Discussion:
Continuous integration
Andreas Politz
2017-03-21 15:45:09 UTC
Permalink
(Sorry for throwing around buzz words here.)

Michael and I are trying to fix some errors with file-notification.
None of us has all the participating libraries and OS at hand, which
means we are unable to test any patches thoroughly.

So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS. The results could be
published via mailing-list.

What do you think ?

-ap
Phillip Lord
2017-03-21 16:11:33 UTC
Permalink
Post by Andreas Politz
(Sorry for throwing around buzz words here.)
Michael and I are trying to fix some errors with file-notification.
None of us has all the participating libraries and OS at hand, which
means we are unable to test any patches thoroughly.
So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS. The results could be published
via mailing-list.
What do you think ?
To some extent it's already there.

https://hydra.nixos.org/jobset/gnu/emacs-trunk/all

It doesnt work for every branch though, which is unfortunate, and I think
this is something that Emacs devel would really benefit from.
Michael Albinus
2017-03-21 19:46:02 UTC
Permalink
Post by Phillip Lord
Post by Andreas Politz
Michael and I are trying to fix some errors with file-notification.
None of us has all the participating libraries and OS at hand, which
means we are unable to test any patches thoroughly.
So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS. The results could be published
via mailing-list.
What do you think ?
To some extent it's already there.
https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
It doesnt work for every branch though, which is unfortunate, and I think
this is something that Emacs devel would really benefit from.
hydra is helpful, although I've stopped reading the mailing list due to
too many false reports. Maybe the situation has been improved last
weeks?

The other point is that it focuses on GNU/Linux. This is OK, but it
doesn't help us with our recent problem, running tests for kqueue on
*BSD and/or OS X.

Best regards, Michael.
Phillip Lord
2017-03-21 20:40:26 UTC
Permalink
Post by Michael Albinus
Post by Phillip Lord
To some extent it's already there.
https://hydra.nixos.org/jobset/gnu/emacs-trunk/all
It doesnt work for every branch though, which is unfortunate, and I
think this is something that Emacs devel would really benefit from.
hydra is helpful, although I've stopped reading the mailing list due to
too many false reports. Maybe the situation has been improved last weeks?
I don't think so. It's a vicious circle, unfortunately. If not enough
people check it, then its more likely to be broken.
Post by Michael Albinus
The other point is that it focuses on GNU/Linux. This is OK, but it
doesn't help us with our recent problem, running tests for kqueue on *BSD
and/or OS X.
I agree with this. I had a look at buildbot and got it working cleanly (on
gnu/linux). It supports BSD, MacOS and windows workers also, as well as
build-any-branch semantics.

From my own perspective, being able to start multi-OS CI builds on any
branch would be a very good addition to Emacs.

Phil
Michael Albinus
2017-03-22 07:00:34 UTC
Permalink
Post by Phillip Lord
Post by Michael Albinus
hydra is helpful, although I've stopped reading the mailing list due to
too many false reports. Maybe the situation has been improved last weeks?
I don't think so. It's a vicious circle, unfortunately. If not enough
people check it, then its more likely to be broken.
It's even worse now. For failed test runs, there is no log file anymore.
This makes it useless.
Post by Phillip Lord
Phil
Best regards, Michael.
Toon Claes
2017-03-22 08:46:40 UTC
Permalink
Post by Andreas Politz
So, I wondered whether it be possible to run the test-suite
automatically for every commit/branch/OS. The results could be
published via mailing-list.
Some time ago Ted Zlatanov proposed to use GitLab to improve the
development process:

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

GitLab could take care of running CI, because it runs CI when commits
gets pushed to it.

The GitLab Runner can be installed on a large number of platforms:

https://docs.gitlab.com/runner/install/

And the builds logs will be available on the GitLab installation.

I know several people on this list are not familiar with
GitLab/GitHub/BitBucket, that's why Ted asked
savannah-hackers-***@gnu.org if it was possible to run a GitLab
installation on FSF/GNU hardware, but I've never heard anything else
from it.

http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

I think it could be really interesting to give GitLab a try in the Emacs
development workflow. And I am also willing to help to set this up.


-- Toon
Thien-Thi Nguyen
2017-03-22 12:16:39 UTC
Permalink
() Toon Claes <***@iotcl.com>
() Wed, 22 Mar 2017 09:46:40 +0100 (CET)

Some time ago Ted Zlatanov proposed to use GitLab to improve
the development process

(tangent) I tried to create GitLab account several times but it
gave me a 422 error (w/o further explanation) each time. What's
the probem, i wonder? My creds ain't good enough, i suppose...

(on topic) I wanted the GitLab account to be able to push (and
thus work on, publicly, using the nice GitLab facilities) ZOW:

http://www.gnuvola.org/software/zow/

which can be used to whip up CI/CD for Emacs (or anything,
really) in Emacs Lisp. (Admit it, you saw it coming! :-D)
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502
Richard Stallman
2017-03-22 16:42:02 UTC
Permalink
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
Post by Thien-Thi Nguyen
(tangent) I tried to create GitLab account several times but it
gave me a 422 error (w/o further explanation) each time. What's
the probem, i wonder? My creds ain't good enough, i suppose...
That could be a serious flaw in GitLab, if it is a policy
rather than a bug.

What sort of "creds" are they checking? What do they _say_ a person
needs in order to make an account?
--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.
Thien-Thi Nguyen
2017-03-31 13:19:44 UTC
Permalink
() Richard Stallman <***@gnu.org>
() Wed, 22 Mar 2017 12:42:02 -0400

What sort of "creds" are they checking?

I don't know what they're checking. My understanding is that
the site is not actually running the "community edition" variant
(from ‘git clone https://gitlab.com/gitlab-org/gitlab-ce.git’),
so even if i could understand the source in short time (i'm just
starting to study Ruby -- hints from those w/ more experience
very welcome!), the answer might not be evident there anyway.

This happens in a web-browser HTTPS session, so implicit creds
are whatever such a session entails (plus whatever the spooks
manage to sidecar into/over/around the session, i suppose).

What do they _say_ a person needs in order to make an account?

AFAIR, there were no instructions on what constitutes an
acceptable login name, password, or email address. Just some
input text fields.

You need to fill these in w/ a login name that's not already
assigned, a password for that login (entered twice to validate),
and an email address. Both the "not already assigned" and the
password validation checks require Javascript, i think.

Almost forgot: there's also the usual spate of big-SaaSS icons.
Presumably you don't *need* them, but can use them (or let them
(ab)use you, more likely) as substitute forms of authentication.
Those creds i definitely lack.
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502
Richard Stallman
2017-04-02 19:48:42 UTC
Permalink
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
Post by Richard Stallman
What sort of "creds" are they checking?
I don't know what they're checking.
Before relying on Gitlab for anything, we need to find out what
in practice a person needs in order to make a Gitlab account.
Until we know that, we can't reach any conclusion about whether it is ok
to use for Emacs development.
--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.
Toon Claes
2017-05-23 20:07:53 UTC
Permalink
Post by Richard Stallman
Before relying on Gitlab for anything, we need to find out what
in practice a person needs in order to make a Gitlab account.
Until we know that, we can't reach any conclusion about whether it is ok
to use for Emacs development.
The issue Thien-Thi Nguyen was having with registering at GitLab.com was
caused by the fact their cookies were turned off completely.

At the moment it is just not possible to register, or sign in, at
GitLab.com with cookies disabled.

I've noticed many other sites, including savannah.gnu.org, rely on
cookies to log in, so I'm hoping this is not an issue. Or am I mistaken?

That being said, all the results of the emacs tests ran by GitLab CI are
public and available without logging in.

The investigation of the issue of not being able to register can be
found here: https://gitlab.com/gitlab-org/gitlab-ce/issues/30401


-- Toon



DISCLAIMER: I'm working for GitLab.com, but I'm an avid emacs user
and available to help finding the suited CI solution, whether that
is GitLab CI, Hydra, Buildbot, or something else.
Thien-Thi Nguyen
2017-03-31 13:20:05 UTC
Permalink
() Richard Stallman <***@gnu.org>
() Wed, 22 Mar 2017 12:42:02 -0400

What sort of "creds" are they checking?

I don't know what they're checking. My understanding is that
the site is not actually running the "community edition" variant
(from ‘git clone https://gitlab.com/gitlab-org/gitlab-ce.git’),
so even if i could understand the source in short time (i'm just
starting to study Ruby -- hints from those w/ more experience
very welcome!), the answer might not be evident there anyway.

This happens in a web-browser HTTPS session, so implicit creds
are whatever such a session entails (plus whatever the spooks
manage to sidecar into/over/around the session, i suppose).

What do they _say_ a person needs in order to make an account?

AFAIR, there were no instructions on what constitutes an
acceptable login name, password, or email address. Just some
input text fields.

You need to fill these in w/ a login name that's not already
assigned, a password for that login (entered twice to validate),
and an email address. Both the "not already assigned" and the
password validation checks require Javascript, i think.

Almost forgot: there's also the usual spate of big-SaaSS icons.
Presumably you don't *need* them, but can use them (or let them
(ab)use you, more likely) as substitute forms of authentication.
Those creds i definitely lack.
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502
Ted Zlatanov
2017-03-22 13:14:58 UTC
Permalink
On Wed, 22 Mar 2017 09:46:40 +0100 (CET) Toon Claes <***@iotcl.com> wrote:

TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
TC> development process:

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html

TC> GitLab could take care of running CI, because it runs CI when commits
TC> gets pushed to it.

Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro. Together with per-branch CI
(so the changes on the branch can be tested before they are merged, as
opposed to post-merge) this could result in a greatly improved developer
experience.

(Hydra is a good service, but it doesn't offer that level of integration
currently, and I think it would be a bit harder to set that up.)

TC> I know several people on this list are not familiar with
TC> GitLab/GitHub/BitBucket, that's why Ted asked
TC> savannah-hackers-***@gnu.org if it was possible to run a GitLab
TC> installation on FSF/GNU hardware, but I've never heard anything else
TC> from it.

TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html

Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).

TC> I think it could be really interesting to give GitLab a try in the Emacs
TC> development workflow. And I am also willing to help to set this up.

Same here.

On Wed, 22 Mar 2017 13:16:39 +0100 Thien-Thi Nguyen <***@gnu.org> wrote:

TN> (tangent) I tried to create GitLab account several times but it
TN> gave me a 422 error (w/o further explanation) each time. What's
TN> the probem, i wonder? My creds ain't good enough, i suppose...

Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
are proposing something different: a FSF/GNU hosted installation of the
GPL-ed GitLab software on local hardware.

Ted
Alex
2017-03-22 14:19:30 UTC
Permalink
Post by Ted Zlatanov
Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).
Other self-hosted options are https://gitea.io/en-US/ and
https://gogs.io; both also use a lot of JS but as far as I can tell it’s
all free.

They don’t have built-in CI, though, so that would probably require a
separate dedicated CI server like Jenkins.
Toon Claes
2017-03-22 15:38:26 UTC
Permalink
Post by Alex
Other self-hosted options are https://gitea.io/en-US/ and
https://gogs.io; both also use a lot of JS but as far as I can tell it’s
all free.
Even if the javascript is free, we should verify if it works properly
with LibreJS.
Toon Claes
2017-03-22 15:36:40 UTC
Permalink
Post by Ted Zlatanov
Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up.
Let's not try to change too many thing at once. But I've been working
with a Pull Request/Merge Request workflow for quite some time now, and
I like it!
Post by Ted Zlatanov
Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).
The javascript on GitLab is free, but at the moment it is not compatible
with LibreJS. There is an issue about this, but there isn't much
progress on it:

https://gitlab.com/gitlab-org/gitlab-ce/issues/15621

If that is a blocking issue, we should trigger the GitLab team so maybe
it will get a higher priority.
Post by Ted Zlatanov
Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com. Toon and I
are proposing something different: a FSF/GNU hosted installation of the
GPL-ed GitLab software on local hardware.
Yes, as far as I understand, the FSF/GNU community does not like relying
on third party hosting, so that why we are suggesting to install a
self-hosted GitLab instance on FSF/GNU systems.

Maybe as temporary solution, we might put a git mirror on GitLab.com and
set up GitLab CI to see if we can get it to run the tests.


-- Toon
Phillip Lord
2017-03-22 18:51:14 UTC
Permalink
Post by Toon Claes
Post by Ted Zlatanov
Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up.
Yes, as far as I understand, the FSF/GNU community does not like relying
on third party hosting, so that why we are suggesting to install a
self-hosted GitLab instance on FSF/GNU systems.
Maybe as temporary solution, we might put a git mirror on GitLab.com and
set up GitLab CI to see if we can get it to run the tests.
That should work, but is a slight pain to mirror since the CI
configuration is committed.

Phil
Eli Zaretskii
2017-03-22 15:41:05 UTC
Permalink
Date: Wed, 22 Mar 2017 09:14:58 -0400
Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro.
Actually, it's more like "obtain write access, then push your changes
directly".
Toon Claes
2017-03-22 15:59:55 UTC
Permalink
Post by Eli Zaretskii
Date: Wed, 22 Mar 2017 09:14:58 -0400
Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro.
Actually, it's more like "obtain write access, then push your changes
directly".
A platform like GitLab relies heavily on pushing code to feature
branches, and merge to master when someone approves. I know this is a
big change compared to the current workflow, that's why I am proposing
to start of by setting up CI (which will work with either workflow).

-- Toon
Phillip Lord
2017-03-22 18:49:14 UTC
Permalink
Post by Ted Zlatanov
TC> Some time ago Ted Zlatanov proposed to use GitLab to improve the
TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg00937.html
TC> GitLab could take care of running CI, because it runs CI when commits
TC> gets pushed to it.
Absolutely. I think the benefits reach beyond that--especially if a pull
request workflow could be set up. Right now it's "push into branch; ask
for comments" which is delightfully retro. Together with per-branch CI
(so the changes on the branch can be tested before they are merged, as
opposed to post-merge) this could result in a greatly improved developer
experience.
I'd certainly agree with this. Inline code review would be nice also,
especially given that Emacs git is set up for non-squashing.
Post by Ted Zlatanov
TC> I know several people on this list are not familiar with
TC> GitLab/GitHub/BitBucket, that's why Ted asked
TC> installation on FSF/GNU hardware, but I've never heard anything else
TC> from it.
TC> http://lists.gnu.org/archive/html/emacs-devel/2016-07/msg01133.html
Also note the recent discussion about why the Docker Hub web site's
Javascript usage made the Docker Hub service unacceptable. I hope we
don't waste time on discussing a GitLab installation if it doesn't fit
that specific requirement (since it runs a web server).
If we are going to test builds across different platforms then, it is
worth noting, that we technically have to use non-free software. We
cannot test the windows/macos builds without it. I don't know whether
this is a problem or not; it is not dissimilar from building and
distributing Emacs windows binaries.

I mentioned earlier that I've also tried buildbot. It works, the
building is nice, and it's just CI. It would plugin to the existing
savannah infrastructure, so might be less distruptive. The web front end
does not, however, work with librejs.


Phil
raman
2017-03-23 00:17:50 UTC
Permalink
Would be nice to target a tool that is usable from within Emacs --
rather than dumbing things down to a What You See Is All You Have
browser environment.

Is it safe to assume that people developing Emacs are using Emacs as an
editor?

Browser-based tools get inflicted on one in a heterogeneous environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
--
Phillip Lord
2017-03-23 14:22:17 UTC
Permalink
Post by raman
Would be nice to target a tool that is usable from within Emacs --
rather than dumbing things down to a What You See Is All You Have
browser environment.
To an extent, although, probably not to the extent that it is usuable
everywhere else. Nor to the extent that we have to develop something
specific rather than using software that already exists.
Post by raman
Is it safe to assume that people developing Emacs are using Emacs as an
editor?
As an editor, but not necessarily for everything else.
Post by raman
Browser-based tools get inflicted on one in a heterogeneous environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
Both of the tools that have been mentioned (gitlab and buildbot) have
IRC interfaces, so that could be contacted this way. Another solution
would be to extend them to serve up an org file, with hyperlinks through
to logs, and so forth.

AFAICT, the current solution (hydra) doesn't offer an emacs front end.

Phil
T.V Raman
2017-03-23 17:11:34 UTC
Permalink
As long as we dont end up doing things like diff/merge of source code
inside a browser window, I'll stay mostly happy.
Post by Phillip Lord
Post by raman
Would be nice to target a tool that is usable from within Emacs --
rather than dumbing things down to a What You See Is All You Have
browser environment.
To an extent, although, probably not to the extent that it is usuable
everywhere else. Nor to the extent that we have to develop something
specific rather than using software that already exists.
Post by raman
Is it safe to assume that people developing Emacs are using Emacs as an
editor?
As an editor, but not necessarily for everything else.
Post by raman
Browser-based tools get inflicted on one in a heterogeneous environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
Both of the tools that have been mentioned (gitlab and buildbot) have
IRC interfaces, so that could be contacted this way. Another solution
would be to extend them to serve up an org file, with hyperlinks through
to logs, and so forth.
AFAICT, the current solution (hydra) doesn't offer an emacs front end.
Phil
--
--
Phillip Lord
2017-03-23 17:55:43 UTC
Permalink
Oh, I was talking about CI, nothing else.

gitlab and that sort of tool doesn't force you to do diffing or merging
of source code in the web interface. The code review tools, I don't know
about. Some systems have pretty good email interfaces for discussing
diffs; I haven't used gitlabs one, so I don't know.

Phil
Post by T.V Raman
As long as we dont end up doing things like diff/merge of source code
inside a browser window, I'll stay mostly happy.
Post by Phillip Lord
Post by raman
Would be nice to target a tool that is usable from within Emacs --
rather than dumbing things down to a What You See Is All You Have
browser environment.
To an extent, although, probably not to the extent that it is usuable
everywhere else. Nor to the extent that we have to develop something
specific rather than using software that already exists.
Post by raman
Is it safe to assume that people developing Emacs are using Emacs as an
editor?
As an editor, but not necessarily for everything else.
Post by raman
Browser-based tools get inflicted on one in a heterogeneous environment
where different individuals use a variety of editors, platforms and
machine types --- in this instance --- emacs-Development --- it would be
nice to leverage the strenghts of Emacs if it is indeed a common
component of our varied environments.
Both of the tools that have been mentioned (gitlab and buildbot) have
IRC interfaces, so that could be contacted this way. Another solution
would be to extend them to serve up an org file, with hyperlinks through
to logs, and so forth.
AFAICT, the current solution (hydra) doesn't offer an emacs front end.
Phil
--
Toon Claes
2017-03-23 21:29:06 UTC
Permalink
Post by Phillip Lord
Some systems have pretty good email interfaces for discussing
diffs; I haven't used gitlabs one, so I don't know.
The initial review would happen online through the web interface. You
can see the diff online and add a comment to a specific line (of the
diff).

Such comments would produce mail notifications, and GitLab supports
replying to that email to add a comment.

If you aware of tools that won't require the initial review through the
web interface, I really am interested to see how they work.


-- Toon
Chad Brown
2017-03-23 22:05:17 UTC
Permalink
Post by Toon Claes
If you aware of tools that won't require the initial review through the
web interface, I really am interested to see how they work.
Would someone with a functional setup for such a system (GitLab,
etc.) be willing to check and see how functional it is in eww, please?
(The only one I have easy access to is github, which is problematic
for other reasons).

Thanks in advance,
~Chad
Yuri Khan
2017-03-24 05:15:11 UTC
Permalink
Post by Chad Brown
Would someone with a functional setup for such a system (GitLab,
etc.) be willing to check and see how functional it is in eww, please?
Nohow, as far as I can tell. I do not have a functional eww setup, but
I disabled Javascript in Firefox and GitLab review functionality
pretty much broke down completely. It can’t even display a diff.

Also, even with Javascript, I find the implementation of code review
in GitLab horrible. I have had better experience with
Bugzilla+Splinter (which still requires Javascript but is more
pleasant UX-wise).
Phillip Lord
2017-03-24 10:37:42 UTC
Permalink
Post by Chad Brown
Post by Toon Claes
If you aware of tools that won't require the initial review through the
web interface, I really am interested to see how they work.
Would someone with a functional setup for such a system (GitLab,
etc.) be willing to check and see how functional it is in eww, please?
(The only one I have easy access to is github, which is problematic
for other reasons).
Both of them have online versions either as service or a demo. As far as
I can tell, neither of them are functional within eww.

Ultimately, if our main criteria for a CI and code review system is
"does it work with EWW" or "can we have an emacs interface", then I
think that it's likely we are going to end up with something that is
poorly functional. Picking a good system and then making Emacs work with
it seems a better way forward.

Phil
raman
2017-03-24 15:22:32 UTC
Permalink
***@russet.org.uk (Phillip Lord) writes:
I'd phrase the requirement slightly differently from what Phillip says
below.

"Functional" is really in the eye of the beholder -- most of the
JS-based browser environments lack good APIs that keep the UI
front-end separate from the app-logic back-end.


So asking "does it work in EWW" itself may be a red-herring -- a
well-designed system would let you implement a clean UI from the emacs
end, eeither with or without EWW.

For a good example of such API separation, see Ipython notebooks and how
package ein in Emacs leverages that separation.
Post by Phillip Lord
Post by Chad Brown
Post by Toon Claes
If you aware of tools that won't require the initial review through the
web interface, I really am interested to see how they work.
Would someone with a functional setup for such a system (GitLab,
etc.) be willing to check and see how functional it is in eww, please?
(The only one I have easy access to is github, which is problematic
for other reasons).
Both of them have online versions either as service or a demo. As far as
I can tell, neither of them are functional within eww.
Ultimately, if our main criteria for a CI and code review system is
"does it work with EWW" or "can we have an emacs interface", then I
think that it's likely we are going to end up with something that is
poorly functional. Picking a good system and then making Emacs work with
it seems a better way forward.
Phil
--
Ted Zlatanov
2017-03-24 16:31:16 UTC
Permalink
On Fri, 24 Mar 2017 10:37:42 +0000 ***@russet.org.uk (Phillip Lord) wrote:

PL> Ultimately, if our main criteria for a CI and code review system is
PL> "does it work with EWW" or "can we have an emacs interface", then I
PL> think that it's likely we are going to end up with something that is
PL> poorly functional. Picking a good system and then making Emacs work with
PL> it seems a better way forward.

Agreed; it's probably going to be possible to use the GitLab API. It's
pretty thorough https://docs.gitlab.com/ee/api/README.html

Ted
Phillip Lord
2017-03-24 18:07:39 UTC
Permalink
Post by Ted Zlatanov
PL> Ultimately, if our main criteria for a CI and code review system is
PL> "does it work with EWW" or "can we have an emacs interface", then I
PL> think that it's likely we are going to end up with something that is
PL> poorly functional. Picking a good system and then making Emacs work
with PL> it seems a better way forward.
Agreed; it's probably going to be possible to use the GitLab API. It's
pretty thorough https://docs.gitlab.com/ee/api/README.html
Just so.

I think we are jagging here. We have two CI systems that people have
looked at (gitlab, and I have buildbot working). gitlab also does code
review, and git front end hosting.

Which leaves us with the three technical/social questions:

1) Are John, Eli and the other devels interested and willing to sanction
adding this form of infrastructure (even as a trial)

2) Does the FSF have resource to provide the bare metal hosting assuming
we want to self-host, even as a trial?

3) Who is prepared to do an installation?

And, I would also pose a fourth moral question:

4) In addition to gnu/linux, can we provide a windows and mac builder,
given that this would by necessity involve non-free software (i.e. the
OSes).


In answer to question 3), I'm happy to do installation for a buildbot
based solution, since I already tried it just for "fun".

Phil
Stefan Monnier
2017-03-24 18:37:53 UTC
Permalink
Post by Phillip Lord
4) In addition to gnu/linux, can we provide a windows and mac builder,
given that this would by necessity involve non-free software (i.e. the
OSes).
FWIW, I have heard rumors that you can build several Windows programs
under GNU/Linux (and I guess that's why Debian includes a mingw32
package). Not sure if Emacs falls into this category. At least I know
it's possible to *run* the w32 version of Emacs under GNU/Linux using
Wine (last time I used it, it suffered from some quirks but was good
enough for me to reproduce a bug locally).


Stefan
Eli Zaretskii
2017-03-24 19:09:56 UTC
Permalink
Date: Fri, 24 Mar 2017 14:37:53 -0400
Post by Phillip Lord
4) In addition to gnu/linux, can we provide a windows and mac builder,
given that this would by necessity involve non-free software (i.e. the
OSes).
FWIW, I have heard rumors that you can build several Windows programs
under GNU/Linux (and I guess that's why Debian includes a mingw32
package). Not sure if Emacs falls into this category.
It doesn't, for the same reason Emacs cannot be cross-compiled
targeting any other system: building Emacs requires running the built
binary. As long as dumping Emacs is being used, it's also impractical
with Emacs.
At least I know it's possible to *run* the w32 version of Emacs
under GNU/Linux using Wine
AFAIK, running Emacs under Wine requires a specially-formatted
command, like "wine emacs.exe" or some such, so the build system needs
changes to support that, and so does CI.
Phillip Lord
2017-03-27 10:30:39 UTC
Permalink
Post by Stefan Monnier
Post by Phillip Lord
4) In addition to gnu/linux, can we provide a windows and mac builder,
given that this would by necessity involve non-free software (i.e. the
OSes).
FWIW, I have heard rumors that you can build several Windows programs
under GNU/Linux (and I guess that's why Debian includes a mingw32
package). Not sure if Emacs falls into this category. At least I know
it's possible to *run* the w32 version of Emacs under GNU/Linux using
Wine (last time I used it, it suffered from some quirks but was good
enough for me to reproduce a bug locally).
That may be true, but even if it works, it's not testing the windows
build, it's testing a windows like build.

Phil
Eli Zaretskii
2017-03-24 18:59:42 UTC
Permalink
Date: Fri, 24 Mar 2017 18:07:39 -0000
1) Are John, Eli and the other devels interested and willing to sanction
adding this form of infrastructure (even as a trial)
Why do you need a sanction? AFAIU, this doesn't require anything from
the project, once there are volunteers willing to set this up. Am I
missing something?
Phillip Lord
2017-03-24 21:35:44 UTC
Permalink
Post by Eli Zaretskii
Date: Fri, 24 Mar 2017 18:07:39 -0000
1) Are John, Eli and the other devels interested and willing to
sanction adding this form of infrastructure (even as a trial)
Why do you need a sanction? AFAIU, this doesn't require anything from
the project, once there are volunteers willing to set this up. Am I
missing something?
Because of question 2 and 4. Do we have a machine to put it on, and do we
also want a non-free build.

Phil
Eli Zaretskii
2017-03-25 06:37:29 UTC
Permalink
Date: Fri, 24 Mar 2017 21:35:44 -0000
Post by Eli Zaretskii
Date: Fri, 24 Mar 2017 18:07:39 -0000
1) Are John, Eli and the other devels interested and willing to
sanction adding this form of infrastructure (even as a trial)
Why do you need a sanction? AFAIU, this doesn't require anything from
the project, once there are volunteers willing to set this up. Am I
missing something?
Because of question 2 and 4. Do we have a machine to put it on, and do we
also want a non-free build.
I think Ted answered that, and I agree with him.
Ted Zlatanov
2017-03-24 21:46:22 UTC
Permalink
Date: Fri, 24 Mar 2017 18:07:39 -0000
1) Are John, Eli and the other devels interested and willing to sanction
adding this form of infrastructure (even as a trial)
EZ> Why do you need a sanction? AFAIU, this doesn't require anything from
EZ> the project, once there are volunteers willing to set this up. Am I
EZ> missing something?

There's no need for a sanction; we need the hardware, and Toon
referenced that I asked the savannah admins:
https://lists.gnu.org/archive/html/savannah-hackers-public/2016-07/msg00021.html

The answer was basically "we have no resources, it's up to you."

So I guess it's up to us.

Ted
Phillip Lord
2017-03-27 10:49:36 UTC
Permalink
Post by Ted Zlatanov
Date: Fri, 24 Mar 2017 18:07:39 -0000
1) Are John, Eli and the other devels interested and willing to sanction
adding this form of infrastructure (even as a trial)
EZ> Why do you need a sanction? AFAIU, this doesn't require anything from
EZ> the project, once there are volunteers willing to set this up. Am I
EZ> missing something?
There's no need for a sanction; we need the hardware, and Toon
https://lists.gnu.org/archive/html/savannah-hackers-public/2016-07/msg00021.html
The answer was basically "we have no resources, it's up to you."
The answer that I read was "we are waiting for more resource from FSF too".
Post by Ted Zlatanov
So I guess it's up to us.
That does sound likely though.

Phil
Toon Claes
2017-03-27 09:54:41 UTC
Permalink
Post by Phillip Lord
3) Who is prepared to do an installation?
In answer to question 3), I'm happy to do installation for a buildbot
based solution, since I already tried it just for "fun".
As a trial I hope to have time somewhere this week to set up CI using
GitLab.com. It probably won't be the final hosting solution, but this is
at the moment the easiest path to set up a trial installation.

When we have both, we can do a objective comparison.


-- Toon
Ted Zlatanov
2017-03-27 13:32:37 UTC
Permalink
Post by Phillip Lord
3) Who is prepared to do an installation?
In answer to question 3), I'm happy to do installation for a buildbot
based solution, since I already tried it just for "fun".
TC> As a trial I hope to have time somewhere this week to set up CI using
TC> GitLab.com. It probably won't be the final hosting solution, but this is
TC> at the moment the easiest path to set up a trial installation.

TC> When we have both, we can do a objective comparison.

I'd strongly advise against using gitlab.com. Please review the
conversation here about the Docker Hub site: if *any* part of the
gitlab.com usage requires non-free software, even just account creation,
it won't be acceptable.

If you are willing to run a standalone instance (maybe a EC2 micro
instance), that's the best way to get it going right now.

Ted
Phillip Lord
2017-03-30 09:47:41 UTC
Permalink
Post by Toon Claes
Post by Phillip Lord
3) Who is prepared to do an installation?
In answer to question 3), I'm happy to do installation for a buildbot
based solution, since I already tried it just for "fun".
As a trial I hope to have time somewhere this week to set up CI using
GitLab.com. It probably won't be the final hosting solution, but this is
at the moment the easiest path to set up a trial installation.
When we have both, we can do a objective comparison.
I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.

http://emacs.bioswarm.net:8010

Currently, it's running a single build (full build from clean, through
to tests). It will build any branch (following a change). The build
takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
probably add a "incremental recompile and test" job which would be much
quicker. The builds are running on the master which is probably not
ideal.

Anyway, comments welcome.

Phil
Lars Brinkhoff
2017-03-30 14:47:51 UTC
Permalink
Post by Phillip Lord
I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.
http://emacs.bioswarm.net:8010
I see two tests are failing: one in ediff-ptch-tests and one
inpython-tests. I don't see those when I build and test myself.

Is there a way to see build results split into separate branches?
Phillip Lord
2017-03-30 17:42:11 UTC
Permalink
Post by Lars Brinkhoff
Post by Phillip Lord
I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.
http://emacs.bioswarm.net:8010
I see two tests are failing: one in ediff-ptch-tests and one
inpython-tests. I don't see those when I build and test myself.
Yep, we need a special target for CI which dumps the output of these files
to screen. AFAICT, there is no way to see the log files. The python-test
is breaking, I think, because it assumes that Emacs is not being built in
a virtual environment (which it is with build bot).
Post by Lars Brinkhoff
Is there a way to see build results split into separate branches?
The UI doesn't seem to uncover this, although it may be possible (I am not
a build bot expert). But I can seperate things into queues. In use I'm
make one for emacs-25, one for master, one for everything else.

I've fiddled a bit with the config -- there are now two builds: one
incremental, and one bootstrap. The quick one should be faster.
Toon Claes
2017-04-04 20:19:37 UTC
Permalink
Post by Phillip Lord
I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.
http://emacs.bioswarm.net:8010
Cool!

Meanwhile, I also have set up CI using GitLab.com.

The project is over here, and is being mirrored from Savannah:

https://gitlab.com/emacs-ci/emacs

To set this up, I had to add a .gitlab-ci.yml file to the root of the
project, with the following content:

#+BEGIN_SRC yaml
image: debian:unstable

before_script:
- apt update -qq
- apt install -y -qq build-essential autoconf automake pkg-config libtool m4 autoconf-archive gtk-doc-tools libxml2-utils gobject-introspection libgirepository1.0-dev libglib2.0-dev libjson-glib-dev libncurses-dev

stages:
- test

test:
stage: test
script:
- ./autogen.sh
- ./configure --without-makeinfo --with-gnutls=no
- make check
#+END_SRC

This file probably can be improved in many ways, but I got a successful
build. You can visit the build log here:

https://gitlab.com/emacs-ci/emacs/builds/13595493

If someone with write access would agree to commit the .gitlab-ci.yml
file, the daily mirroring will pick up commits, and GitLab will build
them automatically.

If anybody would like direct write access to the GitLab.com repository,
please click the "Request Access" button on the project page (see link
above).
Post by Phillip Lord
Currently, it's running a single build (full build from clean, through
to tests). It will build any branch (following a change). The build
takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
probably add a "incremental recompile and test" job which would be much
quicker. The builds are running on the master which is probably not
ideal.
That's great work Phil! I still have to figure out everything it does,
but it seems to be very comprehensive.

The set up at GitLab.com is doing quite the same at the moment. Doing
incremental recompilation would be quite hard on GitLab, because each
build is done in a clean Docker container, so you'll have to export
artifacts and reuse them each time.

GitLab has a feature called pipelines, which allows you to chain builds
together in stages. So this could be an example pipeline:

test --> build some GNU/Linux distro
\
-> build macOS
\
-> build Windows

The build stages won't be executed if the test stage failed.
If I understand it correctly, buildbot does something similar?
At the moment I only have configured 1 stage on GitLab, because only 1
was needed at the moment. The build stages shown in the flowchart above
can be added in a later stage to make regular builds for different
platforms automatically.

At first sight, also the concept of Workers on buildbot (called Runners
on GitLab) are quite similar.

What I do not yet understand is what the Builders are, and what the
difference is between full and quick?


-- Toon
Ted Zlatanov
2017-04-06 13:30:32 UTC
Permalink
On Tue, 4 Apr 2017 22:19:37 +0200 (CEST) Toon Claes <***@iotcl.com> wrote:

TC> Meanwhile, I also have set up CI using GitLab.com.

TC> The project is over here, and is being mirrored from Savannah:

TC> https://gitlab.com/emacs-ci/emacs

Toon, that's great. I've requested access to collaborate.

For RMS and anyone else interested in evaluating the suitability of
GitLab, please note that the GnuTLS project is also hosted there, so
they may have already evaluated the free software suitability of it:
https://gitlab.com/gnutls/gnutls

Thanks
Ted
Toon Claes
2017-04-06 14:23:28 UTC
Permalink
Post by Ted Zlatanov
Toon, that's great. I've requested access to collaborate.
Thanks! I've given you access!
Post by Ted Zlatanov
For RMS and anyone else interested in evaluating the suitability of
GitLab, please note that the GnuTLS project is also hosted there, so
https://gitlab.com/gnutls/gnutls
As Thien-Thi Nguyen mentioned, some people might have trouble to
register at GitLab.com at the moment, depending on the browser and the
addons you have enabled. There is an issue about this (which you should
be able to see without registering):

https://gitlab.com/gitlab-org/gitlab-ce/issues/30401



-- Toon
Richard Stallman
2017-04-07 16:06:20 UTC
Permalink
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It is ok to host a project on gitlab. What I don't know is about the
CI functionalities. If it works to use them with JS turned off, or
with LibreJS active, then they are ok.
--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.
Lars Brinkhoff
2017-04-09 12:25:44 UTC
Permalink
Post by Richard Stallman
It is ok to host a project on gitlab. What I don't know is about the
CI functionalities. If it works to use them with JS turned off, or
with LibreJS active, then they are ok.
If they are ok, I'd like to suggest Toon Claes' .gitlab-ci.yml to be
added to the repository:

https://gitlab.com/emacs-ci/emacs/merge_requests/1/diffs

Then there would be automatic building and testing for all commits with
results visible on the GitLab web stite.
Glenn Morris
2017-04-09 16:35:28 UTC
Permalink
Post by Lars Brinkhoff
Then there would be automatic building and testing for all commits with
results visible on the GitLab web stite.
Why are you ignoring the existing continuous integration system that
Emacs and other GNU projects have been using for years?

http://hydra.nixos.org/project/gnu
Lars Brinkhoff
2017-04-09 18:01:44 UTC
Permalink
Post by Glenn Morris
Post by Lars Brinkhoff
Then there would be automatic building and testing for all commits with
results visible on the GitLab web stite.
Why are you ignoring the existing continuous integration system that
Emacs and other GNU projects have been using for years?
http://hydra.nixos.org/project/gnu
I'm not ignoring it, I posted a comment about it a week ago or so.
Ted Zlatanov
2017-05-31 18:26:39 UTC
Permalink
I'd like to start collecting evaluation criteria for a CI solution for
Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
others people feel should be considered. Everyone will have a chance to
comment, vote, and contribute. But first we have to agree on what we're
evaluating.

What features of a CI solution do you consider important that we can
rate objectively?

Thanks
Ted
John Wiegley
2017-05-31 19:25:28 UTC
Permalink
TZ> I'd like to start collecting evaluation criteria for a CI solution for
TZ> Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
TZ> others people feel should be considered. Everyone will have a chance to
TZ> comment, vote, and contribute. But first we have to agree on what we're
TZ> evaluating.

There has been some exploration done on GitLab already, I wonder if they have
some data to share with you?

TZ> What features of a CI solution do you consider important that we can rate
TZ> objectively?

Letting us know when the build fails, which commit caused it to fail, and the
ability to do this on all the platforms that we care about.

Some of the other features that have been discussed, such as code review and
pull requests, are nice, but I think not necessary just to have a helpful CI.
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Phillip Lord
2017-06-01 12:59:53 UTC
Permalink
Post by John Wiegley
TZ> I'd like to start collecting evaluation criteria for a CI solution for
TZ> Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
TZ> others people feel should be considered. Everyone will have a chance to
TZ> comment, vote, and contribute. But first we have to agree on what we're
TZ> evaluating.
There has been some exploration done on GitLab already, I wonder if they have
some data to share with you?
And buildbot.
Post by John Wiegley
TZ> What features of a CI solution do you consider important that we can rate
TZ> objectively?
Letting us know when the build fails, which commit caused it to fail, and the
ability to do this on all the platforms that we care about.
Some of the other features that have been discussed, such as code review and
pull requests, are nice, but I think not necessary just to have a helpful CI.
Building all branches though, so that pull requests can be build in a
clean environment is important.

Multiplatform (i.e. including windows and mac) builds would be nice.

Local replicability of the environment (via docker or vm) would be good
as well.

Nice GUI. Important, and not to be underestimated.

Phil
Ted Zlatanov
2017-07-14 20:08:20 UTC
Permalink
On Wed, 31 May 2017 12:25:28 -0700 John Wiegley <***@gmail.com> wrote:

JW> There has been some exploration done on GitLab already, I wonder if they have
JW> some data to share with you?

I think everyone is waiting. Besides me, no one else seems to have used
the test GitLab instance. I strongly encourage everyone to look around
the Hydra instance we use today, GitLab, BuildBot, and other CI systems
they may know.

From the responses, here are the criteria for "a helpful CI" as John put it:

Builds:

* build logs and good notifications
* good platform coverage
* clean builds of all branches+commits and reporting on each one's build
* local replicability of build environment via Docker or VM
* store build artifacts (packages, tarballs, etc.)

UI:

* good UI/UX and multiple requests for a Web GUI too
* support special build requests: specific branch, target, test (via web or email)

Software and maintainer/company:

* Free software
* probable long-term support; ie they have a solid business plan
* personal logins to comment on builds or specific code

Nice to have:

* pull request awareness (not necessarily PRs in the CI system itself)
* code review capability

In order to keep the evaluation objective, I'll keep out of the voting.
If you just want to vote, please send your votes to me directly by
e-mail. But please feel free to vote and comment here; just make sure
to make it clear that you're voting so I can keep track.

Thanks
Ted
Dmitry Gutov
2017-07-16 21:36:13 UTC
Permalink
Post by Ted Zlatanov
I think everyone is waiting. Besides me, no one else seems to have used
the test GitLab instance.
All the latest builds there are broken now.

And no email notifications means no actual usage.
Post by Ted Zlatanov
I strongly encourage everyone to look around
the Hydra instance we use today, GitLab, BuildBot, and other CI systems
they may know.
I suspect what we actually need to do is to

- (optionally) Build a table of check marks with how the CI options
correspond to the criteria we've enumerated.
- Nag the Emacs maintainers and the FSF personnel about giving us a
hardware, or even setting up a GitLab instance themselves, for the Emacs
developers to use.
- (important) Somehow deal with the perpetually-broken build and _set
up email notifications_. There is no other practical way to encourage
everyone to use the CI. And that's more essential than a (reasonable)
choice of the CI server.

I think the voting may be considered optional since Richard more or less
okay'd a FSF GitLab installation. If somebody actively disagrees, nobody
said we absolutely have to have just one CI server.

On the subject of checkmarks, though:

- GitLab is reasonably feature-rich, and it has a Web UI that's familiar
to many younger developers. Which is a huge plus. Code Review capability
included.
- BuildBot seemingly has no code review capability, and seems to be only
a lego-like system for a ultimately flexible build pipelines. Not our
primary pain point, I think.
- Hydra is fairly limited in terms of features (not code review, for one
thing) and has a minimal UI. It's also broken at the moment
(http://hydra.nixos.org/jobset/gnu/emacs-trunk isn't functional, and
there are JS errors if you open the Error Console; CORS seems to be the
problem).

So GitLab seems to come ahead as an obvious winner to me. If you really
want us to vote, probably better to put that call into a new thread,
instead of deep in the innards of this one.
Ted Zlatanov
2017-07-17 14:43:52 UTC
Permalink
Post by Ted Zlatanov
I think everyone is waiting. Besides me, no one else seems to have used
the test GitLab instance.
DG> And no email notifications means no actual usage.

We turned off e-mail notifications so we don't annoy people.

DG> I suspect what we actually need to do is to

DG> - (optionally) Build a table of check marks with how the CI options correspond
DG> to the criteria we've enumerated.

I don't think that's necessary.

DG> - Nag the Emacs maintainers and the FSF personnel about giving us a hardware,
DG> or even setting up a GitLab instance themselves, for the Emacs developers to
DG> use.

We already did that. We're at the stage of determining what to set up as
"the Emacs CI system."

DG> - (important) Somehow deal with the perpetually-broken build and _set up email
DG> notifications_. There is no other practical way to encourage everyone to use the
DG> CI. And that's more essential than a (reasonable) choice of the CI server.

If you're talking about the temporary evaluation setup on gitlab.com, I
can work on that. But remember we're evaluating the CI system, not
actually switching to gitlab.com.

DG> So GitLab seems to come ahead as an obvious winner to me. If you really want us
DG> to vote, probably better to put that call into a new thread, instead of deep in
DG> the innards of this one.

I'll record your vote and have posted a new article calling for votes.

Thanks
Ted
Ted Zlatanov
2017-07-17 14:36:25 UTC
Permalink
Reposting as suggested by Dmitry Gutov.

I strongly encourage everyone to look around the Hydra instance we
use today, GitLab, BuildBot, and other CI systems they may know.

From the responses, here are the criteria for "a helpful CI" as John put it:

Builds:

* build logs and good notifications
* good platform coverage
* clean builds of all branches+commits and reporting on each one's build
* local replicability of build environment via Docker or VM
* store build artifacts (packages, tarballs, etc.)

UI:

* good UI/UX and multiple requests for a Web GUI too
* support special build requests: specific branch, target, test (via web or email)

Software and maintainer/company:

* Free software
* probable long-term support; ie they have a solid business plan
* personal logins to comment on builds or specific code

Nice to have:

* pull request awareness (not necessarily PRs in the CI system itself)
* code review capability

In order to keep the evaluation objective, I'll keep out of the voting.
If you just want to vote, please send your votes to me directly by
e-mail. But please feel free to vote and comment here; just make sure
to make it clear that you're voting so I can keep track.

You can vote for multiple CI systems if you think that's a good thing.

Thanks
Ted
John Wiegley
2017-08-11 17:36:37 UTC
Permalink
TZ> I strongly encourage everyone to look around the Hydra instance we
TZ> use today, GitLab, BuildBot, and other CI systems they may know.

What was the final outcome of the voting, in case I missed it?
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Ted Zlatanov
2017-08-11 19:38:40 UTC
Permalink
TZ> I strongly encourage everyone to look around the Hydra instance we
TZ> use today, GitLab, BuildBot, and other CI systems they may know.

JW> What was the final outcome of the voting, in case I missed it?

GitLab has carried the vote by a large majority (8 votes). I got about
half of those in private e-mail. The great majority of Emacs developers
haven't had a strong opinion in emacs-devel or elsewhere.

Meanwhile, gitlab.com has issues mirroring the Emacs repository, but
they said that's a system-wide issue, and they host many thousands of
repositories. So I don't think that will be an issue with a dedicated
private installation, but I'm bringing it up in the interest of full
disclosure: https://gitlab.com/emacs-ci/emacs/issues/3

Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.

Ted
Nicolas Petton
2017-08-11 21:41:08 UTC
Permalink
Post by Ted Zlatanov
Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.
That would be awesome! I guess that the GitLab instance would only be
used for its CI though, or are we planning to use Merge Requests and/or
its issue tracker as well?

Cheers,
Nico
Ted Zlatanov
2017-08-11 23:08:29 UTC
Permalink
Post by Ted Zlatanov
Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.
NP> That would be awesome! I guess that the GitLab instance would only be
NP> used for its CI though, or are we planning to use Merge Requests and/or
NP> its issue tracker as well?

For now, we're only planning to do CI. After the CI piece is set up and
working, we can discuss other features. The next step is a machine that
can host the GitLab instance. Who can get that set up on the FSF/GNU
side? It could live on the GNU ELPA server or on a new machine.

The GitLab version of the emacs.git repository (and any others needed)
will be a pure mirror, so merge/pull requests won't have an effect.

We will disable the web UI as far as possible so users are not misled
into trying to use those features.

Ted
John Wiegley
2017-08-11 23:49:25 UTC
Permalink
TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
TZ> installation. I can help set it up, but it has to be allocated like the
TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
TZ> and start doing CI against all emacs.git commits.

Richard, is this something the sysadmins at the FSF can help with?
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Ted Zlatanov
2017-12-09 23:59:10 UTC
Permalink
TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
TZ> installation. I can help set it up, but it has to be allocated like the
TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
TZ> and start doing CI against all emacs.git commits.

JW> Richard, is this something the sysadmins at the FSF can help with?

Ping again. It's been a while and any help would be welcome in setting
up a GitLab CI system for Emacs.

Thanks
Ted
Ian Kelling
2017-12-19 19:52:48 UTC
Permalink
Post by John Wiegley
TZ> Unless there are objections, I ask (again) for a machine to host a GitLab
TZ> installation. I can help set it up, but it has to be allocated like the
TZ> GNU ELPA server, internally. Once it's up and running, we can set it up
TZ> and start doing CI against all emacs.git commits.
JW> Richard, is this something the sysadmins at the FSF can help with?
Ping again. It's been a while and any help would be welcome in setting
up a GitLab CI system for Emacs.
Thanks
Ted
We recently put a new cluster in operation and can create the machine
pretty easily now. Some of the details like ssh keys and
responsibilities are better discussed off list. Anyone else who wants to
volunteer to admin this system, please reply here or to me and I will
add you to the thread. I'm starting the thread with all the users I see
on the elpa machine.
--
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7 DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org
Toon Claes
2017-08-13 07:13:04 UTC
Permalink
Post by Ted Zlatanov
Meanwhile, gitlab.com has issues mirroring the Emacs repository, but
they said that's a system-wide issue, and they host many thousands of
repositories. So I don't think that will be an issue with a dedicated
private installation, but I'm bringing it up in the interest of full
disclosure: https://gitlab.com/emacs-ci/emacs/issues/3
Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.
Before you start setting up emacs's private GitLab instance for CI, I
should explain something.

GitLab is built in 2 flavours, CE and EE:
- GitLab CE is Free Software, and licensed under MIT Expat License.
- GitLab EE is built on top of GitLab CE, it includes extra features
that are destined for enterprise users. GitLab EE has a proprietary
license. https://gitlab.com/gitlab-org/gitlab-ee/blob/master/LICENSE

The feature to mirror a repo, from Savannah to GitLab, is an EE
feature. So you cannot use that feature without a subscription.
(see https://about.gitlab.com/products/).

So to use GitLab CI, there are a few options:

- Set up a private GitLab EE instance with a paid subscription
=> I doubt GNU/FSF would agree with this

- Set up a private GitLab CE instance and build some custom scripts to
/manually/ sync the repo from Savannah to GitLab.

- Keep using GitLab.com (the SaaS solution GitLab provides, which is
running GitLab EE).

- Instead of the savannah repo, use the GitLab repo as the main one.
=> I don't see this happen in the near future.


-- Toon
Paul Eggert
2017-08-13 07:18:01 UTC
Permalink
Post by Toon Claes
- Set up a private GitLab CE instance and build some custom scripts to
/manually/ sync the repo from Savannah to GitLab.
As this is the only option that uses free software, it's the only one that a GNU
project like Emacs could reasonably use.
Ted Zlatanov
2017-08-14 02:22:33 UTC
Permalink
Post by Toon Claes
- Set up a private GitLab CE instance and build some custom scripts to
/manually/ sync the repo from Savannah to GitLab.
PE> As this is the only option that uses free software, it's the only one that a GNU
PE> project like Emacs could reasonably use.

Yes. It's not hard to automate: `git fetch --all --tags --prune' to a
local clone (bare repo), followed by `git push -f --prune --mirror' to
the GitLab remote. We just need a machine where the script can live.

Ted
Richard Stallman
2017-08-18 15:06:50 UTC
Permalink
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
Post by Ted Zlatanov
Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.
I asked the sysadmins about this. They are setting up new server
infrastructure, which is a big job. It looks like it will take 2 or 3 months.

Once that is done, they will be able to make a new virtual machine.

I'm sorry for the delay.
--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.
Ted Zlatanov
2017-08-18 15:35:34 UTC
Permalink
On Fri, 18 Aug 2017 11:06:50 -0400 Richard Stallman <***@gnu.org> wrote:

RS> [[[ To any NSA and FBI agents reading my email: please consider ]]]
RS> [[[ whether defending the US Constitution against all enemies, ]]]
RS> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
Post by Ted Zlatanov
Unless there are objections, I ask (again) for a machine to host a
GitLab installation. I can help set it up, but it has to be allocated
like the GNU ELPA server, internally. Once it's up and running, we can
set it up and start doing CI against all emacs.git commits.
RS> I asked the sysadmins about this. They are setting up new server
RS> infrastructure, which is a big job. It looks like it will take 2 or 3 months.

RS> Once that is done, they will be able to make a new virtual machine.

RS> I'm sorry for the delay.

Thank you for moving things forward. Let us know if we can help.

Ted
Dmitry Gutov
2017-05-31 20:28:45 UTC
Permalink
Post by Ted Zlatanov
What features of a CI solution do you consider important that we can
rate objectively?
I think:

- Readable, functional web UI listing builds and the branches, and
commits, that they correspond to.
- Email notifications to the mailing list, as well as the authors of
commits (author and committer if they are different) that broke the build.

Nothing else really comes to mind.
Stephen Leake
2017-05-31 23:19:23 UTC
Permalink
Post by Ted Zlatanov
I'd like to start collecting evaluation criteria for a CI solution for
Emacs developers. We'll evaluate BuildBot, Hydra, and GitLab and any
others people feel should be considered. Everyone will have a chance to
comment, vote, and contribute. But first we have to agree on what we're
evaluating.
What features of a CI solution do you consider important that we can
rate objectively?
Free software

probable long-term support; ie they have a solid business plan

support special build requests
specific branch, target, test
via web or email
--
-- Stephe
Philipp Stephani
2017-06-04 13:23:07 UTC
Permalink
Post by Ted Zlatanov
What features of a CI solution do you consider important that we can
rate objectively?
- Good web-based user interface
- Web interface for code reviews for pull requests
- System should automatically test pull requests and notify the author if
they break any unit test
- System should prevent pushing code to non-scratch branches that would
break unit tests
- Running unit tests on a broad selection of supported systems
- Running unit tests on every commit in non-scratch branches, if there is
breakage notifying the author which commit is the culprit
- Integration of test running and code review (feedback in the code review
tool about test status)
Ted Zlatanov
2017-04-11 13:18:47 UTC
Permalink
Post by Lars Brinkhoff
Then there would be automatic building and testing for all commits with
results visible on the GitLab web stite.
GM> Why are you ignoring the existing continuous integration system that
GM> Emacs and other GNU projects have been using for years?

GM> http://hydra.nixos.org/project/gnu

I think it's fair to evaluate software that has the potential to improve
Emacs developers' experience.

IMHO Hydra (compared to GitLab) has a poor UI, is not very configurable
(especially on a per-developer level), and does not integrate deeply
with Git. It also has no support for code review or pull/merge requests.

I think notifications and webhooks are better in GitLab but that's
debatable.

The Nix build system is pretty arcane as well.

Finally, I don't think using GitLab precludes Hydra or any other CI
system.

Ted
Stefan Monnier
2017-04-11 13:37:09 UTC
Permalink
Post by Ted Zlatanov
with Git. It also has no support for code review or pull/merge requests.
Gitlab-CI doesn't have code review or pull/merge requests either, does it?
Post by Ted Zlatanov
Finally, I don't think using GitLab precludes Hydra or any other CI
system.
AFAIU, the current discussion is not about Gitlab but about CI systems
(including Gitlab-CI).


Stefan
Lars Brinkhoff
2017-04-11 13:51:38 UTC
Permalink
Post by Stefan Monnier
Gitlab-CI doesn't have code review or pull/merge requests either, does it?
No, not in Gitlab-CI. For that you'd have to use the full GitLab.
Ted Zlatanov
2017-04-11 14:34:30 UTC
Permalink
Post by Ted Zlatanov
with Git. It also has no support for code review or pull/merge requests.
SM> Gitlab-CI doesn't have code review or pull/merge requests either, does it?

You're right. I meant it can easily integrate with the rest of the
GitLab toolchain that does have those.

Ted
Phillip Lord
2017-04-11 16:48:08 UTC
Permalink
Post by Stefan Monnier
Post by Ted Zlatanov
with Git. It also has no support for code review or pull/merge requests.
Gitlab-CI doesn't have code review or pull/merge requests either, does it?
Post by Ted Zlatanov
Finally, I don't think using GitLab precludes Hydra or any other CI
system.
AFAIU, the current discussion is not about Gitlab but about CI systems
(including Gitlab-CI).
Even as it stands, gitlab-ci would support pull/merge requests better
than hydra because it should build the branches for us. At the moment,
hydra is limited to emacs-25 and master. It's not possible for a
developer to schedule a clean build at the moment. Achieving this would
be a good thing I think.

Phil
Phillip Lord
2017-04-07 16:11:55 UTC
Permalink
Post by Toon Claes
Post by Phillip Lord
I have the buildbot installation up now. Slightly harder work than I
hoped, but not too bad.
http://emacs.bioswarm.net:8010
This file probably can be improved in many ways, but I got a successful
https://gitlab.com/emacs-ci/emacs/builds/13595493
It's very nice, I think nicer that buildbot, especially as it seems to
have got branches sorted out properly.
Post by Toon Claes
Post by Phillip Lord
Currently, it's running a single build (full build from clean, through
to tests). It will build any branch (following a change). The build
takes about 60 mins (or 30 mins with parallel builds). In practice, I'd
probably add a "incremental recompile and test" job which would be much
quicker. The builds are running on the master which is probably not
ideal.
That's great work Phil! I still have to figure out everything it does,
but it seems to be very comprehensive.
The set up at GitLab.com is doing quite the same at the moment. Doing
incremental recompilation would be quite hard on GitLab, because each
build is done in a clean Docker container, so you'll have to export
artifacts and reuse them each time.
GitLab has a feature called pipelines, which allows you to chain builds
test --> build some GNU/Linux distro
\
-> build macOS
\
-> build Windows
The build stages won't be executed if the test stage failed.
If I understand it correctly, buildbot does something similar?
I can do anything I like with buildbot. It's programmatic. The flip side
is, of course, you have to program it; it's not very declarative.
Post by Toon Claes
At the moment I only have configured 1 stage on GitLab, because only 1
was needed at the moment. The build stages shown in the flowchart above
can be added in a later stage to make regular builds for different
platforms automatically.
At first sight, also the concept of Workers on buildbot (called Runners
on GitLab) are quite similar.
What I do not yet understand is what the Builders are, and what the
difference is between full and quick?
"Builders" are different ways of building things. So, you might have one
specific to windows, or one which runs tests and one which does not.

I've tried to set it up to do an incremental build (that's what "quick"
is). But, I haven't done it right. It's building all branches, but
doesn't understand them. So it's doing subsequent incremental builds on
different branches in the same working directory which is causing the
sorts of breakages that you would expect. Having incremental builds
working is kind of important, I think because the bootstrap is so
slow. Working out how to clean them next time after a failure is
important though.

Phil
John Wiegley
2017-03-29 05:01:16 UTC
Permalink
PL> 1) Are John, Eli and the other devels interested and willing to sanction
PL> adding this form of infrastructure (even as a trial)

Not only does it have my sanction, but I actually setup a trial gitlab on my
own server at the beginning of last year -- although it didn't go far. So if
someone has renewed energy to make this happen, I'm all for it.
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Andreas Politz
2017-03-27 10:34:02 UTC
Permalink
Post by Phillip Lord
I mentioned earlier that I've also tried buildbot. It works, the
building is nice, and it's just CI.
How do you run the different OSs ?

-ap
Phillip Lord
2017-03-27 12:00:48 UTC
Permalink
Post by Andreas Politz
Post by Phillip Lord
I mentioned earlier that I've also tried buildbot. It works, the
building is nice, and it's just CI.
How do you run the different OSs ?
Same way they all do -- a master which runs the website, database and
schedules the build jobs on workers. The workers can have different
OSs.

Phil
Thien-Thi Nguyen
2017-03-22 15:17:18 UTC
Permalink
() Ted Zlatanov <***@lifelogs.com>
() Wed, 22 Mar 2017 09:14:58 -0400

TN> (tangent) [...] create GitLab account

Oh, you mean the GitLab hosted CI/CD accounts on gitlab.com.

Yes.

Toon and I are proposing something different: a FSF/GNU
hosted installation of the GPL-ed GitLab software on local
hardware.

I see. Well, i hope ZOW can fill the gap until that time comes.
Certainly, it runs on my (local) machine, which has nowhere near
the requisite amount of RAM for a GitLab instance...
--
Thien-Thi Nguyen -----------------------------------------------
(defun responsep (query)
(pcase (context query)
(`(technical ,ml) (correctp ml))
...)) 748E A0E8 1CB8 A748 9BFA
--------------------------------------- 6CE4 6703 2224 4C80 7502
John Wiegley
2017-03-31 17:30:42 UTC
Permalink
TN> Toon and I are proposing something different: a FSF/GNU
TN> hosted installation of the GPL-ed GitLab software on local
TN> hardware.

TN> I see. Well, i hope ZOW can fill the gap until that time comes. Certainly,
TN> it runs on my (local) machine, which has nowhere near the requisite amount
TN> of RAM for a GitLab instance...

Have we verified yet that every aspect of GitLab is free (libre) software?
--
John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2
Stefan Monnier
2017-03-31 18:24:44 UTC
Permalink
Post by John Wiegley
Have we verified yet that every aspect of GitLab is free (libre) software?
AFAIK, yes, the gitlab "Community Edition" software is fully free (the
FSF looked at that when assessing its recommendation for hosting
providers). And even the JS software you implicitly need to download
and execute when accessing the "Enterprise Edition" (which is what is
used on gitlab.com) is also fully free (tho maybe not fully recognized
as such by librejs).


Stefan
Mike Gerwitz
2017-04-02 01:44:55 UTC
Permalink
Post by Stefan Monnier
AFAIK, yes, the gitlab "Community Edition" software is fully free (the
FSF looked at that when assessing its recommendation for hosting
providers).
I feel that this needs clarification, just in case.

The FSF itself did not look at anything---there were a number of
volunteers that Zak Rogoff organized. I evaluated GitLab.com against rms'
criteria, but that is GitLab.com as a _hosting provider, not
self-hosting_. I did not perform a software evaluation, aside from
ensuring that the JS served to the client is free (I had already worked
with them in the past to change the EE license to liberate all JS and
use Piwik instead of Google Analytics).

If that's a concern, someone else will have to do so. If there is a
licensing issue, it'd certainly be a bug, and from my experience with
them, they'd get it resolved in short order.
--
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B 2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com
Richard Stallman
2017-04-01 23:31:55 UTC
Permalink
[[[ To any NSA and FBI agents reading my email: please consider ]]]
[[[ whether defending the US Constitution against all enemies, ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]
Post by John Wiegley
TN> I see. Well, i hope ZOW can fill the gap until that time comes. Certainly,
TN> it runs on my (local) machine, which has nowhere near the requisite amount
TN> of RAM for a GitLab instance...
Have we verified yet that every aspect of GitLab is free (libre) software?
Actually, that is not what we need. What we want to know is that this
CI procedure works without the _users_ having to run any nonfree
software. (In this case _users_ will be Emacs developers.)

What runs in their server is no direct concern for us.
--
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.
Lars Brinkhoff
2017-04-11 06:09:43 UTC
Permalink
Another option might be Drone.io:
https://github.com/drone/drone

It's a free continuous integration server implemented in Go. Builds are
run in Docker containers.
Loading...