17:00:01 <djmitche> #startmeeting weekly
17:00:01 <bb-supy> Meeting started Tue Oct 31 17:00:01 2017 UTC and is due to finish in 60 minutes.  The chair is djmitche. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:01 <bb-supy> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:01 <bb-supy> The meeting name has been set to 'weekly'
17:00:05 <djmitche> #topic Introduction
17:00:08 * djmitche waves to everyone
17:00:11 <rjarry> o/
17:00:18 <djmitche> tardyp: you around?
17:00:28 <djmitche> http://bit.ly/2rup31x
17:00:56 <rjarry> he seems not :/
17:01:02 <djmitche> ok
17:01:09 <djmitche> #topic Week in Review
17:01:20 <djmitche> anyone know anything to highlight from this week?
17:01:45 <rjarry> sorry, did not follow close enough
17:01:55 <djmitche> the weekly summary email looks pretty simple
17:02:37 <djmitche> #topic Py2.6 + tw<16 support drop for workers
17:02:54 <djmitche> https://github.com/buildbot/buildbot/pull/3715
17:03:27 <djmitche> my understanding is
17:03:37 <djmitche> we need Twisted > 16 for some features
17:03:44 <rjarry> yep I have nothing against this
17:03:52 <djmitche> for ClientService in particular
17:03:59 <djmitche> and that is not compatible with 2.6
17:04:00 <djmitche> yeah
17:04:10 <rjarry> altough this drops support on immobilist (is that even a word) distributions
17:04:17 <djmitche> #info Email was sent earlier to invite any objections to dropping 2.6 support on the workers
17:04:32 <djmitche> what do you mean?
17:04:50 <rjarry> some redhat versions are still on python 2.6
17:05:05 <djmitche> ah
17:05:14 <djmitche> I think those can use worker 0.9.12 or earlier
17:05:18 <djmitche> just no TLS support, aiui
17:05:24 <rjarry> most .deb based distros are with py 2.7
17:05:33 <rjarry> sure they can
17:05:42 <djmitche> #info Workers on older distros that only support 2.6 can use older versions of the worker (0.9.12 and earlier)
17:05:45 <djmitche> ..or install 2.7 :)
17:05:46 <rjarry> they are still stuck on eight for the majority of them
17:05:50 <djmitche> right
17:06:01 <djmitche> so I'm not hearing anyone say "no"
17:06:50 <rjarry> I guess not :)
17:06:52 <djmitche> ok!
17:06:58 <djmitche> #agreed No objections, go ahead :)
17:06:59 <tardyp> hi guys
17:07:01 <djmitche> hi!
17:07:03 <tardyp> sorry for being late
17:07:08 <rjarry> hi tardyp
17:07:12 <djmitche> no worries, I've done it many times :(
17:07:18 <djmitche> anything to add to the above?
17:07:34 <tardyp> looks like I've missed the vote :)
17:07:42 <djmitche> I figured you were in favor :)
17:07:50 <tardyp> correct
17:09:07 <djmitche> #topic “Maintainer” on README change to botherders?
17:09:21 <tardyp> I merged your PR, as we agreed last meeting I think
17:09:21 <djmitche> this already landed
17:09:26 <djmitche> but officially removes me as maintainer :)
17:09:27 <djmitche> yeah
17:09:33 <rjarry> heh
17:09:37 <djmitche> and last but not least..
17:09:40 <tardyp> well, you are still part of the botherders
17:09:42 <djmitche> #topic Expert Twisted Book
17:09:43 <djmitche> it's true
17:10:04 <rodrigc> djmitchie: you haven't done much maintenance lately, so that's a non-event
17:10:05 <tardyp> I finally managed to find some time to work on it
17:10:56 <djmitche> I saw some PDF generation stuff
17:11:02 <tardyp> djmitche: I have made a few more paragraphs, if you want to take a look
17:11:07 <djmitche> rodrigc: yeah, I think I just forgot to delete it from the agenda :)
17:11:19 <djmitche> sure, make a PR whenever you'd like my look
17:11:29 <tardyp> I did :)
17:11:36 <tardyp> there are two already
17:11:57 <rjarry> btw, does anybody know who is responsible of the debian/ubuntu buildbot package?
17:12:08 <djmitche> oh, great!
17:12:10 * djmitche flags self
17:12:11 <tardyp> rjarry: now you are!
17:12:15 <rjarry> haha
17:12:15 <djmitche> haha
17:12:31 <tardyp> for me this is a vacant position
17:12:46 <tardyp> we have a very old debian package downstream
17:12:48 <rjarry> we missed stretch this year to upgrade to nine
17:12:56 <djmitche> jollyroger used to do it
17:13:02 <djmitche> Andriy Senkovych
17:13:06 <djmitche> and Adam Collard
17:13:20 <cmouse> I'm seeing Ozarowski, Bogert and Klose
17:13:24 <tardyp> it is not very easy to package, as debian requires to be able to build everything from source packages
17:13:39 <tardyp> which is very difficult with nodejs toolchains
17:13:39 <djmitche> yeah, the JS is a problem since we vendor it all
17:13:45 <rjarry> ah yeah
17:13:52 <rodrigc> I found submitting patches to Debian in general is quite hard
17:13:53 <tardyp> as you would need to package lots of stuff
17:14:10 <djmitche> #topic Debian Packaging
17:14:14 <rodrigc> and packaging all the dependencies of buildbot just makes the task even harder
17:14:24 <rjarry> wouldn't that require to change buildbot itself so that it sources JS stuff from external libs ?
17:14:24 <tardyp> one solution is to ship the javascript precompiled inside the source tarballs
17:14:32 <tardyp> that is what grafana is doing for example
17:14:46 <rjarry> ok
17:15:07 <rjarry> so no more distinction between .tgz and .whl
17:15:19 <djmitche> oh, the .tgz's currently don't contain that info?
17:15:35 <djmitche> would Debian be OK with it in that case?  It's still not covered under our license..
17:15:39 <rjarry> no, they only contain the .coffee source files, not the compiled .js
17:15:51 <tardyp> technically, I think we just need to whitelist the generated JS in the MANIFEST.in
17:16:23 <tardyp> not covered under out license?
17:16:35 <rodrigc> Is this where the Debian package files live: https://github.com/buildbot/debian-buildbot ?
17:17:02 <rjarry> Latest commit e76919a  on Oct 25, 2014 :)
17:17:09 <djmitche> I don't know what Debian's specific concerns are
17:17:17 <djmitche> do they allow vendoring in general?
17:17:29 <rjarry> they do not like it in any case
17:17:34 <djmitche> or do they require that if project X is hosted at npmjs.org/X then they need to fetch it from there?
17:17:37 <rjarry> I don't know if they can make an exception
17:17:42 <djmitche> yeah
17:18:02 <tardyp> we don't technically have to ship anything from npm
17:18:11 <tardyp> we ship things from bower
17:18:22 <djmitche> right, but we're sourcing a lot of those libraries (like angular) from npmjs
17:18:22 <tardyp> npm is just for the stuff needed to build the js
17:18:42 <tardyp> no, this is from bower, which uses github as a backend
17:18:48 <djmitche> ahh
17:18:52 <djmitche> like Golang does
17:18:53 <djmitche> interesting
17:19:23 <rjarry> this looks quite a tedious work
17:19:25 <djmitche> so should we try to talk to some of the Debian folks cmouse identified?
17:19:40 <djmitche> yeah, I guess outcomes would be (a) we make a simple change that allows Debian packaging to be easy
17:19:49 <rjarry> I always found that python packaging in distos is crappy
17:19:50 <djmitche> (b) Debian makes an exception or does something tedious
17:19:55 <djmitche> (c) We don't get packaged in debian
17:20:02 <djmitche> it's bad for other languages too
17:20:04 <rjarry> maybe (c) is best ?
17:20:10 <tardyp> ahah
17:20:14 <rjarry> or
17:20:26 <rjarry> we have a partially functional package
17:20:44 <rodrigc> what is lost by not having a Debian package for buildbot?
17:21:02 <cmouse> not much, tbh
17:21:05 <rjarry> we are not integrated with the system scripts (systemd, and such)
17:21:14 <rodrigc> I think for people serious enough about using buildbot, they can survive with pip install
17:21:14 <rjarry> appart from than, not much indeed
17:21:34 <cmouse> i guess it might be harder for some places with strict rules to get bb
17:21:41 <djmitche> yeah
17:21:42 <cmouse> some places forbid installing non-packaged software
17:21:49 <tardyp> I think this is still a mark of seriousness to be packaged in debian
17:21:53 <rjarry> my suggestion would be to have only buildbot-worker available in debian
17:22:07 <rjarry> and leave server stuff out of it
17:22:10 <rodrigc> if someone wants to to do the work, then it is OK
17:22:14 <tardyp> we can also have the master
17:22:19 <tardyp> just not the UI :)
17:22:24 <rjarry> why not
17:22:26 <rodrigc> I think partial solutions are worse than no solution at all
17:22:28 <tardyp> that would be a first step
17:22:33 <cmouse> just want to point out that having buildbot-(worker) requires a c compiler.
17:22:40 <cmouse> e.g. for virtualenv setup
17:22:45 <djmitche> yeah, that makes it tricky
17:23:06 <djmitche> is anyone interested in this enough to follow up?
17:23:21 <djmitche> #info Debian packaging for the UI is hard because it involves compiled, packed JS
17:23:24 <cmouse> there is already buildbot debian package
17:23:39 <djmitche> yeah, it would probably be pinging those maintainers and then working with them to find a fix
17:24:20 <rodrigc> the question is are those maintainers motivated enough to update the packaging files to 0.9 and deal with all the hard work
17:24:28 <rodrigc> based on the issues you raised
17:24:35 <rodrigc> can't hurt to ask them
17:24:39 <cmouse> it's the best way, really
17:24:55 <djmitche> yeah
17:25:01 * djmitche looks for excited volunteers :)
17:25:58 <djmitche> #info This is an open task for a motivated person familiar with Debian to pick up
17:26:01 <tardyp> maybe the argument that python did chose buildbot nine for the CI might help to convinced the old maintainer to update
17:26:11 <rodrigc> my experience with node stuff is I ignore the default system packages anyways, and install my own stuff
17:26:17 <djmitche> yeah
17:26:30 <rodrigc> because system packages tend to be way behind, and Javascript stuff moves very, very fast
17:26:34 <djmitche> yep
17:26:36 <rodrigc> faster than Python stuff
17:26:45 <djmitche> I figure the Debian folks are probably just busy and this isn't a pressing concern for them
17:27:02 <rodrigc> so I would say that having half-way incomplete broken packages for Debian is worse than having no packages at all
17:27:06 <djmitche> if we had information that this was causing issues for users that might be more motivating
17:27:16 <djmitche> yeah
17:27:22 <djmitche> I think they are working but old?
17:27:38 <tardyp> actually https://packages.debian.org/fr/sid/buildbot
17:27:42 <tardyp> this is 0.8.12
17:27:44 <tardyp> so latest eight
17:28:08 <djmitche> that's not bad
17:28:11 <djmitche> I need to run shortly
17:28:17 <rodrigc> I got one last thing
17:28:36 <rodrigc> you guys rejected my request, but I'd like to resubmit my request for more Python3 testing resources: https://lists.buildbot.net/pipermail/devel/2017-September/012428.html
17:28:57 <djmitche> #topic More Python3 Testing Resources
17:29:28 <djmitche> https://lists.buildbot.net/pipermail/devel/2017-September/012428.html
17:30:02 <tardyp> rodrigc: do you have more arguments?
17:30:15 <rodrigc> Even if my full request is not implemented, we need more than what we have now
17:30:20 <rodrigc> I need to argue?
17:30:24 <rodrigc> it's not obvious?
17:31:00 <rodrigc> There are a lot of corner cases, and people who don't know what they are doing submit patches to buildbot
17:31:10 <rodrigc> especially with respect to bytes/unicode handling
17:31:28 <rodrigc> so it's better to catch it via automated testing at buildbot
17:32:12 <rodrigc> I wasn't able to attend the last meeting were you rejected my request, but looking at those meeting notes, you basically decided you didn't want to do it and argued against it
17:32:32 <rodrigc> as one data point, python.org is now on buildbot nine + python 3.4: http://buildbot.python.org/all/#/about
17:32:56 <tardyp> I dont hink that doing dogfooding will help catching what peope are catching with those PRs
17:33:12 <rodrigc> well it's better than nothing
17:33:17 <rodrigc> which is what we have now
17:33:32 <tardyp> we have lots of python3 testing
17:33:38 <rodrigc> it's not enough
17:33:39 <tardyp> what we don't have is interrop
17:33:43 <rodrigc> and inadequate
17:33:58 <tardyp> what people have been catching recently is not interrop problems
17:34:09 <rodrigc> the unit tests are inadequate
17:34:15 <tardyp> its integration issues with python3
17:34:23 <rodrigc> how much do I have to argue this?
17:34:41 <rodrigc> have you basically decided you aren't going to do this, and will keep arguing it so that it won't get done?
17:35:01 <tardyp> what I am saying is that dogfooding will not help
17:35:08 <tardyp> doing more e2e test will help
17:35:13 <rodrigc> and I'm saying you're wrong
17:35:20 <tardyp> but we need to have people doing it
17:35:49 <tardyp> last issue is gitlab hook
17:36:00 <rodrigc> no there are other issues
17:36:04 <tardyp> would dogfooding have find it no
17:36:05 <rodrigc> such as bitbucket hook
17:36:17 <tardyp> same! we are not using bitbucket
17:36:20 <rodrigc> tardyp: let me know
17:36:24 <tardyp> so dogfooding wouldn't have helped
17:36:27 <rodrigc> have you basically decided no?
17:36:43 <rodrigc> because I have better things to do with my time than argue with you about something you aren't going to help with
17:36:49 <tardyp> I am just saying this does not help
17:36:53 <tardyp> and it is complicated to do
17:37:05 <tardyp> so for me there is no ROI at doing it
17:37:09 <rodrigc> basically you've made up your mind that you don't want to do it
17:37:28 <tardyp> you don't have any reasonable argument!
17:37:31 <rodrigc> so if you are coming from that direction, it's uphill for me to argue against it
17:37:49 <tardyp> as always, I am open to be convinced. but this needs to be argumented
17:38:02 <rodrigc> are you the only gatekeeper on this?
17:38:14 <rodrigc> I really can't argue any better than what I have described
17:39:00 <tardyp> then I am not convinced, but maybe others are
17:39:21 <rodrigc> it doesn't really matter what other people thing
17:39:30 <rodrigc> if you are going to be the blocker on this, then that's that
17:40:02 <tardyp> I will not. If other people are willing to work on it, I will not block it
17:40:31 <rodrigc> well who has ops on buildbot who can help?
17:40:33 <tardyp> but I think this won't be a reasonable use of my time
17:40:54 <rodrigc> the proposal I put here: https://lists.buildbot.net/pipermail/devel/2017-September/012428.html requires setting stuff up
17:41:19 <rodrigc> well only you can judge what is useful use of your time, and what you are willing to put time into
17:41:34 <tardyp> https://github.com/buildbot/buildbot-infra/graphs/contributors
17:42:03 <tardyp> this is basically the people that have access to our infra
17:42:08 <rodrigc> most of those people are not active
17:42:29 <tardyp> I can give you access if you want
17:43:49 <tardyp> but the work is not really to setup new worker. the work is to make them work under hyper, and improve our CI setup to implement what you suggest
17:44:16 <tardyp> so It is more work to do here: https://github.com/buildbot/metabbotcfg
17:44:20 <rodrigc> I'm not familiar with hyper
17:44:47 <tardyp> It is not as easy as you might think it is actually.
17:45:04 <rodrigc> well my suggestion is to set up a second master
17:45:19 <rodrigc> not sure how easy it is with your setup
17:45:19 <tardyp> ah.. ok
17:45:36 <rjarry> sorry, I had to run get my kid, I might be interrested in following this debian packaging stuff
17:45:56 <tardyp> then you need to create a new jail here https://github.com/buildbot/buildbot-infra
17:46:03 <rjarry> I'll contact the package maintainers and see what they think
17:46:11 <tardyp> like this one https://github.com/buildbot/buildbot-infra/blob/master/jail-nine.yml
17:46:53 <tardyp> frankly I really think it is more easy to create more end to end tests than to create dogfooding
17:47:31 <rodrigc> well, I disagree, but we have gone over that multiple times already
17:48:35 <tardyp> then go ahead and implement a jail-nine-compat
17:48:51 <rodrigc> I'll submit patches to the infra if I can, but it will take me time to figure it out
17:49:02 <tardyp> I think this is the only way you can convince yourself
17:49:22 <tardyp> there is a vagrant setup to test your patches
17:49:23 <rodrigc> I've been setting up my own environments to dogfood and test
17:49:34 <rodrigc> so I'm convinced this is a good idea
17:49:41 <rodrigc> don't worry about that
17:51:16 <rodrigc> essentially buildbot.python.org is now one big e2e test
17:51:47 <tardyp> it is not. it is a dogfooding
17:52:09 <rodrigc> it's both
17:52:25 <tardyp> e2e tests needs to be in the CI, and prevent PR to enter the release
17:52:37 <tardyp> if they brake compat
17:52:52 <rodrigc> ok
17:53:48 <rodrigc> anyways, it looks like this discussion is going to go on and on.  if you want to end the discussion of this point for the meeting, this is fine
17:54:03 <rodrigc> I'll do the best that I can submitting patches to the infra
17:54:14 <tardyp> what I am saying is that it is one thing to setup a version for a one time test, but it is another to setup one that is constantly updating
17:54:29 <tardyp> and dont create false positives, that are preventing people to work
17:54:32 <rodrigc> it would be nice to get more real help on this, but that's fine, I understand you aren't convinced this is worthwhile
17:54:59 <tardyp> ok
17:55:20 <rodrigc> I'm convinced you're wrong on this, so let's leave it at that and move on
17:55:54 <tardyp> ok
17:56:15 <tardyp> rjarry: good to know that you are motivated to make things progress at debian
17:56:36 <rodrigc> on another note, at least two people have complained that buildbot doesn't give an adequate warning when websockets don't work
17:56:49 <tardyp> let me know if they are okay that we ship the JS code in the tarball, and I'll look at how I can enable that
17:57:55 <tardyp> rodrigc: actually more than that. it is a common problem
17:58:38 <tardyp> that only happens once per person, then nobody is motivated enough to enhance the situation
17:59:26 <rodrigc> https://github.com/buildbot/buildbot/issues/3672
18:00:31 <rodrigc> I'm not familiar with the UI code, but that seems like a day 1 issue which would be nice to improve
18:01:16 <tardyp> I have already tried to improve it, but it actually made things worse
18:03:14 <tardyp> djmitche: maybe it is time to close the meeting. we are already 30 min late
18:03:32 <tardyp> #endmeeting