16:30:43 <djmitche> #startmeeting weekly
16:30:43 <bb-supy> Meeting started Tue Aug 30 16:30:43 2016 UTC and is due to finish in 60 minutes.  The chair is djmitche. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:43 <bb-supy> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:30:43 <bb-supy> The meeting name has been set to 'weekly'
16:30:58 <djmitche> #topic Introductions
16:31:14 <djmitche> https://titanpad.com/ep/pad/view/buildbot-agenda/6o1VWh3KO9
16:31:21 <djmitche> who's here?
16:31:38 * tardyp wave
16:31:48 <tardyp> 👋
16:31:51 <gracinet> I'm here
16:31:54 <rutsky> hi!
16:31:59 * SheldonCooper is observing
16:32:00 <djmitche> me too! :)
16:32:14 <djmitche> #topic week in review
16:32:21 <djmitche> looks like a *lot* of Python-3 work!
16:32:24 <jaredgrubb> im watching
16:32:26 <tardyp> cool, we are back at somehow medium attendence
16:32:30 <djmitche> indeed!
16:32:39 <djmitche> I guess it's after the holidays :)
16:32:44 <tardyp> yep
16:33:03 <tardyp> so yes py3 by rodrigc has generated a lot of PRs
16:33:34 <djmitche> hopefully that will be "locked in" with some tests?
16:33:35 <tardyp> in two days probably the same amount compared to lasts year gsoc :-/
16:33:44 <djmitche> haha
16:34:00 <tardyp> if we go far enough I think we can enable those py3k tests in travis
16:34:23 <tardyp> but I think with him we can be confident we'll go further
16:35:01 <tardyp> problem is until we have some of the unittests working, we cannot enable much testing
16:35:08 <djmitche> right
16:35:22 <djmitche> #info lots of Python-3 PRs from rodrigc -- awesome!
16:35:27 <djmitche> I see a guanlecoja upgrade too
16:35:34 <djmitche> any other gems in there that I've missed?
16:35:53 <rutsky> is trial is still not working in Python 3?
16:36:36 <tardyp> rutsky: I am not sure of the exact progress on the unit test yet
16:36:58 <tardyp> but the PRs are starting to be much smaller fixes on details
16:37:29 <djmitche> even suggestions of grepping for from __future__ <something> seemed good
16:38:26 <djmitche> anything else from this week to highlight?
16:38:44 <tardyp> not really more
16:39:38 <tardyp> We got a bunch of discussion with users adopting nine and having some scalability issues
16:39:57 <tardyp> so I wrote https://github.com/tardyp/buildbot_profiler
16:40:16 <djmitche> #info Pierre built "buildbot_profiler"
16:40:27 <tardyp> turns out that outside the obvious db api, a lot of it is about inlinecallabks
16:40:54 <djmitche> wow, that's cool
16:40:58 <tardyp> I am not sure yet if inlinecallbacks are really slow or if this is the profiling method which cannot decode the traceback properly, and only show the inlinecallbacks
16:41:19 <djmitche> #info profiling a typical master shows a lot of time in inlineCallbakcs
16:41:21 <djmitche> #undo
16:41:21 <bb-supy> Removing item from minutes: <ircmeeting.items.Info object at 0x80698e390>
16:41:24 <djmitche> #info profiling a typical master shows a lot of time in inlineCallbacks
16:41:59 <SheldonCooper> we're still working on porting our work scripts for buildbot-0.9, plus I'm still working on the rc2 version bump
16:42:08 <tardyp> mayve python3 with await/async might help with inlinecallback performance
16:42:15 <djmitche> I see callWithContext, too -- I've never figured out what that is (something to do with twisted.log)
16:42:29 <djmitche> SheldonCooper: that's awesome, thanks!
16:42:34 <djmitche> #topic 0.9.0rc2
16:42:53 <SheldonCooper> the autobahn dep bump needed 3 pkg version bumps plus 7 new pkgs to add to the Gentoo tree
16:42:58 <djmitche> #info Feedback about nine scalability led to profiling (see above)
16:43:25 <djmitche> #info SheldonCooper working on bumping to rc2 in Gentoo, testing out 0.9
16:43:34 <SheldonCooper> yes
16:43:56 <tardyp> arg I wasn't aware of it
16:44:08 <tardyp> autobahn version bump is only for tw 16.3
16:44:23 <djmitche> is that the version of twisted with a py3-compatible PB?
16:44:25 <tardyp> older autobahn works fine with tw < 16.2
16:44:40 <tardyp> no, thats only in master right now
16:45:10 <djmitche> ok
16:45:21 <SheldonCooper> I'll play with getting the deps set for that then, currently work is still using twisted 15
16:45:22 <djmitche> sa2ajj: you around to talk about 0.8.x releases?
16:46:09 <tardyp> SheldonCooper: tw15 is fine
16:46:29 <SheldonCooper> tardyp, but you have a hard requirement for autobahn-0.16 in setup.py
16:46:41 <djmitche> could we just let twisted pull the newer autobahn itself?
16:46:53 <djmitche> oh I see
16:46:58 <tardyp> it was to actually simplify the setup.py
16:46:58 <djmitche> this is somethign like what npm calls a "peer dep"
16:47:20 <SheldonCooper> which setuptools tried to use pip to install on me, caused the sandbox violation
16:47:35 <tardyp> well it is a version incompatibility: if tw >= 16.3 autobahn > xx is needed
16:47:36 <djmitche> haha, yuck
16:47:51 <djmitche> right - npm has a way for twisted to express that
16:47:53 <djmitche> anyway
16:48:09 <djmitche> I don't think python packaging can express it
16:48:58 <SheldonCooper> I'd have to patch that out of setup.py to open up the the twisted/autobahn dep
16:49:12 <SheldonCooper> or the install will fail
16:49:16 <djmitche> that might be reasonable
16:49:19 <tardyp> +1
16:49:28 <djmitche> maybe we could add a comment so that your patch doesn't seem totally out of the blue?
16:49:55 <djmitche> gracinet: do you want to talk about tls next?  I just saw the bug update
16:49:59 * sa2ajj is now here :/
16:50:21 <SheldonCooper> I can keep the patch local for Gentoo tree if you can't remove it from setup.py for other install systems
16:50:23 <gracinet> djmitche: not much to say, I've had it on my mind for a long time
16:50:45 <djmitche> ok
16:50:54 <gracinet> and last time I installed a master, I redid your experiment with the connection string, realised that it was more complicated worker-side
16:50:56 <djmitche> SheldonCooper: I think that makes sense
16:51:26 <djmitche> gracinet: ok, let's put something on the agenda for next week then :)
16:51:48 <djmitche> #topic 0.8.x release
16:51:52 <gracinet> can't promise, but I'm attacking it right now following tardyp's tips
16:51:57 <gracinet> yes will tell next week
16:51:57 <djmitche> sa2ajj: ^^ what news
16:52:00 <djmitche> gracinet: well a status update anyway :)
16:52:09 <sa2ajj> a) i remove 0.8.14 visibility on pypi
16:52:18 <sa2ajj> s/remove/removed/
16:52:31 <SheldonCooper> the gentoo ebuild can do complex dependencies for autobahn/twisted
16:52:33 <sa2ajj> b) 0.8.14p1 is still pending :(
16:53:07 <djmitche> #info 0.8.14 is hidden on pypi
16:53:20 <sa2ajj> not an excuse: we transferring 30+ developers to yocto
16:53:20 <djmitche> #info 0.8.14p1, with minor fixes, is not released yet, but soon
16:53:26 <sa2ajj> this is fun :|
16:53:34 <djmitche> that's a lot of work :)
16:53:48 <sa2ajj> during this week i'm focused on the new dev.env.
16:54:15 <sa2ajj> i hope to have time for 0.8.14p1 on the coming sunday/monday
16:54:25 <djmitche> awesome
16:54:31 <djmitche> we are all on open-source time here so that sounds great :)
16:54:57 <djmitche> ok
16:55:04 <djmitche> #topic Buildbot phone-home feature
16:55:05 <sa2ajj> the reason i hid the 0.8.14 is simple: there was a number of questions here like "0.8.14 from pypi and no tag'
16:55:23 <djmitche> #undo
16:55:23 <bb-supy> Removing item from minutes: <ircmeeting.items.Topic object at 0x80698e4d0>
16:55:37 <djmitche> yes, I think that makes sense
16:55:49 <djmitche> it's just basically running through CuttingReleases again to ship 0.8.14p1, right?
16:55:59 <djmitche> no source changes?
16:57:29 <tardyp> looks like he hungup
16:57:34 <djmitche> aww
16:57:38 <djmitche> I don't show parts/joins, thanks
16:57:47 <djmitche> #topic Buildbot phone-home
16:57:54 <djmitche> #info (from agenda) Insert in buildbot nine a "call home" statistic, which would send to bb.net some statistics about current running master (number of workers, number of builds, version, plugins) at every master startup.
16:58:06 <tardyp> I did raise that
16:58:06 <djmitche> back in the 0.7.x days, we had something like this
16:58:14 <tardyp> oh!
16:58:15 <djmitche> the logo on the web UI was hosted on buildbot.net
16:58:22 <tardyp> ah!
16:58:29 <tardyp> nice trick
16:58:40 <djmitche> we didn't disclose it though :(
16:58:47 <djmitche> and it also tracks *users*, not installs
16:58:51 <djmitche> so I didn't really like it
16:59:21 <tardyp> my point is we are really blind in term of who runs which version
16:59:23 <djmitche> yeah
16:59:30 <tardyp> for example who runs rc2 I have no idea
16:59:33 <djmitche> having that info back then was pretty useful, I'll admit
16:59:39 <tardyp> should I consider it well tested?
16:59:48 <djmitche> right
17:00:11 <djmitche> so I'm in favor of this -- IMHO it should be in the default config with a comment that says what it does and how it helps us
17:00:14 <gracinet> I think it should be an opt-in
17:00:19 <tardyp> so I would like to introduce that feature for 0.9.0
17:00:38 <djmitche> I think this is somewhere between "opt-in" and "opt-out"
17:00:40 <djmitche> it's in the file, which hopefully you read over before running
17:00:44 <gracinet> although, well the conf gets generated so
17:00:49 <djmitche> and the comment makes it clear that things will work just fine without it
17:00:55 <tardyp> the problem of opt-in is that with our python config I doubt we will touch the upgrades
17:01:15 <djmitche> true
17:01:19 <djmitche> how about this:
17:01:20 <gracinet> touch the upgrades ? ah you mean those that had been installed before ?
17:01:24 <djmitche> three levels
17:01:24 <bdbaddog> Perhaps also a warning on startup if you're opt'd in?
17:01:27 <djmitche> 1. no phone-home
17:01:33 <tardyp> My proposal was rather opt out with
17:01:35 <djmitche> 2. just the master version
17:01:39 <djmitche> 3. full data
17:01:40 <tardyp> a message on upgrade
17:01:57 <djmitche> and 2 is the default
17:02:04 <djmitche> tardyp: yes, I like that
17:02:06 <bdbaddog> o.k. a message on startup though. so it's not just a one time message that user can miss and then get irate about.
17:02:14 <tardyp> so if there is nothing in master.cfg it will still send minimal info
17:02:15 <djmitche> default to 2, with a message on upgrade and startup if the default is applied
17:02:19 <djmitche> right
17:02:20 <gracinet> I can imagine Debian and co patching this out, maybe I'm over-cautious ?
17:02:36 <djmitche> gracinet: that would be fine
17:02:42 <tardyp> yes
17:02:42 <djmitche> or they can change the default
17:02:58 <gracinet> yes they could, /me checks somethin
17:03:11 <djmitche> if the language in the relnotes, comments, messages, etc. makes it clear that this is just info to help us guide development, I think we might be able to get away with it
17:04:06 <gracinet> (I have no pbm personnally activating it on my masters)
17:04:36 <tardyp> so looks like there is agreement
17:04:39 <bdbaddog> for sure some of my clients will want this disabled.
17:04:50 <bdbaddog> +1 on proposed implementation/docs/etc
17:04:51 <tardyp> I will come up with a proposal implementation this week
17:04:55 <djmitche> cool, thanks
17:05:08 <djmitche> #agreed will default to sending basic information, but with loud warnings
17:05:23 <djmitche> #agreed will offer the option of sending more detailed information (number of builders, etc.)
17:05:31 <djmitche> I like it!
17:05:37 <djmitche> Any Other Business?
17:06:09 <SheldonCooper> sorry, was sidetracked by things
17:06:13 <tardyp> nine.bb.net just went back
17:06:26 * sa2ajj reads the history
17:06:28 <SheldonCooper> just an FYI, the phone-home would have to be optional
17:06:39 <tardyp> I did change the config to run master.cfg
17:06:49 <tardyp> SheldonCooper: yes, that is what we mean by opt-out
17:06:55 <SheldonCooper> our installs are behind firewalls without exteranl internet access
17:06:58 <djmitche> SheldonCooper: definitely -- question was, is it OK to default to sending the master version?
17:07:15 <djmitche> good point -- it would need to fail safe in that situation (like, try to phone, but not crash the master if it fails)
17:07:17 <sa2ajj> djmitche: sorry for being away, yes, no source changes, only minor docs changes ('released on") and pushing the tag
17:07:21 <SheldonCooper> yeah
17:07:54 * sa2ajj sometimes think of looking @ meetingbot :/
17:07:57 <djmitche> #agreed will need to be resilient to failure, as many systems are behind firewalls
17:08:05 <SheldonCooper> and FYI, we are using 0.8.12 one master. will switch to 0.9 next with possibly 5 masters
17:08:11 <djmitche> awesome :D
17:08:23 <djmitche> so I did have one other topic
17:08:28 <djmitche> #topic docs builds failing
17:08:32 <djmitche> the error from ansible:
17:08:34 <djmitche> failed: [localhost] (item=gmake -C master/docs PATH:=/home/bbuser/doctools/bin:$(PATH) VERSION=latest) => {"changed": true, "cmd": ["gmake", "-C", "master/docs", "PATH:=/home/bbuser/doctools/bin:$(PATH)", "VERSION=latest"], "delta": "0:00:25.724609", "end": "2016-08-29 04:47:53.856514", "failed": true, "item": "gmake -C master/docs
17:08:34 <djmitche> PATH:=/home/bbuser/doctools/bin:$(PATH) VERSION=latest", "rc": 2, "start": "2016-08-29 04:47:28.131905", "stderr": "python: not found\nenchant module import failed:\nNo module named enchant\nSpell checking disabled.\n\nException occurred:\n  File \"/home/bbuser/doctools/lib/python2.7/site-packages/sphinx/environment.py\", line 1867, in create_index\n    for
17:08:34 <djmitche> type, value, tid, main in entries:\nValueError: too many values to unpack\nThe full traceback has been saved in /tmp/sphinx-err-RxKPMr.log, if you want to report the issue to the developers.\nPlease also report this if it was a user error, so that a better error message can be provided next tim!
17:08:38 <SheldonCooper> about 5 slaves atm, but that will change too with the new masters
17:08:40 <djmitche> whoops, that should've pastebinned, sorry
17:08:54 <djmitche> 5 slaves / 5 masters would be pretty unusual :)_
17:08:59 <sa2ajj> i think it's related to sphinx changes
17:09:24 <djmitche> yeah, I'm confused because (a) we installed enchant and (b) it looks like it keeps running even without enchant
17:09:25 <tardyp> djmitche: I've run the docs manually and it works
17:09:38 <djmitche> is that too-many-values thing enchant-related?
17:09:48 <tardyp> I created a new venv because I didn't know where the ansible venv was
17:09:59 <tardyp> so probably prunning the venv will solve the situation
17:10:00 <sa2ajj> djmitche: what would be a link to the bb build?
17:10:07 <sa2ajj> s/the/a/
17:10:11 <tardyp> sa2ajj: its ansible
17:10:35 * sa2ajj . o O (english grammar is so different from russian one :/ )
17:10:40 <djmitche> sa2ajj: yeah, it happens in the ansible run
17:10:52 <djmitche> tardyp: ok, I'll try that
17:10:59 <djmitche> anyway, we can probably take that bit offline
17:11:04 <tardyp> djmitche: by the way I registered to the admin mailing list but still waiting for approval
17:11:04 <sa2ajj> well, freebsd does not have python, it only has python2
17:11:07 <djmitche> #action djmitche to work on getting docs builds running again
17:11:25 <sa2ajj> is it docs.bb.net?
17:11:30 <tardyp> yes
17:11:52 <sa2ajj> ok, i'll have a look as well
17:13:30 <djmitche> I'll let you know what I find in the next few minutes (which is about all I have..)
17:13:33 <djmitche> ok
17:13:38 <djmitche> hearing no other business.....
17:13:46 <djmitche> #endmeeting