16:30:04 <djmitche> #startmeeting weekly
16:30:04 <bb-supy`> Meeting started Tue Feb  9 16:30:04 2016 UTC and is due to finish in 60 minutes.  The chair is djmitche. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:04 <bb-supy`> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:30:04 <bb-supy`> The meeting name has been set to 'weekly'
16:30:08 <djmitche> #topic introduction
16:30:11 <djmitche> Hi everyone!
16:30:15 <rutsky> Hi!
16:30:21 <djmitche> agenda is at https://titanpad.com/buildbot-agenda
16:30:21 <infobob> http://paste.pound-python.org/show/K5M79DpVfLxtkUWVcaES/ (repasted for djmitche)
16:30:45 <djmitche> This is our weekly meeting on Buildbot organization and development
16:31:01 <verm__> hi
16:31:02 <djmitche> anyone is welcome to add to the agenda or jump in at any time
16:31:10 <djmitche> wow, big week!
16:31:20 <djmitche> #topic new workers via Docker
16:31:36 <rutsky> is this topic is about BB infra workers?
16:31:40 <djmitche> yes
16:31:51 <djmitche> #info Amar finished the Debian VM, and Dustin installed the docker-worker on it
16:31:51 <verm__> everything seems to be running under bhyve just fine
16:31:58 <verm__> if we want other operating systems please let me know
16:32:00 <verm__> i will add them
16:32:05 <skelly> 10-RELEASE or 11?
16:32:08 <verm__> 11
16:32:15 <djmitche> http://buildbot.buildbot.net/buildslaves/linux
16:32:21 <djmitche> still needs a bit of greening up
16:32:26 <verm__> hoping to get native docker running soon, the postgres docker wouldn't work due to a missing syscall
16:33:06 <djmitche> bdbaddog: has also donated a docker host
16:33:17 <djmitche> http://buildbot.buildbot.net/buildslaves/bdbaddog-eight
16:34:18 <bdbaddog> its sitting on top of centos 7 if that's helpful to know.
16:34:37 <djmitche> works for me :)
16:34:39 <djmitche> so, thanks for that
16:34:43 <djmitche> which brings me to the next topic
16:34:47 <djmitche> #topic DB bugs
16:34:59 <djmitche> http://trac.buildbot.net/ticket/3420 - MySQL failures
16:35:10 <djmitche> http://trac.buildbot.net/ticket/3419 - Postgres failures
16:35:15 <rutsky> djmitche: do you plan to work on them soon?
16:35:27 <rutsky> if not, I would like to take them for this night
16:35:38 <djmitche> so we've fixed some things, and also expanded our compatibility, and that's caused some test failures
16:35:43 <rutsky> currently they are breaking bug2340
16:35:58 <djmitche> I won't get a chance to work on them during my daytime (next 12h)
16:36:20 <djmitche> I think MySQL will be fixed when https://github.com/buildbot/buildbot/pull/1967 lands?
16:36:29 <rutsky> nice, my daytime ends soon, and I'll try to solve them :)
16:36:45 <rutsky> djmitche: I think yes
16:37:01 <rutsky> I propose to add one more test configuration
16:37:11 <rutsky> that will run random subset of tests in random order
16:37:27 <bb-trac> [trac] #3420/undecided (assigned) updated by dustin (To be clear, #3421 is about MySQL-5.7 compatibility.  This bug should be solved by   ...) http://trac.buildbot.net/ticket/3420
16:37:50 <djmitche> rutsky: I like that idea - trial does support it
16:37:51 <rutsky> djmitche: when I'll finish with my SQLite and PG issues, I'll take a look is your PR fixed all my issues
16:38:23 <djmitche> sounds good
16:38:46 <djmitche> the random-tests support takes a random seed, so we can even (hopefully) reproduce failures caused by ordering
16:39:10 <rutsky> good idea!
16:39:43 <rutsky> djmitche: do you writing ticket for that? If not, I'll do it
16:39:50 <djmitche> I'm not yet but I can
16:39:56 <djmitche> go ahead, I'll keep the meeting rolling
16:40:04 <rutsky> ok
16:40:05 <djmitche> #topic CLI tool to talk to API
16:40:16 <djmitche> verm__: ^^ any updates on this?
16:40:27 <djmitche> I think this will help illuminate other uses of the Data API
16:40:35 <verm__> no, working on it soon now that the VM is out of the way. :)
16:40:37 <djmitche> so I'm pretty excited about it conceptually :)
16:40:38 <djmitche> ok
16:40:40 <verm__> i agree there are a few projects i need it for, too
16:40:45 <verm__> so it's a semi-priority for me
16:41:04 <djmitche> #info no updates yet - will work on it now that the Debian VM is finished
16:41:14 <djmitche> #topic Google Summer of Code 2016
16:41:34 <djmitche> #info Mentoring organization applications opened yesterday, close on the 16th
16:41:52 <djmitche> We had a bumper year last year
16:42:00 <djmitche> I think we should do it again!
16:42:04 <tardyp> yeah!
16:42:10 <djmitche> So the first thing we need is a co-admin
16:42:17 <djmitche> tardyp: you've done that for a few years now -- still willing?
16:42:36 <tardyp> if needed yes
16:42:41 <djmitche> OK
16:42:49 <tardyp> if somebody is interrested also I can share
16:42:50 <djmitche> how about
16:42:51 <bb-trac> [trac] #3445/undecided (v:master) created by rutsky (add test configuration that runs random subset of tests in random order) http://trac.buildbot.net/ticket/3445
16:43:02 <djmitche> I'll send an email to the devel list indicating avaialbility of the co-admin role, and mentoring
16:43:19 <djmitche> and check back next week
16:43:50 <djmitche> sorry apps close on the 19th - a week from friday
16:43:59 <rutsky> what responsibilites have co-admins?
16:44:02 <djmitche> so I'll get the application started, and we'll select a co-admin next tuesday
16:44:25 <djmitche> rutsky: organizing the whole process of handling applications, selecting mentors, dealing with errant students, etc.
16:44:40 <djmitche> it's a lot of responsibility but not a lot of time dedicated
16:44:42 <rutsky> I definitely would like to see a log, why nine.bb.net is not working (I have clues, but still it's nice to see exact reason)
16:44:49 <tardyp> looks fine to me
16:44:56 <rutsky> ah, not technical co-admin :)
16:45:11 <djmitche> no - we usually split mentoring into technical and non-technical
16:45:28 <djmitche> #agreed will put out an email to the devel list looking for mentors and co-admin, elect co-admin next tuesday
16:45:42 <djmitche> #info correction, applications due Friday the 19th
16:45:44 <tardyp> basically admin is making sure everything runs fine
16:45:57 <tardyp> has a bunch of admin rights in gsoc app
16:46:18 <djmitche> so speaking of technical co-*,
16:46:19 <tardyp> some of the work consist in removing spamming/irrelevent applications
16:46:40 <tardyp> making sure the mentor are looking at the list of apps, et
16:46:53 <djmitche> we generally have had two mentors - the idea being one is technical and doing reviews, etc., while the other is organizational and talks to the student about schedule, communication, etc.
16:47:08 <djmitche> From those who have mentored, is that worth continuing this year?
16:47:18 <djmitche> bobhood: ^^?
16:47:26 <djmitche> verm__: ^^
16:47:33 <bobhood> Well...
16:47:37 <djmitche> tychoish: ^^
16:47:43 <tardyp> I apply for the UI stuff
16:47:45 <bobhood> What are you asking?  Are you asking if I would recommend it to others?
16:47:56 <djmitche> bobhood: yeah
16:48:04 <djmitche> as admins, should we plan to have 2 mentors per student, or just one?
16:48:10 <bobhood> Yes, I would.  It was a fun experience, and I got lucky with a great student.
16:48:11 <djmitche> it just affects our allocation of students, really
16:48:19 <tardyp> I think its good to have both
16:48:19 <bobhood> I think that was a testiment to how well the applications were filtered.
16:48:45 <bobhood> Having both worked really well, I think.
16:48:47 <djmitche> thanks, yeah, and I think we did a good job of attracting quality students
16:48:48 <djmitche> ok
16:49:05 <djmitche> I'll include that in the email to get others' opinions, but that's good feedback
16:49:21 <djmitche> #info Email will also poll former mentors on the usefulness of having two mentors per student
16:49:33 <djmitche> ok, anything else for GSoC?
16:49:46 <djmitche> rutsky: if you want to co-admin, you've got time to think about it and reply to the ML thread :)
16:50:04 <djmitche> #topic Bug 2340 - worker transition
16:50:09 <djmitche> I think I have a new name for this every week
16:50:19 <rutsky> djmitche: ok, I'll think about it :)
16:50:38 <djmitche> #info DB migrations and DB issues (above) are slowing things down a little
16:50:48 <djmitche> rutsky: and I totally forgot to review your migration until just an hour ago -- sorry about that
16:50:49 <rutsky> we're renaming, renaming, renaming... ism't this is a goal of this task? :)
16:50:52 <djmitche> I didn't put it in taskwarrior :(
16:50:58 <djmitche> true!
16:51:30 <djmitche> anyway, I think the structure was great, and commented on a few of the TODO notes
16:51:43 <rutsky> I made half of refactoring that I planned and stumbled with DB issues with updated SQLAlchemy
16:51:54 <rutsky> djmitche: thanks, I'll take a look at them
16:52:26 <djmitche> I think you might be safest to write the migrations with the old versions for SQLAlchemy and migrate
16:52:31 <djmitche> since that's what the rest of the migrations were written for
16:52:45 <djmitche> if those don't work on newer versions, then those are new problems
16:53:15 <rutsky> I prototyped other topics that needs reviewing in https://github.com/buildbot/buildbot/pull/1962 but since my refactorings are not finished it's nothing to review more than DB migrations atm
16:53:29 <djmitche> OK
16:53:41 <djmitche> are there bits of the migration that can start landing on master yet?
16:53:50 <djmitche> haha, 666 commits :)
16:54:06 <rutsky> unfortunately no --- to many connections with other parts of code
16:54:12 <djmitche> ok
16:54:31 <rutsky> tables columns are used in a lot of places and I can't extract only minimal part from these changes
16:54:40 <djmitche> right, that makes sense
16:54:53 <rutsky> I will provide separate PR for database cleanup
16:55:03 <djmitche> #info not yet able to pull out parts of the migration to land on master - everything is very tied together
16:55:04 <djmitche> ok
16:55:22 <rutsky> database cleanup pisses me off :(
16:55:28 <djmitche> in general I feel like this is going well, but has taken more time than you thought -- are you still comfortable with the amount of work involved?
16:55:56 <rutsky> yes
16:56:06 <djmitche> do you want to (temporarily) remove the compatibility with newer versions of sqlalchemy and migrate? and Postgres-9.5 and MySQL 5.7?
16:56:30 <rutsky> it's slower due to my main work, but it's progressing and I already see end of that PR :)
16:56:33 <djmitche> adding support for new versions isn't part of the bounty project, so that would be absolutely fine
16:56:41 <djmitche> yeah
16:57:37 <rutsky> while that issues are not related to bounty program I feel obligated to handle them too
16:57:56 <djmitche> ok, I can appreciate feelings of obligation :)
16:58:00 * djmitche has those too
16:58:02 <djmitche> pesky things
16:58:14 <rutsky> if they will be too hard to solve, I'll revert to your suggestion
16:58:18 <djmitche> sounds good
16:58:38 <rutsky> but I think it's just few minor things that should be debugged
16:58:44 <djmitche> #agreed if DB issues persist, we will back out some of the new compatibility we've just addd to try to focus on the worker transition
16:58:46 <djmitche> me too
16:58:51 <rutsky> (not easy one, but solvable)
16:58:55 <djmitche> ready for next topic?
16:58:59 <djmitche> almost done :)
16:59:05 <djmitche> (with the meeting I mean)
16:59:11 <rutsky> yes, please
16:59:17 <djmitche> #topic UI testing support
16:59:25 <djmitche> not sure who added this to the agenda -- what's up?
16:59:31 <rutsky> it's me
16:59:36 <tardyp> I think this is a good gsoc project
16:59:46 <rutsky> djmitche: you commented in one of the PRs that this topic should be discussed
16:59:52 <verm__> djmitche: we've always done 2 mentors per student on all the projects i've been part of
17:00:00 <verm__> even if the 'secondary' is just there incase the primary isn't around
17:00:04 <djmitche> verm__: ok, good datapoint too - thanks
17:00:04 <verm__> it happens sometimes
17:00:05 <djmitche> right
17:00:23 <djmitche> rutsky: yes, ok
17:00:24 <verm__> the secondary can be non-technical, too even if all they say is "calm down they'll be back soon" or something similar
17:00:32 <djmitche> haha, yeah
17:00:37 <tardyp> also secondary can help on hard decision issues
17:00:48 <tardyp> wether you shall fail a student
17:00:50 <bobhood> Yeah, I had several meetings with the student when the technical mentor was not there or could not attend.
17:00:57 <djmitche> yeah
17:01:11 <tardyp> its always hard to fail students, but this is necessary, so having another guy who follow the project help a lot
17:01:22 <djmitche> that's a good point - having two people making pass/fail decisions helps with the "one against the other" feelings
17:01:24 <djmitche> yep
17:01:26 <bobhood> But we're off topic.  :)
17:01:27 <rutsky> any ideas of quick'n'dirty solution for UI testing? Is it meaningful to have any?
17:01:30 <djmitche> hehe, it's OK
17:01:37 <djmitche> tardyp: we have *some* karma-based testing for the UI, right?
17:01:53 <tardyp> yes, what we need is e2e tests
17:02:14 <rutsky> tardyp:  not only
17:02:21 <tardyp> we can have more karma tests, but they wont catche the regressions we saw lately
17:02:49 <rutsky> at least tests that emits some events like "press Force Build" and checks that no exceptions are occured would be useful (and not hard to implement)
17:03:11 <rutsky> or at least test that renders whole page and checks for unhandled exceptions
17:03:21 <djmitche> so as a GSoC project, this would not require the student to design the testing framework from scratch
17:03:37 <tardyp> well, the exception are due to what the backend is sending
17:03:37 <djmitche> but more to write some tests, and maybe design a framework for e2e (end-to-end)
17:03:49 <bdbaddog> does trial work with selenium?
17:04:04 <tardyp> so those are really backend frontend interactions
17:04:14 <tardyp> bdbaddog: I've tried this in the beg of the project
17:04:29 <tardyp> but definitly nodejs community have much more powerful tools imho
17:04:36 <bdbaddog> gotcha.
17:04:44 <djmitche> yeah, I think the two need separate test frameworks
17:04:47 <rutsky> tardyp: so it's more like backend sends unexpected stuff for frontend?
17:05:00 <tardyp> yes from what I can tell
17:05:04 <djmitche> the interface between backend and frontend has been tricky, too
17:05:10 <djmitche> we have a way of generating fake data for tests right now, right?
17:05:36 <tardyp> we have some tools, yes
17:05:49 <tardyp> but they are not really generating based on a shared spec
17:05:57 <tardyp> the spec is still separated
17:06:06 <djmitche> so that might be a good part of the GSoC project?
17:06:10 <tardyp> this is *kindof* what I'm working on with the raml
17:06:14 <djmitche> ahh, ok
17:06:16 <tardyp> yes
17:06:29 <djmitche> huh, I missed that you were using raml -- someone else just suggested that to me at Mozilla
17:06:33 <tardyp> how to share the spec with raml, is also a good gsoc idea
17:06:38 <djmitche> gotcha
17:06:54 <djmitche> I'll try to put those in bugs so we can track them, if they aren't already
17:07:00 <tardyp> https://github.com/spotify/ramlfications/pull/80
17:07:18 <tardyp> working with raml python parser upstream to make raml python lib move forward
17:08:25 <djmitche> awesome!
17:08:28 <tardyp> So we agree to manage that via gsoc project (e2e test?)
17:09:05 <tardyp> I think I created a trac already
17:09:08 <djmitche> #agreed will manage testing improvements as a GSoC project, maybe two (E2E tests and improving the fidelity of the backend used for tests, perhaps with RAML)
17:09:08 <djmitche> ok
17:09:29 <djmitche> #topic Development week-in-review
17:09:32 <rutsky> I would like to see some quick'n'dirty solution atm... but if there is no way to get any kind-of-test, then GSoC
17:09:58 <bdbaddog> any progress on http://trac.buildbot.net/ticket/3418 ?
17:10:23 <rutsky> speaking about UI testing :)
17:11:03 <djmitche> that's the sidebar-retraction bug
17:11:08 <bdbaddog> yes.
17:11:11 <djmitche> yes, it'd be nice to see that tested to make sure it doesn't regress on any browser
17:11:34 <djmitche> but I think that means E2E and finding a way to run each of those browsers within the testing infrastructure
17:11:38 <djmitche> docker might help with that
17:11:48 <djmitche> but I don't see anything quick-n-dirty
17:12:09 <bdbaddog> wouldn't any automated testing be better than none at this point?
17:12:15 <tardyp> this one is even trickier, as this is depending on the window size, and css placement
17:12:33 <bdbaddog> for me that bug happens regardless of browsersize.
17:12:44 <bdbaddog> even if I spread across two 1080p monitors
17:13:14 <tardyp> a, yes, you are right, indeed
17:13:37 <tardyp> its a css bug in guanlecoja-ui
17:13:49 <bdbaddog> I'd take a wack at it but looks like quite a bit to ramp up on just to understand the UI code.
17:14:16 <tardyp> yeah its quite an easy one if you are into css magic, but if you are not
17:14:17 <djmitche> right, so just unit-testing a component wouldn't reproduce it
17:14:23 <tardyp> can take weeks..
17:14:39 <djmitche> bdbaddog: and we do have any automated testing, just not a lot of it -- and not an easy way to test the things that we're seeing fail
17:15:18 <djmitche> so as for the week in review
17:15:29 <tardyp> right, and believe me I would really prefer not breaking stuff :-}
17:15:47 <djmitche> haha :)
17:16:09 <djmitche> we've got a few PRs marked merge-me -- but merging is up to rutsky at the moment to avoid conflicts
17:16:10 <bdbaddog> is there any automated UI testing at this point?
17:16:39 <bdbaddog> I know we have trial and lots of that level of testing.
17:17:08 <bb-github> [13buildbot] 15rutsky pushed 6 new commits to 06master: 02https://git.io/vgzG0
17:17:08 <bb-github> 13buildbot/06master 14f299ed1 15Vladimir Rutsky: add docs on using BUILDBOT_TEST_DB_URL with databases in Docker
17:17:08 <bb-github> 13buildbot/06master 149bed6cb 15Vladimir Rutsky: run in tests example whole test suite
17:17:08 <bb-github> 13buildbot/06master 144976c86 15Vladimir Rutsky: fix MySQL SQLAclhemy URI
17:17:39 <rutsky> djmitche: regarding BUILDBOT_TEST_DB_URL, I used psycopg and saw that you use pg8000, any reason to prefer one over another?
17:18:05 <djmitche> rutsky: no, not really
17:18:15 <djmitche> pg8000 is pure-python so easier for me to install
17:18:21 <bb-meta_> build #342 of py27-tw1020 is complete: Success [3build successful]  Build details are at http://buildbot.buildbot.net/builders/py27-tw1020/builds/342
17:18:29 <djmitche> anyway, there are a few new bugs, but I'll go triage those now, offline
17:18:32 <rutsky> pg8000, I believe, should have less dependencies (i.e. psycopg should depend on system-wide libpg-dev or something like that)
17:18:36 <djmitche> I don't want to belabor the meeting :)
17:18:52 <djmitche> rutsky: right, but we should support both (which means we should probably test both....)
17:19:01 <rutsky> hehe
17:19:12 <djmitche> or maybe pick the one that works better and recommend it and test that one
17:19:18 <djmitche> either way is OK by me
17:19:32 <djmitche> I think this is the first time I've seen differences between tehm
17:19:45 <rutsky> maybe there is no difference actually
17:19:46 <djmitche> OK, any other business to attend to?
17:19:58 <djmitche> we can keep chatting obviously, just anything else to get into the notes
17:20:01 <rutsky> I think different errors were due to other differences in envs
17:20:07 <djmitche> that makes sense
17:20:24 <rutsky> I think no, got plenty of work to do :)
17:21:16 <djmitche> haha, me too
17:21:17 <djmitche> ok
17:21:19 <djmitche> #endmeeting