15:00:46 #startmeeting RDO meeting (2016-02-10) 15:00:46 Meeting started Wed Feb 10 15:00:46 2016 UTC. The chair is apevec. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:46 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:46 The meeting name has been set to 'rdo_meeting_(2016-02-10)' 15:00:49 ya, we probably need to advertise that question to have any chance of real feedback 15:00:59 #topic roll call 15:01:02 o/ 15:01:05 present 15:01:07 o/ 15:01:07 o/ 15:01:17 o/ 15:01:18 #info agenda https://etherpad.openstack.org/p/RDO-Meeting (scroll to NEXT=> ) 15:02:00 ok, let's give few min 15:02:49 number80, jruzicka - yt? 15:02:52 o/ 15:04:07 ok, first topic: 15:04:14 #topic Trello bot for #rdo 15:04:53 who proposed it? 15:05:00 there's one example, on #rdo-puppet 15:05:15 I don't like its noise level 15:05:17 I added it, because it was discussed last week and someone said we should look at how it works in #rdo-puppet then decide 15:05:51 EmilienM said it was probably configurable to reduce the amount of noise 15:05:53 so if we add it, I do limit bot notifications 15:05:58 I'm not particularly a fan of the noise it generates but if I may extend the topic 15:06:10 jpena: it's not but we can improve it 15:06:22 yes, we also have the other bot for delorean builds 15:06:25 I really want a bot to spam me with reviews that land in openstack-packages/*, jenkins build jobs and eventually monitoring status from the monitoring PoC 15:06:35 we might want to unify all in one notification bot 15:07:02 dmsimard, is there something like that in monitoring tools you're looking at? 15:07:14 +1 to reviews on openstack-packages getting notified here 15:07:24 EmilienM, where does puppy code live? 15:07:41 https://github.com/fredericlepied/zircbot 15:07:41 is it trello-specific? 15:07:44 apevec: I've built a bot at my previous job using http://errbot.io/, it's dead simple, mature and it's a googler project 15:07:47 and https://github.com/fredericlepied/ztrellohook 15:07:56 o/ 15:07:58 apevec: you can extend it with webhooks and everything 15:08:04 flepied1 is the author 15:08:28 dmsimard, do you want to compare those two bots? 15:08:40 I'm very biased towards what I've already worked with 15:08:47 which one is better at chess 15:08:54 EmilienM seemed to like errbot :p 15:09:28 I have no preference, our use case is much simpler that yours 15:09:28 let's take errbot then, but extend it to include both trello(highly filtered!) and derekh's version.csv checker ? 15:09:41 +1 15:09:43 derekh's version.csv checker is already covered in monitoring 15:09:54 derekh, ^ where is code for your versions.csv check job? 15:10:05 apevec: http://209.132.179.184:3000/#/client/RDO%20Monitoring/rdo-monitoring-01.osop.rhcloud.com?check=check-delorean-mitaka-current-passed-ci 15:10:27 apevec: he sent it to me, it's a bash one-liner that greps for FAILED 15:10:35 nice :) 15:11:01 I made something nagios-friendly in python 15:11:04 apevec: dmsimard should I turn mine off ? 15:11:04 dmsimard, so you can create irc notifications from uchiwa? 15:11:05 so we can eventually graph stats 15:11:23 apevec: irc is one of the "handlers" from sensu, yes 15:11:38 derekh: not yet, this is not in production still 15:11:46 o/ 15:12:03 dmsimard: cool, just shout whenever and I'll turn mine off 15:12:06 ok, let's write down summary 15:12:21 #agree #rdo to have single notifications bot 15:12:34 #agreed #rdo to have single notifications bot 15:13:04 dmsimard: are you taking the action to set that up? 15:13:06 #action dmsimard to put errobt in production 15:13:12 ah guess so :p 15:13:16 I volounteered apparently 15:13:18 :P 15:13:21 it's part of monitoring :) 15:13:55 so far I've been working on monitoring between troubleshooting trunk and with the amount of breakage that has been happening I haven't exactly had a whole lot of time to spend on it 15:14:19 I'll try to at least wrap up the monitoring portion this week so I can move on to bot next week. 15:14:40 sounds good 15:14:56 dmsimard, i can offer some help on errbot if you need. 15:14:59 maybe we can have trunk status as a quick topic now? 15:15:06 apevec: sure 15:15:13 #topic trunk status 15:15:36 consistent is there 15:16:04 today 15:14 UTC 15:16:08 but CI ? 15:16:14 There was a lot of backwards incompatible changes that broke packstack, tripleo, puppet-openstack and others. 15:16:21 #link https://review.openstack.org/#/q/topic:bug/1539793 15:16:29 #link https://review.openstack.org/#/q/topic:bug/1542486 15:16:40 yep we are good from a packaging POV, but https://etherpad.openstack.org/p/delorean_master_current_issues has some issues around nova 15:17:13 And more recently changes in both heat and tempest were identified and submitted for fixes in https://review.openstack.org/#/c/278081/ and https://review.openstack.org/#/c/278068/ 15:17:43 I like having a weekly (hopefully quick) topic of trunk status 15:17:44 puppet-openstack is fixed by now but we are still working on submitting/merging the necessary fixes to tripleo and packstack. 15:17:49 trown: +1 15:18:02 I'm done with trunk status 15:18:16 and I think I wasn't chair so that probably didn't register 15:18:16 thanks 15:18:25 oops 15:18:33 #chair dmsimard trown jpena 15:18:33 Current chairs: apevec dmsimard jpena trown 15:18:34 #link etherpad for helping with trunk troubleshooting https://etherpad.openstack.org/p/delorean_master_current_issues 15:19:00 #link https://review.openstack.org/#/q/topic:bug/1539793 15:19:04 #link https://review.openstack.org/#/q/topic:bug/1542486 15:19:08 so nova api db is merged in packstack 15:19:21 t-h-t is open 15:19:25 #info heat tempest tests are fixed by https://review.openstack.org/#/c/278081/ and https://review.openstack.org/#/c/278068/ 15:19:37 dmsimard: whatever you need for packstack let me know 15:20:01 apevec: I have my code editor opened for one more patch to t-h-t for the nova-neutron with v3 keystone issue 15:20:04 apevec: regarding rhbz #1301384. When do you think you'll push the build to rawhide for python-keystoneauth1? Looks like you only committed the changes to git 15:20:25 o/ 15:20:31 imcsk8, nova-neutron change https://review.openstack.org/#/c/277867/ 15:20:35 \o 15:20:39 apevec: Actually, I am just re-reading the bug report now 15:20:40 * number80 was distracted by HRT crap 15:20:45 missed the depends on issues 15:21:09 pabelanger, OT - let's sync after meeting 15:21:12 number80, HRT ? 15:21:17 #chair number80 15:21:17 Current chairs: apevec dmsimard jpena number80 trown 15:21:28 apevec: will test it 15:21:41 apevec: Talent people, whatever the acronym is now 15:22:21 TA - but OT :) 15:22:49 dmsimard, trown - ok, we are >< close to CI pass? 15:23:39 apevec: I am always pretty pessimistic about that, but I think we have a good handle on the currently known issues 15:24:06 my pessimism revolves around new issues coming before we get everything merged before the current known ones 15:24:16 yeah 15:24:19 which is rooted in experience 15:24:53 we have non-milestone testday planned for the next week, hopefully no new bug issues will come up 15:24:54 apevec: o/ 15:25:05 apevec: apologies, didn't know a meeting was going on 15:25:10 jschlueter, \o 15:25:17 #chair jschlueter 15:25:17 Current chairs: apevec dmsimard jpena jschlueter number80 trown 15:25:17 apevec: yeah, I mean, we're fairly close 15:25:19 pabelanger, np 15:25:20 oh... I did not realize there was another testday next week 15:25:27 *IF* no other breaking changes occur 15:25:33 seems so close to the previous one 15:25:39 ack 15:25:42 Right now breaking changes are occuring faster than we can fix them so we're already behind 15:25:50 https://www.rdoproject.org/testday/ 15:26:01 trown, idea was to keep it monthly 15:26:11 m3 is only in March 15:26:23 makes sense 15:26:44 would be really nice to have a fully consistent current-passed-ci repo for that 15:27:02 yes! 15:27:33 if it looks like tripleo patches will be the only hold up and they are in good shape upstream (at least one +2), I can put fixes in the images again 15:27:43 ack 15:27:57 ok, let's move on 15:28:08 #topic Changes to Delorean 15:28:41 we've few pending which would be good to merge before pushing first RPM 15:29:05 one particular is https://review.gerrithub.io/254505 Use fedora guidlines for pre-release packages 15:29:21 we went back and forth there 15:29:36 it's now following fedora pre-release versioning guidelines 15:30:03 it also wins over current versioning: 15:30:11 rpmdev-vercmp 1-dev9999999999 1-0.120160210.shorthash 15:30:17 1-dev9999999999 < 1-0.120160210.shorthash 15:30:17 yep I just wanted to give one last chance for dissent on that one since it has morphed alot 15:30:31 so new builds with this patch will be able to upgrade current builds 15:30:35 oh that is handy, I wasnt sure about that part 15:30:51 I like that one much more than playing with epoch 15:31:14 yeah, epoch was a hack I suggested 15:31:19 epoch should be avoided if at all possible 15:31:20 ya I did not mind epoch until thinking about our source rpms 15:31:28 it could work but better not mess with it 15:31:39 we may have some cases where this doesn't work, though. It looks like gnocchi has version 1.3.4 in liberty, but delorean is building 1.3.1 15:31:39 *nods* 15:31:51 jpena, that's upstream issue 15:32:00 yep, that's what I was going to say :) 15:32:02 they need to push newer tag on master 15:32:10 2.0.0.0b1 or something 15:32:10 jpena: ya for those we have to solve on case by case, and introduce epoch = epoch+1 if upstream is uncooperative 15:32:28 ack 15:32:32 which is still better than the crazy epoch solution 15:32:48 so if no -1s let's merge this 15:32:49 s/crazy/date based/ 15:33:02 the pre-release versioning? 15:33:04 last call :) 15:33:30 just to be sure, because, the best I can do with epoch is abstention :) 15:33:40 number80 https://review.gerrithub.io/254505 15:33:49 +1 15:33:50 number80: yep the patch as proposed is pre-release versioning 15:33:52 number80, no Epoch 15:34:11 ok, we were speaking both 15:35:08 number80, epoch mentioned was for case by case when upstream has wrong versioning on trunk 15:35:21 e.g gnocchi 15:35:32 apevec: the patch submitted was injecting epoch in all projects 15:35:33 pradk, ^ please push 2.0 beta/alpha tag 15:35:34 and would not be a delorean change, but a spec change on rpm-master 15:35:45 which was ok, if it were injected by delorean 15:35:48 number80, please look at the latest patchset 15:37:13 I don't see the epoch one 15:37:24 while at it, there are few more patches ready for reviews https://review.gerrithub.io/#/q/status:open+project:openstack-packages/delorean+branch:master 15:37:31 number80, there is none, now 15:37:45 we're discussing https://review.gerrithub.io/254505 15:37:50 only 15:37:58 I think we are good on the delorean patch, and could discuss the case by case for inconsistent upstream when/if it next arises 15:38:00 yes, I'm ok with this one 15:38:02 for gnocchi, I'd like to push once more upstream to fix it 15:38:14 yep +1 15:38:32 EmilienM, ^ any connections in gnocchi :) ? 15:38:33 https://review.gerrithub.io/#/c/262031/ 15:38:38 ^ this one is ok 15:38:51 pradk: could you check for the gnocchi tag issue? 15:39:03 pradk is the best guy to help in gnocchi 15:39:32 number80, that one is part of jpena's https://review.gerrithub.io/#/q/topic:delorean-packaging series 15:39:41 apevec, number80, lets chat with jd after the meeting, I spoke to him already but he's not inclined to do so as he manages the tagging for gnocchi 15:40:07 ack 15:40:22 apevec: reviewing the others as well 15:40:39 pradk, just ask him to run python ./setup.py --version on gnocchi master.... 15:41:16 ok, let's finish reviews off-meeting, I don't think others are controversial 15:41:18 apevec, yes already had the conversation .. we'll be releasing 2.0 soon which should bump master to 2.0.0+ 15:41:35 pradk, but there are also pre-release tags 15:41:52 apevec, lets discuss after the mtg 15:42:05 ack 15:42:12 #agreed merge https://review.gerrithub.io/254505 15:42:44 #action pradk et al discuss with jd gnocchi 2.0 pre-release tagging 15:43:24 #action everybody review open delorean reviews for 0.1 delorean RPM 15:43:42 #topic RDO liberty production CI jobs 15:44:08 Sorry I got sidetracked, just a quick update - I managed to get the tempest revert merged to resolve ongoing trunk issues with heat 15:44:42 dmsimard, nice, one more down... 15:44:54 re. liberty CI bz 1304395 is a blocker? 15:45:06 adarazs, ^ 15:45:34 apevec: looking. 15:45:38 trown, ^ https://bugzilla.redhat.com/show_bug.cgi?id=1304395#c3 15:45:56 apevec: yep, it blocks the jobs on centos ci, probably it would block anybody trying to install and build images for rdo-manager. 15:46:17 apevec: for example this https://ci.centos.org/view/rdo/job/rdo_manager-periodic-7-rdo-liberty-production-centos-7.0-templates-virthost-minimal-neutron-ml2-vxlan-smoke/ 15:47:07 how are images built in those jobs? 15:47:21 trown, is quickstart working for liberty? 15:47:39 3o-quickstart 15:47:53 apevec: yep, and I use tripleoclient from liberty release for building all images... even trunk 15:48:13 trown, so what's wrong in that bz? 15:48:19 looking 15:48:50 also what are those jobs, are they not all moving to use 3o-quickstart? 15:48:51 apevec: btw this bug blocks the image building too, but at least for this I could add an easy workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1278972 15:49:39 adarazs, trown - can you sync up about those jobs, I though weshay said all jobs will move to use 3o-quickstart images 15:49:42 apevec: eventually they will, but we don't have the bindings in khaleesi for them yet and this is what we can currently do to test production repos on centos. 15:49:45 or did I misunderstand? 15:49:58 ah I see https://github.com/redhat-openstack/tripleo-quickstart/blob/master/playbooks/roles/images/build/templates/build-images#L12 15:50:42 apevec: there are still a bunch of khaleesi based jobs running on centosci, and the production repo jobs are in that category 15:51:02 adarazs, I don't know is it worth to debug old jobs, can't we just move "eventually" forward? :) 15:51:37 apevec: I am working with khaleesi folks to integrate the quickstart with there jobs so that the quickstart does not have to maintain all possible deployment scenarios and can just do the one thing well of setting up the environment 15:52:21 nice 15:52:30 in any case, the topic for this meeting I think is more around who is responsible for CBS packages? maintainer in rdoinfo? maintainer in fedoradb? 15:53:06 I am neither of these things for a lot of the tripleo packages and would rather not be until we have a simpler workflow to allow managing so many packages 15:53:23 wait, I don't understand something: would these errors go away if we would be using tripleo-quickstart? why? we have production repos and these are packaging problems. 15:53:32 so package is python-tripleoclient ? 15:53:40 yep 15:53:58 adarazs: ya the issue would go away, because the quickstart does not rely on tripleoclient for repo setup 15:54:00 trown, ideally, maintainers in rdoinfo==fedpkg==centos sig(no pkgdb yet) 15:54:02 because it is not so good :p 15:54:51 It's somewhere on my pile, automating the builds in CBS 15:54:53 apevec: ya I think for tripleo there is work to be done to unify those things, and actually have active participants in RDO listed 15:55:35 trown: but if it would set up the same repos it would have the same errors. I don't think it uses a different way to build images... it might be using delorean repos instead of production, but the packaging errors are still there in production, and that's our problem. 15:55:51 ah wait tripleoclient is not in fedora? 15:56:57 adarazs: nope, it is possible to pre-build images up with repos before running the tripleoclient build on them, and not using any of the tripleocient repo setup elements. I can talk detail of that after the meeting. 15:57:35 adarazs: it is true that building images following the docs with python-tripleoclient is broken... not arguing that 15:57:46 ie it is a valid bug 15:57:52 #agreed discuss 3o image build after the meeting 15:58:01 trown, so https://github.com/openstack-packages/python-tripleoclient/blob/rdo-liberty/0001-Remove-hardcoded_RDO_RELEASE.patch is not upstream ? 15:58:20 trown: okay, thanks :) 15:58:27 and issues is that trunk build is missing it? 15:58:40 but ok, as agreed after the meeting! 15:58:53 that was the last topic 15:59:00 #topic open floor 15:59:04 one minute left! 15:59:16 some news from trove 15:59:28 mysql driver? 15:59:31 they postponed the python3/pymysql patch after mitaka release ... 15:59:41 eh 16:00:16 jpena, does that block all of pymysql migration or we can do it for other projects? 16:00:46 it's done for every other project. The only thing we can't do is to remove the mysql-python dependency from oslo-db 16:00:49 I mean, upstream liberty was using pymysql, how did they get away? 16:01:05 jpena, ok, that's not to bad 16:01:17 let's push that to rdo liberty then! 16:01:30 we're over time, 16:01:33 thanks everyone! 16:01:36 #endmeeting