15:00:46 <apevec> #startmeeting RDO meeting (2016-02-10)
15:00:46 <zodbot> Meeting started Wed Feb 10 15:00:46 2016 UTC.  The chair is apevec. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:46 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:46 <zodbot> The meeting name has been set to 'rdo_meeting_(2016-02-10)'
15:00:49 <trown> ya, we probably need to advertise that question to have any chance of real feedback
15:00:59 <apevec> #topic roll call
15:01:02 <dmsimard> o/
15:01:05 <trown> present
15:01:07 <jpena> o/
15:01:07 <chandankumar> o/
15:01:17 <imcsk8> o/
15:01:18 <apevec> #info agenda https://etherpad.openstack.org/p/RDO-Meeting (scroll to NEXT=> )
15:02:00 <apevec> ok, let's give few min
15:02:49 <apevec> number80, jruzicka - yt?
15:02:52 <eggmaster> o/
15:04:07 <apevec> ok, first topic:
15:04:14 <apevec> #topic Trello bot for #rdo
15:04:53 <apevec> who proposed it?
15:05:00 <apevec> there's one example, on #rdo-puppet
15:05:15 <apevec> I don't like its noise level
15:05:17 <jpena> I added it, because it was discussed last week and someone said we should look at how it works in #rdo-puppet then decide
15:05:51 <jpena> EmilienM said it was probably configurable to reduce the amount of noise
15:05:53 <apevec> so if we add it, I do limit bot notifications
15:05:58 <dmsimard> I'm not particularly a fan of the noise it generates but if I may extend the topic
15:06:10 <EmilienM> jpena: it's not but we can improve it
15:06:22 <apevec> yes, we also have the other bot for delorean builds
15:06:25 <dmsimard> I really want a bot to spam me with reviews that land in openstack-packages/*, jenkins build jobs and eventually monitoring status from the monitoring PoC
15:06:35 <apevec> we might want to unify all in one notification bot
15:07:02 <apevec> dmsimard, is there something like that in monitoring tools you're looking at?
15:07:14 <trown> +1 to reviews on openstack-packages getting notified here
15:07:24 <apevec> EmilienM, where does puppy code live?
15:07:41 <EmilienM> https://github.com/fredericlepied/zircbot
15:07:41 <apevec> is it trello-specific?
15:07:44 <dmsimard> apevec: I've built a bot at my previous job using http://errbot.io/, it's dead simple, mature and it's a googler project
15:07:47 <EmilienM> and https://github.com/fredericlepied/ztrellohook
15:07:56 <jschlueter> o/
15:07:58 <dmsimard> apevec: you can extend it with webhooks and everything
15:08:04 <EmilienM> flepied1 is the author
15:08:28 <apevec> dmsimard, do you want to compare those two bots?
15:08:40 <dmsimard> I'm very biased towards what I've already worked with
15:08:47 <trown> which one is better at chess
15:08:54 <dmsimard> EmilienM seemed to like errbot :p
15:09:28 <EmilienM> I have no preference, our use case is much simpler that yours
15:09:28 <apevec> let's take errbot then, but extend it to include both trello(highly filtered!) and derekh's version.csv checker ?
15:09:41 <trown> +1
15:09:43 <dmsimard> derekh's version.csv checker is already covered in monitoring
15:09:54 <apevec> derekh, ^ where is code for your versions.csv check job?
15:10:05 <dmsimard> apevec:
15:10:27 <dmsimard> apevec: he sent it to me, it's a bash one-liner that greps for FAILED
15:10:35 <apevec> nice :)
15:11:01 <dmsimard> I made something nagios-friendly in python
15:11:04 <derekh> apevec: dmsimard should I turn mine off ?
15:11:04 <apevec> dmsimard, so you can create irc notifications from uchiwa?
15:11:05 <dmsimard> so we can eventually graph stats
15:11:23 <dmsimard> apevec: irc is one of the "handlers" from sensu, yes
15:11:38 <dmsimard> derekh: not yet, this is not in production still
15:11:46 <jruzicka> o/
15:12:03 <derekh> dmsimard: cool, just shout whenever and I'll turn mine off
15:12:06 <apevec> ok, let's write down summary
15:12:21 <apevec> #agree #rdo to have single notifications bot
15:12:34 <apevec> #agreed #rdo to have single notifications bot
15:13:04 <trown> dmsimard: are you taking the action to set that up?
15:13:06 <apevec> #action dmsimard to put errobt in production
15:13:12 <trown> ah guess so :p
15:13:16 <dmsimard> I volounteered apparently
15:13:18 <dmsimard> :P
15:13:21 <apevec> it's part of monitoring :)
15:13:55 <dmsimard> so far I've been working on monitoring between troubleshooting trunk and with the amount of breakage that has been happening I haven't exactly had a whole lot of time to spend on it
15:14:19 <dmsimard> I'll try to at least wrap up the monitoring portion this week so I can move on to bot next week.
15:14:40 <apevec> sounds good
15:14:56 <chandankumar> dmsimard, i can offer some help on errbot if you need.
15:14:59 <apevec> maybe we can have trunk status as a quick topic now?
15:15:06 <dmsimard> apevec: sure
15:15:13 <apevec> #topic trunk status
15:15:36 <apevec> consistent is there
15:16:04 <apevec> today 15:14 UTC
15:16:08 <apevec> but CI ?
15:16:14 <dmsimard> There was a lot of backwards incompatible changes that broke packstack, tripleo, puppet-openstack and others.
15:16:21 <dmsimard> #link https://review.openstack.org/#/q/topic:bug/1539793
15:16:29 <dmsimard> #link https://review.openstack.org/#/q/topic:bug/1542486
15:16:40 <trown> yep we are good from a packaging POV, but https://etherpad.openstack.org/p/delorean_master_current_issues has some issues around nova
15:17:13 <dmsimard> And more recently changes in both heat and tempest were identified and submitted for fixes in https://review.openstack.org/#/c/278081/ and https://review.openstack.org/#/c/278068/
15:17:43 <trown> I like having a weekly (hopefully quick) topic of trunk status
15:17:44 <dmsimard> puppet-openstack is fixed by now but we are still working on submitting/merging the necessary fixes to tripleo and packstack.
15:17:49 <dmsimard> trown: +1
15:18:02 <dmsimard> I'm done with trunk status
15:18:16 <dmsimard> and I think I wasn't chair so that probably didn't register
15:18:16 <apevec> thanks
15:18:25 <apevec> oops
15:18:33 <apevec> #chair dmsimard trown jpena
15:18:33 <zodbot> Current chairs: apevec dmsimard jpena trown
15:18:34 <trown> #link etherpad for helping with trunk troubleshooting https://etherpad.openstack.org/p/delorean_master_current_issues
15:19:00 <dmsimard> #link https://review.openstack.org/#/q/topic:bug/1539793
15:19:04 <dmsimard> #link https://review.openstack.org/#/q/topic:bug/1542486
15:19:08 <apevec> so nova api db is merged in packstack
15:19:21 <apevec> t-h-t is open
15:19:25 <dmsimard> #info heat tempest tests are fixed by https://review.openstack.org/#/c/278081/ and https://review.openstack.org/#/c/278068/
15:19:37 <imcsk8> dmsimard: whatever you need for packstack let me know
15:20:01 <dmsimard> apevec: I have my code editor opened for one more patch to t-h-t for the nova-neutron with v3 keystone issue
15:20:04 <pabelanger> apevec: regarding rhbz #1301384. When do you think you'll push the build to rawhide for python-keystoneauth1?  Looks like you only committed the changes to git
15:20:25 <number80> o/
15:20:31 <apevec> imcsk8, nova-neutron change https://review.openstack.org/#/c/277867/
15:20:35 <jruzicka> \o
15:20:39 <pabelanger> apevec: Actually, I am just re-reading the bug report now
15:20:40 * number80 was distracted by HRT crap
15:20:45 <pabelanger> missed the depends on issues
15:21:09 <apevec> pabelanger, OT - let's sync after meeting
15:21:12 <apevec> number80, HRT ?
15:21:17 <apevec> #chair number80
15:21:17 <zodbot> Current chairs: apevec dmsimard jpena number80 trown
15:21:28 <imcsk8> apevec: will test it
15:21:41 <number80> apevec: Talent people, whatever the acronym is now
15:22:21 <apevec> TA - but OT :)
15:22:49 <apevec> dmsimard, trown - ok, we are >< close to CI pass?
15:23:39 <trown> apevec: I am always pretty pessimistic about that, but I think we have a good handle on the currently known issues
15:24:06 <trown> my pessimism revolves around new issues coming before we get everything merged before the current known ones
15:24:16 <apevec> yeah
15:24:19 <trown> which is rooted in experience
15:24:53 <apevec> we have non-milestone testday planned for the next week, hopefully no new bug issues will come up
15:24:54 <jschlueter> apevec: o/
15:25:05 <pabelanger> apevec: apologies, didn't know a meeting was going on
15:25:10 <apevec> jschlueter, \o
15:25:17 <apevec> #chair jschlueter
15:25:17 <zodbot> Current chairs: apevec dmsimard jpena jschlueter number80 trown
15:25:17 <dmsimard> apevec: yeah, I mean, we're fairly close
15:25:19 <apevec> pabelanger, np
15:25:20 <trown> oh... I did not realize there was another testday next week
15:25:27 <dmsimard> *IF* no other breaking changes occur
15:25:33 <trown> seems so close to the previous one
15:25:39 <number80> ack
15:25:42 <dmsimard> Right now breaking changes are occuring faster than we can fix them so we're already behind
15:25:50 <apevec> https://www.rdoproject.org/testday/
15:26:01 <apevec> trown, idea was to keep it monthly
15:26:11 <apevec> m3 is only in March
15:26:23 <trown> makes sense
15:26:44 <trown> would be really nice to have a fully consistent current-passed-ci repo for that
15:27:02 <apevec> yes!
15:27:33 <trown> if it looks like tripleo patches will be the only hold up and they are in good shape upstream (at least one +2), I can put fixes in the images again
15:27:43 <apevec> ack
15:27:57 <apevec> ok, let's move on
15:28:08 <apevec> #topic Changes to Delorean
15:28:41 <apevec> we've few pending which would be good to merge before pushing first RPM
15:29:05 <apevec> one particular is https://review.gerrithub.io/254505 Use fedora guidlines for pre-release packages
15:29:21 <apevec> we went back and forth there
15:29:36 <apevec> it's now following fedora pre-release versioning guidelines
15:30:03 <apevec> it also wins over current versioning:
15:30:11 <apevec> rpmdev-vercmp 1-dev9999999999 1-0.120160210.shorthash
15:30:17 <apevec> 1-dev9999999999 < 1-0.120160210.shorthash
15:30:17 <trown> yep I just wanted to give one last chance for dissent on that one since it has morphed alot
15:30:31 <apevec> so new builds with this patch will be able to upgrade current builds
15:30:35 <trown> oh that is handy, I wasnt sure about that part
15:30:51 <jpena> I like that one much more than playing with epoch
15:31:14 <apevec> yeah, epoch was a hack I suggested
15:31:19 <jruzicka> epoch should be avoided if at all possible
15:31:20 <trown> ya I did not mind epoch until thinking about our source rpms
15:31:28 <apevec> it could work but better not mess with it
15:31:39 <jpena> we may have some cases where this doesn't work, though. It looks like gnocchi has version 1.3.4 in liberty, but delorean is building 1.3.1
15:31:39 <number80> *nods*
15:31:51 <apevec> jpena, that's upstream issue
15:32:00 <jpena> yep, that's what I was going to say :)
15:32:02 <apevec> they need to push newer tag on master
15:32:10 <apevec> or something
15:32:10 <trown> jpena: ya for those we have to solve on case by case, and introduce epoch = epoch+1 if upstream is uncooperative
15:32:28 <apevec> ack
15:32:32 <trown> which is still better than the crazy epoch solution
15:32:48 <apevec> so if no -1s let's merge this
15:32:49 <trown> s/crazy/date based/
15:33:02 <number80> the pre-release versioning?
15:33:04 <apevec> last call :)
15:33:30 <number80> just to be sure, because, the best I can do with epoch is abstention :)
15:33:40 <apevec> number80 https://review.gerrithub.io/254505
15:33:49 <number80> +1
15:33:50 <trown> number80: yep the patch as proposed is pre-release versioning
15:33:52 <apevec> number80, no Epoch
15:34:11 <number80> ok, we were speaking both
15:35:08 <apevec> number80, epoch mentioned was for case by case when upstream has wrong versioning on trunk
15:35:21 <apevec> e.g gnocchi
15:35:32 <number80> apevec: the patch submitted was injecting epoch in all projects
15:35:33 <apevec> pradk, ^ please push 2.0 beta/alpha tag
15:35:34 <trown> and would not be a delorean change, but a spec change on rpm-master
15:35:45 <number80> which was ok, if it were injected by delorean
15:35:48 <apevec> number80, please look at the latest patchset
15:37:13 <number80> I don't see the epoch one
15:37:24 <apevec> while at it, there are few more patches ready for reviews https://review.gerrithub.io/#/q/status:open+project:openstack-packages/delorean+branch:master
15:37:31 <apevec> number80, there is none, now
15:37:45 <apevec> we're discussing https://review.gerrithub.io/254505
15:37:50 <apevec> only
15:37:58 <trown> I think we are good on the delorean patch, and could discuss the case by case for inconsistent upstream when/if it next arises
15:38:00 <number80> yes, I'm ok with this one
15:38:02 <apevec> for gnocchi, I'd like to push once more upstream to fix it
15:38:14 <trown> yep +1
15:38:32 <apevec> EmilienM, ^ any connections in gnocchi :) ?
15:38:33 <number80> https://review.gerrithub.io/#/c/262031/
15:38:38 <number80> ^ this one is ok
15:38:51 <number80> pradk: could you check for the gnocchi tag issue?
15:39:03 <EmilienM> pradk is the best guy to help in gnocchi
15:39:32 <apevec> number80, that one is part of jpena's https://review.gerrithub.io/#/q/topic:delorean-packaging series
15:39:41 <pradk> apevec, number80, lets chat with jd after the meeting, I spoke to him already but he's not inclined to do so as he manages the tagging for gnocchi
15:40:07 <number80> ack
15:40:22 <number80> apevec: reviewing the others as well
15:40:39 <apevec> pradk, just ask him to run python ./setup.py --version on gnocchi master....
15:41:16 <apevec> ok, let's finish reviews off-meeting, I don't think others are controversial
15:41:18 <pradk> apevec, yes already had the conversation .. we'll be releasing 2.0 soon which should bump  master to 2.0.0+
15:41:35 <apevec> pradk, but there are also pre-release tags
15:41:52 <pradk> apevec, lets discuss after the mtg
15:42:05 <apevec> ack
15:42:12 <apevec> #agreed merge https://review.gerrithub.io/254505
15:42:44 <apevec> #action pradk et al discuss with jd gnocchi 2.0 pre-release tagging
15:43:24 <apevec> #action everybody review open delorean reviews for 0.1 delorean RPM
15:43:42 <apevec> #topic RDO liberty production CI jobs
15:44:08 <dmsimard> Sorry I got sidetracked, just a quick update - I managed to get the tempest revert merged to resolve ongoing trunk issues with heat
15:44:42 <apevec> dmsimard, nice, one more down...
15:44:54 <apevec> re. liberty CI bz 1304395 is a blocker?
15:45:06 <apevec> adarazs, ^
15:45:34 <adarazs> apevec: looking.
15:45:38 <apevec> trown, ^ https://bugzilla.redhat.com/show_bug.cgi?id=1304395#c3
15:45:56 <adarazs> apevec: yep, it blocks the jobs on centos ci, probably it would block anybody trying to install and build images for rdo-manager.
15:46:17 <adarazs> apevec: for example this https://ci.centos.org/view/rdo/job/rdo_manager-periodic-7-rdo-liberty-production-centos-7.0-templates-virthost-minimal-neutron-ml2-vxlan-smoke/
15:47:07 <apevec> how are images built in those jobs?
15:47:21 <apevec> trown, is quickstart working for liberty?
15:47:39 <apevec> 3o-quickstart
15:47:53 <trown> apevec: yep, and I use tripleoclient from liberty release for building all images... even trunk
15:48:13 <apevec> trown, so what's wrong in that bz?
15:48:19 <trown> looking
15:48:50 <apevec> also what are those jobs, are they not all moving to use 3o-quickstart?
15:48:51 <adarazs> apevec: btw this bug blocks the image building too, but at least for this I could add an easy workaround: https://bugzilla.redhat.com/show_bug.cgi?id=1278972
15:49:39 <apevec> adarazs, trown - can you sync up about those jobs, I though weshay said all jobs will move to use 3o-quickstart images
15:49:42 <adarazs> apevec: eventually they will, but we don't have the bindings in khaleesi for them yet and this is what we can currently do to test production repos on centos.
15:49:45 <apevec> or did I misunderstand?
15:49:58 <trown> ah I see https://github.com/redhat-openstack/tripleo-quickstart/blob/master/playbooks/roles/images/build/templates/build-images#L12
15:50:42 <trown> apevec: there are still a bunch of khaleesi based jobs running on centosci, and the production repo jobs are in that category
15:51:02 <apevec> adarazs, I don't know is it worth to debug old jobs, can't we just move "eventually" forward? :)
15:51:37 <trown> apevec: I am working with khaleesi folks to integrate the quickstart with there jobs so that the quickstart does not have to maintain all possible deployment scenarios and can just do the one thing well of setting up the environment
15:52:21 <jruzicka> nice
15:52:30 <trown> in any case, the topic for this meeting I think is more around who is responsible for CBS packages? maintainer in rdoinfo? maintainer in fedoradb?
15:53:06 <trown> I am neither of these things for a lot of the tripleo packages and would rather not be until we have a simpler workflow to allow managing so many packages
15:53:23 <adarazs> wait, I don't understand something: would these errors go away if we would be using tripleo-quickstart? why? we have production repos and these are packaging problems.
15:53:32 <apevec> so package is python-tripleoclient ?
15:53:40 <trown> yep
15:53:58 <trown> adarazs: ya the issue would go away, because the quickstart does not rely on tripleoclient for repo setup
15:54:00 <apevec> trown, ideally, maintainers in rdoinfo==fedpkg==centos sig(no pkgdb yet)
15:54:02 <trown> because it is not so good :p
15:54:51 <number80> It's somewhere on my pile, automating the builds in CBS
15:54:53 <trown> apevec: ya I think for tripleo there is work to be done to unify those things, and actually have active participants in RDO listed
15:55:35 <adarazs> trown: but if it would set up the same repos it would have the same errors. I don't think it uses a different way to build images... it might be using delorean repos instead of production, but the packaging errors are still there in production, and that's our problem.
15:55:51 <apevec> ah wait tripleoclient is not in fedora?
15:56:57 <trown> adarazs: nope, it is possible to pre-build images up with repos before running the tripleoclient build on them, and not using any of the tripleocient repo setup elements. I can talk detail of that after the meeting.
15:57:35 <trown> adarazs: it is true that building images following the docs with python-tripleoclient is broken... not arguing that
15:57:46 <trown> ie it is a valid bug
15:57:52 <apevec> #agreed discuss 3o image build after the meeting
15:58:01 <apevec> trown, so https://github.com/openstack-packages/python-tripleoclient/blob/rdo-liberty/0001-Remove-hardcoded_RDO_RELEASE.patch is not upstream ?
15:58:20 <adarazs> trown: okay, thanks :)
15:58:27 <apevec> and issues is that trunk build is missing it?
15:58:40 <apevec> but ok, as agreed after the meeting!
15:58:53 <apevec> that was the last topic
15:59:00 <apevec> #topic open floor
15:59:04 <apevec> one minute left!
15:59:16 <number80> some news from trove
15:59:28 <apevec> mysql driver?
15:59:31 <number80> they postponed the python3/pymysql patch after mitaka release ...
15:59:41 <apevec> eh
16:00:16 <apevec> jpena, does that block all of pymysql migration or we can do it for other projects?
16:00:46 <jpena> it's done for every other project. The only thing we can't do is to remove the mysql-python dependency from oslo-db
16:00:49 <apevec> I mean, upstream liberty was using pymysql, how did they get away?
16:01:05 <apevec> jpena, ok, that's not to bad
16:01:17 <apevec> let's push that to rdo liberty then!
16:01:30 <apevec> we're over time,
16:01:33 <apevec> thanks everyone!
16:01:36 <apevec> #endmeeting