15:00:45 <imcsk8> #startmeeting RDO meeting (2016-06-22)
15:00:45 <zodbot> Meeting started Wed Jun 22 15:00:45 2016 UTC.  The chair is imcsk8. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:45 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:45 <zodbot> The meeting name has been set to 'rdo_meeting_(2016-06-22)'
15:00:56 <apevec> drop of the chair?
15:01:04 <imcsk8> of the connection
15:01:08 <apevec> jk
15:01:09 <chandankumar> https://etherpad.openstack.org/p/RDO-Meeting
15:01:20 <apevec> imcsk8, #chair us all
15:01:20 <Duck> quack
15:01:47 <trown> o/
15:01:48 <number80> o/
15:01:52 <jpena> o/
15:01:52 <amoralej> o/
15:01:57 <chandankumar> \o/
15:02:00 <rbowen> /0\
15:02:05 <apevec> /o/
15:02:20 <dmsimard> mewow
15:02:31 * dmsimard is bad at meowing
15:02:47 <apevec> woof!
15:03:26 <apevec> imcsk8, you need to chair us
15:03:43 <imcsk8> #chair apevec Duck number80 dmsimard
15:03:43 <zodbot> Current chairs: Duck apevec dmsimard imcsk8 number80
15:03:48 <Duck> yes, it's tiring to stand up all along
15:03:55 <misc> o/
15:03:59 <imcsk8> sorry, i lost connections
15:04:01 <apevec> #topic rollcall
15:04:06 <apevec> we did that
15:04:11 <rbowen> Still here
15:04:17 <apevec> #topic graylist review.rdoproject.org
15:04:20 <imcsk8> #chair amoralej jpena rbowen
15:04:20 <zodbot> Current chairs: Duck amoralej apevec dmsimard imcsk8 jpena number80 rbowen
15:04:37 <apevec> graylist should be down to 5min after reverse DNS was fixed
15:04:39 <number80> #chair trown chandankumar
15:04:39 <zodbot> Current chairs: Duck amoralej apevec chandankumar dmsimard imcsk8 jpena number80 rbowen trown
15:04:50 <Duck> #chair misc
15:04:50 <zodbot> Current chairs: Duck amoralej apevec chandankumar dmsimard imcsk8 jpena misc number80 rbowen trown
15:05:15 <chandankumar> apevec, what is graylist?
15:05:16 <apevec> any SMTP experts around who can help us solve remaining 5?
15:05:27 <apevec> temp block by mail server
15:05:32 <misc> apevec: I do not think we can do much
15:05:40 <misc> except on the remote server side
15:05:43 <apevec> from review.rdo -> redhat.com it was hours
15:06:01 <misc> and greylist should be applied on the first connexion
15:06:02 <Duck> but that's rh.c's side
15:06:12 <misc> not sure if RH IT has some kind of whitelist
15:06:25 <misc> we can dig in their puppet config to see that, or open a ticket
15:06:28 <apevec> misc, card is https://trello.com/c/vgSUYFjU/162-email-graylist-review-rdoproject-org
15:06:36 <apevec> jschlueter has filed IT ticket
15:06:40 <imcsk8> Duck: rh.c is graylisting right?
15:06:50 <apevec> yes
15:07:13 <imcsk8> mot graylist programs have an exclusion list
15:07:22 <imcsk8> /mot/most/
15:07:34 <apevec> misc, please check comments in the card, and add suggestion how to debug further
15:07:58 <apevec> but I think we can remove  that topic from recurring list
15:08:06 <apevec> let's move on?
15:08:14 <misc> apevec: looking at the ticket, seems IT is on it
15:08:34 <apevec> yeah, jschlueter said they need one example session
15:09:17 <apevec> fbo, jschlueter ^ anything else to add?
15:09:54 <misc> we can chat after the meeting (or I can go ping fbo, even if that involve me moving by 5 m to the other side of the room)
15:10:18 <apevec> #action apevec to followup remaining graylisting of review.rdoproject.org with fbo jschlueter misc
15:10:32 <apevec> misc, hehe
15:10:33 <Duck> misc: do I need to be a previous member of IT to see tickets unrelated to me?
15:10:37 <apevec> ok next
15:10:44 <apevec> #topic DLRN instance migration to ci.centos infra
15:10:53 <jpena> that's mine
15:11:10 <misc> Duck: yep, unfortunately
15:11:18 <jpena> we should have everything in place now, with the current and consistent links being synced
15:11:46 <apevec> what about passed-ci  ?
15:11:50 <apevec> do they match?
15:11:57 <mburned> o/
15:12:09 <jpena> that's synced to buildlogs, and the public server redirects there
15:12:13 <imcsk8> #chair mburned
15:12:13 <zodbot> Current chairs: Duck amoralej apevec chandankumar dmsimard imcsk8 jpena mburned misc number80 rbowen trown
15:12:50 <apevec> ok so when can we do the cut off?
15:13:03 <amoralej> will we manually sync the current-passed-ci for bootstrap the new server?
15:13:04 <jpena> unless EmilienM/trown/dmsimard have any issue, I'd like to flip the switch tomorrow morning (CET time), so we have enough time to troubleshoot during the day if anything goes wrong
15:13:32 <apevec> EmilienM, trown, dmsimard - speak up now!
15:13:33 <trown> +1, hopefully we get a promote on master today
15:13:51 <dmsimard> I think all my concerns have been addressed
15:14:08 <dmsimard> also yes, we need to re-sync the symlinks
15:14:20 <trown> but if not, I dont think the change should disrupt promote
15:14:36 <dmsimard> I can take care of that, I did a small script to check the symlinks on both servers to compare them
15:14:38 <apevec> trown, you're using trunk-primary ?
15:14:53 <dmsimard> apevec: for the hash check only
15:14:58 <dmsimard> apevec: not for the actual installation
15:15:19 <jpena> dmsimard: ok. If you can, re-sync the symlinks later today and e-mail me if it goes ok, so I can change DNS tomorrow
15:15:19 <apevec> I mean for promote script
15:15:40 <trown> trunk-primary is used to get the hash, and it is used for promote
15:15:43 <dmsimard> yes, promote uses trunk-primary
15:15:50 <EmilienM> jpena: go ahead
15:15:50 <apevec> ok, then it's good
15:15:59 <apevec> jpena, action!
15:16:02 <EmilienM> our CI is completly broken anyway today
15:16:12 <EmilienM> but unrelated to rdo
15:16:16 <jpena> #action dmsimard to re-sync the current-passed-ci symlinks
15:16:29 <jpena> #action jpena to switch DNS entries for trunk.rdoproject.org on Thu Jun 23
15:17:02 <apevec> #topic MM3 (mailman3) installation
15:17:26 <apevec> that was on irc earlier, do we have hosting now?
15:17:36 <apevec> IIUC
15:17:39 <Duck> investigating OSAS hosting
15:17:50 <number80> yep, misc mentionned that OSAS has two unnamed servers
15:18:00 <apevec> there was auth question
15:18:01 <Duck> if a VM on our new machines would fit you, think it would
15:18:15 <Duck> more generally, list the needs
15:18:19 <apevec> looks like MM3 doesn't support old style password per list ?
15:18:27 <Duck> have it written in Trello for ex
15:18:39 <apevec> ack, let's start with that
15:18:39 <Duck> apevec: it should, but not yet
15:19:01 <apevec> rbowen, ^ suggestion is to also migration rdo-list to future @lists.rdoproject.org
15:19:04 <Duck> abompard started working on it but is too busy with the upcoming release to continue it right now
15:19:14 <apevec> rbowen, any suggestion for migration strategy?
15:19:16 <misc> when is 3.1 supposed to be released ?
15:19:26 <apevec> auto re-subscribe all current members?
15:19:45 <rbowen> Yes, I would be a big big fan of having our lists on an rdoproject.org hostname.
15:19:45 <Duck> misc: I did not see an exact date, but seems soon
15:19:47 <number80> apevec: yep
15:20:14 <Duck> misc: but we have the RPM from abompard with 3.1 git + patches
15:20:29 <apevec> number80, would you like to start the trello card and everybody chime-in with requirements?
15:20:57 <number80> #action coordinate requirements for m-l migration in trello
15:21:00 <Duck> we can already work on migrating previous mails and see what is the result
15:21:05 <number80> #undo
15:21:05 <zodbot> Removing item from minutes: ACTION by number80 at 15:20:57 : coordinate requirements for m-l migration in trello
15:21:14 <number80> #action number80 coordinate requirements for m-l migration in trello
15:21:24 <Duck> number80: :-)
15:21:30 <apevec> rbowen, what version of mailman do we have now at @redhat.com ?
15:21:36 <dmsimard> Why no hyperkitty :(
15:21:38 <misc> mailman2
15:21:46 <Duck> dmsimard: ???
15:22:03 <misc> likely mailman-2.1.12-25.el6.x86_64
15:22:09 <dmsimard> Duck: fedora's moved to hyperkitty recently, was it considered ?
15:22:22 <Duck> dmsimard: it's a standard component with MM3
15:22:35 <dmsimard> oh so mm3 ~= hyperkitty ?
15:22:35 <Duck> and Postorius for the webi
15:22:36 <misc> and abompard is hyperkitty upstream
15:22:43 <dmsimard> ahhhhhh
15:22:49 <rbowen> It seems we've been talking about it for 2 years, at least.
15:22:59 <Duck> dmsimard: the components are split, but I was talking about the whole package
15:23:17 <misc> rbowen: yeah, mailman3 was supposed to be released at pycon in Montreal, and it took a few weeks of delay
15:23:23 <misc> then fedora did deploy, found bugs
15:23:23 <rbowen> Also worth looking at, Apache PonyMail
15:23:35 <rbowen> Not that I'm biased or anything. :-)
15:23:48 <misc> (then, it did also requires python3, so it did requires backport on EL7)
15:23:54 <Duck> well, if you want to look at the new interface, I can give you info on the oVirt's experiment
15:24:04 <dmsimard> rbowen: http://ponymail.com/ ? :P
15:24:10 <rbowen> No, not that one
15:24:19 <rbowen> ponymail.incubator.apache.org
15:24:22 <trown> lol
15:24:23 <dmsimard> I know, just messing with you
15:24:35 <trown> flashback to the 90s
15:24:36 <misc> " Q: I can Receive email but I can no longer send any email.." seems like a way to reduce the problem of having too much traffic on the list
15:24:48 <dmsimard> sorry for sidetracking topic
15:24:50 <dmsimard> let's move on
15:25:04 <rbowen> Also http://rdo.fosslists.org/list.html?rdo-list@redhat.com
15:26:05 <apevec> rbowen, oh, so we need to migrate all those references?
15:26:11 <rbowen> No.
15:26:23 <rbowen> That last url was a temporary thing set up by the ponymail guy to show me what it could do.
15:26:27 <rbowen> That is not a stable URL
15:26:43 <apevec> ok
15:26:49 <apevec> let's move on, we have action
15:27:01 <apevec> #topic New release for openstack-utils
15:27:08 <number80> o/
15:27:16 <apevec> yes, let's publish it
15:27:19 <number80> I noticed that since pixelb left no new release
15:27:24 <apevec> even if it is just one patch
15:27:36 <number80> apevec: actually, we have some pending reviews
15:28:29 <apevec> reviews?
15:28:29 <jpena> should we add this package under the DLRN umbrella?
15:28:31 <number80> do you take the action or should i take it?
15:28:37 <apevec> I see only one PR  https://github.com/redhat-openstack/openstack-utils/pull/11
15:28:48 <apevec> we didn't migrate it to review.rdo did we?
15:29:01 <apevec> jpena, I'm not sure
15:29:07 <apevec> it's not openstack project
15:29:13 <apevec> and not much changing
15:29:17 <number80> apevec: nope (well, I reviewed Nir's to add neutron-**aaS services)
15:29:46 <number80> https://github.com/redhat-openstack/openstack-utils/commit/11c3e85609f168a64410354afce1c143fc8661a9
15:30:01 <apevec> https://github.com/redhat-openstack/openstack-utils/issues/13 should be fixed?
15:30:48 <apevec> number80, ok, let's do triage of remaining open issues then release it
15:31:05 <number80> Yes
15:31:09 <apevec> #action apevec and number80 do triage of open issue and release openstack-utils 2016.1
15:31:32 <apevec> next
15:31:35 <apevec> #topic Proposal to manage pinned packages
15:31:46 <number80> (sorry, I'm also dealing w/ CBS tag topic on #centos-devel at the same time)
15:32:24 <apevec> so yes, current approach was to fork problematic project (currently osc)
15:32:31 <apevec> and pin to latest release tag
15:32:56 <dmsimard> I have a problem with that
15:33:02 <apevec> but to build it, we had to delete successfully built commits from dlrn db
15:33:06 <number80> I agree that's error-prone
15:33:15 <dmsimard> We have no visibility on the problems between <release tag> and <latest commit>
15:33:18 <apevec> jpena, please summarize your proposal
15:33:25 <dmsimard> In that sense, it's as bad as having no consistent for a long period of time
15:33:34 <jpena> my idea was that, in these cases, we:
15:33:49 <jpena> a) temporarily disable the pinned project in rdoinfo (setting a tag)
15:33:51 <apevec> dmsimard, yes, but that's projects which don't seem to care about their tip of master branch
15:34:08 <dmsimard> apevec: I don't think these projects don't care
15:34:10 <jpena> b) have the pinned rpm in an overrides repo, included in delorean-deps.repo
15:34:10 <apevec> dmsimard, so time to fix is unkown
15:34:15 <apevec> unknown even
15:34:28 <dmsimard> apevec: for that particular issue, dtroyer was relatively quick in understanding the issue and pushing a fix for review
15:34:43 <dmsimard> apevec: If that use case had testing coverage, it probably wouldn't have merged in the first place
15:34:52 <apevec> yep
15:35:10 <apevec> ideally we help upstream fix it and improve coverage
15:35:20 <trown> the osc issues plagued us for a week... pretty hard to have a weekly promote if a single issue takes a week to resolve
15:35:35 <apevec> but if that takes more than few days, we need planB solution
15:35:40 <trown> +1
15:35:45 <dmsimard> trown: so pinning a package is the same as promoting an inconsistent repo, then ?
15:35:47 <trown> more than 2 days is dire
15:36:02 <apevec> dmsimard, not quite
15:36:12 <apevec> we pin to release tag, which is what upstream gate is actually using
15:36:24 <trown> because it is rare we go more than 2 days without something breaking, so the cycle can repeat
15:36:24 <dmsimard> I would think upstream gate uses trunk, no ?
15:36:25 <apevec> this would be restricted to dep libs only
15:36:42 <trown> dmsimard: nope, not for libs
15:36:43 <apevec> dmsimard, not for libs
15:36:43 <dmsimard> Why would upstream gate devstack use released projects ?
15:36:47 <dmsimard> Oh.
15:36:48 <number80> well, if we keep building every commit in pinned package to track progress (at least display the build state in reporting), I'm fine
15:36:56 <apevec> number80, we don't
15:37:03 <apevec> we pin it to a tag
15:37:14 <apevec> it's exceptional state
15:37:28 <amoralej> with jpena's proposed solution we don't build any new commit
15:37:31 <apevec> and we need to remove it as soon as fix lands on master
15:37:33 <number80> I mean w/ jpena b. proposal, it'd be possible
15:37:50 <trown> I actually think we might be better served to follow upstream here and put libs into deps repo
15:38:13 <apevec> number80, it's both
15:38:15 <number80> trown: master release for libs are too often broken
15:38:18 <number80> ack
15:38:19 <apevec> step a and step b
15:38:34 <number80> suse and mirantis had issue recently w/ oslo.cache changes
15:38:50 <trown> number80: right but at least if a release is broken we have a leg to stand on when bringing the issue upstream
15:39:00 <trown> number80: if master is broken... there is not much urgency
15:39:35 <apevec> ok, let's then make a list,
15:39:51 <apevec> of projects in rdoinfo which are used from pypi in the gate
15:39:51 <trown> putting them in deps repo does then require a deps promotion pipeline, but we want that anyways
15:39:54 <dmsimard> so how does upstream test when they bump a lib release ?
15:39:57 <apevec> that's oslo and clients
15:40:14 <dmsimard> brb kid school (last day at school!)
15:40:19 <trown> ever?
15:40:22 <apevec> dmsimard, it's whatever that projects' have in their gate
15:40:58 <rbowen> Wow. Our kids were out almost a month ago.
15:42:21 <apevec> so conclusion?
15:42:43 <apevec> move all of oslo and clients out of rdoinfo ?
15:42:55 <apevec> but then, how do we keep track of master changes?
15:42:57 <trown> it might be a bit complicated issue to resolve in meeting... maybe we should push to ML?
15:43:03 <apevec> yeah
15:43:06 <jpena> +1 to trown's proposal
15:43:18 <number80> I
15:43:28 <apevec> jpena, ok, since this was your topic, please start thread on rdo-list :)
15:43:28 <number80> 'm -1 to remove oslo libs from DLRN
15:43:37 <jpena> apevec: ack
15:43:49 <jpena> #action jpena to start thread on rdo-list about pinned packages
15:43:53 <apevec> yeah, I'm -1 too for now, but let's discuss details on hte list
15:43:58 <number80> sure
15:44:19 <apevec> next
15:44:21 <apevec> #topic Add openstack-macros in CBS cloud SIG buildroot
15:44:30 <apevec> +2 !
15:44:42 <apevec> and let's migrate all of rdo-rpm-macros to that
15:44:57 <number80> #action number80 migrate rdo-rpm-macros to openstack-macros
15:45:14 <apevec> number80, ^ this build is pure current upstream?
15:45:21 <apevec> openstack-macros-2015.2-0
15:45:41 <number80> apevec: yes, it's to allow jpena to move forward w/ DLRN/rpm-packaging support
15:45:47 <apevec> ack
15:46:07 <apevec> jpena, ^ anything else blocking you on that support?
15:46:08 <number80> I'll handle the transition as we can't have the same macros defined twice in buildroot
15:46:34 <jpena> apevec: we need that, merge https://review.rdoproject.org/r/1346 and then we can setup a test instance
15:46:54 <number80> this one need second opinion but it works and does not break existing feature
15:47:50 <apevec> #action apevec to review dlrn rpm-packaging support https://review.rdoproject.org/r/1346
15:47:53 <apevec> ok, next
15:48:03 <apevec> #topic How to raise an alert when RDO Trunk repos are broken
15:48:13 <apevec> sensu?
15:48:23 <rbowen> We need another bot!
15:48:25 <jpena> this came up today, since the keystonemiddleware thing broke current-passed-ci and current-tripleo
15:48:28 <apevec> but how do you define "broken" ?
15:48:38 <trown> apevec: is this current-passed-ci broken? or current?
15:48:54 <trown> ah, thanks jpena
15:48:55 <jpena> at a minimum: broken symlinks, anything that prevents a package from being read
15:49:07 <apevec> so repoclosure?
15:49:22 <apevec> we need to define a probe
15:49:38 <jpena> apevec: that would be ok. If we can run a probe there, and email rdo-list when it fails, it would be good
15:49:39 <apevec> light enough i.e. not full CI pipeline :)
15:49:52 <jpena> the idea is not to only alert us, but also rdo trunk consumers
15:49:54 <apevec> I'd like this to be part of overall monitoring
15:50:00 <apevec> which is based on sensu
15:50:07 <apevec> not just one-off script somewhere
15:50:27 <jpena> that makes sense, yes
15:50:32 * trown is tempted to action dmsimard in his absence
15:50:38 * jpena too
15:50:42 <apevec> dmsimard, ^ when you're back, suggestions welcome how to put this into sensu
15:50:49 <apevec> trown, jpena ack :)
15:51:00 <number80> remember that he'll be in PTO next week :)
15:51:31 <apevec> #action dmsimard to suggest lightweight sensu probe for basic rdo repo consistency check
15:51:47 <apevec> number80, there's still enough hours until next week :)
15:51:48 <trown> he can always re-delegate :)
15:51:56 <number80> ack
15:52:12 <apevec> #topic Test Day
15:52:15 <apevec> rbowen, ^
15:52:32 <rbowen> So, first thing was stats from the last test day, just FYI
15:52:48 <rbowen> We have traditionally not even done a milestone 1 test day, so it's not really surprising that numbers were down.
15:53:00 <rbowen> But we need to try to spread the word a little wider for M2, if possible.
15:53:14 <apevec> less tickets might mean openstack is maturing :)
15:53:16 <rbowen> Like, I kind of missed notifying various places like CentOS and Fedora this time.
15:53:36 <rbowen> So, just wanted to get the date out there early so that we can get some wider participation.
15:53:45 <rbowen> And, as always, improve test case documentation
15:53:47 <rbowen> /EOL
15:54:06 <apevec> ack for July 21/22 dates
15:54:12 <number80> +1
15:54:15 <apevec> let's keep upstream milestone + 1 week
15:54:25 <rbowen> ok, I'll start getting the word out on that date.
15:54:30 <apevec> that's month from now
15:54:35 <apevec> rbowen, action it :)
15:54:40 <rbowen> I'll be at RHSummit all next week, so will be somewhat unresponsive for the next little bit.
15:54:53 <rbowen> #action rbowen to promote Newton Milestone 2 test day, July 21/22
15:55:04 <apevec> #topic Chair for next meeting
15:55:14 <imcsk8> i'll do it
15:55:18 * chandankumar would like to chair
15:55:28 <imcsk8> i feel bad for sepping dong during this meeting
15:55:32 <apevec> chandankumar, you'll get week after :)
15:55:40 <imcsk8> s/dong/down/
15:55:45 <apevec> imcsk8, no worries
15:55:56 <apevec> just fix those tubes for the next week ;)
15:56:08 <imcsk8> yeah!
15:56:17 <apevec> #info imcsk8 is chair June 29
15:56:34 <apevec> #info chandankumar is chair July 6
15:56:37 <apevec> #topic Open Floor
15:56:52 <apevec> anything else, for few minutes left?
15:56:59 <imcsk8> just one thing
15:57:04 <imcsk8> the packstack refactor
15:57:13 <apevec> merge it! :)
15:57:24 <imcsk8> there's concerns about not having storage host
15:57:35 <apevec> that can be followup
15:57:43 <apevec> we will not release this
15:57:50 <apevec> folks using it will stay on released version
15:58:01 <imcsk8> ok
15:58:12 <imcsk8> ah and other thing hehee
15:58:23 <apevec> jpena, ^ you're fine w/ followup ?
15:58:25 <jpena> and remember, separate storage host support was deprecated long time ago, so they can expect it to break ;)
15:58:47 <jpena> apevec: sure, it should be a followup
15:58:53 <imcsk8> +1
16:00:12 <apevec> imcsk8, what's the other thing?
16:00:24 <imcsk8> we can address it after ending the meeting
16:00:28 <apevec> ok
16:00:32 <apevec> #endmeeting