21:00:01 <nirik> #startmeeting
21:00:08 <nirik> #meetingtopic EPEL
21:00:15 <nirik> who all is here for the epel meeting?
21:02:58 * dgilmore 
21:03:02 * stahnma is
21:03:03 * giesen 
21:03:10 <nirik> hurray. It's not just me. ;)
21:03:12 * vpv 
21:03:49 <giesen> how many people normally show up for these? (this is my first)
21:04:02 <stahnma> 6-7 often times
21:04:03 * muep___ 
21:04:12 <stahnma> we enjoy more :)
21:04:27 <nirik> more input is great! :)
21:05:04 <nirik> I'm dealing with a bunch of day job items, so I might go slowly here... ;)
21:05:09 <nirik> #topic Koji/Bodhi - All done?
21:05:10 <muep___> my first attendance as well... though I use centos only little
21:05:18 <nirik> dgilmore: how are we looking?
21:05:25 <dgilmore> nirik: ok.
21:05:35 <dgilmore> theres some bugs that need fixing
21:05:43 <lmacken> pretty much done
21:05:52 <lmacken> in terms of deployment/hacking...
21:05:58 <dgilmore> the push process need some tuning
21:06:03 * nirik cheers. Thanks for working on that. Very appreciated.
21:06:05 <lmacken> policy-wise... we need to make some decisions and hack them into place
21:06:12 <lmacken> eg: mandatory testing?
21:06:19 <dgilmore> lmacken: right
21:06:21 <nirik> yeah, I had that on the agenda.
21:06:21 <lmacken> autokarma?
21:06:22 <lmacken> etc
21:06:39 <nirik> so, I guess moving on:
21:06:44 <nirik> #topic Pushing Packages to stable too soon.
21:07:00 <nirik> this ties in with the next topic I had, which was: Pushing schedules (stable/testing).
21:07:21 <dgilmore> people keep trying to send things straight to stable
21:07:43 <stahnma> I think if we reestablish the pushing policy/proess and communicate it, we'll be ok
21:07:54 <nirik> yeah, so what should that policy be?
21:07:58 <stahnma> in the recent past, pushes were very non-exact, so people probably got impatient
21:08:01 <stahnma> and just asked for stable
21:08:41 <dgilmore> stahnma: one person in particular continues trying to push to stable even after i asked via email and in bodhi to keep the package in testing
21:09:00 <nirik> I think we should require time in testing for all non security updates.
21:09:07 <stahnma> is there any way to prevent a stable request if it's not a security update?
21:09:09 <dgilmore> i agree
21:09:14 <dgilmore> i think 2 weeks
21:09:21 <stahnma> i mean technically, like disable the request button or something
21:09:37 <nirik> lmacken: ? can we do that for epel?
21:09:39 <lmacken> "time" or "karma"?  if something hits testing and 3 people test it and +1 it, should it really sit around?
21:09:41 <dgilmore> maybe even that we push to stable every second tuesday
21:09:52 <lmacken> it can be done..
21:10:03 <nirik> ie, epel updates have 'testing' and 'security'
21:10:14 <dgilmore> and unless its security related it needs to have some stint in testing
21:10:17 <stahnma> testing trumps time, but time is good for packages that don't get used all that often
21:10:36 <nirik> stahnma: agreed.
21:10:42 <dgilmore> stahnma: right autokarma can trigger erlier pushes
21:11:04 <nirik> I'm ok with karma as long as people are testing/confirming the update works for them.
21:11:05 <muep___> makes sense
21:11:36 <lmacken> a simple solution for the time-based testing->stable moves would be to hack the bodhi client to say "show me all testing packages that have been sitting for 2 weeks"...
21:11:41 <lmacken> that wouldn't require any bodhi-server changes too
21:11:49 <stahnma> that's a pl8us
21:11:57 <nirik> I also like the non security stable pushes only being every 2 weeks on tuesday.
21:12:04 <dgilmore> lmacken: well at least client side
21:12:14 <stahnma> can you do that and everything that has +3 karam
21:12:18 <nirik> yeah, that sounds nice... and a way to promote a list to stable.
21:12:33 <stahnma> like select updates where karma>3 or request_date>2weeks
21:12:36 <lmacken> but yeah, if we only push every 2 weeks, then that takes care of the time-in-testing for us
21:13:04 <ssmoogen> sorry I am a bit late
21:13:19 <dgilmore> lmacken: im going to look to adding to bodhi-client so that it doesnt try to push to stable all teh time
21:13:34 <lmacken> dgilmore: ok
21:13:35 <dgilmore> lmacken: i think we can do most of what we need client side
21:13:38 <lmacken> yeah
21:13:40 <dgilmore> with no server changes
21:13:50 <lmacken> good :) that makes me much happier
21:13:58 <nirik> does everyone like that general policy?
21:14:01 <dgilmore> lmacken: :)
21:14:02 * lmacken pushed out 28 bodhi versions this week to get EPEL working
21:14:14 <stahnma> wow
21:14:16 <giesen> ouch
21:14:25 <dgilmore> lmacken: its much appreciated
21:14:34 <stahnma> is karama able to trump time?
21:14:34 <lmacken> fix bug -> mock -> scp -> rpmsign -> createrepo -> yum update -> puppetd -> apachectl -> test
21:14:37 <lmacken> repeat 28 times :)
21:14:43 <stahnma> or is it strickly time based?
21:14:59 <giesen> even with carm, should there still not be a minimum period?
21:14:59 <dgilmore> stahnma: karma can trump time
21:15:01 * ssmoogen wonders if we will do deltarpms with EPEL-6
21:15:03 <lmacken> stahnma: autokarma can currently trump time
21:15:04 <giesen> *karma
21:15:12 <dgilmore> stahnma: but stable will be every two weeks on a tuesday
21:15:15 <nirik> Proposed: stable pushes will be every 2 weeks on tuesday. Testing pushes will be every weekday or so. All updates must spend time in testing unless security related. 2 weeks or karma to promote to stable.
21:15:37 <nirik> so in practice karma doesn't matter, right?
21:15:40 <nirik> well, I guess it can.
21:15:49 <dgilmore> nirik: security pushes to stable will happen more often that two weeks
21:16:03 <stahnma> +1 to proposal
21:16:08 <dgilmore> nirik: ill start the stable pushes on tuesday
21:16:08 <nirik> Proposed: stable pushes will be every 2 weeks on tuesday. Testing pushes will be every weekday or so. All updates must spend time in testing unless security related. 2 weeks or karma to promote to stable. Security pushes to stable will happen as needed.
21:16:11 <ssmoogen> +1 to proposal
21:16:35 <stahnma> very nice
21:16:45 <dgilmore> nirik: packages wont be autopromoted
21:17:28 <dgilmore> nirik: but if someone tries to move it to stable before 2 weeks  it will wait
21:17:40 <nirik> dgilmore: so a package thats been in testing 2 weeks won't go to stable unless the maintainer requests it?
21:17:50 <nirik> (that doesn't have karma)
21:18:29 <dgilmore> nirik: right
21:18:49 <nirik> thats kinda a pain... as maintainers will have to remember to go back and do that after two weeks...
21:19:00 <nirik> ie, they will need to keep track of a timer on all their updates.
21:19:02 <dgilmore> nirik: bodhi reminds people after 2 weeks
21:19:22 <nirik> can we customize that for epel? or it's a stock email?
21:19:29 <dgilmore> nirik: its stock
21:20:40 <ssmoogen> dgilmore, can other people be notified also?
21:20:43 * nirik looks to see what it says. I guess it's ok.
21:20:59 <dgilmore> ssmoogen: not sure
21:21:20 <nirik> yeah, that seems fine to me.
21:21:25 <ssmoogen> that way if stuff sits there for a looooong time managers could be notified?
21:21:29 <nirik> I guess I will write up a wiki page and post to the list?
21:21:55 <nirik> #agreed stable pushes will be every 2 weeks on tuesday. Testing pushes will be every weekday or so. All updates must spend time in testing unless security related. 2 weeks or karma to promote to stable. Security pushes to stable will happen as needed.
21:21:56 <dgilmore> ssmoogen: im sure we can get reports from bodhi
21:22:06 <nirik> #action nirik will post to the list and update the wiki with this policy info
21:22:33 <nirik> anything more on this topic, or shall we move on?
21:23:01 <ssmoogen> dgilmore, thanks
21:23:19 <nirik> #topic Bug day this weekend.
21:23:28 <nirik> Anything we need to prep for bug day?
21:23:33 <stahnma> starting in just a few hours!
21:23:39 <stahnma> not a whole lot.
21:23:48 <stahnma> I encourage everybody to participate
21:23:51 <nirik> I have friends visiting from out of town this weekend, but I should be around if anyone needs me... will try and poke at some bugs.
21:23:52 <stahnma> http://tr.im/epelbugs
21:24:10 <stahnma> I sent out a notice to lists
21:24:29 <stahnma> I didn't hear much back, but the general level of activity on bugs has certainly upped in recent weeks
21:24:49 <stahnma> I am hoping to close 4-6 bugs (personally), and maybe 20-30 overall
21:24:56 <stahnma> I have no idea how particpation will be though
21:25:05 <ssmoogen> stahnma, I am hoping to put in an hour-2 tomorrow
21:25:06 <stahnma> can we update the topic in #epel also?
21:25:32 <nirik> stahnma: yep. I can do that.
21:26:01 <stahnma> any questions on bug day?
21:26:17 <nirik> oh, it's in the topic... or you mean to note when it's running?
21:26:23 <ssmoogen> i am not authorized to do so.. I should ask kbsingh to add me :)
21:26:23 * nirik doesn't have any.
21:26:24 <stahnma> yeah
21:26:43 <ssmoogen> stahnma, what do we have steps on what to do?
21:26:43 <nirik> ssmoogen: I can add you there too... and whoever else wants to be added.
21:27:29 <stahnma> ssmoogen: the stuff I sent to the list
21:27:37 <stahnma> ssmoogen: basically, touch the bug
21:27:48 <stahnma> solicit information, try to fix, nudge a person, whatever it takes
21:28:01 <stahnma> it's really not very formal
21:28:15 <ssmoogen> stahnma, oh the email I have kept unread for the last couple of days... I feel stupid
21:28:33 <stahnma> no worries
21:28:44 <stahnma> also, marking a bug as 'Triaged' in teh whiteboard field helps the rest of us
21:28:48 <stahnma> if you worked on it
21:29:03 * stahnma should learn more about the ways of bz in teh future
21:29:27 <ssmoogen> same here
21:29:31 <nirik> anything further on bug day? or shall we move along?
21:29:41 <ssmoogen> move along
21:29:45 <ssmoogen> nothing to see here
21:29:53 <nirik> #topic Dealing with incompatible upgrades (zabbix, moinmoin, nagios, rdiff-backup, duplicity,etc)
21:30:08 <nirik> so, should we form some kind of policy here? or should it be case by case? or what
21:30:23 <nirik> I can talk about rdiff-backup specifically if people are interested.
21:30:41 <ssmoogen> I think we should have something here
21:30:58 <ssmoogen> its one of those issues that seperates what people expect from Fedora and Enterprise stuff
21:31:00 <vpv> as a new epel maintainer, I'd like to have some sort of basic policy to help me at least get started, I'm mostly worried about moin
21:31:41 <giesen> personally I'd love to see another repo
21:31:45 <giesen> something like dag
21:31:48 <nirik> sometimes it gets down to: do we want a non upstream maintained old version or a new incompatible version that breaks installs.
21:31:54 <giesen> but rigourous like epel
21:32:33 <ssmoogen> nirik, I think we have to have a little of both.
21:32:35 <stahnma> I tend to think that breakage is ok, but not ideal
21:32:43 <giesen> either that or red hat needs to push out releases yearly
21:33:10 <stahnma> in an enterprise of any size you're probably not hitting epel directly, you're probably importing/testing packages into a yum repo/satellite/spacewalk
21:33:16 <stahnma> I hope :)
21:33:16 <giesen> so packages aren't 30 months old by the time the next release arrives
21:33:21 <ssmoogen> stahnma, I used to until I had to spend a week with a moinmoin upgrade
21:33:31 * sharkcz also prefers new repo
21:33:32 <stahnma> admins also should be able to read the -announce list :)
21:34:01 <ssmoogen> stahnma, heheheheheheeh not hit a repo directly.
21:34:16 <ssmoogen> stahnma, i wish we could enforce that...
21:34:17 <stahnma> giesen: it's very difficult to move some addon software when the core and libs are staying at the same version for the enteprrise linux
21:34:23 <dgilmore> when does needing a new repo stop?
21:34:27 <nirik> well, in the case of rdiff-backup: epel has version 1.0.5. fedora has 1.2.8. The two versions cannot interoperate. 1.0.5 is no longer maintained upstream.
21:34:34 <dgilmore> sharkcz: not really a viable option
21:35:06 <stahnma> ssmoogen: oh, I know we can't.  but if you're responsible for an enterprise setup, you probably have some duties here
21:35:10 <dgilmore> stahnma: im sure most users are not using spacewalk/satellite
21:35:13 * quaid peeks in late, sorry, meeting not rescheduled on phone for some reason
21:35:29 <nirik> if I update epel, anyone who talks to another 1.0.5 will break... they will need to also update all machines backed up at the same time.
21:35:33 <ssmoogen> nirik, I would like us to look at the following proposal
21:35:35 <stahnma> dgilmore: I know very few shops that pull directly from the interwebs for packges
21:35:50 <dgilmore> stahnma: your in a special world
21:36:04 <stahnma> not just my workplace, I am saying lots of shops
21:36:29 <ssmoogen> introduce rdiff-backup105 which replaces rdiff-backup. It is a dead package with a README explaining that and where to get the source
21:36:31 <nirik> we do have customers that pull direct from epel... but they usually ask me about the updates before applying them... having the specific stable time is nice too, as they know to expect a pile at once.
21:36:41 <dgilmore> stahnma: most large installs i know of dont work that way
21:36:43 <ssmoogen> introduce rdiff-backup128 which is the new code
21:36:53 <nirik> ssmoogen: yeah, thats an option. Note however it's not maintained anymore upstream.
21:37:07 <dgilmore> stahnma: they pull from local mirrors/internet
21:37:13 <nirik> ssmoogen: also, fedora has the new version as rdiff-backup. ;)
21:37:31 <ssmoogen> nirik, I understand. In those cases we need to be able to give them source code and say what they need to do in the README
21:38:01 <ssmoogen> nirik I understand upstream Fedora issues.. that might require a proposal to FESCO?
21:38:05 <nirik> well, wouldn't simply an announcement about it be good enough? and a note in the changelog/cvs ?
21:38:29 <nirik> I think we should really try in these cases, but sometimes we have to just drive on and announce as best we can the change.
21:38:29 <ssmoogen> nirik, I have dealt with too many admins who just have yum-updateds turned on
21:38:37 <nirik> nagios may be in a similar bind.
21:38:50 <ssmoogen> s/admins/sites, fortune500, etc/
21:39:09 <stahnma> wow
21:39:12 <ssmoogen> yeah
21:39:19 <muep___> should things like rdiff-backup with an unstable interface even be in epel, in the first place?
21:39:22 <stahnma> I should sell consulting :)
21:39:48 <dgilmore> stahnma: you live in a different world to most
21:39:59 <ssmoogen> the issue is trying to work out a methodology that doesn't break most people
21:40:54 <ssmoogen> stahnma, I got an earful about how broken EPEL was because some cluster had done that... explaining them that it was poor systems administration didn't do anything
21:40:55 <muep___> of course the milk has already been spilled in the case of rdiff-backup, but in general? (and it's sometimes difficult to know when a project decides to break their compatibility)
21:41:09 <nirik> muep___: well, hard to say. They are usefull. No telling in advance how incompatible an upstream will end up being on updates.
21:41:22 <ssmoogen> the other issue is that sometimes people want the old stuff.
21:41:35 <nirik> duplicity was a worse case...
21:41:49 <muep___> personally I'd like a repo with up to date versions of stuff that still compiles on CentOS
21:42:12 <muep___> and maybe rdiff-backup would have been more appropriate there
21:42:34 <nirik> muep___: well, the 1.0.5 version in now works.
21:43:04 <ssmoogen> dgilmore, nirik I was looking at the unison software that is in Fedora
21:43:20 <nirik> up to date and CentOS/RHEL doesn't compute in my mind. ;)
21:43:20 <ssmoogen> there are two versions because they don't interoperate but are almost the same
21:43:29 <nirik> ssmoogen: yeah, I know.
21:43:31 <ssmoogen> what led to the naming scheme there
21:43:36 <ssmoogen> and can we follow it
21:43:49 <dgilmore> ssmoogen: :(
21:44:02 <nirik> ssmoogen: can you write up a proposal to do that kind of thing and send it to the list?
21:44:03 <ssmoogen> s/can/should/
21:44:17 <ssmoogen> I will do so
21:44:40 <nirik> #action ssmoogen will sent a post to the list about using different versioned packages to handle this mess.
21:45:05 <nirik> anything more on this? Or shall we revisit after talking on list?
21:46:14 <nirik> #topic Repoclosure/QA scripts
21:46:17 <nirik> ok, moving on. ;)
21:46:35 <nirik> with the move to bodhi, do we have any qa/repoclosure/broken deps checking?
21:46:38 <nirik> lmacken: ?
21:46:42 <lmacken> nope!
21:46:58 <nirik> ok.
21:47:04 <lmacken> we need some though... badly.
21:47:08 <ssmoogen> i have been asked to take that on
21:47:17 <nirik> so, we should look at setting something up for this. We really don't want to push broken things to stable.
21:47:30 <ssmoogen> lmacken, can the previous tools work as a starting point or should I look at complete rewrite?
21:47:36 <lmacken> EPEL has some hacked up version of repoclosure in the f-i repo i think
21:47:42 <lmacken> but I'm not sure if it handles multilib
21:47:46 <ssmoogen> and can they be plugged into bodhi or something
21:47:51 <lmacken> ssmoogen: you should probably talk to f13/skvidal as well
21:47:52 <lmacken> yeah
21:48:26 <ssmoogen> so that if we push p0wnU-1.5 and it requires rootkit9 and we don't have it.. bodhi will complain.
21:48:49 <ssmoogen> or is that not the proper place ofr it?
21:50:04 <dgilmore> lmacken: they dont do multilib
21:50:20 <lmacken> dgilmore: but that'll probably be Good Enough
21:50:32 <dgilmore> basically one of the biggest things we need to add is checks that pushing a package to stable wont result in broken deps
21:50:40 <dgilmore> lmacken: right
21:50:53 <lmacken> we should seewhat changes are in the epel-repoclosure script, and why the changes aren't upstream
21:51:00 <ssmoogen> nirik, I have to head out right now to pick up my kid from camp. sorry for the short notice. bbs
21:51:03 <lmacken> then we could somehow plug repoclosure into the bodhi push process...
21:51:23 <lmacken> yeah, I've gotta catch the bus now... but this is an important task and we need to get it done :)
21:51:25 <stahnma> I don't think they are very different, other than the yum configuration.  But I could be wrong
21:51:36 <lmacken> last time I did a diff, they were pretty different :\
21:51:39 <nirik> ok, so ssmoogen and lmacken will follow up on this?
21:51:47 <stahnma> it's been a while since  I looked, and it was a train wreck when I did :)
21:51:48 <nirik> should we file a ticket or something?
21:51:49 <lmacken> nirik: yeah
21:51:59 <lmacken> nirik: it's one of the oldest tickets in bodhi already :)
21:52:13 <nirik> lmacken: ok. ;)
21:52:28 <nirik> #action lmacken and ssmoogen will work on getting a repoclosure/deps checker going.
21:52:33 <lmacken> https://fedorahosted.org/bodhi/ticket/79
21:52:50 <nirik> anything else on this? or shall we move on?
21:52:59 <ssmoogen> oh and I am for repotags :) and I am out of here
21:53:08 <nirik> #topic Orphans
21:53:19 <nirik> I posted a list of orphans to the list. Got a few takers.
21:53:32 <nirik> #info check the list for a posting on orphans and how to give them a home.
21:53:43 <nirik> #topic open floor
21:53:49 <nirik> Anything else folks want to bring up?
21:55:03 * nirik will close the meeting in a few if no one chimes in with anything.
21:55:57 * nirik will close in 60 seconds.
21:56:56 <nirik> #endmeeting