18:00:42 <smooge> #startmeeting EPEL (2019-05-29)
18:00:42 <zodbot> Meeting started Wed May 29 18:00:42 2019 UTC.
18:00:42 <zodbot> This meeting is logged and archived in a public location.
18:00:42 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:42 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:42 <zodbot> The meeting name has been set to 'epel_(2019-05-29)'
18:00:42 <smooge> #meetingname epel
18:00:42 <smooge> #topic Chair and Introductions
18:00:42 <zodbot> The meeting name has been set to 'epel'
18:00:42 <smooge> #chair bstinson Evolution nirik smooge pgreco tdawson
18:00:42 <zodbot> Current chairs: Evolution bstinson nirik pgreco smooge tdawson
18:00:58 <pgreco> hi everybody
18:01:15 <smooge> hello everyone. I even emailed an agenda.. even if it was a day late and a dollar short
18:02:10 <smooge> #topic Agenda
18:02:11 <smooge> #info EPEL Proposal: Removal of PPC64 (Not PPC64le) in 2019-06-01
18:02:11 <smooge> #info EPEL Proposal: EPEL Master branch AKA Rawhide
18:02:11 <smooge> #info CentOS-8
18:02:30 * smooge waits for more people to get in
18:02:31 <nirik> morning
18:03:05 <Evolution> morning
18:03:08 <orionp> hello
18:03:10 * bstinson waves
18:03:27 <tdawson> Hello
18:05:26 <smooge> hello all
18:05:39 <smooge> #topic EPEL PPC64
18:05:39 <smooge> #info https://www.scrye.com/wordpress/nirik/2019/05/23/attention-epel6-and-epel7-ppc64-users/
18:06:16 <nirik> so yeah... not too much outcry yet that I have seen...
18:06:34 <smooge> I am currently archiving epel ppc64 to /pub/archive (I am doing everything jic)
18:06:53 <nirik> hum, well, I disabled it in pungi yesterday...
18:07:23 <nirik> ah, but the repos probibly will stay until we rm them... ok
18:07:46 <nirik> or we could just leave them there with no updates, but that might confuse people
18:08:32 <smooge> well I want to make sure that nothing accidently removes them
18:08:39 <smooge> I don't mind them being there dead for a bit
18:08:53 <nirik> ok.
18:09:04 <nirik> if we want to point people to archives it will need some mirrormanager magic
18:09:06 <smooge> I just want to make sure we don't make a change which somehow says 'oh let me clean out that stuff'
18:09:21 <smooge> and we find ppc64 is gone
18:09:45 <smooge> this also gives us a snapshot to point people to when we start making the changes for 7.7
18:10:34 <smooge> i figure I am doing this for f28 today.. I should do it for the other items
18:11:18 <smooge> #info PPC64 Big Endian is no longer composed for Fedora or EPEL. While it will live in regular trees for the time being, it has been archived to /pub/archive/epel just in case
18:11:45 <smooge> any other items from people?
18:11:54 <smooge> #topic EPEL Rawhide
18:11:54 <smooge> #info https://smoogespace.blogspot.com/2019/05/epel-proposal-epel-master-branch-aka.html
18:12:13 <smooge> and I am supposed to wait 30 seconds before clicking.. but new keyboard
18:12:14 * nirik hasn't seen this yet... too many barges of emails
18:12:58 <smooge> this is a pre-pre proposal. we talked about it generally 2 meetings ago so I put some ligaments on the bones
18:13:57 <nirik> I'm not sure I like dealing with betas...
18:15:27 <tdawson> I'm also a little concerned about the mass branching and rebuild.  What about those developers who like to have their Fedora and EPEL packages the same, this will cause a divergence between the two.
18:15:32 * nirik will have to ponder on it. I don't think I can decide anything based on 2min read
18:15:48 <smooge> not asking you to
18:16:15 <smooge> asking you to put it on your list so you can decide in 2-3 meetings. if I don't even put it out there it will be 4-5 months before we get to it
18:16:43 <nirik> I'd be inclined to not deal with betas, but then the question comes up about buildroots... the rawhide one you would want to always populate buildroot, the stable branches, no... so they need to be seperate there at least
18:16:52 * nirik nods
18:17:32 <smooge> the issue is that with RHEL looking to do major changes every 3-6 months.. the only heads up we may get for developers is with betas
18:19:02 <smooge> in any case, we can put some more flesh to the strawman or set it on light
18:19:09 <nirik> modules probibly mitigates this somewhat... you can experement in your rawhide/master module branch
18:19:35 <tdawson> Ya, I think I'm going to have to re-read a few times, and maybe have discussion on email before I can give a final answer.
18:20:53 <smooge> nirik, I am worried it will make ursa packages worse. If module X.Z is now default and we built against X.Y which was the old default.. all those packages get removed when an update occurs
18:21:35 <smooge> but that is probably a boat to worry about later.
18:21:50 <nirik> sure, they will have to adapt no matter what
18:21:55 <smooge> anyway something to read and discuss in the next couple of weeks
18:23:10 <nirik> yep. thanks for setting up the straw man smooge
18:23:20 <smooge> #topic CentOS 8
18:23:21 <smooge> #info bstinson says its all going smoothly. will be out any year now
18:23:26 <smooge> #undo
18:23:26 <zodbot> Removing item from minutes: INFO by smooge at 18:23:21 : bstinson says its all going smoothly. will be out any year now
18:23:38 <tdawson> *laughs*
18:23:54 <smooge> I believe things are working well and they have a koji you can watch progress and such on
18:24:15 <tdawson> Oh, I didn't know they were using koji this time.
18:24:47 <smooge> https://koji.mbox.centos.org/koji/
18:24:52 <pgreco> tdawson, yeap, fun.... :)
18:25:03 <smooge> I believe pgreco has his own one for his stuff
18:25:15 <pgreco> https://koji.armhfp-mbox.centos.org/koji/
18:25:35 <pgreco> that one is a little behind on packages
18:25:46 <pgreco> it will catch up in a few days
18:26:10 <smooge> next up
18:26:49 <smooge> #topic EPEL-8
18:26:49 <smooge> #info met last week on trying to get plans together
18:26:49 <smooge> #info wrote rawhide proposal last night
18:26:49 <smooge> #info sgallagh pushed libmodulemd-2.5.0 and I need to update grobisplitter we can roll then
18:26:49 <smooge> #info make branches in things tomorrow.
18:26:50 <smooge> #info make koji work next week
18:26:52 <smooge> #info make fedpkg work next week?
18:27:35 <sgallagh> smooge: I am cutting a 2.5.0-2 right now to fix a bug that grobisplitter will hit.
18:27:51 <smooge> sgallagh, okie dokie. I still need to make the changes so it can hit them
18:28:00 <sgallagh> 🙂
18:28:21 <smooge> if you keep fixing them before I even get to them.. you will fool me into believing software is flawless
18:28:39 * sgallagh laughs evilly
18:29:03 <smooge> so next week I hope to have a workflow plan on how we will build stuff. And hopefully figure out all the places I need to make changes
18:29:13 <smooge> then we can do so in stg and move from there
18:29:53 <smooge> does the above make any sense to you nirik and sgallagh as a plan?
18:30:02 <pgreco> smooge, once I finish with the c8/armhfp stuff, I can help with that
18:31:23 <sgallagh> smooge: It's a little light on details.
18:31:31 <sgallagh> What does "make koji work next week" mean?
18:31:51 <nirik> sorry, I stepped away for a min, looking
18:32:17 <smooge> make is so staging build-system can do builds for x86_64 and track them as appropriate
18:32:23 <nirik> note: I am away next week.
18:32:38 <nirik> I think it means adding all the koji tags/targets for it in stg?
18:33:06 <nirik> I should have the needed koji there the end of this week... and mergerepos_c was already updated in fedora
18:33:07 <sgallagh> ok
18:33:25 <smooge> nirik, yeah.. that does change my palsn
18:33:58 <smooge> I keep forgetting that it isn't last week when it is now
18:34:20 <smooge> thanks nirik what can I do and how should I help on that?
18:35:05 <nirik> well, I have builds, need to add some more patches now...
18:35:07 <smooge> I do have one thing for the committee to bikeshed: what to call the branch. We have el6, epel7 and we should go to epel8 or something else?
18:35:16 <nirik> epel8 sounds good to me.
18:35:33 <sgallagh> smooge: epel8.0 ?
18:35:49 <nirik> then people have to branch every minor ?
18:35:52 <sgallagh> Aren't we planning to have separate branches matching RHEL minor releases?
18:36:21 <sgallagh> Hmm, probably we should talk about the branching plan
18:36:33 <sgallagh> Is it going to be like Fedora's or opportunistic?
18:36:46 <smooge> so I was.. but then we talked about that causing problems because people won't want to branch every minor so we were looking at epel8-rawhide but that needs time to percolate
18:36:52 <nirik> --verbose? what is 'like fedoras'?
18:36:56 <sgallagh> meaning we only branch at the point where we want to change compatibility
18:37:03 <smooge> fc29 fc30 fc40
18:37:25 <smooge> which would map to epel8.0 epel8.1 epel8.2
18:37:43 <sgallagh> nirik: The plan is to effectively treat each minor release of RHEL as similar to a major release of Fedora, yes?
18:37:44 <smooge> is what I took the --quiet version of sgallagh's comment to mean
18:37:53 <sgallagh> So it's permissible to break compat on those boundaries, yes?
18:38:15 <nirik> I guess we talked about that yes...
18:38:58 <sgallagh> So we probably need to discuss what the branching strategy looks like.
18:39:28 * nirik would prefer whatever needs the least amount of work. ;) but not sure that would meet the goals.
18:39:43 <sgallagh> The least amount of work in what sense?
18:39:46 <tdawson> I thought we were just doing a rhel8 and rhel8-master or rhel8-rawhide
18:40:05 * sgallagh looks to smooge
18:40:30 <nirik> for releng, for maintainers. ;)
18:40:36 <tdawson> ^^ all of that replace rhel with epel
18:40:37 <smooge> so the rhel8 and rhel8-master is part of my https://smoogespace.blogspot.com/2019/05/epel-proposal-epel-master-branch-aka.html
18:41:15 <tdawson> If we had a epel8.1, and then epel8.2 came out, we wouldn't have anyplace to build the epel8.1 packages ... so what is the point of having those branches.
18:41:20 <nirik> anyhow, it would be nice if we wrote up possible ones to decide... I guess looking back over the minor release plan thing...
18:41:37 <sgallagh> tdawson: I'm not following.
18:42:05 <tdawson> We only have one, or two, build enviroments.  Not one for each release.
18:42:15 <nirik> https://fedoraproject.org/wiki/EPEL/Changes/Minor_release_based_composes but it doesn't talk about branching, just the archiving, etc.
18:42:34 <tdawson> When Fedora 31 comes out, we will still have a fedora 30 place to build fedora 30 packages, thus f31 and f30 make sense.
18:42:36 <sgallagh> tdawson: I think you're missing my point. You're assuming that to be true, I'm challenging whether it shoudl be
18:42:45 <pgreco> I don't like the 8.minor idea very much, I'd rather stick with just 2 branches max
18:43:01 <tdawson> When rhel 8.2 comes out, the epel build enviroment will move over to rhel 8.2, thus we will not have anyplace to build epel8.1 packages.
18:43:06 <smooge> ok will write up a branching strategies after this meeting
18:43:14 <nirik> if we have epel8.0 epel8.1 epel8.2... and build something for epel8.0, what happens to it?
18:43:26 <sgallagh> tdawson: That assumes we only have a single buildroot.
18:43:30 <nirik> we would need seperate update streams for all then right?
18:43:38 <smooge> sgallagh, that is all we have resources for
18:43:49 <sgallagh> okay
18:43:56 <tdawson> sgallagh: Yes ... and from everything I've seen, we will only have one build root ... well one buildroot and a rawhide root.
18:44:02 <smooge> s/we have resources/we currently have resources/
18:44:12 <nirik> the problem is about minor releases times.
18:44:57 <nirik> say 8.0 is out and 8.1 is not yet... but about to be. People need to build their incompatible stuff in epel8, but then what if they need a security/major bug fix to 8.0 stuff? unpush that and push the 8.0 thing?
18:45:19 <smooge> nirik, what happens when I try to commit and build against F24
18:45:44 <nirik> smooge: you can't. it's blocked/gone/disabled.
18:45:49 <smooge> that is an honest question because I don't think I have ever tried
18:46:05 <nirik> the koji target is removed, the branch won't accept commits.
18:47:13 <smooge> so one way would be that 8.0 would be disabled/blocked when 8.1 reached a certain time frame. [strawman]
18:48:13 <nirik> yeah, but then we have to mass branch 8.0 to 8.1 and coordinate with users which one they use... tricky, but possible I suppose
18:48:33 <smooge> nirik, I was going to say we mass branch from epel8-master to 8.0 then to 8.1 and then
18:49:17 <pgreco> the positive of that, is that it should be easier to determine what is really being maintained
18:49:40 <pgreco> but nowadays, finding the right branch in pagure is hard
18:49:41 <sgallagh> Might I suggest a straw-man, derived from smooge's last suggestion?
18:49:54 <smooge> we have 2 sets of downloads from CDN and configs epel8-master and epel8.x where the rhel8-sync pulls master from /8/ and /8.N/
18:50:03 <pgreco> after the centos/fedora git unification, and many epel branches
18:50:35 <smooge> in any case I don't plan to fix this here and it needs a better proposal than either my minor-branch or my rawhide one
18:50:41 <smooge> sgallagh, sure
18:51:21 <sgallagh> Two parts to this proposal:
18:52:22 <sgallagh> 1) We only ever have two branches supported: epel8-rawhide and epel8.X, where epel8.X matches the latest stable release of RHEL 8
18:53:05 <sgallagh> 2) At a new stable release, epel8.X+1 is branched from epel8-rawhide and epel8.X is blocked
18:54:14 <sgallagh> By policy, patches should always go to epel8-rawhide and then be backported to epel8.X
18:54:37 <sgallagh> So, functionally it's behaving like Fedora, except without a Branched or FN-2 release to support.
18:54:49 <sgallagh> EOF
18:55:22 <nirik> pretty good...
18:55:38 <nirik> one issue: what if I don't want a rawhide version? just stable?
18:56:08 <tdawson> nirik: As a user or a developer?
18:56:09 <sgallagh> nirik: How is that in conflict? You just keep the two branches in sync.
18:56:12 <nirik> I guess you just keep it in sync...
18:56:18 <nirik> but thats two branches to deal with
18:56:23 <nirik> tdawson: developer
18:56:34 <sgallagh> as an aside, I continue to wish that git allowed you to just alias two branches.
18:56:50 <tdawson> I think it would be the same as fedora.  Make it stable in rawhide and sync it over.
18:57:14 <sgallagh> nirik: we also have the new fedpkg support for building for multiple releases from the same branch
18:57:34 <tdawson> Ooohh
18:57:42 <sgallagh> so we *could* just allow you to keep it only in epel8-rawhide and just have a package.cfg that also builds for the stable release simultaneously
18:57:58 <nirik> that might be nice...
18:59:31 <sgallagh> though if the epel8.X name is changing regularly, we should probably ask for a fedpkg RFE to alias that so it doesn't need to be updated at each branch.
18:59:39 <nirik> how often are minors? 6m or 3m?
18:59:47 <sgallagh> e.g. build for `[ epel8.x, epel8-rawhide ]`
18:59:52 <sgallagh> nirik: 6mo
19:00:17 <sgallagh> That's what they said at the Summit keynotes anyway
19:00:31 <nirik> it would be also nice if fedpkg clone epel8 works to get epel8.x where x is the newest one
19:01:19 <sgallagh> nirik: Cart before the horse. First let's figure out if we agree that this is the branching structure we want, then we can make it nicer
19:01:43 <nirik> sure
19:02:08 <smooge> ok first off. we are over time
19:02:18 <sgallagh> Also, if we went this route we might want to just have s/epel8-rawhide/epel8/ ssince it's the place we want people to do their work.
19:02:21 <nirik> we then just need to set a dist tag...
19:02:43 <sgallagh> So `fedpkg clone epel8` would just get you the standard place.
19:03:07 <sgallagh> And if we make the `package.cfg` part of branch creation, they don't have to know about epel8.x unless they specifically want to separate them
19:03:24 <smooge> would epel8 for master/rawhide make more sense and epel8.x for the latest (or epel8-latest)?
19:03:39 <smooge> or what sgallagh is saying
19:03:51 <sgallagh> smooge: I think you just repeated what I was saying, so +1 :)
19:04:19 <nirik> the problem with that is we then default to 'fast whatever' instead of 'stable support until the next minor release at the soonest'
19:04:20 <sgallagh> Or, wait. I think I see the difference
19:04:58 <sgallagh> nirik: Which, honestly, is probably what Fedora packagers will expect to have
19:05:15 <smooge> so fedpkg defaults to fastest because developers are used to that workflow.. the epel-release will point to the slowest
19:05:20 <smooge> s/will/could/
19:05:36 <sgallagh> smooge: Exactly what I was thinking.
19:05:38 <nirik> if we were starting now yeah, but in the past the expectiation I think was 'support this for 10 years, no incompatible upgrades'
19:05:45 <nirik> but I guess we are starting 8 now.
19:05:59 <sgallagh> And we probably don't actually need to have separate epel8.1, epel8.2 branches, now that we come to it.
19:06:06 <sgallagh> Which I think was the extra piece smooge was saying.
19:06:48 <nirik> so if we don't, we don't mass branch? or ?
19:08:08 <sgallagh> nirik: Well, if we're defaulting to "we expect you to use the rolling branch most of the time", we may not need to mass branch.
19:08:10 <smooge> well I think we need to have 2 proposals and people need to discuss and we as the steering committee decide
19:08:43 <sgallagh> Because the set of people who have separated the two is also the set of people that probably understands why they did so
19:09:09 <nirik> sure...
19:09:10 <smooge> because there is either 'we expect you to use rolling and build to stable with this command whne you want.', 'we expect you to use rolling and we mass branch/rebuild regularly'
19:09:15 <sgallagh> smooge: I just got volunteered to write this up, didn't I?
19:09:36 <smooge> I don't volunteer people who outrank me
19:09:37 <nirik> I think most maintainers will take the easiest path
19:09:50 <nirik> but that will break users who expect epel to try and avoid incompatible changes
19:09:58 <sgallagh> smooge: I'm not on EPSCo. You outrank me here :)
19:10:26 <sgallagh> nirik: Well, that's the thing. I think this addresses that too
19:10:42 <nirik> ok... write up in email to list? ;)
19:10:50 * nirik has to go... contractor is here.
19:11:08 <sgallagh> If you want to make an incompatible change, we set policy that you have to split the stable and rawhide builds and carry the breaking one only in rawhide for N months and announce it coming with the next minor release.
19:11:18 <sgallagh> epel@?
19:11:24 <smooge> epel-devel
19:11:29 <smooge> we can work on this shortly
19:11:36 <smooge> Thank youy all for coming this week
19:11:43 <smooge> #endmeeting