20:00:06 <tdawson> #startmeeting EPEL (2022-08-24)
20:00:06 <zodbot> Meeting started Wed Aug 24 20:00:06 2022 UTC.
20:00:06 <zodbot> This meeting is logged and archived in a public location.
20:00:06 <zodbot> The chair is tdawson. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
20:00:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:06 <zodbot> The meeting name has been set to 'epel_(2022-08-24)'
20:00:08 <tdawson> #meetingname epel
20:00:08 <zodbot> The meeting name has been set to 'epel'
20:00:09 <tdawson> #chair nirik tdawson pgreco carlwgeorge salimma dcavalca dherrera gotmax[m]
20:00:09 <zodbot> Current chairs: carlwgeorge dcavalca dherrera gotmax[m] nirik pgreco salimma tdawson
20:00:11 <tdawson> #topic aloha
20:00:18 <rsc> .hello robert
20:00:19 <zodbot> rsc: robert 'Robert Scheck' <redhat@linuxnetz.de>
20:00:23 <carlwgeorge> .hi
20:00:24 <zodbot> carlwgeorge: carlwgeorge 'Carl George' <carl@redhat.com>
20:00:26 <tdawson> Hi rsc
20:00:29 <jcpunk> .hi
20:00:30 <zodbot> jcpunk: jcpunk 'Pat Riehecky' <riehecky@fnal.gov>
20:00:30 <tdawson> Hi carlwgeorge
20:00:38 <tdawson> Hi jcpunk
20:00:50 * carlwgeorge waves at jcpunk
20:01:00 * jcpunk waves back
20:01:10 <tdawson> It feels like I've just seen ya'll.
20:01:35 <pgreco> .hi
20:01:36 <zodbot> pgreco: pgreco 'Pablo Sebastian Greco' <pablo@fliagreco.com.ar>
20:01:48 <tdawson> Hi pgreco
20:01:59 <rcallicotte> .hi
20:02:00 <zodbot> rcallicotte: rcallicotte 'Robby Callicotte' <rcallicotte@mailbox.org>
20:02:04 <pgreco> on a zoom call in parallel
20:02:12 <rcallicotte> cool!
20:02:37 <nirik> morning
20:02:37 <tdawson> Hi rcallicotte
20:02:42 <tdawson> Morning nirik
20:03:05 <davide> .hello dcavalca
20:03:05 <zodbot> davide: dcavalca 'Davide Cavalca' <dcavalca@fb.com>
20:03:23 <tdawson> Hi davide
20:03:41 <rcallicotte> devconf seemed like a lot of fun from the live streams
20:05:04 <tdawson> It was the first devconf I'd gone to.  Ya, I think it was pretty fun.  It was nice physically being around people.
20:05:13 <tdawson> #topic EPEL Issues https://pagure.io/epel/issues
20:05:15 <tdawson> https://pagure.io/epel/issues?tags=meeting&status=Open
20:05:38 <tdawson> There are two open issues.
20:06:00 <tdawson> I'm going to start with the one relating to jcpunk first
20:06:08 <tdawson> .epel 190
20:06:09 <zodbot> tdawson: Issue #190: Keep everything for 30 days after EOL - epel - Pagure.io - https://pagure.io/epel/issue/190
20:06:13 <jcpunk> the basic goal is to try and get the work required by the infra folks into a sort of "common expectations".  They've got to manage the RHEL subscriptions used for the koji build roots and I'd like to have some sort of official statement from both groups that consume the RHEL bits for builds on when they'd like the subscriptions to expire (which will lock the builds, but that could be done earlier)
20:07:23 <jcpunk> For Stream, we'd like folks to be able to finish some bits that may appear near the end of the subscription
20:08:08 <nirik> yeah, I am not sure this makes sense for the way we have things setup
20:08:44 <jcpunk> Understood, I'm mostly trying to make sure that I've explored the idea - not trying to change EPEL policy
20:08:47 <nirik> well, I guess it could...
20:09:08 <nirik> basically, when a RHEL version goes EOL we disable everything in koji and move the repos to archive/
20:09:45 <jcpunk> tacitly it is a question of "how fast after EOL" do these things happen
20:10:03 <jcpunk> the specific timetable isn't really specified anywhere I found
20:10:32 <nirik> true. Usually it's pretty fast in fedora/epel land... at least the disabling part.
20:11:07 <davide> is there any harm to delaying this by 30 days?
20:11:39 <nirik> It really irks the releng in me... letting people build against EOL things and distribute them.
20:12:46 <nirik> "you don't need to hurry and move to a supported OS... take an extra 30 days when who knows what issues that can never be fixed pile up on you"
20:13:11 <tdawson> :)
20:13:30 <davide> if I'm reading the issue right, the spirit of this is meant to allow for folks to do last-minute builds to adjust for stuff that might have landed in RHEL in the final week
20:13:40 <jcpunk> davide: +1
20:13:46 <jcpunk> that was my intent
20:13:51 <tdawson> jcpunk: How likely is that to happen?
20:14:02 <nirik> so, as smooge notes when we move things to archive, mirrormanager keeps working fine for them.
20:14:16 <jcpunk> On that front, I can't predict when RHEL Eng publishes things
20:14:16 <nirik> if the desire is to just build against things that should work fine...
20:14:41 <nirik> if the desire is for epel builds / composes / updates to still be enabled, thats the part I don't care for. ;)
20:15:28 <jcpunk> This breaks down into a few real bits, when should the RHEL subscriptions expire and when should the tags get locked
20:15:44 <jcpunk> when the tags get locked, things should move to archive
20:17:11 <jcpunk> I'm not totally sure how Infra manages the RHEL subscriptions for the build roots, but they are effectively the ones that asked me to try and get a clear set of "what we'd like to see" for those subscriptions
20:17:13 <nirik> sure. IMHO that should be on EOL day or close to it. ;)
20:17:29 <nirik> jcpunk: you mean centos infra? or ?
20:18:04 <jcpunk> err yeah, I should be more precise.  the folks supplying the packages for the CentOS Community Buildsystem
20:18:43 <nirik> can we clarify if this request is to build and distribute EPEL packages after eol, or just build against them/have repos avaible?
20:20:51 <tdawson> That would be good to clear up, because I might have written the original ticket incorrectly.
20:20:56 <jcpunk> Mostly this is a request for very clear specific expectations for how the EPEL folks want to handle the EOL regarding the RHEL bits.  Having the bits available, but the tags locked could permit some sort of emergency build of $critical_package that has a super terrible zero day that is in EPELX two days after EOL.  On the one hand, folks should get off it and move to current release.  On the other hand, folks tend to miss those s
20:20:56 <jcpunk> orts of deadlines.
20:21:29 <jcpunk> Does EPEL want to have a bit of wiggle room for that sort of use case?
20:21:45 <davide> this is compounded by the fact that the EOL might well happen near/over a holiday season
20:21:56 <davide> so folks could just not be around to respond quickly is something happens
20:22:09 <davide> IMO a 30 day leeway period here seems totally reasonable
20:22:15 <nirik> I think the expectation should be that repos will always be available, but at EOL no further builds will be possible
20:22:50 <tdawson> Which holiday are you thinking of?  All EOL are during May or June
20:22:57 <nirik> 30days post EOL could be a holiday or weekend. ;)
20:23:46 <davide> I was specifically thinking about the CentOS Linux 8 eol which fell across december iirc, but that could have been a oneoff
20:24:18 <tdawson> That was a one-off.  And that wasn't a RHEL EOL.
20:24:43 <jcpunk> I'm not sure on the world wide holiday schedule.....
20:24:54 <davide> nirik: sure, but by then you'd have had 30 days to deal with anything
20:25:10 <nirik> davide: sure, but you have 10 years to know when a rhel eol is. ;)
20:25:28 <davide> the point is that if we EOL on X, and something landed in RHEL on X-2 days, there just isn't a lot of room to respond
20:25:32 <jcpunk> and 6+ years of folks saying they'll upgrade next quarter
20:25:45 <davide> so we'd give folks a little bit of wiggle room to deal with whatever is needed to wrap things up nicely
20:26:52 <tdawson> I'm a little concerned this would cut into our EOL party ... would we have it at the real EOL or 30 days after?
20:26:54 <jcpunk> to my mind, the day after EOL RHEL isn't more insecure than before, it is just that no one is looking to see what is out there any longer.  There probably are security issues, but they aren't really known
20:26:54 <nirik> yes, but it's never ending
20:27:04 <nirik> Oh, RHEL pushed this, I need a new X build.
20:27:13 <nirik> Oh, X build broke Y, I need to rebuild Y
20:27:23 <carlwgeorge> I don't think the hypothetical makes sense here
20:27:59 <carlwgeorge> that late in the RHEL cycle, any update would be security fix only
20:28:05 <tdawson> Sorry, what I said was supposed to be funny ... re-reading I see it came off as sarcastic.
20:28:29 <carlwgeorge> no library changes that would require an EPEL rebuild
20:29:34 <nirik> tdawson: text is bad for nuance. ;(
20:29:45 <jcpunk> indeed, text is hard
20:29:48 <tdawson> Yes it is :(
20:30:10 <Eighth_Doctor> .hello ngompa
20:30:13 <zodbot> Eighth_Doctor: ngompa 'Neal Gompa' <ngompa13@gmail.com>
20:30:16 <nirik> davide: were you involved in this request from the centos side? do you know if the intent was just to build against epel or also do epel updates?
20:30:27 <nirik> hey Eighth_Doctor
20:30:36 <tdawson> carlwgeorge does have a very good point though.  Any update in RHEL at that late in the game shouldn't be requiring rebuilds.
20:30:44 <jcpunk> indeed
20:31:11 <davide> Conan Kudo: I remember some discussion of this, but I'm fuzzy on the details
20:31:15 * Eighth_Doctor is catching up on scrollback
20:31:55 <davide> on the CentOS side, I would expect as long as folks can continue doing builds in CBS against RHEL (and EPEL) after the EOL for a little bit, we'd probably be ok
20:31:57 <carlwgeorge> so the only thing it would allow for would be for epel updates independent of RHEL for x days after the RHEL EOL
20:32:00 <Eighth_Doctor> EPEL is mirrored into CBS infrastructure, so this should not be an issue
20:32:42 <Eighth_Doctor> the ticket request does not make sense for public use perspective
20:32:48 <Eighth_Doctor> neither for CentOS Stream 9 nor EPEL
20:32:57 <Eighth_Doctor> because MirrorManager will just be repointed to archive endpoints
20:33:07 <nirik> yeah, smooge noted that in the ticket
20:33:22 <jcpunk> I'm really looking for "do we have common expectations" and the CentOS docs for what those are for CBS are what I'm writing now.  The EPEL policies say "after EOL" but I'm really looking for something more specific.
20:33:23 <Eighth_Doctor> if CentOS Stream 8 is onboarded into MirrorManager, then this will also apply there
20:34:00 <Eighth_Doctor> we can stop allowing builds after 30 days, but typically it gets done within a week or so, unless there's an infra freeze going on
20:34:20 <nirik> jcpunk: I would say "assume builds are stopped at EOL"
20:34:43 <jcpunk> having really clear expectations could theoretically help "infra freeze" folks schedule their bits
20:34:56 <nirik> RHEL eols are a big event, we usually do them the day of... but moving to archive might be a few days later.
20:34:57 <Eighth_Doctor> there will be no infra freezes in parallel with RHEL EOLs if they follow the pattern going forward, since May is after Fedora releases are out
20:35:02 <Eighth_Doctor> ending EPEL 30 days after would essentially match the Fedora release policy of N+2+1 month
20:35:25 <jcpunk> the Fedora release policy is mostly how I landed on "30 days"
20:35:35 <jcpunk> but they own the whole os
20:35:42 * nirik is really not in favor, but will do it everyone else is.
20:36:17 * jcpunk accepts whatever EPEL thinks is best for their workflow
20:36:36 <tdawson> I can go either way, but I'm standing by nirik on this.
20:37:16 <Eighth_Doctor> that might not necessarily be a bad idea to ensure all the pending updates are flushed and frozen
20:37:43 <jcpunk> Can you get something into https://docs.fedoraproject.org/en-US/epel/epel-policy/ that is unambiguous?
20:37:49 <nirik> it's also going to confuse a bunch of people... "hey, my EOL rhel7 box just got some updates... wth?"
20:37:56 <Eighth_Doctor> I tend to agree more with nirik, especially since there's no public access harm
20:38:09 <tdawson> Words are hard today ... that came out wierd too.  I'm going to vote for what nirik wants.  Since he is most likely the person to do it.  And if he changes his mind, I'll follow it over too.
20:38:15 <Eighth_Doctor> if we were doing something crazy like yoinking the content from MirrorManager, then a delay is appropriate
20:38:17 <Eighth_Doctor> but we're not
20:38:29 <nirik> yeah, repos should always be available... even after eol...
20:39:03 <Eighth_Doctor> as a matter of course, I fundamentally disagree with how CentOS historically handled EOLs and I expect that with MirrorManager and c9s, they'll follow the way EPEL and Fedora do
20:39:36 <Eighth_Doctor> the public access harm that CentOS inflicted in the past is something I never want to see Fedora do
20:40:16 <Eighth_Doctor> since we don't do that, I think we should just document that updates are cut off EOD on EOL day for RHEL
20:40:35 <Eighth_Doctor> we can flush the last days of updates and allow no more
20:40:42 <jcpunk> and tags are by EOD of EOL day
20:40:57 <jcpunk> So long as I've got clear doc on that, I'm happy
20:41:39 <jcpunk> s/tags are/tags are locked/
20:42:03 <tdawson> So ... we're in agreement?
20:42:08 <davide> agreed, I think the important part here is getting whatever we end up settling on codified so folks know what the expectations are
20:42:19 <jcpunk> and having it written somewhere I can find
20:42:46 <nirik> that seens like a reasonable expectation
20:42:46 <tdawson> Anyone volunteering to update the docs?
20:43:26 <nirik> what the heck. I can.
20:43:33 * nirik adds to his todo
20:43:48 <jcpunk> <3 thanks nirik
20:44:13 <jcpunk> I'd say this ticket is settled
20:44:34 <tdawson> And ... the other issue is on retiring ...
20:45:07 <tdawson> Since the first issue took a while, and the retiring issue is having a disucssion on email, are we ok skipping it this week?
20:45:56 * gotmax[m] is kind of here now
20:46:13 <Eighth_Doctor> retiring issue? oh for retiring packages
20:46:29 <Eighth_Doctor> I would anticipate that being a long discussion, so yeah
20:46:34 <gotmax[m]> Although my laptop is dead, so forgive phone typos
20:46:54 <tdawson> gotmax[m]: Are you ok keeping the dicussion in emails for another week?
20:47:03 <gotmax[m]> Sure
20:47:18 <tdawson> Thanks.
20:47:32 <tdawson> With the limited time, I don't think we'd get very far in the discussion.
20:47:34 <gotmax[m]> I was just going to add that I'm working on some improvements to the Docker packaging that would make packaging it in EPEL 9 easier.
20:47:48 <gotmax[m]> I wouldn't mind branching it if I can get a co-maintainer
20:47:56 <tdawson> gotmax[m]: cool
20:48:09 * tdawson takes two steps back from Docker.
20:48:16 <gotmax[m]> I'm just bringing it up, because there was some discussion about this in #centos
20:48:24 <davide> fyi I filed a ticket to get golist branched
20:48:29 <gotmax[m]> Why?
20:48:39 <rcallicotte> did you say docker?
20:48:44 <gotmax[m]> golist as part of the RHEL go-rpm-macros package
20:48:45 <davide> because otherwise most of the golang packaging stuff doesn't work in epel
20:49:04 <davide> are you sure? last I check it wasn't available at all
20:49:08 <Eighth_Doctor> golist isn't in RHEL
20:49:09 <davide> * checked
20:49:15 <gotmax[m]> I know. The RHEL maintainers installed golist to a different directory, but didn't patch the macros
20:49:25 <gotmax[m]> And they still haven't fixed it
20:49:40 <gotmax[m]> There is a workaround
20:49:46 <davide> is there a bugzilla I can track for this?
20:49:47 <gotmax[m]> But we can also package golist
20:50:03 <davide> https://bugzilla.redhat.com/show_bug.cgi?id=2096096 is the golist one
20:50:04 <Eighth_Doctor> huh, oh there is a bundled golist
20:50:14 <davide> I haven't had time to walk over the deps but it didn't look too bad
20:50:23 <gotmax[m]> They patched up Fedora's macros and broke them in multiple ways :/
20:50:26 <tdawson> Since we've only got 10 minutes left, I'm going to move on, golist or no.
20:50:38 <Eighth_Doctor> yeah, that's another long discussion
20:50:43 <davide> yep
20:50:45 <Eighth_Doctor> it looks pretty chainsawed
20:51:03 <gotmax[m]> Look at the go-rpm-macros bugzilla component and the open PRs for more info
20:51:07 <tdawson> The only old business I've got it the epel survey - https://tinyurl.com/epelsurvey2022   goes until the end of august
20:51:09 <gotmax[m]> Anyways...
20:51:25 <tdawson> #topic EPEL-7
20:51:26 <tdawson> CentOS 7 will go EOL on 2024-06-30
20:52:10 <gotmax[m]> I did not realize we were still on Old Business...
20:52:13 <gotmax[m]> I thought it was open floor
20:52:21 <gotmax[m]> Sorry for getting us off track
20:53:06 <tdawson> Not  a problem.  Anything for EPEL7 ?
20:53:37 <tdawson> #topic EPEL-8
20:53:39 <tdawson> CentOS Stream 8 goes EOL in 2024-05-31
20:54:53 <tdawson> So, I have "-retirement party" in my notes right after this .... thus the wierd comment I said earlier ... the comment still looks wierd, but that's the reason for it.
20:55:01 <tdawson> #topic EPEL-9
20:55:03 <tdawson> CentOS Stream 9 goes EOL in 2027-05-31
20:55:48 <tdawson> #topic General Issues / Open Floor
20:56:02 <tdawson> OK ... looks like we're back on docker :)
20:56:02 <carlwgeorge> for epel9, there was an unannounced incompat upgrade of re2
20:56:13 <tdawson> ouch
20:56:28 <carlwgeorge> #info unannounced incompat upgrade of re2 required rebuilding bloaty package
20:56:28 <gotmax[m]> So name bump?
20:56:30 <carlwgeorge> #link https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2022-f063125ebb#comment-2674089
20:56:34 <carlwgeorge> yeah
20:56:43 <carlwgeorge> i think bloaty was the only one
20:56:50 <carlwgeorge> *needing a rebuild
20:59:20 <tdawson> So ... hmm ... I'm a little concerned about this ... cuz the epel maintainer wasn't the person who did it.
21:00:02 <carlwgeorge> yeah it's not great
21:00:15 <carlwgeorge> we really don't have any recourse other than finger wagging
21:00:53 <tdawson> Ya ... now that I lookw I see he did a pull request .... :(   and I thought it was for fedora.
21:00:54 <carlwgeorge> if anyone knows that maintainer and would like to approach them to make sure they saw my bodhi comment, i'd appreciate it
21:01:27 <tdawson> I'll talk to them.
21:01:32 <carlwgeorge> thanks
21:01:53 <tdawson> No, thank you for pointing it out.  I should have been more diligent.  I'm the epel maintainer. :(
21:02:14 <tdawson> Anything else before we wrap up?
21:02:26 <carlwgeorge> i sent the email but please vote on https://pagure.io/epel/issue/192
21:02:36 <carlwgeorge> for jonathanspw to join the epel packagers sig
21:02:48 <jonathanspw> oh, .hi
21:02:54 <jonathanspw> .hi
21:02:55 <zodbot> jonathanspw: jonathanspw 'Jonathan Wright' <jonathan@almalinux.org>
21:03:05 <tdawson> Hi jonathanspw
21:04:26 <tdawson> It looks like there are enough votes.   Does someone want to add jonathanspw ?
21:04:36 <carlwgeorge> policy says to leave it open for a week
21:04:45 <carlwgeorge> or did i misread it
21:05:45 <tdawson> Oh, sorry.  I still haven't caught up on my email, I was thinking it had been a week.
21:06:37 <tdawson> Anyway, before I say anything else stupid ... it looks like we're over time.
21:06:59 <tdawson> Thank you all for comming and for the good discussions.
21:07:23 <tdawson> #endmeeting