<@tdawson:fedora.im>
18:00:09
!startmeeting EPEL (2024-06-26)
<@meetbot:fedora.im>
18:00:10
Meeting started at 2024-06-26 18:00:09 UTC
<@meetbot:fedora.im>
18:00:10
The Meeting name is 'EPEL (2024-06-26)'
<@tdawson:fedora.im>
18:00:17
!meetingname epel
<@meetbot:fedora.im>
18:00:18
The Meeting Name is now epel
<@tdawson:fedora.im>
18:00:22
!topic aloha
<@yselkowitz:fedora.im>
18:00:43
!hi
<@zodbot:fedora.im>
18:00:43
Yaakov Selkowitz (yselkowitz)
<@nirik:matrix.scrye.com>
18:00:44
morning
<@rcallicotte:fedora.im>
18:00:48
!hi
<@zodbot:fedora.im>
18:00:50
Robby Callicotte (rcallicotte) - he / him / his
<@jrichardson:matrix.org>
18:00:54
!hi
<@zodbot:fedora.im>
18:00:56
James Richardson (jrichardson)
<@rcallicotte:fedora.im>
18:01:01
I'm here but fighting fires :(
<@tdawson:fedora.im>
18:01:19
Hi yselkowitz Robby Callicotte and James Richardson
<@nhanlon:beeper.com>
18:01:22
!hi
<@zodbot:fedora.im>
18:01:23
Neil Hanlon (neil) - he / him / his
<@tdawson:fedora.im>
18:01:24
Morning nirik
<@carlwgeorge:matrix.org>
18:01:36
!hi
<@zodbot:fedora.im>
18:01:39
Carl George (carlwgeorge) - he / him / his
<@tdawson:fedora.im>
18:01:42
Hi Neil Hanlon and Carl George
<@nhanlon:beeper.com>
18:01:47
heya folks
<@tdawson:fedora.im>
18:02:15
Robby Callicotte: Those fires need to be fought. But we're glad you are at least partly here.
<@nhanlon:beeper.com>
18:03:20
🪣 🌊
<@salimma:fedora.im>
18:03:39
!hi
<@zodbot:fedora.im>
18:03:40
Michel Lind (salimma) - he / him / his
<@tdawson:fedora.im>
18:03:51
Hi Michel Lind 🎩
<@dherrera:fedora.im>
18:04:14
!hi
<@smooge:fedora.im>
18:04:14
hello
<@zodbot:fedora.im>
18:04:15
Diego Herrera (dherrera) - he / him / his
<@tdawson:fedora.im>
18:04:26
Hi Diego Herrera
<@tdawson:fedora.im>
18:04:33
Hello Stephen J Smoogen
<@tdawson:fedora.im>
18:05:23
!topic End Of Life (EOL)
<@tdawson:fedora.im>
18:05:29
CentOS Stream 8 / epel-8-next went EOL on 2024-05-31
<@tdawson:fedora.im>
18:05:29
https://endoflife.date/rhel
<@tdawson:fedora.im>
18:05:29
CentOS Stream 9 / epel-9-next goes EOL in 2027-05-31
<@tdawson:fedora.im>
18:05:29
https://endoflife.date/centos-stream
<@tdawson:fedora.im>
18:05:29
RHEL 7 / epel-7 will go EOL on 2024-06-30
<@tdawson:fedora.im>
18:05:42
So next week is the end of epel7
<@nirik:matrix.scrye.com>
18:05:57
⌛
<@tdawson:fedora.im>
18:06:04
Do we have everything in place? Or tickets created?
<@salimma:fedora.im>
18:06:21
related: tickets filed against epel 7 will auto-close right?
<@nirik:matrix.scrye.com>
18:06:26
I was going to bring up this and epel8-next eol...
<@carlwgeorge:matrix.org>
18:06:32
we can probably take the cs8/epel8-next eol reminder out of that text since it has already passed
<@nirik:matrix.scrye.com>
18:07:03
it might be nice to have a SOP for eol epel releases and spell out what you want releng to do and what others want to do and such
<@tdawson:fedora.im>
18:07:48
So have it in the epel documentation?
<@carlwgeorge:matrix.org>
18:07:55
indeed, and to spell out what happens differently for epel next. we're going through the first ever eol for one of those. https://pagure.io/releng/issue/12147
<@nirik:matrix.scrye.com>
18:08:17
sure, either in epel or releng, I don't care as long as it's somewhere we can find it. ;)
<@carlwgeorge:matrix.org>
18:08:24
then again, we're only ever going to do it once more for epel9-next, so maybe not worth making a full sop for that part
<@carlwgeorge:matrix.org>
18:08:38
!link https://pagure.io/releng/issue/12147
<@nirik:matrix.scrye.com>
18:08:42
yeah, next is less important...
<@tdawson:fedora.im>
18:09:04
If we did it in the releng documentation, where would be the place for that?
<@nirik:matrix.scrye.com>
18:09:14
but for example, for epel7, do we want to try and ping all open bugzilla bugs telling people it's about to eol and then close them EOL after? (which is what we do for fedora)
<@carlwgeorge:matrix.org>
18:09:39
i'm all in favor of doing things like fedora whenever possible
<@smooge:fedora.im>
18:09:56
In this case, I would probably say it is a little late to ping people
<@tdawson:fedora.im>
18:09:57
Oh, that's much nicer than I was thinking for the bugs. I was just thinking of closing them all.
<@nirik:matrix.scrye.com>
18:10:00
in here: https://docs.fedoraproject.org/en-US/infra/releng_misc_guide/ or possibly in https://docs.fedoraproject.org/en-US/infra/release_guide/release_eol/
<@salimma:fedora.im>
18:10:09
true, I guess for Fedora we ping them
<@yselkowitz:fedora.im>
18:10:34
there is a much bigger gap between epel versions than there are fedora versions, bugs are more likely to be no longer applicable to the next epel
<@nirik:matrix.scrye.com>
18:10:44
note: someone would need to actually do this work. ;) (not it!)
<@smooge:fedora.im>
18:10:54
I was thinking that at this point it is a 'close all EL-7' tickets time as a lot of the items are not going to work in EL-8 or beyond
<@smooge:fedora.im>
18:11:24
not it for doing the closing or not it for writing the instructions?
<@smooge:fedora.im>
18:11:47
I can write up the stuff for EPEL-7 if that is ok?
<@tdawson:fedora.im>
18:11:56
I'm ok doing the bugzilla closings. But it will probrubly not happen until July 8th due to vacation and holidays.
<@smooge:fedora.im>
18:12:06
as an emeritus last act type thing
<@smooge:fedora.im>
18:12:36
'i come here not to praise EPEL-7, but to bury it'
<@nirik:matrix.scrye.com>
18:12:49
note that there's scripts that are used in fedora... could be modified
<@tdawson:fedora.im>
18:14:28
Doing a batch bugzilla <anything>, isn't too hard. But I wouldn't mind looking at Fedora's to make the wording as close as possible.
<@salimma:fedora.im>
18:14:52
I guess we can also help close our individual tickets unless Troy has an automated script
<@salimma:fedora.im>
18:15:09
but yeah might as well do it uniformly and avoid confusion
<@smooge:fedora.im>
18:15:12
I will post something to the list of potential wording?
<@smooge:fedora.im>
18:15:24
Do we have a ticket in the epel tracker for this?
<@nirik:matrix.scrye.com>
18:15:42
note that the fedora scripts close things EOL, which normal users cannot I don't think... might need privs to do that
<@tdawson:fedora.im>
18:15:50
Closing all the epel7 bugs? Not that I've noticed.
<@carlwgeorge:matrix.org>
18:16:46
tangential question for the group related to retirements: i think it would be worthwhile to have epel-release 8 obsolete epel-next-release 8. any systems that still have epel-next-release installed, whether their os is eol or not, should stop checking for that repo, and we have an easy way to accomplish that with an obsoletes in the main epel-release package. i haven't set up a pr for this yet, but can throw that up right after the meeting. any concerns?
<@tdawson:fedora.im>
18:17:15
Oh ... ok. I'll have to check and see if I can do an EOL. But ... I'll wait till next week to check.
<@tdawson:fedora.im>
18:17:47
Carl George: So, even if they are still running CentOS Stream 8, it will just stop them from checking the epel8-next repo ... correct?
<@carlwgeorge:matrix.org>
18:18:27
yup. or if they happen to have epel-next-release installed on rhel8 or similar, which definitely happens even if it isn't recommended.
<@tdawson:fedora.im>
18:19:04
I don't see any problems with that.
<@nirik:matrix.scrye.com>
18:19:21
not sure it's worth the trouble, but sure... ;)
<@tdawson:fedora.im>
18:20:10
Is there already a releng ticket for archiving (and other stuff) for epel7 ?
<@tdawson:fedora.im>
18:20:32
Or will that happen the day it goes EOL?
<@carlwgeorge:matrix.org>
18:21:12
i can file one similar to the epel8-next one for eol, and yeah i think on eol day is the right time for that
<@carlwgeorge:matrix.org>
18:21:31
on that day i can yank the epel7 build targets in koji to kick things off
<@nirik:matrix.scrye.com>
18:21:45
+1
<@carlwgeorge:matrix.org>
18:21:45
or maybe the day after to be safe
<@carlwgeorge:matrix.org>
18:22:27
did we happen to have an issue for epel6 retirement?
<@nirik:matrix.scrye.com>
18:22:44
cannot recall.
<@carlwgeorge:matrix.org>
18:23:34
if anyone happens to run across that, i'd love to bookmark it, and i'm sure Stephen J Smoogen would fine it useful for what he's writing up for epel7 retirement as well
<@carlwgeorge:matrix.org>
18:23:45
if anyone happens to run across that (if it exists), i'd love to bookmark it, and i'm sure Stephen J Smoogen would fine it useful for what he's writing up for epel7 retirement as well
<@smooge:fedora.im>
18:23:59
we did not have an issue for EPEL-6 because I don't think we had this repo when that happened
<@tdawson:fedora.im>
18:24:15
So, this is the tasks I have (along with who is doing them)
<@tdawson:fedora.im>
18:24:23
- have epel8-release obsolete epel8-next - carl
<@tdawson:fedora.im>
18:24:23
- Close bugzillas - tdawson
<@tdawson:fedora.im>
18:24:23
- Write-up SOP - Smooge
<@tdawson:fedora.im>
18:24:23
- File releng issue to EOL epel7 - carl
<@carlwgeorge:matrix.org>
18:24:23
there are epel6 issues in there, but nothing i see for retirement
<@tdawson:fedora.im>
18:25:41
This is exciting. I know we've EOLed epel releases before, but epel7 is rather huge in terms of people using it.
<@tdawson:fedora.im>
18:26:09
Anything else before I move on to the next topic?
<@tdawson:fedora.im>
18:26:38
!topic EPEL Issues https://pagure.io/epel/issues
<@tdawson:fedora.im>
18:26:43
https://pagure.io/epel/issues?tags=meeting&status=Open
<@tdawson:fedora.im>
18:26:55
We don't have any open issues this week.
<@tdawson:fedora.im>
18:27:37
Next week we'll probrubly be talking about "catch", but that can wait for next week.
<@carlwgeorge:matrix.org>
18:27:49
small follow up, https://pagure.io/epel/issue/278 is closed/approved but the implementation isn't done yet. i need to follow up with adrian.
<@carlwgeorge:matrix.org>
18:28:05
implementation being over in https://pagure.io/fedora-infrastructure/issue/11968
<@carlwgeorge:matrix.org>
18:28:28
but good news, the 8.8 test worked as expected
<@tdawson:fedora.im>
18:28:53
cool
<@tdawson:fedora.im>
18:29:16
Thanks for keeping us informed, as well as thanks to those working on it.
<@tdawson:fedora.im>
18:29:32
!topic Old Business
<@tdawson:fedora.im>
18:29:49
do we have any old business? Other than the old business we've already covered?
<@yselkowitz:fedora.im>
18:30:15
any updates on epel 10?
<@tdawson:fedora.im>
18:30:26
I was about to ask about that.
<@tdawson:fedora.im>
18:30:43
Let me give an update on the 10 mass rebuild.
<@tdawson:fedora.im>
18:31:16
It's done, and packages are making their way through gating. As a result ... right now composes are failling, but that's sorta expected.
<@tdawson:fedora.im>
18:31:50
I'm hoping that by next week enough of the packages have made it through the gating that composes are back to normal.
<@jonathanspw:fedora.im>
18:32:52
!hi
<@zodbot:fedora.im>
18:32:53
Jonathan Wright (jonathanspw)
<@tdawson:fedora.im>
18:32:54
How that affects epel10 plans ... I'm not positive.
<@tdawson:fedora.im>
18:32:59
Hi Jonathan Wright
<@jonathanspw:fedora.im>
18:33:07
Sorry I'm late, distracted by contractor here installing 2g fiber at my house :)
<@jonathanspw:fedora.im>
18:33:13
finally joining the modern world...
<@salimma:fedora.im>
18:33:19
2g !
<@jonathanspw:fedora.im>
18:33:32
They assure me 8g is coming this year.
<@carlwgeorge:matrix.org>
18:33:34
humble brag lol
<@tdawson:fedora.im>
18:33:35
making me jealous. ;)
<@rcallicotte:fedora.im>
18:33:58
Jealous... I'm still on carrier pigeon... lol
<@tdawson:fedora.im>
18:34:53
I've been assured that any type of fiber is coming to a house near me* soon** (* 15 miles away ** next few years)
<@salimma:fedora.im>
18:35:00
I remember when 2.5G was fast. for mobile...
<@salimma:fedora.im>
18:35:23
AT&T has been promising to lay fiber to my village for years. when I moved here I was so excited when I tried to signup and it says 5 for the bandwith
<@salimma:fedora.im>
18:35:32
then I realized it's 5... Mbps. no wonder it felt a bit cheap
<@carlwgeorge:matrix.org>
18:35:58
James Richardson switched the batcave sync for c10 from composes to mirrored content, and also added epel10.0 tags to robosignatory
<@jonathanspw:fedora.im>
18:36:04
I'm getting 2g for $99/mo
<@jonathanspw:fedora.im>
18:36:15
it's from windstream
<@carlwgeorge:matrix.org>
18:36:30
next up will be an attempt to verify the robosignatory stuff works in staging
<@tdawson:fedora.im>
18:36:43
Carl George: Very cool.
<@tdawson:fedora.im>
18:36:59
robosignatory is where the packages get signed after being built, correct?
<@carlwgeorge:matrix.org>
18:37:11
correct, and does some tag shuffling
<@carlwgeorge:matrix.org>
18:37:56
my basic understanding is it watches for builds to get the incoming tag, then signs them, then tags them with the outgoing tag and removes the incoming tag
<@nirik:matrix.scrye.com>
18:38:05
it should be pretty easy to test that in prod too (well, easier if you don't want to wait for me to have time to look at staging)
<@tdawson:fedora.im>
18:38:12
Cool.
<@carlwgeorge:matrix.org>
18:38:33
hehe, testing in prod is always fun
<@carlwgeorge:matrix.org>
18:38:44
https://pagure.io/fedora-infra/ansible/pull-request/2066#request_diff is the change
<@tdawson:fedora.im>
18:39:24
I think in a couple of weeks, things should have settled down from the mass rebuild, and maybe, we can start thinking about prod.
<@carlwgeorge:matrix.org>
18:40:00
testing that in prod will involve actually putting stuff in prod koji, which brings me to another question
<@conan_kudo:matrix.org>
18:40:27
!hi
<@zodbot:fedora.im>
18:40:29
Neal Gompa (ngompa) - he / him / his
<@tdawson:fedora.im>
18:40:35
Hi Conan Kudo
<@salimma:fedora.im>
18:40:56
that's not bad. I pay $80 for 1 gbit
<@carlwgeorge:matrix.org>
18:41:06
the plan of record has been a soft launch as soon as we get all the pieces in place, with an official launch in november or december. ideally i'd like the soft launch ready in time for the epel10 hackfest at flock.
<@jonathanspw:fedora.im>
18:41:18
currently paying $110 for "gigabit" (non-symmetrical) cable 1g/35m
<@tdawson:fedora.im>
18:41:44
That sounds good to me, if possible.
<@salimma:fedora.im>
18:41:52
the sooner the better, I plan to dogfood the refactored ebranch and ideally I can say at Flock that it definitely works :)
<@carlwgeorge:matrix.org>
18:42:23
while looking through the steps for bodhi, something occurred to me. we could set up the epel10.0 bodhi release initially to work like rawhide, in that there is not testing repo and updates get autocreated for each build. i think that would be a good initial setup for epel10 during the soft launch phase.
<@nirik:matrix.scrye.com>
18:42:37
should be doable. Just needs the koji setup and then bodhi config and syncing...
<@jonathanspw:fedora.im>
18:42:41
That would be really handy
<@nirik:matrix.scrye.com>
18:43:13
might be unexpected for people... otherwise should be fine
<@carlwgeorge:matrix.org>
18:43:21
it doesn't drastically reduce the amount of work needed, the main thing i think would be we don't have to wait for the fedpkg changes for minor version branch requests to land
<@salimma:fedora.im>
18:43:32
oh... yeah. and when there's a RHEL 10 beta that's when you enable gating but for the 10.0 repo
<@salimma:fedora.im>
18:43:46
we're distinguishing between 'epel10' and 'epel10.0' right?
<@carlwgeorge:matrix.org>
18:44:05
it varies depending on the scope
<@tdawson:fedora.im>
18:44:08
Ya, at some point we'll need to put it back the other way, but for starting, I really like the idea.
<@carlwgeorge:matrix.org>
18:44:35
initial setup would be:
<@carlwgeorge:matrix.org>
18:44:35
<@carlwgeorge:matrix.org>
18:44:35
epel10 branch -> epel10.0 build -> epel10.0 update -> epel10 repo path
<@carlwgeorge:matrix.org>
18:45:09
i'm thinking right before the official announcement in nov/dec is when we turn on the testing repo
<@carlwgeorge:matrix.org>
18:45:24
this will also really help the bootstrap effort i think, like at the flock hackfest
<@yselkowitz:fedora.im>
18:46:02
will make early epel 11 a lot easier too
<@tdawson:fedora.im>
18:46:26
Sorry, you lost me with the "epel10 branch -> epel10.0 build -> epel10.0 update -> epel10 repo path" My brain isn't following.
<@carlwgeorge:matrix.org>
18:46:31
speaking of which, i did get approved for funding, so i'll be at my own hackfest after all
<@salimma:fedora.im>
18:46:37
we can sort out the details later, I suppose
<@tdawson:fedora.im>
18:46:43
Ya!!
<@nirik:matrix.scrye.com>
18:47:17
cool
<@rcallicotte:fedora.im>
18:47:19
YAY!!
<@jonathanspw:fedora.im>
18:47:22
y'all pray for me though
<@smooge:fedora.im>
18:47:24
congrats!
<@jonathanspw:fedora.im>
18:47:42
y'all pray for me though. I get to listen to the freight train at night :p
<@carlwgeorge:matrix.org>
18:48:23
in dist-git, the branch name will be epel10, so that people running `fedpkg request-branch epel10` get what they expect. when they create a build for that, it will land in an epel10.0 koji tag, and feed into an epel10.0 bodhi update. the repo path will be epel/10 so that requests coming in don't need to determine the minor version (it's not in the c10 os-release file).
<@tdawson:fedora.im>
18:48:46
Ahhh .... ok. Yes.
<@tdawson:fedora.im>
18:48:59
Thanks for explaining.
<@carlwgeorge:matrix.org>
18:49:07
sure thing
<@jonathanspw:fedora.im>
18:49:35
Will building against a specific target be possible or only `epel10`?
<@carlwgeorge:matrix.org>
18:50:19
yes, once the targets exist. the initial rollout will only be `epel10` and `epel10.0` targets, going to the same tags.
<@tdawson:fedora.im>
18:50:44
For right now, I believe there can only be the one target. But after RHEL is released, I believe you should be able to target ..... ya ... carl said it better. :)
<@jonathanspw:fedora.im>
18:50:48
perfect
<@carlwgeorge:matrix.org>
18:51:43
i'm going to start drafting an update to post in discourse and the mailing list to set expectations around the soft launch period, and ask maintainers to not start building immediately if they see the build targets, because that doesn't mean all the pieces are working yet. we have eager maintainers that just start building when they notice stuff, as i learned with epel9-next.
<@tdawson:fedora.im>
18:52:12
We're getting a little short of time. I'm going to switch to Open Floor, and if nobody has any open floor stuff, we can go back to epel10.
<@tdawson:fedora.im>
18:52:23
!topic General Issues / Open Floor
<@tdawson:fedora.im>
18:52:34
Does anyone have anything for Open Floor?
<@carlwgeorge:matrix.org>
18:53:10
i was gonna say all that epel10 stuff in the open floor, but it already came up, so i think i'm good
<@jonathanspw:fedora.im>
18:53:27
Will EPEL7 drop off on/near EOL date or is it getting pushed out 30 days or something like how centos 8 was when it hit EOL?
<@jonathanspw:fedora.im>
18:54:02
(I saw stuff about fedora specifically but it didn't mention epel)
<@tdawson:fedora.im>
18:54:30
I believe Carl is going to do start things the day of, or the day after
<@carlwgeorge:matrix.org>
18:54:41
i'm open to either way tbh
<@tdawson:fedora.im>
18:54:42
Starting with the target.
<@carlwgeorge:matrix.org>
18:54:54
yeah, day of or day after, yank the build target for sure
<@nirik:matrix.scrye.com>
18:55:06
depends on what you mean by drop off. ;)
<@nirik:matrix.scrye.com>
18:55:49
the archiving will be sometime soon after that... as far as usage drop off... who knows? ;)
<@carlwgeorge:matrix.org>
18:56:48
considering that mirrormanager is already set up to redirect eol epel versions to archive mirrors, perhaps we don't need a 30 day period like centos
<@jonathanspw:fedora.im>
18:56:50
I mean repos, sorry
<@carlwgeorge:matrix.org>
18:56:58
considering that mirrormanager is already set up to redirect eol epel major versions to archive mirrors, perhaps we don't need a 30 day period like centos
<@tdawson:fedora.im>
18:57:53
My vote is for the day after (for the target). Someone is always going to say they thought it was at the very end of the day. So you might as well do it the day after.
<@carlwgeorge:matrix.org>
18:58:57
centos doesn't do the archive redirect to vault, they force people to manually configure vault repos if they want them. i think that's the reason for having a grace period for taking down the repos.
<@jonathanspw:fedora.im>
18:59:24
I didn't realize epel had an auto-redirect. IMO it shouldn't.
<@nirik:matrix.scrye.com>
18:59:25
ok. I didn't know there was a grace period.
<@tdawson:fedora.im>
19:00:14
Hmm ... we didn't have a grace period for CentOS Stream 8. It got yanked so fast some bits had to be put back for me to do the final release.
<@nirik:matrix.scrye.com>
19:00:22
fedora has always done it that way. Changing that is of course possible, but I don't think it would be very nice to do that for epel7 a few days before it eols
<@jonathanspw:fedora.im>
19:00:47
not worth messing with, wasn't implying we should try to change it.
<@jonathanspw:fedora.im>
19:00:55
not worth messing with, wasn't implying we should try to change it, at least not for 7
<@nirik:matrix.scrye.com>
19:01:27
yeah, a choice was made long ago to do that and now it's kinda expected. ;( But we could change it with enough noise...
<@jonathanspw:fedora.im>
19:01:27
I didn't realize Fedora did it. I've never stayed on an EOL fedora release so never ran into it I guess.
<@carlwgeorge:matrix.org>
19:01:27
https://mirrors.fedoraproject.org/metalink?repo=epel-4&arch=x86_64 still works fwiw
<@tdawson:fedora.im>
19:01:32
Looks like our time is up. Thank you all for coming and for the good disucssions. And thank you all for all the work you do for EPEL and it's community.
<@salimma:fedora.im>
19:01:42
thanks Troy!
<@jrichardson:matrix.org>
19:01:49
thanks Troy!
<@nirik:matrix.scrye.com>
19:01:50
epel4! 🤘
<@tdawson:fedora.im>
19:01:59
I'll talk to ya'll next week, if not sooner.
<@carlwgeorge:matrix.org>
19:02:01
maybe the grace period thing was intentionally a centos linux thing, not stream
<@tdawson:fedora.im>
19:02:17
!endmeeting