20:00:20 #startmeeting EPEL (2022-07-27) 20:00:20 Meeting started Wed Jul 27 20:00:20 2022 UTC. 20:00:20 This meeting is logged and archived in a public location. 20:00:20 The chair is tdawson. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 20:00:20 Useful Commands: #action #agreed #halp #info #idea #link #topic. 20:00:20 The meeting name has been set to 'epel_(2022-07-27)' 20:00:22 #meetingname epel 20:00:22 The meeting name has been set to 'epel' 20:00:23 #chair nirik tdawson pgreco carlwgeorge salimma dcavalca 20:00:23 Current chairs: carlwgeorge dcavalca nirik pgreco salimma tdawson 20:00:25 #topic aloha 20:00:34 .hi 20:00:35 rcallicotte: rcallicotte 'Robby Callicotte' 20:00:37 .hi 20:00:38 carlwgeorge: carlwgeorge 'Carl George' 20:00:41 morning 20:00:48 Hi rcallicotte 20:00:52 Hi nirik 20:00:55 Hi carlwgeorge 20:00:57 .hi 20:00:58 dherrera: dherrera 'Diego Herrera' 20:01:02 Hi dherrera 20:01:08 * rcallicotte waves at tdawson 20:01:24 .hi 20:01:24 salimma: salimma 'Michel Alexandre Salim' 20:01:31 Hi salimma 20:02:44 .hello gotmax23 20:02:45 gotmax[m]: gotmax23 'Maxwell G' 20:02:59 Hi gotmax[m] 20:03:14 .hi 20:03:15 pgreco: pgreco 'Pablo Sebastian Greco' 20:03:20 ok, looks like zodbot is not recognizing me... 20:03:46 * gotmax[m] is having internet problems, but I'm here 20:03:59 Hi pgreco ... if that really is who you are. :) 20:04:13 there we go... 20:04:31 .hello robert 20:04:32 rsc: robert 'Robert Scheck' 20:04:33 tdawson: most of the time I'm not even sure myself 20:04:40 Hi rsc 20:05:02 Okay, it should be working now 20:05:15 How's everyone doing? 20:05:20 gotmax[m]: That was a quick fix. 20:05:31 Doing good ... and it's now 5 after soo ... 20:05:37 #topic EPEL Issues https://pagure.io/epel/issues 20:05:39 https://pagure.io/epel/issues?tags=meeting&status=Open 20:05:43 tdawson: I'm in an airport and I got disconnected 20:06:16 The only open issue is #39 20:06:19 .epel 39 20:06:20 tdawson: Issue #39: Retiring EPEL Packages (End of lifing CEPH in EL7 and process improvement) - epel - Pagure.io - https://pagure.io/epel/issue/39 20:06:35 I renamed it so I would quit being confused each week. 20:07:14 We discussed the orphan issue a bit last week 20:07:30 (tdawson brought up orphaning in the ticket) 20:07:33 I don't see tht anything has changed since the last time we looked at it ... do we need to discuss anything else at the moment? 20:07:39 .hi 20:07:40 jonathanspw: jonathanspw 'Jonathan Wright' 20:07:42 Oh ya ... except that. :) 20:07:46 Welcome! 20:07:52 Hi jonathanspw 20:08:02 Sorry I'm late :) 20:08:14 jonathanspw: Is there anything you are specifically wanting to talk about this week? 20:08:16 .hello dcavalca 20:08:17 davide: dcavalca 'Davide Cavalca' 20:08:22 Hi davide 20:08:28 Nope I'm just a fly on the wall this time. 20:08:28 In summary: orphaning doesn't work the same as it does on Fedora. It can't be done per branch and there's no automated way to pick up EPEL branches. 20:08:48 We could try to create a manual process to handle this, but I'm not sure what it would look like 20:09:14 gotmax[m]: Ya ... it's a pain ... but so it people just retiring packages because they no longer want to maintain them. 20:09:23 it is possible to assign bugs to orphan for epel... not the same tho I agree 20:10:25 It would be nice if we could find someway to work EPEL into the weekly emails Miro sends out 20:10:30 And into the FTBFS/FTI process 20:10:40 FTI would be easiest 20:10:51 The FTI stuff should be covered by will-it already I think 20:10:58 Isn't that what will-it.. 20:11:09 what davide said :) 20:11:15 Does will-it file bugs automatically? 20:11:23 Yep ... will-it currently lists them, but I haven't gotten to the "create bugzilla's" part of it 20:11:35 Sorry ... "yep" was to a previous question. 20:11:50 Does will-it file bugs automatically? - Not yet 20:11:52 we could see if he is willing to add them or show someone else how to ? 20:12:09 This is the FTI bug script I believe: https://pagure.io/releng/blob/main/f/scripts/ftbfs-fti/follow-policy.py 20:12:39 I'm not sure where it runs, but the RHEL mirror being private might be a problem 20:13:29 For will-it I just use centos7 and alma8 and alma9 20:14:13 That script queries the buildroot repo, but I suppose having it use the composed repos wouldn't be a huge deal 20:14:29 *composed repos for EPEL 20:15:04 Ah ... that's right ... you usually can't querry the EPEL buildroot due to restrictions. But maybe that script is special. 20:15:46 Though, I think we've gotten off course of the original thing, which was retiring (instead of uninstallable) packages. 20:16:02 Yes, my fault :) 20:16:21 Not a problem ... but if you want to open an issue about it, that would be ok. 20:17:21 I'm wondering about starting an email about epel orphaning (or maybe call it something else) ... maybe some other people have good ideas about how to do it. 20:17:34 Fine with me 20:17:49 +1 on discussing this on the list 20:18:01 We could create an issue tracker and just manually retire them if nobody picks it up 20:18:10 Or something like that 20:18:25 Do we have data around how many packages we expect? 20:18:38 We'd most likely need a process that's (somewhat) independent from Fedora's process 20:18:41 I haven't a clue. But I expect the nubmers to be low. 20:18:43 The manual approach is fine if it's in the single to low double digits 20:18:49 we meant to start in on the list last week, I didn't get round to it, my bad. gotmax (He/Him) if you want to kick it off that'd be great 20:18:57 But it'll start getting unwieldy quickly otherwise 20:19:59 OK. Let's move this to the mailling list and move on. 20:20:09 +1 20:20:14 #topic Old Business 20:20:33 Did we have any old business from last week? 20:20:47 Do we want to talk more about the modules thing? 20:21:02 I just realized I forgot to read last weeks meeting. Sorry about that. 20:21:13 We didn't come to a conclusion last time but not sure if there's anything left to talk about 20:21:20 What's up with modules? 20:21:44 Yeah, I was about to explain it, but then I got disturbed... 20:22:05 Basically, some of RHEL's default modules that we build against are retired 20:22:29 * nirik thought we came to the conclusion we couldn't do much about this. 20:22:43 Yeah, that was also my conclusions 20:22:44 I suppose we could manually... 20:22:46 *no s 20:23:18 We could fix this if it was possible to build modular packages against RHEL 8 modules 20:23:34 But we can't just start building against different module streams by default 20:23:40 It's effectively a mass SO name bump 20:25:30 so, I wonder... has anyone ever seen anyone complain about this? 20:25:41 is this just PHP, or are there other modules too? 20:25:42 (the PHP situation has been a problem for months I think) 20:25:45 There are many 20:25:49 I had issues with perl before too 20:26:09 In Hyperscale we ended up having to disable perl support in perf because of a missing module 20:26:17 That we couldn't just easily build 20:26:19 I would think this would hit our users a lot more than it seems to... 20:26:47 I'd say within fb I've seen this come up once or twice a year at least 20:26:55 Yeah, it would cause a lot of problems for the users 20:27:06 Here are some problematic default module streams: 20:27:14 php 7.2 - May 2021 20:27:14 But usually we'd find some crappy workaround and move on with life 20:27:40 https://access.redhat.com/support/policy/updates/rhel-app-streams-life-cycle 20:27:41 nginx 1.14 - same date 20:28:06 not much probibly has runtime deps on nginx. 20:28:08 mercurial 4.8 - Nov 2022 20:28:13 but php I would expect to... 20:28:23 is postgres in that list??? 20:28:33 9.6 is 20:28:47 So ... what does "retire" actually mean. If they are still around and installable ... I don't see that as "retired" 20:28:53 10 / 2024 20:28:53 13 / 2026 20:28:55 But that's not the default 20:28:58 yeah, the PHP one has definitely bitten us at FB. 20:29:00 It's not supported 20:29:25 But ... neither is EPEL (by red hat at least) 20:29:34 And they don't recieve updates 20:29:44 to me that's not really retired then... if they are still around and installable. 20:29:45 AFAIK 20:30:05 rcallicotte: but unmaintained 20:30:06 But they are still marked as "Default" ... to me this seems like a Red Hat problem. 20:30:21 Yes, it is mainly a RH problem 20:30:31 But it still affects us 20:30:44 well, they arent even available from redhat are they? it's just that we have old copies of them? 20:30:45 True, but has anyone opened a bug against it? 20:30:52 tdawson: I agree, if the default is retired, it's something RH should fix, but I'm not sure how they would fix it without breaking a lot of things 20:30:55 or are they also still on rhn? 20:31:10 No, I'm on a RHEL 8 machine right now. Everything listed above is marked "d" with default. 20:31:25 they're still available in rhel8 20:31:48 can we petition the redhat folks to make a new module flag for "unmaintained"? bad joke... 20:31:51 ok, in that case, I fear there's nothing we can do. 20:31:53 Grobisplitter just picks up whatever is the default stream in RHEL 20:32:04 tdawson: I agree, we should file a bug 20:32:15 yeah i don't see there being anything epel can do about rhel product decisions 20:32:21 I am 99% sure it will be closed notabug. ;) 20:32:38 My understanding is that they don't even get security updates? 20:33:15 sure, but I bet there's customers who depend on them still existing.. (thats the version they certified X with, etc) 20:33:22 and i also don't think we should block people from building epel packages that depend on packages from retired default module streams. we don't block people from building against rhel7 packages with "declined to fix" cves. 20:33:34 This is what the lifecycle page says: 20:33:34 > Customers never have to update, they can update at their own convenience. If a customer asks for support beyond the specified life of an Application Stream, they may be asked to move to a newer Application Stream to take advantage of bug fixes and CVE advisories. 20:33:54 Agreed, but I do like the idea of somehow flagging these as retired and not picking them by default unless one asks 20:33:56 yeah... 20:34:15 But yeah, that'd be a rhel change and it seems unlikely to happen in a minor release 20:34:38 I just keep thinking to myself ... 7 more years ... just 7 more years until RHEL 8 retires. 20:34:43 But we'd have to figure out what existing packages depend on retired modules if we wanted to "flag" them 20:35:30 Now I know how the Windows team felt with Windows 8. ;) 20:35:37 Wasn't there some koji/mbs issue about the module building issue? I think there was some progress on the koji side, but I'm not sure if it went anywhere. 20:36:33 I think there is, but I don't quite understand the nuance myself. Remi kept insisting you can't build PHP modules in EPEL 8 because of this 20:36:44 You are correct ... that is didn't go anywhere. Everytime someone tried to do something, it exploded in their face. 20:36:58 Last time I asked I was told it plain doesn't work 20:37:41 In https://pagure.io/epel/issue/75 from Neal 5 months ago: 20:37:45 And on the CentOS side there's no mbs at all for cbs, so CentOS sigs can't build modules either 20:37:52 Even though this ticket is closed, I figure I should put a "minor" update in here: as of Koji 1.28, it became possible for Koji to configure modules for a buildroot per tag. 20:37:52 So now only MBS needs updating to handle it. 20:38:21 right, it needs mbs changes. 20:38:25 nothing we can do 20:38:28 I feel like I keep pushing us off topic :( 20:39:01 Not a problem (about the topic) ... I was just about to ask if we can table this (or put a pin in it, whatever the term) and move on. 20:39:16 Yeah, I was about to say that 20:39:23 We're kind of stuck here 20:39:31 #topic EPEL-7 20:39:36 Anything for EPEL7? 20:39:36 But at least we have a better idea of the problem now :) 20:39:47 I would bet no 20:40:43 I didn't see anything in the emails or chat that I can remember. 20:41:01 #topic EPEL-8 20:41:02 CentOS Stream 8 goes EOL in 2024-05-31 20:41:33 Anything for EPEL8 this week, other than the module stuff? 20:42:06 ansible 6.0.0 should have landed in EPEL 8 next a week or two ago to match up with c8s's ansible-core bump, but the ansible-core update in c8s has been failing to build for two weeks 20:42:12 EPEL 8 Next 20:42:26 i did a cve fix update for python-eventlet in epel8, should be in testing soon 20:42:33 I fixed an FTI for novnc due to a missing dep that someone had reported 20:42:58 And filed a few more stalled package tickets 20:43:29 Oh ... talking of stalled packages, I noticed that the last couple of mine have actually been getting the epel-packagers-sig added, and not just me. 20:43:46 I usually ask for the sig group to be added 20:43:48 Did you explicitly ask them to add you? 20:43:56 I ask for both. 20:44:33 I did change the spacing of how I ask though. So I have "please put in the epel-packagers-sig" on one line, and have a blank line, then "and please add me" on the next line. 20:44:48 I've done the last few... 20:44:55 thats what was requested no? 20:44:57 Do we have / should we have a "how to process these" checklist for releng? 20:45:19 Is there a template in the releng repo for this? 20:45:25 That could help 20:45:27 I mean an issue template 20:45:28 no, because it's just using the web interface 20:46:04 I mean the pagure issue templates 20:46:09 Or even better, a tool to automate processing 20:46:20 " I am a member of the epel-packagers-sig and I am requesting epel-packagers-sig be given commit permissions so that I, or a member of the SIG, might branch and build this package in epel9." 20:46:29 that to me says... 'add the epel-packager-sig' 20:46:47 Yup 20:46:51 if we also want to add the requester, it could be more clear? 20:47:16 to be fair I think that would be good, because then we could also make the requestor bugzilla assignee for epel 20:47:19 Do we want that in general? 20:47:30 Anyway, whoever has been processing it has been getting it right lately. 20:47:43 We could also make the sig the bz assignee 20:47:45 I don't want to hammer on them once they start getting it right. 20:47:51 Like it's done for rust packages 20:48:02 Yeah, that's supposed to happen, but I think there were several issues were that didn't happen 20:48:14 I like that better than having an individual 20:48:22 that process would help with my stalled packages for epel9 20:48:27 As people move on and stuff gets missed or forgotten 20:48:37 we currently cannot. 20:48:45 the sig isn't a assignable entity... 20:48:52 I am not sure why 20:48:56 :( 20:49:08 we could probibly fix it... would need some digging. 20:49:31 I've seen plent of others where a sig is the bugzilla default. Specifically lightdm 20:49:32 What's different about the EPEL packagers sig vs. e.g. the rust sig? 20:50:43 pagure let me put @epel-packagers-sig as the Bugzilla assignee for one of my packages 20:50:56 I don't know off the top of my head, would need investigation 20:51:00 Not sure if it actually works on the Bugzilla side though 20:51:09 really? huh. perhaps I just fat fingered it. 20:51:20 we can test out of meeting? 20:51:31 Sounds good ... let's move on. 20:51:39 #topic EPEL-9 20:51:40 CentOS Stream 9 goes EOL in 2027-05-31 20:51:43 nirik: You need the @ 20:52:00 I have 2 epel9 things 20:52:02 this -> https://bugzilla.redhat.com/show_bug.cgi?id=2104624 20:52:07 oops 20:52:15 rcallicotte: you go first 20:52:40 gotmax[m]: yeah, I thought I did... but perhaps there was lag or something. 20:53:00 The salt folks have been slow-walking this request 20:53:27 nirik: I did it on python38-toml-epel, so we can see if it syncs over to Bugzilla. Not sure how long that takes 20:53:29 i've read the whole bug, and i'm of the opinion that rcallicotte can just do the stalled process to get himself added 20:53:38 ... until 20 min ago it looks like. 20:54:15 even in the last reply, it sounds like they want you to create a different package rather than granting you access to the salt package 20:54:27 thats... odd. 20:54:30 yeah thats the tone im getting as well 20:54:32 yeah that was an interesting read 20:54:34 yes, that was my understanding too 20:54:39 which I don't agree 20:55:09 tldr, saltstack inc cannot block interested maintainers from maintaining epel9 branches of the salt package 20:55:15 they seem to have the impression that because they are salt that they can control it how they want in epel 20:55:23 No, to me it sounds like he can package it, just not use "their" package as a template. 20:55:43 but thats not true either... fedora specs are not nonfree 20:55:49 In other words, if they are controlling the Fedora package, make sure you don't sync from Fedora. 20:56:24 I know for a fact that they are re-using that spec in their internal builds due to all the errors 20:56:34 nirik: yeah, it's MIT license by default, no? 20:56:38 yep 20:56:51 unless they indicate in the spec that it's under a different license, but even then it has to be FLOSS 20:56:53 It seems they don't really understand how EPEL packaging works 20:56:57 they do not 20:57:06 no they don't 20:57:12 some of this misunderstanding is worrying and sounds like things that legal might have to weigh in on 20:57:26 a configuration management company not understanding how distros work is a bit.. worrying 20:57:27 Does anybody know these people? It sounds like this could benefit from an side channel conversation with them. 20:57:42 saltstack inc doesn't own the salt package, despite the current maintainers all being saltstack inc employees 20:57:53 yeah, it would be nice to try and nicely explain to them... 20:58:38 i know a guy that used to work there that knows these folks 20:58:51 I had a couple of friends that worked there until the vmware buyout 20:59:06 Also, it might be a good idea to tell them not to store srpms in the distgit repo: https://src.fedoraproject.org/rpms/salt/tree/rawhide 20:59:18 Not really relevant, but it really bothered me 20:59:23 yeah, my clone is going super slow. ;( 20:59:31 that and the mockbuild results dir 20:59:42 Yup :( 20:59:49 what a trainwreck 21:00:20 i say we just ignore them and do the stalled thing 21:00:22 rcallicotte: Well, if/when you do epel9, you can show them how it's supposed to be done. ;) 21:00:29 I agree with carlwgeorge 21:00:40 okie dokie 21:00:41 that sounds bad, but in this case i think it's appropriate 21:00:51 pgreco: You said you had two things ... go for it. 21:00:57 yeah, quick things 21:01:19 I asked for 2 new packages for epel9, tlp (already in qa) and bonnie++ (still pending) 21:01:37 I'm gonna wait a few days and then file the stalled for all the pending ones 21:01:44 the other thing is the email to the devel list 21:01:58 we get emails about updates-testing packages for 7 and 8, but not for 9 21:02:00 I never realized tlp is in Fedora 21:02:13 Thanks! I was gonna need bonnie++ down the road. 21:02:20 I might have to give that a try (I have one laptop where powertop doesn't do well enough) 21:02:40 It is there, but I think there's some sort of conflict between that and power-profiles-daemon 21:02:41 yeap, tlp maintainer did it all in a few hours 21:02:50 pgreco: I've noticed that when ever I have a package submitted that updates-testing doesn't go out for it .... that would explain the missing epel9 emails. ;) 21:03:10 pgreco: huh, I thought I saw some in the past... if they aren;t going there we can figure out why. probibly releng or infra ticket. 21:03:19 tdawson: I also noticed you started working on fedpkg? 21:03:22 Just kidding ... sorta. 21:03:33 Or made some more progress... 21:03:52 gotmax[m]: Well, I started about a month ago, I just barely got throught the stalled packages. 21:04:07 gotmax[m]: Yep ... it's close. 21:04:21 nirik, they do come out, but not as often as the other two 21:04:35 and I don't think that's related to people not building (though it could be) 21:04:45 pgreco: No, they don't. 21:04:54 the last one I have for 9 is July 15 21:04:56 I looked at the code at one point, and it's a bit fragile. 21:05:11 Easy to get errors, which cause it to not be sent. 21:06:27 I can look into why at some point, but right now, I have to leave for an appointment... file a ticket if you want me to remember to look at it. ;) 21:06:38 ack 21:06:56 tdawson: you seem to have a bit more info, do you want to file it or should I? 21:07:17 pgreco: I'll ping you on the #epel channel ... we're over time here. 21:07:22 Sorry, just noticed the time 21:07:33 Thank you all for coming and for the good discussions we've had. 21:07:39 Talk to you all next week if not sooner. 21:07:47 #endmeeting