2024-01-26 17:00:25 <@sgallagh:fedora.im> !startmeeting Fedora ELN (2024-01-26)
!meetingname eln
!topic Init Process
2024-01-26 17:00:26 <@meetbot:fedora.im> Meeting started at 2024-01-26 17:00:25 UTC
2024-01-26 17:00:26 <@meetbot:fedora.im> The Meeting name is 'Fedora ELN (2024-01-26) !meetingname eln !topic Init Process'
2024-01-26 17:00:40 <@sgallagh:fedora.im> !hi
2024-01-26 17:00:41 <@zodbot:fedora.im> Stephen Gallagher (sgallagh) - he / him / his
2024-01-26 17:00:53 <@tdawson:fedora.im> Hello
2024-01-26 17:00:54 <@salimma:fedora.im> !hi
2024-01-26 17:00:55 <@zodbot:fedora.im> Michel Lind (salimma) - he / him / his
2024-01-26 17:01:21 <@davide:cavalca.name> !hi
2024-01-26 17:01:23 <@zodbot:fedora.im> Davide Cavalca (dcavalca) - he / him / his
2024-01-26 17:02:24 <@nhanlon:beeper.com> !hi
2024-01-26 17:02:25 <@zodbot:fedora.im> Neil Hanlon (neil) - he / him / his
2024-01-26 17:02:27 <@nhanlon:beeper.com> g'day folks
2024-01-26 17:02:33 <@nhanlon:beeper.com> happy friday, etc etc
2024-01-26 17:02:39 <@sgallagh:fedora.im> Greetings my friends!
2024-01-26 17:02:47 <@sgallagh:fedora.im> Greetings, my friends!
2024-01-26 17:02:50 <@yselkowitz:fedora.im> !hi
2024-01-26 17:02:51 <@zodbot:fedora.im> Yaakov Selkowitz (yselkowitz)
2024-01-26 17:03:31 <@sgallagh:fedora.im> Alright, I think that's everyone that I expected to see here today, so let's get started!
2024-01-26 17:03:38 <@sgallagh:fedora.im> !topic Mass Rebuild Status
2024-01-26 17:04:03 <@sgallagh:fedora.im> So, this cycle's mass-rebuild has been... difficult
2024-01-26 17:04:54 <@sgallagh:fedora.im> You have probably seen the announcement that Fedora is delaying the start of the F40 Branch and all other dev milestones up to and including the start of the Beta Freeze
2024-01-26 17:05:29 <@sgallagh:fedora.im> We (ELN) were among those that requested this delay, because our mass-rebuild only triggers when the Fedora mass-rebuild is tagged back into Rawhide.
2024-01-26 17:05:33 <@salimma:fedora.im> is it mostly binutils/gcc or is there other difficulties?
2024-01-26 17:05:43 <@salimma:fedora.im> (not python like last cycle I hope)
2024-01-26 17:06:03 <@sgallagh:fedora.im> Our estimates were that there was not going to be sufficient time before the branching date to clean things up in Fedora before EL10 breaks inheritance from Rawhide.
2024-01-26 17:06:23 <@tdawson:fedora.im> There was some authentication issues
2024-01-26 17:06:31 <@salimma:fedora.im> yeah the auth lost us over 2 days :(
2024-01-26 17:06:37 <@sgallagh:fedora.im> There were issues with the GNU Toolchain as well as FAS
2024-01-26 17:06:43 <@salimma:fedora.im> do we have a list of packages we need to prioritize fixing in Fedora/ELN?
2024-01-26 17:06:52 <@yselkowitz:fedora.im> yes, https://github.com/fedora-eln/eln/issues/176
2024-01-26 17:07:00 <@salimma:fedora.im> awesome
2024-01-26 17:07:15 <@sgallagh:fedora.im> Michel Lind🎩: https://tiny.distro.builders/view--view-eln.html ? 😛
2024-01-26 17:07:17 <@salimma:fedora.im> the provenpackagers should be able to chip in. I'll try to prioritize this when fixing
2024-01-26 17:07:45 <@salimma:fedora.im> when is the deadline? I'll be free-er after CentOS Connect next Friday
2024-01-26 17:07:50 <@sgallagh:fedora.im> It would be fantastic to have some help with the cleanups, but we probably want to figure out how to coordinate so we don't end up doubling-up
2024-01-26 17:08:17 <@salimma:fedora.im> load the broken list on a spreadsheet, people put their names to the ones they are fixing?
2024-01-26 17:08:18 <@sgallagh:fedora.im> Michel Lind🎩: Fedora 40 Branching, rescheduled for Feb. 13
2024-01-26 17:08:26 <@salimma:fedora.im> feb 13 should work
2024-01-26 17:08:31 <@sgallagh:fedora.im> So really, Feb 12, since the Branch could occur at 00:00 Feb 13
2024-01-26 17:08:49 <@salimma:fedora.im> with fosdem travel it's probably easier to use the spare time to fix packages than doing focus work anyway
2024-01-26 17:08:53 <@salimma:fedora.im> ack, 12
2024-01-26 17:08:58 <@yselkowitz:fedora.im> which means a fix build through CI gating in rawhide before then
2024-01-26 17:08:58 <@tdawson:fedora.im> Note that the mass rebuild is not finished.  There are still 6500 packages to go ... and at the rate they are going ... that might take until Monday.
2024-01-26 17:09:25 <@sgallagh:fedora.im> Troy Dawson: I've heard rumblings that the failures may be resubmitted as well.
2024-01-26 17:09:41 <@salimma:fedora.im> yeah. I've been prioritizing fixing my packages that I know will not succeed in the mass rebuild this week, to help get them out of the way
2024-01-26 17:09:44 <@sgallagh:fedora.im> Since some internal compiler error issues have been found and fixed since.
2024-01-26 17:09:50 <@salimma:fedora.im> I've definitely seen some that got submitted twice
2024-01-26 17:09:58 <@tdawson:fedora.im> It seems they are doing them in batches of 40 or so ... possibly 50, but I've never seen the count that high.
2024-01-26 17:10:21 <@sgallagh:fedora.im> Michel Lind🎩: If they're early in the alphabet, they may have been resubmitted because of the binutils issue early on
2024-01-26 17:10:40 <@tdawson:fedora.im> Usually they just slam koji and let koji work it's way through them.
2024-01-26 17:10:45 <@salimma:fedora.im> Stephen Gallagher: in my case it's the folly stack. which has tight coupling between the packages :)
2024-01-26 17:10:45 <@smooge:fedora.im> that sounds right.. doing more than that will just end up with stuff really piling up due to the lack of equal build speeds
2024-01-26 17:11:19 <@sgallagh:fedora.im> Stephen J Smoogen: Did they change the approach this cycle?
2024-01-26 17:12:02 <@smooge:fedora.im> I think I read something in one of the channels but since my matrix doesn't search well and I don't have copies of on my system... assume my memory is bad
2024-01-26 17:12:26 <@smooge:fedora.im> I think they altered the batch size to try and deal with an issue but I am not sure.
2024-01-26 17:12:57 <@tdawson:fedora.im> Well ... whatever they did, they changed the time from 2 days to 7.
2024-01-26 17:13:15 <@smooge:fedora.im> well there are some other issues which have slowed things down
2024-01-26 17:13:26 <@nhanlon:beeper.com> I'm down to help with cleanup
2024-01-26 17:13:32 <@tdawson:fedora.im> On the flip side, as someone said above, it gives people a chance to fix they failed builds, as long as they are early in the alphabet.
2024-01-26 17:13:52 <@smooge:fedora.im> there are things which need to be done in a build which talk to bodhi and other parts of the build system. Anything which affects user lookups etc is going to cause slowdowns
2024-01-26 17:14:48 <@sgallagh:fedora.im> Is FAS still running off a single FreeIPA node, or did the replication get fixed?
2024-01-26 17:15:11 <@nhanlon:beeper.com> replication is fixed aiui
2024-01-26 17:16:21 <@salimma:fedora.im> Bodhi's new update page has been timing out all morning for me today (funnily viewing updates is working fine)
2024-01-26 17:16:22 <@sgallagh:fedora.im> OK, let's proceed assuming that the mass-rebuild will be finished sometime next week.
2024-01-26 17:16:40 <@salimma:fedora.im> yeah, how do we want to coordinate this? I think that's the main issue now
2024-01-26 17:16:48 <@sgallagh:fedora.im> I was just about to say that
2024-01-26 17:17:16 <@sgallagh:fedora.im> Trying to work from a shared spreadsheet is one approach, but it might just be better to work in the ticket
2024-01-26 17:17:34 <@nhanlon:beeper.com> https://framacalc.org/abc/en/ if we go the spreadsheet route
2024-01-26 17:18:06 <@salimma:fedora.im> <3 Frama and I have not tried this one yet
2024-01-26 17:18:08 <@sgallagh:fedora.im> As yselkowitz has been leading this effort, I'll defer to him for what he wants to do
2024-01-26 17:18:11 <@salimma:fedora.im> but ticket is fine
2024-01-26 17:18:34 <@yselkowitz:fedora.im> I don't want to have everyone need to sign up for a new service just to track this
2024-01-26 17:18:50 <@yselkowitz:fedora.im> a google spreadsheet works for me, or we use the github ticket
2024-01-26 17:19:23 <@salimma:fedora.im> there's nothing private about this so yeah either works 
2024-01-26 17:19:29 <@salimma:fedora.im> I'm sure Google already knows what I do ;)
2024-01-26 17:19:29 <@sgallagh:fedora.im> Michel Lind🎩: It won't be what they say so much as *how* they say it. Probably delivered by arrow.
2024-01-26 17:19:39 <@salimma:fedora.im> with a poisoned tip
2024-01-26 17:21:21 <@sgallagh:fedora.im> Any strong preferences?
2024-01-26 17:21:32 <@nhanlon:beeper.com> nope, whatever is least resistance 
2024-01-26 17:22:25 <@salimma:fedora.im> weak preference for spreadsheet
2024-01-26 17:22:33 <@sgallagh:fedora.im> OK, a spreadsheet might indeed be easiest, as I think on it. That way the name of who is working on it is closely connected to the package.
2024-01-26 17:22:33 <@salimma:fedora.im> scales better with a lot of packages
2024-01-26 17:22:38 <@sgallagh:fedora.im> Vs. comments in the ticket
2024-01-26 17:22:55 <@salimma:fedora.im> yeah and maybe have a status column so we can track how close it is (if someone is stuck I'm sure they could use help)
2024-01-26 17:23:17 <@sgallagh:fedora.im> One request: if you start working on something, but then need to abandon it, PLEASE make that statement loudly.
2024-01-26 17:23:18 <@nhanlon:beeper.com> framacalc fwiw doesn't seem to require registration, sorta just like an Etherpad or so
2024-01-26 17:23:31 <@nhanlon:beeper.com> and COLORS!
2024-01-26 17:24:07 <@sgallagh:fedora.im> Let's not try to get too fancy. I don't want people spending more than 15s updating the spreadsheet. There's plenty of better things to be doing.
2024-01-26 17:24:37 <@nhanlon:beeper.com> agreed, only mostly joking about the colors thing :P
2024-01-26 17:25:25 <@yselkowitz:fedora.im> well if it doesn't require signup then no objections to trying framacalc
2024-01-26 17:26:31 <@sgallagh:fedora.im> yselkowitz: Can I count on you to populate the list of packages requiring attention?
2024-01-26 17:26:49 <@yselkowitz:fedora.im> yes I already have that in csv form too
2024-01-26 17:27:06 <@sgallagh:fedora.im> yselkowitz++
2024-01-26 17:27:08 <@zodbot:fedora.im> sgallagh gave a cookie to yselkowitz. They now have 8 cookies, 2 of which were obtained in the Fedora 39 release cycle
2024-01-26 17:27:38 <@sgallagh:fedora.im> !action yselkowitz will populate a framacalc spreadsheet with the packages requiring attention post-mass-rebuild
2024-01-26 17:28:20 <@sgallagh:fedora.im> !info Any interested person should feel welcome to jump in and help, instructions on how will be posted to https://github.com/fedora-eln/eln/issues/176
2024-01-26 17:29:12 <@sgallagh:fedora.im> Let's move on to the branching discussion
2024-01-26 17:29:21 <@sgallagh:fedora.im> !topic EL10 Branching
2024-01-26 17:29:29 <@sgallagh:fedora.im> First, a quick status update.
2024-01-26 17:30:08 <@sgallagh:fedora.im> !info sgallagh has updated https://docs.fedoraproject.org/en-US/eln/branching/ with the operations required at branch-time
2024-01-26 17:30:37 <@sgallagh:fedora.im> This now covers both normal Fedora branching and the special events that take place when we branch a new EL release
2024-01-26 17:31:37 <@sgallagh:fedora.im> It covers the parts needed on the Fedora side. The CentOS Stream side will need its own document, but right now that's mostly internal at RH because all of the infrastructure access is behind the RH firewall
2024-01-26 17:31:44 <@salimma:fedora.im> well I'm happy to add color and validation if nobody else is doing it ;)
2024-01-26 17:31:45 <@sgallagh:fedora.im> That doc will be forthcoming
2024-01-26 17:32:16 <@sgallagh:fedora.im> Michel Lind🎩: The only validation we need is telling each other we're doing a great job :)
2024-01-26 17:32:58 <@sgallagh:fedora.im> Now, one thing I realized yesterday that we haven't talked about or, indeed, given enough thought to around EL10: EPEL
2024-01-26 17:33:43 <@sgallagh:fedora.im> We've had 35,000 foot plans in place to fork ELN-Extras into EPEL for RHEL 10 for a while, but it occurred to me that we haven't actually identified the process for how to do that.
2024-01-26 17:34:18 <@tdawson:fedora.im> Oh ... I thoguht that was something that EPEL would take care of.
2024-01-26 17:34:23 <@sgallagh:fedora.im> Obviously, what we decide will need to be coordinated with the EPEL SIG, so I'm going to politely ask Troy Dawson to be our liaison here :)
2024-01-26 17:34:46 <@tdawson:fedora.im> Right now we don't have epel10 branching ability ...
2024-01-26 17:34:57 <@sgallagh:fedora.im> I have no objections to the EPEL folks taking point on this, but I want to make sure we know what's happening in that space
2024-01-26 17:35:12 <@zodbot:fedora.im> salimma has already given cookies to sgallagh during the F39 timeframe
2024-01-26 17:35:17 <@tdawson:fedora.im> Meaning fedpkg branch-release epel10 will not work .... I think
2024-01-26 17:35:58 <@salimma:fedora.im> we have been talking about potentially allowing certain packages to be pre-branched for epel10 but haven't come to a consensus I think
2024-01-26 17:35:59 <@tdawson:fedora.im> Carl has a presentation on it at CentOS Connect ... we can ask him if he's got that in place yet.
2024-01-26 17:36:12 <@salimma:fedora.im> but yeah maybe limiting it to pacakges already in ELN Extras is a nice compromise
2024-01-26 17:36:29 <@salimma:fedora.im> the one we talked about is more wide ranging - if it's in EPEL X we branch it for X+1
2024-01-26 17:36:43 <@tdawson:fedora.im> Ya, I believe we said something along that lines.  If you want it pre-branched, but it in eln-extras
2024-01-26 17:36:50 <@salimma:fedora.im> thing is.. we don't need just the packages directly listed right? we need the transitive dependencies too that are not in the base EL
2024-01-26 17:36:51 <@sgallagh:fedora.im> Michel Lind🎩: This was always the stated intent of ELN-Extras.
2024-01-26 17:36:59 <@sgallagh:fedora.im> That it's basically proto-EPEL
2024-01-26 17:37:02 <@salimma:fedora.im> yeah
2024-01-26 17:37:26 <@salimma:fedora.im> as you said, it's been 30k feet high and we have not previously discussed the specific details of how to tie those two :)
2024-01-26 17:37:26 <@sgallagh:fedora.im> Right, and Content Resolver helps with that, fortunately.
2024-01-26 17:38:03 <@tdawson:fedora.im> The problem is that ELN branches before epel10 even has a buildroot to test with ... well ... maybe not, but I'm pretty sure it's not setup yet.
2024-01-26 17:38:05 <@salimma:fedora.im> does someone have a good writeup for that? e.g. 'given this workload definition, where can I see the packages it pulls in too'
2024-01-26 17:38:58 <@sgallagh:fedora.im> Troy Dawson: Ideally, we want EPEL to branch from ELN-Extras at the same time as CS10 from ELN
2024-01-26 17:39:17 <@sgallagh:fedora.im> Because otherwise, they may start picking up changes intended for F41
2024-01-26 17:39:50 <@yselkowitz:fedora.im> or we wait a bit and branch off of f40
2024-01-26 17:40:12 <@yselkowitz:fedora.im> since presumably it will take some time before epel10 can start being built?
2024-01-26 17:40:36 <@tdawson:fedora.im> Ya ... I'm pretty sure it's not going to be setup in two weeks ... but I do like yselkowitz  idea of branching from f40
2024-01-26 17:40:56 <@sgallagh:fedora.im> Actually, yeah. I hadn't thought about that, but you're right
2024-01-26 17:41:10 <@sgallagh:fedora.im> That's probably our best move
2024-01-26 17:42:04 <@salimma:fedora.im> when I bring in a new package - unless I'm sure it's safe I tend to branch to epel from the latest branched release, yeah
2024-01-26 17:43:52 <@sgallagh:fedora.im> OK, so for packages that don't have a separate `eln` branch, we can branch from F40
2024-01-26 17:44:11 <@tdawson:fedora.im> Sounds good to me.
2024-01-26 17:44:27 <@tdawson:fedora.im> I would have that nothing in -extras has an ELN branch.
2024-01-26 17:44:29 <@sgallagh:fedora.im> Those few that do have an `eln` branch might require some additional attention, but we can probably branch from there (or some point in its history)
2024-01-26 17:44:38 <@sgallagh:fedora.im> "There" being the `eln` branch
2024-01-26 17:44:40 <@tdawson:fedora.im> I would hope that nothing in -extras has an ELN branch.
2024-01-26 17:44:56 <@sgallagh:fedora.im> Oh, right. They wouldn't would they.
2024-01-26 17:45:04 <@sgallagh:fedora.im> Oh, right. They wouldn't, would they.
2024-01-26 17:45:10 <@sgallagh:fedora.im> Oh, right. They wouldn't, would they?
2024-01-26 17:45:34 <@sgallagh:fedora.im> Anyone opposed to that plan?
2024-01-26 17:45:40 <@salimma:fedora.im> seems solid
2024-01-26 17:46:20 <@tdawson:fedora.im> I'm ok with it.  I'll bring it up in the EPEL meetings.
2024-01-26 17:46:55 <@sgallagh:fedora.im> !agreed Packages in ELN-Extras will branch for EPEL 10 from the `f40` branch at some point after we branch EL10.
2024-01-26 17:47:10 <@sgallagh:fedora.im> Thank you, Troy Dawson 
2024-01-26 17:47:18 <@zodbot:fedora.im> sgallagh gave a cookie to tdawson. They now have 62 cookies, 6 of which were obtained in the Fedora 39 release cycle
2024-01-26 17:47:42 <@sgallagh:fedora.im> Does anyone else have a topic around the Branching?
2024-01-26 17:48:51 <@sgallagh:fedora.im> Alright, I'll take that as a no
2024-01-26 17:48:56 <@sgallagh:fedora.im> !topic Open Floor
2024-01-26 17:49:07 <@sgallagh:fedora.im> For the first time in a while, we have time for Open Floor!
2024-01-26 17:49:14 <@yselkowitz:fedora.im> F40FTBFS spreadsheet is ready: https://lite.framacalc.org/tgyaghuoob-a5pt
2024-01-26 17:49:33 <@salimma:fedora.im> woot
2024-01-26 17:49:43 <@salimma:fedora.im> I like how it defaults to Frencn
2024-01-26 17:49:47 <@salimma:fedora.im> I like how it defaults to French
2024-01-26 17:49:49 <@yselkowitz:fedora.im> I'm not entirely impressed with this framacalc, but we'll give it a try
2024-01-26 17:49:58 <@salimma:fedora.im> Allons enfants de la patriee.....
2024-01-26 17:49:59 <@yselkowitz:fedora.im> my French is too rusty unfortunately
2024-01-26 17:50:05 <@salimma:fedora.im> is it open to public? interesting
2024-01-26 17:50:37 <@salimma:fedora.im> bodhi is down today so might as well work on ELN :P
2024-01-26 17:51:05 <@salimma:fedora.im> I was just reciting the first line of the national anthem :D
2024-01-26 17:51:08 <@sgallagh:fedora.im> !link https://lite.framacalc.org/tgyaghuoob-a5pt
2024-01-26 17:53:52 <@sgallagh:fedora.im> I'll look into the libmodulemd issue with Petr.
2024-01-26 17:54:14 <@sgallagh:fedora.im> I haven't had a chance to review the patches in detail yet, but I've been following the discussion
2024-01-26 17:54:29 <@yselkowitz:fedora.im> was the nodejs20 ICE the same or different?
2024-01-26 17:54:47 <@sgallagh:fedora.im> Unclear yet; I'm going to try building it against the latest gcc and see if it just works
2024-01-26 17:55:04 <@sgallagh:fedora.im> Immediately after this meeting
2024-01-26 17:55:41 <@sgallagh:fedora.im> OK, five minutes left: any more topics?
2024-01-26 17:55:53 <@tdawson:fedora.im> Nothing from me
2024-01-26 17:56:37 <@salimma:fedora.im> not from me
2024-01-26 17:57:11 <@sgallagh:fedora.im> Alright, I'll close the meeting then. Thanks for coming!
2024-01-26 17:57:14 <@sgallagh:fedora.im> !endmeeting