16:00:43 <sgallagh> #startmeeting ELN (2023-09-08) 16:00:43 <sgallagh> #meetingname eln 16:00:43 <zodbot> Meeting started Fri Sep 8 16:00:43 2023 UTC. 16:00:43 <zodbot> This meeting is logged and archived in a public location. 16:00:43 <zodbot> The chair is sgallagh. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 16:00:43 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:43 <zodbot> The meeting name has been set to 'eln_(2023-09-08)' 16:00:43 <zodbot> The meeting name has been set to 'eln' 16:00:43 <sgallagh> #topic Init Process 16:00:55 <michel-slm> .hello salimma 16:00:56 <zodbot> michel-slm: salimma 'Michel Lind' <michel@michel-slm.name> 16:01:00 <sgallagh> .hi 16:01:01 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com> 16:01:16 <neil> .hi 16:01:18 <zodbot> neil: neil 'Neil Hanlon' <neil@shrug.pw> 16:01:24 <neil> (brb, finishing making my coffee) 16:01:57 <tdawson> Hello ... I made it this week. 16:02:16 <yselkowitz> .hi 16:02:17 <zodbot> yselkowitz: yselkowitz 'Yaakov Selkowitz' <yselkowi@redhat.com> 16:03:03 <dcavalca> .hi 16:03:04 <zodbot> dcavalca: dcavalca 'Davide Cavalca' <davide@cavalca.name> 16:04:22 <sgallagh> Thank you for joining us this week as we hop into our time machine and explore the deepest reaches of the late 1900s... 16:04:47 <sgallagh> IRC! 16:05:34 <sgallagh> #topic Agenda 16:05:50 <sgallagh> #info Agenda Item: Status Update 16:06:09 * neil plays ephemeral music for the time travel sequence 16:06:10 <sgallagh> Does anyone have any specific topics they'd like to add to the agenda this week? 16:06:14 <tdawson> Woaa ... the future was so bright I had to wear shades. 16:06:39 * neil would like to 'formally' join the ELN cabal, if such things would be allowed 16:07:10 <sgallagh> There's one very specific thing you need to do in order to join up. 16:07:21 <sgallagh> Attend the meeting on IRC... so welcome! 16:07:28 <neil> woo! 16:07:33 <neil> easiest requirements ever 16:07:58 <michel-slm> neil is one of the cool kids now 16:08:01 <dcavalca> I have a specific question about why grub2-efi-x64-modules is missing from recent composes, and a general one on how one would can figure that out by themselves 16:08:07 * neil borrows tdawson's shades 16:08:18 <dcavalca> I checked CR as I thought that's where this would be defined but didn't find any smoking gun 16:08:21 <sgallagh> #info Agenda Item: grub2-efi issues 16:08:46 <yselkowitz> is this a result of the changes to the compose? 16:09:20 <dcavalca> maybe? I can tell you it was definitely there a while ago, but I can't pinpoint exactly when it disappeared because my internal monitoring was broken at the same time... 16:09:20 <sgallagh> Alright, do we want to start with that topic? 16:09:27 * sgallagh takes that as a "yes" 16:09:36 <sgallagh> #topic grub2-efi issues 16:10:24 <sgallagh> dcavalca: Do you mean that it shows up in CR but isn't appearing in the compose? 16:10:50 <dcavalca> I meant I didn't see any reference to it in recent commits to CR 16:10:58 <dcavalca> I didn't think to actually check the output of CR but that's a good idea, looking now 16:11:13 <yselkowitz> it's not in ELN 16:11:26 <dcavalca> yeah I don't see it in CR either 16:11:31 <yselkowitz> looks like it should be in ELN extras openstack workload? 16:12:06 <sgallagh> https://tiny.distro.builders/view-rpm--view-eln-extras--grub2-efi-x64-modules.html 16:12:20 <dcavalca> you're right, yeah 16:12:32 <dcavalca> so I take it this was excluded from ELN on purpose at some point? 16:12:33 <sgallagh> Which means it *should* be appearing in the Extras repo 16:12:44 <yselkowitz> but I don't even see a package by that name in koji 16:12:51 <yselkowitz> was it renamed? 16:12:52 <dcavalca> grub2 is the source package 16:12:52 <sgallagh> yselkowitz it's a subpackage 16:13:07 <yselkowitz> yeah I got that, but look: https://koji.fedoraproject.org/koji/buildinfo?buildID=2282236 16:13:12 <dcavalca> and yeah I see it https://odcs.fedoraproject.org/composes/production/latest-Fedora-ELN/compose/Extras/x86_64/os/Packages/grub2-efi-x64-modules-2.06-97.eln130.noarch.rpm 16:13:45 <dcavalca> yselkowitz: it's this one right? https://koji.fedoraproject.org/koji/rpminfo?rpmID=35794554 16:14:04 <yselkowitz> oh its noarch 16:14:56 <sgallagh> dcavalca: So what is the actual issue you are seeing? 16:15:13 <yselkowitz> the grub2.spec is not exactly easy reading... 16:15:13 <sgallagh> Just that it's not part of the BaseOS/AppStream repos anymore? 16:15:21 <sgallagh> That may have happened when I cleaned it up recently. 16:15:33 <sgallagh> Dropping entries that are no longer in Content Resolver for ELN 16:16:04 <sgallagh> Ye[ 16:16:05 <sgallagh> yep 16:16:10 <sgallagh> https://pagure.io/pungi-fedora/pull-request/1196#request_diff 16:16:12 <dcavalca> yeah, from my point of view, what I was trying to find out was whether this was intentional (i.e. this will be excluded from c10s too) or just a temporary glitch / ELN quirk 16:16:25 <dcavalca> ooh I forgot about pungi 16:16:27 <sgallagh> Looks intentional to me 16:16:31 <dcavalca> thanks, this explains it 16:16:41 <sgallagh> Well, Content Resolver is the official place 16:16:42 <yselkowitz> does it ship in rhel 9? 16:16:50 <sgallagh> I had to update Pungi to match CR 16:16:52 <neil> it does afaik 16:16:52 <dcavalca> yeah, it's in BaseOS 16:17:01 <dcavalca> well, on c9s, I don't have a RHEL 9 box to check 16:17:15 <neil> it's "required" if you want to be able to rebuild grub from source 16:17:24 <neil> (i mean, they're in the SRPM, but) 16:17:45 <neil> perhaps systemd-boot and UKI has something to do with it (?) 16:18:00 <neil> dcavalca: yeah it's in BaseOS on RHEL proper, too 16:18:16 <yselkowitz> confirmed, it ships in 9 16:18:43 <sgallagh> We can probably look into why it's been removed from the Content Resolver input 16:19:02 <sgallagh> But it definitely looks intentional 16:19:40 <yselkowitz> are you sure, I don't see any mention of it in git log -p 16:20:15 <sgallagh> Neither do I. It's possible it's an oversight. 16:20:23 <sgallagh> Maybe something used to Requires: it and stopped? 16:20:51 <yselkowitz> possible, that spec file is complicated to put it nicely 16:21:16 <neil> yeah--and there _was_ stuff moved around due to systemd-boot, iirc 16:22:11 <sgallagh> It might be worth checking with the bootloader folks if that's an intentional change or an oversight. 16:22:28 <neil> looks like `tboot` is the only thing that Requires it on rl9 16:22:30 <neil> el9* 16:23:03 <dcavalca> fwiw I can't actually tell if we're using this for anything at Meta, but we _are_ explicitly installing it (which is how I noticed it went away) 16:23:08 <sgallagh> tboot in ELN doesn't have that dependency 16:23:12 <sgallagh> So that might explain it 16:23:48 <yselkowitz> https://gitlab.com/redhat/centos-stream/rpms/tboot/-/commit/26f7e15a839997e9daf0c2a85393fdfa12a7650b 16:24:00 <yselkowitz> "- Added a Requires line to install grub2-efi-x64-modules" 16:25:10 * yselkowitz used to work with tcamuso, could ask him what's up with that 16:25:20 <sgallagh> Please do 16:25:42 <dcavalca> thanks, yeah knowing one way or another would definitely be helpful here 16:25:44 <sgallagh> It looks like tboot in Fedora is maintained by Intel folks, not Red Hat folks. 16:26:12 <sgallagh> #info There's a disconnect between CentOS Stream and ELN; tboot depends on grub2-efi-x64-modules in CS9 16:26:35 <sgallagh> #action yselkowitz will contact the maintainers to find out what should happen in ELN 16:27:24 <sgallagh> #info The grub2-efi-x64-modules package was removed from ELN because it is not present in the Content Resolver output. 16:27:36 <sgallagh> Anything else to add here? 16:28:17 <dcavalca> I think that's it, thanks 16:28:39 <sgallagh> Actually, let me write out a little more detail 16:28:58 <sgallagh> I'll give a little history about what I changed in the pungi config last week,. 16:29:57 <sgallagh> As you know, RHEL maintains several repositories for packages. The main ones are BaseOS, AppStream and CRB. There are a few others around, but we'll focus on these. 16:31:09 <sgallagh> Content Resolver doesn't currently provide any mechanism for annotating which packages belong in which repo. We have a hack that we use to shunt Extras into a comps group, but that's "special". 16:31:56 <sgallagh> So we have to make use of a pungi config document called `prepopulate.json` to do the split, by listing every binary package produced by each SRPM we are including in each repo. 16:32:30 <sgallagh33> I got disconnected/reconnected: what was the last you saw from me? 16:32:42 <dcavalca> > <sgallagh> So we have to make use of a pungi config document called `prepopulate.json` to do the split, by listing every binary package produced by each SRPM we are including in each repo. 16:32:55 <sgallagh33> (Yes, this is as ugly as it sounds) 16:33:47 <dcavalca> ugh, yeah it sounds like teaching CR to map packages to repos would make this a lot simpler in the long term 16:34:02 <sgallagh33> What we did initially was just copy the `prepopulate.json` from CentOS Stream 9 into ELN and then hand-edit it whenever things changed in a major way. 16:34:19 <neil> we sorta did something like that with how we manage repositories in Peridot--though that's created _from_ preopulate.json lol 16:34:52 <sgallagh33> Last week, people started noticing that some packages that were explicitly removed from ELN were still showing up in the compose, so I ran a script to strip out everything from `prepopulate.json` that is not listed in Content Resolver 16:35:06 <sgallagh33> (That resulted in the MR I linked earlier) 16:35:10 <neil> (Link for the curious :P https://git.rockylinux.org/rocky/peridot-rocky/-/blob/r9/autocatalog.cfg?expanded=true&viewer=simple ) 16:35:48 <sgallagh33> So that brings us to where we are today. 16:36:26 <sgallagh33> We have plans to rework the Content Resolver input format to allow us to annotate where things belong. 16:36:51 <dcavalca> sgallagh: you should consider writing a blog post or something explaining how the sausage is made 16:37:05 <sgallagh> bstinson is planning to present some design options to us in the next week or two 16:37:14 <dcavalca> there are a _lot_ of moving parts here, and it'd be useful to have a birds eye view of how everything fits together written down somewhere 16:37:29 <sgallagh> I will probably do that after we make this CR change 16:37:37 <dcavalca> (I meant you as global you, not necessarily you personally to be clear) 16:37:42 <sgallagh> Because it will change a lot of this information (and probably make it easier) 16:37:55 <dcavalca> makes sense, thanks 16:38:02 <neil> I think that'd be a really informational post, yeah 16:38:27 <sgallagh> Yeah, I need to do that. If only so that someone can take over if this lottery ticket I bought comes through... 16:39:03 <neil> i'd be happy to contribute (to the lottery or the post :) ). i am an okayish technical editor 16:40:01 <sgallagh> I will likely take you up on that. 16:40:05 <sgallagh> Thank you 16:40:57 <sgallagh> My ultimate goal is to have the Content Resolver input be the sole authority for what ends up in the compose. 16:41:23 <sgallagh> Right now, it's the list of what is *supposed* to be there. Which is close, but not quite the same thing 16:42:11 <sgallagh> (Which means that if you see a difference between CR and the compose, it is a bug) 16:42:38 <dcavalca> and ideally we can have automation to compare CR with the compose and file bugs when that happens 16:43:25 <sgallagh> I'd love to see that, but I can't promise anything in the near future. Too many moving parts and less than six months until CentOS Stream 10 kicks off. 16:43:39 <sgallagh> Contributions welcome, of course 16:44:19 <dcavalca> oh yeah, something for the long term for sure, and I wouldn't mind helping out if we get a chance 16:46:00 <sgallagh> Alright, I think we've covered this topic. 16:46:09 <sgallagh> #topic Status Update 16:46:13 <neil> same here. i think that'd be a welcome improvement :) 16:47:26 <sgallagh> I have a minor announcement: we're going to do the first pass of CentOS Stream 10 package import/sync early next week. 16:47:38 <dcavalca> yay! 16:47:48 <dcavalca> are we going to start making c10s composes as well? 16:47:51 <neil> woot! :) 16:48:04 <tdawson> dcavalca: Not yet 16:48:10 <sgallagh> This will involve a mini-mass-rebuild of ELN packages, so we'll be bumping the buildroot number to 132 and triggering a rebuild of most of the packages slated for the CS10 runtime. 16:48:31 <sgallagh> (There are some special cases that we need to exclude while we work out some issues with the kind folks at Gitlab) 16:49:17 <yselkowitz> my status: 3 packages currently FTBFS, still requires investigation. 5 packages still FTBFS in rawhide (down 1 from last time). today's dependency spike should be squashed by this evening, staying approximately level with the last few weeks. 16:49:32 <sgallagh> Composes will come later, but before the end of 2023. We haven't nailed down a firm date yet, but we'll be starting the gears turning 16:49:39 <sgallagh> right after the import completes 16:50:19 <dcavalca> that's excellent 16:50:23 <sgallagh> #info First packages will be synced to CentOS Stream 10 next week. Composes to come later. 16:50:40 <dcavalca> as soon as we have signed composes I'll start getting this bootstrapped internally 16:50:55 <sgallagh> #info Direct CentOS Stream development is still not going to start; all work should continue to happen on ELN until further notice. 16:51:13 <sgallagh> (Where "further notice" means "until Fedora 40 Final Freeze") 16:52:06 <sgallagh> dcavalca Signed composes probably won't happen until the official launch of CS10, which will be once the ELN sync ends 16:52:47 <sgallagh> I mean, the composes might have signatures, but I don't think the public keys will be published before then. 16:53:53 <dcavalca> if we can get the signing in place sooner than later, that'd make my life easier 16:54:10 <sgallagh> We'll have to talk about that some more. 16:54:17 <neil> ditto - would like to get a head start on CS10 in OpenStack-Ansible 16:54:19 <dcavalca> we basically would like to start bootstrapping c10s internally as soon as possible so we can provide ongoing feedback early on 16:54:28 <sgallagh> I'm slightly of the opinion that we don't want to sign anything we're unwilling to stand behind :) 16:56:05 <sgallagh> ELN at least is very clearly a moving development target. CentOS Stream carries a stronger stability promise. 16:56:09 <tdawson> Although it's good to get things in place ... up until the F40 split, this is going to go through many updates/changes ... at least for package version. 16:57:18 <dcavalca> yeah I think as long as we're clear on when Stream 10 is _actually_ released, having early composes out shouldn't be a problem 16:57:25 <dcavalca> we kinda did something similar with c9s and it worked pretty well IMO 16:57:48 <sgallagh> Yeah, I'm sure we can come up with something. Maybe we hand-deliver the public keys to people we know understand what they're getting into :) 16:58:20 <dcavalca> works for me :) 16:58:23 <sgallagh> I'll let folks like bstinson figure that one out. I just turn the crank 16:59:07 <sgallagh> Alright, we're pretty much at time. Any urgent last questions? If so, take them to #fedora-eln 16:59:21 <sgallagh> Thank you everyone for coming 16:59:28 <neil> 👍thanks sgallagh! 16:59:32 <neil> see y'all in a few weeks :) 16:59:47 <dcavalca> thanks! 16:59:50 <sgallagh> #endmeeting