16:00:58 <sgallagh> #startmeeting Fedora ELN SIG (2021-09-10)
16:00:58 <zodbot> Meeting started Fri Sep 10 16:00:58 2021 UTC.
16:00:58 <zodbot> This meeting is logged and archived in a public location.
16:00:58 <zodbot> The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:58 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:58 <zodbot> The meeting name has been set to 'fedora_eln_sig_(2021-09-10)'
16:01:05 <sgallagh> #meetingname eln
16:01:05 <zodbot> The meeting name has been set to 'eln'
16:01:18 <sgallagh> #topic Init Process
16:01:31 <sgallagh> Who is around today?
16:01:32 <sgallagh> .hello2
16:01:33 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
16:01:41 <tdawson> ./hello2
16:01:50 <sgallagh> Drop the slash
16:01:53 <tdawson> .hello2
16:01:54 <zodbot> tdawson: tdawson 'None' <tdawson@redhat.com>
16:02:09 <tdawson> I'll need to add a name sometime.
16:02:25 <sgallagh> Or just remain an international Man of Mystery
16:02:33 <tdawson> :)
16:02:57 <jforbes> .hello2
16:02:58 <zodbot> jforbes: jforbes 'Justin Forbes' <jforbes@redhat.com>
16:03:37 <sgallagh> jkonecny, bookwar dcavalca Are you around today?
16:05:53 <sgallagh> I'll wait just another minute, then get started
16:05:58 <tstellar> .hello2
16:05:58 <zodbot> tstellar: tstellar 'Tom Stellard' <tstellar@redhat.com>
16:07:09 <sgallagh> #topic Agenda
16:07:19 <sgallagh> I have three topics for this week:
16:07:40 <sgallagh> #info Agenda Item: s390x container images
16:07:56 <sgallagh> #info Agenda Item: DistroBuildSync updates
16:08:03 <sgallagh> #info Agenda Item: Content Resolver and ELN-Extras
16:08:10 <sgallagh> Does anyone else have something to put on the agenda?
16:08:19 <tdawson> Not me
16:08:22 <jforbes> Nothing else here
16:08:23 <tstellar> sgallagh: I would like to give an update on https://github.com/fedora-eln/eln/issues/40
16:08:47 <sgallagh> #topic Agenda Item: Status Update on Alternative Composes
16:09:02 <sgallagh> Why don't we take that one first, since it will likely have the most new content.
16:09:11 <sgallagh> #topic Status Update on Alternative Composes
16:09:18 <sgallagh> #chair tstellar
16:09:18 <zodbot> Current chairs: sgallagh tstellar
16:10:19 <Eighth_Doctor> .hello ngompa
16:10:20 <zodbot> Eighth_Doctor: ngompa 'Neal Gompa' <ngompa13@gmail.com>
16:10:25 <tstellar> OK, I haven't been able to get in touch with anyone doing odcs to ask about setting up composes for side-tags.  I think people are very busy.
16:11:11 <tstellar> In the past few months COPR has increased its builder capacity and I've been looking into using that instead.
16:11:47 <tstellar> Downside of copr is that there are less arches, but the upside is it's available, and easier for end users to interact with.
16:11:50 <Eighth_Doctor> it's a shame we can't build ISOs with copr
16:12:01 <Eighth_Doctor> or disk images
16:12:14 <sgallagh> tstellar: I think COPR now supports all arches that Fedora does
16:12:16 <sgallagh> Or am I mistaken?
16:12:16 <Eighth_Doctor> well technically, nobody can build install DVDs
16:12:29 <Eighth_Doctor> those can only be made with pungi :(
16:14:11 <tstellar> sgallagh: Oh you are actually right, I didn't notice that ppc64le support was back.
16:14:26 <tstellar> sgallagh: But s390x is still emulated on x86_64.
16:15:50 <tstellar> sgallagh: Anyway, the summary is that it seems to me that copr might be a better fit for what I'm trying to do than side-tags.  Anyone else have thoughts?
16:15:57 <sgallagh> tstellar: OK, so how would you like to proceed here?
16:16:17 <sgallagh> And what exactly is the problem with side-tags?
16:16:32 <sgallagh> In theory, we should be able to generate a compose from any tag at all
16:16:40 <sgallagh> The release tags are just convention
16:16:48 <Eighth_Doctor> if you're not building media, then ODCS is overkill
16:17:49 <sharkcz> there is a good chance there will be real hw for s390x in copr
16:18:10 * Eighth_Doctor snorts
16:18:42 <tstellar> sgallagh: I thought using ODCS was a requirement, and I wasn't able to get that set up, but maybe I'm wrong.
16:18:44 <Eighth_Doctor> someone has OpenStack on Z systems still?
16:19:11 <sgallagh> That actually ties into another topic on the agenda: s390x container images. I'll get to that in a bit
16:19:37 <tstellar> sgallagh: Who to proceed: I just am looking for advice on what approabh to pursue next.
16:19:42 <sgallagh> tstellar: Not a requirement, but I'd prefer if we used what we already have rather than inventing something new with a 99% overlap
16:19:56 <tstellar> (haven't been able to make a meeting all summer due to conflicts, sorry).
16:20:15 <sgallagh> No problem
16:21:00 <tstellar> sgallagh: OK, I'll need help getting in touch with ODCS team, then if we want to proceed with ODCS.  I haven't had success via email.
16:21:30 <sgallagh> OK, try reaching out to Jan Kaluza directly.
16:21:37 <sgallagh> (jkaluza on IRC)
16:21:53 <sgallagh> He'll either be able to help or will know who can
16:22:12 <sgallagh> For now, I guess we can move on?
16:22:17 <tstellar> sgallagh: Yes.
16:22:28 <sgallagh> #topic s390x Container Images
16:23:08 <sgallagh> I've been working on https://github.com/fedora-eln/eln/issues/16
16:23:32 <sgallagh> I've got container base images successfully generated for most arches, but we have a showstopper on s390x.
16:24:17 <sgallagh> In order to generate the image, we need to run one of the tools on the target hardware. However, we run into a problem with this.
16:24:41 <sgallagh> In ELN, we are building packages with compiler and linker flags that require a minimum platform version of z14.
16:25:06 <sgallagh> However, the image builders only have access to z13 systems.
16:25:25 <sgallagh> This works fine for Rawhide, which still uses the older baseline, but it just won't work on ELN
16:25:38 <jforbes> Oh, right
16:26:22 <jforbes> I don't think there is much of a work around available there.
16:27:03 <sgallagh> I'm hoping we can request an LPAR or two of the newer systems in the near future, but for now we are blocked
16:27:04 <Eighth_Doctor> why was the baseline raised then?
16:27:12 <Eighth_Doctor> if we don't have the resources to actually build for it
16:27:27 <jforbes> We can build packages with it just fine
16:27:47 <jforbes> But mostly because RHEL targets
16:27:50 <Eighth_Doctor> sure, but we can't run them, which means no images
16:28:23 <jforbes> Well, we can't on the Fedora infra.
16:28:37 <sgallagh> This is going to be an issue for EPEL 9 as well, so I'm reasonably confident we can get hardware access.
16:28:38 <sgallagh> I just don't have a timeline on that yet.
16:28:51 * Eighth_Doctor sighs
16:28:59 <Eighth_Doctor> okay, I hope we can get that soon
16:29:27 <Eighth_Doctor> given the progress on getting cs9 on the mirror network, I expect epel9-next to come very soon
16:29:58 <sgallagh> Anyway, that's all I had for this topic. It was mostly just a status update.
16:30:23 <jforbes> Well, soon is relative. I don't know what the timeline is for epel 9, I just nknow that it is being given more attention that previous epel
16:30:29 <sgallagh> I will probably turn off the s390x image generation for the time being so we can get back to `FINISHED` rather than `FINISHED_INCOMPLETE` composes.
16:30:42 <Eighth_Doctor> we're basically talking weeks at this point
16:31:15 <jforbes> Ahh, that's better than I figured, I expected early next year maybe
16:31:47 <tdawson> Early next year is what we like to tell people ... so they will be pleasantly surprised ... hopefully.
16:32:32 <Eighth_Doctor> I think the idea is to "launch" early next year
16:32:35 <Eighth_Doctor> but we want to begin populating things sooner rather than later
16:32:53 <Eighth_Doctor> at least KDE and Fedora Infra applications first
16:33:54 <Eighth_Doctor> that will give us the biggest stress of whether someone figured out CRB properly this time
16:34:18 <Eighth_Doctor> (I suspect no, so doing this earlier means we can actually have a chance of fixing it before GA)
16:36:20 <sgallagh> OK, do we want to move on?
16:36:26 <Eighth_Doctor> sure
16:36:30 <tdawson> yep
16:37:13 <sgallagh> #topic DistroBuildSync update
16:37:55 <sgallagh> The code is mostly reviewed and accepted. We're running a `--dry-run` test of it over this weekend to verify that it recognizes and would build everything that the current implementation does.
16:38:17 <sgallagh> After that, we'll likely merge it on Monday or Tuesday and we're aiming to get it into prod by the end of the week.
16:38:40 <jforbes> nice
16:38:42 <Eighth_Doctor> what does this change?
16:39:00 <tstellar> sgallagh: Do you have a link to more details about what this is?
16:39:33 <sgallagh> Conan Kudo: For existing CentOS Stream 9 -> RHEL 9, hopefully nothing.
16:39:58 <sgallagh> But it adds the implementation we need for better rebuilding of ELN.
16:40:10 <sgallagh> In particular, the ability to notice a mass-tag event and handle it better.
16:40:36 <tdawson> Or even a more simple side-tag event
16:40:48 <sgallagh> By "better" I mean "tagging the Rawhide builds into the ELN buildroot, then rebuilding all of them"
16:41:09 <sgallagh> Thanks tdawson: yes, it will deal with side-tags much more reliabl.
16:41:17 <sgallagh> s/reliabl/reliably/
16:42:22 <jforbes> Hmm, better is a loaded term there, but I think more simple or convenient would apply
16:42:22 * sgallagh looks for it.
16:42:23 <sgallagh> Getting it into prod next week is partially contingent upon getting some credentials from Fedora Infra to use for the rebuild user. I have a ticket open for that.
16:42:45 <sgallagh> https://pagure.io/fedora-infrastructure/issue/10200
16:43:18 <sgallagh> jforbes: "Far less prone to causing cascading failures" == "better" to me :)
16:43:27 <Eighth_Doctor> :)
16:44:14 <jforbes> Sure, that's the more convenient bit, though packages may build differently using rawhide as a buildroot than they would with an eln buildroot, meaning they would need to be built again for correctness
16:44:29 <jforbes> sorry, some packages, certainly not the majority
16:44:41 <sgallagh> True...
16:45:49 <sgallagh> I suspect that most of the packages that would risk that are already on the exclusion list for the rebuilds anyway.
16:45:54 <sgallagh> If not, once we identify them, on they go.
16:46:40 <sgallagh> Any other questions? If not, I'll move to Open Floor.
16:48:00 <sgallagh> #topic Open Floor
16:48:04 <jforbes> Just an update on AWS images. There has been some activity this week, though no real ETA yet. Basically, it hasn't been forgotten about, but isn't moving quickly.
16:48:09 <tdawson> What about the last topic of the agenda?
16:48:30 <tdawson> Content Resolver and ELN-Extras
16:48:45 <sgallagh> Oh, whoops
16:48:46 <sgallagh> You're right...
16:49:01 <sgallagh> #topic Content Resolver and ELN-Extras
16:49:02 <sgallagh> #chair tdawson
16:49:02 <zodbot> Current chairs: sgallagh tdawson tstellar
16:49:16 <sgallagh> Troy, would you be so kind as to take point here?
16:49:30 <tdawson> Sure thing, although I wasn't able to contact davide this week.
16:50:11 <tdawson> So, Content Resolver (which feeds the package list into the builder) is setup with an ELN Extras section. - https://tiny.distro.builders/view--view-eln-extras--x86_64.html
16:51:03 <tdawson> This view has workloads, just like the regular ELN section, but if a dependency of a package in the Extras workload is already in ELN, it isn't listed.
16:51:39 <tdawson> Currently, there is only two workloads.  KDE, and a test nedit workload.
16:52:35 <tdawson> If you look at the KDE workload, you can see what is required (what we have listed in the workload) and what extra runtime dependencies are needed for it to build in ELN - https://tiny.distro.builders/workload--eln_extras_kde--fedora-empty-base--repository-fedora-eln--x86_64.html
16:52:52 <sgallagh> I'll also note that, as requested at the previous meeting, there is now better documentation at https://github.com/minimization/content-resolver/blob/master/README.md
16:53:04 <Eighth_Doctor> this doesn't factor in weak deps, does it?
16:53:45 <tdawson> You will see on that workload page, that there is a link to the workload configuration - https://tiny.distro.builders/config-workload--eln_extras_kde.html ... which also gives a link to the github repo where you can edit and work on the workload - https://github.com/minimization/content-resolver-input/blob/master/configs/eln_extras_kde.yaml
16:53:46 <sgallagh> No, only strong dependencies. If you want something that would be weakly requested, you need to put it in your workload intentionally
16:54:09 <Eighth_Doctor> tdawson: can we get the weak deps added to the kde trace?
16:54:37 <tdawson> Sure
16:54:38 <Eighth_Doctor> most of our "weak deps" are not _really_ weak but are so because people complain
16:54:53 <Eighth_Doctor> crippled KDE is not what I'd like to see tested
16:54:57 <copperi[m]> 👍️
16:55:26 <tdawson> Yep.  But, beyond KDE, others can do pull requests to get their own workloads added, to see what extra things they will need.
16:55:31 <Eighth_Doctor> it'd be nice if there was a flag in the resolver to also include weak deps instead of making it something to populate
16:55:52 <Eighth_Doctor> yeah, I think something we probably want to do for fedora infra is get our applications into the resolver too
16:55:54 <sgallagh> Conan Kudo: File an RFE
16:55:57 <tdawson> No, I don't think that's a good idea.
16:55:57 <Eighth_Doctor> pagure, mailman, etc.
16:56:20 <tdawson> If something is a weak dependency, and you really want it, then list it.
16:56:31 <sgallagh> tdawson: I think it would be okay at a per-workload level
16:56:44 <sgallagh> But not at a per-view level
16:56:44 <Eighth_Doctor> right
16:57:02 <Eighth_Doctor> it's not like you can't indicate that they are weak deps in the graph anyway
16:57:24 <tdawson> A) it would be a pain in the rear, and B) what I said above.  How is someone going to know if you really want the package or not.
16:57:27 <Eighth_Doctor> but imo, it's unreasonable to be able to add every weak dep manually to track for DEs
16:57:45 <Eighth_Doctor> if it were up to me, nearly all of our weak deps would be strong ones in KDE
16:57:56 <tdawson> But, as sgallagh said, put it in as an issue / RFE.
16:58:09 <Eighth_Doctor> yep, where to? eln-sig/distrobuilder?
16:58:18 <sgallagh> Yeah, I don't think this meeting is necessarily the right place to litigate that.
16:58:18 <tdawson> Content Resolver
16:58:24 <sgallagh> https://github.com/minimization/content-resolver/
16:58:39 <tdawson> https://github.com/minimization/content-resolver
16:58:43 <tdawson> Yep.
16:59:12 <sgallagh> OK, we're coming up on the top of the hour, so does anyone have any other concerns they want to raise today?
16:59:13 <tdawson> The content resolver currently doesn't do buildroot dependencies for ELN Extras.  That is a feature we are currently working on.
17:00:47 <tdawson> Nothing more from me.
17:00:59 <Eighth_Doctor> https://github.com/minimization/content-resolver/issues/31
17:01:08 <Eighth_Doctor> RFE filed
17:01:09 <sgallagh> Thanks
17:01:15 <copperi[m]> Thanks sgallagh
17:01:19 <jforbes> nothing more from me
17:01:20 <sgallagh> #endmeeting