2025-01-07 17:01:08 <@zbyszek:fedora.im> !startmeeting FESCO (2025-01-07)

2025-01-07 17:01:09 <@meetbot:fedora.im> Meeting started at 2025-01-07 17:01:08 UTC
2025-01-07 17:01:10 <@meetbot:fedora.im> The Meeting name is 'FESCO (2025-01-07)'
2025-01-07 17:01:13 <@zbyszek:fedora.im> !meetingname fesco
2025-01-07 17:01:13 <@meetbot:fedora.im> The Meeting Name is now fesco
2025-01-07 17:01:17 <@zbyszek:fedora.im> Chairs: @conan_kudo:matrix.org, @ngompa:fedora.im, @nirik:matrix.scrye.com, @humaton:fedora.im, @zbyszek:fedora.im, @sgallagh:fedora.im, @fale:fale.io, @dcantrell:fedora.im, @decathorpe:fedora.im, @salimma:fedora.im
2025-01-07 17:01:24 <@zbyszek:fedora.im> !topic Init Process
2025-01-07 17:01:37 <@nirik:matrix.scrye.com> morning
2025-01-07 17:01:40 <@conan_kudo:matrix.org> !hi
2025-01-07 17:01:40 <@zbyszek:fedora.im> !hi
2025-01-07 17:01:42 <@dcantrell:fedora.im> !hi
2025-01-07 17:01:43 <@zodbot:fedora.im> Zbigniew Jędrzejewski-Szmek (zbyszek)
2025-01-07 17:01:43 <@zodbot:fedora.im> Neal Gompa (ngompa) - he / him / his
2025-01-07 17:01:44 <@zodbot:fedora.im> David Cantrell (dcantrell) - he / him / his
2025-01-07 17:01:51 <@humaton:fedora.im> !hi
2025-01-07 17:01:52 <@zodbot:fedora.im> Tomáš Hrčka (humaton) - he / him / his
2025-01-07 17:02:05 <@zbyszek:fedora.im> We even have quorum.
2025-01-07 17:02:45 <@zbyszek:fedora.im> OK, I think we can start the discussion.
2025-01-07 17:02:51 <@zbyszek:fedora.im> If people join, they can review the log.
2025-01-07 17:02:56 <@zbyszek:fedora.im> !topic #3304 Change: Intel SGX Software Stack
2025-01-07 17:02:56 <@zbyszek:fedora.im> !fesco 3304
2025-01-07 17:03:23 <@zbyszek:fedora.im> Hmm, so the multiline paste is not supported by zodbot again?
2025-01-07 17:03:28 <@zbyszek:fedora.im> !fesco 3304
2025-01-07 17:03:29 <@zodbot:fedora.im> ● **Opened:** 3 weeks ago by amoloney
2025-01-07 17:03:29 <@zodbot:fedora.im> **fesco #3304** (https://pagure.io/fesco/issue/3304):**Change: Intel SGX Software Stack**
2025-01-07 17:03:29 <@zodbot:fedora.im> 
2025-01-07 17:03:29 <@zodbot:fedora.im> ● **Last Updated:** 55 minutes ago
2025-01-07 17:03:29 <@zodbot:fedora.im> ● **Assignee:** berrange
2025-01-07 17:03:34 <@salimma:fedora.im> !hi
2025-01-07 17:03:37 <@zodbot:fedora.im> Michel Lind (salimma) - he / him / his
2025-01-07 17:03:39 <@nirik:matrix.scrye.com> you can't do topic and more lines I think is the bug
2025-01-07 17:03:57 <@berrange:matrix.org> !hi
2025-01-07 17:03:59 <@zodbot:fedora.im> Daniel Berrangé (berrange)
2025-01-07 17:04:09 <@sgallagh:fedora.im> !hi
2025-01-07 17:04:10 <@zodbot:fedora.im> Stephen Gallagher (sgallagh) - he / him / his
2025-01-07 17:04:15 <@zbyszek:fedora.im> danpb: Thanks for joining on the short notice.
2025-01-07 17:04:53 <@zbyszek:fedora.im> We had two +1 votes in the ticket.
2025-01-07 17:05:00 <@zbyszek:fedora.im> Folks who had doubts, please speak up.
2025-01-07 17:05:14 <@nirik:matrix.scrye.com> I just somehow missed this one, I'm +1.
2025-01-07 17:05:19 <@sgallagh:fedora.im> Hi danpb, thanks for coming.
2025-01-07 17:06:07 <@sgallagh:fedora.im> I know in the past, there was a similar feature that we denied because we had no realistic way of testing it, because all testing required access to the vendor's keys. If that's not the case here, I have no remaining issues.
2025-01-07 17:06:57 <@berrange:matrix.org> not sure what you mean by access to the vendor's keys, but everything in my linked copr repo can be tested by any Fedora user with suitable HW
2025-01-07 17:07:38 <@sgallagh:fedora.im> I mean that it wasn't possible to run our compiled software at all if it hadn't been signed by the hardware vendor
2025-01-07 17:07:55 <@sgallagh:fedora.im> There was no "insecure mode" allowing us to test things without a round-trip to the vendor.
2025-01-07 17:08:57 <@berrange:matrix.org> well this proposal is primarily requesting to treat SGX architectural enclaves as firmware, so we wouldn't be compiling them, unless going for the academic exercise of proving reproducibility
2025-01-07 17:11:30 <@conan_kudo:matrix.org> I don't think most folks in Fedora would consider that a reasonable request
2025-01-07 17:11:32 <@sgallagh:fedora.im> From our perspective, would it look any different (or require special consideration) to install an OS in such an enclave?
2025-01-07 17:12:05 <@conan_kudo:matrix.org> Nitro enclaves don't require this kind of weirdness, for example
2025-01-07 17:12:18 <@sgallagh:fedora.im> Conan Kudo: Please refrain from putting words in others' mouths.
2025-01-07 17:12:22 <@berrange:matrix.org> an enclave is not something you install an OS into - it is a execution context for  small bits of code - typically individual functions
2025-01-07 17:13:06 <@conan_kudo:matrix.org> that's not true
2025-01-07 17:13:07 <@berrange:matrix.org> also note we're not talking about all enclaves - *exclusively* the architectural enclaves, which are the low level part that establishes the root of trust to the HW
2025-01-07 17:13:14 <@conan_kudo:matrix.org> an enclave is a secondary computer that can do _anything_
2025-01-07 17:13:18 <@sgallagh:fedora.im> OK, I'll admit I have a very poor understanding of how this works. (Clearly)
2025-01-07 17:13:32 <@berrange:matrix.org> the architectural enclaves are conceptually "firmware" for SGX
2025-01-07 17:13:36 <@conan_kudo:matrix.org> it's purely limited by how much you can store there
2025-01-07 17:13:58 <@salimma:fedora.im> For comparison - how big a footprint are we talking about, and how does it compare to say nvidia’s “firmware” for its open source drivers?
2025-01-07 17:14:08 <@conan_kudo:matrix.org> and enclaves do not have to be small, that's why we're having this discussion about SGX in the first place
2025-01-07 17:14:47 <@salimma:fedora.im> SGX will still be more open than that anyway, as @danpb noted we can rebuild to prove reproducibility if we need
2025-01-07 17:14:50 <@berrange:matrix.org> a enclave still runs in ring-3, so can't do "anything" - and communication with the enclave needs a special call mechanism initiated from OS code, not the reverse
2025-01-07 17:15:35 <@salimma:fedora.im> (I’m already +1 but a size comparison might help assuage concerns or put this in context)
2025-01-07 17:15:37 <@berrange:matrix.org> Conan Kudo: again, we're not talking about everything that *application* enclaves could do - i'm only proposing that the *architectural* encalves are treated as firmware
2025-01-07 17:16:07 <@conan_kudo:matrix.org> but can we fix anything? we are ultimately responsible for the code we deliver, the reproducibility argument is academic only because even if we find a flaw, we can't do anything about it
2025-01-07 17:16:28 <@salimma:fedora.im> That’s the same with firmware issues though
2025-01-07 17:16:34 <@berrange:matrix.org> that's true of all firmware we ship
2025-01-07 17:16:42 <@salimma:fedora.im> We can fix by rolling back a botched version 
2025-01-07 17:16:53 <@zbyszek:fedora.im> sgx-enclave-prebuilt-pce-signed-2.25-1.fc42.x86_64.rpm	2024-Nov-15 17:48:04	933.04K 	RPM File
2025-01-07 17:16:53 <@zbyszek:fedora.im> sgx-enclave-prebuilt-tdqe-signed-2.25-1.fc42.x86_64.rpm	2024-Nov-15 17:48:02	880.94K 	RPM File
2025-01-07 17:16:53 <@zbyszek:fedora.im> sgx-enclave-prebuilt-qe3-signed-2.25-1.fc42.x86_64.rpm	2024-Nov-15 17:48:06	892.32K 	RPM File
2025-01-07 17:16:53 <@zbyszek:fedora.im> sgx-enclave-prebuilt-ide-signed-2.25-1.fc42.x86_64.rpm	2024-Nov-15 17:48:05	90.33K 	RPM File
2025-01-07 17:17:09 <@zbyszek:fedora.im> (This is from the copr.)
2025-01-07 17:17:36 <@conan_kudo:matrix.org> we've never accepted shipping pre-built things that we can actually build though
2025-01-07 17:17:41 <@conan_kudo:matrix.org> not even for firmware
2025-01-07 17:17:47 <@salimma:fedora.im> Thanks, I’m still on my phone (running late)
2025-01-07 17:18:18 <@salimma:fedora.im> If we can build but can’t run that seems functionally equivalent as not being able to build
2025-01-07 17:18:21 <@berrange:matrix.org> most firmware will still require a vendor signature to be trusted by the HW on load, so we'd be in the same scenario there
2025-01-07 17:18:26 <@salimma:fedora.im> It’s like ftbfs vs fro
2025-01-07 17:18:45 <@conan_kudo:matrix.org> not really, we could easily attach a signature back for an artifact we build
2025-01-07 17:18:57 <@zbyszek:fedora.im> Because of the signing, for all intents and purposes, sgx enclave code behaves like firmware. 
2025-01-07 17:19:01 <@fale:fale.io> I think one of the question is what "we can actually build" means in this context. Does the loose of a feature (due to the signing outside the Intel certs chain) make it rebuildable or not?
2025-01-07 17:19:15 <@conan_kudo:matrix.org> the fact we don't do that for secure boot artifacts is a quirk of our own infrastructure, it's not impossible
2025-01-07 17:19:23 <@salimma:fedora.im> We don’t really want to adopt the maximalist FSF position here do we 🫠
2025-01-07 17:19:29 <@berrange:matrix.org> you can't attach a signature unless what we built was identical to the original firmware payload
2025-01-07 17:19:41 <@zbyszek:fedora.im> We can introspect it and do test builds, but to actually use it on real hardware, we need the exact binary blessed by upstream.
2025-01-07 17:20:41 <@salimma:fedora.im> Is that a decent compromise?
2025-01-07 17:20:41 <@salimma:fedora.im> Require a rebuild and gate the release of a new version to having an identical or small enough delta to the signed blob, once the signature is stripped
2025-01-07 17:20:41 <@salimma:fedora.im> 
2025-01-07 17:21:14 <@zbyszek:fedora.im> The Change page tries to answer that question…
2025-01-07 17:21:31 <@berrange:matrix.org> NB, new releases could involve new toolchain versions, so gating releases on rebuilds could delay issuing of CVE updates
2025-01-07 17:21:50 <@berrange:matrix.org> because the update could get blocked behind new package review process
2025-01-07 17:21:57 <@conan_kudo:matrix.org> then maybe Intel should bless _our_ builds
2025-01-07 17:22:28 <@sgallagh:fedora.im> Conan Kudo: That seems unrealistic.
2025-01-07 17:22:39 <@sgallagh:fedora.im> (Intel is having a hard time delivering its own stuff these daysP
2025-01-07 17:22:41 <@conan_kudo:matrix.org> it is only if we don't push for this
2025-01-07 17:22:43 <@sgallagh:fedora.im> (Intel is having a hard time delivering its own stuff these days)
2025-01-07 17:22:54 <@decathorpe:fedora.im> !hi
2025-01-07 17:22:55 <@zodbot:fedora.im> Fabio Valentini (decathorpe) - he / him / his
2025-01-07 17:23:02 <@decathorpe:fedora.im> (sorry, was stuck in a phone call)
2025-01-07 17:23:02 <@conan_kudo:matrix.org> improving the freedom of our systems comes only if we push for it
2025-01-07 17:23:05 <@berrange:matrix.org> fyi, i've spent over a year discussing this with intel and its not going to happen i'm afraid
2025-01-07 17:23:18 <@zbyszek:fedora.im> I don't think that's accurate. Compat packages don't require review. It'd be better to say that it might require a lot of packaging work to have the tools in the appropriate version. (As the Change page says.)
2025-01-07 17:23:26 <@conan_kudo:matrix.org> if we don't even try as a project, then we're not doing a good job for our community
2025-01-07 17:23:31 <@berrange:matrix.org> unless our build is 100% byte-for-byte identical to  the official build
2025-01-07 17:23:54 <@berrange:matrix.org> at which point "signing" our buld just involves sending us the original pre-built enclaves they already ship
2025-01-07 17:24:01 <@sgallagh:fedora.im> Conan Kudo: Effort we know is wasted isn't doing anyone any favors
2025-01-07 17:24:07 <@conan_kudo:matrix.org> can you get them to publicly tell us why they think only they should be offering SGX firmware?
2025-01-07 17:24:55 <@conan_kudo:matrix.org> that is not only uncalled for, but there are so many examples against that it isn't even funny
2025-01-07 17:24:58 <@sgallagh:fedora.im> Conan Kudo: It seems pretty self-explanatory: the whole purpose is to ensure that they're the sole source of trust and it can't be replaced by anyone in the middle
2025-01-07 17:25:04 <@conan_kudo:matrix.org> samba, nvidia, rdp, etc.
2025-01-07 17:25:33 <@berrange:matrix.org> Conan Kudo:  the goal of SGX is to allow software to establish that it is running on genuine Intel hardware and that requires a chain of trust back to Intel's root CA for SGX
2025-01-07 17:25:39 <@sgallagh:fedora.im> Conan Kudo: That's not wasted effort, because actual deliverables come from those efforts.
2025-01-07 17:25:43 <@salimma:fedora.im> I’m not sure how samba and rdp is relevant here
2025-01-07 17:26:02 <@salimma:fedora.im> Those are just reverse engineering and surely we can’t do that for sgx
2025-01-07 17:26:04 <@berrange:matrix.org> that eliminates the OS vendor specific signatures as a viable option
2025-01-07 17:26:11 <@conan_kudo:matrix.org> they are both closed standards that took a lot of pushback and effort to make open
2025-01-07 17:26:39 <@conan_kudo:matrix.org> both smb and rdp have been published standards with open implementations precisely because of pushback
2025-01-07 17:26:44 <@berrange:matrix.org> so the only other option would be for intel to verify and sign OS vendor's builds, like microsoft does for vendor's shim builds
2025-01-07 17:26:54 <@conan_kudo:matrix.org> that's what I want to see, yes
2025-01-07 17:27:35 <@berrange:matrix.org> and they don't consider it acceptable from a security POV to have a huge number of distinct signed builds  all built with slightly different toolchanis
2025-01-07 17:27:49 <@zbyszek:fedora.im> Realistically, I think that spending huge maintainer effort on this would be counterproductive. With all due respect for Dan's work, I don't expect more than a handful of enthusiasts to use this on Fedora. The proposed approach provides a reasonable benefits without creating a crazy amount of work for each package update. 
2025-01-07 17:28:11 <@conan_kudo:matrix.org> whatever
2025-01-07 17:28:33 <@conan_kudo:matrix.org> I think there's a security benefit in this, but we're going to disagree on this point
2025-01-07 17:28:57 <@conan_kudo:matrix.org> regardless, my vote is still -1, I'm backing out of this discussion because it's going in circles
2025-01-07 17:29:03 <@fale:fale.io> is this decision also going to potentially impact other softwares or is SGX-only?
2025-01-07 17:29:24 <@sgallagh:fedora.im> Shall we call a vote and see if we have agreement one way or the other at this point?
2025-01-07 17:29:41 <@fale:fale.io> (ie: will other different software be able to make the claim "it was ok for SGX, so why not for xyz"?)
2025-01-07 17:29:41 <@salimma:fedora.im> SGX only
2025-01-07 17:29:58 <@berrange:matrix.org> fyi, personally i would really like it if intel would sign OS vendor's builds, but having explored that option for over a year, i'm pretty certain it won't ever  happen
2025-01-07 17:30:10 <@salimma:fedora.im> But presumably others could try putting up similar proposals and this is sort of a precedent for evaluating those 
2025-01-07 17:30:27 <@salimma:fedora.im> Jinx
2025-01-07 17:30:30 <@berrange:matrix.org> Fale: anyone else would have to justify why their software meets the 'firmware exception' 
2025-01-07 17:30:48 <@conan_kudo:matrix.org> it's not hard to do if everyone keeps using the coprocessor argument
2025-01-07 17:30:51 <@salimma:fedora.im> Stepping out for 5 mins but I already voted 
2025-01-07 17:30:55 <@zbyszek:fedora.im> Yeah, they'd need to put a CPU model and a foundry first, so I wouldn't worry that much.
2025-01-07 17:30:57 <@sgallagh:fedora.im> Proposal: The Change is accepted.
2025-01-07 17:31:05 <@sgallagh:fedora.im> +1, based on this conversation
2025-01-07 17:31:09 <@dcantrell:fedora.im> +1
2025-01-07 17:31:12 <@conan_kudo:matrix.org> -1
2025-01-07 17:31:18 <@zbyszek:fedora.im> +1
2025-01-07 17:31:26 <@fale:fale.io> 0 = I'm not fully convinced on the ramifications
2025-01-07 17:31:34 <@nirik:matrix.scrye.com> +1
2025-01-07 17:31:36 <@decathorpe:fedora.im> 0
2025-01-07 17:31:45 <@humaton:fedora.im> 0
2025-01-07 17:32:12 <@sgallagh:fedora.im> Michel Lind 🎩 UTC-6 was +1 
2025-01-07 17:32:39 <@sgallagh:fedora.im> I count (+5, 3, -1).
2025-01-07 17:32:43 <@zbyszek:fedora.im> !agreed APPROVED (+5, 3, -1)
2025-01-07 17:33:20 <@zbyszek:fedora.im> danpb++ thanks for coming.
2025-01-07 17:33:22 <@zodbot:fedora.im> zbyszek gave a cookie to berrange. They now have 1 cookie, 1 of which was obtained in the Fedora 41 release cycle
2025-01-07 17:33:23 <@berrange:matrix.org> thank you all for the debates !
2025-01-07 17:33:30 <@zodbot:fedora.im> sgallagh gave a cookie to berrange. They now have 2 cookies, 2 of which were obtained in the Fedora 41 release cycle
2025-01-07 17:33:32 <@zbyszek:fedora.im> danpb: 
2025-01-07 17:33:48 <@sgallagh:fedora.im> danpb: Thanks for putting up with my ignorant questions :)
2025-01-07 17:34:17 <@zbyszek:fedora.im> !topic #3317 Change: Automated Packit Onboarding
2025-01-07 17:34:20 <@zbyszek:fedora.im> !fesco 3317
2025-01-07 17:34:21 <@zodbot:fedora.im> 
2025-01-07 17:34:21 <@zodbot:fedora.im> **fesco #3317** (https://pagure.io/fesco/issue/3317):**Change: Automated Packit Onboarding**
2025-01-07 17:34:21 <@zodbot:fedora.im> ● **Opened:** 23 hours ago by amoloney
2025-01-07 17:34:21 <@zodbot:fedora.im> ● **Assignee:** lbarczio
2025-01-07 17:34:21 <@zodbot:fedora.im> ● **Last Updated:** 2 hours ago
2025-01-07 17:34:47 <@zbyszek:fedora.im> I didn't have time to read the discussion…
2025-01-07 17:35:16 <@sgallagh:fedora.im> I feel like most of the opposition here boils down to misunderstanding what "opt-out" means in this context.
2025-01-07 17:35:25 <@zodbot:fedora.im> salimma gave a cookie to berrange. They now have 3 cookies, 3 of which were obtained in the Fedora 41 release cycle
2025-01-07 17:35:55 <@salimma:fedora.im> Even a PR being done by default would be just wrong for a subset of packages though
2025-01-07 17:36:03 <@salimma:fedora.im> Like Rust and golang
2025-01-07 17:36:08 <@dcantrell:fedora.im> I don't want to have to continually opt-out.  Like will PRs come every 6 months or so and I have to re-opt-out?
2025-01-07 17:36:11 <@sgallagh:fedora.im> Michel Lind 🎩 UTC-6: Sure, which is why it's a merge request and not an automatic push
2025-01-07 17:36:18 <@fale:fale.io> I'm +1 on this, as long as the text is very clear on potential issues and it is one PR in the lifetime of the package, and only applies to new packages
2025-01-07 17:36:24 <@sgallagh:fedora.im> dcantrell: As I understand it, it's only for new package additions
2025-01-07 17:36:32 <@decathorpe:fedora.im> not every packager is experienced enough to know that this is footgun-as-a-service
2025-01-07 17:36:42 <@salimma:fedora.im> Yeah but that would confuse new contributors to a space
2025-01-07 17:36:44 <@decathorpe:fedora.im> which is why I would like to have the PR creation opt-in
2025-01-07 17:36:45 <@conan_kudo:matrix.org> I think being done by default is going to be wrong for a long time to come
2025-01-07 17:36:48 <@salimma:fedora.im> As Fabio said
2025-01-07 17:37:01 <@conan_kudo:matrix.org> we are missing too many pieces to give packagers the opportunity to make informed choices from packit PRs
2025-01-07 17:37:10 <@salimma:fedora.im> I will be plus one if they make the fedpkg flag opt in
2025-01-07 17:37:12 <@sgallagh:fedora.im> As Fale says, I'd like to require that the MR description be very verbose about the possible risks
2025-01-07 17:37:16 <@nirik:matrix.scrye.com> see also: https://discussion.fedoraproject.org/t/f42-change-proposal-automated-onboarding-to-packit-release-automation-for-new-packages-system-wide/139530/25
2025-01-07 17:37:23 <@sgallagh:fedora.im> I think that's sufficient warning about the proverbial "foot-gun"
2025-01-07 17:37:43 <@salimma:fedora.im> And I also would like to see more options about what branches get pushed by packit, presumably as fedpkg options 
2025-01-07 17:37:44 <@zbyszek:fedora.im> I don't think this is a fair statement. A PR is just a PR. Maintainers need to evaluate PRs.
2025-01-07 17:37:50 <@conan_kudo:matrix.org> we lack reverse dependency checks, we lack reverse dependency test rebuilds, we lack compatibility verification
2025-01-07 17:38:04 <@conan_kudo:matrix.org> without these things, those PRs are just asking for trouble
2025-01-07 17:38:16 <@salimma:fedora.im> Also - license audit, esp when vendoring is involved 
2025-01-07 17:38:20 <@decathorpe:fedora.im> reverse installability check would be the minimum I would require for this to be enabled by default
2025-01-07 17:38:24 <@sgallagh:fedora.im> Agreed: if they blindly accept bad merge requests, that's something we should probably try to find out early.
2025-01-07 17:38:35 <@salimma:fedora.im> I’m a big packit fan and I only judiciously turn it on
2025-01-07 17:38:36 <@conan_kudo:matrix.org> that already happens now with opt-in packit
2025-01-07 17:38:43 <@conan_kudo:matrix.org> that was a complaint from both python and rust folks
2025-01-07 17:39:07 <@sgallagh:fedora.im> Conan Kudo: What, specifically?
2025-01-07 17:39:12 <@conan_kudo:matrix.org> we want to use packit with kde and we _can't_ because of this
2025-01-07 17:39:14 <@sgallagh:fedora.im> I'm not sure which comment you were replying to there
2025-01-07 17:39:38 <@conan_kudo:matrix.org> "blindly accept bad [pull] requests"
2025-01-07 17:40:06 <@conan_kudo:matrix.org> and some people even configure packit to automerge and build on rawhide
2025-01-07 17:40:49 <@decathorpe:fedora.im> hm, I thought auto-merge + auto-build was explicitly not possible?
2025-01-07 17:40:50 <@conan_kudo:matrix.org> we don't give people the tools to make the right calls with packit PRs
2025-01-07 17:41:15 <@salimma:fedora.im> Uh… example?
2025-01-07 17:41:24 <@salimma:fedora.im> Pretty sure merges are manual
2025-01-07 17:41:33 <@conan_kudo:matrix.org> I think python-dbusmock and anaconda are managed this way
2025-01-07 17:41:38 <@conan_kudo:matrix.org> same for cockpit too, IIRC
2025-01-07 17:41:43 <@salimma:fedora.im> Auto build was one of my concern, I would not want to see the default be anything but rawhide
2025-01-07 17:42:24 <@conan_kudo:matrix.org> the containers tooling stuff is like this too IIRC
2025-01-07 17:42:26 <@zbyszek:fedora.im> If there's a maintainer who doesn't understand what they are doing, that needs to change. Or they need to stop being a maintainer. Trying to prevent bad changes by some arbitrary limitations on how those changes are introduced doesn't seem helpful at all. 
2025-01-07 17:42:26 <@zbyszek:fedora.im> I find this discussion very strange. It's trivially easy to push a commit that doesn't work using a local workflow: just edit the version number, add sources, push, build. With a PR we at least get a scratch build and some zuul checks.
2025-01-07 17:42:26 <@zbyszek:fedora.im> 
2025-01-07 17:42:37 <@salimma:fedora.im> That sounds like custom automation though?
2025-01-07 17:42:46 <@salimma:fedora.im> I’d be ok with a policy banning that 
2025-01-07 17:42:55 <@conan_kudo:matrix.org> zbyszek: the problem is that if a robot does it, there is no one to teach
2025-01-07 17:42:58 <@salimma:fedora.im> A human must be responsible for code changes 
2025-01-07 17:43:09 <@conan_kudo:matrix.org> computers cannot be held accountable
2025-01-07 17:43:21 <@sgallagh:fedora.im> Michel Lind 🎩 UTC-6: I don't think that's enforceable
2025-01-07 17:43:31 <@sgallagh:fedora.im> Or practical, really
2025-01-07 17:44:09 <@conan_kudo:matrix.org> it has to be, since we have no protection for bad builds/pushes
2025-01-07 17:44:14 <@sgallagh:fedora.im> In the case of e.g. Cockpit, they have (for example) such a rigorous upstream commit review and testing policy that I'm perfectly happy trusting them to push changes automatically and build in Fedora.
2025-01-07 17:44:47 <@decathorpe:fedora.im> the problem with bot-created updates is when *things go wrong* though. nobody gets bodhi notifications, for example.
2025-01-07 17:44:48 <@salimma:fedora.im> I’m all for automation to help.  But a packager must at least either author or review surely?
2025-01-07 17:44:51 <@conan_kudo:matrix.org> and on the flipside, I have no trust for the containers projects because they do regularly break things without qualification
2025-01-07 17:45:01 <@fale:fale.io> I think there are three different issues: 1. are maintainer evaluating properly the PRs? 2. have people set up automation to auto-merge? 3. Should packit open a PR for every new package to "offer" its services? I think this ticket is only about number 3
2025-01-07 17:45:07 <@sgallagh:fedora.im> Michel Lind 🎩 UTC-6: A packager authored the automation...
2025-01-07 17:45:12 <@conan_kudo:matrix.org> and nobody knows until its too late because it's all done by bots
2025-01-07 17:45:21 <@zbyszek:fedora.im> We already allow pulls to be opened automatically (e.g. via packit) and used. The proposed change just makes it easier to have those pull requests opened. It's still up to the maintainers to a) accept the packit config, b) merge the prs. 
2025-01-07 17:45:26 <@salimma:fedora.im> That feels like too much indirection
2025-01-07 17:45:44 <@salimma:fedora.im> FWIW I’m ok granting exceptions to those who prove the processes are safe like cockpit
2025-01-07 17:45:47 <@salimma:fedora.im> But by default?
2025-01-07 17:45:56 <@conan_kudo:matrix.org> Fale: it's also about making fedpkg turn it on by default
2025-01-07 17:45:57 <@fale:fale.io> my understanding is that fedup should not block packit to open the PR by default
2025-01-07 17:46:08 <@conan_kudo:matrix.org> turning it on for everything now, and in the future on by default
2025-01-07 17:46:28 <@conan_kudo:matrix.org> with how things work now, it's a hard "no" from me
2025-01-07 17:46:39 <@salimma:fedora.im> Yeah, points 1 and 2 are relevant because of the opt out nature of the proposal
2025-01-07 17:46:52 <@salimma:fedora.im> We are thus discussing whether it’s even safe 
2025-01-07 17:47:01 <@sgallagh:fedora.im> All of the oppositional arguments sound like they're boiling down to "I don't trust our packagers to review the merge requests".
2025-01-07 17:47:13 <@sgallagh:fedora.im> Which... that seems like a MUCH bigger problem than this Change.
2025-01-07 17:47:14 <@zbyszek:fedora.im> No, not by default. The change page says:
2025-01-07 17:47:14 <@zbyszek:fedora.im> > Performing Koji builds and Bodhi updates automatically after merging the updates by the maintainer.
2025-01-07 17:47:25 <@zbyszek:fedora.im> I.e. the maintainer has to make the merge.
2025-01-07 17:47:27 <@conan_kudo:matrix.org> and I question whether it will ever become safe given recent announcements from the teams managing automation saying they have no time for Fedora
2025-01-07 17:47:33 <@salimma:fedora.im> it’s not that, it’s that for some SIG packages opt out would just be wrong by default
2025-01-07 17:47:41 <@decathorpe:fedora.im> I don't trust *all* packages to *always* know that they're doing. that's a different thing.
2025-01-07 17:47:59 <@decathorpe:fedora.im> I don't trust _all_ packages to _always_ know what they're doing. that's a different thing.
2025-01-07 17:48:22 <@decathorpe:fedora.im> (counting myself in that list, ftr)
2025-01-07 17:48:52 <@sgallagh:fedora.im> I think I missed those announcements. They'd be news to me, since my new role as of Jan 1st is to try to unify the automation in Fedora, CentOS Stream and Red Hat more so they can share effort.
2025-01-07 17:49:01 <@sgallagh:fedora.im> I think I missed those announcements. They'd be news to me, since my new role at Red Hat as of Jan 1st is to try to unify the automation in Fedora, CentOS Stream and Red Hat more so they can share effort.
2025-01-07 17:49:19 <@nhanlon:beeper.com> my opinion is I like the idea but gives me a lot of pause
2025-01-07 17:49:29 <@conan_kudo:matrix.org> there was an announcement from the OSCI team a while back saying that they will not be putting any efforts into Fedora and concentrating on CentOS/RHEL for the foreseeable future
2025-01-07 17:49:40 <@zbyszek:fedora.im> Which SIGs would like to opt out completely? Rust?
2025-01-07 17:49:57 <@conan_kudo:matrix.org> KDE would completely opt out
2025-01-07 17:50:03 <@conan_kudo:matrix.org> Packit is unsafe for the KDE stack
2025-01-07 17:50:17 <@salimma:fedora.im> And the problem is we can’t use globs to determine the opt out
2025-01-07 17:50:24 <@conan_kudo:matrix.org> not being revdep aware makes it dangerous
2025-01-07 17:50:24 <@sgallagh:fedora.im> Conan Kudo: That was overstated; that should have read closer to "we're going into maintenance mode until the new forge is in place".
2025-01-07 17:50:29 <@salimma:fedora.im> Not all Rust packages start with rust for example 
2025-01-07 17:50:38 <@decathorpe:fedora.im> yes - using packit without a custom config is unsafe for Rust packages (rust-*)
2025-01-07 17:50:39 <@zbyszek:fedora.im> OK, so should be ask for the default behaviour for `fedpkg` to be dependent on the package type? E.g. "no" for `rust-*`, `kde-*` or something like that?
2025-01-07 17:50:57 <@salimma:fedora.im> Well, if only it’s that simple
2025-01-07 17:51:07 <@salimma:fedora.im> Why don’t ask them to make this opt in for at least one release
2025-01-07 17:51:17 <@sgallagh:fedora.im> Well, we could definitely conditionalize some *known* cases easily enough
2025-01-07 17:51:19 <@salimma:fedora.im> What’s the rush?
2025-01-07 17:51:28 <@nirik:matrix.scrye.com> what would 'opt in' look like that it's not currently?
2025-01-07 17:51:40 <@sgallagh:fedora.im> But yeah, given the pushback, I suppose I can be on board with asking them to take another Fedora cycle to polish the idea
2025-01-07 17:51:43 <@decathorpe:fedora.im> (I'm hoping that with dynamically generated specparts that restriction can be lifted for most Rust packages in the future, but that's ... ***far*** future, given that our spec files need to support RHEL 9)
2025-01-07 17:51:46 <@sgallagh:fedora.im> And ask again then
2025-01-07 17:51:46 <@conan_kudo:matrix.org> I would have been fine if it was just adding a flag to `fedpkg request-repo` that allows initializing a repo with packit enabled
2025-01-07 17:51:50 <@salimma:fedora.im> Have fedpkg request repo has a flag or several to also configure packit 
2025-01-07 17:52:16 <@conan_kudo:matrix.org> even to this day, we don't auto-onboard everything to koschei
2025-01-07 17:52:22 <@conan_kudo:matrix.org> and it's been 8 years
2025-01-07 17:52:28 <@salimma:fedora.im> (I’d say requesting packit builds for stable branches should be a second flag )
2025-01-07 17:52:30 <@sgallagh:fedora.im> Proposal: Ask that they switch to opt-in on fedpkg for Fedora 42 and revisit opt-out as an option for Fedora 43.
2025-01-07 17:52:38 <@fale:fale.io> +1
2025-01-07 17:52:44 <@zbyszek:fedora.im> Stephen Gallagher: +1
2025-01-07 17:52:51 <@conan_kudo:matrix.org> Stephen Gallagher: +1
2025-01-07 17:52:53 <@salimma:fedora.im> +1
2025-01-07 17:52:57 <@humaton:fedora.im> +1
2025-01-07 17:52:59 <@decathorpe:fedora.im> +1
2025-01-07 17:53:28 <@nirik:matrix.scrye.com> I suppose... +1
2025-01-07 17:53:50 <@dcantrell:fedora.im> +1
2025-01-07 17:54:59 <@zbyszek:fedora.im> "!agreed The proposal is approved with the modification that it's just opt-in for F42. For switch to opt-out can be proposed for F43."  OK?
2025-01-07 17:55:24 <@salimma:fedora.im> "For switch" -> Switching
2025-01-07 17:56:11 <@sgallagh:fedora.im> Proposed: !info FESCo also requests that the Packit team investigate conditionalizing for known problematic packages such as rust-* to be excluded from future opt-out.
2025-01-07 17:56:19 <@zbyszek:fedora.im> !agreed The proposal is approved with the modification that it's just opt-in for F42. Switching to opt-out can be proposed for F43 (+9, 0, 0)
2025-01-07 17:56:32 <@zbyszek:fedora.im> !info FESCo also requests that the Packit team investigate conditionalizing for known problematic packages such as rust-* to be excluded from future opt-out.
2025-01-07 17:56:46 <@zbyszek:fedora.im> Phew.
2025-01-07 17:56:52 <@zbyszek:fedora.im> !topic Meeting time
2025-01-07 17:57:10 <@zbyszek:fedora.im> Are we all OK with not changing the time?
2025-01-07 17:57:38 <@fale:fale.io> I'm ok. I would prefer a time with DST to have constant time through the year but it's ok even without DST 🙂
2025-01-07 17:58:30 <@sgallagh:fedora.im> DST changes depending on your locale, so that's impossible.
2025-01-07 17:59:06 <@fale:fale.io> Italy
2025-01-07 17:59:11 <@nirik:matrix.scrye.com> This time is okish. I'd prefer an hour later, but I know others don't, so sticking with this is fine.
2025-01-07 17:59:23 <@sgallagh:fedora.im> Fale: Out of curiosity, where are you located?
2025-01-07 17:59:48 <@zbyszek:fedora.im> !info FESCo meetings will reuse the same time slot
2025-01-07 17:59:55 <@salimma:fedora.im> for US-EU meetings not following daylight saving seems a decent compromise
2025-01-07 18:00:14 <@salimma:fedora.im> if we follow DST, which one, and also it makes the meetings either permanently late for EU or permanently early for US :)
2025-01-07 18:00:28 <@salimma:fedora.im> following UTC means half the year it's early for US and half the year it's late for EU :p
2025-01-07 18:00:51 <@zbyszek:fedora.im> OK, I think we're mostly in agreement that it's acceptable. Let's move on.
2025-01-07 18:00:55 <@zbyszek:fedora.im> !topic Next week's chair
2025-01-07 18:00:59 <@sgallagh:fedora.im> For West Coast US, yeah. Less terrible for those of us on the good coast ;-)
2025-01-07 18:01:20 <@sgallagh:fedora.im> I can run the meeting next week
2025-01-07 18:01:39 <@zbyszek:fedora.im> !action Stephen Gallagher will chair the next meeting
2025-01-07 18:01:43 <@zbyszek:fedora.im> !topic Open Floor
2025-01-07 18:01:49 <@zodbot:fedora.im> decathorpe has already given cookies to sgallagh during the F41 timeframe
2025-01-07 18:01:58 <@sgallagh:fedora.im> I have a topic
2025-01-07 18:02:09 <@zbyszek:fedora.im> !info The mass rebuild is supposed to start next Wednesday.
2025-01-07 18:02:13 <@zbyszek:fedora.im> Stephen Gallagher: go
2025-01-07 18:03:38 <@sgallagh:fedora.im> First, I'd like to formally apologize for my screw-up when writing the Devel announcement draft about the provenpackager issue. Naming him directly was bad form and I didn't consider it properly at the time.
2025-01-07 18:04:12 <@nirik:matrix.scrye.com> We are all (except not Fale to blame. ;(
2025-01-07 18:04:30 <@decathorpe:fedora.im> Yeah, that's not just on you though. We should have noticed this before it was sent out.
2025-01-07 18:05:03 <@nirik:matrix.scrye.com> I was going to ask if we should look at writing up a policy... or just wait until the council comes back.
2025-01-07 18:05:06 <@salimma:fedora.im> yeah... as nirik said. even those of us who did not think it might be a good idea did not catch it in time
2025-01-07 18:05:16 <@zbyszek:fedora.im> Did we send out the summary that was being prepared for the council?
2025-01-07 18:05:21 <@dcantrell:fedora.im> wait until the council is done investigating this issue
2025-01-07 18:05:35 <@sgallagh:fedora.im> Not that I have seen.
2025-01-07 18:05:45 <@zbyszek:fedora.im> Right, so we should do that.
2025-01-07 18:05:59 <@decathorpe:fedora.im> the HackMD doc has the current state, and I think I addressed all comments
2025-01-07 18:06:00 <@nirik:matrix.scrye.com> Note: the FPL is on the fesco list. So he should have anything mentioned there.
2025-01-07 18:06:07 <@sgallagh:fedora.im> That was going to be the rest of this topic: I realize that the Council has decided to weigh in here.
2025-01-07 18:06:40 <@sgallagh:fedora.im> I think I missed the HackMD doc (I went on PTO about an hour after the announcement got sent and only got back to a computer yesterday). I'll look for it in my email history.
2025-01-07 18:06:53 <@decathorpe:fedora.im> it's linked from the fesco ticket.
2025-01-07 18:06:57 <@sgallagh:fedora.im> OK, thanks
2025-01-07 18:08:02 <@zbyszek:fedora.im> So… let's say one final day for review and on Thursday we send it out to the council?
2025-01-07 18:08:09 <@sgallagh:fedora.im> I kind of want to draft an announcement reminding people of the expected behavior of a provenpackager, but I worry that doing so *right now* would come across as accusatory.
2025-01-07 18:08:30 <@dcantrell:fedora.im> please don't.  wait for the council to finish their investigation first
2025-01-07 18:08:30 <@zbyszek:fedora.im> I was thinking about an announcement like that too. I think we should do it.
2025-01-07 18:08:50 <@zbyszek:fedora.im> dcantrell: Why is that related?
2025-01-07 18:09:11 <@decathorpe:fedora.im> I'd rather not put fuel onto a fire that has thankfully died down to embers over the holidays
2025-01-07 18:09:12 <@nirik:matrix.scrye.com> There is already some more discrussion today on pp stuff
2025-01-07 18:09:20 <@sgallagh:fedora.im> I'll look at it this afternoon.
2025-01-07 18:09:24 <@dcantrell:fedora.im> anything we send now to the community related to this incident before the council has weighed in will just restart the mega thread from December
2025-01-07 18:09:26 <@dcantrell:fedora.im> we don't need to do that
2025-01-07 18:10:31 <@zbyszek:fedora.im> Meh. OK.
2025-01-07 18:11:30 <@dcantrell:fedora.im> I'd also like to add here that I felt like the fallout from this fell on me.  I know that's not right, but it certainly felt that way because we dumped the announcement "with the trash" on a Friday and then everyone seemingly vanished.  I started getting direct messages asking what is going on and aside from posting the one message apologizing for FESCo's actions, I asked council to take over and do an investigation.
2025-01-07 18:11:44 <@dcantrell:fedora.im> TL;DR, this was not how I wanted to wrap up 2024  :)
2025-01-07 18:12:11 <@salimma:fedora.im> it felt heaviest on Josh I feel, as the initial messenger
2025-01-07 18:12:35 <@salimma:fedora.im> but yeah, sorry for any trouble that fell your way dcantrell 
2025-01-07 18:12:47 <@decathorpe:fedora.im> I think none of us did ...
2025-01-07 18:14:32 <@sgallagh:fedora.im> I think we can all agree that we did not handle this particularly well. We'll need to learn from it.
2025-01-07 18:14:55 <@nirik:matrix.scrye.com> Definitely.
2025-01-07 18:16:05 <@decathorpe:fedora.im> there are some docs that definitely need a refreshening, and it would probably be a good idea to put some more formalized process into place for removing people from pp / packager groups . right now it's only vaguely mentioned as "something that FESCo can make happen" 
2025-01-07 18:16:35 <@nirik:matrix.scrye.com> yes, sadly we need a policy.
2025-01-07 18:16:46 <@zbyszek:fedora.im> We also have https://pagure.io/fesco/fesco-docs/pull-request/94 unfinished
2025-01-07 18:17:24 <@zbyszek:fedora.im> Anyhow… any other topics?
2025-01-07 18:17:54 <@decathorpe:fedora.im> please no 😅
2025-01-07 18:17:55 <@zbyszek:fedora.im> If not, I'll end the meeting in a minute.
2025-01-07 18:18:15 <@zbyszek:fedora.im> After a short minute:
2025-01-07 18:18:16 <@zbyszek:fedora.im> !endmeeting