17:00:34 <sgallagh> #startmeeting FESCO (2023-02-28)
17:00:34 <zodbot> Meeting started Tue Feb 28 17:00:34 2023 UTC.
17:00:34 <zodbot> This meeting is logged and archived in a public location.
17:00:34 <zodbot> The chair is sgallagh. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
17:00:34 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:00:34 <zodbot> The meeting name has been set to 'fesco_(2023-02-28)'
17:00:34 <sgallagh> #meetingname fesco
17:00:34 <zodbot> The meeting name has been set to 'fesco'
17:00:34 <sgallagh> #chair nirik, decathorpe, zbyszek, sgallagh, mhroncok, dcantrell, music, mhayden, Conan_Kudo, Pharaoh_Atem, Son_Goku, King_InuYasha, Sir_Gallantmon, Eighth_Doctor
17:00:34 <zodbot> Current chairs: Conan_Kudo Eighth_Doctor King_InuYasha Pharaoh_Atem Sir_Gallantmon Son_Goku dcantrell decathorpe mhayden mhroncok music nirik sgallagh zbyszek
17:00:34 <sgallagh> #topic init process
17:00:37 <zbyszek> .hello2
17:00:39 <zodbot> zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' <zbyszek@in.waw.pl>
17:00:45 <mhayden> .hello2
17:00:48 <zodbot> mhayden: mhayden 'Major Hayden' <mhayden@redhat.com>
17:00:53 <decathorpe> .hi
17:00:53 <sgallagh> .hi
17:00:56 <zodbot> decathorpe: decathorpe 'Fabio Valentini' <decathorpe@gmail.com>
17:00:59 <nirik> morning
17:01:00 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
17:02:01 <salimma> .hi
17:02:03 <zodbot> salimma: salimma 'Michel Alexandre Salim' <michel@michel-slm.name>
17:02:45 <dcantrell> .hello2
17:02:46 <zodbot> dcantrell: dcantrell 'David Cantrell' <dcantrell@redhat.com>
17:02:47 <bcotton> .hello2
17:02:49 <zodbot> bcotton: bcotton 'Ben Cotton' <bcotton@redhat.com>
17:03:44 <sgallagh> OK, we have quorum. Let's get started
17:04:09 <sgallagh> #topic #2960 FESCo blocker bug: Popular third-party RPMs fail to install/update/remove in F38 due to security policies verification
17:04:09 <sgallagh> .fesco 2960
17:04:10 <zodbot> sgallagh: Issue #2960: FESCo blocker bug: Popular third-party RPMs fail to install/update/remove in F38 due to security policies verification - fesco - Pagure.io - https://pagure.io/fesco/issue/2960
17:04:31 * decathorpe ducks
17:04:50 <sgallagh> My reading of this ticket thus far is that we agree that the current situation is blocking for Beta, but we don't agree on what would constitute a valid solution to unblock it.
17:05:38 <decathorpe> In my opinion,l at least the packages and repositories that are shipped with Fedora Workstation OOTB need to work ...
17:06:05 <decathorpe> (that would probably be the google-chrome repo and RPM, I assume that RPMFusion is not affected)
17:06:05 <sgallagh> Define OOTB? Because there's a checkbox in gnome-firstboot to enable some of them
17:06:12 <nirik> Did we say beta?
17:06:27 <mhroncok> .hello churchyard
17:06:28 <zodbot> mhroncok: churchyard 'Miro Hrončok' <mhroncok@redhat.com>
17:06:34 <mhroncok> sorry for being late
17:06:36 <nirik> ah, we did
17:06:54 <salimma> ideally the ones we either enable by default or promote should work, right?
17:07:08 <nirik> I really really don't think we should remove things people installed.
17:07:10 <sgallagh> Michel Alexandre Salim 🎩: *Ideally* everything should work everywhere, all the time
17:07:52 <bcotton> this is definitely a "people will blame us, even if it's not our fault" situation
17:08:16 <nirik> So, would anyone be willing to reconsider the Beta blocker? If we made it Final instead that would be more incentive for fixing and make the problem more obvious to everyone?
17:08:18 <Eighth_Doctor> .hello ngompa
17:08:19 <zodbot> Eighth_Doctor: ngompa 'Neal Gompa' <ngompa13@gmail.com>
17:08:51 <sgallagh> nirik: A large enough subset of Fedora developers use Chrome that I think shipping Beta without fixing this would result in much lower testing
17:09:04 <zbyszek> nirik: I think postponing is not good. It just moves the can down the road a bit, but makes our Beta semi-useless for a large fraction of people.
17:09:17 <Eighth_Doctor> and a significant number of Fedora contributors actually need Chrome working
17:09:20 <mhroncok> there is a new proposal in the ticket
17:09:21 <nirik> well, there's workarounds... and Beta testers would be much more likely to do that than end users.
17:09:28 <zbyszek> Back when we had Alpha's, I'd say that this is OK for Alpha.
17:09:32 <mhroncok> by Workstation WG
17:10:00 * nirik goes to read. grumbles about people updating things right before meetings.
17:10:24 * salimma wonders if the MS repos are affected too, won't be surprised if they are
17:11:05 * zbyszek really likes the WorkstationWG's proposal (https://pagure.io/fesco/issue/2960#comment-843816)
17:11:25 <nirik> well, that sounds ok to me, but... I am not sure how feasable it is.
17:11:26 <zbyszek> Some details would need to be clarified though.
17:11:49 <mhayden> zbyszek: i like it too, but i wonder how much work it would be to go backwards
17:11:53 <zbyszek> I.e. I think I'd be fine with adding back SHA1 to the default policy.
17:11:57 <nirik> unless crypto-policies has come up with a per application way to override things we would have to drop the SHA1/DSA stuff globally.
17:12:05 <sgallagh> How hard would it be to get RPM/DNF to make a loud noise about the signature in a way that spurs action? Something like "WARNING! The software "foo" does not have a valid signature. It may have been compromised by a third-party. Please contact the maintainer of the repository."
17:12:44 <nirik> it also DSA I think?
17:12:50 <decathorpe> Should be possible. At least the new RPM GPG implementation should provide better error messages than the old one
17:13:16 <nirik> the old one didn't honor system wide crypto at all. ;)
17:13:31 <decathorpe> there's that, and the old one also just said "FAILED" ...
17:13:45 <decathorpe> but either way, this probably won't help anyone who doesn't use the dnf CLI
17:14:11 <zbyszek> Failed to import public key "https://dl.google.com/linux/linux_signing_key.pub" to rpmdb.
17:14:25 <sgallagh> Does GNOME-Software not have the ability to report errors back to the user?
17:14:25 <zbyszek> That's what is says right now. I'd say that is not terribly useful.
17:14:59 <decathorpe> zbyszek: that's a different component AFAIK
17:15:04 <mhroncok> I feel like this is a chicken-egg problem. Nobody fixes the chrome RPM if it continues to work, but we are afraid to break it.
17:15:09 <nirik> I'm not sure we should drive off into the implementation details. How about we just say it's a beta blocker and ask rpm/gnome-software/dnf/crypto-policy folks to work together to come up with a solution?
17:15:16 <dcantrell> mhroncok: +1
17:15:46 <sgallagh> nirik: Well, we have already been asked whether "roll back the SHA1/DSA change" is on the table.
17:15:56 <decathorpe> if we break it, there needs to be a well-documented escape hatch ... (i.e. in release notes, common bugs, release announcement blog posts, ... etc)
17:16:40 <sgallagh> And we've been asked whether it's okay to just automatically uninstall packages with bad signatures.
17:16:53 <sgallagh> These are definitely questions that we need to answer.
17:17:06 <nirik> well... I think it could be, but we don't know if it's needed yet do we? has anyone asked crypto-policy folks? are they even aware of this?
17:17:12 <zbyszek> So… in the ticket I wrote that I support uninstalling packages, but after reading the comments, I'm -1 on that now.
17:17:26 <dcantrell> I don't think automatic uninstall is acceptable.  Given the options, reverting seems to be the best option because this will knowingly break behavior for existing users
17:17:43 <nirik> I am -1 to automatic uninstall. I'm fine with 'I can't do anything until you remove this package or work around this issue'
17:17:48 <dcantrell> worth considering is RH is moving to Fedora for CSB, which we probably don't want to break third party RPMs on
17:18:09 <mhroncok> not sure what CSB is
17:18:18 <dcantrell> the IT-supported install of the OS
17:18:22 <dcantrell> previously was RHEL
17:18:23 <sgallagh> It's the OS they preinstall on the company laptops and support
17:18:27 <dcantrell> CSB==corporate standard build
17:18:42 <mhroncok> thanks for clarifying
17:19:08 <mhroncok> (I don't think it's much relevant for us here.)
17:19:26 <sgallagh> mhroncok: It's relevant insofar as it represents a large constituency that will be affected
17:19:28 <dcantrell> no, it's just a known user that will be relying on third party RPMs
17:20:01 <salimma> similar to WSB, we also preinstall Chrome at Meta
17:20:05 <nirik> I'd like to have the crypto policy folks weigh in... see if they can come up with a way to just exempt rpm...
17:20:07 <salimma> sorry, CSB
17:20:42 <mhroncok> I mean whoever creates CSB is probably capable of making their own adjustments
17:21:03 <mhroncok> it's more the users who don't have a customized Fedora we should worry about
17:21:17 <mhroncok> *customized build of Fedora Linux
17:21:39 <dcantrell> is it though?  I don't really consider CSB *that* customized beyond adding third party RPMs and some config files.  that's a full giant build of an OS
17:21:53 <dcantrell> I think it's in the same category as any random user wanting to install RPMs we don't provide
17:22:07 <sgallagh> Proposal: If third-party repo support is enabled, RPM needs to accept SHA-1 hashes for Fedora 38, ideally with a deprecation warning. FESCo strongly advises against allowing SHA-1 hashes elsewhere, but will accept that for F38 if it's the only achievable, timely solution.
17:22:46 <music[m]> .hello music
17:22:47 <zodbot> music[m]: music 'Benjamin Beasley' <code@musicinmybrain.net>
17:22:52 <decathorpe> Am I understanding things correctly that there's two things that break without SHA-1 signatures - RPM package signatures and repo signatures?
17:22:55 <nirik> I can be +1 to that I think.
17:22:57 <music[m]> Sorry I had to be a bit late again.
17:23:04 <zbyszek> sgallagh: a question: just setting update-crypto-policies DEFAULT:SHA1 is *not* enough to get the google chrome repo signature to be imported.
17:23:10 <nirik> Fabio Valentini: yes.
17:23:11 <salimma> we don't check repo signatures in Fedora yet right
17:23:21 <salimma> for Fedora repos anyway
17:23:23 <mhroncok> Stephen Gallagher: this part "If third-party repo support is enabled" -- what is the consequence of that if?
17:23:28 <sgallagh> zbyszek: I'm aware, but I am not specifying a solution, just a guideline for unblocking
17:23:30 <nirik> zbyszek: because it's DSA I think?
17:23:41 <zbyszek> Yes, it is DSA.
17:23:56 <nirik> (which is also blocked in the current policy)
17:23:56 <zbyszek> Do you know how to add DSA to the crypt policy?
17:24:11 <sgallagh> OK, amend my proposal to say SHA-1/DSA wherever it says SHA-1
17:24:16 <nirik> there's no seperate way currently you have to do LEGACY
17:24:28 <sgallagh> mhroncok: Sorry, can you rephrase?
17:24:28 <nirik> they could add a module for DSA like they did SHA1 tho I think
17:24:41 <decathorpe> just saying, there would also be the possibility of making RPM (tempirarily) not honor crypto policy ...
17:24:47 <zbyszek> So should we amend sgallagh's proposal to also include DSA?
17:24:51 <mhroncok> Stephen Gallagher: what does it mean "If third-party repo support is enabled"?
17:24:54 <sgallagh> I mostly mean if they tick the "allow third-party software" box in gnome-initial-setup
17:24:58 <mhroncok> is it a checkbox somewhere?
17:25:08 <sgallagh> And whatever the equivalent is for other DEs
17:25:14 <nirik> I am not sure thats possible. I know it wasn't back when gnutls wanted to except something...
17:25:15 <mhroncok> becasue third party repo support is always enabled
17:25:17 <zbyszek> Just allowing SHA1 doesn't even solve the most visible issue with google-chrome-stable.
17:25:28 <sgallagh> In GNOME, it is.
17:25:30 <decathorpe> (problem: that doesn't work everywhere, and it doesn't consider upgrades)
17:25:31 <Eighth_Doctor> how do we handle that for upgrades?
17:25:58 <Eighth_Doctor> we don't have any state for tracking that?
17:25:59 <sgallagh> If there isn't a common mechanism for that, I'll amend my proposal then:
17:25:59 <mhroncok> in other words: if users add the chrome repo manually, it's OK that it doesn't work?
17:26:06 <decathorpe> yeah, I don't think modifying crypto policy based on which repos are enabled is a good idea ...
17:26:31 <mhroncok> does the polocy auotmatically changes based on what repositories are enabled?
17:26:35 <mhroncok> *policy
17:26:52 <sgallagh> Proposal: RPM must accept SHA-1 hashes and DSA keys for Fedora 38, ideally with a deprecation warning. FESCo strongly advises against allowing these algorithms elsewhere, but will accept that for F38 if it's the only achievable, timely solution.
17:27:12 <mhroncok> thanks
17:27:14 <mhroncok> +1
17:27:16 <zbyszek> sgallagh: +1
17:27:18 <decathorpe> that sounds good to me. +1
17:27:35 <dcantrell> sgallagh: can we say the deprecation warning will note SHA-1/DSA support will be removed in F39?
17:27:39 <sgallagh> mhroncok: I wasn't trying to be prescriptive with that statement. Just that it MUST work if they are enabled. Not specifying that it didn't have to in the ELSE case.
17:27:45 <dcantrell> I mean disabled
17:27:49 <sgallagh> dcantrell: Yes
17:28:08 <dcantrell> sgallagh: in that case with the added text, I'm +1
17:28:18 <sgallagh> Proposal: RPM must accept SHA-1 hashes and DSA keys for Fedora 38, ideally with a deprecation warning that it will be disabled in F39. FESCo strongly advises against allowing these algorithms elsewhere, but will accept that for F38 if it's the only achievable, timely solution.
17:28:19 <nirik> +1 here
17:28:24 <mhroncok> Stephen Gallagher: I udnesratnd now what you meant, but it IMHO made the statement more complex than it needed to be
17:28:30 <nirik> what happens if google fixes it before f38 release? ;)
17:28:35 <decathorpe> still +1
17:28:44 <mhroncok> still +1
17:28:47 <decathorpe> nirik: let's plan for that once hell freezes over
17:28:47 <Eighth_Doctor> Stephen Gallagher: +1
17:28:49 <zbyszek> still +1
17:28:49 <sgallagh> nirik: We throw a *&*^ing party
17:28:59 <zbyszek> nirik: we cancel F38 and start over.
17:29:03 <music[m]> +1 for the latest proposal
17:29:12 <bcotton> do we have approval that the existing behavior is a Beta blocker?
17:29:33 <mhroncok> good question
17:29:47 <mhroncok> In think we do, but Stephen Gallagher's proposal does not say beta
17:29:50 <sgallagh> This presumes that it's a Beta blocker and provides the criteria for unblocking
17:29:56 <sgallagh> I'll rephrase (again)
17:30:01 <bcotton> that's implicit in what's being voted on now, imo, but i know adam would feel better if we were very clear on that
17:30:06 <zbyszek> Yeah.
17:30:19 <nirik> I still think it would be ok to make it final, but I'm the minority there it seems... so beta it is?
17:30:37 <Eighth_Doctor> people are going to do upgrades when the beta goes out
17:30:37 <sgallagh> Proposal: FESCo agrees to block Beta on this proposal. In order to unblock, RPM must accept SHA-1 hashes and DSA keys for Fedora 38, ideally with a deprecation warning that it will be disabled in F39. FESCo strongly advises against allowing these algorithms elsewhere, but will accept that for F38 if it's the only achievable, timely solution.
17:30:38 <Eighth_Doctor> we generally encourage it
17:30:59 <mhroncok> still +1
17:31:02 <nirik> sure.
17:31:05 <Eighth_Doctor> Stephen Gallagher: +1
17:31:05 <nirik> +1
17:31:05 <sgallagh> s/on/for/, s/proposal/issue/
17:31:09 <dcantrell> +1
17:31:18 <zbyszek> FWIW, I upgraded to F38 yesterday and it seemed very smooth.
17:31:19 <music[m]> +1 again
17:31:23 <zbyszek> +1 still
17:31:26 <decathorpe> +1
17:32:37 <sgallagh> mhayden: Vote?
17:33:23 <sgallagh> #agreed FESCo agrees to block Beta for this issue. In order to unblock, RPM must accept SHA-1 hashes and DSA keys for Fedora 38, ideally with a deprecation warning that it will be disabled in F39. FESCo strongly advises against allowing these algorithms elsewhere, but will accept that for F38 if it's the only achievable, timely solution. (+8, 0, -0)
17:33:27 <mhayden> +1 from me
17:33:46 <sgallagh> #topic #2951 Proposal: policy for resubmitting rejected proposals
17:33:46 <sgallagh> .fesco 2951
17:33:47 <zodbot> sgallagh: Issue #2951: Proposal: policy for resubmitting rejected proposals - fesco - Pagure.io - https://pagure.io/fesco/issue/2951
17:34:44 <nirik> Would more people support this if it was amended to say:
17:35:04 <sgallagh> So, my personal take on this is that we probably don't need a formal policy here. FESCo can decide whether to accept a late Change or not on a case-by-case basis.
17:35:08 <nirik> "Proposals that are formally rejected"
17:35:39 <nirik> ie, rejected in a meeting and meeting minutes sent out.
17:36:04 <bcotton> Stephen Gallagher: this isn't about lateness, it's about "rejected and then re-voted"
17:36:37 * bcotton has no objection to "formally rejected"
17:36:43 <sgallagh> We could just revise the process to say it must reach +5 or -5 for a decision to be official.
17:37:07 <sgallagh> Otherwise the vote proceeds to the ticket/next meeting.
17:37:11 <Eighth_Doctor> that would cause a lot of problems though, because we rarely get that many votes
17:37:31 <bcotton> this is more about transparency than outcome, imo
17:38:03 <decathorpe> nirik: I like "formally rejected + announced" better. the problem with just "rejected" is that we often reject proposal as-is, but approve right after with a slight modification or additional conditions.
17:38:47 <decathorpe> sending these things back through the entire process instead would be a massive waste of time
17:38:56 <nirik> yeah, we don't want this to apply to those as it would really slow the process down and add a bunch of red tape
17:39:07 <Eighth_Doctor> and keep in mind our process timelines are too tight as it is
17:39:28 <sgallagh> Transparency is nice, but I feel like it's over-engineering a solution to a rare problem.
17:39:28 <bcotton> fwiw, "formally rejected" captures the intent of my draft
17:39:32 <sgallagh> it == this proposal
17:39:41 <bcotton> i did not mean for it to include "not approved until you fix x,y,z" type scenarios
17:40:03 <zbyszek> bcotton: I still think that it comes down to people who didn't like the result of a particular vote convincing themselves that if the make the process more onerous the next vote will be better. I think this is mistaken, the results will be the same, just more slowly and with additional annoyance.
17:40:20 <Eighth_Doctor> considering this entire thing is my fault, I'd rather try to figure out how to eliminate the pressure problem we have for spring cycles
17:40:32 <Eighth_Doctor> because that's what drove that mess in the first place
17:40:45 <nirik> we could weaken it a bit more by making 'must' into 'should'...
17:40:50 <dcantrell> sgallagh: I don't think it's necessarily rare.  Or maybe it is, but when it does happen we don't have to panic late in the cycle to approve something that did not get approved initially.  This change would give a clear explanation as to how to restart the process, which would mean "next Fedora"
17:41:04 <zbyszek> Eighth_Doctor: actually, I think we have an answer to that question
17:41:08 <bcotton> if we make it should, then it's not worth adding, imo
17:41:09 <nirik> yeah, it does seem like a solution to a rare problem
17:41:36 <bcotton> i agree with zbyszek that this was driven by people upset with the results, but i also think the process was bad in this case
17:41:44 <zbyszek> Essentially, we (FESCo, submitters, commenters) should make an extra effort for complicated proposals.
17:42:08 <zbyszek> I.e. if we feel that something is contentious, we should deal with it with high priority.
17:42:24 <bcotton> like i said when #action'ed to write this, sometimes it's better  for long-term community health to take on the pain of this
17:42:24 <decathorpe> dcantrell: "next Fedora" only if the change has been submitted sufficiently late. earlier submissions can go through more cycles before time runs out
17:42:38 <zbyszek> We have this effect above some level of complexity, as the process drags on, we often slip for a week or two without any discussion or progress.
17:43:01 <dcantrell> decathorpe: that's true, but even then there's still a deadline at which point we can reject things as formally rejected
17:43:16 <sgallagh> Could we just make the timeline a little longer? Have an "early submission deadline" where rejections do not necessarily mean pushing to the next release. And a Final deadline that doesn't get the "tweak and resubmit" treatment?
17:43:21 <zbyszek> But that's something to handle via human effort, not with a change of rules.
17:43:35 <bcotton> but if we can't get agreement on this proposal, i'm okay with rejecting it and moving on with life
17:44:04 <sgallagh> Actually, I withdraw that suggestion
17:44:10 <sgallagh> I'm seeing too many holes in it already.
17:44:17 <bcotton> Stephen Gallagher: good, because i was trying to express how sad it would make me :-)
17:44:32 <Eighth_Doctor> could we just push out everything by two weeks?
17:45:06 <mhroncok> I believe that we need some kind of rules for this to maintain trust
17:46:01 <mhroncok> if we reject this with "it makes things slower" but don't have an alternative, we might think that everything is good, but in fact (part of) the community might no longer trust us
17:46:05 <nirik> do we have enough support for adding the "formally" for it to pass? if not, I think we should just drop it and try not have it happen more if we can help it.
17:46:22 <sgallagh> I honestly feel that the inciting situation was a supremely rare occurrence and doesn't necessitate complicating the process.
17:46:23 <decathorpe> nirik: you have my axe (not sure if that's enough)
17:46:57 <mhroncok> and as always, we can come up with improvements later if we figure out the rules are too strict
17:46:59 <sgallagh> But maybe we can add a simple rule: any FESCo member can add the tag "contentious" to the ticket. That tag requires a +7 majority rather than a +5 to pass.
17:47:22 <zbyszek> sgallagh: the filibuster?
17:47:30 <mhroncok> Stephen Gallagher: I think that's a very bad idea :D
17:47:31 <bcotton> i'm not sure it actually complicates the process. i think in most caes, people would already follow the process
17:47:32 <sgallagh> Dammit
17:47:32 <Eighth_Doctor> oh lord a filibuster
17:47:43 <sgallagh> Withdrawn
17:47:47 <bcotton> and complicating the voting process probably creates more problems than it solves
17:48:09 <sgallagh> Proposal: Acknowledge that the situation was handled poorly and try to just Do Better next time.
17:48:09 <nirik> it's already too complex. ;)
17:48:20 <bcotton> sgallagh++ for setting a new world record for number of proposals made and withdrawn in a single #topic
17:48:20 <zodbot> bcotton: Karma for sgallagh changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:48:28 <sgallagh> hahaha
17:48:48 <mhroncok> Stephen Gallagher: -1
17:48:53 <Eighth_Doctor> instead of adding rules, perhaps providing guidance for future change authors might be better
17:49:09 <mhroncok> and it's contentious, so you now need +7
17:49:11 <Eighth_Doctor> oh god
17:49:16 * sgallagh headdesks
17:49:20 * Eighth_Doctor wants to nope out of this
17:49:29 <bcotton> mhroncok++
17:49:29 <zodbot> bcotton: Karma for churchyard changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:50:14 <mhroncok> Conan Kudo: providing guidance for future change authors is good, but does not solve the trust thing
17:50:30 <decathorpe> I still think having some documentation for how to handle this would be good. Just dropping it doesn't feel right, given that there's obviously serious bad feelings about how we handled this last time
17:51:19 <sgallagh> We're coming up on 20 minutes on this topic and I don't feel like we're getting close to a solution. Shall we defer and take it to the list until next week?
17:51:26 <music[m]> I don’t have strong feelings on this. I don’t oppose the proposal with the “formally.” I see the case for doing something, because I think many of the people who became concerned about the process would be comforted by a good-faith effort to prevent a repeat.
17:51:45 <zbyszek> I'd prefer not to discuss this next week.
17:51:54 <zbyszek> As Neal said, "nope out".
17:52:41 <decathorpe> So, can we vote on the proposal plus nirik's clarification? Or defer until better days? :)
17:53:01 <sgallagh> OK, can we at least define "formally rejected"?
17:53:05 <nirik> We could try a vote?
17:53:23 <nirik> "rejected in a meeting and announced on the list"?
17:53:32 <mhroncok> in the ticket as well
17:53:35 <sgallagh> I can work with that definition
17:53:50 <sgallagh> Including Miro's addendum
17:53:55 <mhroncok> we can only reject in ticket with -9
17:54:11 <nirik> "rejected in a meeting, minutes posted to the list and fesco ticket closed'
17:54:13 <sgallagh> mhroncok: Uhhh, I don't agree with that.
17:54:15 <zbyszek> Can somebody paste the full text? I'm getting lost in all the amendments.
17:54:18 <smooge> hi from the peanut gallery.. could someone state what the proposal plus addendums is before the vote happens?
17:54:22 <mhroncok> I would like to see a proposal that gets -9 and tahn is approved without serious modification :D
17:54:42 <Eighth_Doctor> oh god
17:54:46 <Eighth_Doctor> chaos monkey you are
17:54:48 <decathorpe> I think the idea was "decision posted in the ticket", not "in-ticket voting resulted in rejection"
17:54:59 <sgallagh> Actually, I see what you're getting at. -9 means we can skip the meeting vote.
17:55:03 <sgallagh> I can work with that.
17:55:11 <Eighth_Doctor> yeah because it's totally rejected anyway
17:55:18 <Eighth_Doctor> I think we've had one case of that before
17:55:29 <bcotton> Since we're editing
17:55:32 <sgallagh> More than one, but at least one in the last year or so
17:55:46 <bcotton> Let's not specify the mechanism in the policy here and instead point to the existing vote policy
17:56:01 <mhroncok> thanks
17:56:48 <sgallagh> Well, the critical point is that it's a publicly announced decision
17:56:51 <nirik> yeah, agreed. Lets just say 'formally rejected' there and then add what that means to the vote policy?
17:56:52 <sgallagh> Let's leave it at athat?
17:56:58 <bcotton> "Proposals formally rejected according to the FESCo voting policy must…"
17:57:15 <Eighth_Doctor> I suppose it makes sense to clarify the vote policy on that front anyway
17:57:15 <nirik> formally rejected and announced... ?
17:57:36 <bcotton> redundant, imo, but i'd live with it
17:57:59 <bcotton> all of the decisions get announced anyway
17:57:59 <decathorpe> just "according to voting policy" covers too much
17:58:12 <decathorpe> I'm still +1 with the nirik amendment
17:58:15 <zbyszek> -1
17:58:20 <mhroncok> can we do a try-vote and only spend 20-minutes formalizing this if there is a chance of approval?
17:58:24 <nirik> The entire thing would then be: "Proposals that are formally rejected and announced may be submitted by reconsideration, but they must go through whatever process was originally required before FESCo begins voting."
17:58:28 <sgallagh> Ben Cotton (he/him): Sure, but until they're announced, they're Schrodinger's Decision
17:58:42 <bcotton> and, arguably, that opens up the possibility of rules lawyering in cases where the decision was agreed but didn't get announced yet. we want to avoid rules lawyering
17:58:45 <mhroncok> +1
17:58:58 <zbyszek> So this means +3–4 weeks before the vote can be repeated. Sorry, I think that's madness.
17:59:18 <sgallagh> patch "by reconsideration"->"For reconsideration"?
17:59:24 <mhroncok> zbyszek: why so many?
17:59:28 * decathorpe tired
18:00:16 <sgallagh> nirik: Can we append: "provided that the submission deadline has not passed"?
18:00:17 <zbyszek> Because that's how much it usually takes for a ticket to go through ReadyForWranger–>announced on fedora-devel->fesco ticket open->vote.
18:00:52 <nirik> well, that would be part of 'whatever process' no?
18:00:57 <sgallagh> ok
18:01:06 <nirik> but yeah, I am starting to think we should just drop this entirely... there's too many corner cases.
18:01:27 <bcotton> nirik: i agree it would be part of "whatever process"
18:01:27 <mhroncok> zbyszek: it takes 1 week at devel + 1 week at fesco
18:01:29 <bcotton> zbyszek: no, it's a week from the time it lands on my desk until the time i submit it to FESCo
18:01:31 <mhroncok> could go 1 week at devel + fasttrack
18:01:55 <zbyszek> If there's an ongoing discussion on fedora-devel, we normally would wait with the vote until the discussion dies down again.
18:02:02 <Eighth_Doctor> it takes two weeks, not one
18:02:17 <zbyszek> So "same process" means that we would wait the same, i.e. we could vote e.g. 8 weeks later.
18:02:20 <mhroncok> and when Ben Cotton (he/him) is unavailable and the person who wants to get a new vote knows they are running out of time, they can annoucne the proposal on devel themselves
18:03:38 <sgallagh> mildly sarcastic proposal: Rejected Changes may not be resubmitted for the same Fedora release.
18:04:02 <sgallagh> *formally rejected
18:04:11 <zbyszek> Counterproposal: FESCo acknowledges that there wasn't enough awareness and time for discussion before the repeated vote on issue #TBD. In the future, for contentious issues, we'll make an effort to notify fedora-devel and interested participants and leave time for comments.
18:04:12 <decathorpe> I'm sorry, this endless discussion turned my brain into 🍲, I need to step away and get some food. See you next week
18:04:34 <nirik> zbyszek: +1
18:04:36 <mhroncok> I don't think the 3-4 week is real
18:04:44 <mhroncok> see for example https://pagure.io/fesco/issue/2923
18:05:09 <mhroncok> on 2022-12-28, it was rpoposed for revote. That would ahve been annoucend on devel instead
18:05:40 <bcotton> i have exhausted my supply of interest in defending this proposal and i'm late for a session with a mentee, so i'll abide by whatever y'all decide
18:05:47 <mhroncok> after 7 days, it could go to fesco, that's 2023-01-04
18:06:06 <mhroncok> it would have been approved 6 days later on the next meeting
18:06:22 <smooge> point of information? Should this go to be discussed on the list as people are tired and leaving?
18:06:28 <zbyszek> mhroncok: but that's *not* "whatever process was originally required before FESCo begins voting"
18:06:54 <mhroncok> and what process was originally required then?
18:07:04 <mhroncok> formally required that is
18:07:36 <mhroncok> if we say "let's wait 2 more weeks for the discussion to calm down" I don't consider that as "originally required" in this ocntext
18:08:09 <mhroncok> but if this is the issue, I am happy to propose an alternative
18:08:13 <mhroncok> however.... let's stop?
18:08:25 <mhroncok> people are turning to vegetables here
18:09:26 <sgallagh> Proposal: Move on
18:09:28 <nirik> Yeah, lets just table it back and revisit later or close it.
18:09:34 <zbyszek> Ack.
18:09:38 <mhroncok> Move on +100
18:10:04 <sgallagh> #2951 Proposal: policy for resubmitting rejected proposals
18:10:04 <sgallagh> .fesco 2951
18:10:05 <zodbot> sgallagh: Issue #2951: Proposal: policy for resubmitting rejected proposals - fesco - Pagure.io - https://pagure.io/fesco/issue/2951
18:10:16 <mhroncok> not this again :D
18:10:23 <dcantrell> no, it's starting over!
18:10:25 <sgallagh> Oops, bad copy-paste
18:10:28 <zbyszek> You're not allowed until F39!
18:10:47 <sgallagh> #topic #2956 Does FESCo want to implement the policy for retiring packages
18:10:47 <sgallagh> with security bugs?
18:10:47 <sgallagh> .fesco 2956
18:10:48 <zodbot> sgallagh: Issue #2956: Does FESCo want to implement the policy for retiring packages with security bugs? - fesco - Pagure.io - https://pagure.io/fesco/issue/2956
18:10:57 <nirik> resubmit the resubmit
18:11:23 <sgallagh> Proposal: No
18:11:23 <nirik> I still think this is valuable, but I don't think releng has any cycles to do it.
18:11:31 <mhroncok> it's not about releng
18:11:39 <mhroncok> I can slay packages in my sleep
18:11:42 <music[m]> I’m out for today. Good luck with Zombie Son of Frankenproposal.
18:11:43 <sgallagh> Reasoning: the security bug process is so onerous that it's basically impossible to stay on top of them
18:12:08 <decathorpe> Is it really?
18:12:33 <Eighth_Doctor> Yes
18:12:37 <sgallagh> In the case of Node.js, every upstream security release dumps 12-30 bugs in BZ on me.
18:12:51 <zbyszek> sgallagh: yeah, it's not so bad, I think. I handled two for systemd last year and the process was just like any update.
18:12:54 <decathorpe> well, nodejs ...
18:12:55 <Eighth_Doctor> and I get a bunch of rando security BZes for me on ffmpeg
18:12:57 <nirik> I think there's several things here.
18:13:06 <Eighth_Doctor> ones that don't apply either
18:13:14 <nirik> The security bugs filed by RH are great... but sometimes... wildly wrong.
18:13:24 <mhroncok> random CVE bugzillas for packages that are not affected are not uncommon
18:13:24 <sgallagh> nirik: Those are the ones I'm talking about.
18:13:30 <nirik> like for example:
18:13:35 <sgallagh> mhroncok: Exactly
18:13:54 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=2173769
18:14:01 <sgallagh> I recently had to close almost 20 that were actually caused by the c-ares package, for example.
18:14:29 <decathorpe> Well that's a problem with RH scripts, but not with Fedora security policy ...
18:14:39 <nirik> so, I think we need to work with them and fix that part before we can start bugging maintainers to fix things.
18:15:01 <sgallagh> Except the policy requires maintainers to stay on top of dozens of reports with frequent false reports.
18:15:31 <sgallagh> So I go back to my proposal: We don't try to implement this policy at this time.
18:15:49 <mhroncok> well, I'd love if maintainers closed bogus bugzillas in timely manner
18:16:14 <mhroncok> we are talkign about bugzillas thta are open for months, if not years
18:16:14 <Eighth_Doctor> I got bug reports for electron on ffmpeg
18:16:14 <Eighth_Doctor> we don't even have electron in Fedora yet
18:16:21 <sgallagh> mhroncok: Last week, RH Security opened over 50 bugs against Node.js, nearly all of them ALREADY in the released package.
18:16:39 <music[m]> Ok, jumping back in for a second to note that it is also common to have CVE bugs for which no patch is available, let alone a released one. Also that sometimes the patch is lumped in with an incompatible update from upstream and can’t be easily backported. So sometimes the maintainer’s options are limited.
18:16:56 <sgallagh> I just gave up and closed them all as INSUFFICIENT_DATA in a fit of pique
18:17:02 <mhroncok> Stephen Gallagher: good
18:17:08 <mhroncok> Stephen Gallagher: as you should
18:17:17 <nirik> Or the ones I love... 0 information. No upstream report or patch...
18:17:18 <zbyszek> So that would be an example of the maintainer handling the bugs.
18:17:19 <decathorpe> There are already (in my opinion, too many) mechanisms for delaying package retirements in similar cases (like FTI or FTBFS bugs)
18:17:33 <Eighth_Doctor> yeah I have one for m2crypto that's like that :(
18:17:41 <Eighth_Doctor> I can't do anything because there's nothing given to me
18:17:58 <sgallagh> zbyszek: Right, but I'm a person who is PAID to work on this stuff and even I find it overwhelming.
18:18:04 <decathorpe> Nothing stops a package maintainer to set the bug to MODIFIED and ignore it forever
18:18:08 <sgallagh> I pity the volunteer maintainers out there
18:18:24 * Eighth_Doctor drowns in them
18:18:25 <nirik> do we know who we might be able to engage with there?
18:18:54 <nirik> I mean, I love that they file those bugs when they are valid, but... it's gotten pretty invalid lately
18:19:12 <mhroncok> I agree this needs to get better if we want to implement the policy
18:19:17 <decathorpe> I think Ben had talking to ProdSec about their bug reporting problems on his todo-list ... not sure if that happened yet
18:19:25 <sgallagh> OK, we don't have to solve that part of the problem today.
18:19:29 * nirik wonders if they plan to move and file those in jira
18:19:36 <zbyszek> So, should we discuss this with RHEL folks and then return to the issue?
18:19:43 <Eighth_Doctor> ugh
18:19:43 <sgallagh> What we need to decide is: is it worth implementing this policy today?
18:19:43 <Eighth_Doctor> one problem at at time
18:20:10 <mhroncok> is it worth implementing this policy even if the CVE reports are mostly correct?
18:20:19 <nirik> I don't think we want to bug maintainers more on invalid bugs... so no, I don't think this should be enabled until that part is fixed.
18:20:40 <zbyszek> Today: no. But once the bugs are mostly valid: yes.
18:20:44 <sgallagh> mhroncok: I'd vote 0 on that. I'm -1 on the policy as things stand today.
18:20:50 <nirik> Perhaps? but we could revisit that when/if ?
18:21:27 <mhroncok> we could
18:21:28 <nirik> -1 to implementing the policy per current conditions.
18:21:47 <mhroncok> -1 now, 0 if the bugzillas are accurate
18:22:19 <sgallagh> Proposal: In the current environment, FESCo feels that the CVE bug process is insufficient to support implementing this policy. This can be revisited in the future if conditions improve.
18:22:29 <decathorpe> meh. +1
18:22:35 <zbyszek> sgallagh: +1
18:22:39 <mhroncok> Stephen Gallagher: +1
18:22:49 <nirik> +1
18:23:01 <dcantrell> +1
18:23:40 <sgallagh> #agreed In the current environment, FESCo feels that the CVE bug process is insufficient to support implementing this policy. This can be revisited in the future if conditions improve. (+6, 0, -0)
18:23:45 <sgallagh> On to the last topic.
18:24:05 <sgallagh> #topic #2958 F38 incomplete changes: 100% complete deadline
18:24:05 <sgallagh> .fesco 2958
18:24:06 <zodbot> sgallagh: Issue #2958: F38 incomplete changes: 100% complete deadline - fesco - Pagure.io - https://pagure.io/fesco/issue/2958
18:24:32 <sgallagh> Ben Cotton (he/him): You have the floor
18:25:12 <nirik> He had to leave I think
18:25:13 <mhroncok> I don't think Ben Cotton (he/him) is still here
18:25:19 <mhroncok> we drove him away
18:26:04 <mhroncok> do we go one by one?
18:26:16 <zbyszek> Let's.
18:26:19 <sgallagh> OK
18:26:31 <nirik> not much left I think?
18:26:38 <sgallagh> #topic: Feature Incomplete: LLVM 16
18:27:13 <sgallagh> As I noted in the ticket, this is expected and will be resolved between Beta and GA
18:27:17 <nirik> So yeah, lets just let it do that.
18:27:20 <zbyszek> This one is waiting for a final release by upstream. IIUC, nothing to see, move along.
18:27:40 <sgallagh> I'll wait 60s for anyone to disagree, then move on
18:28:13 <mhroncok> ack
18:28:20 <sgallagh> #topic Feature Incomplete: Unified Kernel Support (Phase 1)
18:28:44 <mhroncok> I still have no idea if this is done
18:28:58 <sgallagh> I don't have any insight here.
18:28:58 <nirik> This is an optional thing, so I am fine with them working on it after beta
18:29:26 <zbyszek> I think not much changed from two weeks ago. The kernel parts are done, the grub2 parts are not done.
18:29:33 <nirik> https://bugzilla.redhat.com/show_bug.cgi?id=2159490#c2 has the status as of a few weeks ago
18:30:03 * nirik nods.
18:30:08 <mhroncok> OK, so we continue doing nothing as we don't really care when this lands? can we still call it f38 change if it doesn't land in time for GA?
18:30:20 <nirik> +1 to that from me.
18:30:28 <sgallagh> If it doesn't land in time for GA, we won't mention it in the release materials
18:30:33 <zbyszek> Yeah. The kernels are usable without grub2 support.
18:30:47 <mhroncok> ack
18:31:30 <sgallagh> #topic Feature Incomplete: RPM Macros for Build Flags
18:31:37 <mhroncok> merge the PR
18:31:43 <zbyszek> merge the PR
18:31:59 <sgallagh> This is only incomplete because the maintainers were waiting for FESCo to approve a rename.
18:31:59 <nirik> +1
18:32:00 <sgallagh> We don't need to do that, and I said as much on the MR
18:32:08 <sgallagh> So this is basically ready.
18:32:20 <nirik> does this need to pass through freeze?
18:32:48 <mhroncok> no
18:32:49 <sgallagh> It's a build-time change
18:33:16 <mhroncok> it is a fedora 38 change, but it only really makes sense for rawhide at this point
18:33:53 <mhroncok> but having it in the redhat-rpm-config package for f38 as well is not harmful and might be useful for some
18:34:11 <sgallagh> Yeah, so let's just give it the all-clear?
18:34:31 <mhroncok> all clear, but no reason for bypassing the freeze imho
18:34:45 <sgallagh> Agreed
18:35:17 <zbyszek> Eighth_Doctor: you still around? Can you press the merge button on #244?
18:35:57 * nirik can do it too if you like
18:36:28 <nirik> anything else?
18:36:28 <sgallagh> #topic Open Floor
18:36:33 <sgallagh> oh wait, sorry
18:36:38 <sgallagh> #topic Next Week's Chair
18:37:02 <zbyszek> nirik: somebody should do it, and also cherry-pick for f38.
18:37:22 <nirik> I did, do we need builds also?
18:37:29 <sgallagh> Yes
18:37:45 <sgallagh> So, who wants to Chair next week?
18:38:14 <mhroncok> no chairing for me until summer, sorry
18:38:31 <mhroncok> (and I need to leave now)
18:39:19 <sgallagh> I might not make it to the meeting next week, so I can't do it
18:39:26 <nirik> ugh, DST is coming.
18:39:32 <Eighth_Doctor> zbyszek: sure
18:39:36 <sgallagh> Wait, is that this weekend?
18:39:40 <sgallagh> sadface
18:39:46 <nirik> no. next
18:39:53 <Eighth_Doctor> yup ugh
18:40:11 <Eighth_Doctor> this will collide with team sync meetings
18:40:19 <Eighth_Doctor> :(
18:40:23 <zbyszek> OK, since it seems nobody is racing to do it, I can.
18:40:47 <sgallagh> Thanks, zbyszek
18:41:12 <sgallagh> #action zbyszek to chair 2023-03-07 meeting
18:41:13 <sgallagh> #topic Open Floor
18:41:16 <sgallagh> Any topics for Open Floor? No? Good :)
18:41:40 <zbyszek> I had something, but it can wait.
18:41:59 <sgallagh> zbyszek: Go ahead, I was being flippant
18:42:10 <zbyszek> No, it's a longer thing.
18:42:25 <Eighth_Doctor> well apparently whoever merged #244 gets to build it for f38 now too
18:42:27 <sgallagh> OK, in that case, open a discussion on a ticket or devel@
18:43:24 <zbyszek> nirik: in #244, was the inclusion of the other two commits from rawhide intentional?
18:43:46 <nirik> Conan Kudo: doing so now.
18:44:15 <nirik> oh, shoot, were those not supposed to be in there? they seemed harmless....
18:44:34 <zbyszek> They probably are ok…
18:45:26 <sgallagh> OK, are we done here, then?
18:45:35 <zbyszek> sgallagh: sorry for derailing the quick wrap up.
18:45:54 <sgallagh> No worries
18:45:55 <sgallagh> #endmeeting