17:00:28 <bcotton_> #startmeeting F33 Final Go/No-Go meeting
17:00:28 <zodbot> Meeting started Thu Oct 22 17:00:28 2020 UTC.
17:00:28 <zodbot> This meeting is logged and archived in a public location.
17:00:28 <zodbot> The chair is bcotton_. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:28 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:00:28 <zodbot> The meeting name has been set to 'f33_final_go/no-go_meeting'
17:00:29 <bcotton_> #meetingname F33-Final-Go_No_Go-meeting
17:00:29 <zodbot> The meeting name has been set to 'f33-final-go_no_go-meeting'
17:00:36 <bcotton_> #topic Roll Call
17:00:40 <coremodule> .hello2
17:00:41 <King_InuYasha> .hello ngompa
17:00:44 <zodbot> coremodule: coremodule 'Geoffrey Marr' <gmarr@redhat.com>
17:00:47 <zodbot> King_InuYasha: ngompa 'Neal Gompa' <ngompa13@gmail.com>
17:00:48 <pwhalen> .hello2
17:00:49 <zodbot> pwhalen: pwhalen 'Paul Whalen' <pwhalen@redhat.com>
17:00:50 <nirik> morning
17:00:55 <coremodule> hey
17:00:58 * nirik gets some coffee.
17:01:07 <bcotton_> this may end up being the most well-attended go/no-go meeting in recent memory
17:01:15 <Southern_Gentlem> .hello jbwillia
17:01:16 <zodbot> Southern_Gentlem: jbwillia 'Ben Williams' <vaioof@gmail.com>
17:01:34 <frantisekz> .hello2
17:01:35 <zodbot> frantisekz: frantisekz 'František Zatloukal' <fzatlouk@redhat.com>
17:01:48 * sumantro is here
17:01:51 <mboddu> .hello mohanboddu
17:01:52 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
17:02:04 <sumantro> .hello sumantrom
17:02:05 <zodbot> sumantro: sumantrom 'Sumantro Mukherjee' <sumukher@redhat.com>
17:02:19 <sgallagh> .hello2
17:02:19 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
17:02:23 <mboddu> I remember bcotton_ saying ship it :P
17:02:59 <mattdm> I am here :)
17:03:05 <cmurf> .hello chrismurphy
17:03:06 <zodbot> cmurf: chrismurphy 'Chris Murphy' <bugzilla@colorremedies.com>
17:03:56 <bcotton_> i'm going to go ahead and start....the process while people join
17:04:01 <bcotton_> because this may take a while :-)
17:04:12 <x3mboy> .hello2
17:04:13 <bcotton_> #topic Purpose of this meeting
17:04:13 <zodbot> x3mboy: x3mboy 'Eduard Lucena' <eduardlucena@gmail.com>
17:04:14 <bcotton_> #info Purpose of this meeting is to check whether or not F33 Final is ready for shipment, according to the release criteria.
17:04:15 <lruzicka2> .hello lruzicka
17:04:16 <bcotton_> #info This is determined in a few ways:
17:04:17 <bcotton_> #info 1. No remaining blocker bugs
17:04:18 <zodbot> lruzicka2: lruzicka 'Lukáš Růžička' <lruzicka@redhat.com>
17:04:19 <bcotton_> #info 2. Release candidate compose is available
17:04:20 <bcotton_> #info 3. Test matrices for Beta are fully completed
17:04:31 <mboddu> ^ Final ?
17:04:35 <bcotton_> DOH
17:04:38 <bcotton_> #undo
17:04:38 <zodbot> Removing item from minutes: INFO by bcotton_ at 17:04:20 : 3. Test matrices for Beta are fully completed
17:04:46 <bcotton_> #info 3. Test matrics for Final are fully completed
17:05:18 <bcotton_> okay, let's do this
17:05:27 <bcotton_> #topic Current status - blockers
17:05:28 <bcotton_> #link https://qa.fedoraproject.org/blockerbugs/milestone/33/final/buglist
17:05:38 <bcotton_> Before we start discussion, I'm going to set expectations
17:05:58 <mattdm> good call :)
17:06:00 <bcotton_> I know this was a very contentious discussion in the FESCo meeting yesterday, so I'll just remind everyone of the Friends foundation
17:06:14 <nirik> +1
17:06:33 <mattdm> Yeah. This is a situation where I have an opinion, but I know it's a hard call and I have a lot of respect for the other point of view and those of you who hold it :)
17:06:36 <bcotton_> try to keep your comments short and on-topic and let's make sure we're taking turns to let people speak
17:07:11 * Southern_Gentlem gets popcorn
17:07:14 * sgallagh puts away the megaphone and looks sad.
17:07:39 <bcotton_> and i'll say this ahead of time because i don't want it to seem like a last minute thing, but traditionally blockers require a clear consensus (generally understood to be a net +3 or so) and the default state is not a blocker
17:07:51 <bcotton_> so having said that, let's actually look at the blockers
17:08:15 <bcotton_> #info 1 Proposed Blockers
17:08:16 <bcotton_> #info 0 Accepted Blockers
17:08:18 <bcotton_> #info 0 Accepted 0-day Blockers
17:08:26 <bcotton_> #topic (1883609) Secure Boot fails to boot F33 Beta image
17:08:27 <bcotton_> #link https://bugzilla.redhat.com/show_bug.cgi?id=1883609
17:08:28 <bcotton_> #link https://pagure.io/fedora-qa/blocker-review/issue/135
17:08:30 <bcotton_> #info Proposed Blocker, shim, ASSIGNED
17:08:48 <bcotton_> #info Vote in ticket is currently FinalBlocker (+7, 1, -6)
17:09:38 <bcotton_> so....anyone want to change some minds?
17:09:40 <nirik> is that the votes from the revoting recently? or includes eariler ones?
17:10:00 <Southern_Gentlem> ?
17:10:11 <sgallagh> The revote reset the count. So it's current.
17:10:13 <bcotton_> nirik: this is since the vote was reset yesterday
17:10:14 <mattdm> Is it useful for me to repeat my points from the ticket?
17:10:35 <nirik> ok
17:10:35 <lruzicka> yes
17:10:56 <bcotton_> i will note, though that the fact that Ubuntu is stopping the rollout of the dbx was not shared until after several votes (in both directions) were lodged
17:11:13 <bcotton_> mattdm: standby. Southern_Gentlem: did you have a question?
17:11:31 <Southern_Gentlem> let mattdm go he may answer my question
17:11:43 <bcotton_> ack. mattdm, all you your Popeliness
17:11:47 <mattdm> lol
17:11:53 <mattdm> So I think there's no good answer here.
17:12:00 <mattdm> But, I'm leaning towards "ship it"
17:12:06 <mattdm> The basic thing is:
17:12:12 <mattdm> Fedora 32 already has this problem.
17:12:26 <mattdm> If we don't ship, we're in the same state of "brokeness"
17:12:28 <mattdm> So why not ship?
17:12:57 <mattdm> We can produce updated images later. Southern_Gentlem proves it is possible :)
17:13:02 <King_InuYasha> so... to me, the fact that Ubuntu *finally* pulled the update makes me more comfortable with releasing now
17:13:09 <Southern_Gentlem> ?
17:13:23 <King_InuYasha> we do not have the capability to produce all the images post-GA, but we should develop that facility
17:13:34 <mhroncok> .
17:13:38 <mattdm> with respins! i mean, not quite the same, but it's not like it's a mystery
17:13:46 <King_InuYasha> mattdm: actually it is
17:13:54 <King_InuYasha> I don't even *know* how half our images are made
17:14:00 <sgallagh> Let's not get too far off-topic.
17:14:13 <mattdm> I take it as granted that we have the ability to solve that. It's work, not a mystery.
17:14:15 <nirik> well, I was looking into that and I think it might not be as hard as I was thinking.
17:14:17 <sgallagh> nirik has indicated that it should be doable,
17:14:19 <King_InuYasha> anyway, I'm willing to change to -1 FinalBlocker now, with the proviso we have a Fedora Magazine article explaining this
17:14:34 <sumantro> bcotton, I want to go -1 from 0
17:14:35 <King_InuYasha> we need it anyway now that all old-world keys are busted
17:14:39 <bcotton_> sumantro: ack
17:14:44 <sgallagh> King_InuYasha: I'll second the magazine article request
17:14:48 <mhroncok> .
17:14:55 <mattdm> Do we have any volunteers to _write_ said article?
17:14:58 <sgallagh> mhroncok: ?
17:15:02 <Southern_Gentlem> so what is the timeline from upstream of getting the new shim? is that known?
17:15:04 <mhroncok> (not sure if this is the established way to get some space)
17:15:08 <King_InuYasha> Southern_Gentlem: no
17:15:19 <mattdm> Southern_Gentlem: no, but pjones was optimistic about a two-week timeline
17:15:24 <mboddu> We can reproduce the images and put them somewhere for people to access
17:15:31 <mattdm> optimistic but no guarantees.
17:15:41 <bcotton_> sumantro: you were a 0 in the ticket
17:15:50 <King_InuYasha> it might also be a rare instance where we update the GA repositories post-GA to add an updated package
17:15:51 <Southern_Gentlem> i can do respins in 2 weeks if needed
17:15:51 * cmurf raises hand
17:15:56 <sgallagh> sumantro: He said he wanted to switch to -1
17:15:58 <mattdm> Southern_Gentlem++
17:16:02 <sumantro> bcotton yes, and I want to go FinalBlocker -1
17:16:05 <sgallagh> err, bcotton_
17:16:09 <bcotton_> sumantro: sorry, i read that backwards
17:16:13 <Southern_Gentlem> also not releaseing and holding 2-4 weeks really gets us anything
17:16:14 <nirik> if we need to, we should make a new point release compose... IMHO.
17:16:21 <frantisekz> mboddu: can we actually respin the GA? so the newer images will be available through Media Writer, from default getfedora links and so on...?
17:16:21 <sgallagh> mhroncok and cmurf would like to interject. Let's allow them to
17:16:25 <bcotton_> okay. let's let mhroncok go
17:16:35 <mhroncok> I have a concern. We have an ability to block here and now and make sure we fix the problem or else we don't release Fedora. That's strong enough motivation to actually fix it and fix it in time. Once we vote -blocker and ship, the "we need to fix this AND release new media" problem has no process and I am afraid that it will be a huge pain.
17:16:53 <mhroncok> if we are OK with that challenge, we shoudl ship
17:16:57 <mboddu> I will also go with -1 blocker since Ubuntu pulled the update as well, FYI bcotton_ I haven't voted on that ticket so far
17:16:59 <mattdm> OOOh. I want that NEVER to be the reason to block.
17:17:00 <mattdm> Wrong hammer.
17:17:13 <bcotton_> mboddu: ack
17:17:14 <mhroncok> but I don't think that "always release on time" is a good enough reason not to slip
17:17:28 <mhroncok> eom
17:17:39 <bcotton_> mhroncok: thanks. cmurf, go ahead
17:17:43 <cmurf> did anyone ask pjones whether it's possible to just resign the current (old 2018 built) shim? and does it matter if it's an option?
17:17:52 <mattdm> I think we have enough motivation to fix that without "we're withholding all other features for everyone so get with it" as a motivator
17:18:00 <mboddu> frantisekz: Probably not, its not that it cant be done, but it will be very tricky and messy
17:18:02 <sgallagh> cmurf: He's been asked, but has not replied.
17:18:21 <Southern_Gentlem> -1 FB THTF
17:18:39 <sgallagh> THTF?
17:18:47 <Southern_Gentlem> too hard to fix
17:18:48 <bcotton_> To Help The Fedora
17:18:55 <sgallagh> ha!
17:19:01 <cmurf> seems like a safe fast option that totally obviates the problem
17:19:08 <King_InuYasha> cmurf: not quite
17:19:25 <Southern_Gentlem> can this be fixed post release
17:19:27 <King_InuYasha> I don't know this for sure, but I think one of the changes with shim is that it can be signed with two signatures
17:19:33 <nirik> I am against 'respinning' f33... but I think we can do a 33.1 (or whatever seperate/different version) with some pain, but possible. It would need desire to do it, qa, etc.
17:19:47 <sgallagh> Southern_Gentlem: It can be fixed for current installs, but the install media can't be changed post-release.
17:20:07 <King_InuYasha> well, we *could* change the media if we really needed to
17:20:24 <mhroncok> but somebody would need to own this problem
17:20:28 <bcotton_> and part of the problem is that we don't know if it will be a problem before f34 is released. "no earlier than Q2" is right in the "well maybe but maybe not" for our schedule
17:20:31 <Southern_Gentlem> sgallagh,  well thats why the respin sig was started so we do that to fix issues
17:20:34 <mhroncok> it won't happen automagically
17:20:41 <frantisekz> mhroncok++
17:20:41 <zodbot> frantisekz: Karma for churchyard changed to 17 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:20:42 <adamw> ahoyhoy
17:20:46 <mattdm> hey adamw!
17:20:51 <frantisekz> hi :)
17:20:51 <sgallagh> Southern_Gentlem: I don't disagree. That's one of the mitigations we've been citing.
17:20:57 <King_InuYasha> Southern_Gentlem: we cannot respin install DVDs, PXE images, or disk images :(
17:21:02 <adamw> so has the Great Fedora UEFI Civil War started?
17:21:06 <bcotton_> mhroncok: agreed, but it also might not be a problem. it's a *potential* problem at this point
17:21:06 * cmurf i think a 33.1 in the middle of the f34 cycle is begging for some combination of f34 slip and burn out by various people
17:21:09 <King_InuYasha> yup
17:21:10 <nirik> King_InuYasha: we can. it would be a 33.1 or something
17:21:11 <bcotton_> adamw: we've been far too civil so far
17:21:19 <coremodule> adamw, yes, and only you can end it
17:21:46 <mattdm> Can we produce a "33.1" that's just the same except the updated shim package?
17:21:48 <bcotton_> so as it stands right now, we've gone from (+7,1,-6) to (+7,0,-9)
17:21:51 <mattdm> Not "33 + all updates so far"?
17:21:58 <nirik> mattdm: yes. but it would be effort still.
17:22:00 <adamw> fwiw, a lot of my +1 is the same concern as mhroncok's
17:22:03 <mboddu> adamw is not around for the first few min of the meeting
17:22:07 <mattdm> because I agree that doing a full additional QA cycle is not viable
17:22:10 <mhroncok> bcotton_: is there a scenario when we won't need to re-create the media?
17:22:11 <adamw> i think he phrased that very well
17:22:12 <mboddu> Oh, he is here
17:22:19 <mattdm> adamw: that it won't be taken seriously?
17:22:23 <lruzicka> I am going to ask a problematic questions - why is it so important that we "release on time"?
17:22:30 <mhroncok> bcotton_: (if we release with old shim now)
17:22:31 <mattdm> or that there is no process?
17:22:34 <adamw> that it would be easy to not manage to do it because there's no existing process or deadlines
17:22:41 <bcotton_> mboddu: yes, if the revocations don't happen until after we release f34, the f33 media is far less important
17:22:41 <sgallagh> mattdm: A change to the bootloader probably *does* warrant a good chunk of the QA cycle, though.
17:22:45 <adamw> we'd be making it up as we went along
17:22:57 <sgallagh> adamw: That *is* our core competency...
17:22:59 <bcotton_> er, sorry,that was for mhroncok
17:22:59 <King_InuYasha> I don't disagree, but I'm burned out now on this discussion
17:23:01 <adamw> =)
17:23:21 <mattdm> Releasing on the schedule has huge benefits for our reputation, with users and with partners like Lenovo
17:23:36 <mattdm> I'm not saying two weeks will kill that, but it's a shame to break the streak
17:23:49 <mattdm> and we have so much stuff in this release that I'm eager to get it to users
17:23:52 <King_InuYasha> I'd really rather have this fixed in GA, but the fact that this isn't progressively getting worse now makes me feel like it's less urgent
17:24:02 <Southern_Gentlem> has any of the partners tested the f33 beta to see if it causes them issues
17:24:04 <sgallagh> Also, given civil unrest in the USA, the news cycle post-Nov. 3rd may not be ideal
17:24:04 <lruzicka> mattdm, ok, are you absolutely sure that releasing with this blocker not fixed will not harm the reputation?
17:24:06 <adamw> having people try to install our current release in 4 months and it not working because of this would not do our reputation a lot of good.
17:24:07 <King_InuYasha> (to note, Ubuntu yanked the dbx update this morning)
17:24:11 <mattdm> Being consistent with releases translates to people feeling confident about the distro as a whole
17:24:25 <mattdm> Southern_Gentlem: Lenovo is testing the F33 beta now, yes.
17:24:35 <Southern_Gentlem> are they seeing this issue
17:24:36 <adamw> sorry if this has already been asked, but do we have any kind of known schedule for when significant entities are planning to (re-)ship the key revocations?
17:24:38 <mattdm> They have, to cmurf's sadness I'm sure, turned of Secure Boot by default.
17:24:40 <bcotton_> lruzicka, adamw: the problem is we don't know if that will happen or not. certainly not before april 1 according ot microsoft
17:24:54 <bcotton_> adamw: "no earlier than Q2" is what i've been told Microsoft is saying
17:24:54 <sgallagh> adamw: That's true, but we will also have the "well, try F34 Beta!" argument there...
17:24:56 <mattdm> adamw: "Q2 2021, probably-ish"
17:24:56 <adamw> microsoft is obviously the big one, but ubuntu and system firmware updates too
17:24:59 <King_InuYasha> adamw: toward the end of Q1
17:25:03 <adamw> sgallagh: i hate that argument.
17:25:09 <sgallagh> Fair
17:25:17 <adamw> "our stable release doesn't work? eh try the beta" is not a reputation-enhancing proposition :P
17:25:28 <nirik> Southern_Gentlem: no, because lenovo ships with secureboot off. :(
17:25:29 <mhroncok> adamw++
17:25:29 <zodbot> mhroncok: Karma for adamwill changed to 11 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:25:29 <bcotton_> if we knew with any precision when this would happen, it would make the decision a lot easier
17:25:38 <adamw> bcotton: yes.
17:25:38 <mattdm> I think the people who are most likely to be affected, though, are enthusiasts for whom "try the beta" isn't a hardship at all
17:25:41 <King_InuYasha> fact of the matter is, nobody ever thought we'd have to do a key rotation for secure boot ever when we designed this whole thing
17:25:48 <sgallagh> mattdm: +1
17:25:51 <adamw> King_InuYasha: "designed"
17:25:55 <cmurf> On the plus side, Lenovo is one of the only manufacturers who publishes hashes of their firmware. Esoteric though it is, there is a way to verify you have manufacturer firmware.
17:25:56 <King_InuYasha> which was, in retrospect, a really bad move
17:26:04 <King_InuYasha> adamw: I'm trying to be somewhat charitable :P
17:26:13 <adamw> mattdm: i'm also not buying that. for me enthusiasts are if anything more likely to fiddle around with turning SB off.
17:26:19 <adamw> non-enthusiasts buy a laptop and leave it alone.
17:26:33 <mattdm> but they're not going to get updated dbx early
17:26:47 <sgallagh> non-enthusiasts read all the websites that advise turning it off, do that, then never think about it again
17:26:57 <adamw> mattdm: do you have next week's lottery numbers in that crystal ball too? :P
17:27:01 <mattdm> yes :)
17:27:06 <adamw> okay, so if microsoft is saying q2 2021
17:27:09 <mattdm> 4 41 56 1 57 3
17:27:09 <adamw> that's april at earliest
17:27:23 <sgallagh> But realistically, not before April 1 is the same as saying "Almost F34 release"
17:27:38 <adamw> f34 'preferred' release is april 20
17:27:39 <lruzicka> so, wj
17:27:44 <cmurf> they haven't said Q2 as far as I know
17:27:48 <adamw> so that's already three weeks potentially, and that assumes f34 doesn't slip
17:27:53 <sgallagh> So even if it happens right on Apr. 1, we do have the  "F34 will be out soon" statement.
17:27:54 <Southern_Gentlem> so we are going to delay f33 to till then or what
17:27:57 <King_InuYasha> I've mostly heard March is the month that things will start shipping
17:28:02 <lruzicka> what about this proposal, can we plan to release F34 a little earlier to be ready for the change?
17:28:06 <mattdm> See, I want us to be able to assume F34 won't slip :)
17:28:07 <cmurf> they've said 2021 publicly, and i've heard it's Q1 or Q2
17:28:12 <sgallagh> Southern_Gentlem: No, the new key is available, the current key is getting revoked then
17:28:19 <adamw> and of course we have absolutely no precedent for RH trying to jam major features into releases-that-will-become-RHEL or anything like that, that never happens. :P
17:28:21 <sgallagh> But we aren't yet signed with the new key
17:28:31 <King_InuYasha> adamw: naturally not :)
17:28:34 <frantisekz> lruzicka: I wouldn't put F34 release date to an earlier point
17:28:56 <frantisekz> there are other schedules around that - GNOME, KDE, gcc,...
17:28:57 <sgallagh> cmurf: pjones reported they said "Not before Apr." explicitly to him in person.
17:28:59 <bcotton_> shortening the F34 schedule is appealing, but given our downstream's development cycle, i would guess it will be too feature-ful to make it realistic
17:29:26 <cmurf> OK
17:30:15 <mattdm> shortening the F34 cycle isn't appealing to me for the exact same consistency reason :)
17:30:19 <cmurf> bcotton_: Asked about escalating F34 schedule in Workstation meeting a week ago and it's pretty much a no go.
17:30:22 <nirik> it's a tough call. ;(
17:30:27 <Southern_Gentlem> i want to thank everyone that has been beating the midnite oil troubleshooting this, but i see no way fedora should wait 2 weeks or 6months to release f33
17:30:34 <cmurf> As in, impractical to make it go faster.
17:30:48 <adamw> Southern_Gentlem: there is no proposal to wait six months.
17:30:50 <bcotton_> cmurf: yeah, unless we get gnome to release early, too
17:31:34 <bcotton_> so the vote as it currently stands is net negative. anyone interesting in going from -1 to 1?
17:31:53 <Southern_Gentlem> adamw, we release a new shim but M$ doesnt then people are mad because we shipped and nuked their windows install
17:31:56 <bcotton_> because otherwise, it seems like we have as stable of an opinion as we're going to have
17:32:05 <mattdm> Southern_Gentlem++
17:32:14 <adamw> Southern_Gentlem: that can't happen.
17:32:31 <mattdm> Southern_Gentlem: on that last one, I think we're just shippign new signed shim not the revocation list
17:32:36 <adamw> Southern_Gentlem: we're not changing the system's trusted keys. just signing our bootloader with a different one.
17:32:52 <Southern_Gentlem> ok
17:32:53 <bcotton_> alternatively: does anyone want to make a last-minute attempt at convincing the group that blocking is the way to go
17:32:58 <adamw> bcotton: i just did :P
17:33:20 <mattdm> loll
17:33:27 <cmurf> Almost everyone of the major distros, and also Microsoft, have new-world key signed bootloaders.So the revocation list being pushed affects mostly us and maybe Debian and one other.
17:33:33 <Southern_Gentlem> someone flip a coin
17:33:46 <King_InuYasha> cmurf: Fedora and Debian are it
17:33:49 <King_InuYasha> everyone else has been fixed
17:33:59 <King_InuYasha> at least of the major secure boot supporting distros
17:33:59 <cmurf> i.e. dbx version >=190
17:34:24 <King_InuYasha> Debian, however, only just introduced secure boot support with this latest release a couple years ago, so...
17:34:33 <mboddu> If we are going for a release, we should set expectations on the new images? Do we build them, release them....
17:34:51 <bcotton_> mboddu: what do you mean?
17:34:53 * cmurf hasn't voted yet
17:34:56 <mattdm> Let's start planning for it, but not say we will
17:34:58 <nirik> mboddu: you mean a point release after f33?
17:35:04 <mattdm> what are your thoughts, cmurf?
17:35:10 <mboddu> bcotton_: ^^ yes
17:35:18 <cmurf> it's a coin toss, consequences either way
17:35:52 <nirik> I'm not sure this meeting is the right place to discuss a possible point release...
17:36:01 <mattdm> cmurf: yeah. :(
17:36:06 <bcotton_> mboddu: yeah, what mattdm said. we should start figuring out what we need to do, but we don't need to _do_ it until we know we have to
17:36:28 <mboddu> Okay
17:36:45 <cmurf> well that was the primary question in the fesco issue i filed that kicked this off :)
17:36:52 <bcotton_> fwiw, i'd be solidly +1 if we knew with reasonable certainty that the new dbx would be widely distributed prior to the f34 release, but since we don't, it's hard for me to justify that
17:36:54 <cmurf> contingency
17:36:59 <mhroncok> without a plan however we are basically creating a huge problem for our future selves
17:37:13 <cmurf> mhroncok: agreed
17:37:17 <King_InuYasha> I would guarantee that we're going to be in trouble no matter what
17:37:20 <adamw> nirik: i mean, i'm much more happy with a -1 if we make a definite commitment to ship at least some kind of respun image with a resigned bootloader
17:37:24 <mhroncok> cmurf: and that ticket was solved by making it a blocker
17:37:25 <adamw> at minimum a new workstation live
17:37:25 <King_InuYasha> we can't do anything now, and we have no way to do anything in the future
17:37:33 <mhroncok> by revoking the blocker status, we have unsolved it
17:37:37 <King_InuYasha> adamw: I would want *all* media respun if we did that
17:37:39 <nirik> adamw: ok, thats fair.
17:37:54 <King_InuYasha> it is not acceptable, IMO, to have less than all of it
17:38:02 <adamw> nirik: i would usually agree that the scope of the meeting is just "is it a blocker or not" but i think this case is unusual :)
17:38:19 <mattdm> We are not *automatically* creating a huge problem. "Everything is basically fine until F34 is released" is actually an outcome with a reasonable possibility
17:38:53 * mhroncok would eb ahppy to go -1 if we have a concrete contingency plan. "we might be able to release new media, fingers crossed" isn't it
17:39:01 <sgallagh> And (as much as cmurf hates it), “disable secureboot” is a valid workaround.
17:39:10 <frantisekz> also, "Everything is on fire until F34 is released" is a reasonable possibility :) , we should have some plan what to do in the fire case, imo
17:39:14 <cmurf> it's a valid work around
17:39:20 <cmurf> but bad advice
17:39:27 <mattdm> cmurf I 100% agree
17:39:28 <sgallagh> Agreed
17:39:40 <King_InuYasha> mattdm: there is no reasonable case where that is true
17:39:53 <cmurf> also ;) there is another work around not discussed
17:40:03 <mattdm> cmurf?
17:40:05 <sgallagh> Resetting the firmware
17:40:05 <King_InuYasha> mattdm: unless we have a plan to fix this very soon
17:40:18 <nirik> so, would it help if we schedule a point after we have new shim signed and decide if we need a new point release then? otherwise it's just a case of it being enough of a problem to do it?
17:40:36 <cmurf> the same WHCP spec that requires (now optional) that SB can be disabled, also requires that a user can put the firmware into Custom mode, which in the simple case wipes all databases including the dbx.
17:40:38 <mattdm> nirik: yes.
17:41:34 <bcotton_> nirik: yes, but let's not decide that schedule in this meeting since 1. we don't know when we'll have it and 2. we need to sit down and figure out what tasks are needed and where they'll fall on the calendar
17:41:40 <cmurf> but i guess there are some firmware floating about that don't do this reliably *shrug*
17:42:05 <Southern_Gentlem> someone flip the coin and lets move on
17:42:14 <nirik> well, deciding to schedule that might help folks -1 blocker this now...
17:42:34 <bcotton_> nirik: yes, we can decide *to* schedule now, just not what the schedule will be :-)
17:42:35 <Southern_Gentlem> we cant fix it now and i like nirik plan
17:43:17 <mattdm> nirik: I think we're at "not a blocker" given the rule bcotton_ said at the beging
17:43:48 <mattdm> But we definitely don't want to take this lightly and rubber-stamp so it's worth the discussion
17:43:54 <Southern_Gentlem> if needed we can do a 33.a if needed but i have a feeling that f34 this issue will be resolved ( i hope)
17:44:14 <bcotton_> proposed #agreed 1883609 - RejectedBlocker (Final) - There's no clear consensus, but the net vote is "not a blocker". The FPgM will work with appropriate parties to develop a schedule for a potential point release that contains a newly-signed shim
17:44:38 <mattdm> and even if we don't respin, we can definitely publicize when we have a new shim and tell people with secureboot off to turn it on
17:45:04 <mattdm> which may actually result in net greater secure boot use overall than if we hadn't run into this :)
17:45:12 <cmurf> :D
17:45:26 <nirik> ack
17:45:32 <mattdm> ack
17:45:43 <cmurf> ack
17:45:48 <adamw> reluctant ack
17:45:53 <sgallagh> ack
17:45:57 <mhroncok> sad ack
17:46:05 <mboddu> Cant do anything ack
17:46:06 <mattdm> lol :)
17:46:07 <Southern_Gentlem> ack
17:46:11 <mattdm> yes
17:46:15 <bcotton_> adamw: anything that could make you less reluctant or is it just a "this situation is sad"?
17:46:18 <frantisekz> ack
17:46:18 <sgallagh> sad sack sad ack
17:46:19 <lruzicka> ack (I am taking the promise)
17:46:29 <sumantro> ack
17:46:35 <adamw> bcotton: i would've liked a firmer commitment to new images. but oh well.
17:46:44 <mboddu> bcotton_: FYI, we can compose those images, and push them somewhere on to the mirrors, but not to the places where mediawriter looks at and people are familiar with
17:47:02 <mboddu> We might have to advertise about them
17:47:06 <lruzicka> mboddu, adamw: that could be enough if properly advertised
17:47:12 <nirik> well, they are still going to need qa, etc
17:47:19 <bcotton_> mboddu: so we'd have to document TF out of it. that's doable, if not fully effective
17:47:54 <lruzicka> nirik, we are always ready
17:47:55 <mhroncok> mbooth: can we make mediawriter look there as well?
17:47:56 <mboddu> Yup and so, its extra work for QA and releng
17:48:10 <frantisekz> mboddu: even if it's F31.x ? can't this be done?
17:48:19 <nirik> if we do a point release, I think we should do it just like a release... make rc's, test, sync, update websites/docs, etc.
17:48:36 <mboddu> mhroncok: We could hardwire the mediawriter to look at the new location for f33 release
17:48:38 <adamw> i suspect doing a release which literally added a .x to the release number would cause all kinds of fun
17:48:43 <adamw> the more places we actually put that .1 in, the more fun
17:48:49 <mattdm> adamw++
17:48:49 <zodbot> mattdm: Karma for adamwill changed to 12 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:48:54 <mattdm> it's a QE dream, isn't it :)
17:48:58 <nirik> could be.
17:49:01 <bcotton_> okay, i'm not seeing any nacks to my proposal
17:49:07 <adamw> remember when we had a unicode character and a space in the release name, and that was so bad we stopped doing release names? yeah like that only worse. :P
17:49:08 <x3mboy> I'm out of my boundaries here, but..... Do we know how many users are using secure boot?
17:49:14 <adamw> x3mboy: no.
17:49:14 <bcotton_> so i think we'll call that a decision made, if not loved
17:49:23 <x3mboy> It will serve to measure the real affectation
17:49:30 <mhroncok> so much more work for QA and releng and docs and websites and whatnot. isn't it worth slipping a bit?
17:49:31 <x3mboy> I'm not using it, e.g.
17:49:56 <mattdm> mhroncok maybe! but it's a hard call that we just already ack'd
17:50:05 <mboddu> frantisekz: Id didn't get your question " even if it's F31.x ? can't this be done?" ?
17:50:10 <mhroncok> right, sorry for bringing it up again
17:50:12 <bcotton_> #agreed 1883609 - RejectedBlocker (Final) - There's no clear consensus, but the net vote is "not a blocker". The FPgM will work with appropriate parties to develop a schedule for a potential point release that contains a newly-signed shim
17:50:27 <bcotton_> okay, so
17:50:32 <bcotton_> with that decided
17:50:36 * nirik will make a releng ticket to track things we think will break with a point release.
17:50:39 <frantisekz> mboddu: I mean, if we do it as another release, let's say 31.5, can't we put that into media writer/ getfedora?
17:50:41 <bcotton_> #info Zero accepted blockers outstanding
17:50:50 <mattdm> mhroncok: it's okay to register your concern :)
17:50:59 <bcotton_> #action bcotton to assemble a super team to figure out a 33.1 schedule
17:51:06 <mboddu> frantisekz: Okay, as I said earlier, we need to hard code that logic into media writer
17:51:18 <frantisekz> okay, thanks
17:51:29 <adamw> frantisekz: the answer to "can we make media writer find it" is already "yes it's just more work"
17:51:37 <adamw> and also "work that we don't exactly have a precedent for how to do"
17:51:57 <mattdm> mboddu: do we? doesn't it get that from a json file?
17:52:08 <bcotton_> and a sincere thank you to everyone who offered their thoughts here. this was a hard decision with lots of good arguments in all directions and i'm really proud of how well we did it
17:52:20 <mattdm> yes, what bcotton_ said. thanks everyone
17:52:23 <bcotton_> y'all make me proud to be a part of Fedora
17:52:23 <adamw> yes. a json file that is produced via fedfind and this process kind of has a hardwired assumption that there is one "fedora 33" that is in one place
17:52:33 <bcotton_> so. before i get too sentimental
17:52:36 <adamw> because that's how it has always been
17:52:51 <bcotton_> #topic Current status - RC
17:52:54 <adamw> we can break that assumption in various places and enter a new exciting world of "oh hey we didn't think about THAT"
17:52:56 <bcotton_> We have one!
17:53:02 <mattdm> YAY
17:53:12 <adamw> we have one, it was built monday, it's been tested.
17:53:18 <bcotton_> #info RC1.2 is the current release candidate
17:53:24 <adamw> no-one can absolutely *prove* it has eaten any babies.
17:53:27 <bcotton_> mboddu: did this one FINISHED?
17:53:28 <mboddu> mattdm: Yeah, I think so, but even that needs some tweaking
17:53:30 <bcotton_> adamw++
17:53:31 <mboddu> bcotton_: Yes
17:53:35 <mboddu> No missing images
17:53:41 <mattdm> amazing
17:53:42 <King_InuYasha> I just installed it this morning and used it for a btrfs demo
17:53:43 <bcotton_> #info RC1.2 FINISHED - no missing images!
17:53:45 <King_InuYasha> it worked great
17:53:49 <bcotton_> that's a first for us, yeah?
17:53:58 <adamw> first for a release compose, yeah.
17:54:00 <bcotton_> at least since i've been in this seat
17:54:06 <King_InuYasha> since we started allowing images to fail out for GA, yeah
17:54:46 <bcotton_> see? the sun is shining, birds are chirping, composes are finishing. life is great
17:54:57 <bcotton_> anything else about the RC itself before we move on to testing?
17:55:02 <bcotton_> test coverage, that is
17:55:09 * King_InuYasha stares into the eyes of bcotton_ with a dead stare
17:55:20 <adamw> nothing i can think of
17:55:29 <mattdm> lol
17:55:45 <adamw> oh, the arm netinsts are technically over size
17:55:56 <adamw> you know, just in case anyone wanted to write them to a CD and put them in their ARM desktop's CD drive
17:55:57 <bcotton_> the best kind of over size?
17:56:06 <mboddu> Lol
17:56:08 <adamw> i am personally not losing sleep over this proposition
17:56:13 <sgallagh> Haha
17:56:17 <bcotton_> we'll live
17:56:23 <King_InuYasha> I think I can take the hit :P
17:56:44 <bcotton_> #topic Current status - test matrices
17:56:45 <bcotton_> #link https://fedoraproject.org/wiki/Category:Fedora_33_Test_Results
17:56:48 <lruzicka> adamw, this is how I install my 1000 arm machines
17:57:09 <King_InuYasha> we all know that ARM desktops are not real :)
17:57:13 <lruzicka> adamw, I even create an extra CD for each one
17:57:46 * sgallagh calls a psychiatrist for lruzicka
17:57:47 <adamw> coverage looks great
17:57:47 <coremodule> me too, lruzicka taught me that's the most efficient way
17:58:04 * King_InuYasha calls a psychiatrist for coremodule
17:58:24 <adamw> we're missing two of the active directory tests, but if joining via cockpit works, and joining via kickstart works for freeipa, chances of either of those two failing are v. small
17:58:36 <pwhalen> IoT is also covered - https://fedoraproject.org/wiki/Test_Results:Fedora-IoT_33_RC_20201020.0_General#ISO_Install
17:58:38 <adamw> everyone remember to blame sgallagh for that, it is all sgallagh's fault
17:58:38 <frantisekz> it's not that long ago when we were technically blocking on cd burning on arm though...
17:58:51 <King_InuYasha> I'm not going to lose sleep over Active Directory
17:59:08 <adamw> frantisekz: i don't think we did, the 'write a physical disc' test was for intel arches only, arm had (has) its own table
17:59:09 <sgallagh> King_InuYasha: I literally did, so you don’t have to!
17:59:12 <King_InuYasha> :D
17:59:14 <lruzicka> thank you, sgallagh, I think I still have those 6 free sessions
17:59:50 <frantisekz> adamw: basic app functionality, xfce was blocking desktop on arm and it did include cd burning app by default :D
18:00:01 <adamw> oh, heh.
18:00:15 <adamw> well, obviously you want to use your arm desktop to burn CDs for your x86 desktop. duh
18:00:20 <frantisekz> :)
18:00:27 <bcotton_> 1000 of them in parallel
18:00:39 <adamw> so yeah, coverage is pretty much 100%, those two AD tests missing, and real printer printing and desktop notification on ARM
18:00:48 <sgallagh> I actually did use an RPi at one point to control a bank of burners...
18:00:49 <adamw> but we have both those for x86_64 and there's no reason to imagine they'd be broken on ARM
18:01:29 <bcotton_> #info coverage is pretty much 100%. two AD tests missing, and real printer printing and desktop notification on ARM (but they work on x86_64 so there's no reason to think they wouldn't on ARM)
18:01:43 <bcotton_> any other questions or comments on the test coverage?
18:02:41 <King_InuYasha> nah, looks great
18:02:48 <bcotton_> okay, here comes the fun part
18:03:08 <bcotton_> everyone put on your thick glasses and your 1960s buzzcuts, we're going to go around the horn
18:03:10 <bcotton_> #topic Go/No-Go decision
18:03:12 <bcotton_> I will poll each team. Please reply “go” or “no-go”
18:03:15 <bcotton_> FESCo?
18:03:19 <nirik> go
18:03:21 <mhroncok> go
18:03:28 <sgallagh> go
18:03:49 <bcotton_> #info FESCo is GO
18:03:53 <bcotton_> RelEng?
18:04:03 <mboddu> Reluctant go
18:04:05 <mboddu> :D
18:04:18 <bcotton_> #info RelEng is Go
18:04:25 <mboddu> (But, we are shipping a "FINISHED" compose, so I am very happy about it)
18:04:29 <bcotton_> (see, i only capitalized one of the letters)
18:04:32 <bcotton_> QA?
18:04:33 <frantisekz> go
18:04:47 <sumantro> go
18:04:47 <lruzicka> go
18:04:59 <bcotton_> #info QA is GO
18:05:02 <bcotton_> #agreed Fedora 33 Final is GO
18:05:03 <bcotton_> #info Fedora 33 Final will release on 2020-10-27
18:05:11 <bcotton_> #action bcotton to announce decision
18:05:19 <bcotton_> #topic Open floor
18:05:21 <bcotton_> Anything else we need to discuss before closing?
18:05:31 <King_InuYasha> nope
18:05:34 <adamw> YOUR IMPENDING DOOM
18:05:34 * King_InuYasha is exhausted
18:05:40 <adamw> ...wait no that's a different channel my bad
18:05:42 <nirik> I thought of another way we could do a point release that would break things in a different way...
18:05:52 <adamw> nirik: we call that progress!
18:06:02 <nirik> just do f34 for the updated shim release. :)
18:06:05 <mhroncok> we could release 34 in couple weeks and rebrand 34 to 35
18:06:08 <bcotton_> too bad we don't still do code names, "Your impending doom" would be a great name for F33.1
18:06:10 <cmurf> adamw :P
18:06:11 <nirik> and call the current f34 f35
18:06:20 <nirik> mhroncok: yep. ;)
18:06:29 <cmurf> oooo
18:06:37 <bcotton_> nirik, mhroncok: devious. i love it
18:07:04 <cmurf> why have a dot release when we can have a WHOLE ONE
18:07:14 <King_InuYasha> can we have codenames again with new-world releases :D
18:07:22 <Southern_Gentlem> no
18:07:24 <x3mboy> Is there a GA link?
18:07:25 <King_InuYasha> aww
18:07:38 <x3mboy> I need to start working on Screenshots with mktg
18:07:45 <mattdm> x3mboy++
18:08:03 <mhroncok> x3mboy: make sure shim is not visible on the screenshot please
18:08:14 <adamw> nirik: yeah, I was thinking approx the same
18:08:14 <mboddu> nirik: Please no, it will be crazy to work map the distag, branching, rebuilding....
18:08:19 <bcotton_> i actually kind of the like the "very small F34" option and i'm not sure how i feel about that
18:08:35 <bcotton_> so last call to give me very confusing feelings
18:08:37 <x3mboy> mhroncok, I will
18:08:48 <nirik> yeah, it breaks other things. ;)
18:09:22 <adamw> bcotton: hold on, let me dig out some pictures of 1990s-era brian molko, that might do the trick
18:09:33 <mboddu> It will break in such a way that it will take the entire F34 cycle to fix things
18:09:45 <bcotton_> mboddu: F35 cycle ;-)
18:09:49 <mboddu> Right
18:09:51 <nirik> mboddu: you mean f35... the f34 cycle would be really short then
18:09:59 <bcotton_> okay, well i'm going to bang the gavel then
18:10:04 <bcotton_> thanks again everyone for being so awesome
18:10:11 * King_InuYasha dies
18:10:22 <x3mboy> See ya
18:10:24 <bcotton_> #action bcotton to plan King_InuYasha's funeral
18:10:27 <bcotton_> #endmeeting