17:00:10 #startmeeting F32 Beta Go/No-Go meeting 17:00:10 Meeting started Thu Mar 12 17:00:10 2020 UTC. 17:00:10 This meeting is logged and archived in a public location. 17:00:10 The chair is bcotton. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:10 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:10 The meeting name has been set to 'f32_beta_go/no-go_meeting' 17:00:11 #meetingname F32-Beta-Go_No_Go-meeting 17:00:11 The meeting name has been set to 'f32-beta-go_no_go-meeting' 17:00:22 #topic Roll Call 17:00:25 morning 17:00:28 * pwhalen is here 17:00:29 * coremodule is present. 17:00:34 * sumantro is here 17:00:54 .hello adamwill 17:00:55 adamw: adamwill 'Adam Williamson' 17:01:05 * nirik is here, but also dealing with an outage. 17:01:18 is that why pagure's acting weird? 17:01:25 yes 17:01:37 .hello2 17:01:38 frantisekz: frantisekz 'FrantiĊĦek Zatloukal' 17:02:09 Hello 17:02:15 .hello mohanboddu 17:02:16 mboddu: mohanboddu 'Mohan Boddu' 17:02:42 .hello kalev 17:02:43 kalev: kalev 'Kalev Lember' 17:03:04 * mhroncok is here, but watching amovie. ping me when needed 17:03:15 mhroncok: ack 17:03:37 okay, let's start the process 17:03:46 #topic Purpose of this meeting 17:03:47 #info Purpose of this meeting is to check whether or not F32 Beta is ready for shipment, according to the release criteria. 17:03:49 #info This is determined in a few ways: 17:03:51 #info 1. No remaining blocker bugs 17:03:53 #info 2. Release candidate compose is available 17:03:54 #info 3. Test matrices for Beta are fully completed 17:04:03 #topic Current status - blockers 17:04:04 #link https://qa.fedoraproject.org/blockerbugs/milestone/32/beta/buglist 17:04:19 hokay 17:04:39 let's start with the accepted blockers, since if they still exist, there's no need to review the proposed blocker 17:04:41 s 17:04:49 #topic (1807661) Display corruption on aarch64 virtual machines 17:04:50 #link https://bugzilla.redhat.com/show_bug.cgi?id=1807661 17:04:52 #info Accepted Blocker, kernel, MODIFIED 17:05:03 fixed in Beta 1.2 17:05:10 pwhalen: yay 17:05:21 #info BZ 1807661 is fixed in Beta 1.2 17:05:44 note here 17:05:50 adamw: go ahead 17:05:52 if you look at openqa results you may see some failures still looking like this bug 17:06:24 don't worry about that, it's due to boring details of how the test process works and is because those tests install what's on the mirrors at present, not the beta candidate ocmpose, so they're booting to an older kernel that still has the bug 17:06:47 once the fixed kernel is pushed stable and we do a nightly compose that will be ok. 17:06:55 it's not a bug in the images. 17:07:18 #info openqa results will wrongly show failures for this bug until the fixed kernel is pushed to stable and a nightly compose is run. it is not in the images 17:07:29 NOTABUGs are my favorite bugs :-) 17:08:01 anything else on BZ 1807661? 17:08:35 sounds like a no 17:08:38 #topic (1798792) blivet.errors.DeviceTreeError: failed to add slave root00p2 of device root00 17:08:39 #link https://bugzilla.redhat.com/show_bug.cgi?id=1798792 17:08:41 #info Accepted Blocker, python-blivet, MODIFIED 17:09:56 this one appears to have a fix (python-blivet-3.2.0-3.fc32). is it confirmed an in Beta 1.2? 17:10:20 not yet, sorry 17:10:31 i guess lnie didn't get to it and i'm still triangulating stuff, haven't got onto running tests yet 17:10:36 i can try to confirm it shortly 17:10:51 so it's in Beta 1.2 and we just need to confirm it? 17:10:58 yes 17:11:07 the intended fix is in the Beta, we just haven't confirmed it yet. 17:11:24 #info the intended fix is in Beta 1.2, but has not been confirmed yet 17:11:58 adamw: if we move on to the rest of the process, will you be able to do a confirmation before it's time to make a decision? 17:12:20 (where "you" is anyone on QA, obi 17:12:26 obviously* 17:12:28 um 17:12:41 this is going to be a sort of theme today: we're not quite done with all the testing 17:12:54 i don't know if we're gonna get ALL of it done during this meeting. it depends how long the meeting is and stuff. :) 17:13:01 so stall, got it :-) 17:13:29 send in the clowns! 17:13:40 i'm already here 17:13:59 okay, so let's move on to the proposed blockers and see what time that buys us 17:14:08 or it may render the delay moot 17:14:19 #topic (1812026) No visible cursor in Workstation Live 17:14:21 #link https://bugzilla.redhat.com/show_bug.cgi?id=1812026 17:14:22 #info Proposed Blocker, gnome-shell, MODIFIED 17:14:43 this doesn't seem to happen on beta 1.2 17:14:56 this is fine in 1.2 17:14:58 frantisekz: great, thanks for checking 17:15:21 #info BZ 1812026 is fixed in Beta 1.2 17:15:22 please set it to VERIFIED then 17:15:33 y'all are really bad at stalling 17:16:21 adamw, done 17:16:24 #topic (1811800) Multiple packages have broken dependencies due to PostgreSQL 12 17:16:25 #link https://bugzilla.redhat.com/show_bug.cgi?id=1811800 17:16:26 #info Proposed Blocker, postgresql, ASSIGNED 17:16:29 -1 17:16:37 mhroncok: was this something you wanted pinged for? 17:16:53 -1 beta blocker 17:16:54 no, I agree that this does not violate any criteria 17:17:26 okay 17:17:32 -1 beta blocker 17:17:33 -1 Beta Blocker 17:17:41 -1 beta blocker 17:17:49 now's the opportunity for someone to make a compelling argument that this should be a blocker 17:17:56 -1 BB 17:18:33 -1BB 17:18:55 -1bb 17:19:24 #agreed BZ1811800 is rejected as a blocker as it does not violate any release criteria 17:19:33 this should be a blocker because compelling argument 17:19:39 coremodule++ 17:19:39 bcotton: Karma for coremodule changed to 3 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:19:43 lol 17:19:58 yes, let's debate this for *looks at watch* the next 60 minutes or so 17:20:04 at least! 17:20:18 #info BZ1811800 was accepted as a Prioritized Bug this week 17:20:26 well the compelling argument is: the contingency could not happen at the time of contingency, becasue the tracker bug was set CLOSED 17:20:42 noooooooooooooooo 17:21:03 but I don't care :/ 17:21:24 yeah, that's a bit of a process failure that i own 17:21:51 * bcotton has a plan for fixing^W mitigating it in F33+ 17:22:01 * mhroncok crawls back under his rock 17:22:28 i'd be open to the idea of FESCo deciding it's a final blocker, fwiw, but it pretty clearly doesn't meet any criteria 17:22:36 mhroncok: they have movies under rocks now? 17:23:00 bcotton: they do in here :) 17:23:07 #topic Current status - blockers 17:23:39 #info 1 accepted blocker has a confirmed fix, 1 proposed blocker has a confirmed fix, 1 accepted blocker has a possible fix that is being tested as we speak 17:23:52 * sgallagh arrives late 17:23:57 anything else we want to say as blockers? have any last-minute blockers been proposed? 17:24:04 "as" -> "about" 17:24:18 previous release blockers bcotton? or are you planning to go through that? 17:24:27 bcotton: I think both versions of that sentence are accurate :) 17:24:32 frantisekz++ 17:24:36 frantisekz: good call 17:25:02 #topic (1767351) Cannot upgrade to Fedora 32: Modules blocking the upgrade path 17:25:02 :) mhroncok pinged me off irc not to forget :D 17:25:03 #link https://bugzilla.redhat.com/show_bug.cgi?id=1767351 17:25:05 #info Accepted Previous Release Blocker, dnf-plugins-extras, ON_QA 17:26:01 looks like this one has an update, which I assume went into Beta 1.2? 17:26:30 it doesn't need to be in beta 1.2 17:26:32 no, only has to be in stable updates for the old releases 17:26:35 bcotton: Previous release blockers go to... previous releases 17:26:37 having it in F30 and F31 is enough 17:26:38 oh duh 17:26:54 we need karma for f30 i think 17:26:58 f31 should be gone or ready to go 17:27:17 adamw: I dunno... I'm not sure I want to get what I deserve from that one... 17:27:38 #info FEDORA-2020-717d521d35 (F31) is waiting for a stable push 17:27:55 #info FEDORA-2020-02ee4b1a1c (F30) is in testing and needs karma 17:28:21 #help testing & karma needed for update FEDORA-2020-02ee4b1a1c 17:29:36 * adamw discovers that he neglected to run ethernet to his testing box 17:29:40 this is all working out great so far! 17:30:30 so for our purposes, do we want previous release blockers to be at least on their way to stable before giving a thumbs up? 17:30:50 bcotton: yes please 17:31:32 okay, so if someone can test this and give it one more +1, then it's good to go, but... 17:31:45 #topic (1804564) Cannot upgrade to Fedora 32: Modules blocking the upgrade path 17:31:47 #link https://bugzilla.redhat.com/show_bug.cgi?id=1804564 17:31:48 #info Accepted Previous Release Blocker, PackageKit, ASSIGNED 17:31:53 naturally this ethernet cable needs to run behind a bookcase i've screwed into the wall 17:32:05 so, this is also prevrel, which means it doesn't need to go in the compose 17:32:22 we *do* need someone to say 'hey this is totally gonna be fixed by monday' though. which...hasn't happenedf 17:32:45 is nayone sitting within poking range of hughsie? 17:32:51 yeah, there appears to be no action on it at the moment 17:33:17 Last I heard, hughsie had abandoned this project. 17:33:26 I think kalev is most on-point here. 17:34:59 well that seems like a problem 17:35:17 * adamw fishes ethernet cable with a toilet-cleaning brush 17:35:21 this is all totally fine and professional 17:35:31 kalev: any chance you're around? 17:35:53 I am, but I haven't worked on packagekit lately. hughsie seemed to take on this bug reading the ticket, I'd suggest asking him 17:36:21 kalev: okay, thanks 17:36:54 well i think we can safely say this will not be fixed before monday 17:37:33 #action bcotton to check with hughsie on plan for BZ 1804564 17:37:55 #topic Current status - blockers 17:38:06 bcotton: i dunno, it shouldn't be too hard. there should be a corresponding change from last cycle. someone else can probably hack it if necessary. 17:39:01 #info BZ 1804564 is a previous release blocker that will likely not be ready in time for target release date 17:39:14 adamw: i'm happy to let someone else do it, too :-) 17:40:03 okay, anything else for blockers (again)? 17:41:01 kalev: Would you be comfortable landing the change if adamw identified what it needs? 17:41:52 sgallagh: pfah, i can just dump it in the package dist-git 17:41:56 no-one gets to tell me no :P 17:41:59 :D 17:42:16 i tried telling adamw "no" once. i got hit with a toilet-cleaning brush 17:42:19 Just pre-empting any complaints 17:42:33 adamw: no 17:42:35 okay, fine, no-one was crazy enough to put me in the group of people who can build bootchain stuff yet. so they get to tell me no. boooo 17:42:47 #topic Current status - test matricies 17:42:49 #link https://fedoraproject.org/wiki/Category:Fedora_32_Test_Results 17:42:55 okay, so, yeah, we're not done yet 17:43:11 compose was done pretty late today to make it in time 17:43:11 well the good news is that you seem to have another week? 17:43:18 * kparal is finishing the KDE install test 17:43:19 i'm working on the usb tests that kamil isn't working on 17:43:29 (hence the toilet brush ethernet fishing) 17:44:00 also will cover hw raid 17:44:07 adamw: just one of the netinst/dvd stuff on uefi should be fine 17:44:24 s/fine/enough 17:44:28 #info Testing is incomplete but ongoing 17:44:39 anyone have a firmware raid with linux support? 17:45:10 we have one in the office, I won't make it there today though 17:45:21 oh, someone needs to run the checksum checks too 17:45:32 someone with shell access to a box with the images on it 17:45:58 arm checksums are good 17:46:09 adamw: I usually ask mboddu 17:46:27 Oh, I can run them 17:46:30 thanks 17:46:37 Sorry, didn't get time to do it 17:46:53 there's a few Base workstation tests that need knocking out 17:47:19 * nirik was hoping to do some test installs on the macbook, but I just haven't had time 17:48:03 anything else that needs to be explicitly called out now? 17:48:38 we haven't done cloud testing in any actual clouds yet 17:48:50 some server coverage is missing for aarch64, i may be able to bodge openqa up to do that later 17:48:55 well the cloud is just other people's comptuers, soo... :-) 17:49:03 most kde tests are missing 17:49:06 adamw, I tested the aarch64 image in aws 17:49:11 #info Cloud testing in actual clouds is missing 17:49:16 pwhalen: sorry, i meant x86_64 17:49:23 patch that bcotton :) 17:49:32 #undo 17:49:32 Removing item from minutes: INFO by bcotton at 17:49:11 : Cloud testing in actual clouds is missing 17:49:32 pssh, who uses x86_64 anymore? ;-) 17:49:37 right? 17:49:39 #info Cloud testing in actual clouds is largely missing 17:50:00 i would've said 'on x86_64' but oaky 17:50:01 so 17:50:03 #info Most KDE tests are missing 17:50:12 status is: we're clearly not done yet. we could probably be done by end of today, US time. 17:50:19 bcotton, I can do the cloud tests after the meeting 17:50:53 #info Testing is not done, but could probably finish by EOD (US time) 17:51:02 so what i'm kind angling towards here is 17:51:08 can we do a 24 hour slip? 17:51:19 if we can do that it would clear up...lots of stuff. 17:51:28 yeah, 24 hours slip would help 17:51:41 that depends. we still have that unfixed previous release blocker looming over us 17:52:02 If QA is comfortable with that (and I assume they are, since they're requesting it), then I suppose it's reasonable 17:52:14 but a 25-hour slip is on the table (i, like an idiot, made plans the day after a go/no-go) 17:52:29 bcotton: that's one of the things i'm hoping to clarify 17:52:35 as in 'can i hack C good enough to fix it' :P 17:52:45 i believe in you 17:52:54 adamw: I'll back you up/do the review for you. 17:53:19 * mboddu also wants to run another rc compose to get silverblue images 17:53:47 but i also don't want to do a short slip unless we're pretty confident we'll be go tomorrow. otherwise, we're putting an undue burden on people (particularly QA and releng) 17:53:50 (and/or 'can i find hughsie and annoy him enough to make him do it') 17:54:03 bcotton: i think there's a pretty reasonable chance. 17:54:11 we haven't found any new blockers yet, coverage isn't *so* terrible. 17:54:15 Probably not if we respin though 17:54:20 (sorry, Silverblue) 17:54:44 okay, well let's go over the RC and then we'll come to a decision on that 17:54:47 no, this would be to ship this candidate 17:54:53 what's up with silverblue? 17:55:03 #topic Current status - RC 17:55:15 #info Beta 1.2 is the current release candidate 17:55:22 what's missing besides silverblue? 17:55:51 adamw: The compose box that I ran for RC is using an older version of pungi which had issues with silverblue composes 17:56:34 ah 17:57:22 Hence, I want to run another rc compose which will get us silverblue images 17:57:42 If we do that, we're committing to a full-week slip. 17:57:59 hmm, with another spin, wouldn't installation tests be enough? 17:58:04 if everything else is same? 17:58:12 * sgallagh looks at adamw 17:58:21 grmph 17:58:41 i feel like 24-hour slip *and* a new compose is kinda pushing things 17:58:48 * sgallagh nods 17:59:10 I request that we either do the 24-hour slip *or* we slip a week and get a new RC 17:59:10 because we had this spin for so long... :-) 17:59:13 do we really need silverblue *in* the beta compose? can we not just point people at a nightly? 17:59:19 *compose 17:59:23 kparal: at least i can do some testing on it today 17:59:26 But... I think policy is that we don't slip for non-blocking media 17:59:32 if we run a new compose it basically knocks out my and tim's and geoff's working day 17:59:32 Which pretty much makes this decision for us 17:59:50 in theory we don't re-compose for non-blocking bugs, right. 17:59:56 we can only re-spin if we have a blocker to fix. 18:00:06 we've broken that rule i think a couple of times for *really bad* FEs. 18:00:17 agreed. i think 1-day and RC is an xor here 18:00:35 I prefer slipping by a week 18:00:49 it's suboptimal for us to not have silverblue in the compose, but it's allowable 18:00:52 Proposal: We slip 24 hours and hope that Beta RC 1.2 passes tests. If not, we respin with blocker fixes and hopefully pick up Silverblue then 18:01:14 mboddu: apart from silverblue, is anything notable missing? 18:01:22 bcotton: Nope 18:01:24 * nirik notes spinning another rc now is dicy, as rabbitmq is blowing up 18:01:38 i'm +1 on that. silverblue *in the beta compose* just doesn't seem that important. 18:01:50 #info Silverblue is missing from Beta 1.2 18:01:52 we'll have silverblue nightlies we can link to from get.fpo or whatever 18:02:07 rabbitmq is blowing up? it must be a day that ends in 'y'! 18:02:23 kparal: everything netinst usb uefi passes 18:02:25 i'll do kde next 18:02:26 #topic Go/No-Go decision 18:02:32 okay, so here's what we're going to do 18:02:40 i'll poll the three teams that get a say in the matter 18:02:48 yay. 18:03:27 reply "go" or "no-go" and if you're "no-go", say if you want a 1-day or 1-week slip. 1-day means we proceed with Beta 1.2 18:03:31 FESCo? 18:03:50 no-go 1-day 18:03:54 #info FESCo is no-go and wants a 1-day slip 18:04:00 Releng? 18:04:03 no-go, whatever works for the packagekit thing 18:04:04 Well, I'm not the only FESCo member here... 18:04:10 * mhroncok was for fesco 18:04:15 mhroncok: ack 18:04:57 any present FESCo members are welcome to object now 18:05:34 no-go, 1 week 18:06:08 #info FESCo is no-go and wants a 1-week slip 18:06:13 QA? 18:06:14 ? 18:06:15 #undo 18:06:15 Removing item from minutes: INFO by bcotton at 18:06:08 : FESCo is no-go and wants a 1-week slip 18:06:22 #info Releng is no-go and wants a 1-week slip 18:06:23 Nope, that was releng 18:06:27 Okay :) 18:06:35 i am having a thursday 18:07:06 bcotton: Haven't we decided that releng should get a request by Tue before go/no-go or else its a week slip or something? 18:07:29 * mboddu remembers talking about it 18:07:35 mboddu: iirc it was proposed and rejected 18:07:43 mboddu, I remember 18:08:09 bcotton: Ah, well, I cant use that as a reason then :( 18:08:30 is it our turn now 18:08:31 ? 18:08:34 adamw: yes 18:08:59 clearly no-go per QA policy. i'd say we can try the 1-day slip as proposed by sgallagh 18:09:11 frantisekz/kparal, wdyt? 18:09:21 no-go 1day slip 18:09:22 also pwhalen and cmurf 18:09:22 will results carry over? 18:09:25 and sumantro :) 18:09:31 1 day slip sounds good for me 18:09:42 pwhalen: i said 1-day slip as proposed by sgallagh, which means no new compose 18:09:46 pwhalen: for a 1-day slip? yes because it's the same RC 18:09:46 so nothing to carry over to 18:09:53 we either ship Beta-1.2 or slip a week and recompose 18:10:03 oh, then no issues here :) 18:10:08 #info QA is no-go and wants a 1-day slip 18:10:29 #agreed Fedora 32 Beta is No-Go 18:10:39 so now we need to #agreed on the duration of the slip 18:10:55 adamw: 1day ack 18:11:08 i'm not going to force releng into a 1-day if they're unwilling 18:11:46 * sgallagh cracks his knuckles 18:11:46 bcotton: We are no-go and at least for 1 day, can we keep this meeting to continue and get back tomorrow and make the final decision? 18:11:58 mboddu: that's what a 1-day slip means... 18:12:07 we come back tomorrow and do this again 18:12:10 Luckily I am not sitting next to sgallagh today 18:12:13 if we're no-go then, we slip for a week (6 days) 18:12:17 at least that'd be my understanding 18:12:24 adamw: you understand correctly 18:12:26 i don't mind either 1 day or 1 week slip, there are advantages/disadvantages either way, is a wash 18:12:31 That matches my understanding 18:13:04 mboddu: i guess it comes down to whether or not you're firmly opposed to a 1-day delay or just prefer to not 18:13:05 adamw: Right, but can we make the decision of either 1 day or 1 week tomorrow? 18:13:31 mboddu: I think you're confused. If we make that decision tomorrow, that *is* a one-day slip 18:13:32 er 18:13:40 It doesn't change the delivery date of Tuesday 18:13:41 yeah 18:13:56 s/delivery/release/ 18:14:31 sgallagh: What if we find a blocker? And also we have the previous release blocker which we are not sure will be fixed by then? I want to punt the decision to tomorrow of when we are going to release 18:14:35 bcotton: ^ 18:14:46 That is literally what I proposed :) 18:14:50 mboddu: that's what the 1-day slip is 18:15:06 calling it a "slip" is probably not the best decision 18:15:12 haha right 18:15:18 ^^ that confused me 18:15:20 yeah. 18:15:21 but it's basically "do we have a go/no-go again tomorrow or a week from now"? 18:15:23 sorry :) 18:15:26 1day delay to decide tomorrow whether to slip a week 18:15:56 Okay, I am okay with 1 day delay in decision, sorry, I got confused 18:16:03 mboddu: sorry for being confusing 18:16:07 I do think we should discuss a policy like the "If no RC by Tuesday, auto slip" to avoid this. 18:16:08 sorry for being sorry 18:16:21 i agree with pwhalen 18:16:23 pwhalen: i'm happy to have that discussion again 18:16:24 adamw: that's the proper canadian way! 18:16:28 pwhalen: we can discuss it again, but maybe not now and here :) 18:16:33 sure 18:17:03 Thanks pwhalen and I can create/reopen the ticket 18:17:29 kparal: haha 18:17:44 okay, so before we call this a final decision, is everyone okay with 1800 UTC tomorrow? (one hour later that our start time today) we'll plan on a 1-hour meeting since it should be pretty obvious either way 18:18:08 * pwhalen is OK with it 18:18:26 wfm 18:18:31 So that's 23:42 from now? 18:18:31 sure 18:18:38 sgallagh: correct 18:18:41 OK 18:18:48 I can probably manage that 18:19:02 Works for me 18:19:05 yep 18:19:10 okay 18:19:41 #agreed We will have another Go/No-Go meeting at 1800 UTC on Friday 13 March to re-evaluate RC 1.2 18:20:22 adamw, sgallagh: let me know as soon as possible if the PackageKit fix won't make it and I'll preemptively cancel the meeting 18:20:28 ack 18:20:31 good idea 18:21:03 adamw: On side note, checksum checks are successful, I will update the test matrix 18:21:11 thanks 18:21:12 * sgallagh goes spelunking 18:21:22 bcotton: will do 18:21:25 #info the 13 March meeting will be canceled if the previous release blocker will not be fixed on time or if any obvious no-go conditions arise 18:21:31 #action bcotton to announce the decision 18:21:51 i'm not sure if being Friday the 13th qualifies as an "obvious no-go condition" in it's own 18:22:00 #topic Open floor 18:22:01 Anything else we need to discuss before closing? 18:22:52 Anyone know where to get a hockey mask by tomorrow? Asking for a friend. 18:23:16 masks are hard to come by those days 18:23:16 Bottom of Crystal Lake. Where it belongs. 18:23:35 okay, sounds like we're set 18:23:51 see you in 37 minutes right here for the release readiness meeting! 18:23:56 thanks everyone 18:23:58 #endmeeting