18:00:22 #startmeeting F32 Beta Go/No-Go meeting 18:00:22 Meeting started Fri Mar 13 18:00:22 2020 UTC. 18:00:22 This meeting is logged and archived in a public location. 18:00:22 The chair is bcotton. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:22 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:22 The meeting name has been set to 'f32_beta_go/no-go_meeting' 18:00:23 #meetingname F32-Beta-Go_No_Go-meeting 18:00:23 The meeting name has been set to 'f32-beta-go_no_go-meeting' 18:00:28 #topic Roll Call 18:00:42 * coremodule is here 18:00:42 morning 18:00:52 .hello2 18:00:53 sgallagh: sgallagh 'Stephen Gallagher' 18:01:39 .hello mohanboddu 18:01:41 mboddu: mohanboddu 'Mohan Boddu' 18:01:44 Hello :) 18:02:03 okay, we're well represented, so let's get started! 18:02:15 i'll skip the boilerplate, y'all remember it from yesterday :-) 18:02:23 #topic Current status - blockers 18:02:24 #link https://qa.fedoraproject.org/blockerbugs/milestone/32/beta/buglist 18:03:27 looks like we verified the one accepted blocker and one of the proposed blockers 18:03:38 .hello adamwill 18:03:38 1813237 would be a proposed previous release blocker, FTR 18:03:40 adamw: adamwill 'Adam Williamson' 18:03:56 #info BZ 1798792 is an accepted blocker, but is VERIFIED 18:04:19 #info BZ 1812026 is a proposed blocker, but is VERIFIED 18:04:33 #topic (1813237) list of installed modules is not remembered across reboots 18:04:35 #link https://bugzilla.redhat.com/show_bug.cgi?id=1813237 18:04:36 #info Proposed Blocker, PackageKit, MODIFIED 18:04:57 so this was discovered overnight, but i believe it goes to previous release, yes? 18:05:15 yeah. we could probably just close it and handle it as part of the existing acceptedpreviousrelease blocker 18:05:35 anyone opposed to this? 18:06:28 +1 18:06:35 #agreed We will handle BZ 1813237 as part of the exisiting AcceptedPreviousRelease blocker 18:06:37 so! 18:06:57 #topic (1804564) Cannot upgrade to Fedora 32: Modules blocking the upgrade path 18:06:58 #link https://bugzilla.redhat.com/show_bug.cgi?id=1804564 18:06:59 #info Accepted Previous Release Blocker, PackageKit, ON_QA 18:07:43 We have a fix for this that we've tested via COPR builds 18:07:45 PackageKit-1.1.12-14.fc31 works fine, I just checked 18:07:49 Official builds are in-progress 18:08:01 kparal is fast, that just completed 10 minutes ago :) 18:08:21 #info PackageKit-1.1.12-14.fc31 works as expected 18:08:35 is there an fc30 build, too? 18:09:07 bcotton: Currently building 18:09:19 There was a glitch with buildroot overrides 18:09:28 okay, have we tested it unofficially at least? 18:09:52 adamw, kparal ? 18:10:13 bcotton: Can we go with "we absolutely will know for sure by Monday"? 18:10:29 yeah. if kparal didn't test yet, we will shortly 18:10:35 sgallagh: maybe, but what happens if the sure answer is "nope, still broken"? 18:10:55 then we are all slip city. ;) 18:10:56 Suggest that people do a stepped upgrade instead of jumping direct and fix that for GA? 18:11:01 I have tested only the previous broken packagekit on FC30 18:11:14 i don't love it, but that's a not-terrible thing for beta, i suppose 18:11:15 which worked, just had some flaws we had to fix 18:11:41 The fixes for the flaws are identical to the fixes for F31, so I have a high confidence level 18:11:49 #info Fedora 30 package is in the works 18:12:08 It just finished building, kparal 18:12:15 If you want to test it during this meeting :) 18:12:20 work, work 18:12:31 zug zug 18:12:35 proposed #agreed Fixes for BZ 1804564 are sufficient and will be in the stable repos in time for the release 18:12:51 ack 18:12:58 sgallagh: I wondered if anybody gets that in text only form :) 18:13:11 sgallagh: thanks much for sheparding that fix! 18:13:32 You're welcome. 18:13:35 ack 18:13:40 (I am never doing that again!) 18:13:48 he lied 18:13:49 sgallagh++ to never doing that again :-D 18:13:56 sgallagh: Until f32 final :P 18:14:10 #agreed Fixes for BZ 1804564 are sufficient and will be in the stable repos in time for the release 18:14:50 #topic (1767351) Cannot upgrade to Fedora 32: Modules blocking the upgrade path 18:14:52 #link https://bugzilla.redhat.com/show_bug.cgi?id=1767351 18:14:53 #info Accepted Previous Release Blocker, dnf-plugins-extras, MODIFIED 18:15:13 i think this one is solved, but the package push changed the state deceptively 18:16:30 well, the f30 update was never pushed stable yet 18:16:37 even the earlier one for the dnf upgrade side only 18:16:40 it just didn't get enough karma 18:16:50 no one uses f30 anymore. ;) 18:16:50 still, we're pretty sure the fixes for that side of things are good 18:16:57 we'll re-test and karma and stuff, but i'd say this is addressed 18:17:13 proposed #agreed Fixes for BZ 1767351 are sufficient and will be in the stable repos in time for the release 18:18:26 ack 18:18:54 ack 18:18:55 ack 18:19:34 #agreed Fixes for BZ 1767351 are sufficient and will be in the stable repos in time for the release 18:19:58 #topic Current status - blockers 18:20:08 #info All outstanding blockers are in an acceptable state 18:20:15 #topic Current status - test matricies 18:20:17 #link https://fedoraproject.org/wiki/Category:Fedora_32_Test_Results 18:20:28 so, we've had another day with Beta 1.2. How do we feel about it? 18:21:28 Best Fedora Beta Ever. Five Stars. Would run it again. 18:21:51 better than cats! 18:21:57 Haha :) 18:22:06 well now let's not be *hasty* 18:22:07 nirik: the animal or the musical? 18:22:07 Have you seen the reviews of that movie? That's a damned low bar... 18:22:09 better than dogs, sure 18:22:22 adamw++ 18:22:25 the musical... 18:23:01 anyhow... 18:23:18 adamw, kparal: How's the coverage? 18:23:24 i'm just looking at it now 18:23:30 please talk among yourselves 18:23:47 I spent the whole day breaking packagekit 18:24:10 I'm glad he pluralized "yourselves". A lot of people forget that and the implications are mind-boggling and disturbing. 18:25:25 not having any ec2 xen coverage makes me v. slightly sad 18:25:34 i don't recall if we committed to having any in that whole discussion 18:25:58 sgallagh: today's pet grammar peeve: people who start "from X to Y" clauses but don't finish them 18:26:01 iirc, we decided to try both xen and kvm based ec2, but that was a long time ago 18:26:05 GIVE ME THE 'TO', DAMNIT 18:26:22 they're both in the matrices, i just don't recall how strongly we committed to testing both and when 18:26:40 coremodule: any chance you could quickly test on a xen instance? 18:26:54 ahhhhhh 18:26:57 ummmm 18:26:59 lemme see 18:27:45 isn't xen always a pain in our backside? 18:27:57 No, sometimes it's a pain all-over. 18:28:07 despite guarantees that the xen folks will make sure to test it each release 18:29:24 coremodule: more or less, but this was the resolution to that 18:29:32 we decided we only care about xen in the context of ec2 instances that are backed by xen 18:29:45 we stopped caring about literally installing a xen dom0 on a fedora box and then booting a domU on it 18:30:00 ec2 xen instances *should* be less painful. i think. 18:30:07 (as amazon gets to deal with most of the xen fun.) 18:30:54 aside from that, we're looking pretty good 18:31:14 there's just a *few* holes in ARM coverage, but nothing i'd lose sleep over, only a few things and there's no reason they'd differ from x86_64 really 18:31:27 a couple of upgrade configs, and printing with a real (not virtual) printer 18:32:47 #info Test coverage is "looking pretty good" apart from a few gaps in ARM coverage and EC2 Xen coverage (which may be done by the time the meeting ends) 18:33:11 any questions, comments, or concerns on test coverage? 18:34:00 adamw: sgallagh: I tested F30->F32 upgrade, works fine, with my "test suite" 18:34:05 kparal++ 18:34:17 cool 18:34:22 #topic Current status - RC 18:34:26 * kparal going afk now 18:34:27 kparal++ 18:34:28 sgallagh: Karma for kparal changed to 6 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:34:39 So Beta 1.2 still exists, right? It hasn't been stolen in the dead of night? :-D 18:34:49 <.< 18:34:52 >.> 18:35:00 * adamw swallows, looks guilty 18:35:06 i, uh, may have eaten it. 18:35:12 was it tasty? 18:35:17 in my defence it was right there! and i was hungry! 18:35:22 kinda of spiky 18:35:40 hm, that may have been a coronavirus, not an RC 18:35:52 *slight cough* 18:36:01 #info Beta 1.2: still a thing 18:36:04 *panicked vacating of the room* 18:36:09 kparal++ 18:36:09 sumantro: Karma for kparal changed to 7 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:36:24 okay, so i guess we've covered all the bases and now it's time to make a decision 18:36:32 bcotton: I thought of stealing 1.2, but then I decided not to :P 18:36:40 unless we want to stall while coremodule does the ec2 xen test? 18:36:42 in my defence for arm coverage, workstation is new for release blocking this release, so dont think we needed those and didnt have time with all the other testing needed 18:36:44 mboddu: we appreciate it 18:37:02 those being upgrade tests 18:37:30 pwhalen: sorry, yeah, should've mentioned that's a bit unclear 18:37:44 whether we need to/should worry about workstation upgrades 18:37:47 f33, fair game. 18:38:21 ideally, we'd cover that for final, but i would 100% buy the "it's not required because it's never been blocking before" interpretation 18:39:32 #topic Go/No-Go decision 18:39:34 I will poll each team. Please reply “go” or “no-go” 18:39:41 FESCo? 18:39:43 GO! 18:39:43 go 18:39:52 #info FESCo is go 18:39:55 Releng? 18:40:06 I still feel like we should punt the release, but from releng perspective we are GO 18:40:23 #info Releng is go 18:40:52 you said 'go' so it counts, but why do you feel like we should punt? 18:41:44 bcotton: Just last minute sheparding makes me feel we might have missed something, thats all 18:41:58 * bcotton nods 18:42:07 mboddu: you officially have the right to say "i told you so" :-) 18:42:23 But I do appreciate all the people who worked hard in the last couple of days 18:42:24 :) 18:42:50 so that leaves us with QA? 18:42:50 bcotton: I am reserving that right from the last couple of releases (at least 1 IIRC) 18:43:07 mboddu: hold on to them as long as you can and then use them all at once :-) 18:43:33 * adamw sets aside a day to be told so by mboddu 18:44:01 bcotton: If you remember about the missing spins and labs for which I had to rerun the unofficial images... 18:44:19 mboddu: ah yes, that's several ITYSes, i believe :-) 18:44:48 adamw, I think you deserve to say the magical word on QA behalf 18:45:11 we haven't automated that yet? 18:45:12 frantisekz: take his power 18:45:13 * nirik runs 18:45:35 =) 18:45:40 (hmm, after missing most of the meeting... I'd say go :D ) 18:45:53 #info QA is go 18:45:55 :-D 18:45:57 :O 18:46:05 i have similar reservations to mboddu, but on policy, we're go 18:46:16 and at least i haven't seen any weirdness in manual or openqa testing with 1.2 18:46:23 I'd like to avoid this late minute testing and fixing for final... 18:46:27 nothing specific that makes me nervous. we don't have any weird feedback on updates or lists 18:46:34 yeah, i would not want to do final this way 18:46:45 so here's the thing: the policy is more like...guidelines 18:46:47 but for beta i think we'll go 18:46:55 bcotton: no, the QA team policy is...rules 18:46:56 =) 18:46:58 Yeah, I am definitely saying "hard no" for final if we do like this 18:47:14 for sure, we will not be doing this for final. and it will take a lot of cash payments to get me to do this for the next few betas :p 18:47:26 okay, well since no one is willing to pull the ejection handle... 18:47:36 #agreed Fedora 32 Beta is GO 18:47:47 * mboddu goes to work 18:47:48 #info Fedora 32 Beta will release on 2020-03-17 18:47:55 flip those bits, mboddu! 18:48:06 #action bcotton to announce decision 18:48:12 #topic Open floor 18:48:13 Anything else we need to discuss before closing? 18:48:16 bcotton: That has to wait until Monday :) 18:48:59 thanks for all the hard work folks 18:49:34 yes, thank you everyone! 18:49:52 sgallagh: it's go home time. tell pfrields i'll fight him if he objects 18:50:06 there's 'what do we do about IoT and silverblue' i guess 18:50:25 adamw: Oh right, good question 18:52:04 iot has been tested as well https://fedoraproject.org/wiki/User:Pwhalen/QA/IoT/Fedora-IoT-32-20200312.0 18:52:22 pwhalen++ 18:52:22 coremodule: Karma for pwhalen changed to 4 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:52:29 i note the empty column for igniton 18:52:36 is that working yet? 18:52:47 does ignition need a remix? 18:52:53 * bcotton couldn't help himself 18:53:10 .fire bcotton 18:53:10 adamw fires bcotton 18:53:28 it should work, but I've not yet tested it. Been frantically trying to get through it all. 18:53:55 during our test day we lost chronyd, which I think is why it failed. 18:54:48 I need to also write a test case for it, which I'll do today 18:55:39 rgr 18:56:23 pwhalen++ 18:56:23 bcotton: Karma for pwhalen changed to 5 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 18:56:35 wow, how have i not enkarma'ed pwhalen already? 18:57:46 okay, so are we done here? 18:57:57 silverblue? 18:58:03 USS Ship It 18:58:13 (assuming for iot the plan is 'ship that nightly if it passes the rest of the tests', i guess) 18:58:15 sliverblue missed the boat, i guess 18:58:16 * cmurf says that most every release at some point 18:58:19 chrony is long back, there's been a bunch of fixes for the install/provisioning in the last few days 18:58:21 cmurf++ 18:58:49 btw, that iot build is probably missing the bits we pulled into beta? 18:58:51 like the updated kernel? 18:58:52 cmurf, what about HMS Ship It for those members of the commonwealth? 18:58:53 well, there are nightlys for silverblue? or did those fail as well? 18:59:10 today's build has the new kernel 18:59:11 nirik: sure, if the plan is 'ship a nightly' that's fine, was just thinking it might be good to write down what the plan is... 18:59:23 indeed. 18:59:32 pbrobinson: right, so, there's a thing: do we re-test and ship that? 18:59:34 right adamw, today's compose has it, and I will repeat testing on that now 18:59:36 adamw: it's on my list, I've been absolutely smashed the last 2 weeks 18:59:53 or wait till we push everything that was sideloaded to the beta stable, then test and ship the next nightly after *that*, for IoT and SB? 19:00:21 coremodule: haha UMM HMMM, not sure I want to extend the metaphor too far :D 19:00:24 (i will try and get a stable push request for that done today, it depends if everything has sufficient karma yet, if not i'll be bugging people for karma) 19:00:30 and if we do ship a nightly do we actually put it somewhere, or do we just say 'go find a nightly and use that' ? 19:00:34 right 19:00:35 most of the sideloading was mostly gnome? Or was there others? 19:00:40 pbrobinson: i don't remember... 19:00:54 of course, GNOME is sort of significant for SB :P 19:01:04 https://pagure.io/releng/issue/9319 has the list 19:01:06 right, and irrelevant for IoT 19:01:50 Silverblue can catch up asynchronously when a successful compose happens? Or not? 19:02:08 I'm happy with today's or doing another run/test when the other bits land 19:02:46 one important point... if we want a compose with just beta stuff in it, we should NOT open the flood gates on updates stable f32 until after composes tonight. 19:02:54 cmurf: There wont be an image, but they can update based on nightly branched commits 19:02:56 nirik: right. 19:03:02 it's not hard to do a compose using the sideload tag, I meant to do that yesterday and then got dragged backwards into another slew of meetings 19:03:25 mboddu: ok thanks 19:04:49 I guess once we have the final stable push request, we can run a branched compose, IOT compose and people can submit them as unofficial beta images 19:05:03 i'd say: me and mboddu will get a stable push done today, then we won't push anything else and wait for the nightly compose, that should get us a silverblue, IoT can do a compose off that or whatever 19:05:13 yeah, that 19:05:16 WFM 19:05:17 adamw: +1 19:05:30 mboddu: or that. :) 19:05:59 Haha :) 19:06:17 +1 19:06:39 you guys remember to turn off kparal? he's gonna find another blocker today if you haven't shut him down 19:07:17 cmurf: too late, once we're go he can't derail us 19:07:30 lol 19:07:52 only 52 minutes left in Czech beer hours, maybe he's busy anyway, lightweight tho he is 19:08:20 okay, that's all the awkward i had 19:08:25 haha 19:08:29 back to debugging mysterious aarch64 machines 19:10:02 alright, time to bang the gavel then 19:10:04 #endmeeting