14:00:07 #startmeeting F31 Final Go/No-Go meeting 14:00:07 Meeting started Thu Oct 24 14:00:07 2019 UTC. 14:00:07 This meeting is logged and archived in a public location. 14:00:07 The chair is bcotton. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:07 Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:00:07 The meeting name has been set to 'f31_final_go/no-go_meeting' 14:00:08 #meetingname F31-Final-Go_No_Go-meeting 14:00:08 The meeting name has been set to 'f31-final-go_no_go-meeting' 14:00:19 .hello mohanboddu 14:00:20 mboddu: mohanboddu 'Mohan Boddu' 14:00:24 #topic Roll Call 14:00:40 .hello2 14:00:41 sgallagh: sgallagh 'Stephen Gallagher' 14:00:43 .hello2 14:00:44 lbrabec: lbrabec 'Lukas Brabec' 14:01:57 * bcotton wonders if adamw set an alarm 14:02:08 * satellit_ listening 14:02:21 * mhroncok around 14:02:54 bcotton: He's a QA engineer. He's probably got alarms ringing constantly 14:03:01 sgallagh++ 14:03:14 #topic Purpose of this meeting 14:03:15 * kparal is partially here 14:03:16 #info Purpose of this meeting is to check whether or not F31 Final is ready for shipment, according to the release criteria. 14:03:26 #info This is determined in a few ways: 14:03:27 #info 1. No remaining blocker bugs 14:03:29 #info 2. Release candidate compose is available 14:03:31 #info 3. Test matrices for Final are fully completed 14:03:43 #topic Current status — Blocker bugs 14:03:45 #link https://qa.fedoraproject.org/blockerbugs/milestone/31/final/buglist 14:04:20 AIUI the 2 proposed blockers have fixes in the compose, but let's go through them real quick-like 14:04:22 .hello2 14:04:23 frantisekz: frantisekz 'František Zatloukal' 14:04:32 #topic (1763875) cut/copy to clipboard sometimes fails 14:04:34 #link https://bugzilla.redhat.com/show_bug.cgi?id=1763875 14:04:35 #info Proposed Blocker, gtk3, ON_QA 14:05:03 adamw said this is fixed: https://bugzilla.redhat.com/show_bug.cgi?id=1763875#c17 14:05:19 has anyone found anything to call him a liar? 14:05:32 nope, looks like he speaks truth :) 14:05:44 i knew we could count on him 14:05:44 bcotton: I can also verify that it is working properly as far as I am able to discern 14:05:45 we don't have much feedback, I guess we'll consider this one fixed 14:06:14 kparal: I installed it last night and haven't had any copy-paste issues today. 14:06:30 proposed #agreed This is fixed in the RC, so the blocker decision is moot 14:06:44 ack 14:06:46 ack 14:07:04 ack 14:07:14 nack 14:07:36 I think we should still declare it a blocker, because if contradictory information comes in, it should impact our decision 14:07:45 *comes in before the decision is made 14:07:57 sgallagh: that's a good point 14:08:04 Hello! 14:08:10 * kparal shrugs. +1 blocker then 14:08:12 anyone -1 to this being a blocker? 14:08:15 Sorry I'm late 14:08:19 welcome, Lailah 14:08:24 fas .lailah 14:08:27 +1 Blocker 14:08:36 .fas lailah 14:08:36 Lailah: lailah 'Sylvia Sánchez' 14:08:40 +1 Blocker 14:08:46 +1 blocker 14:08:46 bcotton: hi, thanks! 14:08:47 .hello2 14:08:48 lruzicka: lruzicka 'Lukáš Růžička' 14:09:05 .hello2 14:09:06 alciregi: alciregi 'Alessio' 14:09:06 * mboddu running checksum test now 14:09:15 mboddu: thanks 14:10:21 proposed #agreed 1763875 - AcceptedBlocker (Final) - This violates the basic functionality and data loss criteria. It appears to be fixed but we are giving it blocker status in case further testing shows that it is not. 14:10:50 ack 14:10:53 ack 14:10:54 ack 14:11:05 ack 14:11:12 coremodule: available for secretary duty? 14:11:13 ack 14:11:16 ack 14:11:17 or we need a different volunteer 14:11:40 ack 14:12:04 #agreed 1763875 - AcceptedBlocker (Final) - This violates the basic functionality and data loss criteria. It appears to be fixed but we are giving it blocker status in case further testing shows that it is not. 14:12:16 #topic (1763831) revert incorrect arm64 <-> aarch64 mapping 14:12:17 #link https://bugzilla.redhat.com/show_bug.cgi?id=1763831 14:12:19 #info Proposed Blocker, rpm, MODIFIED 14:12:31 again: appears to be fixed in the RC 14:13:20 Oh, this one was discussed in the last blockers meeting. 14:13:49 +1 blocker because it is fundamentally wrong and also appears to violate trademarks 14:13:55 do we actually have some confirmation that this is working? 14:14:15 pbrobinson: are you around? 14:14:35 +1 blocker 14:14:45 +1 Blocker 14:14:47 pwhalen: Can you speak to this? 14:16:11 * bcotton just asked in #fedora-arm 14:16:43 so i think we go ahead and call this a blocker and then table it for a bit to see if someone can verify it 14:16:55 +1 blocker 14:17:02 bcotton: yes 14:17:58 I'll let blocker decision to people more knowledgeable in this area. But postponing it until we get some replies sounds good. 14:18:17 proposed #agreed 1763831 - AcceptedBlocker (Final) - This violates Arm's trademark and it will cause long-term support problems for aarch64 images 14:18:44 ack 14:18:47 ack 14:18:50 ack 14:18:52 ack 14:19:13 Ack 14:19:17 #agreed 1763831 - AcceptedBlocker (Final) - This violates Arm's trademark and it will cause long-term support problems for aarch64 images 14:19:33 #info This appears to be fixed, but we are seeking confirmation 14:19:41 okay, on to our accepted blockers 14:19:59 #topic (1691430) dnf.exceptions.Error: Incorrect or unknown "arch": armv7hcnl 14:20:01 #link https://bugzilla.redhat.com/show_bug.cgi?id=1691430 14:20:02 #info Accepted Blocker, libdnf, VERIFIED 14:20:40 pwhalen says it's fixed. i believe him 14:20:45 Morning folks 14:20:50 morning 14:20:56 * bcotton hands adamw some coffee 14:21:10 adamw: Afternoon 14:21:23 adamw, the new day is greeting you 14:21:26 we almost never vote on verified bugs, afaik. but if you prefer, we can 14:21:47 kparal: I'm fine either way 14:21:49 not looking for a vote, just input 14:22:06 does anyone disagree with pwhalen that it's fixed? 14:22:06 doh, this one is already accepted 14:22:13 speak now or forever hold your peace 14:22:32 * kparal holds his peace 14:22:35 (where "forever" is "until the next #topic") 14:22:42 great 14:22:43 * Lailah holds her peace 14:22:59 #info This is fixed in RC1.9 according to pwhalen 14:23:19 and lastly 14:23:23 #topic (1764642) Reporting installer crashes fails when installing from live images due to RPM header string type issue (QA:Testcase_Anaconda_save_traceback_to_bugzilla) 14:23:25 #link https://bugzilla.redhat.com/show_bug.cgi?id=1764642 14:23:26 #info Accepted Blocker, python-meh, VERIFIED 14:23:37 ditto 14:24:11 (except this time it's alciregi who says it works) 14:24:15 objections? 14:24:18 meh 14:24:23 mhroncok++ 14:24:33 I also tested the update 14:24:54 no objections 14:25:04 #info This is fixed in RC1.9 according to alciregi and kparal 14:25:11 I've tested RC 1.9, fixed 14:25:23 #info ...and frantisekz :-) 14:25:26 #topic Current status — Blocker bugs 14:26:10 #info All accepted and proposed blockers are fixed in the latest RC, but we are waiting for someone on the arm team to verify 1763831 14:26:24 so while we wait for that... 14:26:26 #topic Current status — Candidate compose 14:26:53 mboddu has been making RCs like they're going out of style 14:27:13 Haha 14:27:27 #info RC1.9 is the current candidate 14:27:44 anything we should know about 1.9? how do our non-blocking deliverables look? 14:28:30 There are some that missed - https://pagure.io/releng/failed-composes/issue/368 14:28:33 * satellit_ soas seems to have no netwoking but need time to test 14:29:04 * satellit_ 1.8 and 1.3 worked 14:29:21 I think we got all spins, some labs are missing 14:29:36 But we missed armhfp containers, server, workstation 14:30:10 #info All spins are in RC1.9, some labs are missing. armhfp containers, server, workstation also missing 14:30:12 #link https://pagure.io/releng/failed-composes/issue/368 14:30:34 #info soas seems to have no networking, testing ongoing 14:30:37 * mboddu checksum test is done and everything passed 14:31:13 is arm a blocking arch? 14:31:15 mboddu: "I think we got all spins, some labs are missing" Sounds like you have a stamps collection... 14:31:49 Lailah: Haha :D 14:32:05 I see that the Scientific Lab failed...? 14:32:49 Southern_Gentlem: arm is only blocking for Minimal and Xfce https://fedoraproject.org/wiki/Releases/31/ReleaseBlocking 14:33:37 we got all armhfp images, the images that failed are expected (sorry in another meeting) 14:33:53 pwhalen++ 14:33:53 bcotton: Karma for pwhalen changed to 5 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:34:33 pwhalen: while we have you, can you check that https://bugzilla.redhat.com/show_bug.cgi?id=1763831 is truly fixed in 1.9? we'll come back to it when you're ready 14:34:50 any other questions or comments on RC1.9 before we move on to test coverage? 14:36:23 #topic Current status — Test matrix coverage 14:36:25 #link https://fedoraproject.org/wiki/Category:Fedora_31_Test_Results 14:37:07 ok 14:37:10 i'm going through this right now 14:37:33 so, I am testing QA:Testcase_upgrade_gnome-software_previous_workstation right now, for fix for https://bugzilla.redhat.com/show_bug.cgi?id=1762751 on F29 14:37:38 we're looking fairly good; note arm desktop tests are transferred from rc1.3, cloud tests from rc1.8, those should be okay 14:37:39 F30 fix is confirmed working 14:37:48 bcotton: No further questions, Your Lordship 14:37:52 cloud tests haven't been run in a real cloud yet 14:37:54 xen test isn't done yet 14:37:55 Lailah++ 14:38:48 that's about it. i can try and test xen real quick 14:38:53 frantisekz++ 14:38:53 bcotton: Karma for frantisekz changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:38:58 frantisekz: But F29 will be EOL when F31 is released, isn't it? 14:39:16 no. 14:39:20 Lailah: 28 days after the release 14:39:25 Ah, okay 14:39:31 and we specifically support upgrade during that time 14:39:43 bcotton: Why didn't I get my cookies? 14:40:00 Lailah: i must have given you a cookie already during the f31 cycle 14:40:02 adamw: Oh, fair enough 14:40:22 Or maybe you have to write my username with lowercase? 14:40:27 Like lailah? 14:40:35 #info arm desktop tests are transferred from RC1.3 14:40:43 #info cloud tests are transferred from RC1.8 14:41:08 any other questions or comments on test matrices? 14:42:54 pwhalen: are you checking the arm64 <-> aarch64 mapping fix? 14:43:17 Lailah++ 14:43:32 lailah++ 14:43:32 sgallagh: Karma for lailah changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:43:36 Ah! 14:43:39 Yup, seems to be that 14:43:43 It was that, the lowercase 14:43:59 huh. get with the times, zodbot 14:44:06 Lailah, if you set your IRC Nick in FAS, it will also accept that 14:44:07 lailah++ 14:44:07 bcotton: Karma for lailah changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:44:19 * Lailah eats her cookies while happily purring 14:44:33 i don't think cats are supposed to eat cookies 14:44:49 adamw: This one does. 14:44:49 Lailah: What sgallagh said, you need to update your FAS with the exact nick 14:44:51 =) 14:45:06 lailah++ 14:45:06 jlanda: Karma for lailah changed to 3 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:45:10 Another test :) 14:45:11 unicorn does 14:45:16 mboddu: Oh! I will then. Thanks sgallagh & mboddu 14:45:22 * jlanda waves 14:45:26 hi jlanda 14:45:34 * Lailah waves back at jlanda 14:45:36 lailah++ 14:45:39 lruzicka: Karma for lailah changed to 4 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:45:46 lailah++ 14:45:46 frantisekz: Karma for lailah changed to 5 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:45:56 faezebax: That's true 14:46:33 So where did we leave off? 14:46:34 Now that lailah have 5 cookies we can release :D 14:46:42 okay, so that leaves us with making a decision. i'd like independent verification of the arm blocker, but that doesn't seem to be forthcoming 14:47:11 jlanda: That is correct 14:47:28 i'd also like to try and get this xen test run 14:47:37 adamw: how long should we stall? 14:47:39 and if anyone could do cloud in a real cloud 14:47:53 er 14:48:09 so, QA:Testcase_upgrade_gnome-software_previous_workstation is passing too, just upgraded with new libdnf, packagekit, kudos to kalev!!!! 14:48:17 kalev++ 14:48:25 kalev++ 14:48:25 jlanda: Karma for kalev changed to 8 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:48:25 I confirm. 14:48:26 adamw: Not really possible to do a cloud here, I'm still struggling with some weird bugs in KDE 14:48:28 gimme ten minutes 14:48:50 #topic Stalling for a few minutes to run a few final tests 14:48:53 kalev++ 14:49:15 Duh... Didn't work 14:49:32 bcotton, confirmed, fixed in rpm-4.15.0-6.fc31 (sorry) 14:49:33 kalev++ 14:49:33 sgallagh: Karma for kalev changed to 9 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:49:39 pwhalen++ 14:49:41 bcotton: So what do we do now? Do we just wait while having a cup of tea? 14:49:41 pwhalen++ 14:49:41 sgallagh: Karma for pwhalen changed to 6 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 14:49:44 Lailah: yes 14:49:54 Lailah: Sounds good to me. 14:49:55 #info arm64 <-> aarch64 mapping is fixed as confirmed by pwhalen 14:49:56 * sgallagh brews 14:50:01 or a cigarette :) 14:50:02 Okay. 14:50:09 pwhalen++ 14:50:14 frantisekz: I don't smoke 14:50:19 anyone who wants to step away for a moment, i will resume substantive content *no sooner than* 1500 UTC (10 minutes from now) 14:50:27 I only smoke when I set myself on fire 14:50:32 :D 14:50:37 * Lailah shoo away the birds in the balcony and starts preparing tea 14:50:45 (Which has happened an alarming number of times) 14:50:52 sgallagh: That's a good one 14:51:09 sgallagh: You need a new fire alarm me think 14:51:43 Doesn't usually help when the problem is "dangling bathrobe belt and natural gas stovetop" 14:52:03 O_O 14:52:30 sgallagh: At least it can warn you earlier... 14:52:44 Get flame-retardant bathrobes? 14:52:48 It generally doesn't warn you faster than the heat does 14:52:59 Also, sgallagh: turn the gas off 14:53:18 Clearly I just need to do all of my cooking naked. 14:53:38 (Instead of just most of it) 14:53:54 sorry, testing xen is always fun cos i never do it 14:53:56 (iyswim) 14:54:09 sgallagh: put on some underwear, you never know... 14:54:31 Didn't we plan to remove Xen from blocking this release since Amazon doesn't require it anymore? 14:54:37 so... how about anything other than this conversation? :-) 14:55:04 sgallagh: some EC2 instance types still use Xen apparently, but we had discussed making EC2 the requirement as a proxy for xen 14:55:10 iirc we never came to a final conclusion 14:55:45 bcotton: Some use Xen, but the modern ones use KVM and there's no price incentive to use the old ones. 14:56:08 that doesn't mean people won't :-) 14:56:31 sgallagh: i tried, people complained, life sucks 14:56:44 so for now the old criterion still exists 14:56:47 in my epxerience, people will continue using whatever instance type they started using until something forces them to change 14:56:48 i'll hit the pinata again this round 14:56:51 I'll raise the issue again after Go/No-Go 14:57:12 bcotton: Like... Fedora dropping blocking support for Xen? 14:57:17 the proposal to change the criterion got a bit hung up on ec2 details and stuff 14:57:38 adamw: I was convinced XEN was dropped... 14:57:45 I must have dreamed it then 14:58:13 sgallagh :-D 14:59:12 we someway decided to move from xen to ec2 blocker, then we started discussing about what ec2 instamce types... and beta came and we parked it as is 14:59:25 so, I'd need to leave for another meeting, fingers crossed for GO ;) 14:59:56 frantisekz: thanks for your help, enjoy the meeting 15:00:03 sorry everybody, bye! 15:00:15 ok, my xen install is running now 15:00:26 frantisekz bye! Have fun! 15:00:37 so yeah, as well as this if anyone can at least fire up *some* ec2 instance and test it'd be good 15:01:08 adamw: Where do I get the ec2 AMIs? 15:01:15 I'll give it a shot 15:01:25 er 15:01:28 always with the tough questiond 15:01:30 mboddu? 15:01:30 * pwhalen is testing aarch64 cloud on aws 15:01:42 sgallagh: https://dl.fedoraproject.org/pub/alt/stage/31_RC-1.9/Cloud/x86_64/images/ 15:01:53 Thanks 15:01:53 Or other arches 15:02:00 those aren't amis... 15:02:21 list of amis - https://apps.fedoraproject.org/datagrepper/raw?topic=org.fedoraproject.prod.fedimg.image.publish&delta=86400 15:03:08 pwhalen++ 15:03:08 jlanda: Karma for pwhalen changed to 7 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:03:09 pwhalen da boss 15:06:22 jeez, xen installs are slow for some reason 15:06:51 * adamw watches it configure shared-mime-info 15:07:08 I've got an x86_64 AMI up and running, seems to be functioning largely as one would expect. 15:07:14 Any specific tests you want run? 15:08:03 the tests on cloud page i guess 15:08:07 but just that it runs is good news 15:08:11 any failed services? 15:08:39 None 15:08:48 cool 15:08:49 https://fedoraproject.org/wiki/Test_Results:Fedora_31_RC_1.9_Cloud?rd=Test_Results:Current_Cloud_Test tests 15:09:27 aarch64 ami also working, no failed services 15:09:56 xen install just finished 15:10:41 adamw: All passed 15:10:49 I'll run relval real quick 15:11:59 adamw: it took its time... 15:12:29 gah, xen doesn't want to run the installed system 15:14:27 * sgallagh starts baking fudge 15:15:32 soas 1.9 x86 works in VM 15:15:40 hooray! 15:15:40 sees network 15:15:51 eh 15:16:02 i give up on getting xen to co-operate 15:16:11 nuc i5 15:16:15 i don't think it's broken in a criteria-violating way necessarily, it's my dom0 that's playing up[ 15:16:22 at least the install ran 15:16:22 Actually, let me see which instance type I started 15:17:27 * Lailah goes for another cuppa 15:19:06 OK, I had launched an HVM instance. 15:19:13 Let me try for a paravirt one... 15:20:55 Am I reading this wrong, or are we not actually publishing any of the paravirt AMIs to Amazon anyway? 15:21:05 i dunno 15:21:41 sgallagh: No idea 15:22:23 Yeah, looks like we don't even bother building Xen instances for Fedora 15:22:27 err for Amazon 15:22:27 [17:21:30] hvm = xen = adamw can stop fighting with dom0 ? 15:22:40 hum 15:22:40 jlanda: hvm != xen 15:22:41 okay 15:22:47 well i guess that settles that, then 15:22:54 not really 15:23:01 the current criterion says xen has to work. nothing about ec2. 15:23:11 we should stop building any artifacts, then none of our tests can fail 15:23:31 "HVM AMIs are presented with a fully virtualized set of hardware" 15:24:15 so are we blocked from making a decision by the xen tests at this point? 15:24:37 The *only* reason we have that criterion is because we thought EC2 required it. 15:24:53 But we don't even publish EC2 images that work for it. 15:25:07 that's not the only reason, no 15:25:11 but let's not re-litigate that here 15:25:48 let's just say i can't get the test run and we'll waive it on the basis that xen people are supposed to be doing the testing and they're clearly still not 15:25:54 +1 15:26:07 pygrub is failing to find the bootloader in the image i think, but i've no idea if i'm doing something wrong or what 15:26:27 +1 15:26:31 anything else before we poll for go/no-go? 15:27:24 #topic Go/No-Go decision 15:27:25 I will poll each team. Please reply “go” or “no-go” 15:27:27 The release must boot successfully as Xen DomU with releases providing a functional, supported Xen Dom0 and widely used cloud providers utilizing Xen. 15:27:27 not from me 15:27:38 FESCo? 15:27:42 Go 15:27:43 Can we met that without pvhvm ? 15:27:58 The and widely used cloud providers part 15:28:12 Wr twchnically provide images, just for hvm :) 15:28:21 * bcotton pauses 15:29:27 ...i guess we can? 15:29:42 * bcotton resumes 15:29:44 Releng? 15:29:44 jlanda, who are you expecting an asnwer from 15:29:50 Go 15:29:52 it's not something that's changed recently, anyhow. 15:29:56 QA? 15:29:59 * mboddu also wishes we could do more testing 15:30:17 * lruzicka lets adam decide 15:30:29 * Lailah also lets adam decide 15:30:41 adamw ? 15:30:53 i think we can say go according to the policy 15:31:00 no outstanding blockers, test coverage considered acceptable 15:31:01 adamw: you don't sound convinced? 15:31:02 ok, +1 15:31:11 smooge: no one specific, I think we should go 15:31:19 i sound like this meeting is happenign three hours early and i didn't get coffee 15:31:29 well i sent it to you! 15:31:39 okay, so that's "go" around the board 15:31:53 #agreed Fedora 31 Final is GO 15:32:06 YAY!!! 15:32:10 \o/ 15:32:12 take *that* random internet commenter who said we should just call it a november release 15:32:26 #info Fedora 31 Final will release on 2019-10-29 15:32:37 #action bcotton to announce decision 15:32:41 that is a nice date 15:32:49 * mhroncok wasn't paying much attention, should we make sure the libgit2 dnf-plugins and packageit updates both hit stable on 29 and 30 before the release? 15:32:52 however, we could release on Helloween 15:32:57 Great! In time for Halloween, Samhain and Herbstferien 15:33:15 mhroncok: That's part of the policy for PreviousReleaseBlocker already 15:33:28 sgallagh: thanks for confirming 15:33:48 last call! 15:33:52 the packagekit/gnome-software fix be stable for declaring Go? I'm just asking 15:34:12 that was the idea behind my question 15:34:13 hmm, half of my sentence is missing 15:34:39 so, once again, when those previous release updates need to be stable? 15:34:42 mhroncok: yeah, we still need to ensure that. 15:34:44 kparal: sorry, but full sentences are not a blocker criterion 15:34:47 on release date 15:34:59 adamw: Why not before release date? 15:34:59 so we need to push em monday at latest ,really 15:35:03 bcotton: Last call for what? 15:35:04 they can be stable before 15:35:07 but they *must* be stable by 15:35:08 We can push them now if they have karma 15:35:18 Lailah: for people to say things before i end the meeting 15:35:31 I would like to push them asap, so that all mirrors will pick it up before people start upgrading 15:35:42 mboddu++ 15:35:49