17:00:06 #startmeeting F24 Final Go/No-Go meeting #2 17:00:06 Meeting started Thu Jun 16 17:00:06 2016 UTC. The chair is jkurik. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:06 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:06 The meeting name has been set to 'f24_final_go/no-go_meeting_#2' 17:00:08 #meetingname f24-final-go_no_go-meeting 17:00:08 The meeting name has been set to 'f24-final-go_no_go-meeting' 17:00:09 #topic Roll Call 17:00:11 .hello jkurik 17:00:11 jkurik: jkurik 'Jan Kurik' 17:00:32 #chair dgilmore nirik adamw 17:00:32 Current chairs: adamw dgilmore jkurik nirik 17:00:41 morning 17:00:47 evening :) 17:00:49 .hello coremodule 17:00:50 hello! 17:00:50 coremodule: coremodule 'Geoffrey Marr' 17:00:57 hi every body 17:01:25 adamw: do we have you ? 17:01:30 yes 17:01:33 hi 17:01:35 i'm just looking into remaining dual boot issues 17:01:55 ok, we will talk about it shortly 17:02:06 #topic Purpose of this meeting 17:02:07 #info Purpose of this meeting is to check whether or not F24 final is ready for shipment, according to the release criteria. 17:02:09 #info This is determined in a few ways: 17:02:11 #info - No remaining blocker bugs 17:02:12 #info - Release candidate compose is available 17:02:14 #info - Test matrices for Final are fully completed 17:02:15 #link https://qa.fedoraproject.org/blockerbugs/milestone/24/final/buglist 17:02:17 #link http://fedoraproject.org/wiki/Fedora_24_Final_Release_Criteria 17:02:18 #link https://fedoraproject.org/wiki/Category:Fedora_24_RC_Test_Results 17:02:20 #topic Current status 17:02:26 #info This is the second round of the Go/No-Go meeting of Fedora 24 Final 17:02:33 #info The RC for the Fedora 24 Final is available as Fedora 24 RC 1.2 (20160614.0) 17:02:40 #link https://kojipkgs.fedoraproject.org/compose/24/Fedora-24-20160614.0/ 17:02:45 #link https://fedorahosted.org/rel-eng/ticket/6423#comment:13 17:02:51 #info Currently we have 2 proposed and none approved Blockers 17:02:59 #info We have 2 proposed and 5 accepted Freeze Exceptions 17:03:06 #link https://qa.fedoraproject.org/blockerbugs/milestone/24/final/buglist 17:03:16 Let's start with Mini-blocker review 17:03:23 adamw: may I ask you please to chair the mini-blocker review ? 17:03:28 hello 17:03:35 nb: hi 17:03:50 sure 17:03:59 #topic Mini-Blocker Review 17:04:13 so we have two windows dual boot bugs to consider 17:04:14 #topic (1347273) There is no Windows entry in grub after installation to dual boot 17:04:15 #link https://bugzilla.redhat.com/show_bug.cgi?id=1347273 17:04:15 #info Proposed Blocker, grub2, NEW 17:04:47 this one i have not yet reproduced as i'm working on the other one: so far the symptom seems to be that Windows isn't auto-detected and added to the Fedora boot menu if you have a UEFI Windows install on a firmware RAID device 17:05:23 as I understand it, it appears only on RAID and even such it is not reliably reproducible 17:05:38 given that you can still boot windows, I'm -1 blocker. 17:05:50 -1 blocker from my side as well 17:06:08 well, you can boot it from the system firmware 17:06:20 if we thought that was enough we would never have taken *any* of these bugs as blockers 17:06:47 so far we've said we want it to work from the Fedora grub because we're not confident enough that all firmwares provide a usable interface to the boot manager 17:07:01 hum. 17:07:12 * nirik adds another reason to dislike firmware raid. ;) 17:07:24 still, i can go with -1 for limited impact and discovered really late 17:07:33 the more we poke at this the more corner cases we're likely to find 17:07:51 does it applies only for a specific firmware/raid or is it general ? 17:09:54 jkurik: not enough data yet 17:10:06 i've been reproducing the *other* bug (the one that comes next) so i didn't try this one yet 17:10:11 so far we just have pschindl's report 17:10:21 we also don't know if this worked in f23 17:10:22 ok, so I am still -1 to block on this 17:11:12 I can go -1 17:12:44 who else do we have to vote? 17:12:46 so far i see about -3 17:12:57 who is allowed to vote? 17:13:18 striker: for blockers anyone is allowed 17:13:27 I'm -1 on the fact that you can still firmware boot. 17:13:33 then -1, as per comment #2, it seems specific to the environment 17:14:35 and the reporter also changed his +1 to a -1 17:14:37 proposed #agreed 1347273 - RejectedBlocker - this is unfortunate but it's only coming to light very late, we have little data but so far it seems specific to firmware RAID, and there are other mechanisms for achieving boot on UEFI, so the combined impact is too low to consider this a blocker at this point 17:14:50 ack 17:14:54 ack 17:15:06 ack 17:15:14 #agreed 1347273 - RejectedBlocker - this is unfortunate but it's only coming to light very late, we have little data but so far it seems specific to firmware RAID, and there are other mechanisms for achieving boot on UEFI, so the combined impact is too low to consider this a blocker at this point 17:16:13 #topic (1347291) Booting from Windows 10 entry ends with 'relocation failed' error 17:16:13 #link https://bugzilla.redhat.com/show_bug.cgi?id=1347291 17:16:13 #info Proposed Blocker, grub2, NEW 17:16:19 this one may be a little more worrying i guess 17:16:26 adamw, Do we need someone on secretary duty? 17:16:35 coremodule: if you could that'd be great 17:16:43 adamw, Roger that! 17:16:53 coremodule++ 17:16:53 jkurik: Karma for coremodule changed to 1 (for the f23 release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:17:01 so far we have three reports of systems / windows 10 installs where after installing fedora alongside windows (this is all UEFI, btw) the windows entry in fedora's boot manager fails with this 'relocation failed' error 17:17:29 :-/ 17:17:42 we have one physical system (the same one from the previous bug, but without fwraid) where it seems to work (though with UEFI i guess there's always the possibility it's falling through to the next boot manager entry after grub fails?) and it seems to work in a KVM 17:18:01 considering I have an amd machine at home w/ win10 and f24rc1-2, going to give my -1 17:18:12 striker: UEFI or BIOS? 17:18:15 UEFI 17:18:21 so it worked for you? 17:18:34 then that's +3/-3 so far i guess (in terms of systems that work/fail) 17:18:39 do we know whether this is specific for Win 10 ? Do we have a test with Win 7 for example ? 17:18:41 I set prio order in BIOS to boot Fedora UEFI and grub shows both entires - choosing Windows works fine 17:18:42 huh, and it boots after the error on 2+ attemtps to boot? 17:18:58 nirik: no, fedora boots OK after showing the error once 17:19:14 pjones says that's probably just a weird interaction with the shell (the reason the error messages and the boot success/fail get slightly out of sync) 17:19:16 ah, but win never boots? 17:19:23 but basically windows never boots (from fedora grub), fedora does boot 17:19:28 striker: ok, thanks 17:19:40 jkurik: we are not sure of the parameters yet, let's say 17:19:58 for me, i've tested the same win10 image in a KVM and on my bare metal test box, it worked in the KVM, failed on the test box 17:20:09 i have not tried any other windows yet (this test takes for-goddamn-ever) 17:20:41 I have a server 2012 iso here I can also test in kvm 17:20:51 sure, why not 17:21:00 pjones is having a look into my case of this at present 17:21:18 is there any commonality with raid here? or it doesn't seem related? 17:21:27 no 17:21:36 mine fails on install to a single disk 17:21:36 adamw: ok, I just asked as it might be something specific for Win 10 17:21:47 jkurik: it may be; we just don't know yet 17:22:22 yeah, hard to determine here without more info... 17:23:46 should we pause for more testing? or just vote on what we know so far? 17:23:53 building a virt reproducer for 1347291 now 17:23:53 well, non working dual-boot is blocking: https://fedoraproject.org/wiki/Fedora_24_Final_Release_Criteria#Windows_dual_boot 17:23:54 oop, we have another failure report on the bz 17:24:18 jkurik: right, but this is clearly conditional: sometimes it works, sometimes it doesn't 17:24:28 we're kinda tossing coins atm since we don't know *when* it works and *when* it fails 17:24:40 also, you can boot to w10 from uefi selection if it doesn't work from grub 17:24:53 striker: again, this is only if your firmware gives you a reasonable interface to that (and you know about it) 17:24:58 which is why we still care about the grub menu 17:25:03 ah, ok 17:25:06 * striker shuts up 17:25:08 it is a mitigating factor, though, compared to BIOS where that's not an option 17:25:29 striker: we've seen some really awful firmware interfaces that just don't really let you pick what to boot like they should 17:25:48 that's why everyone should run with lenovos 17:27:19 with the currently available info I am 50/50 to block/not-block on this 17:27:30 yeah, i'm kinda on the fence too 17:27:34 i'm hoping pjones will see something 17:27:54 at least give us an idea 17:27:54 * nirik is a weak -1 blocker right now, but if it's more widespread... bah. 17:27:59 sorry full attention is here now 17:28:53 w2k12 installed, installing f24rc1-2 now 17:29:09 hi pjones, thanks for stopping by 17:29:14 hello. 17:29:41 so we're kinda hesitating over whether to block on https://bugzilla.redhat.com/show_bug.cgi?id=1347291 , since all we know so far is it seems to work sometimes and not work other times, it's kinda not much to go on 17:29:56 and it's only been discovered late and dual boot can be kind of a rabbit hole you have to stop jumping into at some point 17:29:56 So, with the version of win10 I got from msdn last week, the current code works. 17:30:48 which doesn't mean much about the problem itself, but does mean some people /won't/ hit it. 17:31:14 pjones: so we can then ask in "know issues" to update windows to the latest version before they install F24 17:31:26 yeah, and we have some successes in bugzilla...but also (by my count) four fails so far, me and pschindl and two bz reporters 17:31:36 jkurik: I do not know if a windows update would fix it or not. 17:31:52 ok 17:31:58 i guess that's one thing i can try 17:32:05 it could be that one is for some 'pro' variant vs some 'home' variant or something like that. 17:32:08 (though it'll be after the fedora install) 17:32:23 i believe i installed pro (both in my kvm where it worked and my bare metal box where it failed) 17:36:15 * adamw downloading updates for the win10 install on his metal box, and re-running test on KVM 17:36:20 can't reproduce with wk12 17:36:25 w2k12* 17:37:05 adamw: okay, I think I have the iso that has both and I picked home. 17:37:25 yeah, my iso has both and i picked pro - but the kvm / metal difference seems odd 17:37:40 i'm *maybe* wondering if it actually failed in kvm and instead of looping back to grub fell through to the next bootmgr entry 17:37:46 is that plausible? or should it always loop back to grub? 17:37:56 striker: thanks for the good news :) 17:39:51 sigh, microsoft's update download progress indicator seems like a good example of the genre 17:40:02 starts at 0, suddenly jumps to 74, jumps back down to 68, then sticks at 92 17:40:35 ooh, now it's preparing to install updates. exciting 17:41:06 I would always expect to get back to grub if that's the failure 17:41:12 adamw: calm down :) 17:42:11 ... 17:42:15 yours works for me in kvm. 17:42:19 with 0.34 17:42:41 there should be a uefi entry for windows still correct? 17:42:48 yes. 17:42:50 yes 17:43:07 should we just say use eufi for dual booting? 17:43:14 I am going -1 for this 17:43:37 dgilmore: the thought is that some firmwares don't show that to the user at all usefully. 17:44:26 nirik: Okay, I have not done a dual boot setup in over 10 years, so I am not sure I am the right person to make a call 17:44:45 * nirik nods. I also have no windows here at all... can't really help testing much 17:44:55 same 17:44:59 About to update the bug, I cannot reproduce on an Intel NUC. 17:45:09 cmurf: so it works for you? 17:45:14 yes 17:45:14 adamw: yeah, your binary works 100% fine for me in kvm with grub2-2.02-0.34.fc24.x86_64 17:45:21 windows boots cleanly. 17:45:21 pjones: yeah, it worked in a KVM for me too - it's only on my metal test box where it doesn't 17:45:33 so, that's odd. 17:45:38 yeah, i know. 17:45:47 firmware specific ? 17:45:47 it's going to take me a while before I can have a look on bare metal. 17:45:50 but i've done it twice now, so it doesn't seem like some freak failure 17:46:00 jkurik: you would not expect it to be, but... 17:46:04 jkurik: seems like the firmware must be involved somehow, but as pjones says this seems weird 17:46:22 I tried SB enabled and disabled, GRUB -34 boots both Fedora and Windows 10 17:46:34 I'm thinking "not a blocker" :) 17:46:40 cmurf: on real hardware? 17:46:45 Intel NUC 17:46:50 i think i can go with -1 for this on the basis that we know it works at least *some* of the time, UEFI boot selection is available as a mitigation for anyone with a sane firmware, and we can't really delay indefinitely on this stuff as there's always gonna be some case that fails 17:47:02 * nirik nods 17:47:22 yeah, the same here, I am becoming more and more -1 to block 17:47:34 NUC5PPYB BIOS 86A 2/23/2016 17:48:00 ok, so we've got i think -4 now 17:48:04 oh, -5 with pjones 17:48:17 adamw I agree however we don't know the scope of the problem, how much hardware is affected 17:48:24 on the bright side it's fixable with an update 17:48:48 -1 here, based on the BZ report + what's been reported above 17:49:09 And like cmurf said, it's update-fixable. 17:49:22 proposed #agreed 1347291 - RejectedBlocker - there definitely seems to be a bug here, but we know it works in at least several cases (so far reports are ~50/50) and the firmware boot interface can be used by anyone with a sensible firmware 17:49:32 ack 17:49:40 ack 17:49:44 ack 17:49:46 cmurf: at least to some degree, yeah 17:49:59 #agreed 1347291 - RejectedBlocker - there definitely seems to be a bug here, but we know it works in at least several cases (so far reports are ~50/50) and the firmware boot interface can be used by anyone with a sensible firmware 17:50:15 ok, with that we have no blockers, unless anyone's proposed one in the last half hour 17:50:21 ack 17:50:24 so there is no point reviewing FEs, i believe 17:50:27 (is this a regression from F23, btw?) 17:50:35 mattdm: we don't know that yet either 17:50:39 ack 17:50:41 mattdm: lots of stuff we don't know :/ i'll keep digging into it today 17:50:44 adamw: ok :) 17:50:45 mattdm: it didn't work at all in f23 on sb systems, did it? 17:50:56 no 17:50:58 pjones: *shrug* 17:51:02 but on non-SB it did 17:51:05 my case here is non-SB 17:51:11 hrm. 17:51:23 oh, that's... interesting. 17:51:26 (i definitely have no SB in my KVM and i'm pretty sure it's off on my metal box) 17:51:30 (sorry not meaning to hang things up.) 17:51:35 on non-sb in the current code you're getting the system firmware's loader in some cases. 17:51:42 just want to be sure I know what I'm talking about :) 17:51:55 ok, so lets move on 17:51:57 ok 17:52:02 i can debug further with pjones elsewhere 17:52:05 adamw: thanks for the blocker review 17:52:07 Has this os-prober blocker been discussed? 17:52:17 adamw++ 17:52:23 adamw++ 17:52:23 striker: Karma for adamwill changed to 25 (for the f23 release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:52:47 #topic Test Matrices coverage 17:52:53 #link https://fedoraproject.org/wiki/Test_Results:Fedora_24_RC_1.2_Summary 17:52:54 cmurf: 1347273 ? yes. it was rejected 17:53:29 that one seems to be pretty corner case-ish (out of all the people reporting so far, the only case where that happened was that one pschindl report with firmware RAID windows install) 17:53:37 we have pretty darn good coverage 17:53:46 so, as I can see we are missing QA:Testcase_install_to_FCoE_target 17:54:16 that is the one we do not HW for it, right ? 17:55:14 yeah...well, theoretically i think it's possible to test entirely with software emulation, but i could not figure it out 17:55:14 If volunteers aren't stepping up and Fedora doesn't have the hardware then it's defacto deprecated. 17:55:29 adamw, I am working on the Xen install currently. My internet is slow for piping X over so give me a bit more time. Worst case, I'll have it tested tonight. 17:55:52 you did test it with 0531, right? 17:56:00 Yeah, and it worked fine there. 17:56:02 that result is probably OK, nothing significant to xen has changed since then i don't think 17:56:17 so we're missing FCoE, SAS, and a couple of ARM desktop tests; one of those was done for RC1.1 (so won't have changed) 17:56:26 pwhalen: around? 17:56:27 ok, great 17:56:38 adamw, i am 17:56:51 it sounds like we are fine with the current coverage 17:56:54 pwhalen: can you check update notification? 17:57:00 oh, there's a couple of image sanity tests missing too, hrm 17:57:07 lemme blow through those quick 17:57:15 adamw: ok, thanks 17:57:48 that compose bug is still around... what did we lose? SoAS and design? 17:57:57 adamw assign me something 17:58:01 but not too much help for it I fear 17:58:07 adamw, I can check that... Yes, the last xen change was on 05-31. 17:59:03 ok, repoclosure and fileconflicts are fine 17:59:19 nirik: yeah, that kind of sucks :( 17:59:36 we could maybe fudge something up, like providing download links for a nightly for those or something 18:00:22 +1 to fudging something up 18:00:24 adamw, hrm need to write out a card for xfce, will take a bit. i did check for updates but there werent any available 18:01:13 pwhalen: ah, yeah, you have to either enable updates-testing or downgrade something 18:01:31 nirik: do you happen to have noticed if update notification is working in xfce? 18:01:52 card in progress now, will update asap 18:01:59 no. It hasn't for years I don't think. 18:02:06 ive not seen it 18:02:22 there's nothing that would provide them... 18:02:42 nirik: oh, okay. well, that doesn't seem worth worrying about, i guess we'll just need to tweak the test matrix a bit. 18:03:12 yeah. Someone wrote a panel plugin to do it long ago, but it had some bugs and then they stopped working on it... so meh 18:03:48 okay. we wrote that criterion when only GNOME and KDE were blocking, so i guess we'll just tweak it a bit 18:04:07 i'm assuming no-one wants to hold up the meeting for some exciting bureaucracy, so, lemme half-ass something... 18:04:20 adamw: ack 18:04:42 Ooh, bureaucracy. 18:04:50 propose #agreed nirik affirms there's no mechanism for update notification in Xfce, so we agree in principle to except Xfce from the criterion "Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image.", which makes the lack of an Xfce result for that test case irrelevant 18:05:10 ack 18:05:13 ack 18:05:35 ack 18:05:36 ack 18:05:40 ack 18:05:43 nirik: dgilmore: can one of you run https://fedoraproject.org/wiki/QA:Testcase_Mediakit_Checksums on the blocking images real quick? since you have shell access (i'm assuming) 18:05:46 ack 18:05:46 #agreed nirik affirms there's no mechanism for update notification in Xfce, so we agree in principle to except Xfce from the criterion "Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image.", which makes the lack of an Xfce result for that test case irrelevant 18:05:47 although I thought only workstation and kde were blocking? 18:05:54 nb: xfce is blocking for ARM these days 18:06:18 adamw, oh 18:06:54 .hello 18:06:54 fale: (hello ) -- Alias for "hellomynameis $1". 18:07:18 * striker has to leave but hopes F24 is a go 18:07:42 striker: thanks for joining, we hope so as well :) 18:08:10 so assuming we're OK with waiving SAS and FCoE for lack of test setup (as we usually do) and we're OK with an 05-31 result for xen, the only missing thing is https://fedoraproject.org/wiki/QA:Testcase_Mediakit_Checksums (which robatino always used to run so i forgot about it) 18:08:58 adamw: thanks 18:09:23 adamw, Xen'll be tested by the end of the day. 18:09:41 coremodule: right, but i'm assuming no-one wants to wait around till the end of the day to finish this meeting. :P 18:09:52 Hah, gotcha. 18:09:56 * adamw is checking the checksums he can right now 18:10:03 nirik, dgilmore: will one of you be able to run the Testcase_Mediakit_Checksums ? 18:10:14 running now 18:10:23 * nirik was also running it. ;) 18:11:02 :) thanks 18:11:59 thanks 18:12:01 arm spins are OK 18:12:07 i'm doing it on openqa, but openqa doesn't have all images 18:12:30 * nirik is running a script on secondary01 18:13:34 Any interest in BIOS Windows Fedora 24 dual boot tested in a VM? I don't have a BIOS system to directly test it on. 18:13:41 It has worked all pre-release period for m. 18:13:56 But the RC1.2 matrix isn't filled out for that particular test. 18:14:42 cmurf: can't hurt 18:15:00 ok, tested all the checksums i have, they look fine 18:15:22 * nirik is still running, all OK so far 18:18:36 so far all checksums are okay 18:18:40 it is still running 18:19:02 ok, lets wait for nirik or dgilmore to complete the test 18:20:46 yeah. with that, i'd say we're good on coverage 18:21:10 every iso checks out checksum wise 18:21:12 yes, QE did a great job 18:21:25 going to check the md5sums implanted 18:22:34 i checked all the ones i have locally and they were fine 18:29:01 couple more to go 18:29:07 but all passed so far 18:29:16 * nirik confirms that all isos with checksums are "OK" 18:29:30 ok, thanks 18:29:53 proposed #agreed Coverage of Test Matrices for RC 1.2 is sufficiently completed 18:30:26 may I have your acks ? or better statement :) 18:30:27 yup, i think we're ok 18:30:28 ack 18:30:42 ack 18:31:04 #agreed Coverage of Test Matrices for RC 1.2 is sufficiently completed 18:31:06 ack 18:31:22 lets make the decision .... 18:31:28 #topic Go/No-Go decision 18:31:34 ack 18:31:56 we need +1 to GO from QE, FESCo and RelEng 18:32:13 releng is go 18:32:19 qa is go on the basis of sufficient coverage and no outstanding blockers 18:32:20 I guess I can +1 for fesco. :) 18:32:35 great, thanks 18:33:19 #agreed The RC 1.2 compose is considered as GOLD and the final decision for Fedora 24 Final is "GO" 18:33:43 thanks a lot to everyone 18:33:47 #topic Open floor 18:33:55 anything else to discuss today on this meeting ? 18:34:02 * nirik has nothing, lets ship this puppy. 18:34:19 * dgilmore notes that the checkisomd5 check on all isos passed 18:34:23 #action jkurik to publish the Go/No-Go result 18:34:27 dgilmore: thanks 18:34:27 Happy to see this guy ship! 18:34:32 the check was still running when we moved on 18:34:56 dgilmore: nirik completed it a bit faster :) 18:35:08 jkurik: I only did the checksums... ;) 18:35:26 jkurik: no he completed the CHECKSUMS, which I had said my check was all okay earlier 18:35:27 ah, so then sorry, I was mistaken 18:35:38 there is two checks to be done 18:35:48 right 18:35:59 i did check most of the blocking images' embedded sums 18:36:03 all the ones i have downloaded here 18:36:16 adamw: I checked them all ;) 18:36:41 for iso in $(find . -name "*iso" -type f); do echo "== $iso =="; checkisomd5 $iso; done 18:36:46 anyway 18:36:50 it is all okay 18:37:23 if there are no other topics for the Open Floor, I will close the meeting in one minute 18:38:34 #endmeeting