17:00:02 <jkurik> #startmeeting F26 Beta Go/No-Go meeting - 2nd round
17:00:02 <zodbot> Meeting started Thu Jun  1 17:00:02 2017 UTC.  The chair is jkurik. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:00:02 <zodbot> The meeting name has been set to 'f26_beta_go/no-go_meeting_-_2nd_round'
17:00:09 <jkurik> #meetingname F26-Beta-Go-No-Go-meeting
17:00:09 <zodbot> The meeting name has been set to 'f26-beta-go-no-go-meeting'
17:00:16 <jkurik> #topic Roll Call
17:00:21 <jkurik> .hello jkurik
17:00:22 <zodbot> jkurik: jkurik 'Jan Kurik' <jkurik@redhat.com>
17:00:28 <jkurik> #chair dgilmore nirik adamw roshi mboddu
17:00:28 <zodbot> Current chairs: adamw dgilmore jkurik mboddu nirik roshi
17:00:36 <nirik> morning
17:00:41 <jkurik> hi
17:00:50 <mboddu> .hello mohanboddu
17:00:51 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
17:01:18 <roshi> .hello roshi
17:01:19 <zodbot> roshi: roshi 'Mike Ruckman' <mruckman@redhat.com>
17:01:30 * roshi needs more coffee
17:01:35 <adamw> .hello adamwill
17:01:36 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com>
17:01:53 <jkurik> hi everybody, we have qe, releng, fesco, ... so we can start
17:02:00 <jkurik> #topic Purpose of this meeting
17:02:04 <jkurik> #info Purpose of this meeting is to check whether or not F26 Beta is ready for shipment, according to the release criteria.
17:02:16 <jkurik> #info This is determined in a few ways:
17:02:20 <jkurik> #info * No remaining blocker bugs
17:02:27 <jkurik> #info * Release candidate compose is available
17:02:38 <jkurik> #info * Release candidate compose is available
17:02:49 <jkurik> #topic Current status
17:02:50 <jkurik> As far as I am aware, the RC for F26 Beta is ready ( https://pagure.io/releng/issue/6808 ) and the testing is ongoing. We have also some proposed blockers.
17:03:06 <dgilmore> hola
17:03:08 <jkurik> #link https://pagure.io/releng/issue/6808 - the ticket to build F26 Beta RC
17:03:10 <jkurik> #link https://kojipkgs.fedoraproject.org/compose/26/Fedora-26-20170531.0/compose/ - the RC compose
17:03:12 <jkurik> #link https://fedoraproject.org/wiki/Test_Results:Fedora_26_Beta_1.4_Summary - Test matrices
17:03:17 <jkurik> hi dgilmore
17:03:39 <jkurik> anyone wants to add something to the current status ?
17:04:08 <adamw> it's complicated
17:04:08 <adamw> :P
17:04:10 <jkurik> #info RC for F26 Beta is ready, testing is ongoing.
17:04:14 <jkurik> :)
17:04:37 <jkurik> ok, so lets figure out the complications...
17:04:48 <jkurik> roshi, adamw: may I ask you please to chair the mini-blocker review ?
17:05:02 <roshi> sure thing
17:05:06 <jkurik> #topic Mini-Blocker Review
17:05:08 <jkurik> #link https://qa.fedoraproject.org/blockerbugs/milestone/26/beta/buglist
17:05:10 <roshi> #topic Introduction
17:05:10 <roshi> Why are we here?
17:05:10 <roshi> #info Our purpose in this meeting is to review proposed blocker and nice-to-have bugs and decide whether to accept them, and to monitor the progress of fixing existing accepted blocker and nice-to-have bugs.
17:05:14 <roshi> #info We'll be following the process outlined at:
17:05:17 <roshi> #link https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting
17:05:19 <roshi> #info The bugs up for review today are available at:
17:05:22 <roshi> #link http://qa.fedoraproject.org/blockerbugs/current
17:05:24 <roshi> #info The criteria for release blocking bugs can be found at:
17:05:27 <roshi> #link https://fedoraproject.org/wiki/Fedora_26_Alpha_Release_Criteria
17:05:29 <roshi> #link https://fedoraproject.org/wiki/Fedora_26_Beta_Release_Criteria
17:05:32 <roshi> #link https://fedoraproject.org/wiki/Fedora_26_Final_Release_Criteria
17:06:05 <roshi> two proposed blockers for Beta
17:06:10 <roshi> #topic (1457336) files from python-libs-2.7.13-2.fc25.x86_64 conflict with files from python2-libs-2.7.13-8.fc26.x86_64
17:06:13 <roshi> #link https://bugzilla.redhat.com/show_bug.cgi?id=1457336
17:06:16 <roshi> #info Proposed Blocker, python, MODIFIED
17:08:36 <roshi> I concur with sgallagh
17:08:47 <roshi> -1 blocking and +1 0Day
17:08:50 <adamw> so afaics we haven't actually strictly defined 'release blocking package set' like we've defined other terms
17:08:53 <adamw> which is a bit of an oversight
17:09:04 <roshi> true
17:09:05 <adamw> but i've always understood it as meaning 'default package set for a release-blocking image'
17:09:07 <nirik> yeah. -1 blocker, and it should hopefully go out after we unfreeze updates.
17:09:11 <adamw> so i'm still -1 to this
17:09:24 <adamw> +1 final, though
17:10:10 <roshi> proposed #agreed - RHBZ#1457336 - RejectedBlocker Accepted0DayBlocker - This bug doesn't actually impact the install media and should be fixed once the freeze is lifted.
17:10:16 <jkurik> yeah, -1 to block beta; sgallagh has explained it in https://bugzilla.redhat.com/show_bug.cgi?id=1457336#c22
17:10:21 <jkurik> ack
17:10:26 * roshi doesn't know that Accepted)DayBlocker is a thing, but hey
17:10:36 <roshi> s/)/0/g
17:10:47 <dgilmore> +1 to 0Dayblocker
17:11:05 <dgilmore> but that does mean we have to have it pushed stable by tuesday assuming we are go
17:11:18 <nirik> it's already submitted.
17:11:21 <dgilmore> we have another 0 day blocker already
17:11:41 <nirik> just needs karma.
17:12:01 <roshi> acks?
17:12:37 <nirik> ack
17:12:46 <adamw> why 0-day blocker?
17:12:59 <adamw> are you guys arguing that any upgrade issue in any package on the server DVD is blocking?
17:13:17 <adamw> if so, that seems inconsistent with the decision made just a week or two ago about a FreeIPA upgrade bug I filed, which was rejected because FreeIPA isn't in a default server install
17:13:23 <nirik> true, doesn't need to be a blocker, IMHO.
17:13:41 <dgilmore> adamw: do we have 0day FE?
17:14:07 <adamw> i don't think we've implemented that, but it can just be accepted as a regular FE without any real problem.
17:14:23 <dgilmore> sure
17:14:35 <adamw> https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2017-05-22/f26-blocker-review.2017-05-22-16.00.log.html
17:14:42 <dgilmore> I was suggesting that it be 0day so that people upgrading to beta get a good experience
17:14:51 <roshi> proposed #agreed - RHBZ#1457336 - RejectedBlocker AcceptedFreezeException - This bug doesn't actually impact the install media and should be fixed once the freeze is lifted.
17:14:55 <roshi> that better?
17:14:57 <adamw> patch
17:15:33 * dgilmore waits
17:16:06 <adamw> actually
17:16:11 <adamw> oh no
17:16:13 <adamw> sigh
17:16:16 <adamw> let me retype that
17:17:00 <adamw> proposed #agreed - RHBZ#1457336 - RejectedBlocker AcceptedFreezeException - This bug doesn't affect the default Server package set so it is not a blocker, but it is accepted as a freeze exception issue as some people will certainly run into it and it'd be good to push the fix stable ASAP.
17:17:14 <roshi> ack
17:17:36 <jkurik> ack
17:18:09 <nirik> ack
17:19:18 <adamw> am i a chair?
17:19:24 <roshi> yep
17:19:47 <nirik> you're not a chair, you're a free man!
17:20:24 <roshi> that's not adamw, that's a chair!
17:20:31 * roshi throws it on the ground
17:20:39 <roshi> not going to be a part of this system
17:20:40 <roshi> :p
17:20:49 <adamw> #agreed - RHBZ#1457336 - RejectedBlocker AcceptedFreezeException - This bug doesn't affect the default Server package set so it is not a blocker, but it is accepted as a freeze exception issue as some people will certainly run into it and it'd be good to push the fix stable ASAP.
17:21:02 <roshi> next up
17:21:04 <roshi> should be quick
17:21:06 <roshi> #topic (1457626) F26 Beta crashes on login with Wayland on nouveau
17:21:06 <roshi> #link https://bugzilla.redhat.com/show_bug.cgi?id=1457626
17:21:07 <roshi> #info Proposed Blocker, xorg-x11-drv-nouveau, NEW
17:22:04 <nirik> yeah, so only that one hardware seeing it? (that we know of)?
17:22:24 <roshi> yeah
17:22:27 <roshi> I have two machines
17:22:42 <roshi> and oddly enough, basic gfx works on the one wayland craps out on
17:22:55 <roshi> and basic gfx doesn't work for the one that wayland does
17:23:13 <roshi> at this point I'm -1 blocker since it just seems to be the GTX 960s
17:23:21 <roshi> but would love for other NIVDIA users to test
17:23:29 * nirik has none here. ;(
17:23:58 <adamw> we're a bit short on time to do much testing here, but i'd default to notablocker
17:24:17 <jkurik> I am -1 beta blocker and I do not know of any NVIDIA user around
17:24:48 <roshi> proposed #agreed - RHBZ#1457626 - RejectedBlocker - This bug only seems to affect specific hardware and isn't wide spread enough to warrant blocking Beta for.
17:25:35 <nirik> ack
17:25:38 <dgilmore> ack
17:26:19 <jkurik> ack
17:26:30 <roshi> #agreed - RHBZ#1457626 - RejectedBlocker - This bug only seems to affect specific hardware and isn't wide spread enough to warrant blocking Beta for.
17:26:50 <roshi> that's it for hte proposed blockers
17:27:04 <roshi> adamw: you want to run through the accepted list? you're more up to speed on them than I am
17:27:07 <adamw> sure
17:27:26 <adamw> #topic (1454897) Release-blocking Cloud base images missing from Fedora 26 composes
17:27:26 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1454897
17:27:26 <adamw> #info Accepted Blocker, distribution, ON_QA
17:27:56 <adamw> erm
17:28:14 <adamw> i, uh, don't see 'em in beta 1.4 either
17:28:21 <roshi> they're there
17:28:30 <adamw> oh yeah
17:28:31 * nirik teseted them
17:28:33 <adamw> why aren't they in the wiki then
17:28:35 <adamw> whew
17:28:37 <roshi> me too
17:28:41 <roshi> they are in the wiki
17:28:46 <adamw> oh they are, just at the bottom
17:28:50 <roshi> yeah
17:28:52 <adamw> my ordering algorithm clearly stinks
17:28:53 <adamw> :P
17:28:59 <jkurik> I have not check all of the images, but I believe we have all the blocking ones
17:29:02 * roshi would have let you know if they weren't there
17:29:14 <adamw> so, this looks fine.
17:29:25 <adamw> #info the relevant images are in Beta 1.4, we can go ahead and close this.
17:29:29 <nirik> I spun up x86_64 and ppc64le instances in our openstack
17:29:38 <adamw> #topic (1438046) initial-setup.service: Failed to set up stdin: Inappropriate ioctl for device
17:29:39 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1438046
17:29:39 <adamw> #info Accepted Blocker, initial-setup, ON_QA
17:30:05 <adamw> pwhalen reported this working with the relevant package version
17:30:38 <adamw> and he reported a pass for the relevant tests in Beta 1.4 matrices
17:30:40 <adamw> so this looks fine
17:30:51 <jkurik> yup
17:30:59 <adamw> #info pwhalen and lbrabec report this working fine in Beta 1.4, so it's 'addressed' for beta validation purposes
17:31:13 <dgilmore> ack
17:31:29 <adamw> #topic (1443206) gnome-shell consistently crashes in the middle of first-login gnome-initial-setup
17:31:29 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1443206
17:31:29 <adamw> #info Accepted Blocker, libgweather, ON_QA
17:31:38 <adamw> #info this has been confirmed fixed by multiple testers
17:31:42 <adamw> #topic (1446879) [abrt] gnome-shell: gweather_location_unref(): gnome-shell killed by signal 11
17:31:43 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1446879
17:31:43 <adamw> #info Accepted Blocker, libgweather, ON_QA
17:31:49 <adamw> #info this has been confirmed fixed by multiple testers
17:32:02 <adamw> #topic (1457645) Fedora 26 KDE x86_64 live image is oversized (size 1490026496, max 1400000000)
17:32:02 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1457645
17:32:03 <adamw> #info Accepted Blocker, LiveCD - KDE, NEW
17:32:09 <adamw> so, this is a brand new automatic blocker i noticed last night
17:32:14 <adamw> rdieter: ping?
17:32:14 <zodbot> adamw: Ping with data, please: https://fedoraproject.org/wiki/No_naked_pings
17:32:19 * nirik voted in bug
17:32:20 <adamw> THE DATA IS IMPLIED ZODBOT
17:32:32 <adamw> nirik: well, changing the target size isn't a voting matter
17:32:40 <adamw> we vote on blocker status, we don't vote on blocker *fixes* :)
17:32:40 <dgilmore> adamw: zodbot is always right
17:32:54 <nirik> sure, ok, I *commented* on the bug then... sheesh. :)
17:33:02 <adamw> changing the target size must be done by the WG/SIG responsible for the image
17:33:08 <adamw> hence, ping rdieter
17:33:55 <adamw> anyone got a more...forceful way to contact rdieter?
17:34:53 <rdieter> oh hi
17:35:29 <rdieter> I see an oversized bug up there, that's the context?
17:35:35 <jkurik> yes
17:35:49 <rdieter> any objections to simply increasing the allowed size?
17:36:06 <rdieter> (though I'm a little surprised the size jumped so much)
17:36:21 * roshi has no objection
17:36:41 <jkurik> I have no objection
17:37:06 <dgilmore> no objections
17:37:20 <dgilmore> rdieter: what was it?
17:37:47 <dgilmore> I know a bunch of arm images needed to be increased last week for rawhide
17:38:39 <adamw> rdieter: we were pinging you because you're the only one who can really change the target size :)
17:38:47 <rdieter> adamw: understood
17:38:59 <rdieter> I'll try to take care of it
17:38:59 <adamw> there is supposed to be some kind of process where we do a 'spins review' every cycle and that's when the target size can be changed, but it doesn't seem to have happened at all for f26
17:39:21 <adamw> i'd say it'd be fine to just declare in this here meeting that it's changed
17:39:31 <adamw> since the spins process doesn't seem to be happening...
17:39:37 <adamw> so:
17:40:12 <adamw> what should we change it to?
17:40:28 <roshi> big enough
17:40:31 <roshi> :p
17:40:44 <rdieter> adamw: 2gb as nirik suggested worksforme
17:41:28 <adamw> roger
17:42:22 <adamw> proposed #agreed as the KDE SIG representative, rdieter affirms that the KDE spin target size is changed to 2GB from F26. Note that the Spins process does not appear to have been properly followed for Fedora 26 cycle, so we consider it appropriate to make the adjustment in this meeting
17:42:31 <adamw> only rdieter gets to ack/nack
17:42:32 <adamw> :P
17:42:44 <jkurik> :)
17:42:44 <roshi> ack
17:42:47 <rdieter> ack
17:42:48 <roshi> you can't stop me!
17:42:51 <adamw> .fire roshi
17:42:51 <zodbot> adamw fires roshi
17:42:55 * rdieter acks roshi's ack
17:43:01 <adamw> #agreed as the KDE SIG representative, rdieter affirms that the KDE spin target size is changed to 2GB from F26. Note that the Spins process does not appear to have been properly followed for Fedora 26 cycle, so we consider it appropriate to make the adjustment in this meeting
17:43:03 <roshi> ha ha!
17:43:05 <rdieter> ack ack ack
17:43:08 <adamw> so, following on from that:
17:43:09 <roshi> thanks for the backup rdieter
17:43:28 <jkurik> rdieter: thanks for joining us; you have saved the F26 Beta release
17:43:56 <adamw> proposed #agreed #1457645 - close as fixed or notabug - as the target size has now been changed, the image is no longer over-sized
17:44:02 <roshi> ack
17:44:09 <jkurik> ack
17:44:13 <mboddu> ack
17:44:21 <adamw> oh, and yeah, someone should definitely look into when and why the images got bigger
17:44:38 <adamw> hey, i know a person with a tool that can quickly generate a series of image size data...that would be me
17:44:40 <adamw> *sigh*
17:44:49 <adamw> #agreed #1457645 - close as fixed or notabug - as the target size has now been changed, the image is no longer over-sized
17:45:20 <adamw> #topic (1348688) Anaconda cannot access LVM partitions in a LUKS-encrypted disk partition after decryption
17:45:20 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1348688
17:45:20 <adamw> #info Accepted Blocker, lvm2, ON_QA
17:46:33 <adamw> hmm, we don't have an absolute confirmation that this is fixed
17:46:43 <adamw> the +1s on the update are those folks that +1 every update
17:47:00 <adamw> but the fix is at least *in* the RC and no-one's said it's broken...
17:47:20 <nirik> I think it was fixed here (I had a encrypted setup on the macbook and unlocked it when I reformated them)... but it's hard to say for sure as my macbook is acting wacky now.
17:47:21 <adamw> any brno folks still around? know if vpodzime ever re-tested this?
17:47:36 <adamw> nirik: there was a specific way of doing the LVM+luks setup to trigger this bug
17:47:42 <adamw> the way anaconda does it by default does *not* trigger the bug, apparently
17:48:12 <nirik> ok. I would have had default setup
17:50:49 <adamw> so...it'd be nice to have confirmation of this, but we can probably treat it as addressed
17:51:14 <adamw> unless someone feels like running through https://bugzilla.redhat.com/show_bug.cgi?id=1348688#c39 while we all wait
17:52:08 * roshi has an install ready to go
17:52:09 <roshi> I'll do it
17:52:34 <jkurik> roshi++
17:52:39 <nirik> roshi++
17:52:40 <zodbot> nirik: Karma for roshi changed to 11 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:53:00 <adamw> shall we do the other one and circle back to this when roshi's done testing?
17:53:02 <adamw> roshi++
17:53:02 <zodbot> adamw: Karma for roshi changed to 12 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:53:06 <adamw> .fire roshi
17:53:06 <zodbot> adamw fires roshi
17:53:07 <roshi> sure thing
17:53:10 <adamw> (just to keep things in balance)
17:53:14 <roshi> :D
17:53:28 <jkurik> yes, lets move on
17:53:42 <adamw> roshi: remember to check with some older image that you can reproduce the bug before checking with 1.4 that it *doesn't* hit the bug...
17:53:56 <adamw> #info roshi is confirming the fix for this, we will discuss the next bug and then circle back to this one
17:54:06 <adamw> #topic (1397087) Rpm hangs after updating to glibc >= 2.25
17:54:06 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1397087
17:54:06 <adamw> #info Accepted 0-day Blocker, rpm, MODIFIED
17:54:08 <roshi> ah, right
17:54:14 <adamw> So, this one is Accepted0Day / AcceptedPreviousRelease
17:54:26 <adamw> as of this morning, the devs have cooked up something they *hope* will fix it properly
17:54:31 <adamw> did anyone test it yet?
17:54:55 * nirik has not yet
17:56:42 <adamw> also, does anyone remember that debate we had about exactly what the requirements for accepted0day / acceptedpreviousrelease should be?
17:56:44 * nb has not seen that before, what is Accepted0Day for?
17:56:45 * roshi burns another usb key
17:56:48 <adamw> istr kparal pushing for a fairly strict definition
17:57:07 <adamw> nb: it means 'fix must be sent out as an update by the day of release but doesn't necessarily need to be in the compose itself'
17:57:19 <nb> oh ok
17:57:25 <adamw> acceptedpreviousrelease is the same, but for the current stable releases rather than the release that's being validated
17:59:04 <jkurik> I do not have a strict definition but I would expect it is something like a blocker which can be delivered as an update and not neccessary be on image
18:00:07 <adamw> the strict definition is at https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Normal.2C_0-Day_and_Previous_Release_blockers .
18:02:03 <mboddu> adamw: what happens if we dont have a fix for Accepted0Day by the release time?
18:02:06 <adamw> i know we've *had* this debate about exactly what the requirements for 0day/previousrelease blockers at go/no-go should be, but i can't find it :/
18:02:22 <adamw> mboddu: that's the tricky part; technically once we've decided Go at this meeting, we can't...un-Go
18:02:38 <adamw> so if we were to say 'oh sure the update will be stable by then' and say Go and then it *isn't* stable by then, we can't abort
18:02:59 <mboddu> adamw: okay
18:03:00 <adamw> which is why, IIRC, kparal argued that the fixes must be pushed stable before the 'go' decision
18:03:11 <adamw> but i really can't find the discussion or our conclusions right now :/
18:04:40 <adamw> so the way i see it, we have three choices (yay choices!)
18:04:44 <jkurik> I guess the fixes must be pushed stable and *verified/tested* before the 'go' decision
18:05:06 <adamw> jkurik: well in theory, going stable means they're verified/tested :P
18:05:15 <jkurik> ok :)
18:05:25 <adamw> 1) rush in some karma on the update, submit it for stable, and say we're good (this kinda ignores the issue that we don't have updates for 24 or 25 yet...)
18:05:51 <adamw> 2) ignore that whole 'no abort button' problem i just explained and say "we're sure the updates will be stable by release day, so we're good"
18:06:18 <adamw> 3) slip
18:06:42 <nirik> it's not clear to me if we need 24/25... or if the fix on the f26 package fixes the upgrades from those older releases...
18:06:49 <adamw> well, I guess, 4) waive the bug as a blocker on...some basis or other...but that's a bit less practical now it seems like the bug *is* fixable
18:06:53 <adamw> nirik: yeah, i asked in the bug
18:06:59 * nirik nods
18:06:59 <mattdm> I'm not loving any of those choices
18:07:00 <adamw> i dunno if any of the relevant devs are online
18:07:08 <adamw> mattdm: feel free to invent a new one :P
18:07:17 <adamw> pwhalen: are you around?
18:07:21 <roshi> glad I'm testing this other bug instead of trying to decide on this one :p
18:07:44 * nirik leans toward 2... but we should indeed test it and try and karma it in the next few days if possible
18:08:10 <pwhalen> adamw, I am, testing upgrades on a couple systems now. but, I'd feel much better if others at least tested installs on x86
18:08:27 <adamw> yeah, we can at least quickly test some of the known-problematic packages on x86...
18:08:37 * adamw goes to do that quickly
18:08:42 <mattdm> In the event that #2 is catastrophically wrong, we can document the problem in several prominent places
18:08:58 <adamw> (see, i told you it was complicated)
18:09:06 <nirik> also if we find it fails we can possibly see if they can try another fix and get that one out...
18:09:09 <mattdm> I guess that's my preference in *this particular case*, but I don't necessarily want that to be precedent-making
18:10:01 <roshi> everything is precedent
18:10:06 <adamw> if folks can go ahead and test some stuff while we argue, that's great
18:12:13 * adamw tests in production, because why not
18:12:37 <roshi> ...there's other places to test?
18:13:35 <adamw> heh
18:13:36 <mattdm> nothing so authentic, though :)
18:16:03 <mattdm> It seems like the arguing is done :)
18:16:32 <adamw> well, we're on the testing now...
18:16:44 <adamw> i do worry a bit about rushing out a change to libdb
18:17:38 <mattdm> just for the record there is an answer to nirik/adamw's question in the bug from pkubat...
18:17:47 <mattdm> #info Note that previous releases do not need to have the fixes applied as all of the magic happens after libdb is updated to the fixed version.
18:19:09 <mattdm> I agree that we don't want to rush out a libdb change. My vote is still on #2
18:19:36 <adamw> other thoughts?
18:20:16 <mattdm> adamw: Am I right in assuming that this will not bite people doing fresh installs of the beta?
18:20:21 <jkurik> after reading the whole story in 1394862 and 1397087 I am to the #2 as well
18:20:27 <adamw> mattdm: yes. that's why it's not AcceptedBlocker
18:20:49 <mattdm> adamw: cool just double-checking
18:24:18 <mboddu> adamw: is #2 is just what you explained? We say Accepted0Day in hope we will have the fix in stable by release time and if we dont, we will release it anyway?
18:24:24 <adamw> no.
18:24:30 <adamw> wait
18:24:54 <adamw> I described #2 as: "ignore that whole 'no abort button' problem i just explained and say "we're sure the updates will be stable by release day, so we're good""
18:25:03 <adamw> that is not really what Accepted0Day is *supposed* to mean
18:25:26 <adamw> Accepted0Day is *supposed* to mean 'we only say Go if we're 100% sure a fix will be stable by the release date"
18:26:12 * roshi is having a harder time replicating this on bare metal than I thought I would
18:26:44 <mattdm> I'm feeling, like, 85% sure :)
18:28:27 <adamw> roshi: this bug or the previous one?
18:31:18 <adamw> so...
18:31:40 <roshi> the LUKS one
18:31:49 * roshi had wanted to do it on bare metal
18:32:19 <adamw> proposed #info 1397087 - A fix for this is now in testing. As this is a sensitive and complex bug, we do not want to rush it stable, but we are quite confident a fix will be pushed stable by release day. If it is not we will make every effort to document the issue in all relevant locations.
18:32:30 <adamw> roshi: eh, virt is fine
18:33:57 <mattdm> adamw: +1
18:35:46 <jkurik> adamw: ack
18:36:15 <nirik> ack
18:37:13 <adamw> #info 1397087 - A fix for this is now in testing. As this is a sensitive and complex bug, we do not want to rush it stable, but we are quite confident a fix will be pushed stable by release day. If it is not we will make every effort to document the issue in all relevant locations.
18:37:14 <pwhalen> both upgrades are stuck. upgrading to libdb-5.3.28-21.fc26 on an existing f26 systems was ok.
18:37:32 <adamw> oof :/
18:37:36 <adamw> so upgrades are failing with -21?
18:37:44 <adamw> where they worked with -19 (but it caused the other problems)?
18:37:57 <mattdm> soooo, hold the presses. :)
18:38:24 <adamw> hmm
18:38:38 <adamw> it occurs to me that the comment in the bug isn't right
18:38:44 <adamw> well
18:38:59 <pwhalen> actually -19 did cause problems 99% of the time, Ive only had a couple successes with it
18:39:16 <adamw> if we only have the update for f26, then isn't it possible for an upgrade transaction to run into the bug before the new libdb is ever installed?
18:39:26 <adamw> if glibc/pthread/whatever is updated before libdb?
18:39:37 <pwhalen> perhaps?
18:40:02 <adamw> #undo
18:40:02 <zodbot> Removing item from minutes: INFO by adamw at 18:37:13 : 1397087 - A fix for this is now in testing. As this is a sensitive and complex bug, we do not want to rush it stable, but we are quite confident a fix will be pushed stable by release day. If it is not we will make every effort to document the issue in all relevant locations.
18:40:51 <mattdm> soooo. My certainty has dropped to, like, 5%
18:41:25 <jkurik> hmm.. does it mean option #3 ?
18:41:44 <mattdm> Well, at least that would answer the July 4 release day issue :)
18:42:37 <adamw> yeah, we're kinda down to #3 - slip or #4 - waive this as a blocker on the basis of complexity and uncertainty about fixing it, or something
18:42:59 <adamw> it'd help if pkubat were around, but i don't think he is :/
18:43:10 <mattdm> I think this is a blocker for very legitimate reasons :(
18:44:58 <roshi> looks like 1.3 fixes the issue for the LUKS bug
18:45:06 <adamw> well, that's good
18:45:12 <adamw> (why'd you test 1.3 not 1.4? doesn't matter much, though)
18:46:06 * jkurik was hoping for the GOLD status today :(
18:46:22 <adamw> i think we all were!
18:46:30 <adamw> but if this is busted, it's busted...
18:46:33 * nirik nods.
18:46:46 <mattdm> ah-yup
18:47:25 <nirik> the sucky part is that if this gets fixed before tuesday we could have gone... but we don't knwo that now.
18:47:31 <nirik> anyone have a time machine? ;)
18:48:02 * roshi tested that first
18:48:13 <jkurik> nirik: I do, bud I have upgraded it to the 1.4 Beta compose and it is stuck now
18:48:20 * adamw just noticed the user named namidairo, possibly the most emo name ever
18:48:39 <jkurik> I assume it makes no sense to go through Test matrices, right ?
18:48:47 <roshi> works with 1.4 too
18:48:48 <adamw> well, we gotta make a formal agreement on this first
18:48:58 <adamw> let's cycle back to that other bug first...
18:49:06 <jkurik> ok
18:49:10 <adamw> #info circling back to the LVM bug to close out on that
18:49:57 <jkurik> roshi: thanks for the testing
18:50:08 <roshi> sorry it took so long
18:50:10 <adamw> #topic (1348688) Anaconda cannot access LVM partitions in a LUKS-encrypted disk partition after decryption
18:50:10 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1348688
18:50:11 <adamw> #info Accepted Blocker, lvm2, ON_QA
18:50:27 <adamw> #info roshi confirms this is fixed in Beta-1.3 and Beta-1.4, so it is considered addressed
18:50:34 <adamw> #topic (1397087) Rpm hangs after updating to glibc >= 2.25
18:50:34 <adamw> #link https://bugzilla.redhat.com/show_bug.cgi?id=1397087
18:50:34 <adamw> #info Accepted 0-day Blocker, rpm, MODIFIED
18:53:14 <adamw> so, sounds like consensus is not to fudge this, but consider it still an unaddressed blocker and slip?
18:53:33 <jkurik> unfortunatelly yes
18:53:36 <nirik> I dont suppose the maintainer is around? they are in brno I guess?
18:54:00 <roshi> I don't suppose we can postpone the decision until we get feedback from the maintainer, can we?
18:54:05 <mboddu> adamw: lets say we got a fix, do we still need to do the compose since its Accepted0Day?
18:54:29 <jkurik> yes, the maintainer is from Brno (9PM already)
18:54:33 <adamw> mboddu: no, no new compose is needed
18:54:40 <adamw> roshi: i don't think so
18:54:48 <adamw> mboddu: that's the point of having the 'special' blocker statuses
18:54:57 <mboddu> adamw: well, thats good news for me :D
18:55:03 * roshi didn't know about waiting until they get in tomorrow
18:55:03 <adamw> yep :)
18:55:14 <adamw> if any other 'regular' blocker appears we may re-spin, of course
18:55:27 <mboddu> adamw: yeah, sure
18:56:06 <nirik> I suppose we could try the 'keep the meeting open until tomorrow' thing... I really don't know how hard it's going to be to fix the current patch to work...
18:56:55 <jkurik> mboddu, dgilmore: what is your POV ? ^^^
18:57:05 <adamw> given the sensitivity of the issue and the fact it seems like we've now tried and failed to fix it like three times, i'm getting less amenable to trying to fudge it in three working days
18:57:09 <jkurik> will releng have enough time to copy to mirrors etc... ?
18:57:13 <adamw> er, sensitivity of the *package*
18:57:21 <nirik> true. ;(
18:58:34 <dgilmore> jkurik: bits have to go on the mirrors tomorrow for Tuesday, Saturday for Wednesday or Sunday for Thursday release
18:59:10 <jkurik> dgilmore: ok, thanks
18:59:40 <dgilmore> jkurik: Wednesday or Thursday release means a releng person has to work on a weekend
19:00:22 <jkurik> I know, I was just checking to be sure I did not miss something
19:00:31 <jkurik> sounds like we are going to convert Accepted0Day to AcceptedBlocker, right ?
19:00:35 <adamw> no
19:00:41 <adamw> i think you don't understand the difference
19:00:48 <adamw> Accepted0Day isn't some kind of 'blocker-lite' thing
19:00:48 <dgilmore> still Accepted0Day
19:00:52 <adamw> it's still release blocking
19:01:11 <adamw> it just means that the bug's nature is such that the fix doesn't have to be included in the *compose*
19:01:18 <dgilmore> it just has to be fixed in an update before Tuesday
19:01:20 <adamw> it's not any less...blocking
19:01:37 <jkurik> yes, but we need to make the decision today
19:01:41 <adamw> (but as noted earlier, we can't really guarantee anything that isn't queued for stable *now* will be stable by Tuesday)
19:01:42 <adamw> right
19:01:55 <nirik> I can sorta see a case for talking to the maintainer tomorrow morning and deciding based on that... but I am also fine with just slipping a week I guess.
19:01:59 <dgilmore> the risk is that if we say we are go for everything else, unless this bug is fixed by Tuesday we slip
19:02:20 <dgilmore> but given we would have the compose on the mirrors waiting we could easily ship Wednesday or Thursday
19:02:57 <nirik> dgilmore: well we could stage things and if it's fixed by tuesday just go tuesday too... we just don't know tho.
19:03:06 <dgilmore> nirik: right
19:03:15 <dgilmore> we can stage what we have tomorrow
19:03:30 <dgilmore> we just could not ship until libdb is fixed
19:04:00 <adamw> dgilmore: we have always said that go/no-go is a binary non-reversible decision
19:04:03 <nirik> normally this isnt an option, but if this is the only thing blocking...
19:04:16 <adamw> that's always been very strongly stated by you, fpl, fpm etc.
19:04:20 <adamw> if we say 'go' we're gone
19:04:36 <adamw> any time the idea of 'go, but...' has come up it's always been shot down, hard
19:04:40 <adamw> i don't really mind if that's changed, but it seems odd
19:05:11 <adamw> sorry, I meant "if we say 'go' we're releasing on Tuesday"
19:05:22 <jkurik> can we propose a deadline for the fix ? like Friday 15:00 UTC ? and then make an "automatic" non-reversible decision based on the fix availability ?
19:05:23 <dgilmore> adamw: I guess i am saying we say this is good stage it
19:05:40 <dgilmore> and wait to make the actual go decision until we have the fix
19:05:55 <nirik> well, I was thinking it was still not reversable
19:06:26 <dgilmore> adamw: I am saying we postpone the go until we have a fix
19:06:30 <adamw> dgilmore: similarly, it's always been stated that we can't decide 'go' any later than this meeting
19:06:31 <nirik> if we say "go, and no release until libdb fix is in updates" thats what we do... we don't then say "Oh, look another blocker monday night, lets make a new compose"
19:06:33 <dgilmore> though thats ugly and messy also
19:06:34 <mattdm> I'm in favor of "go, but..." if it's technically possible. But there are some other logistics too
19:06:35 <adamw> and we can only do this meeting once a week
19:06:56 <mattdm> For example: RH puts out a press release for the betas, and that has to clear legal
19:07:03 <mattdm> and needs to be scheduled with all that stuff
19:07:07 <mattdm> which is somewhat black magic to me
19:07:12 <mattdm> but I know they need some lead time
19:07:37 <adamw> nirik: oh, so you're not really trying to fudge a release by tuesday or on a different day next week, but rather saying 'let's lock down everything but the libdb fix'?
19:07:47 <mattdm> (knowing monday morning is probably okay; knowing monday eob for next morning almost certainly aint)
19:07:49 <nirik> that was my thought...
19:07:57 <adamw> but still release next tuesday?
19:08:03 <adamw> (assuming libdb is fixed by then...)
19:08:07 <nirik> yeah
19:08:10 <adamw> er, tuesday 13
19:08:18 <adamw> man, i'm confused now
19:08:27 <nirik> we all are. ;)
19:08:50 <adamw> can anyone who is proposing anything but a regular one-week slip please lay out, in detail, what they are proposing?
19:09:37 <jkurik> adamw: can we propose a deadline for the fix ? like Friday 15:00 UTC ? and then make an "automatic" non-reversible decision based on the fix availability ?
19:09:52 <adamw> and explain how it is consistent with the 'Meeting Outcomes' section of https://fedoraproject.org/wiki/Go_No_Go_Meeting , the procedure for this meeting?
19:09:59 <dustymabe> start to stage, postpone decision til tomorrow, plan to release wednesday ??
19:10:41 <nirik> proposal: we are go with this compose. If the libdb fix is not available in the updates repo with a fix by tuesday, we slip a week. Otherwise we release then.
19:10:52 <adamw> i thought we had decided releases could only happen tuesdays.
19:11:11 <nirik> I guess the downside of that is that we wouldn't care about any other found bugs.
19:11:15 <adamw> nirik: i don't believe we can wait until tuesday to make that decision, can we?
19:11:19 <nirik> s/care/block/
19:11:26 <adamw> all the other bits of the release process take time, and start happening right after we declare 'go'.
19:11:34 <nirik> I suppose not indeed.
19:12:15 <nirik> so I guess it's just easier to slip a week than try and figure out how to deal with this 0dayupdate blocker. ;(
19:12:41 <adamw> i believe one option that is more or less consistent with all the rules is 'leave the meeting open, start staging bits but don't do anything that commits us to a release on june 6 yet, make a final decision tomorrow as part of this same meeting'
19:13:04 <nirik> yeah, we have done that before... but I think always when the meeting was on wed...
19:13:29 <roshi> heh
19:13:31 <adamw> yeah, i don't know if we made a decision tomorrow morning if that would give enough time for all processes to happen by tuesday
19:13:36 <adamw> does anyone know the answer to that?
19:13:49 <nirik> I think it would.
19:13:51 <jkurik> I do not
19:14:16 <dustymabe> so there was a rule about only releasing on Tuesdays?
19:14:19 * nirik looks at dgilmore and his 'no' stamp. ;) what does it say?
19:14:21 <dustymabe> just confirming
19:14:21 <adamw> yes, i'm pretty sure there is.
19:14:27 <adamw> dgilmore, is that right?
19:14:39 <nirik> dustymabe: yeah. There was a bunch of talk about adjusting that, but it never got anywhere I don't think
19:15:51 <mattdm> tuesday is good from a "getting press" standpoint too
19:16:17 <dgilmore> one sec
19:16:31 <mattdm> rule is documented https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle
19:16:32 <jkurik> there is a set of time extensive tasks to be done from releng side (mirroring etc.) as I understand and it is always good to this over weekend
19:16:55 <dustymabe> jkurik: i think everyone was proposing that we start those processes now anyway
19:17:13 <dustymabe> even if we wait til tomorrow to decide
19:17:17 <nirik> oh, I was wrong
19:17:23 <nirik> https://pagure.io/fesco/issue/974
19:17:41 <dustymabe> 'FESCo accepted the proposal on the 2012-11-28 meeting.'
19:17:43 <dustymabe> nice
19:18:07 <mboddu> afaik, we can get things done for Tue release if we can make a decision by tomorrow morning, but we can wait on dgilmore respond
19:18:20 <mboddu> s/respond/response
19:18:52 <jkurik> yeah, that why I was proposing the deadline to be at 15:00UTC tomorrow
19:19:17 <adamw> i don't have any personal objection to the 'delay final decision till tomorrow' plan, really, except other groups have always pushed back against that kinda thing before. but i suppose it helps that we can at least stage the compose since this is a 0day
19:19:42 <nirik> right, and we don't need to recompose or retest a new compose.
19:19:53 <adamw> "This is one of the reasons why we moved go/no-go to Thursday after F17; so it wasn't possible to have the 'one more day'."
19:19:54 <adamw> heh
19:20:02 <nb> lol
19:20:05 <nirik> nice.
19:20:14 <adamw> regarding that ticket, okay fesco accepted the proposal, but i see zero followup after that
19:20:30 <dgilmore> sorry was on the phone to fedex
19:20:32 <adamw> which doesn't make me super confident that all groups are actually prepared to *deliver* a release on any day but a tuesday
19:20:44 <roshi> yeah
19:21:31 <roshi> but if there isn't a new compose to test, that takes away the main "let's not do heroics" feature of the only tuesday release
19:21:33 <dgilmore> dustymabe: releng has a thing about releasing on tuesdays only
19:21:41 <dgilmore> but that is to do with staging
19:21:51 <dgilmore> if we leave it stage longer we are okay with that
19:22:26 <dgilmore> we could make the decision that what we want to ship is done and good, we can stage it
19:22:32 <dustymabe> dgilmore: since we don't have to recompose, we could go ahead and start staging now?
19:22:41 <adamw> yeah
19:22:46 <dgilmore> we can then decide the actual date to ship it based on the extenral dependency being resolved
19:23:19 <adamw> okay, so let me try and form a coherent proposal out of this...
19:23:29 <dgilmore> so we would not be asking releng people to work weekends
19:23:42 <adamw> well, the agreed for the *bug* is easy
19:23:46 * nirik notes he is out next week, so it's up to you suckers^Wfolks to implement this. ;)
19:23:48 <adamw> it seems like everyone agrees it needs fixing
19:24:07 * jkurik agrees
19:24:12 <dgilmore> adamw: given the number of times i have hit the issue it needs fixing
19:24:20 <nirik> yeah, it needs fixing.
19:24:26 <adamw> proposed #agreed #1397087 - this is agreed to still be a clear blocker, and is at present unaddressed. However, as it's a 0Day / PreviousRelease blocker, it does not in itself invalidate the Beta-1.4 compose
19:24:33 <nirik> and beta is often when people upgrade to the next release stream. ;(
19:24:50 <nirik> ack
19:24:53 <dgilmore> ack
19:24:56 <mboddu> ack
19:24:56 <jkurik> adamw: ack
19:24:59 <adamw> all the wrangling about what we actually *do* in light of this is a separate question
19:25:02 <roshi> ack
19:25:04 <dustymabe> acky bracky heart
19:25:05 <adamw> so jkurik gets to formulate the proposal for that :P
19:25:10 <adamw> #agreed #1397087 - this is agreed to still be a clear blocker, and is at present unaddressed. However, as it's a 0Day / PreviousRelease blocker, it does not in itself invalidate the Beta-1.4 compose
19:25:23 <adamw> okay, that's all the blockers, i believe (unless anyone proposed another in the last hour or two)
19:25:36 <adamw> #info that covers all the outstanding proposed / agreed blockers
19:25:40 <adamw> back to you, jkurik
19:26:01 <jkurik> roshi, adamw: thanks for the blocker review
19:26:07 <roshi> np
19:26:26 <jkurik> Lets check Test matrices before we move on....to make the final decision
19:26:44 <jkurik> #topic Test Matrices coverage
19:26:52 <jkurik> #link https://fedoraproject.org/wiki/Test_Results:Fedora_26_Beta_1.4_Summary
19:27:16 <jkurik> I do see these testcases not covered:
19:27:20 <jkurik> QA:Testcase_install_to_hardware_RAID
19:27:21 <jkurik> QA:Testcase_install_to_SAS
19:27:23 <jkurik> QA:Testcase_realmd_join_kickstart
19:27:25 <jkurik> ARM Domain controller
19:27:26 <jkurik> ARM Database server
19:27:42 <adamw> SAS is our standing joke
19:27:51 <adamw> hw RAID, yeah, i'm usually the one who does that and my test box is broekn
19:28:02 <adamw> i can run out and buy some bits for it and hopefully test it later today
19:28:38 <jkurik> atm, do we see any of these test cases as critical, to wait for test results ?
19:28:40 <adamw> realmd_join_kickstart for AD is untested, true...other join methods are tested for AD, and kickstart is tested for freeipa...
19:28:48 <adamw> sgallagh: can you test kickstart by any chance?
19:29:20 <jkurik> adamw: sgallagh stated he will not be available today
19:29:24 <adamw> i think just x86 coverage for the roles is probably okay, we don't actually mandate every box be filled (yet)
19:29:27 <adamw> ah, sorry
19:29:45 <adamw> so, if we're delaying the final decision anyway, i think we can fill out the coverage acceptably
19:29:48 <dustymabe> coconut is a machine
19:29:55 <dustymabe> coconut++
19:29:55 <zodbot> dustymabe: Karma for coconut changed to 1 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
19:29:56 <jkurik> yeah, I agree that x86 should be sufficient
19:30:02 <adamw> dustymabe: coconut *is* a machine, yes :)
19:30:05 <adamw> dustymabe: coconut is openQA
19:30:12 <dustymabe> haha, i was like, damn
19:30:13 <adamw> hence the bot icon
19:30:39 * roshi read that as dusty letting people know that coconut is a bot
19:30:40 <roshi> lol
19:30:40 * dgilmore can try do the two arm ones
19:30:45 <adamw> funny story: i gave a talk on openQA at a conference once and someone came up after and told me they know the person who designed the icon i chose to denote bot results...
19:31:11 <adamw> dgilmore: manual freeipa testing is kind of a bitch, that's why i automated it :) but thanks
19:31:26 <jkurik> ok, so anyone is opposed to waive the missing test cases ?
19:31:32 <adamw> well, i wouldn't want to waive all of those
19:31:44 <adamw> but i think we can get enough of them done by tomorrow
19:32:08 <adamw> i'd probably be okay with waiving SAS and the one AD join test
19:32:21 <dgilmore> what adamw said
19:33:13 * dgilmore wonders if anyone has tested aarch64, ppc64 or ppc64le?
19:33:29 <dgilmore> they all seem mia from the matricies
19:33:36 * dgilmore runs
19:33:43 <nirik> I tested ppc64le cloud... worked fine. ;)
19:33:49 <dustymabe> nirik: nice
19:34:40 <jkurik> proposed #agreed the following test cases needs to be sucessfully tested by tomorrow: "Testcase_install_to_hardware_RAID", "Testcase_install_to_SAS" and on ARM platform "Domain controller" and "Database server". Beside of this the coverage of the testing is considered as sufficient.
19:35:32 <adamw> nack
19:35:35 <adamw> not SAS
19:35:36 <adamw> never SAS
19:35:41 <roshi> lol
19:35:42 <dgilmore> nak
19:35:50 <adamw> (except that one time someone showed up with an actual SAS device and it frickin' failed, sheesh)
19:36:08 <adamw> aside from that, yeah
19:36:11 * dgilmore goes to buy adamw a sas controller
19:36:28 <adamw> dgilmore: too bad you don't know my address any more :P
19:36:34 <jkurik> proposed #agreed the following test cases needs to be sucessfully tested by tomorrow: "Testcase_install_to_hardware_RAID" and on ARM platform "Domain controller" and "Database server". Beside of this the coverage of the testing is considered as sufficient.
19:36:36 <nirik> you know we could probibly keep around some server that has sas in our datacenter...
19:36:38 * dgilmore has a sas controller but no sas disks
19:36:46 <jkurik> adamw: better ? ^^^
19:36:55 <adamw> ack
19:36:58 <dgilmore> ack
19:37:02 <jkurik> uff :)
19:37:08 <dustymabe> rule of thumb, if nobody complains about it, nobody is using it
19:37:25 <dustymabe> orrr, it's working as expected :)
19:37:41 <jkurik> #agreed the following test cases needs to be sucessfully tested by tomorrow: "Testcase_install_to_hardware_RAID" and on ARM platform "Domain controller" and "Database server". Beside of this the coverage of the testing is considered as sufficient.
19:38:14 <jkurik> #topic Go/No-Go decision
19:38:22 <jkurik> so....
19:38:34 <jkurik> we have two options now
19:38:37 <jkurik> 1) slip
19:39:28 <jkurik> 2) start staging the current RC as "GOLD", delay the decision for one day waiting for the libdb fix
19:39:47 * dgilmore waits on adamw
19:40:11 <dgilmore> I am inclined to go with 2
19:40:42 <adamw> well
19:41:02 <adamw> with my QA hat on, i'm a bit concerned about waving through the libdb fix with one day or less of testing
19:41:16 <adamw> but not enough so to be a hard 'no' on 2), it just worries me
19:41:26 <adamw> i'm OK with locking down Beta-1.4 as the gold compose
19:41:33 <dgilmore> adamw: it worries me also
19:41:36 <adamw> well...except for those missing tests, i guess
19:41:44 <dgilmore> given that the fixes so far have been worse
19:41:45 <mattdm> With my FPL hat on, I have the same feeling
19:41:49 <adamw> if we do that and then find out hardware RAID is busted, then what...
19:41:52 <mboddu> dgilmore: correct me if I am wrong, we cannot stage until we have all the test matrices covered, right?
19:42:03 <dgilmore> mboddu: not true
19:42:04 <adamw> right, there's that too (wasn't thinking of that earlier)
19:42:19 <adamw> dgilmore: what if we find a blocker in one of those tests?
19:42:22 <mboddu> dgilmore: what if there is a blocker issues comes up in testing?
19:42:32 <dgilmore> adamw: we run rmr -rf and say sorry and slip
19:42:35 <adamw> we can probably have the tests done soon, though (the sooner this meeting winds up, the sooner i can go buy some ram)
19:42:37 <adamw> dgilmore: heh, true
19:42:54 <jkurik> we can meet tomorrow on i.e. 15:00UTC and make the final decision knowing results of the testcases and the state of the libdb bug
19:43:02 <mboddu> dgilmore: if you are ok with rm, then I am fine too
19:43:31 <dgilmore> mboddu: I want to avoid you or me having to spend our weekend working on it
19:43:55 <mboddu> dgilmore: +1 to that :)
19:44:05 <dgilmore> adamw: what does the sas test need?
19:44:19 <adamw> dgilmore: hardware. no-one has it.
19:44:30 <adamw> except that one time someone who *did* have it showed up. don't think i've seen him since, though.
19:44:33 <dgilmore> adamw: well i have an old server with a sas controller
19:44:38 <adamw> i suppose we should just go buy some, or something.
19:44:50 <nb> well, if it is that obscure, then either we should have Fedora buy some hardware, or remove the criterion
19:44:51 <nirik> we have tons of it in infra... but not any servers sitting around idle doing nothing that can be reinstalled. ;(
19:44:52 <dgilmore> adamw: but I do not have a sas disk
19:45:15 <dgilmore> so i could test sata disk on sas controller
19:45:23 <adamw> we really should just buy one, i'll propose that the qa team buys one before i forget.
19:45:39 <adamw> i've always treated it as a bit of a joke, but if we have a bunch of servers actually using it, we probably should test it..
19:45:45 <dgilmore> #agreed QA team to buy sas hardware
19:46:02 <dgilmore> #action adamw to not forget to buy sas hardware
19:46:22 <adamw> #action dgilmore to remind adamw not to forget to buy sas hardware
19:46:36 <dgilmore> :D
19:46:49 <dgilmore> adamw: so is using a sata disk enough or no?
19:47:19 <jkurik> can we meet tomorrow for the "Go/No-Go" decision ?
19:47:34 <dgilmore> jkurik: sure
19:47:36 <jkurik> or is anyone proposing slip ?
19:48:05 <jkurik> the proposed time 15:00 UTC is fine ?
19:48:28 <adamw> dgilmore: i dunno enough about SAS to know.
19:48:42 <nirik> whats that in brno time? will we have time to talk to the maintainer?
19:49:01 <jkurik> 15:00 UTC is 17:00 in Brno
19:49:10 * dgilmore proposes jkurik stand over the maintainers desk until we have an answer
19:49:25 <mboddu> jkurik: thats 11 EST, so I think we are good
19:50:00 <jkurik> adamw, roshi ^^^ what about QE ?
19:50:07 <roshi> I'll be there
19:50:10 <mattdm> EDT :)
19:50:10 <jkurik> great
19:50:14 <nirik> sounds good to me.
19:50:14 <adamw> was there an actual formal proposal to ack / nack yetr?
19:50:24 <adamw> but yeah, i guess QE can roll with making a final decision tomorrow
19:50:28 <roshi> now it's just to figure out what else I need to either do tonight or tomorrow morning to be ready
19:50:45 * mattdm must go to another different but equally thrilling meeting now
19:50:52 <adamw> roshi: mainly, test the libdb update and be ready to test any subsequent / additional ones
19:51:02 * adamw must go buy some RAM so he can run the hw raid test
19:51:16 <nirik> Oh, I wanted to bring up a non blocker, but likely a lot of people will hit thing....
19:51:21 <adamw> and junk an old amplifier, and buy a new tv, and go pick up a recycling box from his old apartment...
19:51:24 <adamw> nirik: uh huh?
19:51:24 <dgilmore> adamw: no formal proposal
19:51:41 <roshi> adamw: well, I meant more look at the matrices and see what else we'd want coverage on
19:51:42 <adamw> jkurik: note in order to technically remain in compliance with the meeting procedure, we cannot end the meeting
19:51:47 <nirik> there's claims that i686 live media don't work... https://bugzilla.redhat.com/show_bug.cgi?id=1436096
19:51:50 <adamw> we just have to leave it running till we make a decision
19:51:57 <nirik> I know it's not blocking, but just FYI.
19:52:18 <adamw> nirik: openqa sez it does:
19:52:19 <adamw> https://openqa.fedoraproject.org/tests/overview?distri=fedora&version=26&build=Fedora-26-20170531.0&groupid=1
19:52:25 <roshi> worked for me with 1.3
19:52:33 <adamw> (though it tests the i686 media on x86_64 virt CPU)
19:52:36 <nirik> ok, good... then it's likely hw specific.
19:53:21 <roshi> I'll test bare metal after this image burns
19:53:35 * dgilmore has not had 32 bit x86 hardware in at least 8 years
19:53:40 <jkurik> proposed #agreed: the Go/No-Go decision is postponed. Release Engineering starts to stage the current 1.4 Beta compose as GOLD asap. The "Go/No-Go team" meets at 15:00 UTC on #fedora-meeting-2 channel to finish the meeting and make the ultimate decision.
19:53:59 <dgilmore> ack
19:54:11 <nirik> I'm not sure what to reassign that bug to, but it's clearly not a Xfce related issue.
19:54:15 <nirik> ack
19:54:26 <roshi> ack
19:55:17 <adamw> patch
19:55:20 <adamw> oh, no, ack
19:55:26 <adamw> (as we can cancel the stage if we find a blocker today)
19:55:41 <adamw> jkurik: maybe state a day not just a time?
19:55:46 <jkurik> the next meeting on this channel is on Monday, so we can leave it open ....
19:56:11 <jkurik> proposed #agreed: the Go/No-Go decision is postponed. Release Engineering starts to stage the current 1.4 Beta compose as GOLD asap. The "Go/No-Go team" meets on 2017-Jun-02 at 15:00 UTC on #fedora-meeting-2 channel to finish the meeting and make the ultimate decision.
19:56:35 <roshi> ack
19:56:40 <adamw> jkurik: yeah, that's why we have the meeting here :)
19:56:41 <adamw> ack
19:56:46 <jkurik> :)
19:57:01 <jkurik> #agreed: the Go/No-Go decision is postponed. Release Engineering starts to stage the current 1.4 Beta compose as GOLD asap. The "Go/No-Go team" meets on 2017-Jun-02 at 15:00 UTC on #fedora-meeting-2 channel to finish the meeting and make the ultimate decision.
19:57:51 <jkurik> so, thanks for today and lets meet tomorrow ( I will try to chase the maintainer as much as possible during the morning tomorrow )
19:59:30 <adamw> thanks jkurik
19:59:42 <nirik> jkurik++
19:59:42 <zodbot> nirik: Karma for jkurik changed to 9 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
20:00:03 <dustymabe> thanks all
20:00:51 * adamw goes off to buy stuff
20:01:08 * roshi goes to get that coffee I *should* have got before we started this thing...
00:02:25 <bexelbie[m]> .nextmeeting #fedora-meeting-2
00:02:28 <zodbot> bexelbie[m]: In #fedora-meeting-2 is Fedora Ambassadors Latam Meeting (starting in 2 days)
00:02:31 <zodbot> bexelbie[m]: In #fedora-meeting-2 is Workstation WG (starting in 3 days)
00:02:34 <zodbot> bexelbie[m]: In #fedora-meeting-2 is Fedora Release Engineering (starting in 3 days)
00:02:37 <zodbot> bexelbie[m]: - https://apps.fedoraproject.org/calendar/location/fedora-meeting-2%40irc.freenode.net/
01:30:31 <KQfMtFwD> this meeting is still happening?
01:32:08 <KQfMtFwD> just got in, waiting on a compose or something?
15:00:06 <jkurik> #chair dgilmore nirik adamw roshi mboddu
15:00:06 <zodbot> Current chairs: adamw dgilmore jkurik mboddu nirik roshi
15:00:16 <jkurik> Hi go/no-go team
15:00:22 <mattdm> hello!
15:00:27 <jkurik> Hi mattdm
15:00:37 <mboddu> .hello mohanboddu
15:00:38 <zodbot> mboddu: mohanboddu 'Mohan Boddu' <mboddu@bhujji.com>
15:01:37 <nirik> morning
15:01:58 <jkurik> hi mboddu, hi nirik
15:02:14 <jkurik> roshi, adamw: anyone from QA ?
15:04:33 * mattdm needs to step away for a few minutes. will return shortly
15:04:42 <jkurik> kparal: can you represent QA untill adamw or roshi join ?
15:05:38 <kparal> damn, I wasn't paying attention to the remaining blockers at all. but I can, I guess :)
15:05:44 * roshi is here
15:05:52 <kparal> great :)
15:05:53 <roshi> I wasn't going to not have coffee again
15:05:56 <roshi> like last time
15:06:00 <jkurik> kparal, roshi: thanks for joining
15:06:02 <roshi> sorry
15:06:04 <roshi> :)
15:06:12 <jkurik> so...
15:06:15 <jkurik> So, currently afaik the issue with libdb seems to be solved and the build -21 should be ready to be delivered as 0day update.
15:06:35 <jkurik> I have got confirmation from several people the -21 build of libdb is OK
15:06:39 <dustymabe> hey team
15:06:43 <jkurik> Has anyone any other info ?
15:06:47 <jkurik> dustymabe: hi
15:07:04 <pwhalen> morning folks. to provide an update from my testing yesterday. during the meeting starting with f25 I upgraded to the new libdb-5.3.28-21.fc26 and did a distro-sync excluding libdb. this hung at the same spot in the bz (krb5-libs). After the meeting I set up a lookaside with libdb-5.3.28-21 and did the upgrade with the dnf plugin. this succeeds (multiple upgrades complete on armhfp and aarch64)
15:07:27 <jkurik> pwhalen++
15:07:27 <zodbot> jkurik: Karma for pwhalen changed to 1 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:07:32 <mattdm> pwhalen++
15:07:32 <zodbot> mattdm: Karma for pwhalen changed to 2 (for the f25 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:07:32 <jkurik> thanks for the good news
15:07:43 <kparal> also, it received -1 karma half an hour ago, due to ppc64le bug
15:07:49 <mattdm> So, _online_ update with dnf might hang, but *offline* update safe?
15:07:50 <kparal> https://bodhi.fedoraproject.org/updates/FEDORA-2017-a4c41ecc27
15:07:51 <nirik> there's a report from sharkcz on the bug that it didn't work for him
15:07:56 <roshi> I tested it and didn't get a hang in RPM, but tbh I'm not 100% sure what I'm looking at for this bug
15:07:57 <pwhalen> since the meeting, libdb-5.3.28-21 was built in f25. sharkcz tried the upgrade using that and reproduced what I saw yesterday.,
15:08:26 * nirik thought we didn't need f24/f25 updates. hum.
15:08:43 <pwhalen> I dont think we do, and it seems to reproduce the issue we saw
15:09:02 <jkurik> yeah, for now I would focus on F26 only
15:09:37 <nirik> jkurik: is the maintainer still around, can you invite them here?
15:09:44 <jkurik> beside of this issue ( https://bugzilla.redhat.com/show_bug.cgi?id=1397087 ) I have not seen any new blocker
15:09:51 <jkurik> nirik: let me check...
15:10:31 <jkurik> no, I do not see him online (not even on the internal RH irc)
15:10:53 <nirik> ok
15:11:24 <nirik> so all cases of upgrading using the old libdb initially seem to work?
15:11:35 <roshi> that's what it seems like
15:11:37 <jkurik> afaik yes
15:11:39 <pwhalen> nirik, yes.
15:12:21 * sharkcz can recheck without updated f25 libdb, shouldn't take long
15:12:46 <jkurik> kparal: regarding the ppc64le bug - do you see it as a blocking issue ?
15:12:56 <dustymabe> should we blacklist updating libdb in f25 ?
15:13:33 <nirik> well, they should get -karma at least and not pushed stable
15:13:35 <kparal> jkurik: that depends. will it prevent us to push it stable?
15:13:41 <nirik> and possibly revoked
15:14:44 <sharkcz> mattdm: let me check the offline upgrade first
15:14:52 <pwhalen> thanks sharkcz
15:15:00 <jkurik> do we consider ppc64le arch as blocking one ?
15:15:17 <dgilmore> im kinda here
15:15:35 <jkurik> hi dgilmore
15:15:52 <mboddu> jkurik: I think the problem is if its a blocking arch or not, if it gets enough negative karma then it wont be pushed to stable and that means it will be still a Accepted0Day blocker
15:16:25 <jkurik> sure
15:16:40 <dgilmore> jkurik: ppc64le is not blocking
15:16:52 <nirik> so the f25/f24 updates were pushed by besser82... not adamw or the maintainer... wonder why they did that
15:16:59 <kparal> the fact that we want to fix primary arch doesn't mean we should break an alternate one
15:17:00 <nirik> I don't think the arch matters here.
15:17:06 <pwhalen> I dont think he tests them
15:17:08 <roshi> yeah, ppc64le isn't blocking
15:17:12 <nirik> so far it''s breaking the same way everything else did
15:18:35 <nirik> my understanding: upgrading from the existing stable libdb in f24/f25 -> f26 with the -21 libdb works. upgrading from the -21 libdb in f24/f25 -> f26 with -21 libdb fails.
15:18:48 <roshi> that's what it looks like
15:18:57 <pwhalen> nirik, right
15:19:25 <kparal> so far it seems the fix needs more baking in the oven
15:19:43 <nirik> kparal: well, so far it looks like the fix needs to not push anything to 24/25
15:19:49 <roshi> but not for beta
15:20:08 <roshi> seems like beta is good, and the 24/25 issue can be handled outside this meeting
15:20:26 <dustymabe> roshi: +1
15:20:52 <roshi> or am I getting it wrong?
15:21:17 * nirik is reading the bug... it's not super clear
15:21:55 <jkurik> as I understand it, the F24/F25 fixes were already pushed, so we will not be able to update to F26 Beta now
15:22:02 <jkurik> am I correct ?
15:22:21 <nirik> no.
15:22:31 <nirik> they were built and submitted to updates-testing this morning
15:22:47 <jkurik> ah, that is better then
15:23:40 <besser82> Yes?
15:23:51 <nirik> besser82: you built and submitted libdb updates for f24/f25 this morning?
15:23:59 <besser82> Yes
15:24:02 <nirik> they seem to break things.
15:24:04 <jkurik> if we block the F24/F25 updates, we can go on with F26 Beta as it is now (including the libdb -21 build), right ?
15:24:23 <nirik> jkurik: that's what I was hoping for
15:24:27 <roshi> jkurik: as I understand it, yes
15:25:17 <besser82> nirik, the updates are in updates-testing-peding
15:25:46 <besser82> will unpush them
15:25:55 <nirik> besser82: right, but people testing: apply that update -> upgrade to f26 with libdb -21 see it hang and not work... but upgrading without the new one in 24/25 works
15:26:20 <nirik> upgrading from the existing stable libdb in f24/f25 -> f26 with the -21 libdb works. upgrading from the -21 libdb in f24/f25 -> f26 with -21 libdb fails.
15:26:29 <besser82> mhh…  that's odd somehow…
15:26:47 <nirik> my only concern is panu's comment: https://bugzilla.redhat.com/show_bug.cgi?id=1394862#c80
15:28:34 <jkurik> however the maintainer comments is saying: the install order should be glibc -> libdb -> whatever package that depends on libdb
15:28:50 <jkurik> which is not aligned with panu's comment
15:29:34 <mattdm> I'm feeling like this is all too dodgy. Let's slip and get it right.
15:29:57 <nirik> and they said they removed the patch from f25/f24 branches, but it seems like it's still there?
15:30:19 * kparal supports mattdm
15:30:21 <jkurik> yeah, I am inclined to slip as well. This is becoming a black magick
15:30:35 <roshi> makes sense to me
15:31:24 <besser82> nirik, I'll fix it in SCM by making the patch being applied on F26+ only, in case of any rebuilds…
15:32:10 <nirik> I'm worried that the f26 fixed one only handles some common case ordering or something... and others will still break.
15:33:00 <nirik> how many successes have we had with just the f26 -21 one?
15:33:15 <besser82> According to Panu's comment breakage should not happen with -21 libdb installed in F24+ when upgrading to F26
15:33:33 <dgilmore> mattdm: +1
15:33:54 <jkurik> proposed #agreed The decision from this meeting is No-Go for the F26 Beta compose 1.4. The main concern for not releasing this compose as GOLD is unclear resolution of bug #1397087, which is still causing some issues during testing.
15:33:54 <nirik> besser82: yeah, in theory... but in practice it seems we have several folks who hit the hang there.
15:34:12 <mattdm> jkurik: +1
15:34:30 <dgilmore> ack
15:34:33 <nirik> yeah, sadly I agree.
15:34:39 <mboddu> ack
15:34:40 <roshi> ack
15:34:50 * roshi had been hoping this would work
15:34:56 <jkurik> #agreed The decision from this meeting is No-Go for the F26 Beta compose 1.4. The main concern for not releasing this compose as GOLD is unclear resolution of bug #1397087, which is still causing some issues during testing.
15:35:06 <roshi> but at least, for now, there's not another RC to have to run through the tests for
15:35:06 <jkurik> roshi: yes, I was also hoping :(
15:35:47 <nirik> yeah. oh well, lets get it right/fixed right.
15:35:55 <roshi> yep
15:36:05 <jkurik> #action releng to stop mirroring the F26 Beta 1.4 compose
15:36:40 <mattdm> jkurik: Can you also send a note to devel-announce reminding people that the F27 schedule won't change because of this?
15:36:53 <jkurik> #action jkurik to plan next go/no-go meeting for the next Thursday 2017-June-08
15:37:06 <mattdm> Particularly, that means that system wide change propsals are due the week *before* the F26 release
15:37:08 <jkurik> mattdm: sure
15:37:19 <mattdm> and changes requiring a mass rebuild are due *quite some time* before
15:37:20 <jkurik> #action jkurik to announce the outcome of this meeting
15:37:24 * nirik notes he is gone next week, so happy going hopefully to everyone. ;)
15:37:32 <mattdm> and the mass rebuild I guess starts the day afterward
15:37:39 <jkurik> #action jkurik to announce there are no changes in F27 due to this slip
15:38:32 <besser82> Shall the -21 libdb for F26 be unpushed, too?
15:38:44 * mattdm runs off to lunch before he starves to death
15:38:48 <nirik> no, I would say leave it for now...
15:38:56 <nirik> and we can sort out in the bug what we want to do?
15:39:11 <besser82> Okie, sounds like a plan
15:39:23 <jkurik> mattdm: unfortunatelly the F26 slipped for more than month from the original baseline
15:39:45 <mboddu> jkurik: lets hope we wont hit any delays with final release
15:39:56 * adamw arrives
15:39:57 <jkurik> fingers crossed
15:39:58 <adamw> sorry folks, didn't read the time right
15:40:00 <dustymabe> is it possible for us to still release 1.4
15:40:03 <dustymabe> one week later?
15:40:12 <dustymabe> with the libdb update sorted out
15:40:17 <mboddu> dustymabe: yes, its still a possibility
15:40:24 <jkurik> adamw: we are done
15:40:24 <nirik> dustymabe: sure, if nothing else is found...
15:40:31 <adamw> jkurik: so i see
15:40:33 <dustymabe> would be nice for us to be able to lift freeze sooner than later
15:40:56 <dustymabe> that won't happen though
15:41:08 <jkurik> ok, so is there anything else anyone would like to share/ask ?
15:41:09 <nirik> adamw: you can now tell us what a horrible mistake we made. ;)
15:41:35 <mboddu> dustymabe: we cant lift the freeze until we have a definitive GOLD compose
15:42:16 <dustymabe> mboddu: well we kinda had one of those :)
15:42:36 <dustymabe> 1.4 *was* going to be gold
15:42:46 * dustymabe stops talking now
15:42:48 <dustymabe> sorry
15:42:50 <nirik> yeah, but if we say... "this 1.4 is done" and find some horrible bug next week... why not do another and fix it?
15:43:03 <nirik> since we are slipping to sort out libdb anyhow.
15:43:07 <roshi> just as a reminder
15:43:21 <roshi> people *do* respect that when we find these things we slip and fix it
15:43:30 <roshi> that we don't push out shoddy stuff just to meet a deadline
15:43:43 <roshi> while I wanted to release too, I'm glad that we do this
15:44:07 <jkurik> seems like we can close the meeting in one minute if noone has anything else ....
15:44:28 <mboddu> jkurik: what about test matrices and other tests like hardware raids?
15:44:54 <mboddu> jkurik: Can we just get a status on it?
15:45:01 <jkurik> ok
15:45:11 <mboddu> jkurik: since they are not completely done yesterday
15:45:24 <roshi> imo, our coverage looks good
15:45:46 <jkurik> afaik except "Domain controller" for ARM all the required testcases are done
15:46:20 <roshi> still waiting on SAS :p
15:46:45 * dgilmore failed at doing sas and starts domin controller
15:46:51 <dgilmore> but need to finish it
15:47:45 <jkurik> yesterday we adgreed the SAS testcase is not needed, did not we ?
15:47:57 <dgilmore> we did
15:48:20 <roshi> yeah, we did
15:48:22 <jkurik> roshi, adamw: can you please comment on the "Domain controller" for ARM testcase ?
15:48:38 <roshi> and we're going to try to get some hardware for it in the future for the QA team
15:48:57 <adamw> i think we could probably waive it, but it'd be nice if someone with arm hardware could run it in the next week
15:49:01 <jkurik> roshi: are you talking about the SAS or ARM one ?
15:49:09 <roshi> SAS
15:49:12 <adamw> i guess i could set my dev openQA instance to running it in virt, it *might* finish by next thursday :P
15:49:38 <jkurik> adamw++
15:50:02 <jkurik> so, we will review this on the next Go/No-Go meeting the next Thursday
15:50:12 <roshi> sgtm
15:50:55 <jkurik> anything else on Test matrices ?
15:51:30 <dgilmore> adamw: I started the domaincontroller
15:51:53 <dgilmore> sharkcz: if you oculd point me at how you do it in openqa I will run that
15:51:54 <jkurik> #help it'd be nice if someone with arm hardware could run "Domain controller" for ARM testcase
15:52:54 <jkurik> #info Test matrices wiil be reviewed again on the Go/No-Go meeting on 2017-June-08
15:53:00 * dgilmore got as far as running "rolectl deploy domaincontroller --name=test"
15:53:57 <jkurik> looks we are done; can we close the meeting ... if no one has anything else .... ?
15:54:17 <sharkcz> dgilmore: the f25->f26 upgrade? the IBM guys might have it in their openqa, or did you mean sgallagh rather? :-)
15:54:51 * roshi doesn't have anything
15:55:22 <sgallagh> dgilmore: https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/master/f/tests/role_deploy_domain_controller.pm
15:55:42 <dgilmore> sharkcz: domaincontroller
15:56:29 <jkurik> thanks for comming people
15:56:40 <jkurik> see (most of you) on the next Thursday
15:56:51 <jkurik> #endmeeting