00:00:49 #startmeeting F-13-Final readiness meeting 00:00:49 Meeting started Wed May 12 00:00:49 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:00:49 Useful Commands: #action #agreed #halp #info #idea #link #topic. 00:01:00 #meetingname f-13-readiness 00:01:00 The meeting name has been set to 'f-13-readiness' 00:01:12 #topic Gathering crit-mass 00:01:19 * poelcat 00:01:22 * stickster 00:01:41 * jlaska 00:01:56 * etank is here to lurk 00:02:16 * jlaska experiencing long irssi lag 00:02:33 I see Oxf13 as well 00:02:59 adamw: is likely here (Canucks are on tonight) 00:03:38 I don't think notting is available, perhaps spot in his absence? 00:03:49 i'm here 00:03:55 * jlaska tips hat 00:05:09 let's get moving ... 00:05:19 #topic Why are we here? 00:05:27 Quick recap for folks new to this meeting 00:05:50 * spot is here 00:05:58 #info The purpose is to decide whether the Final release criteria have been met 00:06:05 #link https://fedoraproject.org/wiki/Fedora_13_Final_Release_Criteria 00:06:15 * nirik is lurking around too. 00:06:26 welcome etank spot nirik 00:06:49 For those playing the home game, details on this meeting are available at http://fedoraproject.org/wiki/Engineering_Readiness_Meetings 00:07:06 Now for the main event ... 00:07:13 #topic Go or No Go? 00:07:34 i'll take suitcase #14 00:08:07 First off, there remains OPEN (NEW + ASSIGNED) F13Blocker bugs on the list 00:08:23 #info OPEN F13Blocker bugs -- http://tinyurl.com/2v5hyqu 00:08:31 * poelcat notes that is criteria #2 00:08:44 that should be changed slightly 00:08:48 to open /accepted/ blockers 00:08:56 since anybody can toss a bug on the blocker list 00:09:04 #info All bugs blocking the F13Blocker tracker must be CLOSED 00:09:32 basically we should discuss whether to accept these proposed blockers here 00:09:33 I'm open to walking the list *briefly* for issues QA couldn't negotiate the impact on 00:09:50 are there any obvious ones that can be removed? 00:10:02 From my viewpoint, none of the current bugs on the list constitute a Fedora 13 blocker. 00:10:43 let's just take 'em in order? 00:10:57 sure 00:11:07 Yeah, let's start with 588147 00:11:16 #info https://bugzilla.redhat.com/show_bug.cgi?id=588147 00:11:26 what order are you working in? 00:11:57 Oxf13: I'm going by the tinyurl link above ... I'll just shout out the numbers if that's okay with folks? 00:12:06 topic it 00:12:14 sure, it's just that the bug you mention was in the middle of the list 00:12:40 #topic https://bugzilla.redhat.com/show_bug.cgi?id=588147 00:12:55 anyway, this is an abrt crash, which I can't reproduce, and doesn't seem to be reproducible by many other people. 00:13:08 yeah, this is a very newly-sprung one 00:13:15 it'd be nice to know what the actual problem is here 00:13:29 #info In most cases, there must be no SELinux 'AVC: denied' messages or abrt crash notifications on initial boot and subsequent login (see Blocker_Bug_FAQ) 00:13:41 mclasen seems to be offline 00:13:44 I've only seen 1 report of this issue, so the "in most cases" is under question 00:13:45 the person updated on May 1st, you'd think we'd have seen a lot more noise on this if it was systemic 00:13:49 does anyone have an impact assessment? 00:13:52 Oxf13: agreed 00:14:40 the changelog just says "- Fix a crash due to a race condition at login ", which isn't terribly revealing 00:15:39 yeah, i haven't seen this on any of my installs 00:15:42 ENOTENOUGHINFO 00:15:51 not sure how much more time we need to spend here. Does anybody really think this is a issue worthy of slipping the release for? 00:15:52 * stickster hasn't seen this on any of his F13 00:16:06 * etank has not seen this at all either 00:16:15 * jlaska starts #agreed 00:16:30 given the lack of dupes, no. but it would be nice to know what the actual bug is :( 00:17:08 #agreed 588147 - not a release blocker - unable to reproduce, unclear whether reported problem remains, not a lot of reports from testers about this failure 00:17:19 Next ... 00:17:25 #topic https://bugzilla.redhat.com/show_bug.cgi?id=590640 00:17:38 want me to paste the "agreed" actions to the bugs? 00:18:00 this one worried me at first, but now it seems we can't reproduce it 00:18:02 poelcat: please, that'd be great 00:18:04 jlaska: what's your latest word on this? 00:18:09 I'm still a bit concerned on this 00:18:28 #info discovered while testing NFS installs (http://fedoraproject.org/wiki/Test_Results:Current_Installation_Test) 00:18:48 I thought I had this reproduced, boot.iso + repo=nfs:server:path 00:18:56 to be fair, we don't list what the expected boot method is 00:18:59 jlaska: yeah, i think this is a blocker, although it is troublesome that you can't reproduce it 00:19:04 both for the test, and for anaconda support 00:19:09 However, there may be more to it. So far, kparal, rhe and I have all encountered this issue 00:19:14 when you boot via pxe, this certainly doesn't happen 00:19:24 http://docs.fedoraproject.org/install-guide/f13/en-US/html/ch04s06s02.html <-- documentation for prepping a NFS install 00:19:30 which in my opinion is the more common NFS install boot method 00:19:34 any anaconda folks to give an assessment of this> 00:19:34 Yes, booting just the vmlinuz+initrd.img (PXE) doesn't trigger this failure 00:19:35 ? 00:19:36 "The installer must be able to use all supported local and remote package source options " 00:20:06 I honestly think the more common case is to boot some form of physical (DVD, CD or boot.iso), but I could be wrong 00:20:51 We could always provide an updates.img to use for this network install scenario, but it's just unsettling that we don't know root cause yet 00:20:56 http://docs.fedoraproject.org/install-guide/f13/en-US/html/s1-begininstall-nfs-x86.html <-- the second half of NFS installation docs. 00:20:59 dlehman is lurking, but not sure if he's still around 00:21:09 stickster: thx teading 00:21:28 what i'd really like is a better understanding of the bug, but this meeting is now :( 00:22:20 The installation instructions don't specify the boot media for the NFS install 00:22:29 He Rui reported he was able to reproduce this? 00:22:48 Yeah, she, Kamil and I all hit the bug 00:22:52 ]she 00:23:00 oops, my apologies 00:23:04 * stickster doesn't know He 00:23:12 they encountered it while using boot.iso (I think) and a local NFS mirror of branched 00:23:26 i think even if a reproducer is elusive, the fact that 3 qa testers hit this make it a legit blocker 00:23:31 which could indicate a problem with their local mirror 00:23:35 I encountered this using the boot.iso and a local NFS exploded DVD image 00:23:39 ah 00:23:48 but you can't re-encounter it 00:24:11 My latest theory is that NetworkManager was recycling the IP while I was off doing something else, I hit Next and it crashes 00:24:23 somehow losing the nfs handle or connecting in the interim 00:24:32 but that's just a theory and still need to chase this down 00:24:52 so ... shall we vote? 00:24:53 I can't duplicate it here using boot.iso and the branched NFS tree 00:25:26 Oxf13: I've retested many times today with boot.iso and local nfs exported DVD ... and hit it 1 of 5 attempts 00:25:39 so I think there might be more to it than than just boot.iso + repo=nfs:server:path 00:25:45 but just don't have that info yet 00:26:08 wait, I take it back. I did manage to hit it, it just happened much later than I would have expected. 00:26:21 it's after reposetup, and right when you start package install 00:26:30 so .. I'll start ... 00:26:31 well that's a bummer :/ 00:26:34 so that's another one who's hit it 00:26:39 jlaska: i wonder if cache-fs is to blame here 00:26:51 jlaska: the traceback shows it being initialized 00:26:51 boy that'd be shitty if this is what causes us to slip 00:26:56 +1 blocker...it's too obviously icky and uncertain :/ 00:26:59 +1 00:26:59 spot: hmm, didn't think of that ... could be 00:27:24 so, we have a couple workarounds though. 00:27:50 Oxf13: I was trying to find a more comprehensive list of workarounds, but hadn't pinpointed the reproducer yet to fully expore those 00:27:52 jlaska: it might be worth seeing if there is a way to disable that and see if it works around the issue 00:28:13 spot: I can do that 00:28:24 do we want to agree on whether this is a blocker for this meeting 00:28:32 and we can then discuss how to handle the issues? 00:28:40 whether it's a slip, or continued debugging etc... 00:28:43 sorry. +1 for a blocker. 00:28:44 if we agree it's a blocker then the release slips, afaik. 00:28:44 well, this is the go / no go, so... 00:28:59 jlaska: seems like it is doesn't meet the criteria and there is no clear work around 00:29:18 there is a clear work around 00:29:18 It meets the criteria, at least #3 00:29:27 boot using vmlinzu and initrd instead of boot.iso 00:30:16 Oxf13: That is a valid workaround, but requires a PXE setup, or an existing installed system 00:30:16 Oxf13: I'm not entirely sure that's clear, but even if so, i'm not really convinced that "use a different install method" is an acceptable workaround, given criteria #3 00:30:32 spot: you're confusing "install method" with "boot method" 00:30:32 or boot.fedoraproject.org ? 00:30:50 I'm 100% not comfortable with that workaround 00:31:04 I really don't feel that it's super common to boot with spinning boot media and use an NFS source 00:31:24 well, its either supporter or it isn't. 00:31:28 Oxf13: so you're a -1 for blocker 00:31:35 i don't think "super common" is in the criteria. 00:31:52 supported, rather. 00:31:56 spot: for the sake of slipping the release for a week I think "super common" absolutely matters. 00:31:57 * spot is tired. 00:31:58 * stickster could say what he feels about that kind of scenario based on his previous job, but we're just throwing around guesses. 00:32:02 * etank has personally never done a NFS based install. 00:32:26 jlaska: yes, I'm -1 00:32:55 I think we have the votes that this is a blocker :( 00:33:15 that or we change the criteria 00:33:39 * poelcat thinks now is not the time to go down that road 00:33:45 poelcat: +1 00:33:48 poelcat: +1 00:33:56 that's why we did this in December :) 00:34:13 which is great and all, if we had a 8-ball and could see into the future 00:34:22 sometimes you just don't think through all situations until you hit them. 00:34:24 poelcat: down the road of adjusting the criteria? 00:35:08 oh sorry, misread 00:35:24 yeah, I'm not sure what else to say on this bug 00:35:53 * jlaska prepares the #agreed stamp 00:35:57 * nirik plays the sad sliping trombone 00:36:24 really going to ruin my day. 00:36:46 it just seems silly we're going to throw it all away for an issue that hardly seems common, certainly not common enough to find it before a day or so ago. 00:36:47 do we want to look at the rest of them before deciding for sure? or is this it... 00:37:07 if a bug's a blocker, we slip. 00:37:10 nirik: If we have a blocker identified that's not cleared, that's it. 00:37:11 #agreed 590640 - a valid blocker bug, still uncertain how far the impact may extend. Boot.iso + repo=nfs:server:path likely impacted 00:37:21 It's still worth going through the rest 00:37:22 we could look at what else we would take fixes for, though, i guess. 00:37:41 although right now, I really don't feel like spending any more time on Fedora. 00:37:44 adamw: why would we do that if it is blocker only from here on out? 00:38:06 #info some debate around how common NFS installs are for users 00:38:06 because we might want to fix other blockers before we try again? 00:38:24 poelcat: to decide if they're blockers and hence also need to be fixed :) 00:38:30 * Oxf13 thinks we're getting a little too wrapped up on following the letter of our criteria instead of applying a little judgment. 00:38:32 let's clear as much of this list 00:38:33 adamw: sorry, i took that the wrong way 00:38:53 Oxf13: what part of the criteria do you feel needs ammending? 00:39:13 poelcat: the part that is making us slip because of this bug. 00:39:18 Oxf13: I agree there is a degree of judgement that is needed, but in this case we don't fully understand the failure conditions. So it's hard to say what the full impact is for me 00:39:24 Oxf13: i don't agree, i tink it's reasonably clearcut: we decided NFS is a supported install path and we can't release the product with our supported install paths not working. nfs install clearly is not working properly. ergo we don't release the product that way. 00:39:42 adamw: NFS as an install path works, provided you boot via pxe 00:39:51 which in my experience has been the far more common case when doing NFS base dinstalls 00:40:03 Whereas my experience has been the opposite :-) 00:40:05 never mind, circular. 00:40:34 if NFS didn't work with any boot method, that'd be one thing. 00:40:38 Next bug ... 00:40:55 yeah, lets move on, this horse isn't getting any deader. 00:41:04 stickster: if that were really the case, why did it only show up... yesterday? We haven't changed anaconda in while. 00:41:44 Oxf13: good questions, and we'll definitely have that debate once we figure out root cause 00:41:51 #topic https://bugzilla.redhat.com/show_bug.cgi?id=590874 00:42:30 "funny", this might be the cause for the NFS failure 00:42:40 but again, speculation 00:42:42 it's not 00:43:04 simple work around here, don't check UTC, or set your PC clock correctly before doing the install 00:43:16 I was seeing a lease renew attempt from NM in my failure cases of the previous bug 00:43:17 uhh, don't check UTC? 00:43:23 Isn't UTC default? 00:43:26 yeah. 00:43:54 I think the situation is when the PC clock is not already set to UTC, or is wildly off 00:43:56 the report says to *uncheck* system clock uses UTC 00:44:16 I managed to reproduce the clock jump. Just set your timezone to the other half 00:44:17 (to reproduce the bug) 00:44:17 oh right 00:44:20 of the world and uncheck "system clock uses UTC", that should do it. 00:44:28 ^^that was from Orion in the bz 00:44:31 so if you leave the default, it'll work 00:44:57 * jlaska has the spidey sense that this and the previous bug are related 00:45:03 yes, but 'the default' isn't always the right choice here; it's a simple fact whether your system clock is set to UTC or not, it's not really a matter of preference whether you select it or not 00:45:31 does this issue impact the final release criteria? 00:45:37 (i believe the point here being that windows doesn't really work right with the system clock set to UTC?) 00:45:37 So am I correct that this would bite just about anyone in Asia who's already using UTC (e.g. a current Linux user)? 00:45:57 stickster: no, it affects you if your clock does *not* use UTC (and you're aware of this and intentionally uncheck that box) 00:46:04 Ah, OK 00:46:12 Thanks -- so more like the dual-boot user then. 00:46:13 stickster: I think you'd have to uncheck use UTC, and/or have your system clock already set to something vastly wrong. 00:46:52 adamw: I think that's older versions of windows. My dualboot system handles UTC just fine with vista 00:47:23 ok 00:47:36 yeah, i think this is a reasonably minor issue, nice to fix given that we're slipping, but not a blocker. 00:48:00 given that we're now essentially asking 'do we take a fix for this bug', it might be good to have an opinion from hughsie on the impact and the impact of a fix 00:48:37 * nirik cautions against 'oh, we are slipping, lets fix these 100 things' 00:48:38 hughsie? 00:48:44 I gather he's not a Canucks fan? 00:48:52 nirik: agreed 00:48:54 nirik: Yup, we need to be very selective 00:49:08 adamw: perhaps you mean dcbw ? 00:49:14 d'oh 00:49:21 does it show i've been drinking all night? :) 00:49:47 I don't think we'll get an answer on this one here 00:50:13 if we comfortable taking a well tested and isolated fix, we can F13Target this issue and move on 00:50:29 jlaska: Insert a comment in bug looking for impact assessment? Or manage that elsewhere? 00:50:29 jlaska: +1 00:50:34 bottom line we have to decide if this was the last bug on the list, would we slip the release for it 00:50:40 in my opinion, that's no. 00:50:59 stickster: both, CommonBug, and ping the bz for impact assessment 00:51:04 jlaska: thanks 00:51:17 I'm +1 to F13Target for this 00:51:38 sure 00:51:39 +1 to F13Target 00:51:42 anyone else? 00:52:39 move on, looks like 00:52:49 #agreed 590874 - Move to F13Target, add to CommonBugs and seek impact assessment 00:52:59 #topic https://bugzilla.redhat.com/show_bug.cgi?id=590960 00:53:31 adamw's comment#5 sums it up for me 00:53:52 yea, -1 on blocker. 00:54:04 -1 on blocker 00:54:12 I have a 8400M GS that works fine, fwiw 00:54:27 although on comment #8 i'd like to know if it's the exact same card that fails in the other machine 00:54:32 i have 2 different systems with nvidia cards and i have never seen this 00:54:33 but overall, -1 00:54:39 -1, workaround known anyways 00:55:12 #agreed 590960 - not a blocker, no widespread reports of 8400M failures 00:55:31 jlaska: You may want to back that out -- I checked the OP's smolt and it's a 8600 GT 00:55:34 I'm skipping the preupgrade bug 587627 .. hughsie has a handle on this, is aware that this needs to be fixed and available in F12 00:55:41 #undo 00:55:41 Removing item from minutes: 00:55:52 so same decision, but with 8600GT? 00:55:59 yes 00:56:02 My fault -- I misread comment #5 00:56:18 even if all 8600GT's fail, it's a single model, we don't usually block for that. especially since there's an (ugly) workaround. 00:56:27 agreed 00:56:34 #agreed 590960 - not a blocker, no widespread reports of failures with reported chipset 00:56:51 last one ... one we're all familiar with 00:56:52 #topic https://bugzilla.redhat.com/show_bug.cgi?id=590661 00:57:25 fwiw, i still think this is a blocker, behavior regression. 00:57:42 #info I reached out to FESCO for their opinion on the blockery-ness of this issue (see https://fedorahosted.org/fesco/ticket/377) 00:57:43 frankly if the fix for this is low-impact i'd *really* like to take it. 00:58:10 Fesco voted to not slip specifically for it 00:58:10 perhaps it's book keeping at this point, but based on FESCO's review, shall we move this to F13Target 00:58:21 and accept a well tested fix for the issue? 00:58:22 but since we're slipping, and since the fix is very low impact, we can take it 00:58:33 jlaska: sure, end result is the same. 00:58:37 adamw: +1, doubly so (+2?) because a number of factors coincide to make this exactly the kind of bug that could be a ridiculous amount of bad press for us for no good reason 00:58:38 spot: yeah 00:58:47 however, I think we should still move this to Target, since we would likely make the same decision if it for whatever reason was still broken next week 00:59:22 sure, make it target not blocker, that's fine. 00:59:30 #agreed 590661 - should move to F13Target, will accept a well tested fix for this issue considering the potential negative press 00:59:37 * spot isn't sure i agree with that, but given that there is a tested fix in hand... whatever. 00:59:51 spot: roger 01:00:03 * stickster neither, but it's not worth arguing right now since we have our decision for slip already 01:00:07 * nirik thinks it's kinda moot. 01:00:09 #partial-agree :) 01:00:13 "I get the car, I get the house" 01:00:19 heh 01:00:21 okay, that's the list 01:00:38 what about 587627? 01:00:43 again, I'm skipping the F-12 preupgrade issue 01:00:51 okay. 01:01:10 rationale: the bug is in f12's preupgrade, it doesn't affect f13 compose 01:01:12 588597? 01:01:18 We can discuss that if folks want, but hughsie is aware of the deadline for deliverying the F-12 preupgrade 01:01:26 it blocks f13 _release_ but not _compose_, we only need to ship an f12 update in time for f13 release to be okay 01:01:29 so down to 3 blockers then. less is better. 01:01:42 (we've been through this dance with f12, we more or less have a procedure for it) 01:01:46 not even 3 01:01:54 1 ? 01:01:56 AFAIKT there is exactly one issue we're slipping for 01:01:57 spot: That's MODIFIED, I think it's tagged ? 01:02:07 stickster: yeah, you're right, sorry 01:02:16 Oxf13: you can move that issue to VERIFIED now? 01:02:17 * stickster willing to be corrected by smart folk 01:02:25 588021 01:02:30 which issue? 01:02:32 i don't think that there is a confirmed fix there 01:02:40 even though it is in MODIFIED 01:02:43 Oxf13: 588597 01:03:15 do we want to #topic through the MODIFIED F13Blockers ? 01:03:29 jlaska: the only one that seems to be noteworthy imho is 588021 01:03:39 .bug 588021 01:03:41 nirik: Bug 588021 iwlagn hangs the system randomly due to a rate scaling bug - https://bugzilla.redhat.com/show_bug.cgi?id=588021 01:04:00 especially given that there is only a scratch build done for it 01:04:02 adamw: have you seen any forum reports of success with F-13-RC? 01:04:40 spot: adamw: The current kernel is -85, right? That would have the iwlagn fix I see noted 01:05:07 stickster: are you sure? 01:05:14 yes, -85 ought to have the fix 01:05:23 spot: I have a local mirror of f-l-development/13/x86_64/ I'm referring to 01:05:29 $ koji latest-pkg --quiet dist-f13 kernel 01:05:29 kernel-2.6.33.3-85.fc13 dist-f13 ajax 01:05:30 jlaska: the forum threads may not be reproducers; there's no direct feedback etiher way 01:05:45 thanks Oxf13 01:05:47 spot: -85 is definitely the kernel in rc2. i've been testing it all day. =) 01:06:05 no, i mean, are you sure it has the fix from the scratch build? 01:06:06 recommendations? 01:06:49 Tue Mar 30 2010 John W. Linville 2.6.33.1-24 01:06:49 - Avoid null pointer dereference introduced by 'ssb: check for sprom' (#577463) 01:06:51 spot: the changelog indicates that was in -82 01:07:01 Oxf13: that's not it 01:07:07 whoops, wrong kernel 01:07:11 * Tue May 04 2010 John W. Linville 2.6.33.3-82 - iwlwifi: recalculate average tpt if not current (#588021) 01:07:14 That's it 01:07:25 * Tue May 04 2010 John W. Linville 2.6.33.3-82 01:07:25 - iwlwifi: recalculate average tpt if not current (#588021) 01:07:28 okay. just being thorough. :) 01:07:31 adamw beat me to it. 01:07:34 it wasn't clear from the bz 01:07:35 :-) 01:07:37 so all signs seem to indicate a fix is available, still waiting on test confirmation? 01:08:05 FWIW I'm using -85 and iwlagn here for some time now. 01:08:38 yeah, confirmation from reporters would be nice. 01:08:38 Any other bugs before moving to next #topic ? 01:08:49 I'm on iwlagn but kernel -72 01:08:51 and no issues 01:08:57 so I don't know that I can help 01:09:09 #topic https://bugzilla.redhat.com/588021 01:09:32 #info fix appears to be included in 2.6.33.3-82 kernel (in RC#2) 01:09:45 #help confirmation of fix from reporters would be appreciated 01:09:57 Okay, let's end this meeting ... 01:10:06 #topic What's next? 01:10:23 announcements, schedule adjustments? 01:10:43 a warm fuzzy alcohol-induced coma? 01:10:46 * adamw is all for option C 01:10:55 that was a given 01:11:26 * poelcat still planning to do Release Readiness meeting on Thursday--hopefully everything else is on track 01:11:51 #info QA to continue to test bug#590640 and work with dlehman on problem determination 01:12:06 #info Poelcat planning to do Release Readiness meeting on Thursday 01:12:14 #action stickster changing [[Schedule]] page on wiki 01:12:30 * poelcat will update taskjuggler 01:12:34 * jlaska recommends liberal use of #info and #action for all 01:13:18 anything else need ownership? 01:13:22 #action stickster notifying Red Hat components for PR and redhat.com web promo 01:13:33 who is sending out the fedora announcement? 01:13:53 I think Oxf13 generally does 01:14:07 I suppose I can, just not tonight. 01:14:16 Oxf13: I'll do it for you if you like 01:14:18 #action jlaska (Mr. Minutes) will send meeting minutes to test@ and devel@ 01:14:25 I'll refer to your previous text for a pony 01:14:28 stickster: go for it 01:14:37 #action stickster send slip notice in the style of Oxf13 01:15:20 okay gang ... if nothing else, I'll #endmeeting in 2 minutes 01:16:42 1 minute until #endmeeting 01:16:44 I'll note that rel-eng+QA plan to be very conservative of accepting any fixes other than those identified for blockers or critical bugs 01:16:57 s/of/when/ <-- ugh, grammar fail. 01:17:13 yes please! We're cutting into some quality Retrospective time :) 01:17:53 alright folks, thanks for your time and brains 01:18:07 #endmeeting