16:03:05 #startmeeting Fedora 13 Alpha blocker bug review #3 16:03:06 Meeting started Fri Feb 19 16:03:05 2010 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:03:07 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:03:31 I'm here, on another call too 16:03:37 * maxamillion is here ... sorry I'm late 16:04:35 alright, let's get going, with the dulcet tones of Tiger in the background 16:04:55 :P 16:05:24 #topic https://bugzilla.redhat.com/show_bug.cgi?id=557386 16:05:41 alright, this is abrt/kernel again 16:06:12 chatter i've noticed this week seems to indicate it's fixed 16:06:24 it's in MODIFIED now, so that's a good sign 16:06:57 from the looks of comments 8/9 we should re-assign to abrt 16:07:19 is that captured in another bug already on the list ... 16:08:15 566460 16:08:16 i don't believe so 16:08:45 that looks like something new/different 16:09:03 yeah 16:09:21 so i think perhaps the first bug should be assigned to abrt, but depending on this bug? 16:09:21 * jlaska looks in https://admin.fedoraproject.org/pkgdb/packages/bugs/abrt?_csrf_token=9f092d288101fe29f222eb7df98274ab5ee97a6f 16:09:56 nothing jumps out as being already filed to track that change 16:10:29 okay ... so there is some more work needed 16:10:46 do we have a sense if that should be on the F13Alpha list? 16:11:10 I don't see Jiri around, perhaps we can pull in nhorman 16:11:45 go for it 16:12:24 jlaska: *random* side note on that, I got "scolded" recently for posting a link that included the csrf token .... just a though for future reference :) 16:13:02 maxamillion: oh boy, thanks for the heads up 16:13:02 jlaska: i think the second bug is essentially the same in character as the first - a breakage in the kernel which stops abrt from being able to work 16:13:20 jlaska: anytime 16:13:33 my understanding of the second is that it impacts only c/c++ apps 16:13:46 which ... is most of the desktop 16:13:54 is abrt not being functional considered an alpha blocker? (just curious) 16:14:00 okay, nhorman said he'd join shortly 16:14:17 maxamillion: we decided it was for 557386 16:14:44 maxamillion: the main issue is that if abrt is broken due to the kernel, people testing the live images can't possibly file abrt reports 16:14:57 adamw: ah, makes sense 16:15:01 adamw: regarding the 2nd bug, do you see it as a bug in the kernel? 16:15:09 or in abrt? 16:15:26 jeff_hann: it's clearly a kernel bug 16:15:36 look at the reproduction steps, it doesn't involve abrt at all 16:16:05 hmmm 16:17:34 wait a moment 16:18:19 #agreed 557386 is back with abrt team but probably depends on 566460 16:18:22 yes, adamw, you were right 16:18:30 #topic https://bugzilla.redhat.com/show_bug.cgi?id=566460 16:18:35 my mind was wondering 16:18:36 (since we're talkin about it anyway) 16:19:07 so this is the second bug, right? 16:19:28 I added this to the list for discussion since it seems to match the criteria used for the previous bug 16:19:30 yea 16:19:39 yeah 16:19:49 but definitely wanted additional input 16:19:58 shall we quickly discuss another one while we wait for nhorman? 16:20:08 adamw: sure, no objections 16:20:31 #agreed suspending discussion on 566460 while we wait for nhorman 16:20:33 #topic https://bugzilla.redhat.com/show_bug.cgi?id=560477 16:21:21 I propsed this since it touches point#10 of the Alpha criteria 16:21:32 was this tested with a more recent anaconda? 16:21:35 it's in MODIFIED, and I've confirmed the fix with an updates.img, just waiting for an official build 16:21:39 sure, we discussed it last week 16:21:41 it appears to have a patch mentioned in the bug 16:21:53 04.02, right? 16:22:13 jlaska: could you add a comment to say you've tested with updates.img btw? your last comment doesn't say that 16:22:14 if it's still not fixed, we'll see 16:22:35 adamw: I can do that ... just fyi ... I'm not posting feedback in any of my MODIFIED bugs until the build is available for test 16:22:42 but I'll note that now in the bz 16:22:55 ah k 16:23:11 it would be useful to know that the updates.img had worked, though, just so we know where we're up to 16:23:28 good point 16:23:31 okay, posted 16:23:34 ok 16:23:45 so basically we're happy with the situation of this one as we're just waiting for the next build point, right? 16:23:51 yeah 16:24:22 #agreed 560477 is probably fixed, waiting for next anaconda build to confirm fix 16:24:33 #topic https://bugzilla.redhat.com/show_bug.cgi?id=562209 16:25:02 puff of smoke ... and nhorman appears 16:25:04 jlaska, sorry, was in a mtg w/ lwang 16:25:11 * nhorman choughs 16:25:13 :) 16:25:14 nhorman: no worries, thanks for joining 16:25:15 aha, okay, re-topicing 16:25:25 jlaska, what are we trying to solve here? 16:25:27 #agreed resuming 566460 discussion 16:25:33 #topic https://bugzilla.redhat.com/show_bug.cgi?id=566460 16:25:56 nhorman: we are discussing F13 Alpha blocker potential for /topic bug 16:25:58 ah, this jem 16:26:13 nhorman: we just want an update basically 16:26:17 downloading the iso to test a bit in KVM 16:26:24 as we understand it this is another kernel problem which makes it impossible for abrt to work, right? 16:26:28 (can't do much without a core dump) 16:26:51 adamw, yeah.....sort of.... 16:27:02 do elaborate :D 16:27:03 I can give the medium sized story w/ backround if you like 16:27:05 :) 16:27:06 sure 16:27:47 so....setting asside the snit that jmoscovk and I got into over it, basically 6 months ago I changed how kernel coredump to pipe worked to fix a race/bug/oops 16:28:05 um, remember this is 566460 not 557386 16:28:15 adamw, I know, they're related 16:28:21 ah k 16:28:27 * adamw shuts up again 16:28:34 to continue, that change hit rawhide and abrt broke 16:29:15 we have two solutions, the one which I recommended, and jmoscovk implemented in user space, and the one which I promised to try and push through upstream in the kernel 16:29:38 I was figuring no one would want the kernel solution, since its a bit wierd....and that the user space solution would stick 16:29:41 I was wrong :) 16:30:01 the kernel solution was successful, and I pulled it back to rawhide....and there was moderate rejoicing 16:30:12 hehe 16:30:41 unfortunately, to do the backport I had to suck in some other work from andi kleen in the same area that seems to have busticated core_pipe_limit somehow 16:30:47 hence bz 566460 16:31:19 ah, okay. 16:31:23 I'm confident I can fix it given enough time (I just don't know how much time I have) 16:31:27 so it's a kernel bug but you broke it =) 16:31:27 when is f13 deadline? 16:31:38 https://fedoraproject.org/wiki/Releases/13 16:32:16 alpha is 'frozen' now but blocker fixes can go in up until...probably the go/no-go 16:32:21 which is on 2010-02-24 16:32:45 if this doesn't get fixed, abrt is effectively broken, right? 16:33:04 13 is essentially frozen until we hit GA 16:33:29 adamw, well, I think thats a question for the abrt guys. you can still get cores, you just can't access the /proc/pid of a crashing process when you get it 16:33:39 don't know how much they depend on that 16:33:42 no capture of c/c++ apps 16:34:04 euf, so a question for jmoskovc after all...do we have any other abrt contacts? 16:34:27 how can they be that reliant on gathering information out of /proc/pid? Its almost all completely available in the core itself 16:34:31 *sigh* 16:34:39 i don't know if they are, that's what we'll ask them 16:34:54 ok, so reading this schedule 3/2 is the alpha freeze? 16:35:17 no, that's when the alpha is released. 16:35:40 ah, alpha freeze was....2 days ago 16:35:41 great 16:35:41 alpha is already frozen, as I said, since 16th. but fixes for critical issues get in. 16:35:58 ok, so exactly how long do I have to fix this jem? 16:36:09 * Oxf13 notes that the wiki pages and schedules and freeze pages are slightly outdated due to no frozen rawhide 16:36:26 nhorman: you do a build in F-13, you push it into bodhi for updates-testing, and if it solves the issue we can push it stable 16:36:50 adamw: Can't find Jiri, but asking kklic 16:36:55 Oxf13: jlaska: would you agree with 'up until the go/no-go'? 16:36:55 so basically, just fix it asap? 16:37:01 jlaska, he's in bed by now 16:37:18 nhorman: in bed by 6pm, early sleeper :) 16:37:36 adamw: that doesn't resolve the live image scenario? 16:37:38 jlaska, well, home at least, brno office is getting a 3rd floor added to it 16:37:44 nhorman: right 16:37:47 no one can work through the noise after they start construction :) 16:37:54 jlaska: i mean *alpha* go/no-go 16:38:28 adamw: what is the question again? 16:38:28 adamw: right ... so that's too late for testing 16:39:09 Oxf13: how long does nhorman have to fix this to get it in alpha 16:39:33 it should be tested in updates-testing and then tagges table by go-nogo 16:39:38 ok 16:39:50 so when is the go/nogo descision made? 16:39:52 of course, if we consider it a blocker, we ought to delay the alpha if it's not fixed. that depends on what abrt tell us 16:40:02 nhorman: 24th. 16:40:02 kklic: thanks for joining! 16:40:45 adamw, ok, that works. I'll start hammering on it this afternoon. with any luck, I can have it fixed by wed. of next week or so 16:41:01 I'd like to get the fix upstream first, but it sounds like we're going to have to do this in parallel to make the schedule 16:41:02 nhorman: 24th *is* wednesday 16:41:14 nhorman: we'd need to have it available for testing on 23rd probably to test it 16:41:16 it'd need to get into updates-testing a day or 3 before that 16:41:25 but, we need to ask kklic... 16:41:32 kklic: does https://bugzilla.redhat.com/show_bug.cgi?id=566460 stop abrt from working? 16:41:36 or can abrt work around it? 16:43:15 adamw: I need a minute or two to check the source code 16:44:04 ok 16:44:06 we'll wait 16:45:04 adamw: I have to step out in 16 minutes ... but you can put me down to update all the installer MODIFIED bugs as you suggested with the previous installer issue 16:45:36 jlaska: ok, so basically all your installer bugs in modified are in the same state? probably fixed, waiting for next anaconda drop to confirm 16:46:11 yeah, I've been maintaining a mega-updates.img that has all the fixes planned for the next build of anaconda 16:46:44 there's only 1 unresolved issue that concerns me from our TC2 testing for the alpha 16:47:33 ^---> 562209 16:47:43 let's talk about that one then, while kklic is looking... 16:47:56 #agreed kklic is checking out 566460, going to discuss another bug for now 16:47:59 #topic 562209 16:48:02 That #566460 basically prevents ABRT from catching C/C++ cratches 16:48:12 gah! 16:48:16 doh!, sorry Adam :( 16:48:21 #topic https://bugzilla.redhat.com/show_bug.cgi?id=566460 16:48:23 :P 16:48:28 :) 16:48:46 kklic: nhorman says: 16:48:47 adamw, well, I think thats a question for the abrt guys. you can still get cores, you just can't access the /proc/pid of a crashing process when you get it 16:48:53 how can they be that reliant on gathering information out of /proc/pid? Its almost all completely available in the core itself 16:48:53 *sigh* 16:49:03 so abrt does need the /proc/pid ? 16:50:28 yes, we read data from /proc/pid, it cannot be changed easily afaik 16:50:52 okay...so we need the kernel side fix 16:51:04 given the tight timelines it'd be great if you could do anything to help nhorman with the kernel side 16:51:10 ok, so I need to get that fixed, I'll try focus on it this afternoon. With any luck, it won't be too convoluted to figure out 16:51:17 we read cwd, stat, cmdline, exe, limits(?)...not sure how much work would it be to find all that stuff in the coredump 16:51:22 nhorman: would it help to have more eyes on it? 16:51:30 nhorman: if we could ask for any other kernel devs to help out? 16:52:01 adamw, I really can't say right now to be honest 16:52:05 ok 16:52:18 well do let us know if you feel like you're running into a wall as we'll have to try and do something about it :) 16:52:19 I have a feeling its just going to be dumping a few printks into the kernel to figure it out 16:52:34 kklic, unless you are intimately familiar with how the coredump path works in the kernel? 16:52:38 * nhorman asks hopefully :) 16:53:23 #agreed 566460 should be a blocker, must be fixed kernel side, nhorman will try and fix it in time to be tested before go/no-go 16:53:25 nhorman: no experience with kernel, sorry 16:53:47 * nhorman makes a note to himself to _never_ touch the coredump path again 16:54:05 hehe 16:54:08 lol 16:54:18 okay, i think we've covered that one...nhorman please do keep us up to date ;) 16:54:36 #topic https://bugzilla.redhat.com/show_bug.cgi?id=562209 16:54:39 jlaska: still around? 16:54:41 adamw, ok 16:54:42 you bet 16:54:51 thanks a lot nhorman / kklic 16:54:59 np 16:55:09 so what's the story on 562209 jlaska 16:55:33 well .. it affects any ISO boot scenarios (boot.iso, CD and DVD) 16:55:33 adamw: the latest boot.iso has anaconda 13.25 16:55:50 the bug prevents access to any network package repositories 16:55:53 but the latest in repos is 13.27 16:55:55 which is the default case for boot.iso 16:56:27 so I think this fits well into the Alpha release criteria 16:56:38 yeah, it does, boot.iso is supposed to work 16:56:42 there is a workaround ... add the boot argument "updates=" 16:56:52 but that's messy and no one will use it for this common case 16:56:57 clumens updated the bug with several options this morning 16:57:21 I think the devs need to choose an option forward 16:57:32 I can ask clumens + skvidal to join if that would help move things forward 16:57:37 "This means no connection sharing, though." - don't quite understand that 16:57:41 yeah it would be good 16:57:56 adamw: that's just internals of how they are passing around the libcurl network object 16:58:04 vs having to reinstantiate the connection every time 16:58:23 so it's inelegant but not fundamentally going to break anything? 16:58:29 it reads like 1) is the best option, certainly for alpha 16:58:40 yeah, this issue might need a short-term, then a long-term fix 16:58:52 what's up? 16:59:06 hey skvidal, thanks for joining 16:59:18 looking over clumens feedback in bug#562209 16:59:40 clumens: welcome to the party 17:00:20 woo hoo 17:00:43 so ... just trying to get an understanding of how to move forward on this bug 17:00:53 * adamw passes around the weak lemon punch 17:01:33 clumens: skvidal: we're coming down to the wire on alpha so we need to decide and approach and test a fix fairly quick 17:02:25 clumens: I'll work on whatever you prefer, here. - though a 'drop and reinit' option to urlgrabber is probably easiest 17:02:44 skvidal: i am okay with that 17:02:55 clumens: and given curl's tendency toward 'odd' behavior it'll probably be needed elsewhere 17:03:18 yeah, joy 17:03:22 before I d 17:03:23 o 17:03:26 and just b/c I want to be sure 17:03:37 we can reliably repeat this, right? 17:03:50 i can do it in anaconda every time 17:03:56 skvidal: yeah 17:04:19 clumens: and when curl is not built with c-ares the problem still exists, right? 17:04:32 * jlaska has to run to conflict ... will catch up after 17:04:42 b/c it looks like in comment 24 on the bug 17:04:47 that w/o c-ares it works 17:05:04 so if we can just rebuild curl and have it all work - well, yay 17:05:08 I think that depends on what other thing it uses, and whether or not someting calls glibc res_init() after resolv.conf changes 17:05:26 there may very well be something triggering a res_ini(), but since curl doesn't currently use glibc, that does nothing for it 17:06:04 skvidal: looks like if we rebuilt curl to use the same linker as the rest of the universe, we'd still have to call res_init in anaconda. 17:06:15 ok 17:06:16 but that's not such a big deal. 17:06:49 it's something we should probably do anyway 17:07:52 ao again, whatever's being done, we'd want it available for testing by tuesday at latest 17:08:06 so is this the plan: 17:08:13 1. rebuild curl w/o c-ares 17:08:20 2. call re_init() in anaconda? 17:08:22 or 17:08:26 is it: 17:08:41 1. write a 'restart_curl_obj' method to urlgrabber 17:08:47 2. call that from anaconda 17:09:03 5. profit 17:09:45 either way we require modifying one pkg + anaconda 17:09:49 seems to me that the rebuild curl plan is likely to turn up additional weirdness. 17:10:16 okay, then I'll shelve the other things I was doing and work on a patch to urlgrabber 17:10:40 and i'll be standing by to test 17:10:45 as will jlaska 17:10:53 and i agree, i was hoping someone else would say that :) 17:11:01 rebuilding curl this close to alpha wouldn't make me warm and fuzzy 17:11:32 oh but adding code to urlgrabber 17:11:34 (though that really is the right fix. what the hell, use the same resolver as everyone else.) 17:11:35 that's okay 17:11:37 * skvidal boggles 17:12:21 so, there is probably planA for alpha, and planB for after alpha 17:12:30 +1 17:12:37 planB should be the 'make curl use the same damn resolver as everything else' 17:12:38 switching curl's resolver library is something you probably want a little more lead time on 17:13:13 #agreed 562209 for now will be handled by having urlgrabber be able to reinit connections, skvidal will work on this, clumens and jlaska will test 17:13:30 but the urlgrabber thing is a temporary band-aid until then, right? 17:13:38 #agreed post-alpha a more 'correct' fix may be put in place 17:13:43 big fat "REMOVE THIS AFTER CURL STOPS BEING CRAZY" comments and all? 17:14:45 thanks folks... 17:14:54 anyone have anything else on this bug, or next bug? 17:16:45 okay, next bug then 17:16:52 #topic https://bugzilla.redhat.com/show_bug.cgi?id=563212 17:17:53 this is the intel bug. we can probably close it now. uh, if -4 is definitely in the f13 repos after the big exciting switchover 17:19:12 * adamw goes to check. manually, since his yum config appears to be totally useless at present. 17:20:16 * Oxf13 is still working on getting the 13 directory published 17:20:37 yup, -4 is in the 13 tree. so I think we can close this, since there's multiple confirmations in the bug that -4 is okay 17:20:38 it's 4, alright, adamw 17:20:45 and we were pretty confident about what caused it and how to fix it anyway 17:20:55 any objections? 17:21:23 ok! 17:21:28 #agreed 563212 is fixed, closing 17:21:58 #topic https://bugzilla.redhat.com/show_bug.cgi?id=565306 17:22:21 damn, we lost clumens 17:22:33 given that jlaska was able to create an updates.img i assume this should be fixed in next anaconda drop 17:22:36 anyone know? 17:24:46 well, let's figure it is. :) 17:25:08 #agreed 565306 should be fixed in next anaconda drop, jlaska will test 17:25:10 #topic https://bugzilla.redhat.com/show_bug.cgi?id=565497 17:25:42 this is fixed by the updated pl-parser, I believe 17:25:49 obviously jlaska didn't get around to retesting 17:26:57 i have a totem-pl-parser installed with the correct gmime dependency. so this is fixed. closing. 17:26:58 what a lazy bum! 17:27:20 yeah, he sucks :) 17:27:28 #agreed 565306 is fixed by updated totem-pl-parser, closing 17:27:45 okay, so now we have a chunk of MODIFIED anaconda bugs: 17:28:05 565592, 565599, 565611, 565639, 565840, 565873 17:28:34 jlaska says all of these should be fixed in next anaconda drop 17:28:42 and he's just waiting on that to confirm and close them 17:28:51 does anyone want to go through them individually or should we just take that as OK? 17:30:03 *crickets* 17:30:11 :P 17:30:32 okay, we'll just leave 'em 17:30:38 #topic 565592, 565599, 565611, 565639, 565840, 565873 17:30:53 #agreed these are all anaconda bugs which have been patched but no new anaconda build has been made to test them yet 17:31:13 #agreed we will re-test and confirm the fixes after the next anaconda drop 17:31:19 #topic open floor 17:31:24 anyone have any other bugs to bring to the party? 17:32:05 adamw: was the bug regarding installing a minimal system pulls unwanted deps fixed? 17:32:23 jeff_hann: don't know off the top of my head 17:32:28 ...heh. 17:33:44 sorry, timed out 17:35:06 jeff_hann: i don't know that bug off the top of my head 17:35:09 do you have the number? 17:35:21 nope :( 17:35:51 but it's exactly what I'm doing now with boot.iso (rawhide) in KVM 17:35:59 uf. 17:36:12 i don't believe minimal install is in the alpha criteria... 17:36:24 I'm sure it isn't 17:36:30 just a quirk of mine 17:36:41 it is something it'd be nice to have working, though, yeah 17:37:00 well, working on it and will file a bug or smth. 17:37:04 if need be 17:38:42 ok, thanks 17:38:52 any other business? 17:39:29 okey dokey, thanks everyone 17:39:37 #endmeeting