21:03:15 #startmeeting F15 Beta Go No Go Meeting Round 2.5 21:03:15 Meeting started Thu Sep 29 21:03:15 2011 UTC. The chair is tflink. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:03:15 Useful Commands: #action #agreed #halp #info #idea #link #topic. 21:03:36 #meetingname F16 Beta Go No Go Meeting Round 2.5 21:03:36 The meeting name has been set to 'f16_beta_go_no_go_meeting_round_2.5' 21:03:42 #topic Who's Here? 21:03:53 alrighty, shall we get this started? 21:03:59 * Jsmith is here 21:04:11 * tflink is flying by the seat of his pants, let me know if I'm not doing something right 21:04:27 Walking back from dinner 21:04:32 * satellit_ listening 21:04:38 * nirik waves. 21:04:43 walking and on IRC? That takes talent 21:04:47 * adamw is here 21:06:21 * pjones is here as well 21:08:07 sorry, trying to get a list of the current blockers going 21:08:36 just the tree from f16beta should be close enough. 21:08:45 how do you find that? 21:09:08 https://bugzilla.redhat.com/showdependencytree.cgi?id=713564&hide_resolved=1 21:09:12 "show dependency tree" 21:09:31 doesn't take accepted / proposed status into account, but it'll do 21:09:33 ah, that helps 21:09:43 i've thrown all the bugs we might want to talk about in the meeting onto that list (afaik) 21:09:50 #topic why are we here 21:09:56 if I miss one, let me know 21:10:22 #info We're here to see whether or not the beta release criteria have been met 21:10:30 #link http://fedoraproject.org/wiki/Fedora_16_Beta_Release_Criteria 21:10:44 #topic current release blockers 21:10:53 #link https://bugzilla.redhat.com/showdependencytree.cgi?id=713564&hide_resolved=1 21:11:00 * Jsmith switches over to his laptop 21:11:02 #info that list doesn't take accepted/proposed into account 21:11:41 * tflink is going to go over all the bugs currently not VERIFIED 21:11:57 #topic (725185) grubby doesn't add the initrd line at the kernel update 21:11:59 go for it 21:12:04 #link https://bugzilla.redhat.com/show_bug.cgi?id=725185 21:12:15 so, this is one that's been lying around for a while and got rejected as we thought it didn't hit anything important 21:12:18 so we realize it semi-does 21:12:34 the bug is that if you have both grub and grub2 packages installed (and, i think, grub as the active bootloader), kernel updates don't work right 21:12:44 the grub entry doesn't get an initrd line, hence the kernel likely doesn't boot 21:13:02 does the new build fix that? 21:13:06 so, usually you should never have grub and grub2 installed together - they even conflict now - *but*, in fact, you can 21:13:15 no, the 'fix' in grub was to make it conflict with grub2 21:13:16 Does having both grub and grub2 installed, but grub as the active bootloader hit the beta criteria? 21:13:28 yes. let me tell my story. =) 21:13:33 you can get into that situation by doing an EFI install 21:13:37 * nirik waits for how it's possible 21:13:44 EFI uses grub, so that has to be installed. but grub2 is 'default' in comps. 21:13:50 nirik: it's because I screwed up comps. 21:14:03 so because anaconda has superpowers, when you do an efi install, you get a warning that grub and grub2 conflict, you proceed, and you wind up with both installed 21:14:18 ah. ick 21:14:24 so, you do in fact wind up hitting this bug in a supported scenario. 21:14:26 HOWEVER! all is not lost 21:14:36 it happens on *first update*, not install. the kernel you install with works oaky. 21:14:38 there's a patch on anaconda-devel-list to fix anaconda and another to fix comps, but those seem like a stretch since there are workarounds. 21:14:58 and we have a patch to grubby which actually fixes bootloader installation in the grub+grub2 case. 21:15:02 bcl has tested this. 21:15:13 so, we can ship a grubby update, and then have future kernel updates require at least that version of grubby 21:15:18 which *should* ensure you can't hit this bug. 21:15:46 but an install with the newest grubby avoids this? 21:15:48 hurray. 21:15:50 (i guess it should be a Requires:post in kernel, not just requires) 21:15:52 tflink: yes 21:15:53 or does it require other workarounds? 21:15:58 well, it avoids it being a problem 21:16:05 you still have both things installed, which is wrong 21:16:06 tflink: as long as you have that grubby installed when you do the kernel update, it works. at least, in a quick test. 21:16:14 pjones: let's not get into that can of worms =) 21:16:31 adamw: yeah, I think we can fix that after beta. 21:17:14 so, i think that ought to be a reasonable solution to this. 21:17:15 As long as you're reasonably certain this will work, I'm reasonably certain we can call this a non-blocker :-) 21:17:35 the dumb option would simply be to document that you should do 'yum remove grub2' right after you install via efi. 21:17:39 which is the 'manual workaround'. 21:17:39 but we'd need another compose for this, right? 21:17:42 no. 21:17:48 like i said, it only happens on first update. 21:17:52 oh, it's only on update 21:17:59 so as long as we ensure that the first kernel update happens with the fixed grubby, it should be okay. 21:18:07 (he said with all digits crossed) 21:18:34 proposed #agreed - 725185 - RejectedBlocker - This is a problem, but as long as the next kernel update requires this version of grubby, things should be OK 21:19:01 i.e., can be fixed with an update. yeah. 21:19:04 ack 21:19:20 ack 21:19:24 proposed #agreed - 725185 - RejectedBlocker - This is a problem, but as long as the next kernel update requires this version of grubby, things should be OK and this potential issue can be fixed with an update post-beta. 21:19:34 any other ack/nak/patch? 21:20:12 ACK 21:20:20 #agreed - 725185 - RejectedBlocker - This is a problem, but as long as the next kernel update requires this version of grubby, things should be OK and this potential issue can be fixed with an update post-beta. 21:20:39 #topic (731529) grub making uefi calls without aligning stack pointer 21:20:45 #link https://bugzilla.redhat.com/show_bug.cgi?id=731529 21:21:00 this is just ON_QA because no-one but pjones has the hardware to test it, afaik. 21:21:17 pjones: do you actually have the hardware, or were you just diagnosing remotely when you fixed it? 21:21:17 adamw: AFAIK you've been running this for a day now. 21:21:35 there's a very real chance your EFI machine would have showed the symptoms 21:21:39 i've been running the 'fixed' grub, yup. i dunno if my system was actually susceptible to the bug, though, since i'd never installed EFI before. 21:21:51 I do also have machines that fail 21:21:52 i think it's reasonable to just take this one on trust, though, as all our EFI tests are pretty much working. 21:21:53 #info AcceptedBlocker, ON_QA 21:22:48 proposed #agreed - 731529 - Move to VERIFIED - All of the EFI tests are pretty much working, other issues would have likely surfaced by now 21:22:48 this bug came across from RHEL testing, fwiw. pjones just thought it would be a good idea to get the fix in fedora, since it was clearly a big bug. i think we can just say we wanted the fix, we took it, it hasn't made anything worse, let's call it verified and move on... 21:22:53 ack 21:22:56 ACK 21:23:12 Ack 21:23:25 #agreed - 731529 - Move to VERIFIED - All of the EFI tests are pretty much working, other issues would have likely surfaced by now 21:23:49 #topic (737731) Bootloader is left in F15 configuration when preupgrading to F16 21:23:54 #link https://bugzilla.redhat.com/show_bug.cgi?id=737731 21:24:02 #info AcceptedBlocker, ASSIGNED 21:24:15 this is the bug that needs to be fixed in F15 PreUpgrade 21:24:18 can be fixed before release. 21:24:19 yeah. 21:24:22 is it being worked on? 21:24:26 it does not affect spinning of beta 21:24:31 well.. 21:24:39 i applied some shoe leather to hughsie about it i think three days back 21:24:47 adamw: And... 21:24:48 at which point he said he'd work on it the next afternoon, i.e., two days ago 21:24:53 * jdulaney is here 21:24:56 since then, radio silence, but then, it hasn't been my priority 21:24:58 Apologies for lateness 21:25:04 ah, well, perhaps revisit in the coming days 21:25:06 jdulaney: welcome 21:25:10 but yeah, if people can apply force to hughsie that will probably be a good idea. 21:25:30 * jdulaney just got out of Jazz Band 21:26:11 proposed #agreed - 737731 - This needs to be fixed in F15 PreUpgrade and doesn't affect Beta compose. It doesn't appear to be in progress, need to revisit in the next couple of days. 21:26:15 ack 21:26:16 ack/nak/patch? 21:26:19 +1 21:26:28 ACK 21:26:36 aCK 21:26:52 #agreed - 737731 - This needs to be fixed in F15 PreUpgrade and doesn't affect Beta compose. It doesn't appear to be in progress, need to revisit in the next couple of days. 21:27:13 #topic (739253) unable to shut down from gdm greeter 21:27:19 #link https://bugzilla.redhat.com/show_bug.cgi?id=739253 21:27:25 #info AcceptedBlocker, MODIFIED 21:27:34 this one just hasn't got re-tested yet. it ought to be 'fixed', though. 21:27:45 I've looked for this, forgot to move to VERIFIED 21:27:52 tflink: ah, you checked it? cool 21:27:57 shutdown option is not present for shell, works for panel 21:28:04 which is what we expected 21:28:06 that's as we agreed was okay, yup. 21:28:21 thanks for testing 21:28:38 (and people will get the power button back for shell on first update, i believe) 21:28:53 proposed #agreed - 739253 - Move to VERIFIED - this has been fixed to the point that we expected - no power option in shell that fails. The power option will be restored in gnome-shell updates 21:28:56 ack 21:29:00 acK 21:29:23 ACK 21:29:26 #agreed - 739253 - Move to VERIFIED - this has been fixed to the point that we expected - no power option in shell that fails. The power option will be restored in gnome-shell updates 21:29:45 * adamw is envious of jsmith's huge ack cannon 21:30:04 #topic (742207) No usable bootloader option during a text mode f15->f16 upgrade 21:30:12 #link https://bugzilla.redhat.com/show_bug.cgi?id=742207 21:30:22 #info Proposed Blocker, NEW 21:30:23 adamw: I bought it from a former Soviet-bloc country for a small fortune, but it comes in handy 21:30:52 note that text-mode install is not explicitly part of the beta criteria 21:31:05 i think text *install* is, isn't it? 21:31:09 but text upgrade isn't 21:31:31 alpha install is 21:31:40 gah, text install is an alpha criteria 21:31:53 The installer must be able to successfully complete an upgrade installation from a clean, fully updated default installation (from any official install medium) of the previous stable Fedora release, either via preupgrade or by booting to the installer manually. The upgraded system must meet all release criteria 21:31:53 yeah, and criteria inherit, so text install is required to work. but the upgrade criterion doesn't really specify an interface. 21:32:00 but you're right - this is upgrade 21:32:16 so if we take a strict interpretation of the criterion - "The installer must be able to successfully complete an upgrade installation from a clean, fully updated default installation (from any official install medium) of the previous stable Fedora release, either via preupgrade or by booting to the installer manually. The upgraded system must meet all release criteria" - then hey, it passes. 21:32:21 'booting the installer manually'... hard to say if text is in there. ;) 21:32:24 And we can document an easy workaround, right? 21:32:29 the installer *can* complete an upgrade installation. you just have to be in graphical mode to do it. =) 21:32:39 jsmith: well, the easy workaround is 'don't use text mode' 21:32:46 * nirik nods. 21:32:48 there's probably a console workaround for text mode 21:32:53 adamw: Or manually install the bootloader after install but before reboot? 21:32:55 (run commands foo and bar before rebooting) 21:32:56 yeah 21:32:57 I'd be ok with release noting this, and fixing it before final. 21:33:04 nirik: +1 21:33:22 yeah, this just doesn't seem like a hugely important path, and it's not like it's an irretriveable situation. you just need to get a bootloader on there somehow. 21:34:03 or even just use 'skip bootloader' and fix the bootloader config, which is even less of an issue. 21:34:20 proposed #agreed - 742207 - RejectedBlocker, CommonBugs - This doesn't directly hit any beta release criterion, isn't the most common upgrade path and can be worked around. The workaround needs to be documented 21:34:24 ack/nak/patch 21:34:25 ack 21:34:29 aCk 21:34:34 Who is going to document it? 21:34:42 (ACK) 21:34:46 * adamw votes for someone who's had some sleep 21:34:51 I'll action adam or I 21:35:10 #action adamw or tflink - Document workaround for 742207 21:35:14 tflink: Sounds like it's you :-) 21:35:22 #agreed - 742207 - RejectedBlocker, CommonBugs - This doesn't directly hit any beta release criterion, isn't the most common upgrade path and can be worked around. The workaround needs to be documented 21:35:23 you'll action yourself, or you'll be fired. ;) 21:35:39 adamw: you fired me already, that only works once 21:35:49 GET BACK HERE SO I CAN FIRE YOU AGAIN 21:36:14 * tflink is glad to be in a different country from adamw some times 21:36:16 :-D 21:36:26 no-one's out of reach of the orbital laser 21:36:27 blame canada 21:36:37 #topic (742226) /sbin/grub2-probe: error: cannot find a GRUB drive for /dev/mapper/nvidia_cjfffajep2 21:36:40 adamw: except those of us that are in underground bunkers, of course. 21:36:44 well played, sir. 21:36:44 #link https://bugzilla.redhat.com/show_bug.cgi?id=742226 21:36:51 #info Proposed Blocker, NEW 21:36:52 aaaaand this is where a light coating of fudge may be required. 21:37:05 seems like other installs on other bios raid controllers might be working? 21:37:09 yeah 21:37:17 if so, this is hardware specific, and not really necessarily a blocker. 21:37:31 so, we have a beta criterion "The installer must be able to create and install to software, hardware or BIOS RAID-0, RAID-1 or RAID-5 partitions for anything except /boot " 21:37:48 the BIOS RAID bit of this gets pretty lightly tested 21:37:55 which, if some bios raid works, is fulfilled as written. 21:38:03 yeah, if we want to be strict about it. 21:38:06 clearly we can't say it's "all hardware" 21:38:16 yes, it's a judgment call area 21:38:24 gather all the data we have and decide how much fail is too much fail 21:38:30 * nirik nods. 21:38:36 so, jlaska tested on some NVIDIA board, with RAID-0, and it failed. 21:38:37 so far looks like just nvidia? 21:38:38 it's safe to assume we've never shipped a release in which all bios raid worked on every bios. 21:38:59 i have tested with my motherboard, which has two SATA controllers that are RAID capable, and got these results... 21:39:26 RAID-1 on the Intel controller succeeds. RAID-0 on the Intel controller fails at bootloader install with an error that looks quite similar to jlaska's, plus it seems weird all over the shop 21:39:42 anaconda is convinced it's a RAID-1 array, and it's showing up as /dev/dm-X not /dev/mdXXX as it should 21:39:48 so there may be some other kind of weirdness going on there 21:40:22 the intel ones at least for a time used mdraid... oddly. 21:40:27 but I thought that was backed out. 21:40:33 RAID-0 on the Marvell controller succeded, RAID-1 failed to install once (odd error where it seemed like the array essentially crapped out when the bootloader tried to install), installed successfully but booted to a flashing cursor once. 21:40:39 nirik: no, they still are supposed to use mdraid. 21:40:46 and the successful RAID-1 on Intel test did. 21:40:51 really? I thought that was reversed. oh well. 21:41:10 plus my marvell controller may actually be an actual hardware RAID controller, strangely enough. 21:41:27 so...we have something of a paucity of data here. but I did do at least two installs to two motherboard RAID controllers which worked. 21:41:36 anyhow, I am ok with punting on this... "some bios raid controllers are not functional, will try and fix for final" 21:41:45 nirik: +1 to that 21:42:04 there's also the background here that it's not as if we're regressing code that worked before, we're dealing with a grub2 migration here, and it seems like grub2's BIOS RAID support in general is still a work in progress. 21:42:10 yeah, it sounds like that or slip another week 21:42:26 and even if we slipped a week we might find all kinds of dragons here and find we couldn't really 'fix' it anyway. 21:42:40 adamw: right, that might cause us to look at changing criteria.. if our tools changed and don't support the same things. 21:42:53 as i mentioned in the bug, it's not like this is some kind of one-liner in anaconda's code, this is 'large chunks of stuff that grub1 did, grub2 doesn't really seem to' 21:43:10 nirik: yes, it might be good to have some kind of formal process by which we can relax criteria in this sort of case 21:43:26 proposed #agreed - 742226 - RejectedBlocker CommonBugs- BIOS Raid does work for some controllers but not all. Re-propose as final blocker and document for beta. 21:43:29 i'm usually the criteria nazi, but this doesn't feel like excessive fudge, just a realistic adjustment to the state we're in 21:43:31 ack/nak/patch 21:43:56 ACK (with a slightly not-so-fresh feeling) 21:43:58 AcK 21:44:10 yeah, the criteria are meant for a fairly stable situation; switching bootloaders isn't that stable. 21:44:12 ack on that. it'd be nice if this worked, but i really don't see the cost/benefit in slipping to maybe possibly fix this (and break other stuff, most likely). 21:44:21 #agreed - 742226 - RejectedBlocker CommonBugs- BIOS Raid does work for some controllers but not all. Re-propose as final blocker and document for beta. 21:44:32 ok, that's all the non-verified blockers I see 21:44:41 yeah, when we wrote up the anaconda criteria, it was from the PoV of 'well, this mature RAID code we've been using forever shouldn't break, and if it does, we did something silly and should fix it'. 21:44:43 are there any others that I missed? 21:45:04 that's all on my list 21:45:17 there is one other thing to note, which is that we didn't quite make 100% test coverage 21:45:21 any other issues to bring up before we start taking votes? 21:45:54 #topic test coverage 21:46:00 #chair adamw 21:46:01 Current chairs: adamw tflink 21:46:06 #topic Test Coverage 21:46:07 #topic test coverage 21:46:21 awesome 21:46:23 #undo 21:46:23 Removing item from minutes: 21:46:26 we have 100% desktop and base test coverage, if we accept RC2 and RC3 results (which we can do safely given the deltas) 21:46:48 the only fail there is tflink didn't see update notifications in fallback mode, but i'm pretty sure that is actually working and he just didn't wait long enough for the PK gods 21:46:59 someone else filed a pass for RC2, iirc 21:47:08 install coverage is about 99% 21:47:13 two tests are missing: 21:47:21 QA:Testcase_Kickstart_File_Path_Ks_Cfg 21:47:27 https://fedoraproject.org/wiki/QA:Testcase_Kickstart_File_Path_Ks_Cfg 21:47:36 https://fedoraproject.org/wiki/QA:Testcase_Install_to_Hardware_RAID 21:48:01 the first one is just a really, really silly method of kickstart delivery which involves crowbarring the kickstart into the initrd of the installer or something. 21:48:11 we support some really exotic ways of delivering kickstarts. 21:48:38 all the other kickstart methods work, it'd be very odd that this one didn't. and frankly, if it didn't, i'd be like 'the workaround is to use a less weird kickstart delivery method'. 21:49:04 * nirik nods. 21:49:14 WORKSFORME 21:49:20 is it not tested because no one got to it? or that the initramfs stuff changed? 21:49:28 no-one got to it 21:49:34 i was going to do it today, but got sidetracked by bios raid 21:49:41 i'm just checking back to see when we last tested it 21:50:16 personally, I'd vote "go" right now just so I can go get some dinner. 21:50:34 (and because I'm tired of fixing things, of course) 21:50:54 heh. 21:51:00 well, it sounds like we have a pretty reasonable beta compose... and we aren't blocking, so, it seems reasonable to me to go. 21:51:03 i do want to be really strict on test coverage, but this one just feels silly. 21:51:18 the hardware RAID test is a hardware availability issue 21:51:45 but actually, i could claim a pass for that if we decide my marvell controller is a hardware RAID controller, not a fakeraid controller. it seems to present the array as /dev/sdX , not /dev/dm-X or /dev/mdXXX . 21:52:01 and google results kinda suggest it is one. but i'm not entirely sure. 21:52:24 it's extremely unlikely hardware raid would fail, anyway, as to fedora a hardware raid array just looks like, well, a disk. 21:52:24 there's really no other way for it to present as /dev/sdX 21:52:40 that's not entirely true - some controllers show up as other things (think cciss) 21:52:49 oh, right. 21:52:59 I will point out that two of my friends were bitten by RAID issues on F15, for what (little?) it's worth 21:52:59 still, i'll call my marvell test a hardware RAID pass and we can all get some sleep. 21:53:02 but at the same time - the criteria doesn't actually say every controller has to work. 21:53:13 jsmith: you have friends? 21:53:13 jsmith: software raid, bios raid, or hardware raid? 21:53:21 pjones: right. 21:53:24 I didn't think we were allowed to have friends. 21:53:43 pjones: it's one of those criteria where the intent is 'it has to be not completely broken', not 'it has to work all the time for all hardware in every possible configuration'. 21:53:46 pjones: Yeah, although they're not nearly as friendly now that we killed their data :-/ 21:54:04 blame them for not contributing or testing enough ;) 21:54:09 proposed #agreed - missing test results deeped acceptable for beta, will do better for final 21:54:18 deeped. 21:54:22 jsmith: anyway, you can tell them that things will be very different for f16. just be careful not to make 'differenet' imply 'better'. =) 21:54:23 proposed #agreed - missing test results deemed acceptable for beta, will do better for final 21:54:32 let's make that suck less 21:54:43 adamw: go for it 21:55:01 * jsmith fires the ACK cannon again 21:55:02 proposed #agreed - only one exotic kickstart delivery method is missing from test coverage, and arguably the stranger kickstart delivery methods should not be beta critical 21:55:11 * pjones is +1 21:55:15 * jsmith is +1 21:55:15 ack 21:55:15 acky 21:55:30 if i was to propose that we make only http and nfs kickstart delivery methods beta critical, and move the others out to final, would everyone be good with that? 21:55:54 * nirik would be, but not sure we should be adjusting that now/here? 21:56:02 no, but let's call it a pending change 21:56:10 like we do for criteria adjustments we come up with in blocker review meetings 21:56:21 whatevs. 21:56:22 ok. 21:56:33 pjones: just to make me feel better. =) thanks 21:57:06 * jsmith has no problems with that 21:57:21 #agreed - only one exotic kickstart delivery method is missing from test coverage, and we provisionally suggest the stranger kickstart delivery methods should not be beta critical; therefore deeming test coverage effectively complete, as we would not block even if that test failed 21:57:40 * adamw does contortions 21:57:48 alrighty...i think that's everything qa-y 21:58:21 ok, any other issues to bring up before voting? 21:58:51 * adamw has nothin' 21:59:12 #topic Is Fedora 16 Beta Ready to Go? 21:59:13 oh, just a reassurance; i checked the rc3 vs. rc4 package lists and the delta is as intended 21:59:30 i.e. we didn't screw up rc4 compose at all, so far as i can tell, so we shouldn't have any old bugs creeping up to bite us in the ass. 21:59:32 lets start with QA - go or no-go? 21:59:34 Me expresses a huge "thank you" for everyone who went above and beyond the call of duty to help get RC4 delivered and tested 21:59:38 go! 21:59:48 #info QA is go for F16 Beta 21:59:59 Development? 22:00:02 * nirik cheers for him not screwing up the rc4 compose. 22:00:02 ship it 22:00:02 (as per our policy on voting in this meeting, test coverage is effectively complete, no open unaddressed blockers remain: qa vote is automatic) 22:00:10 #info RelEng is go 22:00:24 I'll go ahead and say that Robyn is a "go!" 22:00:37 * jsmith is go! 22:00:41 #info Devel is go 22:00:42 * nirik is good to go. 22:01:06 proposed #agreed - Fedora 16 Beta RC4 is accepted as beta 22:01:13 * tflink thinks that is worded right 22:01:21 sure. 22:01:25 woot! 22:01:26 * tflink thinks that is worded right 22:01:29 whoops 22:01:35 #agreed - Fedora 16 Beta RC4 is accepted as beta 22:01:43 dgilmore: will you be able to handle the syncing out/etc? or you need me to do anything more on that... 22:01:58 holy mother of cookie, at last. 22:02:03 no kidding 22:02:20 bah.. you guys have it easy. in my day we would get up to double digits 22:02:20 nirik: yeah ill do it when i wake up 22:02:21 I don't think that there is anything else that needs to be covered here 22:02:23 big vote of thanks to pjones and bcl for the last minute efi fixes 22:02:34 and to everyone who got the validation done so quickly 22:02:36 I'd like to thank all the QA folks who went over and beyond the call of duty on (re)testing things. 22:02:49 big thanks to adamw for not sleeping this wek 22:02:52 week 22:02:55 dgilmore: cool. 22:03:19 if I'm not missing anything ... 22:03:22 okay, and that being said I'm going to PTO the hell out of tomorrow 22:03:25 #topic Open Discussion 22:03:29 so if you find anything wrong, feel free to not call me. 22:03:45 pjones: Enjoy! 22:03:47 pjones: sounds like a plan and a half 22:03:52 * tflink sets fuse for 2 minutes 22:04:55 #endmeeting