17:04:54 #startmeeting F25 Alpha Go/No-Go meeting - #2 17:04:54 Meeting started Thu Aug 25 17:04:54 2016 UTC. The chair is jreznik. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:04:54 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:04:54 The meeting name has been set to 'f25_alpha_go/no-go_meeting_-_#2' 17:05:10 .hello kevin 17:05:11 nirik: kevin 'Kevin Fenzi' 17:05:13 .hello pfrields 17:05:13 .hello adamwill 17:05:14 stickster: pfrields 'Paul W. Frields' 17:05:17 adamw: adamwill 'Adam Williamson' 17:05:43 #meetingname F25-Alpha-Go_No_Go-meeting 17:05:43 The meeting name has been set to 'f25-alpha-go_no_go-meeting' 17:05:50 you're too fast for me :) 17:05:56 #topic Roll Call 17:05:59 ha 17:06:02 .hello jreznik 17:06:03 jreznik: jreznik 'Jaroslav Reznik' 17:06:20 .hello jbwillia 17:06:22 Southern_Gentlem: jbwillia 'Ben Williams' 17:06:25 .hello mohanboddu 17:06:26 mboddu: mohanboddu 'Mohan Boddu' 17:06:30 I'm sorry for the meeting room issue, seems like it was scheduled by mistake in -1, there are two overlapping meetings in fedocal 17:06:34 .hello kparal 17:06:35 kparal: kparal 'Kamil Páral' 17:06:51 it's all fine. ;) 17:06:57 let's wait a minute for more folks to find the right channel :) 17:07:14 * mattdm is lurking 17:07:22 as you can see, I'm covering for jkurik today, go/no-go after a while ;) 17:07:28 hola 17:07:54 #chair stickster dgilmore adamw kparal nirik 17:07:54 Current chairs: adamw dgilmore jreznik kparal nirik stickster 17:08:11 #topic Purpose of this meeting 17:08:12 #info Purpose of this meeting is to check whether or not F25 Alpha is ready for shipment, according to the release criteria. 17:08:14 #info This is determined in a few ways: 17:08:15 #info No remaining blocker bugs 17:08:17 #info Release candidate compose is available 17:08:18 #info Test matrices for Alpha are fully completed 17:08:20 #link https://qa.fedoraproject.org/blockerbugs/milestone/25/alpha/buglist 17:08:21 #link https://fedoraproject.org/wiki/Test_Results:Fedora_25_Branched_20160819.n.1_Summary 17:08:37 #topic Current status 17:08:57 adamw: could you give us current status of alpha pls? 17:09:55 (or anyone with fresh info) - I can see two proposed blocker bugs, 1 accepted bug in POST and 2 ON_QA 17:10:17 the accepted blockers are all addressed in 1.2. 17:10:26 ok, good to hear 17:10:33 .hello bowlofeggs 17:10:34 bowlofeggs: bowlofeggs 'Randy Barlow' 17:10:41 #info all accepted blockers are addressed in 1.2 17:11:00 let me do a bit of bz bureaucracy quick 17:11:07 ok 17:13:01 anything else to report for current status? 17:13:31 * stickster thinks of https://www.youtube.com/watch?v=1Iu08kXoh7o 17:13:50 otherwise I'd say we can go to mini blocker review, as I can see two bugs proposed there 17:14:07 unless adamw is not doing bz bureaucracy there too :) 17:14:13 okay, so now one of the accepted blockers isn't a blocker any more, and the other two are verified. 17:14:18 yes, we have a couple of proposed blockers. 17:14:48 can I ask you to go thought them in the mini blocker review? 17:14:56 sure. 17:15:00 thx 17:15:10 * stickster pops the corn 17:15:16 #info commence Alpha mini blocker review, hold on to your lunchpails 17:15:19 #topic (1370222) KDE Live doesn't boot to the system with network cable connected 17:15:19 #link https://bugzilla.redhat.com/show_bug.cgi?id=1370222 17:15:19 #info Proposed Blocker, NetworkManager, NEW 17:15:35 stickster, myself i was thinking https://www.youtube.com/watch?v=gSnxvFCEwfE 17:16:00 Workstation Live boots every time on the same system 17:16:08 so far this seems to be a single system bug, so i'd be inclined to -1 it... 17:16:18 satellit did say he saw a long pause during boot before KDE eventually showed up 17:16:21 (on the KDE live) 17:16:28 i just booted it on my test system and it was fine 17:16:30 we waited 5+ minutes 17:16:49 probably even 30 minutes once 17:16:50 i'd be inclined to suspect some bug between plymouth and X in this case, not anything to do with networking, but dunno for sure. 17:17:11 adamw: but unplugging the cable helps 17:17:19 and there's simple workaround acceptable for alpha - just unplug the cable 17:17:43 but I can't imagine what could be different between workstation and kde at this stage of boot 17:17:44 yes, we have seen it on one system only, -1 blocker at this moment 17:17:48 so anyway, yeah, dunno what's going on but i incline to the -1. 17:18:07 -1 blocker 17:18:37 * nirik is also -1 blocker 17:18:44 adamw: I'll secretarialize 17:18:57 thanks kparal 17:18:57 * stickster -1 blocker for this too. 17:19:01 proposed #agreed 1370222 - as so far this only appears to affect a single system, and a couple of workarounds were already discovered, this doesn't seem to merit blocker status 17:19:04 grr 17:19:09 proposed #agreed 1370222 - RejectedBlocker - as so far this only appears to affect a single system, and a couple of workarounds were already discovered, this doesn't seem to merit blocker status 17:19:15 ack 17:19:15 ack 17:19:17 ack 17:19:19 ack 17:19:25 ack 17:19:37 #agreed 1370222 - RejectedBlocker - as so far this only appears to affect a single system, and a couple of workarounds were already discovered, this doesn't seem to merit blocker status 17:19:43 #topic (1370136) glibc update corrupts display of a running system 17:19:43 #link https://bugzilla.redhat.com/show_bug.cgi?id=1370136 17:19:43 #info Proposed Blocker, systemd, NEW 17:19:53 this is what we get for letting kparal at a computer today 17:20:20 * stickster distracts kparal with something shiny 17:20:29 ooh, shiny! 17:20:43 HOLY MOLEY IT WORKED 17:20:47 This is a weird one... 17:21:01 well, i've seen this 'systemd reinitializes on glibc update' behaviour before 17:21:06 I guess I'd be inclined to -1 blocker based on what we know now... we do want to fix it of course... 17:21:13 pretty sure i've seen it result in vt flips and stuff too 17:21:25 we could also unpush glibc from updates-testing before we resolve this 17:21:28 * nirik isn't sure it's systemd but it's hard to say what it is 17:21:28 don't recall ever having graphical corruption, though. 17:21:59 currently it seems that everyone with KDE will hit this once they start updating their Alpha installations, and some people with GNOME (with AMD cards) 17:22:08 I did see some weirdness here, switching to a vty and back and everything was fine 17:22:18 yeah, that's what i got, with an intel adapter 17:22:28 happens during update though, and there's a work around 17:22:34 i can't remeber if mine was intel or radeon 17:22:35 * stickster is in the group of people with GNOME+Radeon, but fwiw doesn't feel inclined to hold on this 17:22:39 * adamw is waiting for significant other to leave so he can test on his system 17:22:50 cmurf: the workaround is offline update, which is only available for gnome 17:22:53 I also do not use Gnome 17:23:02 kparal: did you test running the update from a VT? 17:23:03 or reboot after glibc updates 17:23:07 which, i mean, is good practice anyway 17:23:09 adamw: nope 17:23:22 nirik++ 17:23:22 stickster: Karma for kevin changed to 18 (for the f24 release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:23:37 this is an (extreme) example of why there's offline updates 17:23:41 nirik: the problem is that with corrupted display you don't know if the update finished already. and how to reboot safely 17:23:59 yeah update with a vt 17:24:01 true. 17:24:30 cmurf: unless you try offline updates on windows that can block you for hours from using your computer :) 17:25:49 jreznik: sure, although rpm-ostree are out of tree updates that don't block and also don't yank libraries out from under the running system 17:25:52 kparal: can you test it from a VT and see how that goes? 17:25:55 unfortunately I can't test VT update on those radeon systems right now 17:25:55 so there's a way forward 17:25:59 ah k. 17:26:01 oh yeah, you're at home 17:26:12 if people wanna wait about ten minutes i should be able to try on a radeon box i have here. 17:26:20 but I'd expect there's 50% change it will work ok ;o) 17:26:47 did I read correctly there's a workaround in that one could switch VTs back/forth to fix the corruption? 17:26:48 probably more, because offline updates worked ok on those systems 17:27:02 stickster: sometimes. sometimes it doesn't work 17:27:06 ah 17:27:17 it worked on wayland more than on X11 17:27:20 * adamw brb, call of nature 17:28:56 I tried VT update at least in a VM, no issue 17:29:26 not even the cosmetic bug with visible cursor somewhere else on the screen 17:29:42 so is that a valid workaround? 17:29:45 so the glibc post must be triggering this somehow right? 17:29:55 nirik: yes 17:30:04 stickster: probably 17:31:32 so most probably, we have a workaround for gnome - use offline updates, and for other desktops - update in a VT 17:31:32 there's not all that much in there... so dunno 17:31:42 postinstall program: /usr/sbin/glibc_post_upgrade.x86_64 17:31:55 I'd guess it's related to systemd reloading itself 17:32:01 that's a binary, so i can't tell you offhand what it does. but it clearly triggers a systemd reinit. 17:32:25 * kparal tried VT update in VM twice, no issue 17:33:00 ah ha. 17:33:07 it's glibc's fault 17:33:26 it's sending a '/sbin/telinit u' 17:33:34 hey now 17:33:36 I can easily reproduce it by running that as root here 17:33:46 nirik: :) 17:33:54 don't do that 17:33:56 stickster: sorry, didn't mean to harsh on glibc folks. ;) 17:33:56 "Serialize state, reexecute daemon and deserialize state again. This is equivalent to systemctl daemon-reexec." 17:34:09 so...basically, that's what tells systemd to reinit. 17:34:14 nirik: Oh no, I was being funny about "that's rude of you, glibc" 17:34:26 I don't think anyone's harshing on glibc folks :-) 17:34:27 adamw: except it doesn't do it right 17:34:36 what should it do instead? 17:34:44 not telinit u at least 17:35:08 systemctl daemon-reexec according to man page 17:35:10 oh, I see what you are saying... 17:35:20 anyhow, thats drifting off topic I guess. 17:35:39 to be clear, what i posted was a quote from 'man telinit', which says what 'telinit u' does. 17:35:46 * stickster touts the value of having the right answer long after everyone else :-) 17:35:59 <-- meaning this guy 17:36:48 adamw: right, I got there, just a bit after you did. ;) 17:36:51 * nirik is updating the bug 17:37:17 ok, SO cleared out, i can test on his system as soon as this stick finishes writing. 17:37:23 I don't see anything happening on my system either way, but it could just be the age of my Radeon card 17:39:57 * adamw plays 'now testing' jingle 17:40:16 my screen goes blank witha yellow flashing cursor 17:40:27 if i go to a vt and back things are fine 17:41:07 dgilmore: what GPU? 17:41:10 it switches me to a vt... I can switch back and it's back, but alt leftarrow is now messed up and takes me back to a vt 17:41:31 kparal: 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM] 17:41:49 "systemctl daemon-reexec" runs fine 17:41:56 with no anything 17:42:35 I would say glibc should call "systemctl daemon-reexec" not "/sbin/telinit u" 17:43:07 probibly yeah 17:43:13 dgilmore: if I run sudo systemctl daemon-reexec in KDE, it has completely the same symptoms as described in the bug 17:43:26 kparal: I am using XFCE 17:43:49 so after all this seems to be a systemd bug, provided it's OK to call that command while having a live session 17:44:03 i think dgilmore/nirik's tests were invalid 17:44:04 could we skip the blocker discussion now if we unpush the glibc as kparal proposed in the beginning 17:44:21 if you run telinit -u *then* run systemctl daemon-reexec , the fact that the latter doesn't show the bug could simply be down to the fact that you ran the former first 17:44:22 we're on this one for more than 20 minutes :) 17:44:33 jreznik: i was basically waiting till i could try it on this other system i have... 17:44:45 adamw: how so? 17:44:57 yeah, in order to reproduce the bug again, you have to reboot the system 17:44:59 dgilmore: what if it only happens on the first reexec, regardless of how you trigger it?> 17:45:06 adamw: exactly 17:45:13 try a clean boot and then just run systemctl daemon-reexec without running telinit -u first... 17:45:13 adamw: I guess so 17:45:23 * dgilmore will bbs 17:45:26 will reboot 17:46:16 ah, crap. that system has an NVIDIA now. i forgot we switched the adapter. 17:46:25 I tried both GNOME and KDE in a VM, and systemctl daemon-reexec triggered exactly the same bug 17:46:26 so, that one just does the 'briefly switch to a VT, switch back, no corruption' behaviour. 17:48:08 meh. i think i'm ready to vote -1 on this 17:48:25 doesn't affect all hardware, there's probably a workaround, it's not really *fatal* you're probably gonna get out of it somehow, and this is alpha. 17:48:55 ah true. I didn't reboot... 17:48:56 * kparal is fine with -1 with those workarounds 17:49:07 I'm ok with that, -1 17:49:09 good summary, -1 here. 17:49:31 adamw: you are correct 17:49:46 but i am -1 to blocker 17:49:55 its simple to document and work around 17:50:34 proposed #agreed 1370136 - RejectedBlocker - this is obviously annoying if you hit it, but it's not exactly fatal (you can just reboot), it doesn't affect all systems (many show no significant symptoms), and it's probably workaroundable by doing the update from a VT (or offline, for GNOME) 17:50:49 ack 17:50:54 ack 17:51:41 ack 17:51:52 #agreed 1370136 - RejectedBlocker - this is obviously annoying if you hit it, but it's not exactly fatal (you can just reboot), it doesn't affect all systems (many show no significant symptoms), and it's probably workaroundable by doing the update from a VT (or offline, for GNOME) 17:52:46 so that's all for proposed blockers - thanks adamw 17:53:56 #topic Test Matrices coverage 17:54:03 so there's a couple of little holes 17:54:07 i'm just filling some of them in 17:54:49 i'm doing UEFI basic graphics mode install on one box, and i'm gonna do database role deployment on a VM. 17:55:02 is there, by any chance, an sgallagh in the house? 17:55:16 is there anything we can help with? 17:55:32 the other thing missing is Active Directory tests for server 17:55:42 so, if you happen to have an AD domain handy...https://fedoraproject.org/wiki/Test_Results:Fedora_25_Alpha_1.2_Server?rd=Test_Results:Current_Server_Test 17:57:32 nope here 17:58:25 sgallagh does, but he's our only hope 17:59:07 well, unless he left that vnc server running and i still have the connection details... 18:01:02 hmm :( 18:01:17 hello 18:02:19 nope, doesn't look like I can reach sgallagh's AD server is up atm. 18:02:30 so, i kinda can't test that. 18:02:32 :-( 18:03:14 other options? do we have any recent results to waive it based on it for example? 18:03:32 nope, no-one's run it for 25 at all. 18:03:42 sgallagh is basically the only person who ever does it (and i did it just once using his server) 18:04:12 anyone have sgallagh's cell or anything to bug him? 18:04:20 * stickster checks... 18:04:45 if we can't get him i'd probably be ok with waiving this just for alpha on the understanding that we either have sgallagh commit to test in future (now he's back) or we drop the AD requirements on the basis we can't test them 18:04:47 * stickster may have it, SMS'ing 18:05:46 UEFI basic graphics is fine, just booting a VM to test database role now. 18:05:56 should really put that in openqa...tum ti tum. 18:08:12 adamw (et al.): sgallagh texted back. He is out of town and can't test, but he also pointed out none of the code has changed since F24 release so he believes it would work for Alpha. 18:08:25 famous last words. ;) 18:08:26 ahahahahahahahaha 18:08:27 hahahaha 18:08:32 haahaahahahahaaaahaaahaaaahaaaahhhahahahahaha 18:08:37 I trust him! Completely! 18:08:38 (I assume he means just the realmd stuff) 18:08:39 oh man, that was a good one. *wipes away tears* 18:09:04 :-) only toting the message, don't get your mirth all over me!! 18:09:20 so are we ok waiving this? 18:09:38 i'm ok with waiving just for alpha, as i described above 18:09:50 database server role seems okay. well, i've got a postgre instance at least! 18:09:56 * stickster would love to see a way not to have SPOF for this, agrees with adamw too 18:10:29 that means we're covered now, right? 18:10:38 i think so 18:10:45 * adamw intentionally doesn't look too hard 18:10:51 we should probably do a #agreed on the waiving 18:10:59 working on it 18:11:27 adamw: waiving is okay 18:12:01 proposed #agreed to waive Active Directory tests for server for alpha on the understanding that we either have sgallagh commit to test in future or we drop the AD requirements on the basis we can't test them 18:12:12 ack 18:12:37 ack 18:12:37 ack 18:12:55 nirik: you tested cloud in ec2? it works? network and all? 18:13:05 #agreed to waive Active Directory tests for server for alpha on the understanding that we either have sgallagh commit to test in future or we drop the AD requirements on the basis we can't test them 18:13:24 adamw: no. openstack 18:13:41 oh yeah, i can haz read 18:13:48 can we try it in ec2? is it there? 18:14:17 oops, ack (for record) 18:15:06 yes it uploaded. 18:15:11 ack 18:15:12 I don't have any access to test with... 18:15:59 does anyone else? i don't either, never used it. 18:16:40 * mattdm hasn't used said access in a while but presumably could 18:16:49 hum, this is a bit troubling: 18:16:51 fedimg.image.test -- Fedora-Cloud-Base-25_Alpha-2.x86_64 failed testing on EC2 (us-east-1) (ami-2f751638, par 18:16:52 avirtual, gp2) 18:17:01 * dgilmore is in the same boat as mattdm 18:17:08 * nirik has no idea what testing it does there 18:17:22 nirik: all the testing in autocloud seems to have failed 18:17:34 mboddu: was looking at it earlier 18:17:43 huh. openstack worked fine. 18:17:58 and someone tested in a vm with a dvd datasource and it was fine 18:18:26 * mattdm signing in now 18:18:32 * kparal goes afk 18:18:56 nirik: is that the ami i should test? 18:19:07 dgilmore: with the grub2 fix, testing is passing on autocloud now. https://apps.fedoraproject.org/autocloud/jobs/463/output 18:19:11 dgilmore: what autocloud test failed? 18:19:17 sure, or if you want another region I can get you that 18:19:22 dgilmore: the autocloud tests look passed to me 18:19:25 But that doesn't address ec2 18:19:28 of course, the autocloud tests are kind of a weird bunch 18:19:32 *nod 18:19:38 hmm can't find it 18:19:49 are they marked public? 18:20:20 * mattdm apologizes -- have not done this in a while and everyhting is different now ;) 18:20:59 mattdm: I have not done go/no-go for while and it's all same as always :) 18:21:13 jreznik: lol no i mean the cloud tests 18:21:28 mattdm: I got it :) just joking 18:22:01 stickster: whatever one mboddu was looking at 18:22:02 nirik: any idea? 18:22:08 adamw: ^^ 18:22:27 adamw: any idea on what? 18:22:31 dgilmore: i think he may have been looking at the wrong thing. the cloud base Fedora-25-20160824.0 test is passed, you can see that. 18:22:36 nirik: where mattdm can find the iamge in ec2 18:22:41 https://apps.fedoraproject.org/autocloud/jobs/153 18:22:50 that is the Alpha compose 18:22:50 no idea. I can get ami numbers from fedmsg... 18:22:53 and it looks fine 18:23:03 stickster: I was checking it in the morning, but I saw only nightlies failing 18:23:03 stickster: and I think alpha is https://apps.fedoraproject.org/autocloud/jobs/153 18:23:10 mboddu: correct. 18:23:11 stickster: which is working fine 18:23:16 dgilmore: right. he might have looked at the nightly instead, or the atomic image tests (which aren't blocking) 18:24:15 I do not see any F25 amis -- private or public -- uploaded to EC2 18:24:45 we would need to check if fedimg needs some configuration 18:24:48 us-east-1) (ami-76701361, hvm, gp2) 18:24:55 hmm hold on there's one 18:25:05 Fedora-Cloud-Base-25-20160825.n.0.x86_64-us-east-1-HVM-gp2-0 18:25:07 and 18:25:11 Fedora-Cloud-Base-25-20160825.n.0.x86_64-us-east-1-HVM-standard-0 18:25:20 they are the nightlies 18:25:24 not the Alpha compose 18:25:38 although at this point they should be basically identical 18:25:48 ah got one for the alpha 18:25:55 mattdm: do you see a 20160824.0 ? that'd be alpha 18:26:00 (note: no .n.) 18:26:04 *just* gpt2, no alpha 18:26:15 it wouldn't have a dat 18:26:16 date 18:26:18 Fedora-Cloud-Base-25_Alpha-2.x86_64-us-east-1-HVM-gp2-0 <- no date 18:26:23 oh k. that'd be it too. 18:26:26 Fedora-Cloud-Base-25_Alpha-2.x86_64 18:26:29 sorry not "no alpha" -- no "standard" 18:26:31 Fedora-Cloud-Base-25_Alpha-2.x86_64-us-east-1-HVM-gp2-0 18:26:33 composes go by many names. 18:26:45 ami-76701361 18:26:52 dgilmore yep that's the one 18:26:59 this is where me not being up on things is hurting 18:27:06 i dunno what the implication of that being missing is 18:27:10 that is the only one :( 18:27:15 anyway I'm launching that now 18:27:18 there should be more than one afaik 18:27:33 we should still have a PV image as well as the HVM one 18:27:46 https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html 18:27:53 suggests 'standard' is a 'Previous Generation Volume' 18:27:58 so maybe they don't make it any more? 18:28:10 gp2 is 'General Purpose SSD'... 18:29:17 amazon definitely still has those. I didn't see a cloud wg decision to not support them but maybe i missed it 18:29:27 my instance is now launching.... 18:30:40 it's booting 18:30:57 it's kinda slow 18:31:10 i thought this cloud thing was supposed to be hot stuff 18:31:29 am logged in 18:31:45 it looks fine at first glance 18:31:50 storage resized itself correctly 18:32:28 quick someone who knows what's going on tell me what tests to try and where to put results :) 18:32:40 mattdm: there are specific test cases at https://fedoraproject.org/wiki/Test_Results:Fedora_25_Alpha_1.2_Cloud 18:32:45 if you could just run the two Alpha ones that'd be good 18:32:52 ack 18:32:57 which is basically, 'did it boot' and 'does journalctl work; 18:33:26 first one is good 18:33:48 * stickster waits with bated breath 18:33:55 journalctl is also good 18:34:13 as a bonus, no errors in there, also 18:34:26 alrighty, thanks a lot 18:34:26 ooh good! 18:34:38 so with that we can say test coverage is OK, i think 18:34:44 * mattdm updates matrix mage 18:34:47 oooh. 18:34:49 page. 18:34:57 #info Alpha test coverage is ok 18:35:04 but if we had a mage maybe that'd be better 18:35:17 so let's move to the most important topic! 18:35:19 well, wikitcms pretty much works by magic 18:35:24 i have to sacrifice three gerbils a day, regular 18:35:25 #topic Go/No-Go decision 18:35:42 given that we have no outstanding blockers and coverage is OK with the AD waive...QA votes Go 18:35:52 Releng is go 18:35:58 * nirik is go. Nothing blocking that I can see 18:36:29 note the image file names for the alpha aren't quite correct, but i don't think anyone's going to care much. in fact we've never once, ever, had totally 'correct' image names (as in, matching that policy i pulled out of my ass) 18:37:12 proposal #agreed Fedora 25 Alpha status is go by Release Engineering, QA and Development 18:37:14 \o/ 18:37:22 ack 18:37:23 ack 18:37:25 ack 18:37:30 ack 18:37:34 ack 18:37:34 #agreed Fedora 25 Alpha status is go by Release Engineering, QA and Development 18:37:38 awesome! 18:37:54 #topic Open floor 18:38:03 thank the maker 18:38:17 it was nice to meet you after while on go/no-go! 18:38:25 anyone has anything else for today? 18:38:27 thanks for steppin' in, jreznik 18:38:34 jreznik: nada 18:38:42 indeed thanks jreznik 18:38:53 well in this case, makers 18:38:57 dgilmore: you gonna stage today? 18:39:11 nirik: possibly 18:39:22 either this arvo or the morning 18:39:29 adamw, dgilmore: everything for fedora :) 18:39:32 the schedule calls for it tomorrow 18:39:35 ok 18:40:44 Thank you jreznik! 18:40:50 it's 8:40 pm here, time to leave home - I'm going to end this meeting in 3... 18:41:18 and thank you everyone! jkurik is going to be happy, relaxed after vacations :) 18:42:01 * jreznik is going to send the go email 18:42:05 2... 18:42:07 jreznik: for about an hour 18:42:58 1... 18:43:10 #endmeeting