20:01:40 <smooge> #startmeeting infrastructure 20:01:40 <zodbot> Meeting started Thu Mar 10 20:01:40 2011 UTC. The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 20:01:47 <smooge> #meetingname infrastructure 20:01:47 <zodbot> The meeting name has been set to 'infrastructure' 20:01:57 <smooge> #chair nirik skvidal 20:01:57 <zodbot> Current chairs: nirik skvidal smooge 20:02:15 <nirik> ok, over here? 20:02:35 <smooge> I think this will be short and it seemed what people wanted from the emails 20:02:54 <smooge> my left hemisphere is pounding so lets just do this 20:03:38 <nirik> ok. ;) 20:04:05 <nirik> goozbach should have the agenda. 20:04:14 <smooge> #chair goozback 20:04:14 <zodbot> Current chairs: goozback nirik skvidal smooge 20:04:18 <smooge> #chair goozbach 20:04:18 <zodbot> Current chairs: goozbach goozback nirik skvidal smooge 20:04:41 <smooge> ok #topic roll call 20:04:49 <smooge> #topic roll call 20:04:49 <goozbach> here 20:04:51 * skvidal ishere 20:04:58 <marchant> here 20:04:58 * nirik is around 20:05:42 <smooge> ok goozbach could you run the agenda. my head hurts too much to be on top of it 20:06:33 <goozbach> smooge: on it 20:06:37 * abadger1999 here but will be back in session in a moment 20:07:02 <goozbach> #topic serverbeach outage 20:07:11 <goozbach> who has the update about the outage? 20:07:30 <nirik> it's serverbeach1/2/3 machines. 20:08:01 <goozbach> and what's on them? 20:08:07 <nirik> which is ns1, asterisk1/2, and collab1 20:08:07 <goozbach> who needs to be notified? 20:08:12 * nirik doublechecks. 20:08:15 <goozbach> has the notification happened? 20:08:36 <smooge> I started writing the outage last night and got 20:08:39 <smooge> sidetracked 20:08:41 <smooge> serverbeach2.fedoraproject.org:asterisk1:running 20:08:41 <smooge> serverbeach2.fedoraproject.org:collab1:running 20:08:41 <smooge> serverbeach2.fedoraproject.org:ns1:running 20:08:42 <smooge> serverbeach3.fedoraproject.org:asterisk02:running 20:08:42 <smooge> serverbeach3.fedoraproject.org:collab2:running 20:08:52 <smooge> serverbeach1 is a backup downloader for some things 20:09:36 <smooge> the email was going to go to fedora-announce, fedora-devel, and fedora-infrastructure. 20:09:41 <nirik> so, mailing lists (collab*) and talk will be down. 20:09:51 <nirik> ns2 should be able to handle our ns needs? 20:10:01 <smooge> nirik, I don't know. 20:10:19 <skvidal> nirik: in terms of scale - I bet so 20:10:21 <nirik> and we probibly don't have time to add a ns3 and add it to whois? 20:10:36 <goozbach> how long is the outage? 20:10:37 <nirik> the actual outage is pretty short... 15min somewhere in the 6 hour window. 20:10:37 <skvidal> nirik: correct - it wouldn't make it out ot the world 20:10:42 <skvidal> goozbach: theoretically? 20:10:44 <skvidal> not that much 20:10:48 <skvidal> what nirik said 20:10:55 <goozbach> I see 20:11:13 <goozbach> who's gonna be on hand to hold hands to get it back up if things go south? 20:11:14 * CodeBlock here - why are we not in -admin? 20:11:20 <goozbach> CodeBlock: last min change 20:11:22 <goozbach> :) 20:11:32 <goozbach> I'm being the slave driver today so we'll end on time 20:11:35 <goozbach> :) 20:12:03 <goozbach> what kind of outage is it? reboot? power? networking? 20:12:08 <skvidal> networking 20:12:10 <nirik> goozbach: networking. 20:12:22 <skvidal> goozbach: it's not ours - its serverbeach's 20:12:24 <goozbach> so the hosts themselves dont need babysat to reboot 20:12:24 <nirik> upgrading switches. 20:12:27 <goozbach> gotcha 20:12:54 <nirik> so, we send outage annouce today and moving forward we look at adding a ns3 that is elsewhere? 20:13:00 <skvidal> nod 20:13:11 <goozbach> #info serverbeach outage affects asterisk1, collab1, ns1, asterisk02, and colab02 20:13:15 <nirik> collab and asterisk will be harder to multihome, but we should look at that long term 20:13:39 <goozbach> #info outage is switch upgrade so servers don't need babysitter other than check they are alive 20:14:10 * nirik thinks thats all on this. 20:14:14 <goozbach> #action -- multi-home asterisk 20:14:21 <skvidal> umm 20:14:22 <skvidal> no 20:14:27 <skvidal> kick asterisk in the face 20:14:30 <skvidal> and be done with it 20:14:33 <nirik> well, or remove. ;) but thats another topic. 20:14:39 * skvidal watches the clock 20:14:44 <skvidal> tic tic tic 20:14:56 <goozbach> #action blow asterisk away 20:14:59 <nirik> topic++ ? 20:15:05 <skvidal> +++ 20:15:07 <skvidal> next 20:15:12 <skvidal> +++ATH0 20:15:15 <skvidal> NOCARRIER 20:15:18 <nirik> ha. 20:15:19 <goozbach> #topic kick cvs in the head 20:15:28 <goozbach> nirik: you again? updates? 20:15:30 <nirik> ok, so I looked at updates on this... 20:15:48 <nirik> there are 2 CVS using projects that don't want to go to fedorahosted because the loves the cvs. 20:15:59 <nirik> I think we should tell them to go to sourceforge or whoever still does cvs... 20:16:05 <nirik> and set a deadline. 20:16:05 <goozbach> #action multi-home collab 20:16:17 <skvidal> goozbach: multihoming collab isn't happening this week 20:16:26 <nirik> .ticket 1519 20:16:26 <skvidal> so it is kinda a silly action item 20:16:28 <zodbot> nirik: #1519 (Add packages to infrastructure repo) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/1519 20:16:29 <goozbach> no, but I'm recording minutes of things we brought up 20:16:44 <goozbach> should prob be an #idea instead 20:16:46 <skvidal> nirik: nod. I suspect we'll get some other push back from them 20:16:46 <nirik> goozbach: might be better to "#info look into ways of multihoming collab" 20:16:52 <skvidal> nirik: but here's another option 20:17:07 <skvidal> nirik: if they do not need cvsweb 20:17:07 <goozbach> I'm still getting the hang of this meetingbot thing 20:17:07 <nirik> also, there are 'elvis' changes? 20:17:08 <skvidal> or something 20:17:15 <nirik> goozbach: no worries. 20:17:19 <skvidal> nirik: put the cvs repo on people01 20:17:21 <skvidal> in a shared dir 20:17:25 <skvidal> and tell them to do it via ssh 20:17:34 <goozbach> #idea look into building ns3 and getting it out to the world 20:17:51 <skvidal> nirik: I think I mentioned this before - possibly in the ticket 20:18:02 <nirik> skvidal: yeah, that could work, but someone who doesn't want to move from cvs, probibly doesn't want to change their workflow at all. 20:18:26 <skvidal> nirik: this is not our problem 20:18:34 <nirik> but we don't want to keep cvs forever, and times change. ;) 20:18:35 <skvidal> nirik: we are not here to provide perpetuity of their workflow 20:18:54 <skvidal> i'll ask in the ticket 20:18:56 * nirik nods. 20:19:01 <goozbach> tell them CVS over ssh, or move to git? 20:19:08 <skvidal> or to svn 20:19:09 <skvidal> or bzr 20:19:12 <skvidal> or hg 20:19:14 * skvidal doesn't care 20:19:15 <goozbach> yeah 20:19:18 <skvidal> just NOT CVS 20:19:22 <nirik> There is one more project to move to git, the 2 in cvs, and the elvis stuff and then I think we can turn out the lights. ;) 20:19:41 <goozbach> #info two projects in CVS, Elvis, then lights out 20:19:52 <goozbach> #info one more to move to git 20:20:03 <goozbach> nirik, skvidal anything else? 20:20:10 <nirik> nope. Move on. 20:20:18 <skvidal> nope 20:20:23 <goozbach> #topic server updates 20:20:28 <goozbach> skvidal: take it away! 20:20:51 <skvidal> okay 20:20:53 <skvidal> so here you go 20:21:03 <skvidal> very basic update info we have 20:21:03 <skvidal> http://skvidal.fedorapeople.org/hidden/update-info-2011-03-10/ 20:21:12 <skvidal> these boxes get a kernel update 20:21:12 <skvidal> http://skvidal.fedorapeople.org/hidden/update-info-2011-03-10/kernel-updates 20:21:26 <skvidal> these boxes have something that someone has labeled as 'security' 20:21:27 <skvidal> http://skvidal.fedorapeople.org/hidden/update-info-2011-03-10/security 20:21:35 <skvidal> and here is what they are 20:21:35 <skvidal> http://skvidal.fedorapeople.org/hidden/update-info-2011-03-10/security-updates-by-host 20:21:41 <skvidal> a lot are just git-related 20:22:06 <skvidal> and here are all the boxes with updates 20:22:07 <skvidal> http://skvidal.fedorapeople.org/hidden/update-info-2011-03-10/update 20:22:21 <skvidal> all the boring ones will get applied 20:22:29 <skvidal> like x86## 20:22:33 <rbergeron> ;) 20:22:35 <skvidal> and the xen## boxes 20:22:49 <skvidal> when I get to the app## and db## servers I'll be pinging people 20:22:55 <skvidal> to find out what I'm about to light on fire 20:23:07 <skvidal> I'll probably wait until toshio is back intown to light the app servers on fire 20:23:17 <smooge> yeah. where is he? 20:23:23 <abadger1999> <nod> 20:23:26 <abadger1999> Atlanta for pycon 20:23:27 <skvidal> smooge: pycon 20:23:30 <nirik> what about reboots? 20:23:44 <skvidal> nirik: well all the kernels will need them - but not right away 20:23:44 <smooge> cries a tear at not being able to go 20:23:49 <abadger1999> I'll be back at the end of next week. 20:23:52 <skvidal> nirik: it's not like we remove the old ones 20:23:58 <skvidal> abadger1999: END of next week? 20:23:58 <nirik> yeah, should we schedule those sometime? 20:24:01 <skvidal> abadger1999: pycon is a week long? 20:24:15 <skvidal> nirik: well - that's the reason to apply the updates - to schedule them all 20:24:22 <skvidal> nirik: our biggest problem is so many of them are on xen instances 20:24:26 <skvidal> and we need to reboot those.... 20:24:28 <nirik> yeah... 20:24:32 <abadger1999> skvidal: conference is 3 days. Then a language summit before and hackfests afterwards. 20:24:36 <skvidal> so we end up in a reboot cacauphony of doom 20:24:38 <goozbach> #info skvidal to start applying updates and will ping when hitting special boxes 20:24:40 <skvidal> abadger1999: ha 20:24:58 <nirik> also, it would be cool if someday we could get to the point where we could do mass reboots of everything and not have it service impacting at all... 20:25:05 <nirik> ie, enough redundency. 20:25:20 <goozbach> #idea get enough redundancy to have mass reboots without service impact 20:25:22 <goozbach> :) 20:25:22 <skvidal> nirik: it would be 20:25:27 <skvidal> nirik: but that's not today most of the time 20:25:37 <skvidal> nirik: a lot of that is b/c we're doing our own hosting 20:25:41 <nirik> yeah... with the db boxes, and lots of stuff on xen, etc. 20:25:44 <skvidal> so we have to reboot the xen boxes 20:25:47 <skvidal> and not just the host 20:25:51 <skvidal> s/host/guest/ 20:26:00 <skvidal> if I only had to reboot guests this would be much simpler 20:26:20 <nirik> well, if we could migrate guests around that would do it too... 20:26:28 <nirik> but that usually wants shared storage. ;) 20:26:43 <skvidal> nirik: for some of the hosts 20:26:46 <skvidal> it is not important 20:26:54 <skvidal> but for a bunch of that - migrating would be expensive 20:26:57 <skvidal> super-duper expensive 20:27:00 <skvidal> b/c of all the disk they have 20:27:05 <nirik> anyhow, back to updates... should we send an outage announcement on this? make a window? or ? 20:27:11 <skvidal> for the updates? 20:27:13 <skvidal> nah 20:27:23 <skvidal> justapplying them is normally danger-free for most of the cases 20:27:26 <nirik> well, if they had sheepdog or cloudfs in the backend, it would be not expensive. ;) 20:27:28 <skvidal> the weird cases are app* 20:27:42 <skvidal> nirik: I meant time-expensive - but I agree 20:27:50 <skvidal> nirik: which is another reason to pursue those paths in the future 20:27:54 * nirik nods. 20:28:03 <nirik> I meant outage for the reboots. 20:28:03 <skvidal> and as time goes on 20:28:10 <skvidal> I think right this moment 20:28:14 <skvidal> we can delay on the reboots a bit 20:28:28 <skvidal> until we have something to pair it with 20:28:32 <skvidal> or a better time 20:28:39 <skvidal> alternatively we can do them a bit at a time 20:28:47 <skvidal> invariably, whenever we do reboots 20:28:52 <skvidal> we get a new kernel update the next day 20:28:59 <nirik> sure, the kernel update doesn't seem too critical to me. 20:29:06 <nirik> https://rhn.redhat.com/errata/RHSA-2011-0303.html 20:29:26 <skvidal> it's not 20:29:36 <nirik> https://rhn.redhat.com/errata/RHSA-2011-0329.html 20:29:50 <nirik> go forth and update. ;) 20:29:53 * nirik has nothing more on this 20:30:01 <skvidal> nod 20:30:06 <CodeBlock> skvidal: probably want to reboot before next freeze though right? 20:30:15 <skvidal> seems reasonable 20:30:19 <skvidal> that's 3/28 iirc 20:30:22 <CodeBlock> yeah 20:30:26 <goozbach> #idea reboot before next freeze 20:30:37 <smooge> ツ 20:30:42 <goozbach> skvidal: anything further? 20:30:50 <skvidal> not at the moment 20:30:54 <goozbach> very well 20:30:54 <skvidal> thx 20:31:09 <goozbach> #topic "gather community feedback" 20:31:21 <goozbach> do we want to persue this further? 20:31:40 <goozbach> or did we leave it as "give us a RFR and we'll setup a team on it"? 20:31:45 <nirik> I don't think there is anything for infra to do yet here. 20:32:01 <skvidal> agreed there 20:32:04 <goozbach> I thought as much, but didn't want it to fall off the radar 20:32:12 <nirik> If someone wants to run with a webapp that gathers feedback, great... 20:32:23 <goozbach> #agreed community feedback will wait till we recieve a RFR 20:32:29 <nirik> but until then, I think we have enough to keep busy. 20:32:29 <goozbach> good good 20:32:40 <goozbach> fresh cup move down 20:32:44 <goozbach> #topic meeting tickets 20:33:05 <goozbach> I'm not certian which of these tickets are most pressing, but I've picked the ones which look the most important to me 20:33:23 <goozbach> #url https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assigned&status=reopened&group=milestone&keywords=~Meeting&order=priority 20:33:33 <goozbach> .ticket 2502 20:33:36 <zodbot> goozbach: #2502 (Retrace Server) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2502 20:34:02 <goozbach> retrace server RFR 20:34:04 <smooge> Ok this is basically going to need for hardware to be bought by the team to meet their larger goals 20:34:26 <goozbach> #info needs more hardware to meet the goals 20:34:32 <goozbach> smooge: so it's in a holding pattern? 20:35:15 <smooge> yes 20:35:21 <goozbach> ok 20:35:28 <nirik> I suspect they need lots and lots of disk 20:35:33 <goozbach> .ticket 2517 20:35:35 <zodbot> goozbach: #2517 (Need mod_evasive for EL6) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2517 20:35:56 <goozbach> anyone working on this? what's the priority? 20:35:59 <skvidal> do we actually need this? 20:36:15 * nirik doesn't know. 20:36:52 <goozbach> we'll move on then 20:37:04 <goozbach> .ticket 2531 20:37:05 <zodbot> goozbach: #2531 (DB03 update) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2531 20:37:42 <goozbach> this is the postgres upgrade, the plan has been laid out in the ticket, but nothing further 20:38:03 <goozbach> skvidal? 20:38:11 <nirik> is this something we can get to before the next freeze? 20:39:01 <skvidal> I don't know 20:39:05 <skvidal> the plan went away 20:39:28 <skvidal> mainly due to notenoughtime 20:39:30 * nirik notes the schedule of freezes is at http://rbergero.fedorapeople.org/schedules/f-15/f-15-infrastructure.html 20:39:40 <smooge> I don't think we can do this by next freeze 20:39:55 <smooge> It requires dgilmore who is busy with family for a bit 20:40:08 <smooge> We are going to have enough issues with moving systems next week 20:40:11 <goozbach> if we have reached NOTENOUGHTIME and the big part of help is busy, then it's gonna have to hold 20:40:12 <nirik> yeah. 20:40:23 <nirik> so, lets push it out to between beta and final sometime? 20:40:27 <goozbach> #agreed needs to be held as not enough time before freeze 20:40:37 <nirik> and/or ask dgilmore when 20:40:37 <goozbach> #idea move between beta and final 20:40:46 <smooge> nirik, since it affects the db for builds.. I think it will have to wait til after final 20:41:02 <goozbach> smooge: good point 20:41:09 <nirik> yeah, possibly... although it's never a good time to stop builds. ;) 20:41:17 <nirik> someone will always want to be bulding something. 20:41:35 <goozbach> nirik: but more so when we are alpha or beta 20:41:41 <goozbach> ok next 20:41:54 <goozbach> .ticket 2539 20:41:55 <zodbot> goozbach: #2539 (decom xb-01 and reallocate bxen01) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2539 20:42:07 * nirik shrugs. It's always the case... security issue comes up, or a bad bug is found. :) There's never a good time. 20:42:23 <smooge> ok this needs to be another dgilmore needs time issue. 20:42:50 <goozbach> #info another dgilmore item 20:42:53 <smooge> I can rebuild xb-01 on bvirthost01 but I don't know how useful that will be. 20:42:59 <skvidal> nod 20:43:04 <goozbach> got it, and moving on... 20:43:04 <nirik> what is xb-01 for? 20:43:09 <goozbach> or not... 20:43:09 <skvidal> good question 20:43:12 <goozbach> :) 20:43:20 <goozbach> action item to find out? 20:43:22 <smooge> no idea. it is a xen build host. 20:43:59 <nirik> ok. 20:44:01 <smooge> I don't know if its there for testing whehter builds work on xen or something else. I don't think it is used much 20:44:10 <smooge> because its ssssllllloooooowwwww 20:44:49 <nirik> ok, move on, we will find more info. ;) 20:44:51 <goozbach> well we'll look later 20:44:59 <goozbach> #action find info xb-01 20:45:12 <goozbach> .ticket 2574 20:45:13 <zodbot> goozbach: #2574 (Perform regular inactive account prunings and possibly a password reset policy.) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2574 20:45:16 <goozbach> that's the pruning ticket 20:45:18 <goozbach> any updates? 20:45:56 <smooge> I keep getting sidetracked 20:46:23 <smooge> I need to talk with abadger after meeting on a better way to get the info I need out of fas because its all using zodbot at the moment 20:47:02 <abadger1999> smooge: You can start here: https://fedorahosted.org/releases/p/y/python-fedora/doc/existing.html#fas 20:47:06 <smooge> is 2544 and 2542 on your list 20:47:19 <goozbach> yup 20:47:22 <goozbach> let's do those 20:47:28 <goozbach> .ticket 2544 20:47:31 * nirik notes we have about 13min left. 20:47:35 <zodbot> goozbach: #2544 (migrate autoqa01 elsewhere) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2544 20:47:39 <smooge> ok 2544 goes down next week 20:47:42 <goozbach> last two tickets, then open floor 20:47:53 <smooge> I will not be at the meeting most likely as it will be my whole focus 20:48:00 <smooge> [if I get network data ever] 20:48:14 <goozbach> #info ticket 2544 will be happening next week, smooge will be doing that during meeting time 20:48:18 <smooge> we have racks, we have people to move things. I just ened ips and firewall changes 20:48:36 <nirik> smooge: you going out on site? 20:48:42 <nirik> or just wrangling from there? 20:48:42 <smooge> no. no money to do so. 20:48:46 <nirik> ok. 20:48:56 <smooge> I would feel better if I could but it owuld be out of pocket 20:49:05 <smooge> if something goes wrong, I will do so though 20:49:31 <goozbach> #info just needs IP address and firewall changes 20:49:42 <smooge> things that I am waiting on to get this done: 1) network ips 2) word from the s390 and ppc community people that I can get their help 20:50:06 <smooge> other than that its ready 20:50:26 <smooge> as it is tied in with autoqa as they are going to the same network 20:50:32 <smooge> ok I am done on that. 20:50:39 <goozbach> .ticket 2542 20:50:40 <zodbot> goozbach: #2542 (reinstall fas01 on rhel6 kvm host) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2542 20:51:04 <skvidal> we have 3 rhel6 kvm hosts now, right? 20:51:06 <smooge> We got the hardware finally (it was supposed to be ready before fudcon) I am hoping to get it networked and ready next week 20:51:15 <smooge> then we will have 3. 20:51:25 <nirik> cool. 20:51:58 <smooge> I am really hoping it can happen tomorrow as I would like to have monday and tuesday set up of moving stuff off of old xen boxes. 20:52:14 <goozbach> #info hardware finally here, will have three RHEL6 kvm hosts when finished 20:52:24 <smooge> and I am done 20:52:27 <nirik> the other two are internetx and ? 20:52:43 <goozbach> very well 20:52:47 <goozbach> #topic open floor 20:52:51 <goozbach> wait 20:52:55 <goozbach> #topic meeting time 20:53:08 <goozbach> next week is DST change for the US 20:53:23 <nirik> oh yeah, I was going to suggest we move back an hour (or two) and meet here again moving forward. 20:53:25 <goozbach> we also have been bumping up against the cloud meeting here in this channel 20:53:39 <nirik> yeah, thus I was suggesting we move back an hour. ;) 20:53:41 <goozbach> so moving the meeting to 1900 UTC 20:53:51 <goozbach> +1 from me 20:53:56 <goozbach> +1 from nirik 20:54:03 <goozbach> anyone else want to chime in? 20:54:17 <smooge> +1 as long as I get nirik to take over soon 20:54:34 <goozbach> nirik: you want to run meeting next week? 20:54:55 * skvidal is fine 20:55:00 <skvidal> whatever 20:55:02 <goozbach> I'll still do agenda unless I'm at the hospital delivering my baby 20:55:03 <nirik> goozbach: doesn't matter to me... 20:55:03 <skvidal> my criteria are 20:55:05 <skvidal> not interrupted 20:55:10 <skvidal> and not pushed out of a virtual space 20:55:29 <goozbach> skvidal: having someone sheperd the meeting helps with the second 20:55:34 <goozbach> having an agenda helps too 20:55:37 <goozbach> ok 20:55:39 <goozbach> so it's 20:55:42 <smooge> no I meant more in nirik taking over infra team soon 20:55:44 <nirik> goozbach: I can... is your baby due soon? 20:55:53 <skvidal> goozbach: I don't like timelines 20:56:09 <goozbach> #agreed meeting moved to Fedora-meeting at 1900UTC 20:56:24 <goozbach> nirik: any day now actually 20:56:35 <goozbach> #topic open floor 20:56:38 <goozbach> anything else? 20:57:07 * nirik looks forward to working with folks more moving forward. ;) 20:57:38 * CodeBlock looks forward to working with nirik more too :D 20:57:58 <nirik> oh, I note that I will probibly be somewhat off line the first week of april. 20:58:33 * CodeBlock wishes nirik the best of luck with the new position btw 20:58:34 <nirik> a minute to spare for the cloud folks. ;) 20:58:39 <goozbach> yay! 20:58:46 <goozbach> ok nothing else I move to 20:58:50 <goozbach> #endmeeting