20:00:35 <smooge> #startmeeting infrastructure
20:00:35 <zodbot> Meeting started Thu Nov  4 20:00:35 2010 UTC.  The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:49 <smooge> #chair mmcgrath
20:00:49 <zodbot> Current chairs: mmcgrath smooge
20:00:55 <goozbach> yay! meeting
20:00:56 * nirik is hanging around.
20:00:59 <smooge> #meetingname infrastructure
20:00:59 <zodbot> The meeting name has been set to 'infrastructure'
20:01:10 <smooge> #topic Robot Roll Call
20:01:19 <smooge> crowbot here
20:01:44 * CodeBlock[s] here
20:01:48 * waltJ is still awake
20:01:51 * ninjazjb here
20:01:52 * goozbach here
20:02:09 <CodeBlock[s]> heading to class though; be back in about 5
20:02:19 <smooge> ah the wonders of technology.
20:02:26 <CodeBlock[s]> XD
20:02:33 <smooge> the ability to do something constructive in class
20:02:38 <CodeBlock[s]> i know! :D
20:02:44 <smooge> #topic Release 14 cycle
20:02:52 <smooge> ok the release happened on tuesday
20:02:59 <CodeBlock[s]> smooge: It's a personal health class; I think I can handle discussing something I care about while I learn to not smoke.
20:03:18 <abadger1999> :-)
20:03:20 * mdomsch needs a sugar+caffiene kick, bbiab
20:03:25 <smooge> https://fedorahosted.org/fedora-infrastructure/report/9
20:04:01 * skvidal is here
20:04:04 <skvidal> sorry for the delay
20:04:20 <smooge> the only tickets open at the moment are lessons learned, two web tickets, and I missed closing 11->archives
20:04:42 <smooge> how did the release go for people?
20:04:48 <skvidal> quiet for me
20:05:00 <nirik> went fine... smooth release
20:05:12 <goozbach> from an outsider's perspective it went really well
20:05:22 <goozbach> preupgrade worked like a charm
20:05:35 <ninjazjb> Mine as well
20:05:54 <smooge> ok lessons learned: 1) mmcgrath wrote a nice checklist that if followed works.
20:06:51 <smooge> anything else?
20:07:14 <smooge> ok next topic
20:07:32 <smooge> #topic adding a new report to the trac
20:07:47 <smooge> I am going to add a meeting keyword and report
20:08:36 <smooge> I am not as good as Glorious Leader at keeping things separated to keep meetings going.
20:09:14 <smooge> Next week I would like to get some tickets reviewed and added to the meeting so we can go through and fix and/or close our old ones
20:09:35 <smooge> #topic download server changes
20:09:56 <smooge> Ok this will affect sugar fix mdomsch the most
20:10:21 <smooge> but we have new hardware in PHX2 and soon in RDU that will be moving upstream downloads to.
20:10:51 <mdomsch> ?
20:10:52 <smooge> I am hoping to get them configured next week and hten we can move over in December for PHX2 and January for RDU
20:10:54 <mdomsch> hi
20:11:14 <smooge> basically RHIT's download servers will be phased out and replaced with Fedora download servers
20:11:24 * mdomsch hears angels singing
20:11:37 <smooge> too much sugar in the coffee
20:12:01 <mdomsch> oh to be in control of all our own download servers...
20:12:19 <smooge> we will still be using RHIT's netapps in RDU and PHX2 but the front ends will be ours.
20:13:06 <smooge> so I will be helping mdomsch in what he wants and using lessons learned from warthog9 to get things set up
20:13:48 <smooge> ok questions?
20:13:59 <nirik> nice.
20:14:01 * nirik cheers
20:14:06 <mdomsch> #idea we will need a new rsync module for the combination of epel, enchilada, and archive content
20:14:09 <mdomsch> that's a good time to do it
20:14:27 <nirik> fedora-bellgrande ?
20:14:35 <mdomsch> now that they're all on the same underlying netapp file system
20:14:48 <mdomsch> tengallon
20:15:00 <mdomsch> bbiab
20:15:43 <smooge> spew
20:15:53 <smooge> spew-stew
20:16:22 <smooge> #topic updating hardware
20:16:29 * CodeBlock[s] has returned
20:17:04 <smooge> we also have some replacement hardware that has arrived in PHX2 to update our soon to be out of warranty boxes.
20:17:38 <smooge> I am looking for help on best way to play the update-two-step.
20:17:59 <smooge> as in we have xen01 going away and new hardware for it. How to migrate its children etc.
20:18:13 <smooge> also do we put RHEL-5 on the new box or go to RHEL-6 and KVM?
20:18:46 <smooge> I would like to make 1 box  a RHEL-6 box so we can move a fas onto it.
20:18:54 <CodeBlock[s]> If 6, let's...make sure it doesn't reboot randomly like fas does ^^
20:19:04 <smooge> Mainly to see if we get rid of our nightly frak storms
20:20:36 <skvidal> smooge: okay - so I do have one problem with that, personally
20:20:48 <skvidal> there is nothing in our charter to suggest we need to be QA for rhel
20:20:48 <smooge> ok
20:20:53 <skvidal> and we're down one person
20:21:43 <skvidal> to me
20:21:57 <skvidal> it makes more sense to fallback to rhel5 until the rhel6 thing gets sorted
20:22:03 <smooge> yeah. I don't plan to move too much to EL-6.I think  FAS is currnelty EL-6 due to various libs not being in EL-5
20:22:05 <skvidal> but not make it be a nightly mess until then
20:23:00 <waltJ> what about RHEL-5 and KVM ?
20:24:29 <smooge> can't remember if KVM is tech-preview or fully supported in EL-5. Been doing too much EL6 stuff lately
20:25:11 <waltJ> think it is supported but a quite old version. not sure.
20:25:44 <ninjazjb> I thought it was fully supported as of 5.4
20:26:26 <smooge> the bigger issue though is that FAS is still EL-6 and would require a bunch of work to make it EL-5 (I believe). abadger1999 ?
20:26:28 <abadger1999> smooge: Yeah, that's the story for fas.  Rather than backporting all sorts of stuff, we went to rhel6.
20:27:16 <abadger1999> We could theoretically start rebuilding all of those packages for el-5 and stick them in the infrastructure repo.
20:27:20 <skvidal> the issue we have is that hosts are rebooting nightly(?) and we don't have a real good cause
20:28:16 <abadger1999> I don't know what all the packages are, though -- and it would be a big change from what we're running now since we have the rhel6 stack underneath it which means tons of things all the way down to python itself will be different.
20:28:18 <smooge> the cause seems to be el-6 kernel in xen
20:28:38 <skvidal> okay
20:28:56 <skvidal> so  - could we also do rhel6 on the bare metal + kvm to support fas?
20:29:26 <smooge> that was what I was thinking for one of the new boxes.
20:29:40 <skvidal> that seems like a good plan to me
20:29:45 <abadger1999> <nod>
20:29:54 <waltJ> +1
20:29:55 <skvidal> if we can reasonably verify that rhel6 + kvm makes it all work
20:29:56 <smooge> we have it currently on xen13 but that hardware has had MULTIPLE replacemtns so I have not tried it there
20:30:09 <skvidal> understood
20:30:10 <skvidal> the new boxes
20:30:12 <abadger1999> Minimize the update to just what we have problems with now to see if it makes things better.
20:30:16 <skvidal> what's the blockers there?
20:31:02 <smooge> boxes need to be racked and wired. new names/ip addresses given to them.
20:31:18 <skvidal> okay
20:31:23 <smooge> systems then need to be 'built' and configured
20:31:32 <skvidal> nod
20:31:42 <smooge> we have an excellent new sysadmin in the PHX2 facility so that won't be a problem
20:31:58 <skvidal> great. Do they have an ETA?
20:32:22 <smooge> the boxes arrived last week. He is working on the download servers currnetly.
20:32:40 <smooge> I will need to open a new ticket and see about getting it done in the next 2 weeks.
20:33:40 <smooge> so quick question.. do we want to call them xenXX or something else?
20:33:52 <smooge> kvm is NOT a good name.
20:34:04 <skvidal> virt##?
20:34:09 <waltJ> vmXX ?
20:34:10 <skvidal> virthost##
20:34:22 <skvidal> since they are not guests
20:34:33 <goozbach> virthost makes sense
20:35:13 <smooge> and for the silly geek question.. C arrays or Fortran arrays :) [eg start with 00 or 01?]
20:35:43 * skvidal doesn't care
20:36:00 <skvidal> but if anyone changes the frelling width of the numbers they will taste wrath :)
20:36:09 <skvidal> looks like 01 is what we use all over
20:36:13 <skvidal> bastion01
20:36:15 <skvidal> puppet01
20:36:16 <skvidal> people01
20:36:20 <skvidal> let's stick w/that
20:36:44 <smooge> yeah. I can rename xen13 to virthost00 since its crap hardware :).
20:37:02 <skvidal> umm
20:37:05 <skvidal> wait why?
20:37:17 <skvidal> just call it 01
20:37:18 <skvidal> or 0N
20:37:24 <skvidal> whatever the next number is
20:37:35 <skvidal> why break track with what we do everywhere else?
20:39:13 <smooge> sorry I was being silly.
20:39:20 <skvidal> oh - I didn't catch the joke
20:39:42 <smooge> np my humour is rough.. which means the pain medicine is wearing off.
20:39:52 <smooge> ok i think htat is it at the moment.
20:39:58 <smooge> #topic Open Floor
20:40:14 <goozbach> I'd like to introduce myself
20:40:19 <smooge> hello goozbach
20:40:22 <smooge> how are you?
20:40:35 <goozbach> I've been using fedora for a good long while and wanna join in on the infrastructure fun
20:40:39 <goozbach> so I
20:40:45 <goozbach> am looking for a sponsor
20:40:49 <ninjazjb> hello
20:41:04 <goozbach> smooge: doing well.  excited to be here
20:41:16 <goozbach> I've already signed up for the infra list
20:41:27 <goozbach> is there anything else I need to do to get started?
20:42:38 <smooge> well say hi on the list, find out what you would work on, and listen regularly on #feodra-admin and #feodra-noc for problems
20:42:42 <CodeBlock[s]> goozbach: take a look on https://fedorahosted.org/fedora-infrastructure/report/1?sort=created&asc=0 and see if anything sparks your interest basically
20:43:17 <smooge> Our big issues will be keeping up documentation, figuring out where things are not clear, and making them more clear.
20:43:37 <goozbach> I've already read the "getting started" guide.
20:44:09 <smooge> ninjazjb, is a new fellow too and so is waltJ. CodeBlock is actually turning into an old-timer :).
20:44:26 * ninjazjb waves
20:44:46 <goozbach> welp, I've joined both channels, I'll craft an email to the list.
20:45:46 <ke4qqq> smooge: mind if I make an announcement?
20:46:07 <CodeBlock[s]> smooge: :D
20:46:27 <smooge> ke4qqq, sure
20:46:35 <CodeBlock[s]> smooge: I've had so much fun here, I don't plan on leaving any time soon... if I'm not yet, I will soo be an old-timer ;)
20:46:41 <CodeBlock[s]> soon*
20:46:47 <smooge> ke4qqq, I mean "I do not mind if you make an announcement."
20:47:06 <ke4qqq> So, FAmSCo manages a bit of budget and we regularly don't spend it all. In addition there's talk of us taking on more of Fedora's budget responsibilities.
20:47:41 <smooge> cool
20:48:01 <ke4qqq> And rather than effectively lose that which we don't spend, (which we recognize as lost opportunity) we wnat to make sure that other groups in Fedora know they can come obtain resources. So if you need something to help your efforts in Fedora, please don't hesitate to ask
20:48:20 <ke4qqq> Be that attending a conference like LISA or Pycon, or some piece of hardware.
20:48:37 <CodeBlock[s]> smooge: can we tackle/discuss infra #2461 (it has the meeting keyword, not sure if you saw it)
20:48:48 <smooge> ok ke4qqq where to ask?
20:48:55 <ke4qqq> I'll send a message to the list about this as well, but wanted to get it in front of you.
20:49:09 <ke4qqq> smooge: the easiest way is to talk to a famsco member or a ticket on famsco's trac
20:49:12 * ke4qqq goes to get a link
20:49:27 <ke4qqq> https://fedorahosted.org/famsco/
20:49:32 <ke4qqq> #link https://fedorahosted.org/famsco/
20:49:36 <smooge> thanks
20:49:45 <ke4qqq> EOF - thanks smooge
20:50:23 <smooge> your welcome. I look forward to seeing about USENIX stuff :)
20:50:41 <smooge> CodeBlock, ok ticet 2461
20:50:47 <smooge> .ticket 2461
20:50:48 <zodbot> smooge: #2461 (Allow some people outside of sysadmin-main to increase people01 quotas) - Fedora Infrastructure - Trac - https://fedorahosted.org/fedora-infrastructure/ticket/2461
20:51:28 * CodeBlock[s] talks briefly
20:51:58 <CodeBlock[s]> I see requests for that quite a lot, and I'd like to help with that/be able to increase those... mmcgrath said that maybe we could allow -tools to do that or something along those lines
20:53:06 <skvidal> it's a sudo thing - it's not hard
20:53:14 <skvidal> though we don't get many requests
20:53:25 <skvidal> and I don't remember when the last one went un-answered for more than like an hour
20:53:30 <CodeBlock[s]> skvidal: I see quite a lot on the tracker for those
20:53:44 <skvidal> CodeBlock[s]: un fulfilled?
20:53:55 <smooge> skvidal, since you deal with this a lot.. this looks workable just a policy and guidelines issue and then dealing with people02 when it comes up.
20:54:07 <nirik> oh, I have something to mention I guess too.
20:54:07 <skvidal> smooge: yah  - I'm fine with that
20:54:17 <CodeBlock[s]> skvidal: https://fedorahosted.org/fedora-infrastructure/ticket/2421 https://fedorahosted.org/fedora-infrastructure/ticket/2423 https://fedorahosted.org/fedora-infrastructure/ticket/2411
20:54:18 <skvidal> and added people02 back onto my todo list too
20:54:22 <CodeBlock[s]> three that have gone for days
20:54:31 <skvidal> CodeBlock[s]: hmm - I hadn't seen them
20:54:33 <skvidal> which is odd
20:54:35 <skvidal> b/c I get the emails
20:54:38 * skvidal grumbles
20:54:44 <smooge> me either.. so looks like another issue to deal wioth
20:55:31 <smooge> #action find out where email tickets are going
20:55:42 <smooge> #action work on SOP and policy
20:55:48 <skvidal> 'assigned to nobody'
20:56:24 <smooge> we have 3 minutes untiul the cloud meeting
20:56:28 <nirik> looking at ticket 2413: Does http://fpaste.org/pb0j/ look like everything to do to rename that list?
20:56:52 * nirik is happy to have folks look and give feedback out of band.
20:57:14 <CodeBlock[s]> smooge: is -tools going to be allowed to do that then, or what?
20:57:33 <CodeBlock[s]> sorry I'm kind of fading in and out here between class and here
20:57:48 <smooge> CodeBlock, a) probably yes. b) need to figure out changes first
20:58:01 <CodeBlock[s]> smooge: alright sounds good
20:58:01 <smooge> one last thing
20:58:08 <smooge> meeting time next week
20:58:14 <smooge> I would like to keep it at the same UTC
20:58:25 <smooge> does that work or kill people?
20:58:35 <goozbach> WFM
20:58:37 <goozbach> :)
20:58:43 <CodeBlock[s]> mmm, that would be what EST?
20:58:57 * CodeBlock[s] adds the 4 hours in his head I guess
20:59:06 <CodeBlock[s]> well
20:59:10 <CodeBlock[s]> it's +1 hour right?
20:59:12 <CodeBlock[s]> ble
20:59:15 <CodeBlock[s]> bleh* have to go
20:59:17 <smooge> correct
20:59:21 <smooge> see you
20:59:28 <smooge> rbergeron, you ready to take over?
20:59:44 <smooge> ok I am going to end this and put it on the list
20:59:48 <smooge> see you next week.
20:59:52 <smooge> #endmeeting