19:00:01 <nirik> #startmeeting Infrastructure (2011-08-25)
19:00:01 <zodbot> Meeting started Thu Aug 25 19:00:01 2011 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:00:01 <nirik> #meetingname infrastructure
19:00:01 <zodbot> The meeting name has been set to 'infrastructure'
19:00:01 <nirik> #topic Robot Roll Call
19:00:01 <nirik> #chair smooge skvidal codeblock ricky nirik abadger1999
19:00:01 <zodbot> Current chairs: abadger1999 codeblock nirik ricky skvidal smooge
19:00:14 * skvidal is here
19:00:21 * rfelsburg is here
19:00:23 * CodeBlock here
19:00:34 <abadger1999> boa tardes
19:00:48 <skvidal> what did you just call us?
19:00:56 <skvidal> :)
19:00:57 <rfelsburg> you heard him
19:01:00 <abadger1999> skvidal: Do you really want to know? ;-)
19:01:06 <skvidal> abadger1999: I think I do know
19:01:09 <jsmith> skvidal: He called you a fine fella...
19:01:09 <skvidal> that's the problem!
19:01:11 <abadger1999> hehe
19:01:17 <nirik> ha
19:01:22 * skvidal storms around in a huff
19:01:23 <nirik> #topic New folks introductions and Apprentice tasks.
19:01:27 <skvidal> *huff*
19:01:40 <abadger1999> skvidal: too much paint is bad for the brain
19:01:45 <nirik> Any new folks want to say hi or ask questions? any apprentice tasks/tickets anyone wants to discuss?
19:01:54 <skvidal> abadger1999: you mean wall candy?
19:02:42 * rfelsburg says hi.
19:02:48 <nirik> welcome rfelsburg.
19:02:56 <rfelsburg> ty.
19:03:11 <nirik> ok, moving along...
19:03:18 <nirik> #topic Upcoming Tasks/Items
19:03:27 <nirik> so, we are out of freeze.
19:04:01 <nirik> any upcoming plans folks would like to talk about/
19:04:04 <nirik> I have a few...
19:04:22 <nirik> we finally are getting all out new machines on line in phx2.
19:04:30 * abadger1999 is finally making tie to deploy raffle
19:04:41 <nirik> abadger1999: cool.
19:05:01 <nirik> once we have new machines installed and added to monitoring, we can:
19:05:39 <nirik> look at whats on xen03/05/09/15 and migrate those things to (hopefully) new rhel6 instances on the new virthosts.
19:05:52 <nirik> those 4 machines are out of warentee at the end of next month.
19:06:22 <nirik> also, will look at migrating more rel-eng stuff to rhel6
19:06:49 <nirik> we still need community to work to move to rhel6 app servers.
19:06:49 * skvidal looks at virthost-lists.out and wonders what broke
19:07:11 * skvidal runs vmdiff
19:07:13 <lmacken> nirik: yeah, i'm working on the whole tg2/moksha/fedoracommunity on RHEL6 thing...
19:07:39 <nirik> lmacken: yeah, thanks for that. ;)
19:08:04 * lmacken just deployed a new fedoracommunity bugfix release... hopefully no more log spamming too :)
19:08:10 <nirik> hurray
19:08:13 <skvidal> lmacken: cool
19:08:47 <nirik> I have some random projects on my todo list anyone else is welcome to take and run with:
19:09:11 <nirik> * setup arpwatch on dhcp01/noc01, so we can notice if something new plugs into our networks in phx2.
19:09:26 <nirik> * add ipv6 to noc02 and get ipv6 monitoring working. ;)
19:09:45 <nirik> * more bcfg2 work in qa
19:10:09 <nirik> * get clamav filter working with mailman
19:10:12 <skvidal> * setup and config sec on log02 for specific log event alert
19:10:23 <lmacken> arpwatch sounds interesting. would it trigger a nagios alert or something?
19:10:30 <nirik> lmacken: it does email by default.
19:10:41 * lmacken runs arpon --static on all of his machines to force static arp tables (to mitigate mitm)
19:10:44 * rfelsburg will brb
19:10:52 * skvidal does not need a page about a new mac :)
19:10:56 <nirik> mails you the mac address and a thing looking it up to see what vendor, etc.
19:11:04 <nirik> it should be rare.
19:11:36 <tibbs> I use arpwatch here and find it useful.
19:11:49 <skvidal> * plan combining hosted01's mailing lists with collab1's lists
19:11:51 * nirik uses it at home.
19:12:01 <skvidal> tibbs: it is useful - not sure we need to stow stuff into nagios for it, though
19:12:09 <skvidal> an email or even just a log notice about it sshould be sufficient
19:12:32 <nirik> On the hosted/collab front, I am still waiting to hear back from serverbeach... I wonder if they got confused about our status as donated or something.
19:12:39 * nirik will followup on that.
19:13:09 <skvidal> nirik: wouldn't shock me :)
19:13:48 <nirik> so the xen move stuff will entail some outage... since db02 is in there... I'll try and come up with a schedule/plan.
19:14:10 <nirik> anyhow, any other upcoming plans? any releases for pkgdb/other apps?
19:14:30 <lmacken> i'll probably do a bodhi bugfix release tomorrow
19:14:40 <abadger1999> We might hotfix some pkgdb stuff before next freeze but we're not going to fit a new release in.
19:15:06 <abadger1999> I'm waiting with elections until we get tg2 stack fixed for rhel6 and rhel6 app servers.
19:15:09 <nirik> ok.
19:15:30 <abadger1999> Since it'll be a tg2 port.
19:15:31 <nirik> lmacken: will that fix the override needs commit on rawhide thing?
19:15:54 <lmacken> nirik: yes, it will
19:15:59 <nirik> excellent.
19:16:12 <lmacken> that patch isn't written yet, but I'll make sure to get it done before the release. should be simple.
19:16:32 <nirik> so it should just need commit on that branch or provenpackager?
19:16:59 * rfelsburg back
19:17:11 <lmacken> nirik: yep. that logic is already written for the updates portion of bodhi, so I just need to copy/paste/tweak
19:17:23 <nirik> cool.
19:17:58 <skvidal> the virthost-lists output is fixed
19:18:08 <skvidal> and the bug causing it ot be mostly empty is corrected
19:18:15 <skvidal> (sorry for the interruption)
19:18:16 <nirik> oh, I am also going to make the log02 changes I posted to the list a while back... unless folks have objections? basically allow apprentice and make the logs readable to apprentice...
19:18:24 <skvidal> worksforme
19:19:05 <nirik> #topic Meeting tagged tickets:
19:19:05 <nirik> https://fedorahosted.org/fedora-infrastructure/query?status=new&status=assigned&status=reopened&group=milestone&keywords=~Meeting&order=priority
19:19:21 <nirik> any specific tickets folks want to talk about?
19:19:45 * nirik is going to look at cleaning out the meeting tagged tickets.
19:19:56 <nirik> just put it on those ones we really do want to talk about at meetings.
19:20:45 * nirik listens to the crickets.
19:20:50 <nirik> #topic Open Floor
19:20:57 <nirik> Anyone have items for open floor?
19:22:04 <skvidal> anyone want to work on the func2nagios stuff?
19:22:27 <skvidal> it's not hard to get started and it would mean never having to ask for the basic host entry for just seeing if the box is running, etc
19:22:59 <rfelsburg> Since this has been circulating for the last week or so give or take, are we doing anything to mitigate the range header problem in apache?
19:23:23 <skvidal> rfelsburg: I suspect wait for the patch
19:23:32 <nirik> skvidal: might file a ticket on it and we can point people to it?
19:23:34 <rfelsburg> skvidal: is that puppet? the func2nagios stuff
19:23:43 <skvidal> rfelsburg: is it puppet?
19:23:50 <nirik> rfelsburg: yes, waiting for the fix. None of the workarounds are that good.
19:24:00 <rfelsburg> nirik: thanks.
19:24:11 <skvidal> rfelsburg: no - func2nagios - take list of hosts from func - do some magic to them, ask them for listening ports, dump out host entries and some basic service entries for them
19:24:11 <rfelsburg> skvidal: is the func2nagios stuff puppet related?
19:24:27 <rfelsburg> skvidal: gotcha, sorry no experience with that or i would
19:24:39 <skvidal> rfelsburg: it is mainly b/c maintianing simple, common, repeptive host entries in puppet is irritating
19:25:21 <rfelsburg> skvidal: makes sense.
19:25:25 <nirik> oh, also:
19:26:01 <nirik> #info ask fedora RFR: I am going to see if I can help mether get an initial cut added to puppet for staging. It doesn't look too hard to puppetize
19:26:21 <nirik> #info paste RFR: need to ask herlo status. He was fixing bugs in the epel package
19:27:00 <nirik> ok, if nothing else, we can call it a short meeting and get back to infraing.
19:27:04 <abadger1999> There's been some changes to hosted that people should be aware of
19:27:13 <nirik> oh yes, might be good to note those...
19:27:47 <abadger1999> I changed the hosted restricted ssh command script in the past week for a few issues
19:27:55 <abadger1999> Added ability to upload to releases via rsync
19:28:00 <abadger1999> Removed monotone support
19:28:39 <skvidal> yay
19:28:49 <skvidal> can we remove svn, hg and bzr support too?
19:28:53 * skvidal watches abadger1999 twitch
19:28:53 <nirik> abadger1999: did you see the followup about timestamps with rsync?
19:28:54 <abadger1999> And did my best to lock down bzr and hg -- before they were able to write to the user's ome dir (and thus overwrite authorized_keys and thus get them an ssh shell instead of being locked into only running SCM commands)
19:29:14 <abadger1999> nirik: I did... but I don't know what we'd need to do there.
19:29:17 <skvidal> abadger1999: good job
19:29:24 <abadger1999> nirik: I'm only limiting the directory paths in the script.
19:29:27 <nirik> yeah, I'm not sure if thats possible if you aren't root.
19:29:37 <nirik> but perhaps it should be. Will look.
19:30:06 <abadger1999> nirik: Thanks.  I can help with coding.. but I don't know what rsync options (if they exist) we'd need to inject.
19:31:19 <nirik> we don't really have any way to announce things to hosted people do we?
19:31:56 <skvidal> nirik: I guess we can email every account on the machine
19:31:59 <abadger1999> anyhow.. If there's some sort of authenticated, scm-specific issue in the next week, let me know as it might be the updated script.
19:32:05 <nirik> I thought once about: make a fedorahosted-announce list, invite all project owners to it. Would that be viable?
19:33:06 <nirik> it would be nice to have a place to announce stuff, but allow them to opt into that spamming. ;)
19:33:29 <skvidal> nirik: how much just make a list of all of them on-the-fly
19:33:38 <skvidal> and only enable the list when we need it
19:34:12 <nirik> I suppose. I bet we get people telling us they don't care... but that could work.
19:34:20 <nirik> ie, please remove me from your list.
19:34:53 <nirik> we can work out the details out of band...
19:35:13 <nirik> in addition to this rsync addition, I'd like to be able to announce the new hosted machine, migration, etc.
19:35:38 <skvidal> nirik: if they want to be removed from the list then they can stop using hosted
19:35:40 <skvidal> :)
19:35:50 <nirik> indeed.
19:35:52 <skvidal> nirik: but as long as the list defaults to 'off' when not in use
19:35:57 <skvidal> it's not like it will be a spam sync
19:36:01 <nirik> true.
19:36:02 <skvidal> I dunno
19:36:04 <skvidal> that might be silly
19:36:11 * skvidal doesn't really have a dog in the fight much
19:36:38 <nirik> ok, any other items? or shall we call it a meeting?
19:37:26 <nirik> ok. Thanks for coming everyone!
19:37:29 <nirik> #endmeeting