18:00:08 <nirik> #startmeeting Infrastructure (2012-08-23)
18:00:08 <zodbot> Meeting started Thu Aug 23 18:00:08 2012 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:08 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:08 <nirik> #meetingname infrastructure
18:00:08 <zodbot> The meeting name has been set to 'infrastructure'
18:00:08 <nirik> #topic Greets!
18:00:08 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean
18:00:08 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean
18:00:15 * abadger1999 here
18:00:16 * skvidal is here
18:00:21 * Betapanda is here
18:00:31 <WolfKit> excuse me.  sillyness from another channel
18:00:32 * WolfKit is here
18:00:33 <iambryan> /me is here
18:00:35 * lmacken 
18:00:46 * threebean 
18:00:50 * relrod here
18:01:25 * pingou is here
18:01:31 <nirik> ok, lets go ahead and dive on in...
18:01:40 <nirik> #topic New folks introductions and Apprentice tasks.
18:01:45 <nirik> If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy
18:02:19 <smooge> here
18:02:40 * rossdylan is here
18:02:53 <nirik> any new folks want to say hi? or shall we move on...
18:03:02 <iambryan> Hey all, I am a fedora user for the past 9 years and have admin'd RHEL boxes with NOC infra management apps
18:03:36 <iambryan> I would like to help out on the main admin side and learn some python while I'm at it
18:03:46 <pingou> welcome :)
18:03:49 <ctria> hi all
18:03:54 <ctria> sorry for being late
18:03:55 <iambryan> Thanks
18:03:57 <nirik> welcome iambryan. :) We can point you to some things to look at after the meeting over in #fedora-admin
18:04:12 <nirik> no worries ctria.
18:04:16 <nirik> #topic Applications status / discussion
18:04:16 <ctria> hm i'm on time! i'm back after some months away
18:04:23 <iambryan> Great, will stick around then :)
18:04:57 <nirik> Any applications news this week or upcoming? abadger1999 / relrod / threebean / lmacken / pingou / anyone I missed?
18:05:18 <pingou> I'm slowly but surely trying to move FAS's db model to full SA
18:05:39 <pingou> eof for me
18:06:03 <nirik> pingou: cool. so thats in prep for moving it to TG2? or ?
18:06:07 <relrod> Just pushed another Flask build to el6 testing, to fix the broken deps warnings on the doc subpackage. So please continue testing flask from epel, poke me/file bugs on it if it does anything insane
18:06:25 <nirik> relrod: any news on the plugins issues tflink was hitting?
18:06:38 <relrod> nirik: Haven't had a chance to talk with him much yet, no :(
18:06:43 <pingou> nirik: I think that will help whatever framework we choose (if it uses SA)
18:07:07 <relrod> nirik: will try and catch up with him today if he's around
18:07:12 <nirik> cool.
18:07:38 <rossdylan> I need to look over fedbadges once more and make sure evrything works in EPEL, and the fedora user streams application has a basic outline at this stage
18:07:52 <relrod> nirik: do we want to talk about the status app here or save it for later in the meeting?
18:07:57 <relrod> we were going to revisit it this week
18:08:05 <nirik> relrod: oh yeah... either way. we can talk about it now...
18:09:01 <nirik> So, background is:
18:09:32 <nirik> we have http://status.fedoraproject.org/ setup. This is intended to be a place people can see if there are outages or issues...
18:09:50 <nirik> it's running in openshift, and does not depend on our infra for anything.
18:10:00 <relrod> sorry about that, had to answer the door, back now
18:10:09 <nirik> do folks want to announce and start using this? or are their concerns/issues/outstanding?
18:10:27 <smooge> 1) how is it used. 2) where is it documented?
18:10:30 <nirik> I used it the other day for the hosted outage... no idea if people noticed. (but of course we have not yet announced it)
18:10:47 <skvidal> and it is documented in infradocs, iirc
18:10:50 <nirik> When there is an outage, it's manually updated by someone with info about the outage.
18:10:54 <relrod> smooge: http://infrastructure.fedoraproject.org/infra/docs/status-fedora.txt
18:11:02 <nirik> and yeah, ^
18:11:33 <nirik> we need to make sure ssh keys are syncing, I don't know that that is enabled yet.
18:12:00 * lmacken noticed, thanks to the favicon :)
18:12:09 <nirik> the rss feed is also nice.
18:12:27 <nirik> you can see when an outage started and ended (or at least when an admin noted it/removed it)
18:12:52 <smooge> thanks.. the next question.. who can make changes? sysadmin-main, sysadmin-?
18:12:59 <nirik> http://status.fedoraproject.org/changes.rss
18:13:18 <nirik> if/when the sync script is working, it should be sysadmin-main + sysadmin-noc
18:13:31 <nirik> we could change that if folks prefer...
18:13:46 <smooge> no it sounds right.. I just wanted to add it to the documentation :)
18:14:05 <nirik> yeah, good idea.
18:14:16 <abadger1999> We should "link" together all the SOPs that go into making an outage.
18:14:25 <nirik> abadger1999: also a good idea.
18:14:46 <abadger1999> there's topic in irc, status app, outage ticket.  planned outages: nagios, mail to list.
18:15:06 <nirik> yep.
18:15:41 <abadger1999> app itself sounds good -- but some way to remember to keep it and everything else accurate is what I think we currently lack.
18:15:46 <nirik> also for outages, it might be helpfull if the first responder handles the actual issue and gets others to handle the paperwork stuff... but I guess that depends on others being there to do so.
18:16:05 <nirik> we have an outage sop
18:16:06 <abadger1999> <nod>
18:16:12 <nirik> it likely needs updating.
18:16:16 <nirik> any takers? ;)
18:16:39 <iambryan> I can take a look at that
18:17:16 <abadger1999> <nod>  Yeah -it currently has a busted hyperlink to the outage template and procedure for ticket and email.
18:17:25 <relrod> abadger1999: yeah, we have a nagios check when a thing is marked as down - it will alert us so we remember to set it back when the service comes back. Nothing in place to remind people to actually set things as down though
18:17:41 <nirik> iambryan: that would be great. ;) please do ask lots of questions and we can get you access to update the doc. ;)
18:18:03 <iambryan> Great, already a member with docs if that helps
18:18:26 <nirik> docs always help. Just keeping them up to date can be a chore. ;)
18:19:07 <nirik> #action make sure everything is set with status (ssh key sync and docs) and then announce/use.
18:19:16 <nirik> #action iambryan to help update outage docs.
18:19:25 <nirik> Any other applications news?
18:19:27 <threebean> Lots of stuff happening here.  The main thing is a new release of packages and tagger in stg that contains some of the features of pkgdb migrated over.  A more complete list of other stuff is here -> http://fpaste.org/OQPD/
18:19:59 <threebean> Could use an extra pair of eyes over packages and tagger to make sure stuff's busted.
18:20:10 <threebean> er.. make sure its not busted.  ;)
18:20:15 <abadger1999> :-)
18:20:31 <nirik> cool.
18:21:04 <threebean> Oh, question.. (I should know this, but) when is the freeze over?
18:21:18 <nirik> as soon as alpha goes out... which is sadly now a week later. ;(
18:21:33 <nirik> oh, another application item to note:
18:21:34 <threebean> ah, I saw that.  Ok. I'll keep an eye on announce and -devel.
18:21:47 <nirik> #info testing python-bugzilla update on app01.stg
18:22:07 <nirik> we should try and test anything else we can think of in stg for the new python-bugzilla so we can update prod at some point.
18:22:22 <nirik> #topic Sysadmin status / discussion
18:22:36 <nirik> any sysadmin side things this week or pending?
18:22:53 <nirik> skvidal has been working on our cloud... it even had an instance work on it this morning. ;)
18:23:01 <skvidal> they are working now
18:23:01 <relrod> still slowly making progress on IMM password changes
18:23:02 <threebean> cool
18:23:06 <skvidal> and I have f16 images
18:23:09 <WolfKit> is this the part where I talk about 2fa?
18:23:11 <WolfKit> :)
18:23:17 <nirik> hosted03 is dead. Long live hosted01/02. ;)
18:23:19 <skvidal> so we're not running those awful centos/ubuntu monsters
18:23:30 <abadger1999> nirik: Re: python-bugzilla -- is RH IT tracking whether they see an increase in queries?
18:23:47 <nirik> abadger1999: not that I know of. we can ping them? would you like to do that? or would you like me to?
18:24:02 <nirik> #info IMM password changes ongoing
18:24:15 <abadger1999> nirik: I'm not really sure who to contact there now that dkl has left.
18:24:16 <nirik> #info euca cloud has f16 images working now.
18:24:24 <nirik> WolfKit: sure.
18:24:46 <nirik> abadger1999: kbaker? (who is in -noc actually)
18:24:48 <WolfKit> well.  I had some code changes that made pam_url stop segfaulting on 64-bit hosts
18:25:12 <WolfKit> of course, I went to spin up the VM so I could give a live demo.. and what do you think happened.. it's *gone* :/
18:25:24 <nirik> :(
18:25:26 <WolfKit> awilcox on ind-wkst041 virt-machines: ls -al
18:25:30 <WolfKit> awilcox on ind-wkst041 virt-machines:
18:25:32 <abadger1999> nirik: Cool.  I'll ping him to see -- since we also have things that run in production that don't run in stg probably need to also alert him when we turn on production.
18:25:44 <nirik> abadger1999: thanks.
18:25:48 <WolfKit> I have no idea how this happened.  running a disk recovery software on it now..
18:25:53 <nirik> WolfKit: so, you get to fix the bug a second time? :(
18:26:04 <WolfKit> nirik: it seems like it.  at least I know it's fixable though
18:27:05 <nirik> yeah, hopefully it will look familliar.
18:27:44 <nirik> ok, any other sysadmin stuff?
18:27:53 <nirik> #topic F18 Alpha Release tickets/Freeze
18:28:08 <nirik> as noted, alpha has slipped a week, so our freeze continues an extra week.
18:28:23 <nirik> not sure we need to go over alpha tickets, since it's a ways away now.
18:28:51 <smooge> what is the ticket that tracks tickets for it?
18:29:26 <nirik> we have:
18:29:28 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3427
18:29:30 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3428
18:29:32 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3429
18:30:17 <nirik> #topic Upcoming Tasks/Items
18:30:30 <nirik> here comes the info dump of upcoming items:
18:30:34 <nirik> #info 2012-08-14 to 2012-09-04 F18 Alpha Freeze
18:30:34 <nirik> #info 2012-08-31 end of 2nd quarter
18:30:34 <nirik> #info 2012-09-04 F18 Alpha release.
18:30:34 <nirik> #info 2012-09-10 to 2012-09-24 Skvidal is out
18:30:34 <nirik> #info 2012-09-25 to 2012-10-09 F18 Beta Freeze
18:30:35 <nirik> #info 2012-09-26 to 2012-09-28 Puppetconf
18:30:37 <nirik> #info 2012-10-09 F18 Beta release
18:30:39 <nirik> #info 2012-10-23 to 2012-11-13 F18 Final Freeze
18:30:41 <nirik> #info 2012-11-13 F18 release.
18:30:43 <nirik> #info 2012-11-26 to 2012-11-29 Security FAD
18:30:45 <nirik> #info 2012-11-30 end of 3nd quarter
18:30:51 <nirik> anything anyone wants to note or schedule or amend?
18:31:42 <nirik> #topic Open Floor
18:31:51 <nirik> any general items for open floor or new topics?
18:33:26 * nirik listens to crickets chirp.
18:33:29 <smooge> not from me
18:33:35 <smooge> working on my talk..
18:33:36 <relrod> I start school next week, not sure what my availability will be like yet, from here on out until the end of the semester
18:33:46 <relrod> hopefully not too bad though
18:33:47 <smooge> well school comes first
18:33:52 <smooge> then this fedora thing
18:33:57 <relrod> smooge: ;)
18:33:57 <nirik> yeah. :)
18:34:22 <nirik> I need to work on finalizing the FAD stuff... see if we can meet budget and reserve things, etc... will try and do that more soon.
18:34:43 <relrod> sounds fun, I look forward to that
18:35:29 <nirik> ok, thanks for coming everyone... lets get back to it in #fedora-admin / noc / apps
18:35:30 <smooge> nirik, let me know what I can do to help
18:35:34 <nirik> #endmeeting