18:00:08 <nirik> #startmeeting Infrastructure (2012-08-23) 18:00:08 <zodbot> Meeting started Thu Aug 23 18:00:08 2012 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:08 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:08 <nirik> #meetingname infrastructure 18:00:08 <zodbot> The meeting name has been set to 'infrastructure' 18:00:08 <nirik> #topic Greets! 18:00:08 <nirik> #chair smooge skvidal CodeBlock ricky nirik abadger1999 lmacken dgilmore mdomsch threebean 18:00:08 <zodbot> Current chairs: CodeBlock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge threebean 18:00:15 * abadger1999 here 18:00:16 * skvidal is here 18:00:21 * Betapanda is here 18:00:31 <WolfKit> excuse me. sillyness from another channel 18:00:32 * WolfKit is here 18:00:33 <iambryan> /me is here 18:00:35 * lmacken 18:00:46 * threebean 18:00:50 * relrod here 18:01:25 * pingou is here 18:01:31 <nirik> ok, lets go ahead and dive on in... 18:01:40 <nirik> #topic New folks introductions and Apprentice tasks. 18:01:45 <nirik> If any new folks want to give a quick one line bio or any apprentices would like to ask general questions, they can do so in this part of the meeting. Don't be shy 18:02:19 <smooge> here 18:02:40 * rossdylan is here 18:02:53 <nirik> any new folks want to say hi? or shall we move on... 18:03:02 <iambryan> Hey all, I am a fedora user for the past 9 years and have admin'd RHEL boxes with NOC infra management apps 18:03:36 <iambryan> I would like to help out on the main admin side and learn some python while I'm at it 18:03:46 <pingou> welcome :) 18:03:49 <ctria> hi all 18:03:54 <ctria> sorry for being late 18:03:55 <iambryan> Thanks 18:03:57 <nirik> welcome iambryan. :) We can point you to some things to look at after the meeting over in #fedora-admin 18:04:12 <nirik> no worries ctria. 18:04:16 <nirik> #topic Applications status / discussion 18:04:16 <ctria> hm i'm on time! i'm back after some months away 18:04:23 <iambryan> Great, will stick around then :) 18:04:57 <nirik> Any applications news this week or upcoming? abadger1999 / relrod / threebean / lmacken / pingou / anyone I missed? 18:05:18 <pingou> I'm slowly but surely trying to move FAS's db model to full SA 18:05:39 <pingou> eof for me 18:06:03 <nirik> pingou: cool. so thats in prep for moving it to TG2? or ? 18:06:07 <relrod> Just pushed another Flask build to el6 testing, to fix the broken deps warnings on the doc subpackage. So please continue testing flask from epel, poke me/file bugs on it if it does anything insane 18:06:25 <nirik> relrod: any news on the plugins issues tflink was hitting? 18:06:38 <relrod> nirik: Haven't had a chance to talk with him much yet, no :( 18:06:43 <pingou> nirik: I think that will help whatever framework we choose (if it uses SA) 18:07:07 <relrod> nirik: will try and catch up with him today if he's around 18:07:12 <nirik> cool. 18:07:38 <rossdylan> I need to look over fedbadges once more and make sure evrything works in EPEL, and the fedora user streams application has a basic outline at this stage 18:07:52 <relrod> nirik: do we want to talk about the status app here or save it for later in the meeting? 18:07:57 <relrod> we were going to revisit it this week 18:08:05 <nirik> relrod: oh yeah... either way. we can talk about it now... 18:09:01 <nirik> So, background is: 18:09:32 <nirik> we have http://status.fedoraproject.org/ setup. This is intended to be a place people can see if there are outages or issues... 18:09:50 <nirik> it's running in openshift, and does not depend on our infra for anything. 18:10:00 <relrod> sorry about that, had to answer the door, back now 18:10:09 <nirik> do folks want to announce and start using this? or are their concerns/issues/outstanding? 18:10:27 <smooge> 1) how is it used. 2) where is it documented? 18:10:30 <nirik> I used it the other day for the hosted outage... no idea if people noticed. (but of course we have not yet announced it) 18:10:47 <skvidal> and it is documented in infradocs, iirc 18:10:50 <nirik> When there is an outage, it's manually updated by someone with info about the outage. 18:10:54 <relrod> smooge: http://infrastructure.fedoraproject.org/infra/docs/status-fedora.txt 18:11:02 <nirik> and yeah, ^ 18:11:33 <nirik> we need to make sure ssh keys are syncing, I don't know that that is enabled yet. 18:12:00 * lmacken noticed, thanks to the favicon :) 18:12:09 <nirik> the rss feed is also nice. 18:12:27 <nirik> you can see when an outage started and ended (or at least when an admin noted it/removed it) 18:12:52 <smooge> thanks.. the next question.. who can make changes? sysadmin-main, sysadmin-? 18:12:59 <nirik> http://status.fedoraproject.org/changes.rss 18:13:18 <nirik> if/when the sync script is working, it should be sysadmin-main + sysadmin-noc 18:13:31 <nirik> we could change that if folks prefer... 18:13:46 <smooge> no it sounds right.. I just wanted to add it to the documentation :) 18:14:05 <nirik> yeah, good idea. 18:14:16 <abadger1999> We should "link" together all the SOPs that go into making an outage. 18:14:25 <nirik> abadger1999: also a good idea. 18:14:46 <abadger1999> there's topic in irc, status app, outage ticket. planned outages: nagios, mail to list. 18:15:06 <nirik> yep. 18:15:41 <abadger1999> app itself sounds good -- but some way to remember to keep it and everything else accurate is what I think we currently lack. 18:15:46 <nirik> also for outages, it might be helpfull if the first responder handles the actual issue and gets others to handle the paperwork stuff... but I guess that depends on others being there to do so. 18:16:05 <nirik> we have an outage sop 18:16:06 <abadger1999> <nod> 18:16:12 <nirik> it likely needs updating. 18:16:16 <nirik> any takers? ;) 18:16:39 <iambryan> I can take a look at that 18:17:16 <abadger1999> <nod> Yeah -it currently has a busted hyperlink to the outage template and procedure for ticket and email. 18:17:25 <relrod> abadger1999: yeah, we have a nagios check when a thing is marked as down - it will alert us so we remember to set it back when the service comes back. Nothing in place to remind people to actually set things as down though 18:17:41 <nirik> iambryan: that would be great. ;) please do ask lots of questions and we can get you access to update the doc. ;) 18:18:03 <iambryan> Great, already a member with docs if that helps 18:18:26 <nirik> docs always help. Just keeping them up to date can be a chore. ;) 18:19:07 <nirik> #action make sure everything is set with status (ssh key sync and docs) and then announce/use. 18:19:16 <nirik> #action iambryan to help update outage docs. 18:19:25 <nirik> Any other applications news? 18:19:27 <threebean> Lots of stuff happening here. The main thing is a new release of packages and tagger in stg that contains some of the features of pkgdb migrated over. A more complete list of other stuff is here -> http://fpaste.org/OQPD/ 18:19:59 <threebean> Could use an extra pair of eyes over packages and tagger to make sure stuff's busted. 18:20:10 <threebean> er.. make sure its not busted. ;) 18:20:15 <abadger1999> :-) 18:20:31 <nirik> cool. 18:21:04 <threebean> Oh, question.. (I should know this, but) when is the freeze over? 18:21:18 <nirik> as soon as alpha goes out... which is sadly now a week later. ;( 18:21:33 <nirik> oh, another application item to note: 18:21:34 <threebean> ah, I saw that. Ok. I'll keep an eye on announce and -devel. 18:21:47 <nirik> #info testing python-bugzilla update on app01.stg 18:22:07 <nirik> we should try and test anything else we can think of in stg for the new python-bugzilla so we can update prod at some point. 18:22:22 <nirik> #topic Sysadmin status / discussion 18:22:36 <nirik> any sysadmin side things this week or pending? 18:22:53 <nirik> skvidal has been working on our cloud... it even had an instance work on it this morning. ;) 18:23:01 <skvidal> they are working now 18:23:01 <relrod> still slowly making progress on IMM password changes 18:23:02 <threebean> cool 18:23:06 <skvidal> and I have f16 images 18:23:09 <WolfKit> is this the part where I talk about 2fa? 18:23:11 <WolfKit> :) 18:23:17 <nirik> hosted03 is dead. Long live hosted01/02. ;) 18:23:19 <skvidal> so we're not running those awful centos/ubuntu monsters 18:23:30 <abadger1999> nirik: Re: python-bugzilla -- is RH IT tracking whether they see an increase in queries? 18:23:47 <nirik> abadger1999: not that I know of. we can ping them? would you like to do that? or would you like me to? 18:24:02 <nirik> #info IMM password changes ongoing 18:24:15 <abadger1999> nirik: I'm not really sure who to contact there now that dkl has left. 18:24:16 <nirik> #info euca cloud has f16 images working now. 18:24:24 <nirik> WolfKit: sure. 18:24:46 <nirik> abadger1999: kbaker? (who is in -noc actually) 18:24:48 <WolfKit> well. I had some code changes that made pam_url stop segfaulting on 64-bit hosts 18:25:12 <WolfKit> of course, I went to spin up the VM so I could give a live demo.. and what do you think happened.. it's *gone* :/ 18:25:24 <nirik> :( 18:25:26 <WolfKit> awilcox on ind-wkst041 virt-machines: ls -al 18:25:30 <WolfKit> awilcox on ind-wkst041 virt-machines: 18:25:32 <abadger1999> nirik: Cool. I'll ping him to see -- since we also have things that run in production that don't run in stg probably need to also alert him when we turn on production. 18:25:44 <nirik> abadger1999: thanks. 18:25:48 <WolfKit> I have no idea how this happened. running a disk recovery software on it now.. 18:25:53 <nirik> WolfKit: so, you get to fix the bug a second time? :( 18:26:04 <WolfKit> nirik: it seems like it. at least I know it's fixable though 18:27:05 <nirik> yeah, hopefully it will look familliar. 18:27:44 <nirik> ok, any other sysadmin stuff? 18:27:53 <nirik> #topic F18 Alpha Release tickets/Freeze 18:28:08 <nirik> as noted, alpha has slipped a week, so our freeze continues an extra week. 18:28:23 <nirik> not sure we need to go over alpha tickets, since it's a ways away now. 18:28:51 <smooge> what is the ticket that tracks tickets for it? 18:29:26 <nirik> we have: 18:29:28 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3427 18:29:30 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3428 18:29:32 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3429 18:30:17 <nirik> #topic Upcoming Tasks/Items 18:30:30 <nirik> here comes the info dump of upcoming items: 18:30:34 <nirik> #info 2012-08-14 to 2012-09-04 F18 Alpha Freeze 18:30:34 <nirik> #info 2012-08-31 end of 2nd quarter 18:30:34 <nirik> #info 2012-09-04 F18 Alpha release. 18:30:34 <nirik> #info 2012-09-10 to 2012-09-24 Skvidal is out 18:30:34 <nirik> #info 2012-09-25 to 2012-10-09 F18 Beta Freeze 18:30:35 <nirik> #info 2012-09-26 to 2012-09-28 Puppetconf 18:30:37 <nirik> #info 2012-10-09 F18 Beta release 18:30:39 <nirik> #info 2012-10-23 to 2012-11-13 F18 Final Freeze 18:30:41 <nirik> #info 2012-11-13 F18 release. 18:30:43 <nirik> #info 2012-11-26 to 2012-11-29 Security FAD 18:30:45 <nirik> #info 2012-11-30 end of 3nd quarter 18:30:51 <nirik> anything anyone wants to note or schedule or amend? 18:31:42 <nirik> #topic Open Floor 18:31:51 <nirik> any general items for open floor or new topics? 18:33:26 * nirik listens to crickets chirp. 18:33:29 <smooge> not from me 18:33:35 <smooge> working on my talk.. 18:33:36 <relrod> I start school next week, not sure what my availability will be like yet, from here on out until the end of the semester 18:33:46 <relrod> hopefully not too bad though 18:33:47 <smooge> well school comes first 18:33:52 <smooge> then this fedora thing 18:33:57 <relrod> smooge: ;) 18:33:57 <nirik> yeah. :) 18:34:22 <nirik> I need to work on finalizing the FAD stuff... see if we can meet budget and reserve things, etc... will try and do that more soon. 18:34:43 <relrod> sounds fun, I look forward to that 18:35:29 <nirik> ok, thanks for coming everyone... lets get back to it in #fedora-admin / noc / apps 18:35:30 <smooge> nirik, let me know what I can do to help 18:35:34 <nirik> #endmeeting