19:00:01 #startmeeting Infrastructure (2011-11-10) 19:00:01 Meeting started Thu Nov 10 19:00:01 2011 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:01 Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:01 #meetingname infrastructure 19:00:01 #topic Robot Roll Call 19:00:01 #chair smooge skvidal Codeblock ricky nirik abadger1999 lmacken dgilmore 19:00:01 The meeting name has been set to 'infrastructure' 19:00:01 Current chairs: Codeblock abadger1999 dgilmore lmacken nirik ricky skvidal smooge 19:00:08 * abadger1999 here 19:00:12 * lmacken 19:00:18 Here 19:00:29 \me here 19:00:30 Here. And a new face. 19:00:33 * nirik waves at everyone. 19:00:35 * pingou 19:00:39 hello :) 19:00:51 * adimania is here 19:01:23 hola 19:01:31 hello everyone 19:01:38 * nirik will wait a minute or two for folks to wander in. 19:03:20 ok, I guess lets go ahead and dive on in. ;) 19:03:23 #topic New folks introductions and apprentice tasks/feedback 19:03:42 welcome to new folks. ;) Could you introduce yourselves and what kind of things you are looking to work on? 19:04:00 and if there are any apprentices who want to talk about tickets or tasks, chime right in. ;) 19:04:19 ! 19:04:28 Should I go first? 19:04:39 I'm a sysadmin with a startup that hosts a SaaS app. Would like to work on anything from app dev to provisioning and maintaining boxes. 19:04:43 here 19:04:46 ish 19:05:36 I'm currently as SysAdmin on RedHat and AIX, and I'm interested in learning more about the larger sysadmin world and more about Linux in general 19:06:14 My name is Aditya. I am from India and currently working as system admin. I have some experience with puppet and setting up Koji and can do some python scripting as well. 19:06:26 p3ll0n: What programming languages do you know? 19:06:29 awesome. Welcome all. 19:07:08 abadger1999: Python, C/C++ very well, Ruby/Perl decent knowledge 19:07:14 Excellent. 19:07:52 p3ll0n, adimania: If either of you want to work on developing our web apps, I'll get you started sometime. 19:07:53 please do take a look at https://fedoraproject.org/wiki/Infrastructure_Apprentice and http://fedoraproject.org/wiki/Infrastructure/GettingStarted and do hang out in #fedora-admin and #fedora-noc... do chime in there with questions or asking about tickets or the like. Folks who would like to be apprentices can see me over there after the meeting. 19:08:29 Nathaniel McCallum here... I'm hoping to pimp a bof at FUDCon on krb5 and OTP authentication :) 19:08:43 nirik: will do thanks! 19:09:13 npmccallum: looking forward to it. Will see if it can meet our needs. 19:09:25 abadger1999, I am more into admin stuff like puppet than development. So I would prefer something on that line. 19:09:44 adimania: No problem. We have plenty of that work too :-) 19:10:00 nirik, will catch you for sure. 19:10:05 * LoKoMurdoK here 19:10:28 great. 19:10:47 * adimania was directed to https://fedorahosted.org/fedora-infrastructure/report/14 by ricky 19:11:02 ok, look forward to seeing folks in #fedora-admin/#fedora-noc. Don't feel shy about asking questions in those channels... 19:11:14 adimania: yes indeed. 19:11:20 * nirik just added a new one before the meeting. 19:11:45 ok, any other new folks or apprentice questions? shall we move on to the next topic? 19:12:32 #topic Release retrospective 19:12:41 so, tuesday was the Fedora 16 release day. 19:12:53 How did everyone feel it went? anything we should change or improve? 19:13:53 The main thing I was thinking is that we could silence some specific nagios alerts before hand... ie, process limits on download servers always alert on release day. 19:14:44 doh 19:14:47 DST switch got me 19:14:54 mdomsch: happens to us all. ;) 19:15:17 we had a remark on the little amount of seeder for some spins, don't know if we can help on this though 19:15:28 nirik: we'd almost want a "if processes fall below this limit" check on release day. 19:15:46 pingou: yeah, I was seeding, and a few others, but perhaps we could coordinate it better. 19:16:01 abadger1999: yeah. 19:16:19 nagios: take this check and reverse it. :) 19:16:53 p3ll0n: I'm always looking for people to help maintain MirrorManager, written mostly in python 19:16:58 pingou: perhaps next cycle we could post to the test list when torrents were made and anyone with the last rc or otherwise had content could seed. 19:17:31 nirik: that would be a way for sure 19:17:33 nirik: there was a request from mirrors to get the torrent files into the master mirror tree, rather than posted separately 19:17:56 mdomsch: hum. I suppose thats a thought. Although this is getting a bit more rel-engy. ;) 19:18:01 yep 19:18:12 mdomsch: Okay cool, let's connect after the meeting. 19:18:34 ok, if nothing more on release, will move on to the next topic. 19:19:01 #topic Fudcon Hackfests/Workshops 19:19:11 OTP! 19:19:15 So, I started a thread on this on the infrastructure list... 19:19:25 download* boxes saw a small spike in traffic on Tues and Wed, but nothing they couldn't handle 19:19:47 I was going to go add the ones suggested to the wiki... did anyone have other ideas? or changes to those? 19:20:12 nirik: MM needs some work, mostly in the TG2 migration 19:20:45 how many folks here are going to make it to the next north american fudcon out of curiosity? 19:20:54 could be broken up and lend itself to a hackfest 19:20:56 https://fedoraproject.org/wiki/FUDCon:Blacksburg_2012 19:20:56 * abadger1999 raises his hand 19:21:05 * mdomsch is planning on attending 19:21:11 * nirik should be there. 19:21:11 * npmccallum will be there 19:21:26 * lmacken too 19:21:40 * dgilmore should be there 19:21:59 cool. Hopefully lots of folks can make it... 19:22:22 mdomsch: I'll try to get a bit more done on the model this week-end, no hard promise though 19:22:26 #action nirik to add workshops/hackfests to the wiki. 19:22:41 if anyone thinks of other good ones they want to run or the like, do add them. 19:22:44 pingou: no worries - I haven't had any time for it lately either 19:22:59 mdomsch: too many things in parallel :) 19:23:47 #topic serverbeach machine changes 19:23:55 So, I thought I would give an update on this. 19:24:09 we currently have 5 machines at serverbeach, that we got many years ago. 19:24:18 We are getting 5 new machines to replace them. 19:24:38 wheee 19:24:38 this gives us a chance to migrate things to rhel6 and redo stuff. 19:24:43 The deadline for migration is 2011-12-11 19:25:03 * adimania claps 19:25:06 nirik: is this set by them? 19:25:06 So, we need to move pretty fast on this stuff. 19:25:14 dgilmore: yeah, they are giving us a month to move. 19:25:19 nirik: ok 19:25:21 we could possibly ask for more, but I think we can do it. 19:25:34 biggest effected things will be mailing lists, and fedorahosted 19:25:40 yep. 19:26:00 so, I'm thinking: 19:26:28 serverbeach1 -> goes away, and we replace it with a download-ibiblio01 machine instance at ibiblio. 19:26:28 thanksgiving break move? 19:26:45 that sounds good 19:27:03 * dgilmore is taking off the whole week of thanksgiving 19:27:13 We have space there, and that leaves us more room at sb for hosted stuff. 19:27:17 not that me being away should change anything 19:27:46 serverbeach2/3 (collab1/2 and ns1) -> migrate to new serverbeach06/07 19:27:56 and I have been considering the idea of using drbd there. 19:28:03 for collab01/02 19:28:58 ok 19:29:16 serverbeach4/5 (hosted1/2) -> migrate to sb08/09/10. We need to decide if we want to just migrate the pile to rhel6, work on our plans to distribute it more or what. 19:29:35 * nirik is still thinking on that one, but will come up with a plan. 19:30:09 Any questions or thoughts on all that? 19:30:19 that sounds good 19:30:45 ns is going to be the pain 19:30:54 * nirik thinks he will be around thanksgiving, but isn't 100% sure. 19:31:08 smooge: oh yeah, changing ip's on that will be anoying. 19:31:21 It can take a while to get nameservices dealt with 19:31:32 * nirik can try and ask them if we can migrate that ip. 19:32:02 nirik, I doubt it.. and with the problems we have had with special networks I would prefer to just do the change. 19:32:11 yeah, probibly so. 19:32:30 I think the collab migration should be pretty simple. 19:32:42 hosted could be more difficult depending on how we do it. 19:32:47 6 19:32:52 oop 19:33:24 #action nirik to come up with a concrete plan for hosted by next week. 19:33:39 ok, moving along... 19:33:46 #topic Upcoming Tasks/Items 19:34:15 flood: 19:34:17 #info 2011-11-10 infra meeting. 19:34:17 #info 2011-11-11 kevin out on PTO 19:34:17 #info 2011-11-14 - send out nag about pass/keys change 19:34:17 #info 2011-11-17 infra meeting. 19:34:17 #info 2011-11-21 - send out nag about pass/keys change 19:34:18 #info 2011-11-24 infra meeting. 19:34:20 #info 2011-11-28 - send out nag about pass/keys change. 19:34:23 #info 2011-11-30 Flag day for password reset/new ssh keys. 19:34:24 #info 2011-12-01 nag fi-apprentice folks for december 19:34:26 #info 2011-12-11 Serverbeach migration deadline. 19:34:30 #info 2011-12-26 to 2011-12-30 - rh shutdown week. 19:34:39 any other stuff coming up that folks would like to note? 19:34:59 I will be out tomorrow. ;) 19:35:09 Might be checking in from time to time, but hopefully everything is quiet. 19:36:02 2011-11-17 final fas upgrade 19:36:03 At some point we need to get going on migrating ibiblio01 -> ibiblio02 guests. 19:36:13 abadger1999: cool. Adding. 19:36:34 We've already upgraded to the beta -- final will just bring a few small features (ssh key clearing) and translations. 19:37:06 excellent work on that abadger1999. :) 19:37:17 very nice to close all those hotfix tickets and get the new captcha. 19:37:31 others did most of the work :-), I just put it together for release. 19:38:20 yeah. ;) Kudos to all those folks as well. 19:38:27 #topic Meeting time 19:38:43 Do we want to stick with 19UTC ? or move due to no more daylight savings time? 19:39:23 * dgilmore is down with whatever ;) 19:39:47 * abadger1999 can do either... the meeting is nicely in the middle of his day 19:40:01 * nirik is also fine with whatever. ;) 19:40:10 * adimania later than 19UTC would be a big bummer for me. 19:40:21 likes 1900 UTC 19:41:05 ok, lets keep it then... and see if anyone complains? 19:41:10 * pingou fine for both theoratycally 19:41:16 * nirik notes skvidal isn't here, he may have a conflict. 19:41:42 * mdomsch is rarely here, but the 2pm US time (DST or not) works better for me. 19:41:52 don't plan anything around me though 19:42:10 how does earlier look? 19:42:12 This time works well for me 19:42:46 today's time slot works for me only because another standing meeting got cancelled 19:43:01 the irc-support-sig is at 17utc. 19:43:14 we could do 18, but that would be a bit back to back 19:43:43 yeah 19:44:00 I would like to make it good for EU people to stay with us. 19:44:10 * nirik nods. 19:44:13 * StylusEater is a dolt and forgot about the time switch 19:44:48 #action will stay at 19UTC for now, objections/other plans to the list. 19:44:53 #topic Open Floor 19:45:00 anyone have any topics for open floor? 19:45:47 mediawiki looks like it will have 118 out soon which means our 116 will be EOL I think 19:46:05 I need to make new sets of packages for review 19:46:12 fun. 19:46:22 we can migrate when we get rhel6 app servers. 19:47:03 mdomsch: is the mirror manager trac ticket page up to date? 19:47:05 cool. I am dropping EL4 support for mediawiki.. and will need to look at EL5 support changes anyway as the PHP it wants is newq 19:48:39 any other items for open floor? 19:49:36 StylusEater: mostly. ticket for TG2 migration would be nice 19:49:40 * nirik will close things out in a minute if nothing else comes up. 19:50:03 mdomsch: once I get more changes in for abadger1999 I'll take a look at mirror manager 19:50:16 I don't have anything for open floor. 19:50:34 * dgilmore has nothing 19:51:04 ok, thanks for coming everyone! 19:51:08 #endmeeting