20:00:02 <nirik> #startmeeting Infrastructure (2012-03-22)
20:00:02 <zodbot> Meeting started Thu Mar 22 20:00:02 2012 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:02 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:03 <nirik> #meetingname infrastructure
20:00:03 <nirik> #topic Robot Roll Call
20:00:03 <nirik> #chair smooge skvidal Codeblock ricky nirik abadger1999 lmacken dgilmore mdomsch
20:00:03 <zodbot> The meeting name has been set to 'infrastructure'
20:00:03 <zodbot> Current chairs: Codeblock abadger1999 dgilmore lmacken mdomsch nirik ricky skvidal smooge
20:00:06 * skvidal is here
20:00:08 <CodeBlock> here
20:00:09 <nirik> who all is around for a infra meeting?
20:00:09 * lmacken 
20:00:16 <smooge> I am here
20:00:24 <CodeBlock> hating on openshift and etherpad. but here.
20:00:24 * pingou not there
20:00:40 <skvidal> CodeBlock: talk to me in #noc
20:00:52 <dgilmore> buenos tadres amigos
20:01:02 <nirik> pingou: does that mean you are here if you aren't there? ;)
20:01:07 <davevz> here
20:01:47 <marcelk1985> hey
20:01:50 <marcelk1985> i am here
20:01:54 * abadger1999 here
20:02:10 <nirik> welcome everyone. Lets go ahead and get started...
20:02:14 <pingou> nirik: I'm here but I won't be shortly :)
20:02:18 <nirik> #topic New folks introductions and Apprentice tasks.
20:02:18 <nirik> If any new folks want to give a quick one line bio or any apprentices
20:02:18 <nirik> would like to ask general questions, they can do so now. Anyone?
20:03:24 <marcelk1985> I have the feeling that almost all the easyfix tickets are coding related :-( Am i looking in the wrong place?
20:03:36 <marcelk1985> Or am I just out of luck
20:03:42 <nirik> marcelk1985: they shouldn't be...
20:03:48 <nirik> but I can try and file a few more...
20:04:08 <nirik> https://fedorahosted.org/fedora-infrastructure/report/14 should be the sysadmin type ones.
20:04:25 <nirik> note that when they are assigned, sometimes that person has moved on, and it's not really getting worked on.
20:04:30 <pingou> the easyfix page shows them all :)
20:04:39 <marcelk1985> And I had a link regarding ssh access and that kind of starter info, which I seem to have lost
20:04:41 <nirik> pingou: yeah. link?
20:04:58 <nirik> marcelk1985: should be off the apprentice page: https://fedoraproject.org/wiki/Infrastructure_Apprentice
20:05:37 <marcelk1985> cool
20:05:38 <nirik> marcelk1985: see me in admin after the meeting and I can answer any of those getting started type questions. ;)
20:05:58 <davevz> I've been looking into a ticket for a day or two (3094) is it possible to get it assigned to me? Or should I just comment when I have something ready
20:06:05 <skvidal> yes
20:06:06 <skvidal> that
20:06:09 <skvidal> comment when ready
20:06:13 <skvidal> .ticket 3094
20:06:15 <zodbot> skvidal: #3094 (Assigning a default acl to the pkgs repo) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/3094
20:06:53 <davevz> right thanks
20:07:16 <nirik> I'd ping abadger1999 on questions on that... or ask in ticket.
20:07:29 <davevz> will do
20:07:47 <nirik> cool.
20:08:00 <abadger1999> <nod>
20:08:00 <nirik> any other questions on apprentice tickets or new folks? if not, will move on...
20:08:27 <nirik> #topic two factor auth status
20:08:30 <nirik> Any news here?
20:08:37 * nirik looks to see who was doing what
20:09:04 <nirik> wolfkit was going to look at pam_url...
20:09:07 <wolfkit> unfortunately I haven't finished hacking on pam_url yet
20:09:23 <nirik> wolfkit: ok. No worries. Running into blockers? or just haven't had time...
20:09:25 <wolfkit> I am planning to try to hack it all this weekend
20:09:31 <skvidal> wolfkit: cool
20:09:34 <skvidal> wolfkit: thanks, btw
20:09:38 <skvidal> also a bit more info
20:09:46 <skvidal> totp.cgi that icon wrote
20:09:51 <skvidal> he's having some people look it over, too
20:09:54 <wolfkit> no blockers, the code seems pretty sane, but been under a crunch with other things
20:09:59 <skvidal> dotted t's and crossed i's and all that
20:10:08 <nirik> excellent.
20:10:12 <skvidal> wolfkit: the biggest concerns about the code are
20:10:22 <skvidal> 1. some of the functions used are 'odd'
20:10:26 <skvidal> 2. memory leaks
20:10:38 <skvidal> 3. a bit of how the user/password are submitted
20:10:40 <wolfkit> yeah
20:10:50 <skvidal> if the first 2 are solvable, easily
20:10:55 <skvidal> that would be a big win
20:11:24 * skvidal is done
20:11:32 * nirik nods.
20:11:36 <nirik> Sounds great.
20:11:42 <nirik> ok, moving along then...
20:11:44 <wolfkit> will try to get things done this weekend and talk in -admin with questions/concerns
20:11:56 <nirik> #topic Staging re-work status
20:12:07 <nirik> I want to wait until after the freeze to finish this up...
20:12:16 <marcelk1985> okay
20:12:22 <nirik> I need to document things and then switch over all the staging boxes.
20:12:36 <nirik> more info as we get closer to that.
20:12:47 <skvidal> wolfkit: thanks
20:13:02 <nirik> #info will work on finishing this after the beta freeze.
20:13:12 <nirik> #topic Applications status / discussion
20:13:27 <nirik> lmacken / abadger1999 / pingou / threebean: any application side news or info?
20:13:48 <nirik> I'd like to note that we have darkserver deployed now.
20:14:07 <smooge> yay
20:14:10 <abadger1999> not really -- back from pycon where I worked on mailman3 stuff -- that'll all result in long term stuff but nothing immediately.
20:14:11 <nirik> https://fedoraproject.org/wiki/Features/Darkserver and https://fedoraproject.org/wiki/Darkserver
20:14:38 <abadger1999> threebean had a messaging meeting earlier this week and it looks to be shaping up.
20:15:32 <nirik> I may decommission darkserver01.dev and darkserver01.stg... there shouldn't be much development moving forward and our stg env isn't good enough to really test it there.
20:15:44 <nirik> #info darkserver is deployed in production now.
20:15:58 <abadger1999> I'd like to see API docs/simple example on that and maybe a notification server at some point... the latter would let us start pulling various notification (via email) things out of the individual web apps at the same time as we put in the event hooks.
20:16:03 <nirik> #info messaging work is moving forward, there was a meeting on it earlier this week.
20:17:00 <nirik> #info logs from the messaging sig meeting at http://meetbot.fedoraproject.org/meetbot/fedora-meeting/2012-03-20/messaging-sig.2012-03-20-16.15.html
20:17:08 * CodeBlock updates a bit on search: I enabled crawling of fpeople and fhosted the other day, also enabled caching of search results to try to speed it up... But that's still a work in progress. It's slow...very slow, right now.
20:17:11 <nirik> abadger1999: I think having a small demo would be really good.
20:17:31 <abadger1999> <nod>
20:17:56 <abadger1999> I'd like it to be something that we can start deriving immediate value from :-)
20:17:56 <smooge> do we need someone with DB experience to see if its the queries or schema which is slowing it all down?
20:18:29 <nirik> CodeBlock: is it db related do you think? or ?
20:18:53 <nirik> #info dpsearch development instance at https://search-dev.fedoraproject.org/search.cgi
20:19:00 <CodeBlock> smooge: Probably wouldn't hurt to check that. I need to look into it more and ping/email daMaestro (who nirik said offered to help/has used dpsearch before) and possibly poke upstream and ask if they have any ideas
20:19:34 <nirik> #action CodeBlock to work on fixing speed issues with dpsearch.
20:20:13 <nirik> any other general application news or info?
20:20:59 <nirik> ok, moving along then...
20:21:06 <nirik> #topic Upcoming Tasks/Items
20:21:12 <nirik> (get ready for dump:)
20:21:18 <nirik> #info 2012-03-20 to 2012-04-03 - F17 Beta Freeze
20:21:18 <nirik> #info 2012-03-26 - 21UTC migrate fedorapeople.org
20:21:18 <nirik> #info 2012-03-27 - drop inactive maintainers from packages.
20:21:18 <nirik> #info 2012-04-01 - nag fi-apprentices.
20:21:18 <nirik> #info 2012-04-03 - F17Beta release day
20:21:18 <nirik> #info 2011-04-03 - gitweb-cache removal day.
20:21:20 <nirik> #info 2012-04-10 - drop inactive fi-apprentices
20:21:22 <nirik> #info 2012-04-24 to 2012-05-08 - F17 Final Freeze.
20:21:24 <nirik> #info 2012-05-01 - nag fi-apprentices.
20:21:26 <nirik> #info 2012-05-08 - F17 release
20:21:40 <nirik> a reminder that we are in Beta pre-release freeze.
20:21:50 <nirik> https://fedorahosted.org/fedora-infrastructure/browser/architecture/Environments.png
20:22:44 <nirik> anyone have anything they want to schedule or note thats upcoming?
20:23:17 <dgilmore> nope
20:23:26 <skvidal> fedorapeople outage
20:23:32 <skvidal> once it is done
20:23:38 <nirik> yeah... thats upcoming monday.
20:23:39 <skvidal> we'll have a bunch of new space where people can put stuff
20:23:49 <skvidal> it is for 'groups' or 'projects'
20:23:56 <skvidal> the space will be outside of the quota'd area
20:24:18 <skvidal> and all I wanted to say is that if anyone asks for space, unless they are obviously hosting something illegal or illicit, that our default answer should be 'yes'
20:24:36 <skvidal> but I am not sure we want to make it self-service
20:24:55 <nirik> should we look at moving the last two things we moved to alt back to this? anaconda images and live repins?
20:24:55 <skvidal> does anyone have any input on that?
20:25:07 <nirik> or does it matter? I guess not too much
20:25:29 <skvidal> nirik: I'm thinking moving things back from alt wouldn't be a bad idea
20:25:43 <skvidal> b/c it would mean we could remove people's accesses to alt
20:25:49 <skvidal> which seems like a good idea to me
20:25:50 <nirik> yeah.
20:26:05 <nirik> I'll talk to those groups about that.
20:26:51 <nirik> #action nirik to talk to alt groups about new fedorapeople space
20:26:58 <skvidal> cool.
20:27:07 <nirik> #info fedorapeople outage monday night to move to new host.
20:27:23 <skvidal> did anyone else have any input on the discussion about builders on infrastructure?
20:27:29 <smooge> skvidal, that sounds good to me
20:27:51 <smooge> skvidal, one thing.. various people I know who use alt also do "composes" and such on secondary to get to it.
20:27:52 <skvidal> it sounded to me, optimistically like we had general consensus
20:28:01 <skvidal> wait
20:28:01 <nirik> skvidal: I think it sounds good. Setting them up to reinstall automagically seems like a win.
20:28:06 <skvidal> people are doing composes on secondary?
20:28:18 <skvidal> nirik: cool
20:28:21 <smooge> well soemthing on secondary.. delta isos?
20:28:28 <nirik> smooge: they shouldn't be. Yes, deltaisos are made there.
20:28:35 <nirik> but thats all I know of that runs on there.
20:28:57 <skvidal> so I would not be pleased with deltaisos being made on fedorapeople
20:29:01 <nirik> right.
20:29:03 <skvidal> but making them somewhere else in phx2
20:29:09 <skvidal> and xferring to people should be pretty quick
20:29:15 <abadger1999> skvidal: Your initial email seemed to outline the pros and cons well -- I haven't had a chance to read throught hte rest of the thread.
20:29:26 <nirik> I think making them on secondary still is ok...
20:29:52 <skvidal> nirik: ok
20:30:12 <nirik> we don't need to move everything off alt... just groups that would be ok with using people instead
20:30:14 <nirik> IMHO
20:30:23 <skvidal> nod
20:30:24 <skvidal> +1
20:30:39 <skvidal> i suspect, though, in the future people will be a simpler solution than alt for anumber of things
20:30:56 <nirik> yeah. The only downside is no mirroring...
20:31:19 <skvidal> I agree
20:31:35 <skvidal> nirik: with that in mind - we could discuss mirroring the people space
20:31:40 <skvidal> well not mirroring
20:31:43 <skvidal> but duplicating it
20:31:55 <nirik> yeah. Can look into that.
20:32:01 <skvidal> we've discussed it before and maybe with gluster...
20:32:05 <skvidal> it could be worth considering
20:32:17 <nirik> gluster would only really work if it's in the same datacenter...
20:32:22 <skvidal> right
20:32:29 <skvidal> which is the only downside to ibiblio really
20:32:55 <nirik> We should all ponder on that some more and see what we think....
20:33:02 <skvidal> nod
20:33:17 <skvidal> and we should nag the gluster people to make wan's more possible :)
20:33:18 <nirik> it might be worth rsyncing it to have another remote node as a warm spare...
20:33:38 <nirik> and/or spin up one in the euca cluster once thats running. ;)
20:33:45 <skvidal> okay
20:33:56 <skvidal> if we want to do that...
20:34:03 <nirik> anyhow, we can think about it...
20:34:04 <skvidal> feels like we would want a constant backup of global
20:34:25 <skvidal> I'd be onboard with using older hw to do that.
20:35:01 <daMaestro> yeah, I can help with dpsearch
20:35:06 * nirik wants to think about it some more I think... see if we can come up with something clever and/or more useful.
20:35:29 <nirik> daMaestro: great. See CodeBlock after meeting?
20:35:53 <CodeBlock> :)
20:35:57 <skvidal> nirik: okay. i'm open to suggestions. replicating all that data is tricky
20:36:16 <nirik> yep. agreed.
20:36:21 <nirik> ok, moving along then...
20:36:47 <nirik> #topic Tickets from Ages past
20:36:59 <nirik> anyone have any old ticket they would like us to discuss and dispatch? ;)
20:37:14 <nirik> https://fedorahosted.org/fedora-infrastructure/report/1?sort=created&asc=1 is the full list in age order
20:37:15 <daMaestro> nirik, sounds good; and yea i just ran a test query and ouch
20:37:48 <CodeBlock> Yeah it got really slow .. pretty much all of a sudden, and I'm not sure why
20:38:00 <skvidal> I would like to declare that the age of a majority of a ticket is 3yrs - if it is older than that it is a full adult and should go free
20:38:36 <nirik> skvidal: not a bad plan. I was pinging some of the older ones and asking for update and closing after a bit...
20:39:43 <nirik> .ticket 2572
20:39:45 <zodbot> nirik: #2572 (Consider using Eucalyptus for test VM hosting) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/2572
20:39:55 <nirik> should we close that with 'thanks, we are going to set one up' ?
20:39:59 <wolfkit> nirik: #503 seems silly, it was closed at wontfix twice already and I don't think 'new ticket' should be visible for people who aren't logged in
20:39:59 <skvidal> hee hee
20:40:00 <skvidal> yes
20:40:14 <wolfkit> .ticket 503
20:40:16 <zodbot> wolfkit: #503 (fedorahosted: trac: new ticket) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/503
20:41:08 <nirik> I suppose we could package that plugin...
20:41:28 <nirik> but yes, its old and seems silly.
20:41:53 <smooge> Can I close tickets.. please. I would feel useful.
20:42:30 <nirik> or there may be some newer way to solve it with the new trac?
20:42:50 <nirik> smooge: I would say we can close 2572
20:43:17 <nirik> .ticket 2134
20:43:18 <zodbot> nirik: #2134 (Ticket email notification (Cc) should recognize FAS usernames) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/2134
20:43:41 <nirik> 2134 can also be closed. I think they want all fas accounts to work even if they have no alias, but I don't see how we could allow that to work easily.
20:44:38 <nirik> any other ones for the axe? ;)
20:45:06 <nirik> .ticket 2352
20:45:08 <zodbot> nirik: #2352 (inotifier - make fedora-updates-push (and other such rsync scripts) smarter) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/2352
20:45:16 <nirik> can be closed with: please talk to the messaging sig?
20:45:23 <nirik> or was that more specific...
20:46:42 <smooge> well its looking for inotify and such
20:46:46 <wolfkit> .ticket 668 also seems silly and unnecessary
20:46:46 <zodbot> wolfkit: Error: '668 also seems silly and unnecessary' is not a valid integer.
20:46:51 <wolfkit> woops
20:46:53 <wolfkit> .ticket 668
20:46:54 <zodbot> wolfkit: #668 (packages.fedoraproject.org/ alias for https://admin.fedoraproject.org/pkgdb/packages/name/ ) – Fedora Infrastructure - https://fedorahosted.org/fedora-infrastructure/ticket/668
20:47:29 <nirik> wolfkit: hey, we already have this. ;)
20:47:36 <nirik> https://apps.fedoraproject.org/packages/<packagename>
20:47:51 <wolfkit> aah, so can be closed as fixed then!
20:47:58 <nirik> yes. please do. ;)
20:48:01 <smooge> closed
20:48:04 <nirik> smooge: thanks
20:48:18 <smooge> hmm zodbot isn't yelling about me closing tickets
20:48:34 <wolfkit> that should kind of be fixed instead wontfix, but ok
20:48:56 <nirik> smooge: it will. oh it will
20:49:23 <nirik> ok, thats some good old ticket slaying for the day. ;)
20:49:27 <nirik> #topic Open Floor
20:49:40 <nirik> anyone have any items for open floor? questions? concerns?
20:50:42 <skvidal> I have a couple
20:50:45 <skvidal> well just one
20:50:45 <marcelk1985> ditto
20:50:47 <nirik> skvidal: fire away
20:51:01 <skvidal> so this has come up again and I'm curious folks' feelings on it
20:51:19 <skvidal> if we moved to a model where we wanted to ssh into a machine as root - only using an ssh key and only from a specific system
20:51:26 <skvidal> how would we feel about that?
20:51:30 <skvidal> bad?
20:51:32 <skvidal> horrible?
20:51:34 <marcelk1985> scary
20:51:52 <nirik> somewhat scary, but not 'run and hide under the table'
20:52:33 <skvidal> I'm torn on it
20:52:41 <skvidal> and I'm working on a workaround for it
20:52:47 <dgilmore> skvidal: im pretty torn
20:53:10 <nirik> well, we sorta already have that with funcd right? it's a root process that we connect to from a specific machine and make it do things?
20:53:10 <dgilmore> can we not use sudo for something
20:53:18 <skvidal> nirik: yes
20:53:25 <skvidal> dgilmore: so sudo is a difficulty in some ways
20:53:29 <skvidal> but it is part of a workaround
20:53:32 <skvidal> here's the gist
20:53:37 <skvidal> to streamline our install process
20:53:55 <skvidal> I would like to do
20:54:02 <skvidal> virt-install/kickstart
20:54:04 <skvidal> monitor for ssh
20:54:07 <skvidal> ssh in as root
20:54:15 <skvidal> fix up the host in a myriad of ways
20:54:28 <skvidal> and have this all automated - not requiring someone to log in and muck with the system
20:54:38 <skvidal> AFTER the first puppet run, though
20:54:44 <skvidal> you cannot connect back in as root
20:54:58 <skvidal> so sudo Might be possible
20:55:01 <nirik> huh... is there possibly a way to boot single and talk to it via libvirt / serial /
20:55:06 <skvidal> but sudo + python_paramiko is.... interesting
20:55:09 <skvidal> nirik: no
20:55:17 <skvidal> nirik: that would expect-based and extremely fragile
20:55:31 <dgilmore> skvidal: any reason we cant just do the fixups in %post of kickstart
20:55:36 <skvidal> nirik: and it would mean a different system for our hw and vm hosts
20:55:39 <skvidal> dgilmore: yes
20:55:49 <skvidal> dgilmore: we're talking about shoving the right root pw across that connection
20:55:58 <skvidal> I don't want the rootpw in our kickstarts
20:56:15 <marcelk1985> Ok I have to punch out. Sorry. I will read the meeting's minutes. I will post the question I had to the mailing list
20:56:28 <nirik> marcelk1985: sorry for going long. Please do...
20:56:34 <dgilmore> skvidal: have %post fetch a script from infra.fp.o
20:56:37 <dgilmore> and run that
20:56:43 <dgilmore> though its still in there
20:56:46 <skvidal> yes
20:56:48 <skvidal> which is the problem
20:56:50 <nirik> skvidal: randomly generate and notify people what it is?
20:56:55 <skvidal> how does that help?
20:57:18 <nirik> well, it's not in the ks then... but notifying could be not so great if it was email.
20:57:19 <skvidal> nirik: and how would you notify?
20:57:22 <dgilmore> disable roots passwd
20:57:22 <skvidal> right
20:57:27 <skvidal> okay
20:57:38 <skvidal> so we're off in the weeds
20:57:42 <nirik> yeah, sorry.
20:57:44 <dgilmore> skvidal: what about if the script got func running and we did it that way?
20:57:50 <skvidal> can't do that
20:57:50 <dgilmore> sorry making noise now
20:57:56 <skvidal> anyway
20:57:59 <skvidal> I'm going to work on a solution
20:58:08 <skvidal> and if everyone hates me for it
20:58:09 <skvidal> great
20:58:19 <smooge> what if people just hate you :)
20:58:20 <nirik> you thrive on hate? :)
20:58:20 <skvidal> but if not
20:58:25 <skvidal> so much so
20:59:04 <skvidal> anyway - I'll figure it out
20:59:08 <skvidal> and y'all can poke holes in it later
20:59:09 <nirik> ok
20:59:12 <smooge> I have one item
20:59:19 <nirik> smooge: go ahead
20:59:36 <smooge> mediawiki118 has had some security updates.. will get that packaged up today. 119 will probably be out RSN.
21:00:08 <nirik> should we just target 119 then?
21:00:14 <smooge> I am wanting input on how to handle this.. they are giving these releases something like a 1 year lifetime.. which doesn't work well with EPEL
21:00:59 <nirik> yeah, thats not very ideal...
21:01:03 <smooge> I will have 119 packaged up also soon. getting plugins and such is a fun fun thing
21:01:15 <nirik> but you could just leave a trail of old ones and keep going on the new ones...
21:01:20 * nirik is happy to try and help review.
21:01:51 <nirik> I guess we could just switch to only doing our own for infra.
21:01:54 <smooge> yeah.. I didn't know if that worked well either. Blech.. I can see all our web infrastructure in openshift at this rate
21:02:11 <nirik> but then we lose out on all the other people who might use/help/fix it
21:02:15 <smooge> because webstuff don't like to be stable more than a week
21:02:38 <skvidal> nirik: I think there's a happy medium betwee nthere
21:02:43 <skvidal> our own pkgs for stuff which obscure
21:02:49 <skvidal> public ones for stuff that's more common
21:02:52 <skvidal> (like mediawiki)
21:02:57 <nirik> yeah.
21:03:21 <nirik> if we skip like we have been... 116 -> 119 perhaps thats enough time for a epel package to be worth doing...
21:03:35 <smooge> we went from 112 to 116
21:03:46 <smooge> or was it 110
21:04:01 <smooge> anyway.. I will get them ready for people to look at today.
21:04:07 <smooge> tomorrowish.
21:04:09 <smooge> eol
21:04:22 <nirik> #action smooge to get mediawiki packages setup.
21:04:37 <nirik> ok, anything else? or shall we call it a meeting?
21:04:59 * CodeBlock has to head to class, so either way, I'm out :)
21:05:03 * nirik will close in a minute if nothing else comes up.
21:05:05 * wolfkit calls smooge a meeting. :)
21:05:33 <nirik> thanks for coming everyone!
21:05:36 <nirik> #endmeeting