18:00:42 <nirik> #startmeeting Infrastructure (2014-06-12)
18:00:42 <zodbot> Meeting started Thu Jun 12 18:00:42 2014 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:42 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:42 <nirik> #meetingname infrastructure
18:00:42 <nirik> #topic Namaste
18:00:42 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk
18:00:42 <zodbot> The meeting name has been set to 'infrastructure'
18:00:42 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean
18:00:49 * mirek-hm here
18:00:53 * lmacken 
18:00:53 * webpigeon waves
18:00:58 * adimania is here
18:01:02 * ausmarton_ here
18:01:07 * oddshocks beams in
18:01:42 <smooge> here
18:01:43 * banas here too, first time :)
18:01:46 * pingou here
18:01:50 * bwood09 is here
18:01:52 <pingou> banas: about time man ;-)
18:02:05 <banas> pingou: :D
18:02:15 * danofsatx-work is here, but distracted
18:02:17 * ausmarton here
18:02:50 <danofsatx-work> that is.....unless anyone has a quick-n-dirty howto on getting apache to authenticate against freeipa?
18:02:53 * threebean is here
18:02:53 <nirik> cool. lots of folks today.
18:03:30 <nirik> danofsatx-work: you typed 'quick' and 'freeipa'. ;) sorry no.
18:03:30 * tflink 
18:03:33 <nirik> #topic New folks introductions and Apprentice tasks
18:03:42 <nirik> ok, any new folks like to introduce themselves?
18:03:47 * danofsatx-work shuffles back into the corner
18:03:48 <lalit_> Hello everyone. My name is Lalit Khattar. I am a GSoC student this year and I am working on waartaa under mentorship of rtnpro and sayan. http://waartaa.com
18:03:52 <nirik> or apprentices with questions/comments/ideas?
18:03:58 <webpigeon> o/ lalit_
18:04:02 <lalit_> I am attending this meeting first time :)
18:04:08 <oddshocks> lalit_: welcome!
18:04:11 * abadger1999 here
18:04:13 <nirik> welcome lalit_
18:04:15 <lalit_> oddshocks, thanks :)
18:04:16 <threebean> lalit_: welcome :)
18:04:25 * abompard is here with pingou
18:04:32 * pingou confirms
18:04:34 <banas> Hi, I'm Sarup, first time here. I'm doing a GSoC as well :) I kinda tinker around more in the design team.
18:04:39 <webpigeon> o/ abompard
18:04:40 <abompard> (yes, in the same room, but don't get any ideas)
18:04:49 <banas> .fasinfo sarupbanskota
18:04:50 <zodbot> banas: User: sarupbanskota, Name: Sarup Banskota, email: sbanskota08@gmail.com, Creation: 2013-03-28, IRC Nick: banas, Timezone: Asia/Kolkata, Locale: en, GPG key ID: , Status: active
18:04:53 <zodbot> banas: Approved Groups: designteam cla_done cla_fpca
18:04:57 <danofsatx-work> I am unable to respond to this month's apprentice interest email due to an ongoing problem with outlook.com handling alias addresses
18:05:02 <oddshocks> banas: welcome as well
18:05:17 <banas> oddshocks: thanks!
18:05:17 <nirik> danofsatx-work: noted. :(
18:05:42 <ootbro> I'm back from my vacation (feeling a little crispy) and about ready to dive into my FP project.
18:05:49 * abadger1999 erases SciFi ideas of downloading consciousness to a shared body from memory.
18:05:55 * danofsatx-work needs to figure out how to send from @fp.o address
18:06:00 <nirik> welcome banas. lalit_ / banas: how far along is GSoC ? has it reached midpoint yet?
18:06:20 * pingou provides cafeine to abadger1999
18:06:29 <lalit_> I have completed first version of my first two major tasks.
18:06:37 <lalit_> My first task was to create a search API and an interface for searching irc logs collected by waartaa backend. I completed that task last week. Here is the blog post:
18:06:38 <lalit_> http://lalitkhattar.wordpress.com/2014/06/04/implementation-of-chat-log-browser-gsoc-week-2/
18:06:40 * bwood09 needs caffeine and the soda machine won't dispense any :/
18:06:49 <oddshocks> Last year GSoC went on through August I think
18:06:56 * oddshocks worked with a GSoC student last year
18:07:17 <bwood09> stupid thing keeps declining my NFC payment -_-
18:07:35 <nirik> lalit_: alright.
18:07:46 <banas> nirik: not midpoint yet, although the work on GG is going decent. Rohit has  moved to a new library for the git stuff, I've started the redesign. Some updates on my last blogposts, pending ones on the next: http://sarupbanskota.github.io/fedora/2014/06/07/init-redesigning-glittergallery-responsively.html
18:07:56 <lalit_> My second task was to create a p2p(peer to peer) video chat in waartaa. I completed that task yesterday. Blog post: http://lalitkhattar.wordpress.com/2014/06/11/p2p-video-chat-gsoc-week-3/
18:08:30 <nirik> cool. Any other new folks or apprentices with questions?
18:08:33 <nirik> banas: great!
18:08:40 <nirik> lalit_: interesting. webrtc is up and coming. ;)
18:09:17 <ootbro> I'm back from my vacation (feeling a little crispy) and about ready to dive into my FP project.
18:09:34 <banas> I guess GG is now in the stage where newcomers have come in and are doing cool stuff the authors don't understand right away :D
18:10:00 <banas> btw, hi to lalit_ as well!
18:10:05 <lalit_> nirik, yep. I am glad I got a chance to work on it :)
18:10:14 <lalit_> banas, hello
18:10:20 <hammad> hello, my name is hammad, A Gsoc student working on fedora College project. This week I was working on the API for the product, Almost done with that ..
18:10:38 * oddshocks waves to hammad
18:10:59 <nirik> ootbro: sounds good. :)
18:11:00 <banas> hi hammad!
18:11:21 * adimania sees a lot of GSoC students :)
18:11:28 <webpigeon> indeedy
18:11:44 <nirik> yep.
18:11:50 <ootbro> nirik: I just sent a reply to your reply....  can wait until after we're done here.
18:12:00 <nirik> ootbro: ok, haven't had a chance to read it yet, but will. ;)
18:12:09 <ootbro> not a problem
18:12:15 <nirik> ok, moving along then...
18:12:17 <nirik> #topic Applications status / discussion
18:12:27 * pingou got some releases out
18:12:28 <nirik> any applications news/plans/discussion this week?
18:12:32 <pingou> new pkgdb2, new pkgdb-cli
18:12:41 <pingou> new fedocal coming up
18:12:43 <pingou> new elections
18:12:59 <pingou> pkgdb2 is deployed, so is elections, pkgdb-cli is pending for -testing
18:13:12 <nirik> and all these are just bugfixes right, no api breaks or anything?
18:13:33 <pingou> yes
18:13:51 <pingou> api improvements but no api backward compatibility
18:13:58 <pingou> the rest is bugfixes in the UI
18:14:04 <threebean> pingou: nice work :)
18:14:10 <nirik> cool.
18:14:14 <oddshocks> Lots of progress on fedimg. Should easily have automatic AMI creation and registration in all EC2 regions for every successful image build that passes through Koji done by our next meeting, if not by tomorrow.
18:14:20 <pingou> fedocal coming up either later this week or early next week
18:14:26 <nirik> abompard: hey, since you are around, where are we on hyperkitty? ;)
18:14:28 <oddshocks> up next is GCE, HP, and Rackspace
18:14:34 <oddshocks> among others
18:14:47 * tflink has a question about Taskotron deployment, not sure if it's good meeting material or if it's better to discuss later
18:15:03 <tflink> mostly about timeline and how long things should be in stg
18:15:05 <nirik> tflink: either way.
18:15:24 <threebean> datanommer got an update in production with a bugfix this week.
18:15:35 <nirik> well, depends on the scope of changes... but of course with no production changes are infinite. ;)
18:15:39 <tflink> I'm wondering how long Taskotron should be in infra staging before we can start creating a production instance
18:15:41 <pingou> threebean: and fmn got 2 releases \ó/
18:15:43 <abompard> nirik: lots of improvements around hyperkitty these days, for example in the mailman-bundler script
18:15:45 <threebean> pingou: true
18:16:03 <nirik> abompard: can we move over a few production lists then?
18:16:03 <abompard> which should make it much easier for people to install and test mailman+hyperkitty+postoriys
18:16:15 <abompard> lots of stability fixes too
18:16:50 <threebean> oddshocks: as you get closer to having something releasable you want to try out in infra, take a look at filing a "request for resources" ticket - https://fedoraproject.org/wiki/Request_For_Resources
18:17:02 <lmacken> oddshocks: did you get the broken pip issue resolved?
18:17:06 <lmacken> *pipe
18:17:12 <abompard> nirik: yeah, I need to deploy the new version to prod though
18:17:32 <oddshocks> threebean: Will do, thanks!
18:17:37 <nirik> abompard: ok. Also, do we now want to redeploy on rhel7?
18:17:42 <abompard> nirik: I still have a small permission issue on the second prod server
18:17:44 <pingou> abompard: got any timeline in mind?
18:17:55 <abompard> Oh yeah, let's move to rhel7 first
18:17:58 <pingou> \ó/
18:18:03 <oddshocks> lmacken: Yeah, all good, it was caused by another problem. Rewriting some stuff right now but it should work and also be a lot faster than before :)
18:18:10 <oddshocks> lmacken: thanks for checking it out
18:18:12 <lmacken> oddshocks: excellent, nice work :)
18:18:15 <oddshocks> :)
18:18:16 <pingou> nirik: I have the feeling the move to el7 is going to be easier than the move to el6 was :)
18:18:33 <nirik> pingou: hard to say... but we will see.
18:19:41 <oddshocks> I'm sure everyone who worked on rhel 7 will be happy to see that 4-year loop closed
18:19:50 <pingou> nirik: more up to dates apps :)
18:20:02 <abadger1999> tflink: Does taskotron impact existing services?  (Like make automated queries to an existing service?)
18:20:34 <nirik> pingou: I suspect we will need to do a fair bit of packaging work in epel for some of our stacks... and if we want to try and move to py3 also that will be more work too. ;)
18:20:58 <abadger1999> tflink: If it does not, then I think it doesn't need to sit in stg for very long.
18:21:20 <pingou> nirik: flask has barely any deps ;-)
18:21:25 <pingou> nirik: but for sure we'll see
18:21:39 <lmacken> and the deps it does have, you won't be able to pronounce ;)
18:21:58 <abadger1999> tflink: If it does, we need to evaluate the risk but we probably should deploy it sooner (before we get into release freezes) rather than later.
18:22:30 <nirik> pingou: actually it looks like they are pulling flask into rhel...
18:22:45 <pingou> ah, that might get tricky :/
18:23:25 <nirik> tflink: yeah, I guess I would say for a first prod run sooner is better, but of course it needs to be at least somewhat functional. ;)
18:23:26 <abadger1999> :-(
18:23:44 <abadger1999> heh.  I imagine our reaction is the opposite of what many rhel customer's is...
18:24:47 <nirik> except I am not seeing it... but someone filed a bunch of bugs asking for epel maintainers to remove it.
18:25:09 <pingou> nirik: for flask?
18:25:17 <nirik> yeah
18:25:25 <nirik> anyhow, can do that outside meeting time. ;)
18:25:29 <pingou> sure
18:25:32 <abadger1999> Hmmm -- maybe they were jumping the gun?  It was in a "layered product" not in the pieces of rhel that we care about.
18:25:51 * abadger1999 notes that last sentence should end in a question mark :-)
18:26:33 <threebean> oh, we got a cloud/dev instance of bodhi2 up today
18:26:34 <nirik> yeah, dunno.
18:26:38 <threebean> http://bodhi.dev.fedoraproject.org
18:27:00 <tflink> abadger1999: yeah, it queries both koji and bodhi
18:27:02 <threebean> any feedback appreciated.. perhaps just mention it in #fedora-apps or file bugs on github?
18:27:05 <oddshocks> threebean: nice!
18:27:20 <oddshocks> gratz bodhi folks
18:27:56 <tflink> nirik: it's mostly functional - our current staging instance has been running for a month or so now
18:28:01 <abadger1999> tflink: k.  I still think we want to get it deployed a bit sooner... We don't put the same sort of load onto stg that we do on prod so we sadly don't test that side of things too well in stg.
18:28:27 <tflink> abadger1999: I still want to do a load/stress test of a dev-ish instance to see where the bottlenecks are
18:28:36 <abadger1999> Sounds good.
18:28:58 <tflink> unofortunately, my todo list has been growing with higher-priority tasks faster than I'm taking care of them :-/
18:29:08 <nirik> tflink: I hear ya there. ;(
18:29:14 <tflink> I'd like to switch autoqa off around branch (July 8)
18:29:38 <tflink> but that's not very far off
18:29:50 <nirik> yeah.
18:30:00 <nirik> we would want to get it in prod well before that I think if we can
18:30:31 <tflink> it can't be 100% production until autoqa is turned off due to how feedback is provided via bodhi comments
18:30:46 <oddshocks> nirik: Congrats on being the first badger to earn over 100 Fedora Badges :)
18:30:55 <nirik> oddshocks: :)
18:31:28 <nirik> tflink: sure, but it can be ready to switch to, etc.
18:31:46 <tflink> yeah, my plan was to use our fake_fedorainfra instead of bodhi until the switchover
18:32:31 <tflink> we've been using it for autoqa-stg for a year or two and other than a few error cases when bodhi times out, it mostly solves the issue of having multiple things commenting to bodhi
18:34:16 <nirik> ok, cool.
18:34:22 <nirik> anything else app side?
18:34:53 <threebean> real quick..
18:35:19 <threebean> fmn's backend is getting faster and faster
18:35:23 <pingou> \ó/
18:35:25 <nirik> abompard: do think on moving to rhel7... if we want to do that before we move any prod lists...
18:35:31 <nirik> threebean: cool. :)
18:35:33 <threebean> I'm focusing on trying to get the QueueLength graph at the bottom of this report down to 'zero' as much as possible
18:35:37 <threebean> .tiny https://admin.fedoraproject.org/collectd/bin/index.cgi?hostname=notifs-backend01.phx2.fedoraproject.org&plugin=fedmsg&timespan=154800&action=show_selection&ok_button=OK
18:35:38 <zodbot> threebean: http://tinyurl.com/pzmmdug
18:35:44 <abompard> nirik: yeah, we should do that first
18:36:22 <nirik> FYI for anyone who doesn't know... 'fmn' is a fedora notifications server.
18:36:26 <threebean> if people want to help test it and report bugs and feature requests, that would be helpful
18:36:27 <threebean> yes
18:36:27 <oddshocks> cool
18:36:27 * bwood09 wishes he could get RHEL7 licensing :/
18:36:32 <threebean> https://apps.fedoraproject.org/notifications
18:36:37 <nirik> abompard: ok, can let you know when we get thigns setup for that
18:36:50 <abompard> nirik: yeah, please do :-)
18:37:30 <nirik> hopefully not too long
18:37:37 <abompard> great
18:38:08 <nirik> ok, moving along...
18:38:15 <nirik> #topic Sysadmin status / discussion
18:38:36 <nirik> we got value* machines moved to ansible earlier this week. Thanks for the help on that threebean
18:38:43 <threebean> :)
18:39:03 <nirik> we have just around 100 machines left in puppet. about 32 of those are virthosts we just haven't switched yet because we haven't re-installed them yet.
18:39:12 <nirik> so moving along nicely.
18:39:51 <smooge> yay.
18:39:59 <bwood09> I'm still rounding the edges of that fedorahosted stuff; I need a bit of help though.
18:40:06 <mirek-hm> what's the status of machines for new cloud?
18:40:08 <adimania> I can pick up another module.
18:40:10 <nirik> bwood09: you did get into the machine finally right?
18:40:14 <bwood09> I did :)
18:40:17 <nirik> cool.
18:40:18 <bwood09> In it now, actaully, lol
18:40:21 <smooge> mirek-hm, I have 3 higher priority tasks at the moment
18:40:40 <mirek-hm> ok
18:40:42 <nirik> smooge / mirek-hm: I think we have them on the net now, just need a baseos install?
18:40:54 <smooge> mirek-hm, I got a minimal install on them and got their networks tested. I need to put a full os on them and then turn them over
18:41:10 <smooge> I would like to get a box on the network I can use to do faster installs on first
18:41:26 <smooge> as I am tired of spending 4 hours waiting for a minimal OS to upload
18:41:45 <nirik> smooge: fed-cloud02.cloud does have a dhcpd and pxe setup I used for the arm03 SOCs.
18:41:51 <nirik> might be able to use that.
18:42:34 <smooge> nirik, yeah. that is my next choice.
18:42:47 <nirik> it's kludgy, but would at least get them up
18:43:02 <smooge> I wanted to get a permanent one up with an ansible set so I could reinstall the dhcp on devel etc and cut that out of our list
18:43:31 <smooge> I figure that I should have that by early next week.
18:43:42 <smooge> if not I am going with dhcpd on cloud2
18:43:49 <nirik> ok.
18:44:02 <bwood09> If I could get a couple of people to help me with narrowing this list down, getting metrics on who it'll affect, and planning implementation, that'd be awesome.
18:44:19 <smooge> I am working on getting osuosl working and dl-ib01 turned off as higher priority at the moment
18:44:23 <bwood09> I can probably do it by myself, but it would take (literally) forever. lol
18:44:41 <nirik> bwood09: cool. Might ask on the list? or perhaps start in a bit and then post to the list with some beginning data to get people interested?
18:45:01 <bwood09> If you don't mind pointing me toward how I could grab some of it, sure
18:45:25 <bwood09> I don't need specific instructions, just a general idea of how
18:45:47 <nirik> bwood09: can try after the meeting?
18:45:50 <bwood09> Not a problem
18:46:29 <nirik> lets see... I've been trying to setup things so we can start rhel7 installs... hopefully will have that before too long.
18:46:43 <nirik> #topic nagios recap
18:46:48 <nirik> .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=5&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=5&eday=15&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=2&hoststates=3&servicestates=56&limit=25
18:46:48 <zodbot> nirik: http://tinyurl.com/q8j48o9
18:47:06 <nirik> doing better than last week by a fair bit. ;)
18:47:28 <smooge> yeah. that is the best I have seen in a while
18:47:31 <nirik> I might try adjusting the telia datacenter checks to make them less trigger happy.
18:47:42 <nirik> or if anyone else wants to look into that that would be great.
18:47:45 <smooge> would that be by turning them off :)
18:48:35 <nirik> yeah. The sad thing is that it's just phx2 < -- > telia that don't like talking sometimes...
18:48:53 * bwood09 will be back in ~20 minutes; highlighting bwood09_ so IRC will be flashing on my taskbar when I get home
18:49:38 <nirik> #topic Upcoming Tasks/Items
18:49:38 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/
18:49:47 <nirik> anything upcoming folks would like to note or schedule?
18:50:47 <nirik> smooge: I guess we should start/finish planning our phx2 trip...
18:50:54 <nirik> I guess late july will be ok for me.
18:51:00 <mjnovice> hello everybody, was a bit late!
18:51:07 <mjnovice> meeting about to end ?
18:51:16 <nirik> mjnovice: getting near end yeah. ;) welcome tho.
18:51:32 <smooge> nirik, yes. I think that July 22->July 25
18:52:00 <smooge> hardware should arrive by then
18:52:23 <danofsatx-work> Texas Linux Fest this weekend. I will be attending.
18:52:30 <nirik> smooge: yeah. I guess lets float those dates to rhit and others and make sure there's no conflicts and then start booking thing.
18:53:21 <mjnovice> Wanted to update on bugspad. Due to some fixing, deployment for dev-testing got delayed. But will do it by the end of this week. :)
18:53:36 <nirik> mjnovice: cool. :)
18:53:45 <mjnovice> nirik: :)
18:54:17 <nirik> #topic Open Floor
18:54:30 <nirik> anyone have items for open floor? questions, comments, favorate editors?
18:54:44 <adimania> I know nagios, python and little bit of Django.  Where can I help?
18:54:44 <danofsatx-work> whoops, the TLF shoulda gone here....sorry
18:55:38 <nirik> adimania: take a look at easyfix tickets? or jump in with stuff mentioned on irc...
18:55:48 <nirik> danofsatx-work: no worries. say hi to the fedora folks there.
18:55:55 <danofsatx-work> sure will.
18:56:01 <danofsatx-work> I need some swag ;)
18:56:06 <threebean> adimania: yeah, ask.fedoraproject.org is our only django app that I know of.
18:56:32 <nirik> but there's outstanding tickets on it. ;)
18:56:38 <adimania> threebean, that's okay. I understand that we have a lots of python projects.
18:56:39 <threebean> hey
18:56:49 <pingou> w/ hyperkitty (in the future)
18:56:58 <threebean> well, ask.fedoraproject.org needs some help.  no one is actively working on it at the moment iirc.
18:57:15 <pingou> threebean: +1
18:57:29 <adimania> threebean, nirik cool. I'll try to get the tickets resolved there.
18:57:35 <threebean> awesome :)
18:57:40 <adimania> any sysadmin stuff where I can help?
18:57:43 <nirik> adimania: great. :)
18:57:59 <threebean> hm, datagrepper could be migrated from puppet to ansible :P
18:58:11 <nirik> https://fedoraproject.org/easyfix/
18:58:13 <pingou> \ó/
18:58:33 <nirik> yeah, thats ticket 4393. ;)
18:58:35 <adimania> nirik, looked at easyfix already. :)
18:59:26 <nirik> ok. Aside from those hanging out in irc and asking questions or chiming up when someone says we should do something will often find you tasks too. ;)
18:59:34 * nirik sees we are nearly out of time.
19:00:06 * adimania got a couple of leads to work on. datagrepper migration and ask.fp.o
19:00:17 <nirik> please do continue over in #fedora-admin, #fedora-noc and #fedora-apps. ;)
19:00:20 <nirik> Thanks for coming everyone!
19:00:23 <nirik> #endmeeting