18:00:33 <nirik> #startmeeting Infrastructure (2015-01-08) 18:00:33 <zodbot> Meeting started Thu Jan 8 18:00:33 2015 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:33 <nirik> #meetingname infrastructure 18:00:33 <nirik> #topic aloha 18:00:34 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:00:34 <zodbot> The meeting name has been set to 'infrastructure' 18:00:34 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:00:38 * pingou 18:00:42 * webpigeon waves 18:00:46 * danielbruno here 18:01:04 * puiterwijk is here 18:01:07 <smooge> here 18:01:13 * lmacken 18:01:16 <nirik> morning everyone. 18:01:22 * nirik will wait a few min for folks to wander in 18:01:30 <kushalk124> hello :) 18:01:32 * sborza is here 18:02:00 <webpigeon> it's morning o.O 18:02:01 * threebean threebean 18:02:14 <webpigeon> threebean: that makes sixbean 18:02:23 <mpduty> hi 18:02:38 <threebean> webpigeon: or worse, nine! :| 18:03:14 <nirik> heh. 18:03:18 <nirik> #topic New folks introductions and Apprentice tasks 18:03:25 <nirik> any new folks like to introduce themselves? 18:03:34 <nirik> or apprentices with questions, suggestions, comments? 18:03:46 <mhurron> ya hi, I'm Sean, new here 18:04:05 <fale> I'm Fabio A Locati, an Italian freelance sysop, dba and I'm new here :) 18:04:14 <threebean> fale: hi :) 18:04:23 <nirik> welcome fale and mhurron. :) 18:04:24 <mhurron> I'm a sysadmin, know a little python, looking to help out 18:04:24 * threebean waves to mhurron 18:04:47 <nirik> question for both of you: are you more interested in sysadmin work? or application development? or both? 18:05:33 <mhurron> I know more sysadmin, I don't know if I can be useful for development, So more admin I suppose 18:05:35 <fale> nirik: a little bit more on sysadmin because I already have competence in the area so it would be easier to start from there for me 18:06:00 <nirik> ok, cool. 18:06:18 <nirik> If you are interested I can add you to our apprentice group after the meeting... see me in #fedora-admin and I can get you setup. 18:06:26 <fale> nirik: thankyou 18:06:34 <kushalk124> Hey, I am Kushal currently located in Germany, I plan to help in application development in python ! 18:07:06 <nirik> kushalk124: welcome. Do see the kind folks in #fedora-apps after the meeting... 18:07:23 <kushal> kushalk124, also please say your full name or else people will be confused. 18:07:31 <kushalk124> nirik, Thanks :) I would definitely do so, 18:07:46 <kushalk124> kushal, Oh yes, Kushal Khandelwal here 18:08:06 <sborza> nirik: not sure if I'm already a member of the apprentice group, but would like to be added 18:08:17 <nirik> kushal: yeah... :) 18:08:29 <sborza> I've been out for a few weeks (new baby), but am now back and ready to contribute 18:08:39 <nirik> sborza: can check. ;) see me after the meeting in #fedora-admin and I can see... 18:08:49 <sborza> nirik: will do, thanks! 18:09:07 <pingou> sborza: and congratulation 18:09:09 <pingou> s 18:09:15 <pingou> :) 18:09:25 <nirik> cool. Any other new folks or apprentices with questions? 18:09:31 <sborza> pingou: thank you! 18:09:55 <nirik> #topic Applications status / discussion 18:10:08 <nirik> any applications side news or discussion this week? ;) 18:10:17 * pingou has been working on unit-tests for mirrormanager2 18:10:30 <nirik> I'll note that I sent out a email asking for status on various stuff... please do read it and reply if you have info on those items. 18:10:37 * pingou should 18:10:40 <threebean> pingou: thanks for that :) 18:11:21 <lmacken> I pushed out a new bodhi release to prod this week that contains bochecha's patches to work around the weird \n behavior that started happening after our postgres upgrade 18:11:28 <threebean> I'm aiming to get an FMN release out in the next week with lots of little enhancements and bugfixes all over. -- https://apps.fedoraproject.org/notifications 18:11:32 <pingou> threebean: got a couple of (simple) bugs so far :) 18:11:42 <lmacken> hopefully just a weird SQLObject issue that will be resolved with bodhi2 + SQLAlchemy 18:12:07 <nirik> lmacken: yeah. Oh, have you had a chance to try and setup bodhi2 on bodhi02.stg ? ;) 18:12:19 <nirik> #info lots of work on mirrormanager 2 unit tests. 18:12:31 <nirik> #info bodhi1 update out in production to fix db issue with newlines 18:12:42 <lmacken> threebean: will it support anitya filters? 18:12:44 <nirik> #info work on FMN to get it ready for prod 18:13:06 <lmacken> nirik: not yet, that's the goal for the next couple of weeks 18:13:08 <threebean> lmacken: sure will 18:13:14 <lmacken> threebean: excellent :) 18:14:14 <lmacken> having irc notifications for all upstream releases for all of my packages will be great 18:14:38 * nirik nods 18:14:45 * pingou likes 18:15:18 <nirik> we do have the f22 schedule from fesco now, so we will have to figure out when we want to land new apps/etc in production... 18:15:26 <nirik> and avoid freezes, etc 18:16:38 <nirik> anything else application wise? 18:16:41 <threebean> https://fedoraproject.org/wiki/Releases/22/Schedule 18:16:52 <lmacken> bodhi atomic composes have been going smoothly :) 18:16:52 <nirik> yep 18:17:01 <threebean> wow, alpha freeze starts at end of february? 18:17:13 <threebean> it's always seems like such a short window inbetween. 18:17:14 <nirik> lmacken: excellent. Wonder how many people are consuming them... I guess as always we never know. ;) 18:17:24 <nirik> threebean: yeah, it's a short ramp. ;( 18:17:47 <lmacken> nirik: heh, yep 18:18:40 <nirik> #topic Sysadmin status / discussion 18:18:48 <nirik> ok, lets see... on the sysadmin side... 18:19:10 <threebean> ooo 18:19:12 <nirik> we are down to 53 hosts in puppet. 18:19:20 <pingou> \ó/ 18:19:22 <nirik> threebean: your proxy01.stg seems to be mostly working well. ;) 18:19:33 <threebean> the proxy layer redone in ansible has been put in place in stg now. :) 18:19:38 <nirik> I wanted to get puiterwijk to look at the varnish config. I just hacked it up so it would work 18:19:38 <threebean> cool, cool. :) 18:19:58 <nirik> it also needs selinux work... (the varnish setup, won't start in enforcing) 18:20:31 <threebean> eh, yeah. we'll definitely want that for the front door. 18:20:57 <nirik> yeah, once thats sorted, I think we could look at making a proxy10 or something somewhere and adding it to the mix... 18:21:01 <threebean> haproxy needed some selinux help too, but that's nailed down now. 18:21:07 <nirik> cool. 18:21:50 <nirik> I think haproxy in stg is all green (aside gallery01.stg, which doesn't matter) 18:22:00 <nirik> https://admin.stg.fedoraproject.org/haproxy/proxy01 18:22:27 <threebean> could do a rube run too -- if anyone has some free time to mess with it. 18:22:31 <nirik> #info proxy01.stg needs a bit more work, but overall is looking great in ansible 18:22:37 <nirik> yeah, great idea 18:22:39 <threebean> maybe an apprentice? 18:22:49 <threebean> https://github.com/fedora-infra/rube 18:22:57 <nirik> sure... ^ 18:23:11 <threebean> there's always something new broken with it each time I go back to mess with it.. but it's useful to figure out where we have fires burning we didn't know about. 18:23:18 <lmacken> threebean: yep, always. 18:23:27 <nirik> what happened with running it in jenkins? 18:23:31 <nirik> just too broken from run to run? 18:23:59 <threebean> nirik: yeah, and jenkins itself kept blipping out and I haven't made time to follow up on it. 18:24:05 <lmacken> looks like a virtualenv issue or something 18:24:09 <threebean> (if anyone wants to try running it.. use these instructions..) 18:24:11 <threebean> https://github.com/fedora-infra/rube/tree/develop/rube.fedora#running 18:24:26 <nirik> ok, fair enough 18:24:57 * lmacken got a stack trace when logging in to jenkins :( 18:25:02 <nirik> Also, I have been fighting a curl issue on f21... trying to move our builders to f21 but they sometimes hang in builds. ;( 18:25:05 <lmacken> javax.servlet.ServletException: java.lang.RuntimeException: Failed to serialize hudson.model.User#properties for class hudson.model.User 18:25:06 <webpigeon> oooh dear 18:25:13 <nirik> lmacken: ;( 18:25:16 * pingou ducks 18:25:31 <threebean> rock solid :p 18:25:43 <pingou> gotta love jenkins... 18:25:53 <threebean> I hear about teams doing really incredible stuff with jenkins, but it seems like we just don't have the person-power to keep it afloat. 18:26:06 <nirik> also this month we will be having some storage moving fun... no details or exact time yet, but hopefully this month... we will need an outage to move nfs server mounts. 18:26:26 <nirik> threebean: yeah. Keep wondering if buildbot would do better... 18:26:43 <nirik> #info storage moves likely later this month. 18:26:43 <lmacken> threebean: I think we'd have the same type of issue with elasticsearch 18:26:51 <lmacken> but I really want to use it :P 18:26:54 <nirik> #info smooge and nirik will be in phx2 datacenter in feb 18:27:09 * tflink might be able to help with buildbot 18:27:14 <smooge> hopefully :) 18:27:24 * smooge will be booking tickets later today 18:27:55 <pingou> the best point of Jenkins is its UI 18:28:07 <pingou> stability wise, we're not managing apparently :( 18:28:22 <nirik> there's also that mozilla one... tinderbox? dunno. 18:28:44 <lmacken> let's just run tests in taskotron :) 18:28:49 <pingou> \ó/ 18:28:51 <tflink> the new UI for buildbot nine will be much better 18:29:06 <pingou> tempting :) 18:29:38 <nirik> tflink: oh, while you are around: any thoughts about moving taskotron clients and other stuff thats f20 now to f21? or rhel7? or you want to keep it with 20 for now and move at f22? 18:30:01 <tflink> http://nine.buildbot.net/ 18:30:41 <tflink> I'd rather keep stuff as is for now - the bits we're currently working on will require some major client changes 18:31:01 <nirik> ok, fair enough. just wanted to mention it. ' 18:31:01 <tflink> sorry for being late, still have the pre-DST meeting time in my head 18:32:24 <nirik> #topic nagios/alerts recap 18:32:44 <nirik> I've been trying to fix up things that alert, but hard to say if it's having too much effect. ;) 18:33:18 <nirik> just need to keep plugging away. ;) 18:33:20 <nirik> #topic Upcoming Tasks/Items 18:33:20 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:33:31 <nirik> anything anyone has upcoming that they would like to schedule or discuss? 18:33:56 <pingou> dev-conf early february 18:33:59 <nirik> Do we want to look at a FAD for 2015? or is it too soon to talk about? 18:34:15 <nirik> pingou: you and puiterwijk are going? 18:34:16 <pingou> puiterwijk bochecha dgilmore and pbrobinson will be there 18:34:19 <pingou> and I 18:34:21 <nirik> ah, nice 18:34:29 <pingou> nirik: +1 for FAD 18:34:35 * pingou was thinking rel-eng tooling FAD 18:34:47 <threebean> also, PyCon US in April. 18:34:57 <threebean> pingou: and yeah, rel-eng tooling FAD would be sweet. 18:35:03 * nirik nods. 18:36:04 <nirik> #topic Open Floor 18:36:16 <nirik> anyone have anything for open floor? ideas, comments, suggestions? 18:36:47 * tflink is also going to be at devconf 18:36:53 <smooge> cool 18:36:55 <nirik> nice 18:37:27 * pingou adds tflink to the list :) 18:38:25 <nirik> alright, if nothing else will close things out in a minute... 18:38:32 <smooge> anyone going to be at FOSDEM? 18:38:55 <pingou> smooge: I will 18:39:28 <fale> smooge: I will if I'll be able to fit it with my exam schedule 18:39:47 <smooge> cool 18:40:00 <smooge> ok close it out 18:40:19 <nirik> alright. Thanks for coming everyone. 18:40:22 <nirik> #endmeeting