18:00:35 <nirik> #startmeeting Infrastructure (2014-07-17)
18:00:35 <zodbot> Meeting started Thu Jul 17 18:00:35 2014 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:35 <nirik> #meetingname infrastructure
18:00:35 <nirik> #topic aloha
18:00:35 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk
18:00:35 <zodbot> The meeting name has been set to 'infrastructure'
18:00:35 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean
18:00:40 * pingou 
18:00:41 * lmacken 
18:01:00 <oddshocks> holla
18:01:01 <dgilmore> hola
18:01:14 <oddshocks> haha that was pretty good
18:01:17 <smooge> here
18:01:17 * mirek-hm waves
18:01:26 * hammad_ is here.
18:01:38 * threebean is here, off working on a problem
18:01:43 * charul is here
18:01:48 <ootbro> here
18:01:57 * relrod here, working on noc01 stuff
18:02:17 <nirik> morning everyone. ;)
18:02:19 * adimania is here
18:02:22 * abadger1999 here
18:02:29 * lanica is here for the infrastructure meeting.
18:02:32 <lanica> good morning
18:02:52 <nirik> #topic New folks introductions and Apprentice tasks
18:03:14 <nirik> any new folks want to give a short introduction?
18:03:22 <nirik> or apprentices with questions, comments, ideas?
18:03:30 <ootbro> nirik: (from the apprentice realm) ....  I got your e-mail with how to pull the proxy configs from puppet and will work through it very shortly.
18:03:59 <nirik> ootbro: excellent. :) I'm looking forward to what you are putting together there. I think it will definitely help new folks see how things are setup.
18:04:35 <ootbro> scaling back the scope for Phase 1 makes it easy -- draw a picture  :P
18:05:01 <christianh814> Hi, my name is Christian and I am here in hopes to volunteer my time to the Infrastructure team :)
18:05:23 <pingou> hi christianh814, welcome :)
18:05:23 <christianh814> Been a sysadmin for about 8 years now and would like to give back to the community
18:05:37 <nirik> welcome christianh814
18:05:54 <nirik> christianh814: are you more interested in sysadmin work, or application development? or both?
18:06:27 <christianh814> A little of both. Mostly sysadmin but been playing around with rails/ruby and python too
18:07:08 <nirik> ok, cool. See me in #fedora-admin after the meeting and I can add you to our apprentice group and point you at our easyfix tickets.
18:07:55 <christianh814> nirik: okay will do! Thanks
18:08:55 <nirik> cool. :) Any other new folks or apprentices with questions?
18:09:03 <charul> Hi everyone. I am Charul, working on Shumgrepper as GSoC intern. I want to update my progress.
18:09:12 <pingou> awesome :)
18:09:17 <charul> I have implemented the following till now - comparison of packages and tar_files and display common and different files, get files by shum's value, display files of a package and tar_file and diplay packages list.
18:09:17 <threebean> hi charul
18:09:25 <charul> threebean: hi
18:09:34 <charul> Currently i am working on implementation of a search feature on packages list page and to get the history of a package.
18:09:46 <charul> I wrote a blog yesterday on overview of all the functionalities of shumgrepper http://t.co/gbJXXsOl5b.
18:09:51 <pingou> sweet :)
18:10:04 <charul> url for dev instance of shumgrepper http://209.132.184.120/
18:10:13 <nirik> cool.
18:10:23 * nirik saw the blog post, but hasn't had a chance to dig into it too much
18:10:34 * pingou only saw it today
18:10:44 <lmacken> it looks great, nice work charul :)
18:10:51 <oddshocks> charul: hi!
18:11:14 <charul> pingou, nirik: please do have a read and give your views
18:11:21 <charul> lmacken: thanks :)
18:11:24 <nirik> will try and do so. ;)
18:11:28 <charul> oddshocks: hi :)
18:11:37 <pingou> charul: sure :)
18:11:38 <nirik> when is GSoC over?
18:11:51 <charul> nirik: by mid of August
18:11:53 <lanica> charul: very cool, but how does it handle binary files that have the "preloaded stuff" which changes the md5sum?
18:12:34 <pingou> lanica: ?
18:12:35 <nirik> ok, cool.
18:13:04 <charul> lanica: Sorry, i didn't got your point. I have not considered it. Can you explain ?
18:13:27 <lanica> I suppose its more of a question for whatever collects the md5sums
18:13:44 <lanica> "prelink"
18:13:57 <lanica> prelink can cause md5sums of installed binaries to be different across different systems
18:13:57 <nirik> lanica: the sums are gotten from source uploads...
18:14:05 <nirik> so, we are assuming those are fixed and do not change.
18:14:48 <nirik> this is also for source files, not binaries I think
18:15:35 <lanica> Ahh ok.  Anyway, something to think about/test if it does apply to binaries once installed if prelink is to be involved.
18:16:08 <pingou> lanica: well, it does not apply to binaries, only the sources :)
18:16:54 <nirik> sure.
18:17:03 * nirik removes prelink from everywhere. ;)
18:17:10 <nirik> #topic Applications status / discussion
18:17:12 <lanica> Problem solved ;)
18:17:17 <nirik> any applications news this week?
18:17:21 <pingou> new pkgdb2 in prod today
18:17:30 <pingou> as well as fedocal, elections and nuancier yesterday
18:17:42 <pingou> so allow dgilmore to log in these apps :)
18:18:12 <nirik> :)
18:18:21 <dgilmore> :(
18:18:29 <tflink> the firewall rules for taskotron went through today so taskotron staging should be up and running later today, hopefully
18:18:34 <oddshocks> Fedimg will be in staging this week. Right at this moment I'm working on the .spec file and other packaging stuff
18:18:34 <pingou> dgilmore: come on, be happy, now you can log in :)
18:18:37 * dgilmore is the breaker of the web
18:18:48 <mirek-hm> I upgraded copr-be to F20
18:18:49 <pingou> dgilmore: and we thank you for that ;)
18:18:55 <mirek-hm> so all copr machines are now F20
18:19:31 <nirik> mirek-hm: excellent. ;)
18:19:49 <nirik> #info copr-be updated to f20
18:19:52 <pingou> oh, and I fixed the git hook in progit to allow closing/refering to a ticket from the git commit messages :)
18:20:09 <nirik> #info new releases this week: pkgdb2, fedocal, elections, nuancier.
18:20:16 <pingou> I think we also rolled-out a new release of FAS earlier this week
18:20:16 <nirik> #info taskotron stg should be up very soon.
18:20:31 <nirik> #info fedimg should be in staging soon too.
18:20:40 <nirik> #info fas update rolled out.
18:20:42 <nirik> yes.
18:20:53 <nirik> fixes the initial groups bug we have been dealing with for years. :)
18:22:07 <nirik> any other applications news?
18:22:36 <nirik> oh, there was a call for wider testing for FMN
18:23:14 <pingou> which seems to have worked already :)
18:23:28 <nirik> yeah, saw a bunch of folks tweaking settings.
18:24:27 <nirik> #info please test FMN, https://apps.fedoraproject.org/notifications
18:24:46 <nirik> #topic Sysadmin status / discussion
18:24:54 <nirik> lets see... lots of stuff ongoing.
18:25:06 <nirik> we have been migrating things to ansible and rhel7 where possible.
18:25:30 <nirik> I've been focusing on moving instances off a few virthosts we plan to decomission next week.
18:25:42 <nirik> Still in process:
18:26:10 <nirik> log02 -> log01... I need to get a epylog build from smooge, and migrate a few scripts over, and data rsync needs to finish.
18:26:30 <nirik> also, we had awstats on log02, I guess I can set that up on the new one, but it's not really very great.
18:27:02 <nirik> db04/koji03 (this is our main koji hub/db): I'm going to try and live migrate them some evening soon... hopefully no disruption.
18:27:13 <nirik> noc01: relrod is working on this even as we speak.
18:27:21 <mirek-hm> I had big problem with openstack in Copr. I had to recreate floating IPs (again). As usually I fidlle whole day with that and then problems suddently dissappeared.
18:27:50 <nirik> mirek-hm: ugh. Yeah, we should really try and move forward on new cloud. Just everyone has been so swamped. ;(
18:28:24 <adimania> Apologies if this has been discussed before but are we planning to have a FAD for ansible migration during flock?
18:28:31 <mirek-hm> nirik: if you can give me access to those two machines, I can give it try.
18:29:08 <nirik> adimania: I floated the idea, but I don't think we have time before/after... we might try and do some at flock, but not sure if there's a formal timeslot.
18:29:14 <nirik> adimania: you going to make it to flock?
18:29:22 <adimania> yes,
18:29:25 <nirik> mirek-hm: ok. I can try and do so after the meeting.
18:29:32 <mirek-hm> ook
18:29:34 <nirik> mirek-hm: its actually 3 machines, we got all of them up.
18:30:11 <nirik> mirek-hm: we still don't have our storage up tho we are going to try and set that up next week on site
18:30:34 <mirek-hm> roger that
18:30:48 <nirik> but they do have local disk, so we should be able to set them up at least as a test.
18:32:01 <nirik> lets see...
18:32:13 <nirik> #info we got more networks for phx2. They are in process of being setup
18:32:21 * michel_slm is here (apologies, overslept, was lurking until there's a gap in discussions)
18:32:52 <ootbro> nirik:  more networks means more IP addresses for the phx2 dc ?
18:33:00 <smooge> maybe
18:33:02 <nirik> michel_slm: welcome.
18:33:18 <nirik> ootbro: yeah... it will eventually. I can let you know when I know for sure.
18:33:32 <michel_slm> nirik: thanks. will introduce myself at the end if there's time
18:33:44 <nirik> michel_slm: sure. :)
18:34:01 <nirik> michel_slm: or you can go now if you prefer?
18:34:47 <ootbro> nirik: sounds good.  I'll pull a fresh copy of the DNS zone files when I get closer to building that inclusive server list.  (From the list of services via proxy -- which I just sent you in e-mail -- I may be busy writing for a while ;) )
18:35:23 <nirik> ok.
18:35:33 <nirik> and new feature for the meeting:
18:36:03 <pingou> (and especially for you tonight <drums...>)
18:36:10 <michel_slm> after this announcement ;)
18:36:11 <nirik> OS scorecard: 181 RHEL6 instances, 43 RHEL7 instances, 144 Fedora 20 instances, 4 Fedora 19 instances
18:36:27 <nirik> so we are already making some good headway to rhel7 migrations.
18:36:32 <adimania> yay!
18:36:33 <oddshocks> right on
18:36:34 <mirek-hm> wow
18:36:34 <pingou> nirik: including the private cloud?
18:36:38 * lanica cheers.
18:37:01 <nirik> pingou: some, only things in ansible... not transient/copr builders/etc.
18:37:07 <nirik> also: 88 hosts left in puppet.
18:37:17 <nirik> #info OS scorecard: 181 RHEL6 instances, 43 RHEL7 instances, 144 Fedora 20 instances, 4 Fedora 19 instances
18:37:18 <pingou> nirik: ok, so the jenkins instances are in :)
18:37:23 <nirik> #info 88 hosts left in puppet
18:37:28 <michel_slm> is the goal to move F19 -> RHEL 7, or to F20?
18:37:36 <nirik> yeah, one of the 19's is the jenkins f19 builder
18:38:16 <nirik> michel_slm: to rhel7 for the most part...
18:38:30 <nirik> but things like the f19 jenkins builder will be f19 until it goes end of life
18:38:37 <michel_slm> aha, ok
18:39:08 <nirik> other goals tho include 0 hosts in puppet, and 0 rhel6 instances (aside again some jenkins builder or something)
18:39:31 <pingou> that would be sweet :)
18:39:57 <nirik> we are getting there. ;)
18:40:08 <nirik> #topic nagios/alerts recap
18:40:21 <nirik> this might be down.
18:40:26 <nirik> due to our migrating noc01. ;)
18:42:05 <nirik> #topic Upcoming Tasks/Items
18:42:05 <nirik> https://apps.fedoraproject.org/calendar/list/infrastructure/
18:42:17 <nirik> anything upcoming anyone would like to note or schedule?
18:42:28 <nirik> I'll note that smooge and I will be in phx2 next week... tuesday->friday.
18:42:49 <threebean> week after that I'm on vacation ;)
18:42:51 <nirik> we may not be as available online as we are traveling or off wandering the racks.
18:43:24 <smooge> 2-3 weeks from now we will be in Prague?
18:43:24 <relrod> I'll be away next week at a conference from late 21st until the 24th sometime
18:43:29 <relrod> I'll have connectivity but limited
18:43:36 <nirik> we are supposed to enter the f21 alpha freeze on the 22nd. However, I am going to push that to the 25th (friday)
18:44:02 <relrod> and I'll have my phone, call/text in an emergency if needed
18:44:03 <nirik> smooge: yeah, then lots of people traveling at the beginning of aug to flock
18:45:05 <nirik> For those that haven't been around for a freeze, it means that for any host that is marked 'freezes' in ansible, you need to post changes to the list and get at least 2 +1's from sysadmin-main or sysadmin-releng before you make that change.
18:45:18 <nirik> this is to make sure we don't destablize something before a release.
18:46:02 <nirik> Also, while folks are away and traveling if you see someone asking about something and can't solve it, please ask them to file a ticket and be patient. ;)
18:46:14 <nirik> #topic Open Floor
18:46:23 <nirik> anyone have any items for open floor? michel_slm ?
18:46:25 <michel_slm> here goes my deferred intro
18:46:36 <michel_slm> hi everyone. I'm Michel, salimma on FAS
18:47:15 <michel_slm> been a long time packager / provenpackager / sponsor, but new to infra. figured I'd chip in seeing as I've been doing this on the day job for several years
18:47:23 <nirik> welcome michel_slm! :)
18:47:49 <michel_slm> interested in maintenance and app dev in Python - I'm familiar with Django; not so much with TG2 (do we still use that?)
18:48:02 <pingou> michel_slm: mostly flask and pyramid these days
18:48:26 <michel_slm> ah, awesome. been meaning to use those more
18:49:02 <nirik> we do have 2 Django apps actually tho...
18:49:04 <michel_slm> so yes. nirik added me to fi-apprentice a couple of days ago, and I'm still looking around. if someone has an #easyfix they want looked at that fits, ping me
18:49:17 <nirik> mailman3/hyperkitty and askbot
18:49:39 <pingou> we can still use some help on askbot or is this covered?
18:49:51 <michel_slm> so one production app and one in .. staging (hyperkitty)?
18:50:02 <nirik> yeah, we could def use help on askbot... ;)
18:50:07 <nirik> aand hyperkitty.
18:50:17 <michel_slm> askbot just lost my login so I have a personal itch to scratch there
18:50:20 <nirik> yeah, ask is in prod, hyperkitty is in stg mostly
18:50:38 <pingou> michel_slm: fedoauth changed from your personnal email to user@fp.o
18:50:40 <nirik> I think the production rhel7 version is up for testing now.
18:50:50 <pingou> michel_slm: I would not be surprised if this is the source of your itch
18:51:20 <nirik> I can assist you looking at it after the meeting (or later today when I get time,, etc)
18:51:45 <michel_slm> sounds likely. ankur was looking at it but I had to leave for home so didn't get to hear the answer - sure, after the meeting
18:51:48 <oddshocks> michel_slm: welcome :)
18:51:54 <michel_slm> oddshocks: thanks
18:52:00 <nirik> fair enough.
18:52:08 <nirik> ok, anything else for open floor?
18:52:57 <nirik> ok, will close out in a minute... thanks everyone for coming. ;)
18:53:16 <oddshocks> thanks nirik
18:53:21 <nirik> #endmeeting