18:00:26 <nirik> #startmeeting Infrastructure (2015-07-30)
18:00:26 <zodbot> Meeting started Thu Jul 30 18:00:26 2015 UTC.  The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:26 <nirik> #meetingname infrastructure
18:00:26 <nirik> #topic aloha
18:00:26 <nirik> #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk pbrobinson
18:00:26 <zodbot> The meeting name has been set to 'infrastructure'
18:00:26 <zodbot> Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pbrobinson pingou puiterwijk relrod smooge threebean
18:00:27 <nirik> #topic New folks introductions / Apprentice feedback
18:01:23 <nirik> any new folks like to introduce themselves? or apprentices with questions or comments?
18:02:06 <nirik> lots of people will be out today I'll note. Smooge is onsite at our datacenter, threebean and pingou couldn't make it, puiterwijk is off today.
18:02:10 <p_klos> It's quiet here, so I can make a comment
18:02:40 * rahulrrixe_ is here
18:02:46 * ^conner is here
18:02:58 * nyazdani is here
18:03:04 <p_klos> We will discuss that later but I think we should du a good review of tickets including easyfixes. A lot of them are outdated/inactual :(
18:03:19 <nirik> p_klos: yes lets wait until we actually get to that. ;)
18:03:30 <nirik> #topic GSoC student update - kushal
18:03:39 <nirik> any GSoC folks like to update on their progress?
18:03:45 * nirik isnt sure if kushal is around or not...
18:03:51 <rahulrrixe_> nirik: yes
18:04:12 <kushal> nirik, today yes.
18:04:19 <rahulrrixe_> Hi I have written a blog for recent work which I have done https://medium.com/@rahulrrixe/making-code-reviews-a-fun-task-391bfd04b6c2
18:04:20 <kushal> Which all students are here/
18:04:21 <kushal> ?
18:04:26 <nirik> hey kushal. ;)
18:04:34 <rahulrrixe_> Kushal: Hi :)
18:04:35 <AnuradhaW> Hi, I have resolved the error I had last time with the help of puiterwijk and I have modified my mockups by removing the hamburger menu and adding a footer like in getfedora.org.
18:04:37 <prth> i implemented the gzipped tarfile download of the wallpaper segments & will soon finish the progress bar when uploading
18:04:50 <kushal> I will create a spreadsheet with details from the students.
18:04:51 * sonalkr132 is here
18:04:54 <AnuradhaW> Kushal: Hi :)
18:04:57 <nirik> when is GSoC final?
18:05:02 <kushal> So that we know who all are coming in time.
18:05:48 <nirik> rahulrrixe_: opened in a tab to read later. ;)
18:06:00 <sonalkr132> As of my updates: I have been working on setting up the testing instance for glittergallery. So far, didn't have much luck with nginx. I am getting permission denied error for any directory other than default one used by nginx.
18:06:05 <kushal> Any other updates?
18:06:12 <rahulrrixe_> nirik: Thanks :)
18:06:28 <AnuradhaW> I had mid semester examinations in our univeristy this week so there haven't much work going on with integration this week.and how I resolved the error and modification on the mockups can be found here: http://anuradhanotes.blogspot.com/2015/07/got-issue-resolved-and-back-to-work.html
18:07:30 <nirik> sonalkr132: might check selinux denials?
18:08:55 <sonalkr132> nirik: I gave ngnix user permissions it needed and chmod 755 the dir I wanted to use. I am not sure what else I would need.
18:09:09 <^conner> sonalkr132, is selinux enabled?
18:09:10 <nirik> sonalkr132: selinux permissions? does it work if you do 'sudo setenforce 0'
18:09:56 <nirik> anyhow, we can help debug that out of meeting in #fedora-apps or the like. ;)
18:10:01 <nirik> anyone else?
18:10:07 <sonalkr132> nirik: thanks.
18:10:13 <sonalkr132> I will find someone there
18:10:50 <nirik> kushal: shall we move on? or are you looking for anyone else?
18:12:53 <nirik> #topic announcements and information
18:12:53 <nirik> #info old cloud to new cloud migration moving along, hopefully done by friday - kevin/patrick
18:12:53 <nirik> #info ibiblio05 installed and ready for service - kevin
18:12:53 <nirik> #info virthost19/20/21/22 installed and ready for service - smooge
18:12:53 <nirik> #info smooge onsite at phx2
18:12:54 <nirik> #info bapp02/app01.stg now gone. Down to just 9 hosts in puppet - kevin
18:12:55 <nirik> #info We are now in Fedora 23 Alpha freeze! - kevin
18:12:57 <nirik> #info jenkins-dev in new cloud should be ready for move to production, more details on mailing list - mizdebsk
18:13:00 <nirik> #info migrate planet.fedoraplanet.org to planetfedora.org - https://fedorahosted.org/fedora-infrastructure/ticket/4670 - tyll
18:13:03 <nirik> #info migrated qa stuff off the old cloud that needs to be migrated - tflink
18:13:09 <nirik> anything in the info dump anyone wants to discuss further or note anything about?
18:13:36 <nirik> ok, moving on then to discussion...
18:13:46 <nirik> #topic TRAC tickets review - p_klos
18:14:00 <nirik> p_klos: you wanted to review older tickets? I think it's a great idea...
18:14:09 <p_klos> Yes,
18:14:13 <nirik> however with so many people gone today we might not have background on some of them.
18:14:36 <^conner> I also attempted to look at quickfixes last week and ran into two issues
18:14:47 <p_klos> yes, I'm sad that pingou and puiterwijk can't be with us today.
18:14:49 <roshi> I think a good pruning would be good - at least to udpate the tickets
18:14:54 <^conner> * the issues seemed to be stale, resolved, or no longer relevant
18:15:14 <^conner> * I found that virtually any host mentioned I couldn't ssh into
18:15:21 <p_klos> I habe taken some tickets but when crawling through them found many inactual
18:15:31 <nirik> we are also bad at filing new ones... for several reasons: a) sometimes it's just easier to do something simple and get it done, b) we are all really busy so it's hard to think of things for new folks often.
18:15:33 <roshi> when I was looking through for easy fixes I didn't immediately find any that seemed relevant and so haven't been able to knock any of them out
18:15:35 <kushal> nirik, oops, sorry, too many meetings at the same time :(
18:15:53 <^conner> I confirmed I can get a shell on the bastion but stopped for fear of triggering an IDS if I started trying to sh into every host listed in icinga
18:16:13 <p_klos> We're all busy :)
18:16:14 <nirik> ^conner: there's no ids. ;)
18:16:26 <nirik> anyhow, lets take a look at a few each meeting and try and sort them out?
18:16:40 <p_klos> But as an apprentice I was looking around to find something I could do
18:16:50 <p_klos> ;)
18:17:13 <p_klos> nirik: sounds good! I can make a list for each meeting
18:17:33 <p_klos> after a few months we well clean everything
18:17:54 <nirik> there's also some things that would be great for people to do like docs updates, but new folks don't have the info to update them... so thats a bit of a bad circle.
18:18:07 <p_klos> right
18:18:08 <^conner> indeed
18:18:14 <mizdebsk> from my experience fi-apprentice has access to only few machines, for some of them i've been asking why there is no access, but weren't given any definitive answer
18:18:35 <nirik> mizdebsk: really? where did you ask?
18:18:43 <^conner> we have the same issue with new hires, trying to come up with projects that are reasonably self contained
18:18:45 <mizdebsk> #fedora-admin
18:18:47 <nirik> in any case apprentice should have access to most machines...
18:18:59 <nirik> well, if you get no answer, please file a ticket or ask again when people are around.
18:19:03 * nirik does sleep sometimes.
18:19:20 <roshi> pics or it doesn't happen nirik :p
18:19:21 <^conner> nirik, how about I war-drive what's listed in icinga and report to the list?
18:19:38 <nirik> ^conner: well, it should be very easy to get a list from ansible. ;)
18:20:15 <^conner> oh, right, the inventory
18:20:20 <nirik> look for 'fas_client_groups' variable that contains fi-apprentice
18:20:28 <mizdebsk> examples of machines within my interest, where apprentice has no access: jenkins, copr, anitya, koji (this one is documented)
18:20:31 <nirik> in fact, the scripts/freezelist could be modified to list them
18:21:04 <nirik> I have no problems added apprentice to the first 3 of those.
18:21:53 <mizdebsk> cool, i will prepare patch
18:21:55 <nirik> anyhow, should we look at some tickets today? or start next week?
18:22:13 <nirik> mizdebsk: often we just forget. Perhaps we should make apprentice opt-out rather than opt in.
18:22:20 <^conner> https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/inventory/group_vars/secondary?id=72d3038598e9b7f48401954dd562b9685194d2ec
18:22:23 <nirik> ie, it's always assumed unless listed.
18:22:24 <p_klos> I've pasted 3 examples to Gobby. We could look...
18:22:27 <p_klos> :)
18:22:46 <nirik> p_klos: sure. ;)
18:22:56 <p_klos> the first #link https://fedorahosted.org/fedora-infrastructure/ticket/1684
18:22:58 <mizdebsk> nirik: that's a good idea
18:23:08 <nirik> so, lets start with the oldest?
18:23:11 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/1180
18:23:19 <p_klos> ok
18:23:23 <nirik> this is still the case, but we are moving to mailman3.
18:23:38 <nirik> so, I suppose we could close this and say that, but it's not done yet.
18:23:50 <nirik> there's not really anything for anyone to do other than help abompard migrate us. ;)
18:24:03 <p_klos> when will end migration to mailman3?
18:24:10 <nirik> we don't have a set date.
18:24:22 <nirik> we ran into some bugs that abompard is working on before we move further on
18:24:33 <nirik> some lists are moved already
18:24:46 <^conner> does it still require python2 and pyhton3?
18:24:51 <p_klos> So maybe would be better to support abpompard? :)
18:25:22 <nirik> ^conner: mailman3 uses python3
18:25:53 <nirik> p_klos: yep. So, lets close this with "we are moving things to mailman3, archives should be much nicer then, please file bugs on hyperkitty if you want changes there"
18:26:14 <nirik> you want to do that, or shall I?
18:26:29 <p_klos> If I have your permission ;)
18:26:33 <nirik> sure!
18:26:52 <p_klos> So I'll do just after meeting.
18:27:18 <nirik> ok, sounds good.
18:27:21 <nirik> ok, next:
18:27:25 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/1684
18:27:36 <nirik> so, we still have a bunch of .wsgi's in ansible
18:28:08 <p_klos> ok. So I'll grep for them
18:28:21 <nirik> so, I am not sure if that applies tho. I am not sure I follow the reason for the ticket. ;)
18:28:36 <nirik> the apps toshio mentioned in the first set are all done I think.
18:28:40 <p_klos> ticket was commented that only bodhi left and doesn't need any fixe ;)
18:29:25 <nirik> yeah, Toshio (who filed the ticket) is no longer around...
18:29:41 <^conner> ack finds a bunch of wsgi hits in the ansible tree
18:29:47 <nirik> so I guess I'd say close this one too and perhaps we could ask pingou / threebean / lmacken if it needs reopening.
18:30:03 <p_klos> ok, I will.
18:30:12 <nirik> all our new apps are not turbogears too, so I am not sure this applies to them.
18:30:32 <nirik> and a few apps (like geoip-city) actually don't have a wsgi upstream, so we have to make one locally.
18:30:54 <threebean> oh my, I wasn't aware of that ticket.
18:31:03 <nirik> hey threebean
18:31:14 <threebean> and recently I've added some logic to the .wsgi files we push out.
18:31:15 <nirik> it's really old. I don't know if it's turbogears specific or what
18:31:55 <threebean> it may have been feasible and desirable to do before.. but now we'd have to also un-do some of that fedmsg stuff I was working on.
18:32:21 <threebean> (this new stuff is connecting the number of processes & threads that mod_wsgi gets with the number of fedmsg endpoints we declare and the number of ports we open in iptables..)
18:33:01 <nirik> ok. So you want to poke at that ticket and close or adjust it?
18:33:21 <nirik> (and we should likely remove easyfix until we know what if any fixes we have there)
18:33:43 <threebean> so, I guess I'd vote to close it.  I'm a fan of the new feature that would have to be un-done.  but.. I'm open to discussion about it.
18:34:25 * threebean will close it after the meeting unless someone objects.
18:35:44 <p_klos> Ok, and the last? https://fedorahosted.org/fedora-infrastructure/ticket/3627
18:36:05 <nirik> yeah, another one we were not sure about...
18:36:22 <nirik> I think we agreed in a previous meeting to check apps.fedoraproject.org apps against logs on log01
18:36:58 <nirik> so I guess we could update it with that info?
18:37:45 <p_klos> sounds ok
18:37:59 <nirik> cool. can you do that? or would you like me to?
18:37:59 <threebean> here's a text file with the list of all the apps on apps.fp.o if that helps -> http://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/apps-fp-o/files/apps.yaml
18:38:49 <p_klos> I could check the logs but unfortunately it looks like I don't have permissions to log01...
18:39:46 <nirik> p_klos: apprentice is definitely allowed there.
18:39:56 <p_klos> hmmm...
18:40:07 <nirik> make sure you setup ssh to proxy via bastion01
18:40:47 <p_klos> yes, I have the proxy :)
18:40:51 <^conner> log01.phx2.fedoraproject.org correct?
18:40:55 <p_klos> nirik: is it log01.phx2.fedoraproject.org?
18:40:55 <nirik> yep.
18:40:59 <p_klos> oh correct
18:41:14 <nirik> anyhow, we can sort it after the meeting in #fedora-admin? you should definitely have access.
18:41:49 <p_klos> sure. I have Could not resolve hostname log01.phx2.fedoraproject.org: No address associated with hostname
18:42:10 <p_klos> maybe sth with dns? doesn't matter. We will look after meeting
18:42:10 <nirik> then your ssh config is trying to resolve it locally rather than using the proxy in ~/.ssh/config
18:42:23 <nirik> yeah.
18:42:34 <nirik> ok, shall we move on then... and do some more tickets next week?
18:42:34 <^conner> p_klos, it works for me via the ssh bounce
18:42:40 <^conner> nirik, please
18:42:49 <p_klos> I've got ssh: connect to host bastion.fedoraproject.org port 22: Connection timed out
18:42:52 <p_klos> ssh_exchange_identification: Connection closed by remote host
18:43:05 <p_klos> lockbox and other hosts are ok
18:43:13 <nirik> thanks for the idea. It's great to clean up old tickets. ;)
18:43:20 <p_klos> so, leave it for after-meeting :)
18:43:35 <^conner> nirik, I'd propose that become a regular meeting agenda item
18:43:46 <nirik> ^conner: sure, we can give it a go for a bit. ;)
18:43:51 <nirik> #topic Idea: We could create official Fedora Project Docker repo - p_klos
18:43:59 <nirik> so, this has actually come up before. ;)
18:44:17 <^conner> I vote yes :)
18:44:21 <nirik> it's kind of beyond our scope tho. It needs releng and cloud working group to coordinate on what they want.
18:44:25 * p_klos vote yes also ;)
18:44:28 <^conner> I use centos:6 & centos:7 constantly
18:44:37 <nirik> we aren't going to make something without them telling us what they want.
18:44:59 <nirik> so, I'd say (re)open discussions with those groups. ;)
18:45:10 <nirik> I know it's been discussed before, but I dont have any idea on the status
18:45:18 <^conner> so the cloud sig is the place to start?
18:45:34 <nirik> yeah and then releng I think...
18:45:37 <p_klos> OK, I will connect to releng meeting and try to discuss about ;)
18:46:04 <nirik> p_klos: there may already be people working on things in cloud working group. I don't know off hand.
18:46:33 <nirik> I know we talked about it also because fedora is shipping 32bit and secondary arch docker stuff now and the official one doesn't do that
18:46:34 * p_klos will make small investigation ;)
18:46:50 <nirik> sounds good. ;)
18:47:02 <nirik> once we know what people want/need we can look at what it will take to run a service.
18:47:46 <nirik> #info p_klos to talk to cloud working group and releng to see what status is
18:47:50 <nirik> anything else on this one?
18:48:14 <^conner> who wons https://registry.hub.docker.com/_/fedora/ ?
18:48:21 <^conner> s/wons/owns/
18:48:58 <nirik> dunno
18:49:18 <p_klos> weird...
18:49:41 <nirik> #topic Open Floor
18:49:44 <^conner> I think docker must be maintaining them
18:49:47 <nirik> ok, anything for open floor?
18:49:55 <^conner> https://github.com/docker-library/docs/tree/master/fedora
18:49:59 <nirik> ^conner: I think cloud working group and/or releng has some accounts, etc.
18:50:05 <nirik> but I have 0 idea which one is which
18:51:04 <sonalkr132> just an update. changing SELINUX=enforcing to permissive did the trick. Thanks nirik, ^conner
18:51:12 <^conner> I'm still looking for a good starter project.
18:51:22 <^conner> Ended up short on time the last few days
18:51:28 <mizdebsk> note: jenkins should be ready for migration to new cloud
18:51:44 <^conner> sonalkr132, `ls -z <file> will show you the selinux labels
18:51:53 <mizdebsk> i can do vast of migration myself if that's what i'm expected to do, but i would need someone from sysadmin-main to assist if something goes wrong
18:52:30 <sonalkr132> ^conner: thanks.
18:52:57 <mizdebsk> so my idea was to schedule an outage some time and migrate stuff
18:53:04 <mizdebsk> does that sound ok?
18:53:18 <nirik> mizdebsk: sounds good. I should be able to assist.
18:53:55 * nirik just filed a new easyfix: https://fedorahosted.org/fedora-infrastructure/ticket/4848
18:54:16 <mizdebsk> i'm not sure how to do redirect (proxies and dns are in freese iirc)
18:54:27 <nirik> mizdebsk: jenkins is not in proxies
18:54:52 <mizdebsk> right, but dns
18:55:01 <nirik> updating dns is fine.
18:55:09 <mizdebsk> ok, great
18:55:11 <nirik> it's changing dns config that would need a freeze break
18:55:45 <nirik> I am hoping now that we can decomission the old cloud next week...
18:56:11 <nirik> ok, if nothing else, will close out in a minute.
18:57:11 <nirik> Thanks for coming everyone. ;)
18:57:14 <nirik> #endmeeting