18:00:22 #startmeeting Infrastructure (2014-11-06) 18:00:23 Meeting started Thu Nov 6 18:00:22 2014 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:23 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:23 #meetingname infrastructure 18:00:23 The meeting name has been set to 'infrastructure' 18:00:23 #topic aloha 18:00:23 #chair smooge relrod nirik abadger1999 lmacken dgilmore mdomsch threebean pingou puiterwijk 18:00:23 Current chairs: abadger1999 dgilmore lmacken mdomsch nirik pingou puiterwijk relrod smooge threebean 18:00:42 * oddshocks is here 18:00:47 * danielbruno is here 18:00:48 * sebito91 is here 18:00:50 * relrod here 18:00:50 * puiterwijk is here 18:01:20 * mpduty is here 18:01:48 * pingou here 18:01:52 * threebean here 18:01:55 * adimania here 18:02:04 kushal is here 18:02:22 #topic New folks introductions and Apprentice tasks. 18:02:23 hello. 18:02:31 Any new folks like to introduce themselves? 18:02:39 * sebito91 waves hello 18:02:43 sritanu is here 18:02:44 or apprentices with questions, comments, suggestions, ideas, ? 18:02:55 hi team, i missed last week's meeting and therefore couldn't introduce myself 18:03:15 very excited to help out where i can...quick hits: very interested in metrics collection/analysis 18:03:45 currently working as an SA in NYC 18:04:15 sebito91: welcome! 18:04:24 oddshocks: thank you! 18:04:32 welcome sebito91! 18:04:46 we could definitely use some ideas around metrics and the like... 18:04:58 if I feel some application can be packaged how do I know whether it meets Fedora standards or requirements 18:05:01 I'm going to post a suggestion to the distro, but thinking of possibly adding a graphite or grafana based view to current collectors 18:05:20 nirik: thanks 18:05:41 mpduty: it's mostly a matter of looking at it and the guidelines... you can often also ask #fedora-devel or the devel mailing list for others to help... 18:06:36 nirik, okay 18:07:04 cool. 18:07:19 #topic Unfreeze reminder 18:07:29 just a reminder that we are currently NOT in freeze. ;) 18:07:40 we go back on the 18th... so not too much time really. 18:07:45 \o/ 18:07:47 #topic Applications status / discussion 18:07:58 any applications news or status to talk about? 18:08:45 I have some pull-requests pending review 18:08:56 on fedocal, pkgdb2 and anitya 18:09:08 during this non freeze it would be good to move stuff to ansible/rhel7 where possible/easy. ;) 18:09:14 mostly bug fixes :) 18:09:28 bodhi2 taskotron integration is done in dev. I'm hoping we can triage what tickets need to be closed before staging in the next week or so. 18:09:31 https://github.com/fedora-infra/bodhi/pull/109 18:10:26 on those pull-requests, anybody can review them. Just read through the code change and see if it looks sane. Ask questions about anything you don't understand or looks funny. 18:10:41 an easy way for apprentices to get involved on the apps/dev/maintenance side of the house. 18:10:43 yeah, it would be nice to get bodhi2 in stg where we can look at it mor, etc 18:11:16 cool. 18:11:41 abompard: any news on hyperkitty? I read that upstream was very happy with the sqlalchemy stuff. ;) 18:12:17 nirik: yeah, that branch has finally been merged. I've adapted hyperkitty to work with it 18:12:19 testing it now 18:12:23 pingou: do we have a timeline for announcing aytia more widely? 18:12:38 abompard: cool. So next step is likely moving some automated lists over? 18:12:43 (after that testing) 18:12:47 nirik: yes 18:13:03 nirik: as I won't be there next week, I didn't want to do it this week, but when I get back +1 for me 18:13:36 Oh, another thing to note: there's a new askbot version... if someone would like to rebase our patches and get it building we can look at upgrading. 18:13:56 #info bodhi2 pr's pending: https://github.com/fedora-infra/bodhi/pull/109 (great way to get started) 18:14:16 #info hyperkitty testing sqlalchemy changes 18:14:30 #info anytia announce sometime after next week possibly. 18:14:43 #info askbot upstream update. Help wanted to update el6 package in epel. 18:14:55 not much on the qa front - going to be looking into beaker again soon, expecting the blocking bug there to be fixed in the next month or so. still waiting on a warranty quote for some of the old qa0* boxes that we want to use for beaker clients 18:14:57 threebean: how is the-new-hotness doing? 18:15:07 it is just waiting to be deployed. 18:15:11 cool :) 18:15:34 so if we move anitya on, we can push a new pkgdb2 and deploy the-new-hotness :) 18:15:42 tflink: if we could possibly move blockerbugs to rhel7 when we ansible it that would be great. If you like I can test and see what might be missing... 18:15:46 going to be decomissioning some of the autoqa clients soon because the queues on taskotron keep getting longer than I'd like 18:15:55 nirik: what timeline are you thinking of? 18:16:09 Fedimg successfully uploaded good F21 beta AMIs for base and atomic cloud images, so I think it's about time to actually add in the ansible files I wrote and let it start doing automatic uploads this week, at least in stg 18:16:13 tflink: well, I could look in the next few days, but perhaps next week for migrating/ 18:16:14 ? 18:16:26 oddshocks: excellent. 18:16:42 #info fedimg seems to be working, next step to automate it starting in stg. 18:16:57 * tflink isn't against migration, just concerned about balancing that with the other tasks he's supposed to be working on 18:17:05 Those AMIs are on the fp.o page proper now, so they are public and the official ones we're offering for beta. 18:17:20 I suppose that it should be a pretty simple migration, though 18:17:47 tflink: hopefully, 1 change in ansible and re-run the playbook :) 18:17:59 pingou: blockerbugs isn't in ansible, though 18:18:17 its still puppet 18:18:26 isn't there a patch for that on trac? 18:18:35 yes... 18:18:36 https://fedorahosted.org/fedora-infrastructure/ticket/4392#comment:3 ? 18:18:41 pingou, yes 18:18:43 so, hopefully it will not be too bad... 18:18:46 but you never know. ;) 18:19:00 true enough :) 18:19:00 oh, I didn't see that ticket 18:19:08 tflink: I'm happy to do the bulk of the work on it... as long as you are around to ping if it blows up? 18:19:32 nirik: yeah, that's fine. we have a bug to fix in there, anyways 18:19:40 the app, I mean 18:19:57 ok. 18:20:08 #info nirik to work on migrating blockerbugs. 18:21:14 tflink: oh, another thing I wanted to run by you... but I guess it's more sysadminy... I want to re-install virthost-comm01.qa... I think it has no prod stuff on it, but would need an hour downtime for the re-install sometime. 18:21:40 the only thing that could affect me is bastion-comm01 18:21:46 but an hour isn't a problem 18:22:01 cool. will schedule that out of meeting... 18:22:11 any other application type news/ 18:22:11 ? 18:22:27 I have some questions about what exactly you have in mind but that sounds like an out-of-meeting topic 18:22:33 er, exact process 18:22:51 some of the stuff on virthost-comm01 isn't backed up and isn't easily reinstallable 18:23:54 backup libvirt info for guests, reinstall with rhel7 over the existing install (leaving lvm alone), bring up, redefine guests, bring them up. ;) 18:24:06 so, only the host should be affected. 18:24:14 the guests lv's stay the exact same 18:24:50 * nirik has done this with a number of other virthosts just fine. 18:25:05 #topic Sysadmin status / discussion 18:25:14 ok, wasn't sure if the reinstall involved rebuilding the guests 18:25:23 so yeah, I have been migrating things to rhel7/ansible where possible... I will keep doing that until freeze. 18:25:32 tflink: nope, it could, but it doesn't have to. 18:25:45 no need to tie them together 18:25:49 #info smooge going to USENIX LISA next week to work a booth and answer questions on EPEL 18:26:01 we had a mass reboot last night, went pretty well with a few hiccups. 18:26:16 smooge: do have fun! :) 18:26:42 #info mass reboot went pretty well aside puppetmaster restart issues and ns03 issues (still ongoing). 18:27:01 I'm also going to run the master playbook today... catch things up with ansible. 18:27:10 I mentioned this to smooge yesterday but we have 3 more hosts coming to phx2 soon (order approved, submitted to vendor). I'd guess an eta of 2-3 weeks 18:27:22 tflink: cool. :) 18:27:40 which is the rest of the stuff we ordered for FY15 18:28:01 nice. These are all qa0* taskotron virthosts? or were we going to make another virthost-comm? 18:28:26 I guess we can decide when we need to. ;) 18:28:49 nirik: not sure yet, to be honest 18:29:03 yeah. no need to decide really, we can adjust... 18:29:33 if they're compatible, it might make sense to add them as compute nodes to the cloud 18:29:33 #topic nagios alerts 18:29:57 tflink: we could possibly do that, although that might require some re-wiring as the cloud network is isolated. 18:30:06 tflink: we are also getting some more cloud nodes in Q4 18:30:12 yeah, that would be a different conversation, I think 18:30:14 https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=11&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=11&eday=6&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=3&hoststates=7&servicestates=120&limit=25 18:30:20 .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=11&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=11&eday=6&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=3&hoststates=7&servicestates=120&limit=25 18:30:21 that -> what we may end up doing with openstack 18:30:25 nirik: http://tinyurl.com/pzzwx2r 18:30:32 tflink: yeah, we really need to get our new one on-line. ;) 18:30:42 .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=10&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=10&eday=9&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=3&hoststates=7&servicestates=120&limit=25 18:30:42 puiterwijk: http://tinyurl.com/qevmn8u 18:31:00 threebean: you looked at the tagger thing right? what was that? 18:31:57 I have no idea what was going on with bodhi... lmacken has been traveling this week, but hopefully we can figure it out. 18:32:02 (or the reboots helped it) 18:33:23 the report looks more real when you only do hard states. 18:33:33 .tiny https://admin.fedoraproject.org/nagios/cgi-bin//summary.cgi?report=1&displaytype=3&timeperiod=last7days&smon=11&sday=1&syear=2014&shour=0&smin=0&ssec=0&emon=11&eday=6&eyear=2014&ehour=24&emin=0&esec=0&hostgroup=all&servicegroup=all&host=all&alerttypes=3&statetypes=2&hoststates=7&servicestates=120&limit=25 18:33:34 nirik: http://tinyurl.com/oz9c4ym 18:34:06 anyhow... 18:34:20 #topic Upcoming Tasks/Items 18:34:21 https://apps.fedoraproject.org/calendar/list/infrastructure/ 18:34:32 we have our FAD coming up in the beginning of december... 18:34:39 freeze on the 18th of this month. 18:34:51 anything else anyone would like to note or schedule? 18:35:10 I'll be away next week 18:35:26 no idea about my access to internet :) 18:35:32 alrighty. :) well, have fun... 18:35:53 will do :) 18:36:01 #topic Open Floor 18:36:10 anyone have any items for open floor? 18:37:00 alrighty then, will close out in a minute if nothing more. 18:38:00 Alright. Thanks for coming everyone! Do continue in #fedora-admin, #fedora-noc, and #fedora-apps 18:38:03 #endmeeting