18:00:57 <smooge> #startmeeting Infrastructure (2017-11-30)
18:00:57 <zodbot> Meeting started Thu Nov 30 18:00:57 2017 UTC.  The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:57 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:57 <zodbot> The meeting name has been set to 'infrastructure_(2017-11-30)'
18:00:57 <smooge> #meetingname infrastructure
18:00:57 <zodbot> The meeting name has been set to 'infrastructure'
18:00:57 <smooge> #topic aloha
18:00:57 <smooge> #chair smooge relrod nirik dgilmore threebean pingou puiterwijk pbrobinson maxamillion
18:00:57 <zodbot> Current chairs: dgilmore maxamillion nirik pbrobinson pingou puiterwijk relrod smooge threebean
18:01:11 <marc84> hi everyone
18:01:18 <netcronin> hi all
18:01:33 * pingou here :)
18:01:51 <smooge> hello. This will be a pretty short meeting today. I forgot to send out an announcement
18:01:54 <nirik> morning everyone.
18:02:45 * jcline waves
18:02:52 <smooge> #topic New folks introductions
18:02:53 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
18:03:26 * cverna waves
18:03:39 * relrod waves
18:03:44 <smooge> hello.
18:04:19 <smooge> #topic announcements and information
18:04:19 <smooge> #info PHX2 Colo Trip, Dec 4th - 9th
18:04:19 <smooge> #info 2017-12-06 meeting cancelled
18:04:19 <smooge> #info new fedora-packages on https://apps.stg.fedoraproject.org/packages/ still few minor things to fix.
18:04:20 <smooge> #info new pagure put in production?
18:04:27 <pingou> new pagure is out yes
18:04:31 <nirik> yep. it was.
18:04:44 <smooge> thanks. I remembered the announcements so stuck it in there but wasn't sure
18:05:04 <pingou> I didn't send it
18:05:06 * pingou does so
18:05:40 <smooge> any other announcements? for this week?
18:05:42 <nirik> thanks to puiterwijk and pingou and cverna packages is working in stg... just needs some tweaking and then hopefully we can upgrade production
18:05:57 <pingou> that'll be nice for sure :)
18:06:05 <nirik> yeah. very
18:06:18 <smooge> oh and wiki was updated?
18:06:28 <nirik> yes.
18:06:39 <smooge> #info wiki updated to newer mediawiki
18:06:48 <cverna> we might stop to have weekly ticket about missing packages :)
18:06:54 <nirik> 1.29.1
18:07:03 <smooge> #info proxies being moved to Fedora versus EL7
18:07:43 <nirik> there's still a bunch more of them to do...
18:08:02 <nirik> but the ones that are fedora now are providing http/2 so should be a bit faster.
18:08:33 <pingou> cverna: ticket*s*
18:09:14 <smooge> ok next up
18:09:15 <nirik> Note that next week is the big datacenter move...
18:09:18 <smooge> #topic Ticket cleanup
18:09:18 <smooge> #info none this week.
18:09:18 <smooge> #topic REMINDER INFRASTRUCTURE WILL BE DOWN NEXT WEEK
18:09:28 <nirik> yeah, that. ;)
18:09:39 <nirik> https://www.fedorastatus.org/q4maint.html
18:09:41 <smooge> do you have a link to your email
18:09:47 <smooge> yeah that
18:09:59 <smooge> #info please read through https://www.fedorastatus.org/q4maint.html
18:10:09 <nirik> smooge: perhaps I should file an outage ticket and add the announcements and status there? or do we need one?
18:10:28 <smooge> I was going to file an outage ticket after this meeting but if you want ot :)
18:10:53 <nirik> if any apprentices want to help out, one big thing would be watching for people with questions and pointing them at the status page, etc... on #fedora-admin, social media, etc.
18:11:02 <smooge> yes..
18:11:41 * pingou notes: pagure.io is unaffected :-p
18:11:44 <nirik> I can file a ticket after meeting.
18:12:04 * smooge fixes that bug with pagure.io
18:12:06 <nirik> yeah, I noted a (Partial) list of those in the email
18:12:15 <nirik> ie, things not affected
18:12:27 <pingou> pagure not being in phx2 it should be fine, as long as you don't need to log in :D
18:12:54 <nirik> true.
18:13:55 <smooge> getting sudo on external hosts will not work during certain times
18:14:11 <pingou> hm good point
18:15:02 <smooge> anything else here?
18:15:19 * pingou has something buf for later
18:15:21 <pingou> but*
18:15:59 <smooge> #topic Pingous Later Item
18:15:59 <nirik> it would be nice to have batcave13 up and able to push dns.
18:16:29 <pingou> so last week or the one before Patrick and I got simple-koji-ci work in stg
18:16:47 <pingou> I was wondering if we want to try it for real :)
18:16:57 <pingou> https://pagure.io/fedora-ci/simple-koji-ci
18:17:19 <pingou> for those who don't know it, it's a simple fedmsg consumer listening for new PR being opened in dist-git
18:17:38 <pingou> it will create a srpm (in a mock chroot) and kick off a scratch build in koji
18:17:44 <pingou> and report to the PR the outcome of that build
18:18:08 <nirik> would this be something we just globally enable? or a setting to enable ? optin/optout?
18:18:18 <pingou> globally
18:19:21 <pingou> select count( distinct uid) from pull_requests; -> 676
18:19:32 <pingou> so the added load to koji shouldn't be too bad :)
18:20:22 <nirik> and these are all submitted with super low pri?
18:20:42 <mizdebsk> PR as new way of scratch build from scm - /me likes it
18:21:13 <pingou> I reused the code from anitya
18:21:38 <pingou> priority 30 by default
18:21:43 <pingou> but it's a configuration option
18:22:01 <nirik> ok. I'd be fine with enabling it. If it causes problems we can disable it again.
18:22:06 <pingou> +1
18:22:18 <pingou> and ftr, I'm entirely fine with this running in the cloud
18:22:29 <pingou> there is nothing relying on it (neither should there be any)
18:22:41 <pingou> I see it as purely informational and kinda handy :)
18:23:09 <smooge> I have no problems as long as it is easily disabled by someone at 2am because someone decided to put in 2000 PR's
18:23:21 <pingou> long term I'd like to see it replaced by builds in copr so we can do things like: cross PR dependencies
18:23:32 * nirik nods.
18:23:32 <pingou> smooge: systemctl stop fedmsg-hub
18:23:47 <nirik> I think they are working on it
18:24:09 <pingou> yeah I've discussed this with clime a bit already :)
18:24:32 <pingou> it's already all in ansible
18:24:43 <pingou> but I agree with smooge, could use a SOP
18:24:45 * pingou takes this
18:24:50 <smooge> thanks
18:27:15 <pingou> so if we all agree, I could just use some help to create that cloud instance
18:27:20 <pingou> (haven't done it in a while)
18:27:29 <pingou> but otherwise, that's all for me :)
18:27:39 <nirik> pingou: sure, it should be all ansible.
18:27:45 <pingou> nice
18:27:58 <pingou> nirik: is it in our SOPs as well?
18:28:23 <nirik> It should be in the readme in ansible repo...
18:28:30 <nirik> but I can assist if not
18:28:54 <smooge> ok do we have anything more here?
18:29:27 <smooge> #topic Next Meeting 2017-12-13 1800 UTC
18:29:51 <smooge> OK I would like to see if we can prepopulate the meeting for 2017-12-13 with topics
18:29:52 * nirik notes he will be on PTO and not around there most likely
18:29:54 <pingou> nirik: cool, I'll poke at the README then
18:30:16 <smooge> have a good pto
18:30:59 <smooge> if there are no topics then I think we can have a short summary meeting and not have another one until January
18:31:04 <smooge> Does that sound good to everyone?
18:31:42 <nirik> sure. might be nice for everyone to think on what we want to do for next year...
18:31:52 <nirik> if there's any big plans or projects people have
18:31:53 <pingou> well there will likely be a debrief from the moving no?
18:32:08 <smooge> yes. that will be the 2017-12 meeting but I mean other items
18:32:15 <nirik> could be some yeah... but it should be mostly everything the same hopefully.
18:32:20 <nirik> (just more power and space)
18:32:27 <pingou> more power!!
18:32:34 <smooge> or "it was all in flames.. flames... the humanity"
18:32:57 <smooge> ok last item for this week
18:33:02 <smooge> #topic Open Floor
18:33:11 <nirik> hopefully it's not 'why are we still down' ? :)
18:33:40 <mizdebsk> more and more critical servers are now running on fedora (instead of rhel), is this considered as acceptable practice in general? i would like to move koschei-fontend to fedora too, which would simplify development and maintenance (single python version, no need to build and test on rhel and so on)
18:33:52 <smooge> if it is that I won't be running the meeting but will be in PHX
18:34:09 <nirik> mizdebsk: yeah, should be ok IMHO
18:34:23 <nirik> and provide some advantages (like http/2 from proxy to it)
18:34:50 <nirik> we need to make sure we are aggressive about upgrading tho
18:35:01 <smooge> mizdebsk, my main requests are : Is it updatable? [AKA we have to keep it at Fedora 24 forever because of X] and Will you do the updating?
18:35:02 <nirik> (ie, f25 goes eol in a few weeks)
18:35:29 <mizdebsk> smooge, yes, it's upgradeable, and yes, i'm doing the work
18:35:47 <mizdebsk> (backend is already on fedora)
18:35:49 <smooge> if the answers to the first are yes to both... I don't have a problem with doing it in stg as a test first
18:36:09 <smooge> and then going to production when poc is shown that we aren't going to be paged every night
18:36:21 <smooge> and htis isn't just for your service.. I am meaning for general
18:36:58 <smooge> aka joe-bob-bowlofeggs decides that bodhi needs to be run on rawhide or something like htat
18:37:23 <smooge> mizdebsk, nirik does that make sense?
18:37:27 <mizdebsk> :)
18:37:30 <mizdebsk> yes, thanks
18:37:43 <nirik> sure. so far fedora has been fine as application hosts...
18:37:45 <bowlofeggs> rawhide++
18:37:57 <nirik> speaking of which we need to upgrade bodhi-backend01. ;)
18:38:04 <bowlofeggs> indeed
18:38:19 <bowlofeggs> i was actually planning to do that with the release/deployment of 3.1.0
18:38:20 <nirik> and I think copr is all f25 still
18:38:31 <bowlofeggs> which i've been neglecting due to this huge masher refactor :(
18:38:32 <nirik> bowlofeggs: when it that planned for?
18:38:42 <smooge> ok sounds like a new year plan
18:38:47 <bowlofeggs> nirik: well i guess it'll have to be after the DC outage week
18:38:48 <smooge> OH fudge
18:39:05 <nirik> bowlofeggs: right.
18:39:08 <smooge> #topic End Of Year (2017-12-22 -> 2018-01-03 )
18:39:36 <smooge> so this will be the first reminder for a bit. Red Hat normally does a close of business during the end of the year.
18:39:53 <smooge> During that time most employees will be on vacation and not able to answer things quickly
18:40:22 <smooge> Normally Fedora Infrastructure goes into a slush during this time to keep breakages down.
18:40:27 <bowlofeggs> i will be eating pizza
18:40:50 <smooge> 1. If you are going to push something get 2 +1 by emails before deploying
18:41:25 <smooge> 2. If you are pushing changes please make sure that there is at least one other sysadmin to watch things
18:41:42 <smooge> sorry that should be by emails/irc (versus just emails)
18:41:55 <nirik> 3. Try not to do anything that causes alerts for people who might be away except for alerts. ;)
18:42:19 <smooge> 4. If things are burning and no one shows up... please call.
18:42:36 <smooge> 5. If no one answers the phone.. walk towards the light.
18:42:52 <smooge> ok that is about it
18:43:07 <nirik> FYI, I am not planning any trips, and should be around to answer pages and calls and fires...
18:43:34 <mizdebsk> call how? just curious; last time i tried pages (dnssec related outage) it didn't work
18:45:05 <smooge> well it worked.. I got mine but was not in a place I could respond for a bit
18:45:22 <nirik> I think it also notified me, but I was asleep and slept thru it.
18:45:51 <mizdebsk> i see
18:45:59 <nirik> I can crank up my volume for the holidays.
18:46:07 <smooge> I will do so also
18:46:57 <relrod> same
18:46:58 <smooge> or we could set aside who is dedicated oncall for what days.. and oh look that is a large can of purple worms
18:47:12 <smooge> ok I think I am definitely silly and it is time to call this meeting
18:47:16 <smooge> #endmeeting