15:02:54 <smooge> #startmeeting Infrastructure (2019-05-15)
15:02:54 <zodbot> Meeting started Thu May 16 15:02:54 2019 UTC.
15:02:54 <zodbot> This meeting is logged and archived in a public location.
15:02:54 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:54 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:02:54 <zodbot> The meeting name has been set to 'infrastructure_(2019-05-15)'
15:02:54 <smooge> #meetingname infrastructure
15:02:54 <zodbot> The meeting name has been set to 'infrastructure'
15:02:54 <smooge> #topic aloha
15:02:54 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs
15:02:54 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink
15:03:00 <clime> hi
15:03:03 <nirik> morning party people.
15:03:08 <smooge> I have only 1 job...
15:03:09 <braydon_kains> Hello!
15:03:17 <mkonecny> .hello zlopez
15:03:18 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:03:24 <tflink> morning
15:03:27 <bcotton> .hello2
15:03:28 <zodbot> bcotton: bcotton 'Ben Cotton' <bcotton@redhat.com>
15:03:41 <relrod> morning
15:04:43 * pingou 
15:04:49 <smooge> #topic New folks introductions
15:04:49 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:04:49 <smooge> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:05:29 <braydon_kains> Hey everyone, I'm new. Just finally sent my introduction to the mailing list yesterday but have been looking over projects and guides for a while.
15:06:30 <mkonecny> Welcome here :-)
15:07:01 <smooge> hello braydon_kains
15:07:27 <smooge> #topic announcements and information
15:07:28 <smooge> #info Zombieland reboots of important servers have been done.
15:07:28 <smooge> #info bodhi-4.0.0 beta is in staging. Please test because we ship soon
15:07:28 <smooge> #info only N days until F31 freeze
15:07:28 <smooge> #info bowlofeggs will be going on extended leave in June
15:07:29 <smooge> #info nirik will be on leave from 2019-06-03 to 2019-06-07
15:07:30 <smooge> #info many people will be at a community platform engineering f2f 2019-06-10 to 14
15:07:32 <smooge> #info Another blogpost about release-monitoring.org is published (https://communityblog.fedoraproject.org/stories-from-the-amazing-world-of-release-monitoring-org-5/) - mkonecny
15:07:35 <smooge> #info FAS2 has been moved to openshift (puiterwijk)
15:07:37 <smooge> #info Ipsilon has been moved to openshift (puiterwijk)
15:07:38 <nirik> welcome braydon_kains!
15:07:39 <smooge> #info community openshift is progressing
15:07:43 <smooge> #info next reboot cycle?
15:07:50 <smooge> braydon_kains, I can add you to the apprentice gorup after meeting if you aren't already
15:08:09 <braydon_kains> smooge: Yeah I'm not in that yet, would appreciate it thanks.
15:09:02 <smooge> OK we have a LOT of things starting to go on in the next 3 weeks
15:09:04 <rpbarlow> .hello2
15:09:05 <zodbot> rpbarlow: Sorry, but you don't exist
15:09:14 <relrod> #info new getfedora.org has been deployed. Thank you to puiterwijk for deployment help
15:09:24 <rpbarlow> .hello bowlofeggs
15:09:25 <zodbot> rpbarlow: bowlofeggs 'Randy Barlow' <rbarlow@redhat.com>
15:10:13 <smooge> hello bowlofeggs/rpbarlow
15:10:18 <rpbarlow> heh
15:10:19 <mkonecny> smooge: What is the N in "only N days until F31 freeze"
15:10:20 <smooge> thanks relrod
15:10:33 <rpbarlow> i guess freenode bumped me from being my usual self but that's ok because i have more than one personality
15:11:01 <mkonecny> relrod: I worked on translation and reported few things I encountered
15:11:26 <nirik> https://fedoraproject.org/wiki/Releases/31/Schedule
15:11:36 <nirik> 2019-08-29... so we have a while.
15:12:03 <smooge> mkonecny, looks like ~100 days til freeze and ~60 days til mass rebuild
15:12:25 <nirik> oh, another info:
15:12:29 <relrod> mkonecny: thank you
15:12:41 <mkonecny> smooge: thanks
15:12:48 <nirik> #info flocktofedora is aug 2nd-5th
15:12:49 <mkonecny> relrod: You are welcome
15:13:02 <pingou> no meeting that week either :)
15:13:08 <nirik> sorry, 8th-11th
15:13:13 <mkonecny> Hooray Flock \o/
15:13:16 <nirik> https://flocktofedora.org/
15:13:32 <nirik> everyone should start thinking about talks they might like to give...
15:13:56 <mkonecny> I already have two in mind
15:14:09 <smooge> nirik, ha! you had me there for a moment with 'Oh I can go!'
15:14:12 <nirik> I think we should definitely do a infrastructure workshop, those are always good
15:14:27 <nirik> mkonecny: oh yeah?
15:14:34 * pingou has at least one
15:15:01 * nirik isn't sure. There's a bunch I could do, but not sure what sounds good this year...
15:15:15 <pingou> community openshift?
15:15:20 <mkonecny> nirik: One is about release-monitoring.org with a little magic added :-)
15:15:35 <pingou> mkonecny: are you going to wear a pointy hat?
15:15:55 <mkonecny> The second one should be about privacy, but I'm not sure if this is a good speech for Flock
15:16:29 <mkonecny> pingou: Unfortunately I don't have any, but I can borrow one from my brother :-D
15:16:36 <nirik> pingou: yeah, could... with a lot of help from puiterwijk, we now have it up and running (initial install/not for public use yet)
15:17:49 <nirik> mkonecny: sounds cool. Magic is great!
15:18:10 * nirik almost gave a privacy talk a few years back, but decided not to.
15:18:29 <smooge> pingou, community openshift is the replacement for community openstack that is most of Fedora Infra Cloud
15:18:52 <smooge> or was that a different question?
15:19:25 <pingou> smooge: it was a talk suggestion for flock :)
15:19:30 <smooge> sorry
15:19:33 <nirik> smooge: I think he was suggesting I give a talk on it...
15:19:54 <pingou> smooge: no worries :)
15:19:55 <smooge> I came back from a ping and thought it was about the topic line
15:20:08 <smooge> #topic Oncall
15:20:09 <smooge> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:20:09 <smooge> #info nirik is on call from 2019-05-09 -> 2019-05-16
15:20:09 <smooge> #info relrod is on call from 2019-05-16 -> 2019-05-23
15:20:09 <smooge> #info smooge is on call from 2019-05-23 -> 2019-05-30
15:20:10 <smooge> #info ?????? is on call from 2019-05-30 -> 2019-06-06
15:20:11 <smooge> #info no one is on call from 2019-06-06 -> 2019-06-20
15:20:15 <smooge> #info ?????? is on call from 2019-06-20 -> 2019-06-27
15:20:17 <smooge> #info Summary of last week: (from nirik )
15:20:38 <nirik> a fair pile of pings... on various topics
15:20:48 <nirik> fas had a few people as it was moved and needed tweaking
15:21:03 <rpbarlow> i bothered you a lot this round too!
15:21:04 <nirik> there was some bodhi tweaking to work around issues with other packages
15:21:10 <nirik> yeah. ;)
15:21:32 <nirik> a fair number of incoming tickets to triage...
15:21:44 <nirik> I think we are still over 100...
15:22:01 <nirik> yeah, 105
15:22:36 <nirik> you going to take the coffee cup smooge ?
15:22:48 <smooge> relrod said he would
15:23:00 * nirik reads up more, ah... ok. ;)
15:23:02 <smooge> I take it next week so he can do something during that week
15:23:23 <nirik> I'm going to be out early june, so I can't take the week after that
15:23:26 * smooge needs to see if relrod has stopped partying from graduation
15:23:30 <relrod> smooge: actually... so I mixed up my weeks. Is there any way we could flip? I won't be in tomorrow and this weekend
15:23:48 <smooge> zodbot: take_coffee_cup_of_power
15:24:06 <relrod> smooge: not much partying but a small trip to Dayton OH this weekend
15:24:17 <smooge> did you go to the air force museum?
15:25:06 <relrod> no, I need to
15:25:38 <relrod> Two years ago when I went, I stayed in a hotel that basically sits on the afb though.
15:25:55 <smooge> zodbot: oncall_take_us
15:25:55 <zodbot> smooge: Kneel before zod!
15:26:08 * smooge goes to make an alias for coffe_cup
15:26:19 <smooge> ok schedules are changed
15:26:26 <relrod> thank you and sorry
15:27:13 <smooge> nirik, puiterwijk how does 2019-06-20 -> 2019-06-27 look for either of you
15:27:34 <nirik> I should be able to do that week.
15:28:01 <nirik> I don;'t think I have any plans to go anywhere after the face to face meetings and until flock
15:28:18 <nirik> but... we may want to figure out phx2 site visit sometime
15:28:21 <pingou> it's been a while since I took one, I should put my name in the hat
15:29:08 <smooge> nirik, yeah.. I was figuring mid July for the phx2 site visit
15:29:20 <smooge> pingou, I will put you on that one
15:29:24 <smooge> we can change later
15:29:40 <smooge> #topic Monitoring discussion
15:29:40 <smooge> #info https://nagios.fedoraproject.org/nagios
15:29:40 <smooge> #info Go over existing out items and fix
15:30:06 <nirik> not sure whats up with those buildvmhost-aarch64's...
15:30:11 <nirik> rhit firewall?
15:30:12 <pingou> smooge: I expect I'll have some days off on that week, I'll just be slower at ticket triaging
15:31:21 <nirik> 3 boxes show raid issues, likely need rebooting to re-add drives?
15:31:22 <smooge> the raids should be fixed on reboots
15:31:31 <nirik> there's also one more machine with a dropped drive
15:31:54 <nirik> autocloud-backend-libvirt2
15:32:27 <smooge> I tried to do them via the drac but they will need someone on console to clear hw controller
15:32:38 <smooge> oh I didn't know that one.. oof
15:32:53 <smooge> nirik, is that a service which is going away after 28 is EOL?
15:33:04 <nirik> no, after f29
15:34:39 <nirik> so that brings us to reboots... perhaps next week?
15:34:57 <nirik> when the fallout from the latest cpu attacks is more known?
15:35:02 <smooge> yeah
15:35:12 <smooge> in case we have to move everything to AMD and ARM
15:36:04 <smooge> Do the usual cadence? Monday Staging, Tuesday External, Wednesday Internal, Weekend Build?
15:36:28 <smooge> most of that cadence was because it was easier for just one person to do parts as they had time
15:36:44 <nirik> yeah, that works for me, unless others have better ideas
15:36:49 <smooge> and also to try and not make it a OMG fire to meet an outage window.
15:36:58 * pingou has to step out shortly
15:37:04 <smooge> good seeing you pilotmatt
15:37:10 <smooge> good seeing you pingou
15:37:12 <pingou> :)
15:37:17 <smooge> auto-complete.. you complete me
15:37:23 <rpbarlow> pilotmatt++ tho
15:37:39 <smooge> ok I will put in tickets after meeting
15:37:52 <smooge> #topic Tickets discussion
15:37:52 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:38:25 <nirik> dammit. session crashed.
15:38:36 <smooge> so I wanted to bring up something here which nirik has been bringing into work force
15:39:00 <smooge> if it takes longer than a few minutes or needs more than 1 person, it needs to have a ticket
15:39:10 <smooge> no ticket, no work
15:39:42 <nirik> yeah, it might be a hassle, but it really helps us keep better track of things and how much people are doing.
15:40:25 <rpbarlow> +1
15:40:46 <nirik> also related:
15:40:54 <rpbarlow> it also helps keep people working efficiently by not being interrupted
15:41:04 <nirik> if you have a ticket, please keep that updated with questions/progress...
15:41:32 <nirik> and please do NOT update the ticket and immediately ping people about it. Tickets send email, people get it and can act when they are able.
15:42:26 <nirik> ideally we make tickets efficent enough people don't feel they need to do that.
15:43:08 <smooge> and conversely.. opening a ticket and then pinging people about it lowers it in the queue
15:43:23 <smooge> the cabal queue
15:44:34 <rpbarlow> haha yes that's a good policy
15:44:58 <smooge> ok anything else on this?
15:45:10 <smooge> #topic Open Floor
15:46:09 <smooge> Well thank you all for attending. And thank you all for your work
15:46:20 <rpbarlow> small announcement - i have bodhi-backend02's tasks in openshift in stg (though they currently crash - investigating)
15:46:22 <clime> thanks smooge
15:46:28 <rpbarlow> so with bodhi 4, we should be able to drop backend02
15:46:28 <smooge> cool
15:46:37 <mkonecny> thanks smooge
15:46:58 <smooge> more resources for my fedora block chain package signing tool
15:47:25 <puiterwijk> smooge--
15:47:31 <rpbarlow> haha
15:47:45 <braydon_kains> Looking forward to being part of the team! Thanks everyone!
15:47:56 <rpbarlow> is it impossible or just not worth it to move backend01 into openshift?
15:48:00 <smooge> so I figured that if we have all these people wasting electricity to 'mine' things why not let them do builds and package signing for us
15:48:16 <rpbarlow> i know there's something weird abotu the NFS share, but i confess that i don't really understand it
15:48:31 <smooge> wait wait wait.. I am cracking jokes and rpbarlow is asking serious questions
15:48:35 <rpbarlow> hahaha
15:48:43 <nirik> well, we could in theroy... but it does pungi runs...
15:48:47 <rpbarlow> yeah
15:48:51 <nirik> so right now it's a vm with a ton of memory/cpu
15:49:00 <rpbarlow> yeah more than 90 GBs!
15:49:01 <nirik> I am not sure how pungi would handle being in a container
15:49:10 <rpbarlow> yeah i don't know either
15:49:12 <nirik> there was talk about moving that into koji...
15:49:30 <rpbarlow> that's true i guess we could consider that
15:49:31 <nirik> we could make them runroot tasks possibly.
15:50:22 <rpbarlow> is a runroot task basically a "please run this shell command for me koji?"
15:50:24 <nirik> I'm not sure how difficult that would be, but it sounds like it should work
15:50:30 <nirik> yep.
15:50:34 <rpbarlow> ah
15:50:34 <nirik> with these mounts, etc
15:50:44 <rpbarlow> well that is basically all i'm doing with bodhi's composer anyway
15:50:51 <rpbarlow> instead of asking koji it asks the OS
15:50:58 <rpbarlow> but you know, maybe it wouldn't be that hard?
15:50:59 <nirik> random task: https://koji.fedoraproject.org/koji/taskinfo?taskID=34778643
15:51:07 <nirik> see the 'command' there
15:51:17 <rpbarlow> bodhi does directly write files itself too though
15:51:21 <rpbarlow> to add the updateinfo.xml
15:51:40 <rpbarlow> is there a "koji please accept this updateinfo.xml that i made for you" command? haha
15:52:00 <nirik> nope. ;( But if there was a seperate tool to do that it could run it as part of the runroot task?
15:52:03 <rpbarlow> i.e., it's not just pungi though that's the bulk of the work
15:52:06 <nirik> or it could run a second one to do that
15:52:26 <rpbarlow> it would be hard for it to be a separate tool because it uses bodhi's DB
15:52:34 <rpbarlow> well, unless that tool also used bodhi's DB i guess
15:53:13 <smooge> ok lets move this part to #fedora-admin. There will be another meeting here in 5 minutes
15:53:15 <rpbarlow> maybe a topic for further discussion another day i suppose - that's all i have on it ☺
15:53:21 <smooge> ok cool
15:53:38 <smooge> thank you all again. see you next week. same bat-channel same bat time
15:53:42 <smooge> #endmeeting