15:02:22 <smooge> #startmeeting Infrastructure (2019-08-29)
15:02:22 <zodbot> Meeting started Thu Aug 29 15:02:22 2019 UTC.
15:02:22 <zodbot> This meeting is logged and archived in a public location.
15:02:22 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:22 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:02:22 <zodbot> The meeting name has been set to 'infrastructure_(2019-08-29)'
15:02:22 <smooge> #meetingname infrastructure
15:02:22 <zodbot> The meeting name has been set to 'infrastructure'
15:02:22 <smooge> #topic aloha
15:02:22 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs
15:02:22 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink
15:02:33 <smooge> Morning
15:02:34 <nirik> morning
15:02:40 <relrod> morning
15:03:04 <cverna> morning all
15:03:08 <tflink> morning
15:03:18 * tflink is here but has another meeting at the same time, may get distracted
15:03:53 <smooge> #topic announcements and information
15:03:53 <smooge> #info bowlofeggs is back
15:03:53 <smooge> #info smooge will be on PTO 2019-09-07 -> 2019-09-15
15:03:53 <smooge> #info we are in F31 beta freeze
15:03:54 <smooge> #info pagure updates to 5.7.?
15:03:55 <smooge> #info ?? updates
15:04:27 <smooge> any other updates for this week?
15:04:53 <nirik> can people stop scheduling meetings and sending emails? :)
15:05:13 <tflink> that would make life less busy :)
15:05:20 <smooge> we would just complain that we are never told why we didn't deliver something when it was due last week
15:05:47 <nirik> we do that anyhow. ;) but I digress...
15:05:56 <smooge> this way we have the illusion that we knew we wouldn't be able to deliver it on time
15:06:44 <smooge> I am personally just going to be blocking large sections of my day unavailable
15:06:56 <pingou> 5.7.8 for pagure, both stg hosts
15:08:02 <smooge> thanks pingou
15:08:21 <smooge> any other announcements?
15:08:30 <smooge> i asked that already
15:08:34 <smooge> #topic Oncall
15:08:35 <smooge> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:08:35 <smooge> #info cverna is on call from 2019-08-22 -> 2019-08-29
15:08:35 <smooge> #info ?????? is on call from 2019-08-29 -> 2019-09-05
15:08:35 <smooge> #info ?????? is on call from 2019-08-05 -> 2019-09-12
15:08:36 <smooge> #info Summary of last week: (from cverna )
15:08:53 <cverna> not much happening
15:09:14 <cverna> I had to cleanup the Openshift node because there were running out of space
15:09:37 <cverna> and helping a few people here and there
15:09:51 <cverna> thanks smooge for dealing with the fires yesterday evening
15:09:51 <smooge> cool
15:09:56 <cverna> (my evening)
15:10:25 <smooge> I have found that we seem to have 2 windows of pings.. late-day EU time and late day US EST time
15:10:48 <nirik> there also seems to be a number at start of day us...
15:10:49 <smooge> one is too early for people in the US and one is too late for people in the EU
15:10:56 <nirik> which I guess is late day eu
15:11:20 <smooge> yeah..
15:11:47 <cverna> filing a ticket is also fine in that case :-)
15:11:57 <nirik> more than fine for us.
15:12:08 <smooge> ok who is taking oncall starting today?
15:12:11 <nirik> oh, one thing we might add to our docs...
15:12:47 <nirik> ask people to indicate urgency only _in ticket_ not by pinging and mailing and calling people... otherwise the people not called and pinged don't know the ticket is urgent...
15:13:12 <nirik> smooge: I can take it again... or... we could make bowlofeggs do it since he's back now. ;)
15:14:50 <relrod> Or I can, but I'll be out for a few hours tonight my time
15:15:02 <smooge> that is ok..
15:15:14 <smooge> will you be on jury duty next week?
15:15:24 <relrod> nope, done with it
15:15:32 <smooge> ok then please do
15:15:41 <cverna> We also talked about having a bit more flexible oncall ie on a day/hours base
15:16:37 <relrod> .alias add oncall "echo relrod (Rick Elrod) is oncall. If not around, please file a ticket: https://pagure.io/fedora-infrastructure/issues"
15:16:37 <zodbot> relrod: Kneel before zod!
15:16:39 <nirik> we could... but that makes it harder to be fair I think...
15:16:48 <cverna> ok :)
15:17:04 <nirik> dunno, I'm open to whatever works...
15:17:52 <cverna> yes I think I agree it might be harder to organize
15:19:04 <smooge> i say we get an app to do this for us :)
15:19:08 <smooge> #topic Monitoring discussion
15:19:08 <smooge> #info https://nagios.fedoraproject.org/nagios
15:19:08 <smooge> #info Go over existing out items and fix
15:19:53 <nirik> ok, so we should power cycle those 2 buildhw-aarch64s
15:20:06 <nirik> the osbs one is I think not deployed yet?
15:20:32 <nirik> autocloud* and qa09 have disks out
15:21:05 <cverna> yes I will be working on OSBS more next week hopefully
15:21:08 <nirik> I am not sure about the notifs-backend01 things...
15:21:22 <nirik> it's been sending digests ok and irc messages...
15:22:43 <smooge> me either.
15:23:33 <nirik> and the updates one is known/being worked.
15:28:40 <smooge> sorry phone call
15:28:49 <smooge> #topic Tickets discussion
15:28:49 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:29:39 <smooge> Any tickets people would like to focus on?
15:29:47 <cverna> I quickly wanted to discuss this one https://pagure.io/fedora-infrastructure/issue/8147
15:30:36 <cverna> about deleting the dangling images our our OpenShift nodes, I am looking for feedback :)
15:31:00 <nirik> how does it define 'dangling' ?
15:31:22 <chris787> have you tried docker system prune ?
15:31:22 <cverna> dangling is an image that is not used by a container
15:32:17 * nirik notes this is not a problem on 4.x clusters. ;(
15:32:33 <cverna> chris787: docker system prune is a bit too agressive
15:32:52 <cverna> nirik: what I think is that the build images end up on the registry
15:33:05 <cverna> nirik: so deleting them from the node should be fine
15:33:07 <nirik> yes, they do
15:33:09 <cverna> in theory :)
15:33:10 <nirik> yep
15:33:19 <nirik> so we can give it a try.
15:33:32 <nirik> at worst it means another pull or build.
15:33:54 <cverna> I ll send a FBR with a patch and we can have it in stg at first
15:35:03 <nirik> sure. +1
15:36:25 <cverna> the script that you use if fine but was deleting only images that were "weeks" old, leaving the ones that are "months" old
15:36:35 <cverna> s/if/is/
15:36:53 <nirik> ah, ok.
15:38:33 <cverna> that's it from me
15:39:04 <smooge> ok
15:39:16 <smooge> #topic Open Floor
15:39:23 <smooge> oops mised one
15:39:34 <smooge> #topic Where to put these docs in the future?
15:39:34 <smooge> #info infinote is going away and this is my 'private hackmd'
15:39:34 <smooge> #info a new central doc place would be good.
15:39:59 <cverna> :-)
15:40:11 <smooge> so this meeting outline was houses in infinote but we are truning that off tomorrow
15:40:15 <nirik> well, we have the cpe one that pingou was working on... but I am not sure the status.
15:40:56 <nirik> or you mean hackmd docs? or ?
15:41:48 <smooge> yeah sorry https://hackmd.io/@ssmoogen/HyqIIdlmS is in my private space
15:41:57 <cverna> I think a new place for the meeting outline
15:42:05 <smooge> so I have to give people the rights to edit it etc
15:42:34 <smooge> and I am leery of having people depend on personal stuff for infrastructure
15:43:07 <nirik> I don't know if you can make group spaces on hackmd...
15:43:17 <pingou> the cpe doc is online: https://docs.fedoraproject.org/en-US/cpe/
15:43:24 <cverna> yes we have a infra github bot that we could maybe use to create an hackmd account
15:43:27 <pingou> but isn't linked to from any place in the doc currently
15:43:33 <bowlofeggs> i'm back baby
15:43:42 <bowlofeggs> (i was afk to pick up a car from a shop)
15:44:06 <cverna> https://github.com/fedora-infra-bot
15:44:09 <cverna> bowlofeggs++
15:44:16 <pingou> now we can sync hackmd with github btw :)
15:44:32 <smooge> pingou, yeah I saw that. I would have preferred somewhere else :)
15:45:15 <pingou> to pagure? :)
15:45:16 <smooge> but anyway.. if someone could help set up a central place for docs I would appreciate it
15:45:19 <nirik> so perhaps we make a project and have that sync the hackmd docs on github?
15:46:20 <cverna> +1
15:46:20 <pingou> I've been wanting to have a infra-docs repo where we can put the public facing docs
15:46:37 <pingou> I don't think this solves the meeting structure question though
15:47:07 <nirik> I'd love to move our infra-docs to docs.
15:47:11 <pingou> but then perhaps these can be added to the current infra-docs repo (which is more oriented to members of the team)
15:47:25 <bowlofeggs> for the infra meeting we could just use mozilla's etherpad
15:47:56 <nirik> it's dead?
15:48:07 <smooge> that was my understanding.. it was going away
15:48:08 <pingou> hm, good point, it's not just the structure it's also easily adding content to it, the infra-docs repo isn't a good place then
15:48:10 <nirik> etherpad at least is dead.
15:48:26 <nirik> .title https://public.etherpad-mozilla.org/
15:48:27 <zodbot> nirik: HTTP Error 500: Internal Server Error
15:48:29 <pingou> honestly, I won't think hackmd is such a bad place then
15:48:37 <bowlofeggs> ha
15:49:16 <nirik> there's other ones... dunno if they are better. I guess we should take a survey of them
15:49:19 <smooge> ok we have 5 minutes and I don't think we have an answer
15:49:34 <nirik> lets take it to the list?
15:49:35 <smooge> so lets take it ot the list and have an new person give some ideas
15:49:43 <smooge> #topic Open Floor
15:49:53 <smooge> alright anything for the open floor
15:50:35 <austinpowered> Is now a good time?
15:50:53 <austinpowered> With apologies, this is my first IRC meeting. I didn't see New folks introductions #topic go by so I have been lurking and trying to keep up.
15:51:26 <cverna> welcome austinpowered :-)
15:51:31 <nirik> welcome austinpowered. :) Sorry we didn't have that section...
15:52:24 <smooge> sorry my fault
15:52:27 <smooge> I missed it
15:52:35 <smooge> hi austinpowered
15:52:56 <austinpowered> no worries - I thought it was coming later until I read the agenda again
15:53:06 <smooge> nope.. I walked right over it
15:53:19 * pingou needs to step away, cheers folks!
15:53:20 <smooge> grey text on black background is hard on my old eyes
15:53:28 <smooge> what are you interested in austinpowered ?
15:53:35 <austinpowered> If you saw my post to the list you will see I'm not without some skills.
15:53:52 <austinpowered> I'm open to any direction at this point. Just let me know where to start.
15:53:56 <cverna> smooge: there is a light mode (look for a small wheel on the bottom of the page)
15:54:11 <austinpowered> I believe joining the Fedora Infrastructure Apprentice (fi-apprentice) group is my next step.
15:54:12 * cverna prefers the light mode
15:55:09 <smooge> austinpowered, ok pm your fas account and I will add you after this meeting
15:56:57 <smooge> ok another meeting usually starts soon. so I am going to end this meeting now.
15:57:06 <smooge> thank you all for coming and i will see you next week
15:57:09 <smooge> #endmeeting