15:07:51 <smooge> #startmeeting Infrastructure (2019-08-01)
15:07:51 <zodbot> Meeting started Thu Aug  1 15:07:51 2019 UTC.
15:07:51 <zodbot> This meeting is logged and archived in a public location.
15:07:51 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:07:51 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:07:51 <zodbot> The meeting name has been set to 'infrastructure_(2019-08-01)'
15:07:51 <smooge> #meetingname infrastructure
15:07:51 <zodbot> The meeting name has been set to 'infrastructure'
15:07:51 <smooge> #topic aloha
15:07:51 <smooge> #chair nirik pingou puiterwijk relrod smooge tflink cverna mizdebsk mkonecny abompard bowlofeggs
15:07:51 <zodbot> Current chairs: abompard bowlofeggs cverna mizdebsk mkonecny nirik pingou puiterwijk relrod smooge tflink
15:08:01 <mkonecny> .hello zlopez
15:08:02 <zodbot> mkonecny: zlopez 'Michal Konečný' <michal.konecny@packetseekers.eu>
15:08:04 * pingou salutes the company
15:08:15 <relrod> morning
15:08:20 <abompard> .hello2
15:08:21 <zodbot> abompard: abompard 'Aurelien Bompard' <aurelien@bompard.org>
15:09:20 <nb> .hi
15:09:21 <zodbot> nb: nb 'Nick Bebout' <nick@bebout.net>
15:09:32 <smooge> #topic New folks introductions
15:09:33 <smooge> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
15:09:33 <smooge> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
15:09:33 <smooge> #knstn introduction
15:09:48 <nirik> morning
15:10:29 <tflink> morning
15:10:45 <smooge> #topic announcements and information
15:10:45 <smooge> #info bowlofeggs is on extended leave
15:10:45 <smooge> #info smooge is on PTO 2019-08-14 -> 2019-08-16 (Someone needs to run that meeting)
15:10:45 <smooge> #info No meeting 2019-08-08
15:10:46 <smooge> #info Need someone to run meeting on 2019-08-15
15:10:47 <smooge> #info Flock2Fedora 2019-08-08 -> 2019-08-11
15:10:50 <smooge> #info Blog post on retiring apps https://communityblog.fedoraproject.org/application-service-categories-and-community-handoff/
15:10:53 <smooge> #info CPE/Infrastructure workshop at FLOCK (date to be announced)
15:10:55 <smooge> #info infinote will go away asap.
15:11:03 <smooge> any other announcements?
15:12:02 <smooge> #topic Oncall
15:12:02 <smooge> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
15:12:02 <smooge> #info relrod is on call from 2019-07-25 -> 2019-08-01
15:12:02 <smooge> #info smooge is on call from 2019-08-01 -> 2019-08-13 #flock
15:12:03 <smooge> #info ?????? is on call from 2019-08-13 -> 2019-08-15
15:12:04 <smooge> #info Summary of last week: (from relrod )
15:12:25 <smooge> .oncall_take_us
15:12:25 <zodbot> smooge: Kneel before zod!
15:12:46 <smooge> relrod, anything for oncall this week?
15:13:04 <relrod> So, there were tons of little things. It was quite a busy week, but all the pings were fairly small tasks individually. And random alerts that I acked/dealt with like proxy11 being dumb.
15:13:04 <nirik> smooge: I can take oncall from you on the 14th
15:13:35 <relrod> and proxy09
15:13:37 <smooge> ok I will be away from keyboard probably so just do so
15:13:57 <smooge> relrod, yeah.. it has been papercuts all week long
15:15:13 <nirik> and meetings.. lets not forget the meetings. ;)
15:15:19 <relrod> yeah.........
15:16:28 <smooge> i think the meetings will continue until morale improves
15:16:44 * nirik nods.
15:17:49 <smooge> #topic Monitoring discussion
15:17:49 <smooge> #info https://nagios.fedoraproject.org/nagios
15:17:49 <smooge> #info Go over existing out items and fix
15:18:25 * nirik looks
15:18:35 <abompard> I have a question on the topic of monitoring, but not about the out items
15:18:54 <abompard> so, out items go first :-)
15:18:57 <nirik> the osbs stuff is because cverna is going to reinstall it.
15:19:53 <nirik> ppc64le-13 is because it hit OOM.
15:19:56 <nirik> restarting it
15:20:22 <nirik> badges is badges.
15:21:03 <nirik> I think the rest are all known
15:23:08 <abompard> So, I'd like to add some checks on the rabbitmq servers. I think I have an ansible changeset ready, but it's going to modify the Nagios config and I'd rather have someone from ops available when I push it in case I break Nagios
15:23:31 <smooge> abompard, ok
15:23:32 <nirik> abompard: sure. nagios is super fragile, so you should expect to break it. :)
15:23:43 <abompard> yeah I've worked with it in the past
15:23:45 <nirik> but thats ok, you can iterate and get it working
15:23:52 <smooge> thanks and what nirik says.. if it breaks it is not the end of the world
15:23:55 <abompard> the staging env is not monitored, right?
15:24:05 <nirik> it's not currently, no
15:24:06 <smooge> not really we were having to ack everything
15:24:13 <abompard> ok that's why I saw
15:24:35 <abompard> also, I don't think rbac-playbook will let me redeploy the nagios server
15:24:59 <nirik> we can get you access... sysadmin-noc I think?
15:25:04 <nirik> but we can do that out of meeting.
15:25:09 <abompard> sure
15:25:30 <abompard> would someone from ops be available to work with me on that? Which timeslot?
15:25:43 <smooge> yes. when do you want to do it?
15:25:54 <abompard> after the meeting works for me, tomorrow works too
15:26:10 <smooge> could we do tomorrow. I need to work on 2 meetings after this
15:26:14 <abompard> sure
15:26:19 <smooge> .time
15:26:19 <zodbot> smooge: 03:26 PM, August 01, 2019
15:26:26 <abompard> what would be a good timeslot for you tomorrow?
15:26:34 <abompard> I'm at UTC+2
15:26:43 <nirik> I could possibly do after this meeting too....
15:26:44 <relrod> abompard: can you email me the patch (or send to infra list) and I can do a quick review?
15:26:54 <nirik> I mean I have a ton of things, but would be good to get this. ;)
15:27:01 <abompard> Yeah I'll send it to the list
15:27:01 <smooge> 14:00 or later tomorrow
15:27:16 <abompard> 14:00 UTC?
15:27:26 <smooge> sorry ... I added +2
15:27:31 <smooge> 12:00 UTC or later
15:27:36 <abompard> nirik: as you prefer, if you don't mind right now is even better because I'm on it
15:27:47 <smooge> or work with nirik after meeting
15:27:58 <abompard> smooge: I'll try with kevin, and if we need more time I'll bug you for tomorrow OK ?
15:28:02 <smooge> yep
15:28:07 <abompard> thanks folks
15:28:08 <nirik> whatever works. :)
15:28:11 <smooge> #topic Tickets discussion
15:28:11 <smooge> #info https://pagure.io/fedora-infrastructure/report/Meetings%20ticket
15:28:37 <nirik> smooge: can you close the epel8 ticket now? or not quite yet?
15:29:07 <nirik> .ticket 7806
15:29:08 <zodbot> nirik: Issue #7806: Update communityblog DNS entry - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/7806
15:29:11 <nirik> this one is done ? ^
15:29:20 <smooge> maybe? I don't know what one that is
15:29:24 <nirik> .ticket 7777
15:29:27 <zodbot> nirik: Issue #7777: EPEL-8 Project Outline - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/7777
15:29:51 <smooge> done
15:30:44 <nirik> .ticket 6801
15:30:45 <zodbot> nirik: Issue #6801: run hardlink on fedora-secondary - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/6801
15:31:04 <nirik> this was being talked about the other day
15:31:36 <smooge> yeah.. it needs to be done over archive also
15:32:14 <nirik> relrod: can we update or close the fpaste tickets? ie, 'we are going to be retiring this' ?
15:32:21 <nirik> .ticket 5816
15:32:22 <zodbot> nirik: Issue #5816: [fpaste] Could we please darken the theme a bit? - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5816
15:32:27 <nirik> .ticket 5815
15:32:28 <zodbot> nirik: Issue #5815: [fpaste] Please enable expiration in the new modernpaste instance - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5815
15:32:32 <pingou> +1
15:32:35 <nirik> .ticket 5780
15:32:36 <zodbot> nirik: Issue #5780: Currently modernpaste login form deadlinked - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/5780
15:32:56 <pingou> (or should we wait post-announcement to do this?)
15:32:58 <relrod> nirik: yeah, will do once I contact the fpaste maintainer, which has been on my list of things to do but keep getting pulled away from
15:34:14 <nirik> pingou: not sure.
15:34:36 <nirik> but really we aren't going to work on those things anymore... there's 0 chance, so I think as soon as we know that we should tell them and close the ticket(s)
15:34:51 <pingou> fair, +1 then
15:35:03 <relrod> ok, then I'll close them
15:36:12 <nirik> I can dig up more tickets or we can call that a day for today. ;)
15:38:30 <smooge> ok calling it a day
15:38:36 <nirik> sure...
15:39:01 <smooge> if you want to keep at it I am ok also... I just got pinged on 2 other things
15:39:23 <smooge> #topic Open Floor
15:39:35 <nirik> I have 2 things...
15:39:39 <smooge> cool
15:39:59 <nirik> First: is there anything we want to try and land before flock? (there is very little time for it, but I thought I would check)
15:40:15 <nirik> pingou: were you talking about landing those pagure plungins? or not enough time?
15:41:07 <pingou> nirik: we could try in staging but we won't have it in prod by then
15:41:17 <pingou> it needs a newer pagure version
15:41:26 <nirik> ah, ok.
15:41:30 <pingou> and tomorrow is my last day before flock
15:41:38 <nirik> fair enough
15:41:44 <cverna> I would like to redeploy OSBS in staging but I think the OTP registration is busted in staging :( and I need a token since new phone :(
15:41:56 <pingou> landing in prod also needs a bit of coordination with Michal
15:42:16 <nirik> cverna: yeah, it is... I have been meaning to look and see how that was fixed in prod and see if I could just do that in stg. ;)
15:42:32 <nirik> pingou: stg might still be nice to announce on devel and ask people to test?
15:42:34 <mkonecny> nirik: I had issue with token even on prod
15:42:47 <cverna> nirik: happy to help here if I can :)
15:42:51 <nirik> mkonecny: yeah, I saw your email... weird. It didn't error? just didn't add it?
15:43:25 <mkonecny> nirik: It said "Token was added", but nothing happened
15:43:27 <pingou> nirik: +1
15:44:17 <mkonecny> And when I closed the page, it thought that I had already added it
15:44:37 <nirik> mkonecny: I was gonna clear it again for you and see if it worked. we can do that after meeting (after nagios work? or before)
15:44:50 <mkonecny> However, this could be issue in FreeOTP+
15:45:16 * smooge has to remember to clear his token and get a new one
15:45:56 <smooge> ok we have 10 minutes left for this meeting
15:45:59 <mkonecny> nirik: I will try to add it again, not sure why it isn't working
15:46:03 <nirik> mkonecny: I can also test here, I have that same app. ;)
15:46:21 * pingou gtg
15:46:25 <pingou> talk to you all tomorrow :)
15:46:32 <smooge> see you otmorrow
15:47:20 <smooge> .time
15:47:20 <zodbot> smooge: 03:47 PM, August 01, 2019
15:47:47 <smooge> how about we do the nagio change starting at 16:00 UTC and the token changes at 17:00 UTC?
15:48:20 <nirik> sure, but not sure mkonecny and cverna will be around after 17?
15:48:30 <nirik> or will abompard be around after 17?
15:48:54 <mkonecny> nirik: I will still be online
15:49:06 <abompard> nirik: I'll be online but not for long
15:49:38 <smooge> so I think we start with nagios and hten do token itmes
15:49:40 <abompard> nirik: actually I'll be around after 16:00UTC but not 17:00 UTC
15:50:20 <nirik> ok, sounds good.
15:50:24 <abompard> is 16:00 UTC good for you nirik ? (in 10 minutes)
15:50:32 <nirik> abompard: yep. in #fedora-noc?
15:50:37 <abompard> nirik: yes
15:51:06 <smooge> ok I want to thank everyone for coming to the meeting this week. Next meeting will need to be run by someone else on 2019-08-15
15:51:21 <smooge> #endmeeting