16:00:00 <nirik> #startmeeting Infrastructure (2021-11-11)
16:00:00 <zodbot> Meeting started Thu Nov 11 16:00:00 2021 UTC.
16:00:00 <zodbot> This meeting is logged and archived in a public location.
16:00:00 <zodbot> The chair is nirik. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:00 <zodbot> The meeting name has been set to 'infrastructure_(2021-11-11)'
16:00:00 <nirik> #meetingname infrastructure
16:00:00 <zodbot> The meeting name has been set to 'infrastructure'
16:00:00 <nirik> #chair nirik siddharthvipul mobrien zlopez pingou bodanel dtometzki jnsamyak computerkid
16:00:00 <nirik> #info Agenda is at: https://board.net/p/fedora-infra
16:00:01 <nirik> #info About our team: https://docs.fedoraproject.org/en-US/cpe/
16:00:01 <zodbot> Current chairs: bodanel computerkid dtometzki jnsamyak mobrien nirik pingou siddharthvipul zlopez
16:00:01 <nirik> #topic greetings!
16:00:44 <nirik> morning everyone
16:01:10 <austinpowered> .hi
16:01:11 <zodbot> austinpowered: austinpowered 'T.C. Williams' <fedoraproject@wootenwilliams.com>
16:02:07 * nirik fears the DST change has confused folks when the meeting is. ;(
16:02:43 <austinpowered> I move DST should be banned
16:03:32 <austinpowered> Let's just spring forward one more time and leave it that way. :D
16:03:46 <nirik> yeah, I'd be completely fine with that. ;)
16:04:08 <darknao> .hi
16:04:09 <zodbot> darknao: darknao 'Francois Andrieu' <darknao@drkn.ninja>
16:04:14 <zlopez> .hi
16:04:15 <zodbot> zlopez: zlopez 'Michal Konecny' <michal.konecny@psmail.xyz>
16:04:42 <nirik> morning all. Will wait a few more minutes for folks to wander in
16:06:14 <zlopez> It looks like the IRC bridge doesn't work
16:07:47 <nirik> oh? figures it's always having problems for our meetings. ;(
16:08:01 <nirik> Shall I go ahead? or wait a bit more...
16:09:18 * nirik waits till :10 after
16:10:04 <mkonecny> The Fedora infra meeting is happening on libera.chat, the IRC bridge is down
16:10:07 <nirik> #topic New folks introductions
16:10:07 <nirik> #info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
16:10:07 <nirik> #info Getting Started Guide: https://fedoraproject.org/wiki/Infrastructure/GettingStarted
16:10:25 <nirik[m]> The bridge seems up to me?
16:10:46 <zlopez> Strange, I don't see any message from IRC\ on Matrix
16:11:02 <nirik[m]> odd. I see em...
16:11:23 <nirik> Dunno what could be going on then...
16:11:28 <zlopez> I now see messages from :08
16:12:02 <nirik> huh, ok.
16:12:06 <nirik> anyhow, any new folks?
16:12:44 <nirik> moving along...
16:12:47 <nirik> #topic Next chair
16:12:48 <nirik> #info magic eight ball says:
16:12:48 <nirik> #info chair 2021-11-11 - nirik
16:12:48 <nirik> #info chair 2021-11-18 - eddiejennings
16:12:48 <nirik> #info chair 2021-11-25 - zlopez
16:12:48 <nirik> #info char 2021-12-02 - ?
16:13:04 <nirik> anyone want the 2021-12-02 chair? it's quite comfy....
16:13:31 <nirik> if not no biggie, we have 2 weeks ahead at least
16:13:50 <nirik> #topic announcements and information
16:13:50 <nirik> #info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1030 Europe/paris in #centos-meeting
16:13:50 <nirik> #info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
16:13:50 <nirik> #info If your team wants support from the Fedora Program Management Team, file an isssue: https://pagure.io/fedora-pgm/pgm_team/issues?template=support_request
16:13:52 <nirik> #info Fedora 35 is out!
16:13:59 <nirik> any other announcements ?
16:15:14 <nirik> #topic Oncall
16:15:14 <nirik> #info https://fedoraproject.org/wiki/Infrastructure/Oncall
16:15:14 <nirik> #info https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
16:15:14 <nirik> ## .oncalltakeeu .oncalltakeus
16:15:14 <nirik> #info leothecat on call from 2021-11-04 to 2021-11-11
16:15:15 <nirik> #info eddiejennings on call from 2021-11-11 to 2021-11-18
16:15:17 <nirik> #info mobrien on call from 2021-11-18 to 2021-11-25
16:15:19 <nirik> #info LenkaSegura on call from 2021-11-25 to 2021-12-02
16:15:24 <nirik> #info Summary of last week: (from current oncall )
16:15:34 <nirik> Leo[m]1234: any oncall this last week?
16:16:23 <nirik> again I think we are fine since we have several weeks ahead
16:16:28 <nirik> #topic Monitoring discussion [nirik]
16:16:28 <nirik> #info https://nagios.fedoraproject.org/nagios
16:16:28 <nirik> #info Go over existing out items and fix
16:16:55 <nirik> lets see...
16:17:08 <nirik> 3 down arm builders, thats pretty normal it seems
16:17:32 <nirik> we have an alert about our bugzilla2fedmsg... I need to get a new cert for it, it expired. ;(
16:17:47 <nirik> we have an alert on mdapi... it needs a fix.
16:18:03 <nirik> https://pagure.io/mdapi/issue/119
16:18:28 <nirik> I guess pingou is the only maintainer there?
16:18:46 <nirik> mkonecny: was this on our sheet of apps?
16:20:11 <nirik[m]> I wonder if I shouldn't just echo myself here on the matrix side. 😉
16:21:18 <zlopez> nirik: Let me check
16:21:43 <zlopez> nirik: akash should work on it this quarter
16:22:15 <nirik> zlopez: ah ha. so perhaps we should ask pingou to add him so he can fix this.
16:23:31 <zlopez> +1
16:23:48 <nirik> or I can add him... but it would be good for pingou to give him a tour anyhow.
16:24:22 <nirik> anyhow, theres random wrong/small alerts and then some eln messaging queues that are big. Will ask them about that
16:24:28 <nirik> thats it for nagios I think.
16:24:58 <nirik> #topic Fedora Infra backlog refinement
16:24:58 <nirik> #info Refine oldest tickets on https://pagure.io/fedora-infrastructure/issues
16:25:32 <nirik> so, lets see...
16:25:34 <eddiejennings> Sorry I'm late :(.  I am on-call starting today, right?
16:25:51 <nirik> eddiejennings: no worries. yep.
16:26:00 * eddiejennings nods.
16:26:03 <nirik> .ticket 8213
16:26:08 <zodbot> nirik: Issue #8213: fedmsg -> fedora-messaging migration tracker - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8213
16:27:07 <zlopez> This is still in progress, we still have a few apps that need to be migrated
16:27:08 <nirik> so, I don't think much has changed here...
16:27:11 <nirik> yep.
16:27:48 <nirik> I can add a note that not much has changed and we need to still migrate things.
16:27:52 <zlopez> For those wondering why there isn't any learning topic today, we postponed it a week, because mobrian is sick today
16:28:11 <zlopez> nirik: I will check the ticket and see if there is any progress
16:28:41 <nirik> zlopez: ok, thanks!
16:28:54 <zlopez> I would say that few of the thinks could be removed from the list
16:29:12 <zlopez> For example zanata2fedmsg is no longer relevant
16:29:16 <nirik> yeah, there's a few things we likely will never migrate
16:29:20 <nirik> or that
16:29:39 <nirik> busgateway is fedmsg centric...
16:30:12 <zlopez> And some of them could be changed to toddlers
16:30:23 <nirik> I am not sure if leo (other leo) migrated resultsdb when moving to openshift or not.
16:30:49 <zlopez> I will check the ansible and we sill see
16:30:58 <nirik> ok, sounds good.
16:31:11 <nirik> .ticket 9846
16:31:12 <zodbot> nirik: Issue #9846: openid_login() from python-fedora to id.fedoraproject.org fails with 500 when you forget 2nd factor - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9846
16:31:36 <nirik> so this was hopefully going to be solved by a new ipsilon release... but I am not sure where abompard is in that.
16:31:48 <nirik> There's a number of tickets pending on that.
16:32:29 <zlopez> This one is really confusing, it got me few times
16:32:39 <nirik> yeah.
16:34:01 <nirik> I'll ask abompard about this one... and the others ipsilon related.
16:34:39 <zlopez> Yeah, he should know what is happening
16:35:11 <nirik> will do so and update tickets.
16:35:28 <nirik> .ticket 9512
16:35:30 <zodbot> nirik: Issue #9512: various Fedora servers are not logging correctly. - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9512
16:36:52 <nirik> so... no one has really had time to investigate this and share options.
16:37:00 <nirik> might be a good one for a apprentice?
16:37:28 <zlopez> We could add easyfix label
16:38:17 * austinpowered raises hand
16:38:18 <nirik> yeah. It's not fully easy to fix, but it would be easy to research and give options... do we increase something in rsyslog to prevent it? do we filter things to reduce amount of logs?
16:38:49 <nirik> austinpowered: if you want to look into it that would be great!
16:39:48 <austinpowered> I will look at the ticket and reach out when I need help
16:40:21 <nirik> great
16:40:45 <nirik> .ticket 9624
16:40:46 <zodbot> nirik: Issue #9624: s390x builders run out of memory or disk while building ceph - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9624
16:41:10 <nirik> I was waiting on these finishing their move to the new mainframe, then was going to redo them with f35.
16:41:40 <nirik> will note that in the ticket.
16:42:40 <zlopez> Nice
16:46:17 <nirik> sorry, got sidetracked
16:47:10 <nirik> .ticket 8455
16:47:11 <zodbot> nirik: Issue #8455: Move mailman to newer release of Fedora or CentOS - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/8455
16:47:17 <nirik> we have discussed this one before...
16:47:28 <nirik> I don't know that there's been much progress. we could ask on ticket tho
16:48:14 <nirik> .ticket 9831
16:48:16 <zodbot> nirik: Issue #9831: Patches from lists.fedoraproject.org contain trailing ^M which cause the failure of git am - fedora-infrastructure - Pagure.io - https://pagure.io/fedora-infrastructure/issue/9831
16:48:35 <nirik> I am not sure about this one, again I think it will take investigation... it might take moving mailman to newer.
16:48:59 <zlopez> So this is probably related to #8455
16:49:57 <nirik> it could be. not sure.
16:51:21 <nirik> ok, one more? or open floor?
16:51:43 <zlopez> I think we can switch to open floor
16:51:51 <nirik> #topic open floor
16:51:52 <zlopez> We went through plenty of tickets today
16:52:04 <nirik> yeah.
16:52:11 <nirik> anyone have anything else to bring up?
16:52:49 <jg-dev> I was going to ask about getting added to the fi-apprentice apprentice group, but happy to do that on the admin channel if more appropriate :)
16:53:21 <nirik> sure, can do that after... we need to know your fas account login...
16:54:17 <jg-dev> cool, thank you, I think it should be jg-dev
16:55:31 <nirik> added you. :) welcome!
16:55:37 <nirik> ok, thanks for coming everyone...
16:55:40 <nirik> #endmeeting