#meeting-3:fedoraproject.org log

<@phsmoura:fedora.im>

17:00:06

!startmeeting Infrastructure (2025-01-09)

<@meetbot:fedora.im>

17:00:07

Meeting started at 2025-01-09 17:00:06 UTC

<@meetbot:fedora.im>

17:00:07

The Meeting name is 'Infrastructure (2025-01-09)'

<@phsmoura:fedora.im>

17:00:14

!info About our team: https://docs.fedoraproject.org/en-US/cpe/

<@phsmoura:fedora.im>

17:00:14

!meetingname infrastructure

<@phsmoura:fedora.im>

17:00:14

!chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp

<@phsmoura:fedora.im>

17:00:14

!info Agenda is at: https://board.net/p/fedora-infra

<@phsmoura:fedora.im>

17:00:14

!topic ahoy

<@phsmoura:fedora.im>

17:00:14

!info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra

<@meetbot:fedora.im>

17:00:15

The Meeting Name is now infrastructure

<@phsmoura:fedora.im>

17:00:22

hello

<@nirik:matrix.scrye.com>

17:00:38

morning

<@Zlopez:matrix.org>

17:01:51

!hi

<@zodbot:fedora.im>

17:01:54

Michal Konecny (zlopez)

<@james:fedora.im>

17:02:03

!hi

<@zodbot:fedora.im>

17:02:04

James Antill (james)

<@phsmoura:fedora.im>

17:02:38

!info This is a place where people who are interested in Fedora Infrastructure can introduce themselves

<@phsmoura:fedora.im>

17:02:38

!info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/

<@phsmoura:fedora.im>

17:02:38

!topic New folks introductions

<@phsmoura:fedora.im>

17:02:52

do we have new people around?

<@phsmoura:fedora.im>

17:03:05

feel free to introduce yourselves

<@nirik:matrix.scrye.com>

17:04:10

It's a new year, does that count? 😄

<@Zlopez:matrix.org>

17:04:39

You think the 2025 will be the year of Fedora?

<@Zlopez:matrix.org>

17:04:58

With Fedora 42 it's possible :-D

<@phsmoura:fedora.im>

17:05:25

!info chair 2025-01-16 - saibug

<@phsmoura:fedora.im>

17:05:25

!info chair 2025-01-23 - Yaash

<@phsmoura:fedora.im>

17:05:25

!topic Next chair

<@phsmoura:fedora.im>

17:05:25

!info magic eight ball says:

<@phsmoura:fedora.im>

17:05:25

!info chair 2025-01-30 - ???

<@phsmoura:fedora.im>

17:05:53

we are good for the next 2 wks, but is there any volunteer for 2025-01-30?

<@Zlopez:matrix.org>

17:06:36

Let's see next week

<@phsmoura:fedora.im>

17:07:04

ok, moving on then

<@phsmoura:fedora.im>

17:07:09

!info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org

<@phsmoura:fedora.im>

17:07:09

!topic announcements and information

<@phsmoura:fedora.im>

17:07:09

!info CLE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org

<@phsmoura:fedora.im>

17:07:24

any other announcement?

<@nirik:matrix.scrye.com>

17:08:13

!info mass rebuild is next wed

<@Zlopez:matrix.org>

17:09:16

I completely forgot about that one

<@nirik:matrix.scrye.com>

17:10:46

coming up fast this time.

<@nirik:matrix.scrye.com>

17:11:36

I'm hoping to improve ppc64le builders before then... but we will see if I can. Probibly wouldn't hurt to update/reboot all the builders before then too...

<@nirik:matrix.scrye.com>

17:12:49

no other announcements I can think of

<@phsmoura:fedora.im>

17:13:04

lets see oncall

<@Zlopez:matrix.org>

17:13:23

I don't think we had any during shutdown

<@nirik:matrix.scrye.com>

17:13:58

we didn't. but... I forgot to remove myself... so I guess I was.

<@nirik:matrix.scrye.com>

17:14:19

but only one oncall that I recall. Someone with questions about one of the blogs

<@james:fedora.im>

17:15:06

Yeh, I remember one instance of nirik running !oncall to show nobody was ... and it returned nirik is on call. Was kind of amusing, while slightly drunk.

<@nirik:matrix.scrye.com>

17:15:55

heh, yeah, thats the one.

<@nirik:matrix.scrye.com>

17:16:03

The user DM'ed me their question...

<@phsmoura:fedora.im>

17:16:42

!info ??? is on call from 2025-01-23 to 2025-01-30

<@phsmoura:fedora.im>

17:16:42

!topic Oncall

<@phsmoura:fedora.im>

17:16:42

!info https://docs.fedoraproject.org/en-US/infra/day_to_day_fedora/#_the_oncall_role_in_our_team

<@phsmoura:fedora.im>

17:16:42

!info nirik is on call from 2024-12-06 to 2024-12-12

<@phsmoura:fedora.im>

17:16:42

!info NOBODY is on call from 2024-12-13 to 2025-01-09

<@phsmoura:fedora.im>

17:16:42

!info ??? is on call from 2025-01-09 to 2025-01-16

<@phsmoura:fedora.im>

17:16:42

!info ??? is on call from 2025-01-16 to 2025-01-23

<@phsmoura:fedora.im>

17:17:07

volunteers for next wks tobe oncall?

<@phsmoura:fedora.im>

17:17:19

volunteers for next wks to be oncall?

<@Zlopez:matrix.org>

17:17:25

I will not be here next week, but I can take the one from 16th January

<@nirik:matrix.scrye.com>

17:17:36

I guess I could just keep it another week. ;)

<@nirik:matrix.scrye.com>

17:17:51

It doesn't seem like people use it as much on matrix vs irc. I am not sure why.

<@Zlopez:matrix.org>

17:18:02

And I remember there was the proposal to rotate the oncall and chairing, not sure if we got to any conclusion

<@nirik:matrix.scrye.com>

17:18:42

I'm fine doing that... as long as we allow new folks to take it if they want to....

<@Zlopez:matrix.org>

17:18:54

That was the idea

<@nirik:matrix.scrye.com>

17:19:09

but we might ping back in that discssion and make sure everyone is ok and then start doing it next week or two?

<@Zlopez:matrix.org>

17:19:31

We need to clear the chair list first

<@Zlopez:matrix.org>

17:19:47

And then use it for the rotation

<@Zlopez:matrix.org>

17:20:41

But I'm for it to start use it soon

<@nirik:matrix.scrye.com>

17:20:41

<@nirik:matrix.scrye.com>

17:21:01

I don't think anyone really objected

<@Zlopez:matrix.org>

17:21:31

I don't remember any response in the discussion that would be against it, or here on the meeting

<@phsmoura:fedora.im>

17:22:43

<@phsmoura:fedora.im>

17:22:43

!info nirik is on call from 2025-01-09 to 2025-01-16

<@phsmoura:fedora.im>

17:22:43

!info zlopez is on call from 2025-01-16 to 2025-01-23

<@phsmoura:fedora.im>

17:22:43

so we have this for the next weeks:

<@seddik:fedora.im>

17:23:05

!hi

<@zodbot:fedora.im>

17:23:06

seddik alaouiismaili (seddik)

<@nirik:matrix.scrye.com>

17:23:20

👍

<@nirik:matrix.scrye.com>

17:23:31

hey saibug. Hope you had a good holidays.

<@seddik:fedora.im>

17:24:15

Yes it was great

<@phsmoura:fedora.im>

17:24:48

!info Summary of last week: (@nirik)

<@phsmoura:fedora.im>

17:24:48

there was nobody oncall during the holidays and the last person oncall was nirik..

<@phsmoura:fedora.im>

17:25:08

there was nobody oncall during the holidays and the last person oncall was nirik..

<@phsmoura:fedora.im>

17:25:08

!info Summary of last week: ( nirik )

<@nirik:matrix.scrye.com>

17:25:32

just that one oncall that I recall... nothing else. Was pretty quiet... just a few alerts and such...

<@phsmoura:fedora.im>

17:26:14

!info Go over existing items and fix them

<@phsmoura:fedora.im>

17:26:14

!info https://nagios.fedoraproject.org/nagios

<@phsmoura:fedora.im>

17:26:14

!topic Monitoring discussion [nirik]

<@nirik:matrix.scrye.com>

17:26:48

lets see...

<@nirik:matrix.scrye.com>

17:27:22

we have proxy37 thats not behaving (I think due to network issues between iad2 and there)

<@nirik:matrix.scrye.com>

17:27:45

for logdetective02 I think there was a PR from saibug. Should look back at it

<@nirik:matrix.scrye.com>

17:28:09

there's a copr-dist-git, but it's acked, so I assume they know about it.

<@nirik:matrix.scrye.com>

17:28:15

otherwise, looking good. ;)

<@nirik:matrix.scrye.com>

17:28:33

oh, one other thing to note:

<@nirik:matrix.scrye.com>

17:29:23

we have been getting mailman alerts most days... when the updates pushes are happening. I looked slightly at it, and it looks like it's when sending to the package-announce list its getting a lot of bounces. But I am not sure why. We should look into it and fix it, getting daily alerts on it is anoying.

<@nirik:matrix.scrye.com>

17:30:16

alerts are like: PROBLEM [noc01]: mail_queue on mailman01.iad2.fedoraproject.org is CRITICAL 1014 mail(s) in queue

<@nirik:matrix.scrye.com>

17:31:00

If someone would like to investgate, feel free. ;)

<@nirik:matrix.scrye.com>

17:31:09

I don't think there's anything else nagios wise.

<@phsmoura:fedora.im>

17:31:24

maybe I can take a look while I dont have access to work on issue 12118?

<@nirik:matrix.scrye.com>

17:32:20

sure, it's probibly looking over logs from that time to see whats going on...

<@phsmoura:fedora.im>

17:33:13

ok, lets see blocked tickets

<@phsmoura:fedora.im>

17:33:26

!topic Revisit blocked tickets

<@phsmoura:fedora.im>

17:33:26

!info Check if any blocked ticket is unblocked

<@phsmoura:fedora.im>

17:33:26

!link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status=

<@nirik:matrix.scrye.com>

17:34:20

still blocked I guess... although OIDC isn't really... blocked so much as needs some last few things done

<@nirik:matrix.scrye.com>

17:34:52

copr and anitya are left?

<@Zlopez:matrix.org>

17:35:58

I need to release new Anitya version, it has OIDC support

<@Zlopez:matrix.org>

17:36:51

Only COPR will be left

<@nirik:matrix.scrye.com>

17:37:51

I'll mention that on the copr bug.

<@phsmoura:fedora.im>

17:39:03

!topic Fedora Infra backlog refinement

<@phsmoura:fedora.im>

17:39:03

!link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc

<@phsmoura:fedora.im>

17:39:03

!info Refine oldest tickets on Fedora Infra tracker

<@nirik:matrix.scrye.com>

17:40:22

ok, we haven't done this in a while... shall we take them one by one (we do only have ~20min tho)

<@Zlopez:matrix.org>

17:40:51

I'm for going through few of them

<@nirik:matrix.scrye.com>

17:41:25

go for it then...

<@Zlopez:matrix.org>

17:44:10

!ticket 7361

<@zodbot:fedora.im>

17:44:12

**fedora-infrastructure #7361** (https://pagure.io/fedora-infrastructure/issue/7361):**Signing infrastructure for aarch64**

<@zodbot:fedora.im>

17:44:12

<@zodbot:fedora.im>

17:44:12

● **Opened:** 6 years ago by pbrobinson

<@zodbot:fedora.im>

17:44:12

● **Last Updated:** 5 months ago

<@zodbot:fedora.im>

17:44:12

● **Assignee:** Not Assigned

<@Zlopez:matrix.org>

17:44:30

That is one of the blocked ones, I see

<@nirik:matrix.scrye.com>

17:44:31

this is... being worked on, but is complicated.

<@Zlopez:matrix.org>

17:45:00

!ticket 12144

<@zodbot:fedora.im>

17:45:02

● **Last Updated:** 5 months ago

<@zodbot:fedora.im>

17:45:02

**fedora-infrastructure #12144** (https://pagure.io/fedora-infrastructure/issue/12144):**Add support for creating RDS instances under `testing-farm-` prefix**

<@zodbot:fedora.im>

17:45:02

● **Assignee:** Not Assigned

<@zodbot:fedora.im>

17:45:02

<@zodbot:fedora.im>

17:45:02

● **Opened:** 5 months ago by mvadkert

<@nirik:matrix.scrye.com>

17:45:25

hum, must have missed this one when i landed. ;(

<@nirik:matrix.scrye.com>

17:45:44

just needs doing. I can or perhaps David Kirwan might ?

<@Zlopez:matrix.org>

17:45:56

We didn't even responded to this one and it's sitting there for 5 months already

<@nirik:matrix.scrye.com>

17:46:05

yeah, not sure why I missed it.

<@nirik:matrix.scrye.com>

17:46:16

it did get triaged.

<@nirik:matrix.scrye.com>

17:46:54

proposal: say we are sorry in ticket, cc David Kirwan and if he can't get to it, I can try in the next few days.

<@Zlopez:matrix.org>

17:47:39

I did that already

<@nirik:matrix.scrye.com>

17:48:04

ok, cool

<@Zlopez:matrix.org>

17:48:20

!ticket 12146

<@zodbot:fedora.im>

17:48:21

● **Assignee:** Not Assigned

<@zodbot:fedora.im>

17:48:21

**fedora-infrastructure #12146** (https://pagure.io/fedora-infrastructure/issue/12146):**Manage our new testing.farm domain via AWS Route53**

<@zodbot:fedora.im>

17:48:21

<@zodbot:fedora.im>

17:48:21

● **Opened:** 5 months ago by mvadkert

<@zodbot:fedora.im>

17:48:21

● **Last Updated:** 5 months ago

<@Zlopez:matrix.org>

17:48:33

Another one for testing.farm

<@nirik:matrix.scrye.com>

17:48:49

I can do this one.

<@Zlopez:matrix.org>

17:48:54

And another one without reply for 5 months :-/

<@nirik:matrix.scrye.com>

17:49:04

again, not sure why it was missed. ;(

<@Zlopez:matrix.org>

17:49:07

They were probably opened at the same time

<@nirik:matrix.scrye.com>

17:49:20

but also... wonder why they didn't ping back to ask us?

<@Zlopez:matrix.org>

17:49:46

Maybe they solved that differently in the meantime and forgot about the ticket

<@nirik:matrix.scrye.com>

17:50:30

humm... .

<@nirik:matrix.scrye.com>

17:50:34

it seems to be done already

<@nirik:matrix.scrye.com>

17:51:01

So, let me ask them and close it? or just ask?

<@nirik:matrix.scrye.com>

17:51:27

I wonder if that other one is already done also

<@Zlopez:matrix.org>

17:51:30

If it's done let's ask and close it, they can reopen if something is still missing

<@Zlopez:matrix.org>

17:51:57

Probably somebody did this with AWS rights and just forgot about the ticket

<@nirik:matrix.scrye.com>

17:52:44

yeah, I'd like to add more auditing so we can see things like this, but haven't gotten to it.

<@Zlopez:matrix.org>

17:53:27

At least we see that we didn't do a backlog refinement for some time

<@nirik:matrix.scrye.com>

17:53:37

yeah

<@Zlopez:matrix.org>

17:54:07

!ticket 11804

<@zodbot:fedora.im>

17:54:09

**fedora-infrastructure #11804** (https://pagure.io/fedora-infrastructure/issue/11804):**rhel7 EOL - github2fedmsg**

<@zodbot:fedora.im>

17:54:09

<@zodbot:fedora.im>

17:54:09

● **Opened:** 11 months ago by zlopez

<@zodbot:fedora.im>

17:54:09

● **Last Updated:** 5 months ago

<@zodbot:fedora.im>

17:54:09

● **Assignee:** abompard

<@nirik:matrix.scrye.com>

17:55:01

yeah, I don't know where we are here... but we should retire it soon

<@Zlopez:matrix.org>

17:55:02

We need to announce the sunset and replacement

<@Zlopez:matrix.org>

17:55:21

Replacement is already in place, people just need to migrate

<@nirik:matrix.scrye.com>

17:55:24

The replacement was announced... I think tickets were opened for all existing ones.

<@nirik:matrix.scrye.com>

17:55:52

trying to recall where that tracker was

<@Zlopez:matrix.org>

17:56:00

I will need to push this a little, as it's the last thing blocking both fedmsg and rhel7 EOL

<@nirik:matrix.scrye.com>

17:56:57

yeah, I can't seem to find that tracker, I know there was one... at least I think there was.

<@Zlopez:matrix.org>

17:57:10

Found it https://pagure.io/fedora-infra/w2fm-migration/

<@nirik:matrix.scrye.com>

17:57:37

ah yep.

<@nirik:matrix.scrye.com>

17:57:40

I just got there too.

<@Zlopez:matrix.org>

17:57:55

I went through the announcement blog post :-D

<@nirik:matrix.scrye.com>

17:57:57

so yeah, we should really push on getting this done

<@Zlopez:matrix.org>

17:58:12

I think this is the last one for backlog refinement

<@james:fedora.im>

17:58:20

https://pagure.io/fedora-infrastructure/issue/8213

<@james:fedora.im>

17:58:47

I think that's the tracker

<@nirik:matrix.scrye.com>

17:59:42

one of them. ;) Thats the fedmsg->fedora-messaging, but the last one of that is the github2fedmsg -> webhook2fedoramessaging (which is that other tracker)

<@nirik:matrix.scrye.com>

18:00:05

so, once we finish this we can close a bunch of things and shut off a bunch of vm's and remove a bunch of ansible

<@Zlopez:matrix.org>

18:00:36

I replied in the github2fedmsg ticket, if @abompard:fedora.im doesn't have time to work on that I will do the announcement

<@nirik:matrix.scrye.com>

18:00:55

I assume we want to retire fedmsg at the same time?

<@Zlopez:matrix.org>

18:00:55

Let's give users a month till we pull the plug

<@Zlopez:matrix.org>

18:01:01

Yes

<@nirik:matrix.scrye.com>

18:01:17

I'd say a few weeks... but I just want it done. ;)

<@Zlopez:matrix.org>

18:02:07

Let's go with three weeks than

<@nirik:matrix.scrye.com>

18:02:59

sure

<@Zlopez:matrix.org>

18:03:07

I think we are already overtime

<@Zlopez:matrix.org>

18:03:18

We can discuss that in the ticket itself

<@nirik:matrix.scrye.com>

18:03:19

yep

<@phsmoura:fedora.im>

18:05:21

we can talk about anything else on noc

<@phsmoura:fedora.im>

18:05:34

thanks everyone for joinning today :)

<@phsmoura:fedora.im>

18:05:49

!endmeeting