#meeting-3:fedoraproject.org log

<@lenkaseg:fedora.im>

16:00:25

!startmeeting Infrastructure (2024-05-16)

<@meetbot:fedora.im>

16:00:27

Meeting started at 2024-05-16 16:00:25 UTC

<@meetbot:fedora.im>

16:00:27

The Meeting name is 'Infrastructure (2024-05-16)'

<@lenkaseg:fedora.im>

16:00:39

!meetingname infrastructure

<@lenkaseg:fedora.im>

16:00:45

!chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp

<@lenkaseg:fedora.im>

16:00:52

!info Agenda is at: https://board.net/p/fedora-infra

<@leo:fedora.im>

16:00:52

hey!

<@lenkaseg:fedora.im>

16:00:58

About our team: https://docs.fedoraproject.org/en-US/cpe/

<@lenkaseg:fedora.im>

16:01:03

Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra

<@lenkaseg:fedora.im>

16:01:09

!topic namaste

<@lenkaseg:fedora.im>

16:01:19

Morning/evening/afternoon everybody!

<@aggraxis:fedora.im>

16:01:25

Hello!

<@Zlopez:matrix.org>

16:02:47

!hi

<@zodbot:fedora.im>

16:02:49

Michal Konecny (zlopez)

<@lenkaseg:fedora.im>

16:03:46

!topic New folks introductions

<@lenkaseg:fedora.im>

16:03:55

!info This is a place where people who are interested in Fedora Infrastructure can introduce themselves Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/

<@lenkaseg:fedora.im>

16:04:06

Are there any new folks today?

<@jnsamyak:matrix.org>

16:04:36

!hi

<@zodbot:fedora.im>

16:04:38

Samyak Jain (jnsamyak) - he / him / his

<@lenkaseg:fedora.im>

16:06:12

Nobody new this time but nevermind!

<@lenkaseg:fedora.im>

16:06:19

!topic Next chair

<@lenkaseg:fedora.im>

16:06:25

!info magic eight ball says:

<@lenkaseg:fedora.im>

16:06:33

!info chair 2024-05-16 lenkaseg

<@lenkaseg:fedora.im>

16:07:02

That is me and we have no more chairs. Any volunteer for May23rd?

<@lenkaseg:fedora.im>

16:08:12

(Ideal for new folks as well!)

<@Zlopez:matrix.org>

16:08:21

I can take it

<@lenkaseg:fedora.im>

16:08:31

sold to zlopez!

<@lenkaseg:fedora.im>

16:08:41

!info chair 2024-05-23 zlopez

<@lenkaseg:fedora.im>

16:08:51

Anybody for May30th?

<@lenkaseg:fedora.im>

16:09:13

Unfortunately I will not be around that time

<@lenkaseg:fedora.im>

16:10:41

No worries, can be decided next time

<@lenkaseg:fedora.im>

16:10:48

!topic announcements and information

<@lenkaseg:fedora.im>

16:10:58

!info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org

<@nirik:matrix.scrye.com>

16:11:01

morning

<@lenkaseg:fedora.im>

16:11:06

!info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3

<@nirik:matrix.scrye.com>

16:11:48

!info tomorrow is a Red Hat day of learning and next friday is a Red Hat Recharge day.

<@Zlopez:matrix.org>

16:12:28

!info ARC dist-git investigation started https://communityblog.fedoraproject.org/2024-git-forge-evaluation/

<@Zlopez:matrix.org>

16:13:36

!info There was problem with release-monitoring not filling bugs for packagers that is solved now

<@lenkaseg:fedora.im>

16:15:10

if that's all from announcements, let's move on to...

<@Zlopez:matrix.org>

16:15:34

!info F38 is EOL next week

<@lenkaseg:fedora.im>

16:16:05

!topic Oncall

<@nirik:matrix.scrye.com>

16:16:26

wiki and 2 osbuild vm's are all thats left on f38 that I know of for vm's.

<@lenkaseg:fedora.im>

16:16:56

Ah yes, for me time when I usually move to newly releases fedora

<@lenkaseg:fedora.im>

16:17:06

Ah yes, for me time when I usually move to newly released fedora

<@lenkaseg:fedora.im>

16:17:34

!info https://fedoraproject.org/wiki/Infrastructure/Oncall https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/

<@lenkaseg:fedora.im>

16:17:43

!info aheath1992 is on call from 2024-05-09 to 2024-05-16

<@lenkaseg:fedora.im>

16:17:53

??? is on call from 2024-05-16 to 2024-05-23

<@leo:fedora.im>

16:18:06

i'll take it

<@lenkaseg:fedora.im>

16:18:08

Any volunteer for oncall starting today?

<@lenkaseg:fedora.im>

16:18:17

thank you!

<@lenkaseg:fedora.im>

16:18:32

!info leo is on call from 2024-05-16 to 2024-05-23

<@lenkaseg:fedora.im>

16:18:55

Anybody willing to take following week?

<@nirik:matrix.scrye.com>

16:19:02

I can take it

<@lenkaseg:fedora.im>

16:19:25

Excellent, thanks!

<@lenkaseg:fedora.im>

16:19:28

!info nirik is on call from 2024-05-23 to 2024-05-30

<@lenkaseg:fedora.im>

16:19:35

!info Summary of last week: (from current oncall)

<@lenkaseg:fedora.im>

16:19:56

I believe the last oncall was aheath1992

<@lenkaseg:fedora.im>

16:20:13

Over to you aheath1992

<@aheath1992:matrix.org>

16:21:48

No on call messages that i was aware of

<@aheath1992:matrix.org>

16:21:51

all quiet

<@lenkaseg:fedora.im>

16:22:05

Thanks for the report!

<@lenkaseg:fedora.im>

16:22:16

Let's move on to

<@lenkaseg:fedora.im>

16:22:17

!topic Monitoring discussion [nirik]

<@lenkaseg:fedora.im>

16:22:28

Over to nirik

<@nirik:matrix.scrye.com>

16:22:59

so... aside from a proxy14 blep just now... we only have 2 alerts left.

<@nirik:matrix.scrye.com>

16:23:13

"Check datanommer for recent fedbadges messages"

<@nirik:matrix.scrye.com>

16:23:33

I am not sure if this is checking the old badges? or is looking for something the new one isn't emitting? we need to investigate

<@nirik:matrix.scrye.com>

16:24:21

and "Check MySQL Backup" this is on the new db03... selinux is denying nagios/nrpe perm to look at a link in /backups/ ;( So, needs a selinux policy adjustment or the like.

<@nirik:matrix.scrye.com>

16:24:35

thats it. Everything else is green... no more disk or other issues. ;)

<@lenkaseg:fedora.im>

16:24:56

Thanks nirik !

<@Zlopez:matrix.org>

16:25:32

I have something as well

<@lenkaseg:fedora.im>

16:25:37

Alright, next topic could be:

<@lenkaseg:fedora.im>

16:25:46

go on Zlopez

<@Zlopez:matrix.org>

16:26:24

I noticed that there are plenty of errors reported by cron on staging proxy01 for `/usr/local/bin/restart-mirrorlist-processes`

<@Zlopez:matrix.org>

16:26:34

``` Error: mirrorlist1 is not running as expected. Trying to restart ERROR: mirrorlist2 not processing correctly ```

<@nirik:matrix.scrye.com>

16:26:51

yes, phsmoura was reinstalling that and looking into it

<@Zlopez:matrix.org>

16:27:10

<@lenkaseg:fedora.im>

16:27:58

So let's decide what we want to look into:

<@lenkaseg:fedora.im>

16:28:06

RHEL7 EOL

<@lenkaseg:fedora.im>

16:28:14

Revisit blocked tickets

<@lenkaseg:fedora.im>

16:28:20

backlog refinement

<@lenkaseg:fedora.im>

16:28:36

something else

<@lenkaseg:fedora.im>

16:28:43

Please vote!

<@lenkaseg:fedora.im>

16:29:53

Seems that RHEL7 EOL it is (and will be till June30th) 😀

<@lenkaseg:fedora.im>

16:30:00

!topic RHEL7 EOL

<@lenkaseg:fedora.im>

16:30:05

metaticket here:

<@lenkaseg:fedora.im>

16:30:14

!ticket https://pagure.io/fedora-infrastructure/issue/11815

<@Zlopez:matrix.org>

16:30:39

The ticket number should be enough

<@Zlopez:matrix.org>

16:30:45

!ticket 11815

<@zodbot:fedora.im>

16:30:47

**fedora-infrastructure #11815** (https://pagure.io/fedora-infrastructure/issue/11815):**rhel7 eol** ● **Opened:** 2 months ago by zlopez ● **Last Updated:** 2 months ago ● **Assignee:** zlopez

<@lenkaseg:fedora.im>

16:31:08

ok...where did we stopped last time..

<@lenkaseg:fedora.im>

16:31:20

ok...where did we stop last time..

<@Zlopez:matrix.org>

16:31:31

We now have db-fas01 migrated thanks to nirik

<@Zlopez:matrix.org>

16:31:55

The Badges were migrated as well

<@nirik:matrix.scrye.com>

16:32:24

I did a few more with the outages...

<@lenkaseg:fedora.im>

16:33:27

How is it with Fedora People?

<@lenkaseg:fedora.im>

16:33:35

!ticket 11822

<@zodbot:fedora.im>

16:33:37

**fedora-infrastructure #11822** (https://pagure.io/fedora-infrastructure/issue/11822):**rhel7 - Fedora people** ● **Opened:** 2 months ago by zlopez ● **Last Updated:** a week ago ● **Assignee:** lenkaseg

<@nirik:matrix.scrye.com>

16:33:56

waiting for planet still.

<@nirik:matrix.scrye.com>

16:34:11

phsmoura is sending out a devel-announce about that today hopefully.

<@nirik:matrix.scrye.com>

16:34:21

retiring it on the 29th

<@Zlopez:matrix.org>

16:34:21

There is a onset date for planet 21st May

<@nirik:matrix.scrye.com>

16:34:34

yes, but that was too soon, so I asked to push it out a week

<@Zlopez:matrix.org>

16:34:40

<@nirik:matrix.scrye.com>

16:34:50

less than a week is not enough time for some folks

<@Zlopez:matrix.org>

16:35:02

I'm currently working on mailman3

<@Zlopez:matrix.org>

16:35:11

!ticket 8455

<@zodbot:fedora.im>

16:35:13

**fedora-infrastructure #8455** (https://pagure.io/fedora-infrastructure/issue/8455):**Move mailman to newer release of Fedora or CentOS** ● **Opened:** 4 years ago by smooge ● **Last Updated:** 3 days ago ● **Assignee:** zlopez

<@Zlopez:matrix.org>

16:35:25

And sundries server

<@Zlopez:matrix.org>

16:35:39

!ticket 11821

<@lenkaseg:fedora.im>

16:35:41

me as well, working on mailman-sar,py

<@zodbot:fedora.im>

16:35:41

**fedora-infrastructure #11821** (https://pagure.io/fedora-infrastructure/issue/11821):**rhel7 - sundries servers** ● **Opened:** 2 months ago by zlopez ● **Last Updated:** a week ago ● **Assignee:** zlopez

<@lenkaseg:fedora.im>

16:35:50

me as well, working on mailman-sar.py

<@nirik:matrix.scrye.com>

16:35:53

how are we looking on pdc these days?

<@lenkaseg:fedora.im>

16:36:34

So, owner-sync-pagure is tested on bodhi-backend staging, seems to be working

<@lenkaseg:fedora.im>

16:36:46

New PR opened here:

<@lenkaseg:fedora.im>

16:37:09

https://pagure.io/fedora-infra/ansible/pull-request/2022

<@nirik:matrix.scrye.com>

16:37:13

nice. we can try and push that out and see. Thanks for adjusting it.

<@Zlopez:matrix.org>

16:37:24

There are still few other scripts using PDC from what I remember

<@lenkaseg:fedora.im>

16:37:43

And jednorozec is working on fedfind

<@lenkaseg:fedora.im>

16:38:34

in ansible repo there should not be, there were 4: 3 removed as no longer necessary and 1 fixed (that was the owner-sync-pagure)

<@lenkaseg:fedora.im>

16:38:47

In toddlers there are still some pdc calls

<@Zlopez:matrix.org>

16:38:55

I saw something in releng repo and toddlers

<@lenkaseg:fedora.im>

16:38:56

I think Anton was working on it

<@nirik:matrix.scrye.com>

16:39:45

what abount the src.fp.org commit hook? did we aready change that?

<@lenkaseg:fedora.im>

16:40:10

in releng repo there should be nothing more to fix if I recall correctly from taliing with jednorozec

<@lenkaseg:fedora.im>

16:40:15

only cleanup possibly

<@Zlopez:matrix.org>

16:40:55

It would be nice to update the PDC ticket with what is remaining

<@lenkaseg:fedora.im>

16:40:58

I think so, let me find the PR, but yeah, pagure needs to be released. We were waiting for some last PRs to land

<@Zlopez:matrix.org>

16:41:07

!ticket 11820

<@zodbot:fedora.im>

16:41:09

**fedora-infrastructure #11820** (https://pagure.io/fedora-infrastructure/issue/11820):**rhel7 - PDC** ● **Opened:** 2 months ago by zlopez ● **Last Updated:** a month ago ● **Assignee:** humaton

<@lenkaseg:fedora.im>

16:41:14

Good idea, I can do that

<@nirik:matrix.scrye.com>

16:41:38

ah, waiting for a pagure release? I thought it was just a hook edit

<@lenkaseg:fedora.im>

16:42:00

Here it is: https://pagure.io/pagure-dist-git/pull-request/164

<@nirik:matrix.scrye.com>

16:42:13

ah, a pagure dist git release

<@lenkaseg:fedora.im>

16:42:26

Yes, sorry

<@Zlopez:matrix.org>

16:42:49

But I see that this is waiting for pagure changes as well :-/

<@nirik:matrix.scrye.com>

16:43:26

I'm not sure the status on a new pagure 5.x release... they have been working on 6.0 mostly.

<@lenkaseg:fedora.im>

16:43:42

in releng repo there should be nothing more to fix if I recall correctly from talking with jednorozec

<@lenkaseg:fedora.im>

16:44:05

Correct, both should be released

<@Zlopez:matrix.org>

16:44:09

I have few changes that are waiting for pagure 6.0 as well

<@nirik:matrix.scrye.com>

16:44:36

6.0 will be a big release... we will need to move to rhel9

<@nirik:matrix.scrye.com>

16:44:59

so fun that all this is so tangled up. ;)

<@lenkaseg:fedora.im>

16:45:07

after 2 years or so

<@Zlopez:matrix.org>

16:45:48

At least we are getting slowly off fedmsg

<@nirik:matrix.scrye.com>

16:45:50

we could backport the pagure patches to epel8 for us...

<@aggraxis:fedora.im>

16:45:53

Yeah, I just started reading the history on this. Tangled up inded.

<@aggraxis:fedora.im>

16:46:04

Yeah, I just started reading the history on this. Tangled up indeed.

<@Zlopez:matrix.org>

16:46:26

I'm not sure how long it will take for pagure 6.0 to be released

<@nirik:matrix.scrye.com>

16:46:28

oh, and that reminds me. Do we intend to retire fedmsg at rhel7 eol? If not, we need to move busgateway* vm's to rhel9.

<@Zlopez:matrix.org>

16:46:46

Let me check

<@nirik:matrix.scrye.com>

16:46:50

I don't know if we have anything still using it.

<@Zlopez:matrix.org>

16:46:55

!ticket 8213

<@zodbot:fedora.im>

16:46:58

**fedora-infrastructure #8213** (https://pagure.io/fedora-infrastructure/issue/8213):**fedmsg -> fedora-messaging migration tracker** ● **Opened:** 4 years ago by kevin ● **Last Updated:** 13 minutes ago ● **Assignee:** zlopez

<@Zlopez:matrix.org>

16:47:51

This is what remains: * busgateway.yml: will die when fedmsg dies * fedimg.yml: to be retired I think? * github2fedmsg.yml: waiting for rewrite (webhook2fedmsg) * mailman.yml: waiting for server upgrade to EL9 * mbs.yml: to be retired * mirrormanager.yml: rewrite done, to be deployed * pdc.yml: Retirement in progress

<@Zlopez:matrix.org>

16:48:07

We have somebody working on fedimg replacement

<@Zlopez:matrix.org>

16:48:24

We have outreachy intern working on github2fedmsg replacement

<@nirik:matrix.scrye.com>

16:48:32

yep.

<@Zlopez:matrix.org>

16:48:34

I'm working on mailman

<@nirik:matrix.scrye.com>

16:48:40

any status on github2fedmsg?

<@Zlopez:matrix.org>

16:49:15

We had first meeting with outreachy intern, he should have enough time to work on webhook2fedmsg replacement

<@nirik:matrix.scrye.com>

16:49:43

Is mailman actually using fedmsg in prod... I am not seeing anything in the zmq-amqp gateway for it.

<@Zlopez:matrix.org>

16:50:03

I'm interested about mirrormanager, how it is looking with it?

<@nirik:matrix.scrye.com>

16:50:16

I'm not sure, we need to ping Aurélien B.

<@Zlopez:matrix.org>

16:50:42

nirik: I'm not sure the fedmsg archiver plugin was already ported to fedora messaging, so it depends what version the mailman in prod is using

<@nirik:matrix.scrye.com>

16:51:02

yeah, I think it's not using fedsmg... but I guess it doesn't matter really.

<@nirik:matrix.scrye.com>

16:51:12

The only fedmsgs emitted I see are from github2fedmsg

<@nirik:matrix.scrye.com>

16:51:19

fedimg only consumes.

<@nirik:matrix.scrye.com>

16:51:41

anyhow.

<@nirik:matrix.scrye.com>

16:52:04

perhaps we should update all the rhel7 eol tickets remding about the eol date and asking for status?

<@Zlopez:matrix.org>

16:52:22

I can do that

<@nirik:matrix.scrye.com>

16:53:15

on pagure/pagure-dist-git what should we do? ask for thoughts on another 5.x update soon?

<@Zlopez:matrix.org>

16:53:42

Or we can push the 6.0 update

<@lenkaseg:fedora.im>

16:54:13

Yeah, we were planning to make the release with jednorozec soon. 6.0

<@zodbot:fedora.im>

16:54:46

zlopez gave a cookie to lenkaseg. They now have 22 cookies, 3 of which were obtained in the Fedora 40 release cycle

<@nirik:matrix.scrye.com>

16:55:06

well, I am not sure upstream is ready for 6.0.

<@nirik:matrix.scrye.com>

16:55:11

we need to ask... but great. ;)

<@Zlopez:matrix.org>

16:55:36

They are currently trying to fix CI, as there are some issues there

<@lenkaseg:fedora.im>

16:55:41

Let's open the discussion on the ticket, has to be solved soon.

<@nirik:matrix.scrye.com>

16:56:01

I think CI is fixed in 6.0 now, but they are trying to finish some big changes... dropping things no longer used, etc.

<@nirik:matrix.scrye.com>

16:56:30

I can ask tho

<@lenkaseg:fedora.im>

16:57:40

In case there's some discussion around it, link it please to the pdc ticket 11820 please, so I don't miss it!

<@nirik:matrix.scrye.com>

16:57:44

asked in their channel

<@nirik:matrix.scrye.com>

16:57:52

#pagure:fedora.im

<@lenkaseg:fedora.im>

16:57:59

In case there's some discussion around it, link it please to the pdc ticket 11820, so I don't miss it!

<@leo:fedora.im>

16:58:00

let me know if there's anything i can do to help :)

<@lenkaseg:fedora.im>

16:59:47

alright, we're out of time, thank you everybody for attending!

<@lenkaseg:fedora.im>

17:00:09

#endmeeting

<@nirik:matrix.scrye.com>

17:01:28

need a ! there. ;)

<@lenkaseg:fedora.im>

17:01:31

I think RHEL 7 EOL is our priority right now, all kinds of help welcomed from here for example: https://pagure.io/fedora-infrastructure/issue/11815

<@lenkaseg:fedora.im>

17:01:51

!endmeeting