2024-05-16 16:00:25 <@lenkaseg:fedora.im> !startmeeting Infrastructure (2024-05-16)
2024-05-16 16:00:27 <@meetbot:fedora.im> Meeting started at 2024-05-16 16:00:25 UTC
2024-05-16 16:00:27 <@meetbot:fedora.im> The Meeting name is 'Infrastructure (2024-05-16)'
2024-05-16 16:00:39 <@lenkaseg:fedora.im> !meetingname infrastructure
2024-05-16 16:00:45 <@lenkaseg:fedora.im> !chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp
2024-05-16 16:00:52 <@lenkaseg:fedora.im> !info Agenda is at: https://board.net/p/fedora-infra
2024-05-16 16:00:52 <@leo:fedora.im> hey!
2024-05-16 16:00:58 <@lenkaseg:fedora.im> About our team: https://docs.fedoraproject.org/en-US/cpe/
2024-05-16 16:01:03 <@lenkaseg:fedora.im> Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
2024-05-16 16:01:09 <@lenkaseg:fedora.im> !topic namaste
2024-05-16 16:01:19 <@lenkaseg:fedora.im> Morning/evening/afternoon everybody!
2024-05-16 16:01:25 <@aggraxis:fedora.im> Hello!
2024-05-16 16:02:47 <@Zlopez:matrix.org> !hi
2024-05-16 16:02:49 <@zodbot:fedora.im> Michal Konecny (zlopez)
2024-05-16 16:03:46 <@lenkaseg:fedora.im> !topic New folks introductions
2024-05-16 16:03:55 <@lenkaseg:fedora.im> !info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
2024-05-16 16:04:06 <@lenkaseg:fedora.im> Are there any new folks today?
2024-05-16 16:04:36 <@jnsamyak:matrix.org> !hi
2024-05-16 16:04:38 <@zodbot:fedora.im> Samyak Jain (jnsamyak) - he / him / his
2024-05-16 16:06:12 <@lenkaseg:fedora.im> Nobody new this time but nevermind!
2024-05-16 16:06:19 <@lenkaseg:fedora.im> !topic Next chair
2024-05-16 16:06:25 <@lenkaseg:fedora.im> !info magic eight ball says:
2024-05-16 16:06:33 <@lenkaseg:fedora.im> !info chair 2024-05-16 lenkaseg
2024-05-16 16:07:02 <@lenkaseg:fedora.im> That is me and we have no more chairs. Any volunteer for May23rd?
2024-05-16 16:08:12 <@lenkaseg:fedora.im> (Ideal for new folks as well!)
2024-05-16 16:08:21 <@Zlopez:matrix.org> I can take it
2024-05-16 16:08:31 <@lenkaseg:fedora.im> sold to zlopez!
2024-05-16 16:08:41 <@lenkaseg:fedora.im> !info chair 2024-05-23  zlopez
2024-05-16 16:08:51 <@lenkaseg:fedora.im> Anybody for May30th?
2024-05-16 16:09:13 <@lenkaseg:fedora.im> Unfortunately I will not be around that time
2024-05-16 16:10:41 <@lenkaseg:fedora.im> No worries, can be decided next time
2024-05-16 16:10:48 <@lenkaseg:fedora.im> !topic announcements and information
2024-05-16 16:10:58 <@lenkaseg:fedora.im> !info CPE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2024-05-16 16:11:01 <@nirik:matrix.scrye.com> morning
2024-05-16 16:11:06 <@lenkaseg:fedora.im> !info CPE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in #fedora-meeting-3
2024-05-16 16:11:48 <@nirik:matrix.scrye.com> !info tomorrow is a Red Hat day of learning and next friday is a Red Hat Recharge day.
2024-05-16 16:12:28 <@Zlopez:matrix.org> !info ARC dist-git investigation started https://communityblog.fedoraproject.org/2024-git-forge-evaluation/
2024-05-16 16:13:36 <@Zlopez:matrix.org> !info There was problem with release-monitoring not filling bugs for packagers that is solved now
2024-05-16 16:15:10 <@lenkaseg:fedora.im> if that's all from announcements, let's move on to...
2024-05-16 16:15:34 <@Zlopez:matrix.org> !info F38 is EOL next week
2024-05-16 16:16:05 <@lenkaseg:fedora.im> !topic Oncall
2024-05-16 16:16:26 <@nirik:matrix.scrye.com> wiki and 2 osbuild vm's are all thats left on f38 that I know of for vm's.
2024-05-16 16:16:56 <@lenkaseg:fedora.im> Ah yes, for me time when I usually move to newly releases fedora
2024-05-16 16:17:06 <@lenkaseg:fedora.im> Ah yes, for me time when I usually move to newly released fedora
2024-05-16 16:17:34 <@lenkaseg:fedora.im> !info https://fedoraproject.org/wiki/Infrastructure/Oncall
https://docs.fedoraproject.org/en-US/cpe/day_to_day_fedora/
2024-05-16 16:17:43 <@lenkaseg:fedora.im> !info aheath1992 is on call from 2024-05-09 to 2024-05-16
2024-05-16 16:17:53 <@lenkaseg:fedora.im>  ??? is on call from 2024-05-16 to 2024-05-23
2024-05-16 16:18:06 <@leo:fedora.im> i'll take it
2024-05-16 16:18:08 <@lenkaseg:fedora.im> Any volunteer for oncall starting today?
2024-05-16 16:18:17 <@lenkaseg:fedora.im> thank you!
2024-05-16 16:18:32 <@lenkaseg:fedora.im> !info leo is on call from 2024-05-16 to 2024-05-23
2024-05-16 16:18:55 <@lenkaseg:fedora.im> Anybody willing to take following week?
2024-05-16 16:19:02 <@nirik:matrix.scrye.com> I can take it
2024-05-16 16:19:25 <@lenkaseg:fedora.im> Excellent, thanks!
2024-05-16 16:19:28 <@lenkaseg:fedora.im> !info nirik is on call from 2024-05-23 to 2024-05-30
2024-05-16 16:19:35 <@lenkaseg:fedora.im> 
!info Summary of last week: (from current oncall)
2024-05-16 16:19:56 <@lenkaseg:fedora.im> I believe the last oncall was aheath1992
2024-05-16 16:20:13 <@lenkaseg:fedora.im> Over to you aheath1992 
2024-05-16 16:21:48 <@aheath1992:matrix.org> No on call messages that i was aware of 
2024-05-16 16:21:51 <@aheath1992:matrix.org> all quiet
2024-05-16 16:22:05 <@lenkaseg:fedora.im> Thanks for the report!
2024-05-16 16:22:16 <@lenkaseg:fedora.im> Let's move on to
2024-05-16 16:22:17 <@lenkaseg:fedora.im> !topic Monitoring discussion [nirik]
2024-05-16 16:22:28 <@lenkaseg:fedora.im> Over to nirik 
2024-05-16 16:22:59 <@nirik:matrix.scrye.com> so... aside from a proxy14 blep just now... we only have 2 alerts left.
2024-05-16 16:23:13 <@nirik:matrix.scrye.com> "Check datanommer for recent fedbadges messages"
2024-05-16 16:23:33 <@nirik:matrix.scrye.com> I am not sure if this is checking the old badges? or is looking for something the new one isn't emitting? we need to investigate
2024-05-16 16:24:21 <@nirik:matrix.scrye.com> and "Check MySQL Backup" this is on the new db03... selinux is denying nagios/nrpe perm to look at a link in /backups/ ;( So, needs a selinux policy adjustment or the like.
2024-05-16 16:24:35 <@nirik:matrix.scrye.com> thats it. Everything else is green... no more disk or other issues. ;)
2024-05-16 16:24:56 <@lenkaseg:fedora.im> Thanks nirik !
2024-05-16 16:25:32 <@Zlopez:matrix.org> I have something as well
2024-05-16 16:25:37 <@lenkaseg:fedora.im> Alright, next topic could be:

2024-05-16 16:25:46 <@lenkaseg:fedora.im> go on Zlopez 
2024-05-16 16:26:24 <@Zlopez:matrix.org> I noticed that there are plenty of errors reported by cron on staging proxy01 for `/usr/local/bin/restart-mirrorlist-processes`
2024-05-16 16:26:34 <@Zlopez:matrix.org> ```
Error: mirrorlist1 is not running as expected. Trying to restart
ERROR: mirrorlist2 not processing correctly
```
2024-05-16 16:26:51 <@nirik:matrix.scrye.com> yes, phsmoura was reinstalling that and looking into it
2024-05-16 16:27:10 <@Zlopez:matrix.org> OK
2024-05-16 16:27:58 <@lenkaseg:fedora.im> So let's decide what we want to look into:
2024-05-16 16:28:06 <@lenkaseg:fedora.im> RHEL7 EOL
2024-05-16 16:28:14 <@lenkaseg:fedora.im> Revisit blocked tickets
2024-05-16 16:28:20 <@lenkaseg:fedora.im> backlog refinement
2024-05-16 16:28:36 <@lenkaseg:fedora.im> something else
2024-05-16 16:28:43 <@lenkaseg:fedora.im> Please vote!
2024-05-16 16:29:53 <@lenkaseg:fedora.im> Seems that RHEL7 EOL it is (and will be till June30th) 😀
2024-05-16 16:30:00 <@lenkaseg:fedora.im> !topic RHEL7 EOL
2024-05-16 16:30:05 <@lenkaseg:fedora.im> metaticket here:
2024-05-16 16:30:14 <@lenkaseg:fedora.im> !ticket https://pagure.io/fedora-infrastructure/issue/11815
2024-05-16 16:30:39 <@Zlopez:matrix.org> The ticket number should be enough
2024-05-16 16:30:45 <@Zlopez:matrix.org> !ticket 11815
2024-05-16 16:30:47 <@zodbot:fedora.im> **fedora-infrastructure #11815** (https://pagure.io/fedora-infrastructure/issue/11815):**rhel7 eol**

● **Opened:** 2 months ago by zlopez
● **Last Updated:** 2 months ago
● **Assignee:** zlopez
2024-05-16 16:31:08 <@lenkaseg:fedora.im> ok...where did we stopped last time..
2024-05-16 16:31:20 <@lenkaseg:fedora.im> ok...where did we stop last time..
2024-05-16 16:31:31 <@Zlopez:matrix.org> We now have db-fas01 migrated thanks to nirik 
2024-05-16 16:31:55 <@Zlopez:matrix.org> The Badges were migrated as well
2024-05-16 16:32:24 <@nirik:matrix.scrye.com> I did a few more with the outages...
2024-05-16 16:33:27 <@lenkaseg:fedora.im> How is it with Fedora People?
2024-05-16 16:33:35 <@lenkaseg:fedora.im> !ticket 11822
2024-05-16 16:33:37 <@zodbot:fedora.im> **fedora-infrastructure #11822** (https://pagure.io/fedora-infrastructure/issue/11822):**rhel7 - Fedora people**

● **Opened:** 2 months ago by zlopez
● **Last Updated:** a week ago
● **Assignee:** lenkaseg
2024-05-16 16:33:56 <@nirik:matrix.scrye.com> waiting for planet still.
2024-05-16 16:34:11 <@nirik:matrix.scrye.com> phsmoura is sending out a devel-announce about that today hopefully.
2024-05-16 16:34:21 <@nirik:matrix.scrye.com> retiring it on the 29th
2024-05-16 16:34:21 <@Zlopez:matrix.org> There is a onset date for planet 21st May
2024-05-16 16:34:34 <@nirik:matrix.scrye.com> yes, but that was too soon, so I asked to push it out a week
2024-05-16 16:34:40 <@Zlopez:matrix.org> Ok
2024-05-16 16:34:50 <@nirik:matrix.scrye.com> less than a week is not enough time for some folks
2024-05-16 16:35:02 <@Zlopez:matrix.org> I'm currently working on mailman3
2024-05-16 16:35:11 <@Zlopez:matrix.org> !ticket 8455
2024-05-16 16:35:13 <@zodbot:fedora.im> **fedora-infrastructure #8455** (https://pagure.io/fedora-infrastructure/issue/8455):**Move mailman to newer release of Fedora or CentOS**

● **Opened:** 4 years ago by smooge
● **Last Updated:** 3 days ago
● **Assignee:** zlopez
2024-05-16 16:35:25 <@Zlopez:matrix.org> And sundries server
2024-05-16 16:35:39 <@Zlopez:matrix.org> !ticket 11821
2024-05-16 16:35:41 <@lenkaseg:fedora.im> me as well, working on mailman-sar,py
2024-05-16 16:35:41 <@zodbot:fedora.im> **fedora-infrastructure #11821** (https://pagure.io/fedora-infrastructure/issue/11821):**rhel7 - sundries servers**

● **Opened:** 2 months ago by zlopez
● **Last Updated:** a week ago
● **Assignee:** zlopez
2024-05-16 16:35:50 <@lenkaseg:fedora.im> me as well, working on mailman-sar.py
2024-05-16 16:35:53 <@nirik:matrix.scrye.com> how are we looking on pdc these days?
2024-05-16 16:36:34 <@lenkaseg:fedora.im> So, owner-sync-pagure is tested on bodhi-backend staging, seems to be working
2024-05-16 16:36:46 <@lenkaseg:fedora.im> New PR opened here:
2024-05-16 16:37:09 <@lenkaseg:fedora.im> https://pagure.io/fedora-infra/ansible/pull-request/2022
2024-05-16 16:37:13 <@nirik:matrix.scrye.com> nice. we can try and push that out and see. Thanks for adjusting it.
2024-05-16 16:37:24 <@Zlopez:matrix.org> There are still few other scripts using PDC from what I remember
2024-05-16 16:37:43 <@lenkaseg:fedora.im> And jednorozec is working on fedfind
2024-05-16 16:38:34 <@lenkaseg:fedora.im> in ansible repo there should not be, there were 4: 3 removed as no longer necessary and 1 fixed (that was the owner-sync-pagure)
2024-05-16 16:38:47 <@lenkaseg:fedora.im> In toddlers there are still some pdc calls
2024-05-16 16:38:55 <@Zlopez:matrix.org> I saw something in releng repo and toddlers
2024-05-16 16:38:56 <@lenkaseg:fedora.im> I think Anton was working on it
2024-05-16 16:39:45 <@nirik:matrix.scrye.com> what abount the src.fp.org commit hook? did we aready change that?
2024-05-16 16:40:10 <@lenkaseg:fedora.im> in releng repo there should be nothing more to fix if I recall correctly from taliing with jednorozec
2024-05-16 16:40:15 <@lenkaseg:fedora.im> only cleanup possibly
2024-05-16 16:40:55 <@Zlopez:matrix.org> It would be nice to update the PDC ticket with what is remaining
2024-05-16 16:40:58 <@lenkaseg:fedora.im> I think so, let me find the PR, but yeah, pagure needs to be released. We were waiting for some last PRs to land
2024-05-16 16:41:07 <@Zlopez:matrix.org> !ticket 11820
2024-05-16 16:41:09 <@zodbot:fedora.im> **fedora-infrastructure #11820** (https://pagure.io/fedora-infrastructure/issue/11820):**rhel7 - PDC**

● **Opened:** 2 months ago by zlopez
● **Last Updated:** a month ago
● **Assignee:** humaton
2024-05-16 16:41:14 <@lenkaseg:fedora.im> Good idea, I can do that
2024-05-16 16:41:38 <@nirik:matrix.scrye.com> ah, waiting for a pagure release? I thought it was just a hook edit
2024-05-16 16:42:00 <@lenkaseg:fedora.im> Here it is: https://pagure.io/pagure-dist-git/pull-request/164
2024-05-16 16:42:13 <@nirik:matrix.scrye.com> ah, a pagure dist git release
2024-05-16 16:42:26 <@lenkaseg:fedora.im> Yes, sorry
2024-05-16 16:42:49 <@Zlopez:matrix.org> But I see that this is waiting for pagure changes as well :-/
2024-05-16 16:43:26 <@nirik:matrix.scrye.com> I'm not sure the status on a new pagure 5.x release... they have been working on 6.0 mostly.
2024-05-16 16:43:42 <@lenkaseg:fedora.im> in releng repo there should be nothing more to fix if I recall correctly from talking with jednorozec
2024-05-16 16:44:05 <@lenkaseg:fedora.im> Correct, both should be released
2024-05-16 16:44:09 <@Zlopez:matrix.org> I have few changes that are waiting for pagure 6.0 as well
2024-05-16 16:44:36 <@nirik:matrix.scrye.com> 6.0 will be a big release... we will need to move to rhel9
2024-05-16 16:44:59 <@nirik:matrix.scrye.com> so fun that all this is so tangled up. ;)
2024-05-16 16:45:07 <@lenkaseg:fedora.im> after 2 years or so
2024-05-16 16:45:48 <@Zlopez:matrix.org> At least we are getting slowly off fedmsg
2024-05-16 16:45:50 <@nirik:matrix.scrye.com> we could backport the pagure patches to epel8 for us...
2024-05-16 16:45:53 <@aggraxis:fedora.im> Yeah, I just started reading the history on this. Tangled up inded.
2024-05-16 16:46:04 <@aggraxis:fedora.im> Yeah, I just started reading the history on this. Tangled up indeed.
2024-05-16 16:46:26 <@Zlopez:matrix.org> I'm not sure how long it will take for pagure 6.0 to be released
2024-05-16 16:46:28 <@nirik:matrix.scrye.com> oh, and that reminds me. Do we intend to retire fedmsg at rhel7 eol? If not, we need to move busgateway* vm's to rhel9.
2024-05-16 16:46:46 <@Zlopez:matrix.org> Let me check
2024-05-16 16:46:50 <@nirik:matrix.scrye.com> I don't know if we have anything still using it.
2024-05-16 16:46:55 <@Zlopez:matrix.org> !ticket 8213
2024-05-16 16:46:58 <@zodbot:fedora.im> **fedora-infrastructure #8213** (https://pagure.io/fedora-infrastructure/issue/8213):**fedmsg -> fedora-messaging migration tracker**

● **Opened:** 4 years ago by kevin
● **Last Updated:** 13 minutes ago
● **Assignee:** zlopez
2024-05-16 16:47:51 <@Zlopez:matrix.org> This is what remains:
* busgateway.yml: will die when fedmsg dies
* fedimg.yml: to be retired I think?
* github2fedmsg.yml: waiting for rewrite (webhook2fedmsg)
* mailman.yml: waiting for server upgrade to EL9
* mbs.yml: to be retired
* mirrormanager.yml: rewrite done, to be deployed
* pdc.yml: Retirement in progress
2024-05-16 16:48:07 <@Zlopez:matrix.org> We have somebody working on fedimg replacement
2024-05-16 16:48:24 <@Zlopez:matrix.org> We have outreachy intern working on github2fedmsg replacement
2024-05-16 16:48:32 <@nirik:matrix.scrye.com> yep.
2024-05-16 16:48:34 <@Zlopez:matrix.org> I'm working on mailman
2024-05-16 16:48:40 <@nirik:matrix.scrye.com> any status on github2fedmsg?
2024-05-16 16:49:15 <@Zlopez:matrix.org> We had first meeting with outreachy intern, he should have enough time to work on webhook2fedmsg replacement
2024-05-16 16:49:43 <@nirik:matrix.scrye.com> Is mailman actually using fedmsg in prod... I am not seeing anything in the zmq-amqp gateway for it.
2024-05-16 16:50:03 <@Zlopez:matrix.org> I'm interested about mirrormanager, how it is looking with it?
2024-05-16 16:50:16 <@nirik:matrix.scrye.com> I'm not sure, we need to ping Aurélien B.
2024-05-16 16:50:42 <@Zlopez:matrix.org> nirik: I'm not sure the fedmsg archiver plugin was already ported to fedora messaging, so it depends what version the mailman in prod is using
2024-05-16 16:51:02 <@nirik:matrix.scrye.com> yeah, I think it's not using fedsmg... but I guess it doesn't matter really.
2024-05-16 16:51:12 <@nirik:matrix.scrye.com> The only fedmsgs emitted I see are from github2fedmsg
2024-05-16 16:51:19 <@nirik:matrix.scrye.com> fedimg only consumes.
2024-05-16 16:51:41 <@nirik:matrix.scrye.com> anyhow.
2024-05-16 16:52:04 <@nirik:matrix.scrye.com> perhaps we should update all the rhel7 eol tickets remding about the eol date and asking for status?
2024-05-16 16:52:22 <@Zlopez:matrix.org> I can do that
2024-05-16 16:53:15 <@nirik:matrix.scrye.com> on pagure/pagure-dist-git what should we do? ask for thoughts on another 5.x update soon?
2024-05-16 16:53:42 <@Zlopez:matrix.org> Or we can push the 6.0 update
2024-05-16 16:54:13 <@lenkaseg:fedora.im> Yeah, we were planning to make the release with jednorozec soon. 6.0
2024-05-16 16:54:46 <@zodbot:fedora.im> zlopez gave a cookie to lenkaseg. They now have 22 cookies, 3 of which were obtained in the Fedora 40 release cycle
2024-05-16 16:55:06 <@nirik:matrix.scrye.com> well, I am not sure upstream is ready for 6.0.
2024-05-16 16:55:11 <@nirik:matrix.scrye.com> we need to ask... but great. ;)
2024-05-16 16:55:36 <@Zlopez:matrix.org> They are currently trying to fix CI, as there are some issues there
2024-05-16 16:55:41 <@lenkaseg:fedora.im> Let's open the discussion on the ticket, has to be solved soon.
2024-05-16 16:56:01 <@nirik:matrix.scrye.com> I think CI is fixed in 6.0 now, but they are trying to finish some big changes... dropping things no longer used, etc.
2024-05-16 16:56:30 <@nirik:matrix.scrye.com> I can ask tho
2024-05-16 16:57:40 <@lenkaseg:fedora.im> In case there's some discussion around it, link it please to the pdc ticket 11820 please, so I don't miss it!
2024-05-16 16:57:44 <@nirik:matrix.scrye.com> asked in their channel
2024-05-16 16:57:52 <@nirik:matrix.scrye.com> #pagure:fedora.im
2024-05-16 16:57:59 <@lenkaseg:fedora.im> In case there's some discussion around it, link it please to the pdc ticket 11820, so I don't miss it!
2024-05-16 16:58:00 <@leo:fedora.im> let me know if there's anything i can do to help :)
2024-05-16 16:59:47 <@lenkaseg:fedora.im> alright, we're out of time, thank you everybody for attending!
2024-05-16 17:00:09 <@lenkaseg:fedora.im> #endmeeting
2024-05-16 17:01:28 <@nirik:matrix.scrye.com> need a ! there. ;)
2024-05-16 17:01:31 <@lenkaseg:fedora.im> I think RHEL 7 EOL is our priority right now, all kinds of help welcomed from here for example: https://pagure.io/fedora-infrastructure/issue/11815
2024-05-16 17:01:51 <@lenkaseg:fedora.im> !endmeeting