<@zlopez:fedora.im>
16:00:26
!startmeeting Infrastructure (2025-10-16)
<@meetbot:fedora.im>
16:00:28
Meeting started at 2025-10-16 16:00:26 UTC
<@meetbot:fedora.im>
16:00:28
The Meeting name is 'Infrastructure (2025-10-16)'
<@zlopez:fedora.im>
16:00:35
!topic Hola y bienvenido
<@zlopez:fedora.im>
16:00:35
!info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
<@zlopez:fedora.im>
16:00:35
!info About our team: https://docs.fedoraproject.org/en-US/cle/
<@zlopez:fedora.im>
16:00:35
!info Agenda is at: https://board.net/p/fedora-infra
<@zlopez:fedora.im>
16:00:35
!chair @nirik:matrix.scrye.com @zlopez:fedora.im @jnsamyak:matrix.org @james:fedora.im
<@zlopez:fedora.im>
16:00:35
!meetingname infrastructure
<@meetbot:fedora.im>
16:00:36
The Meeting Name is now infrastructure
<@zlopez:fedora.im>
16:00:54
Welcome everyone to today Fedora Infra weekly meeting
<@nirik:matrix.scrye.com>
16:02:02
morning
<@zlopez:fedora.im>
16:03:11
Anybody else today?
<@humaton:fedora.im>
16:03:20
!hi
<@zodbot:fedora.im>
16:03:20
Tomáš Hrčka (humaton) - he / him / his
<@zlopez:fedora.im>
16:05:10
!topic New folks introductions
<@zlopez:fedora.im>
16:05:10
!info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
<@zlopez:fedora.im>
16:05:10
!info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
<@zlopez:fedora.im>
16:05:16
Anybody new around?
<@zlopez:fedora.im>
16:07:54
Doesn't seem so
<@zlopez:fedora.im>
16:08:08
!topic Next chair
<@zlopez:fedora.im>
16:08:08
!info chair 2025-10-30 - ???
<@zlopez:fedora.im>
16:08:08
!info chair 2025-10-16 - zlopez
<@zlopez:fedora.im>
16:08:08
!info magic eight ball says:
<@zlopez:fedora.im>
16:08:08
!info chair 2025-10-23 - ???
<@zlopez:fedora.im>
16:08:15
Anybody for next chair
<@zlopez:fedora.im>
16:08:42
I cleaned the chairs list, so we can use that to nominate next person
<@phsmoura:fedora.im>
16:09:48
I can take next wk
<@zlopez:fedora.im>
16:10:28
It's yours
<@zlopez:fedora.im>
16:10:41
!info chair 2025-10-23 - phsmoura
<@zlopez:fedora.im>
16:11:11
Anybody for the week after
<@zlopez:fedora.im>
16:11:15
?
<@nirik:matrix.scrye.com>
16:11:35
sure, I could...
<@zlopez:fedora.im>
16:13:37
OK
<@zlopez:fedora.im>
16:13:45
!info chair 2025-10-30 - nirik
<@zlopez:fedora.im>
16:14:18
!info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1900 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
<@zlopez:fedora.im>
16:14:18
!info CLE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0815 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
<@zlopez:fedora.im>
16:14:18
!info f43 final freeze started on oct 7th
<@zlopez:fedora.im>
16:14:18
!info Started working on Fedora Infra application list in hackmd https://hackmd.io/@fnF231raRruGKZbPBWKsHQ/B1F1QhRseg [zlopez]
<@zlopez:fedora.im>
16:14:18
!topic announcements and information
<@zlopez:fedora.im>
16:14:25
Anything else to anounce?
<@humaton:fedora.im>
16:15:20
not sure if announcement but I have updated the meeting template.
<@humaton:fedora.im>
16:16:08
Add regular weekly session to refine the backlog. Having a 10-15 tops segment where we go through the oldest tickets.
<@humaton:fedora.im>
16:16:25
Add regular weekly session to refine the backlog. Having a 10-15 min tops segment where we go through the oldest tickets.
<@nirik:matrix.scrye.com>
16:16:30
this meeting? or the daily meeting?
<@humaton:fedora.im>
16:16:36
this one
<@humaton:fedora.im>
16:16:48
weekly go through 1-3 tickets
<@nirik:matrix.scrye.com>
16:17:04
ah, well, we were doing that already, just supposedly only every other meeting
<@nirik:matrix.scrye.com>
16:17:33
but it's also hard when we haven't gotten much attendence. ;( Perhaps this time is no good for more folks.
<@humaton:fedora.im>
16:18:27
yeah, Should we maybe start posting minutes to discussions? To get some attention? Or somethign else?
<@nirik:matrix.scrye.com>
16:18:50
not sure. happy to try other things.
<@zlopez:fedora.im>
16:19:16
I remember that we had more folks joining, not sure what changed
<@nirik:matrix.scrye.com>
16:20:22
well, we adjusted the time recently, but only an hour eariler.
<@nirik:matrix.scrye.com>
16:20:59
anyhow, will ponder on it.
<@zlopez:fedora.im>
16:21:26
There will be DST happening soon, so it will shift again
<@humaton:fedora.im>
16:21:50
DO you see any disadvantage to spend 10 mins weekly on the refinement?
<@zlopez:fedora.im>
16:21:59
No
<@nirik:matrix.scrye.com>
16:22:19
ugh, DST. Don't remind me. ;(
<@zlopez:fedora.im>
16:22:23
It's just usually more than 10 minutes if there is a discussion around some issues
<@nirik:matrix.scrye.com>
16:22:24
sure, we can.
<@nirik:matrix.scrye.com>
16:22:33
yeah, 10min isn't very much
<@humaton:fedora.im>
16:22:51
well that is kind of the point be it 15.
<@nirik:matrix.scrye.com>
16:22:56
but we have already been talking about this for 10min. so. ;)
<@humaton:fedora.im>
16:22:58
its ok to go throug just 1 ticket
<@humaton:fedora.im>
16:23:19
maybe we will just close 3 because we find out they are EOL or something.
<@humaton:fedora.im>
16:23:46
Or spen 15 talking about 1. Its more about the regular cadence, than anything else.
<@zlopez:fedora.im>
16:24:23
Let me continue than
<@humaton:fedora.im>
16:24:33
sorry for the derail
<@zlopez:fedora.im>
16:24:38
!topic Oncall
<@zlopez:fedora.im>
16:24:38
!info on call from 2025-10-10 to 2025-10-16 - nirik
<@zlopez:fedora.im>
16:24:38
!info on call from 2025-10-17 to 2025-10-23 - ???
<@zlopez:fedora.im>
16:24:38
!info https://docs.fedoraproject.org/en-US/infra/day_to_day_fedora/#_the_oncall_role_in_our_team
<@zlopez:fedora.im>
16:24:49
Any news on getting rid of oncall?
<@nirik:matrix.scrye.com>
16:25:06
no, I have had no time to start that discussion. I will try if fires go down
<@nirik:matrix.scrye.com>
16:25:19
also no oncall pings this last week either
<@zlopez:fedora.im>
16:26:09
Let me take the next week than
<@zlopez:fedora.im>
16:26:19
!info on call from 2025-10-17 to 2025-10-23 - zlopez
<@zlopez:fedora.im>
16:26:42
!topic Monitoring discussion [nirik]
<@zlopez:fedora.im>
16:26:42
!info https://nagios.fedoraproject.org/nagios
<@zlopez:fedora.im>
16:26:42
!info Go over existing items and fix them
<@nirik:matrix.scrye.com>
16:27:14
it's pretty much the same as last week...
<@nirik:matrix.scrye.com>
16:27:18
nothing too new.
<@zlopez:fedora.im>
16:27:20
!oncall
<@zodbot:fedora.im>
16:27:23
If they do not respond, please file a ticket (https://pagure.io/fedora-infrastructure/issues)
<@zodbot:fedora.im>
16:27:23
● @Zlopez:matrix.org (zlopez) Current Time for them: 18:27 (Europe/Prague)
<@zodbot:fedora.im>
16:27:23
<@zodbot:fedora.im>
16:27:23
The following people are oncall:
<@zlopez:fedora.im>
16:27:27
OK, set
<@nirik:matrix.scrye.com>
16:27:39
There are 2 openshift certs to renew... I can do that sometime, and/or train someone else to.
<@zlopez:fedora.im>
16:27:44
What about the fedorainfracloud machines
<@zlopez:fedora.im>
16:27:46
?
<@nirik:matrix.scrye.com>
16:28:05
yes, that was there last week too. We can talk about them again...
<@nirik:matrix.scrye.com>
16:28:36
datanommer01.fedorainfracloud.org I still cannot seem to login to (it's in aws). It's unclear to me if it's going to be needed still.
<@nirik:matrix.scrye.com>
16:29:03
logdetective01.fedorainfracloud.org is being worked on... it has some different firewall rules needs as it runs a bunch of containers I think
<@zlopez:fedora.im>
16:29:29
I have a question respins01.fedorainfracloud.org, that is in AWS too?
<@nirik:matrix.scrye.com>
16:29:42
yep.
<@nirik:matrix.scrye.com>
16:30:04
we provide that to respins sig... but don't monitor it, or do anything much with it. They manage it for the most part.
<@zlopez:fedora.im>
16:32:00
OK
<@nirik:matrix.scrye.com>
16:32:12
I think James said it got rebooted yesterday, but I did not do it...
<@zlopez:fedora.im>
16:32:12
Let's go to backlog refinement than
<@zlopez:fedora.im>
16:33:27
!info Refine oldest tickets on Fedora Infra tracker
<@zlopez:fedora.im>
16:33:27
!link https://pagure.io/fedora-infrastructure/report/weekly_refinement
<@zlopez:fedora.im>
16:33:27
!topic Fedora Infra backlog refinement https://pagure.io/fedora-infrastructure/report/weekly_refinement
<@zlopez:fedora.im>
16:34:01
!ticket 7361
<@zodbot:fedora.im>
16:34:02
● **Assignee:** kevin
<@zodbot:fedora.im>
16:34:02
● **Last Updated:** 3 weeks ago
<@zodbot:fedora.im>
16:34:02
● **Opened:** 6 years ago by pbrobinson
<@zodbot:fedora.im>
16:34:02
<@zodbot:fedora.im>
16:34:02
**fedora-infrastructure #7361** (https://pagure.io/fedora-infrastructure/issue/7361):**Signing infrastructure for aarch64**
<@nirik:matrix.scrye.com>
16:34:32
yes, this is in my current sprint, but I have had no time to actually work on it.
<@nirik:matrix.scrye.com>
16:34:39
I hope to someday
<@nirik:matrix.scrye.com>
16:35:36
Well, the part of this I want to work on is getting out secure boot signing moved to the new setup, then we can move forward on adding aarch64 (but there will be more work on it too)
<@smooge:fedora.im>
16:35:54
this is the sprint task that never ends.. it just goes on and on my friend
<@humaton:fedora.im>
16:37:09
hm
<@nirik:matrix.scrye.com>
16:37:30
if we have time at the end I would like to talk about the timeout issue. ;)
<@zlopez:fedora.im>
16:37:41
Should we move to next ticket?
<@humaton:fedora.im>
16:37:47
yup
<@zlopez:fedora.im>
16:37:50
!ticket 10241
<@zodbot:fedora.im>
16:37:51
**fedora-infrastructure #10241** (https://pagure.io/fedora-infrastructure/issue/10241):**Port apps to OIDC**
<@zodbot:fedora.im>
16:37:51
● **Assignee:** zlopez
<@zodbot:fedora.im>
16:37:51
● **Last Updated:** a week ago
<@zodbot:fedora.im>
16:37:51
● **Opened:** 4 years ago by zlopez
<@zodbot:fedora.im>
16:37:51
<@zlopez:fedora.im>
16:37:58
This is going forward
<@humaton:fedora.im>
16:38:31
what are we missing?
<@zlopez:fedora.im>
16:38:40
I have ipsilon instance for OpenID separated in staging, there is issue with certificate to test everything, but I should be able to do the same on production after freeze
<@nirik:matrix.scrye.com>
16:39:15
Michal Konečný: I sent you email on certs, you should be able to get a new one for *.id.stg.fedoraproject.org. Or I can walk you through it sometime...
<@zlopez:fedora.im>
16:39:49
We can then look at Keycloak as replacement and just sunset the OpenID after some time, right now the page shows that the support will end at 1st May 2026, I assume that is enough time for anybody to switch to OIDC
<@zlopez:fedora.im>
16:40:12
I noticed it, maybe we can set some meeting for next week to go through it
<@nirik:matrix.scrye.com>
16:40:15
Aurélien B is setting up keycloak / ipa-tura in staging
<@nirik:matrix.scrye.com>
16:40:29
sure, fine with me
<@zlopez:fedora.im>
16:41:44
OK, let's go to next one
<@nirik:matrix.scrye.com>
16:41:47
then we can hopefully roll to prod after f43 is out the door
<@zlopez:fedora.im>
16:42:15
!ticket 10458
<@zodbot:fedora.im>
16:42:16
<@zodbot:fedora.im>
16:42:16
**fedora-infrastructure #10458** (https://pagure.io/fedora-infrastructure/issue/10458):**Help me move my discourse bots to production?**
<@zodbot:fedora.im>
16:42:16
● **Assignee:** ryanlerch
<@zodbot:fedora.im>
16:42:16
● **Last Updated:** 3 months ago
<@zodbot:fedora.im>
16:42:16
● **Opened:** 3 years ago by mattdm
<@zlopez:fedora.im>
16:42:31
I think this is still waiting for @mattdm
<@nirik:matrix.scrye.com>
16:42:39
yeah, I was gonna ask mattdm about this... but keep forgetting.
<@nirik:matrix.scrye.com>
16:42:55
I wonder if we shouldn't just close it and say 'open when you have time/are ready to move forward'?
<@humaton:fedora.im>
16:43:16
I was just going to type ^^
<@nirik:matrix.scrye.com>
16:43:44
I think thats fine... since it's been sitting there for so long
<@zlopez:fedora.im>
16:44:09
I'm +1 for closing it as there is no response
<@humaton:fedora.im>
16:44:25
Yeah we might even get more strict. 2-3 pings in 3 months and closing
<@humaton:fedora.im>
16:44:57
It does not have to sit there if the requester is not interested or available.
<@nirik:matrix.scrye.com>
16:45:19
sure, I don't think we need a hard rule, but this one is clearly just not happening.
<@nirik:matrix.scrye.com>
16:45:28
close and move on?
<@humaton:fedora.im>
16:45:33
yup
<@humaton:fedora.im>
16:45:45
and I think we can close the refinement for today
<@humaton:fedora.im>
16:46:02
we did 3 tickets in 10mins have some updates and 1 down.
<@zlopez:fedora.im>
16:46:14
I switch to open floor, do you want me to close the ticket?
<@nirik:matrix.scrye.com>
16:46:21
sure, or I can if you prefer?
<@zlopez:fedora.im>
16:46:31
I will do it
<@nirik:matrix.scrye.com>
16:46:34
I have actually 2 items for open floor/general discussion
<@zlopez:fedora.im>
16:46:46
jednorozec was quicker 😄
<@zlopez:fedora.im>
16:46:47
!topic Open Floor
<@nirik:matrix.scrye.com>
16:47:50
first. I'd like us to move to forge after f43 is out the door. We should do any planning we need beforehand to be ready for it. ie, what goes where, how we redirect people, docs updates, etc. We have a ticket on this, but just wanted to mention it...
<@nirik:matrix.scrye.com>
16:48:28
I'll try and check the ticket / update it if things every quiet down.
<@zlopez:fedora.im>
16:48:29
That sounds like a good way to do it
<@nirik:matrix.scrye.com>
16:48:58
we need to be good about telling people about the new place since we get so many people filing infra tickets.
<@nirik:matrix.scrye.com>
16:49:23
and since so many people work on infra ansible
<@humaton:fedora.im>
16:50:08
Visibility is the key in this case
<@humaton:fedora.im>
16:50:20
should we start talking about it on discussions?
<@nirik:matrix.scrye.com>
16:50:39
https://pagure.io/fedora-infrastructure/issue/12737 I guess is the one I was thinking of?
<@humaton:fedora.im>
16:50:40
the audience there seems to be bigger
<@nirik:matrix.scrye.com>
16:50:49
yeah. Sounds good.
<@humaton:fedora.im>
16:51:08
Yup that one and releng have their own
<@nirik:matrix.scrye.com>
16:51:29
we have a number of forge requests tickets... I don't know if ryan is handling those or what...
<@humaton:fedora.im>
16:51:58
I am
<@james:fedora.im>
16:52:05
Yeh, I don't think it was running at one point. It then booted, but the person in the ticket asked for a reboot ... so I did that this morning.
<@zlopez:fedora.im>
16:52:06
How it will be with the mirroring to batcave?
<@humaton:fedora.im>
16:52:11
created bunch of the groups and ryan did some as well
<@nirik:matrix.scrye.com>
16:52:14
I suppose we could do a seperate ticket for infra and use that for our stuff since that ticket above is more general?
<@humaton:fedora.im>
16:52:44
yes, that one was more intended to discuss the new structure but there are some links and lists.
<@james:fedora.im>
16:52:45
Sorry for the delay ... also in the FPC meeting, and that's going great today ;)
<@nirik:matrix.scrye.com>
16:53:10
https://pagure.io/fedora-infrastructure/issue/12806 and https://pagure.io/fedora-infrastructure/issue/12793 and https://pagure.io/fedora-infrastructure/issue/12793 and https://pagure.io/fedora-infrastructure/issue/12409 and https://pagure.io/fedora-infrastructure/issue/12755
<@nirik:matrix.scrye.com>
16:53:23
(perhaps we should add a forge tag. ;)
<@nirik:matrix.scrye.com>
16:53:52
good question. we will need to figure that out. If it emits fedora-messaging messages we could use the same listener perhaps with minor adjustments.
<@nirik:matrix.scrye.com>
16:54:34
also there's https://pagure.io/fedora-infrastructure/issue/12558 about commops
<@zlopez:fedora.im>
16:55:23
I'm not sure how it is handled right now, I remember there were some issues during the datacenter move
<@nirik:matrix.scrye.com>
16:56:14
ok, we only have a few min left, I wanted to mention the gigantic pain timeout issue ( https://pagure.io/fedora-infrastructure/issue/12814 ). Folks are welcome to look and see if they can find it... I mentioned some more invasive things I could try (reinstalling proxies or virthosts ). We are of course in freeze, but this is kinda a outage too...
<@nirik:matrix.scrye.com>
16:57:24
I think Akashdeep Dhar was working on webhook fedora messaging for forge. I am not sure the status tho.
<@zlopez:fedora.im>
16:57:31
We tried to investigate it a little with nils, but didn't noticed the ticket that is already open
<@nirik:matrix.scrye.com>
16:58:11
I could have missed something for sure, but I have tried everything I could think of over the last few weeks.
<@humaton:fedora.im>
16:58:20
we can emit messages on any event from the new forge.
<@nirik:matrix.scrye.com>
16:58:58
we just need a 'ansible git repo updated' message to trigger a git mirror on. Should be hopefully easy to adjust.
<@zlopez:fedora.im>
16:59:01
I don't think we tried something you didn't
<@zlopez:fedora.im>
17:00:25
We can definitely try that before moving completely
<@zlopez:fedora.im>
17:00:35
We are at end of our time now
<@zlopez:fedora.im>
17:00:41
Let me end the meeting
<@zlopez:fedora.im>
17:00:50
!endmeeting