2025-10-16 16:00:26 <@zlopez:fedora.im> !startmeeting Infrastructure (2025-10-16)
2025-10-16 16:00:28 <@meetbot:fedora.im> Meeting started at 2025-10-16 16:00:26 UTC
2025-10-16 16:00:28 <@meetbot:fedora.im> The Meeting name is 'Infrastructure (2025-10-16)'
2025-10-16 16:00:35 <@zlopez:fedora.im> !topic Hola y bienvenido
2025-10-16 16:00:35 <@zlopez:fedora.im> !info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
2025-10-16 16:00:35 <@zlopez:fedora.im> !info About our team: https://docs.fedoraproject.org/en-US/cle/
2025-10-16 16:00:35 <@zlopez:fedora.im> !info Agenda is at: https://board.net/p/fedora-infra
2025-10-16 16:00:35 <@zlopez:fedora.im> !chair @nirik:matrix.scrye.com @zlopez:fedora.im  @jnsamyak:matrix.org  @james:fedora.im
2025-10-16 16:00:35 <@zlopez:fedora.im> !meetingname infrastructure
2025-10-16 16:00:36 <@meetbot:fedora.im> The Meeting Name is now infrastructure
2025-10-16 16:00:54 <@zlopez:fedora.im> Welcome everyone to today Fedora Infra weekly meeting
2025-10-16 16:02:02 <@nirik:matrix.scrye.com> morning
2025-10-16 16:03:11 <@zlopez:fedora.im> Anybody else today?
2025-10-16 16:03:20 <@humaton:fedora.im> !hi
2025-10-16 16:03:20 <@zodbot:fedora.im> Tomáš Hrčka (humaton) - he / him / his
2025-10-16 16:05:10 <@zlopez:fedora.im> !topic New folks introductions
2025-10-16 16:05:10 <@zlopez:fedora.im> !info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
2025-10-16 16:05:10 <@zlopez:fedora.im> !info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
2025-10-16 16:05:16 <@zlopez:fedora.im> Anybody new around?
2025-10-16 16:07:54 <@zlopez:fedora.im> Doesn't seem so
2025-10-16 16:08:08 <@zlopez:fedora.im> !topic Next chair
2025-10-16 16:08:08 <@zlopez:fedora.im> !info chair 2025-10-30 - ???
2025-10-16 16:08:08 <@zlopez:fedora.im> !info chair 2025-10-16 - zlopez
2025-10-16 16:08:08 <@zlopez:fedora.im> !info magic eight ball says:
2025-10-16 16:08:08 <@zlopez:fedora.im> !info chair 2025-10-23 - ???
2025-10-16 16:08:15 <@zlopez:fedora.im> Anybody for next chair
2025-10-16 16:08:42 <@zlopez:fedora.im> I cleaned the chairs list, so we can use that to nominate next person
2025-10-16 16:09:48 <@phsmoura:fedora.im> I can take next wk
2025-10-16 16:10:28 <@zlopez:fedora.im> It's yours
2025-10-16 16:10:41 <@zlopez:fedora.im> !info chair 2025-10-23 - phsmoura
2025-10-16 16:11:11 <@zlopez:fedora.im> Anybody for the week after
2025-10-16 16:11:15 <@zlopez:fedora.im> ?
2025-10-16 16:11:35 <@nirik:matrix.scrye.com> sure, I could...
2025-10-16 16:13:37 <@zlopez:fedora.im> OK
2025-10-16 16:13:45 <@zlopez:fedora.im> !info chair 2025-10-30 - nirik
2025-10-16 16:14:18 <@zlopez:fedora.im> !info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1900 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-10-16 16:14:18 <@zlopez:fedora.im> !info CLE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0815 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-10-16 16:14:18 <@zlopez:fedora.im> !info f43 final freeze started on oct 7th
2025-10-16 16:14:18 <@zlopez:fedora.im> !info Started working on Fedora Infra application list in hackmd https://hackmd.io/@fnF231raRruGKZbPBWKsHQ/B1F1QhRseg [zlopez]
2025-10-16 16:14:18 <@zlopez:fedora.im> !topic announcements and information
2025-10-16 16:14:25 <@zlopez:fedora.im> Anything else to anounce?
2025-10-16 16:15:20 <@humaton:fedora.im> not sure if announcement but I have updated the meeting template.
2025-10-16 16:16:08 <@humaton:fedora.im> Add regular weekly session to refine the backlog. Having a 10-15 tops segment where we go through the oldest tickets.
2025-10-16 16:16:25 <@humaton:fedora.im> Add regular weekly session to refine the backlog. Having a 10-15 min tops segment where we go through the oldest tickets.
2025-10-16 16:16:30 <@nirik:matrix.scrye.com> this meeting? or the daily meeting?
2025-10-16 16:16:36 <@humaton:fedora.im> this one
2025-10-16 16:16:48 <@humaton:fedora.im> weekly go through 1-3 tickets
2025-10-16 16:17:04 <@nirik:matrix.scrye.com> ah, well, we were doing that already, just supposedly only every other meeting
2025-10-16 16:17:33 <@nirik:matrix.scrye.com> but it's also hard when we haven't gotten much attendence. ;( Perhaps this time is no good for more folks.
2025-10-16 16:18:27 <@humaton:fedora.im> yeah, Should we maybe start posting minutes to discussions? To get some attention? Or somethign else?
2025-10-16 16:18:50 <@nirik:matrix.scrye.com> not sure. happy to try other things.
2025-10-16 16:19:16 <@zlopez:fedora.im> I remember that we had more folks joining, not sure what changed
2025-10-16 16:20:22 <@nirik:matrix.scrye.com> well, we adjusted the time recently, but only an hour eariler.
2025-10-16 16:20:59 <@nirik:matrix.scrye.com> anyhow, will ponder on it.
2025-10-16 16:21:26 <@zlopez:fedora.im> There will be DST happening soon, so it will shift again
2025-10-16 16:21:50 <@humaton:fedora.im> DO you see any disadvantage to spend 10 mins weekly on the refinement?
2025-10-16 16:21:59 <@zlopez:fedora.im> No
2025-10-16 16:22:19 <@nirik:matrix.scrye.com> ugh, DST. Don't remind me. ;(
2025-10-16 16:22:23 <@zlopez:fedora.im> It's just usually more than 10 minutes if there is a discussion around some issues
2025-10-16 16:22:24 <@nirik:matrix.scrye.com> sure, we can.
2025-10-16 16:22:33 <@nirik:matrix.scrye.com> yeah, 10min isn't very much
2025-10-16 16:22:51 <@humaton:fedora.im> well that is kind of the point be it 15. 
2025-10-16 16:22:56 <@nirik:matrix.scrye.com> but we have already been talking about this for 10min. so. ;)
2025-10-16 16:22:58 <@humaton:fedora.im> its ok to go throug just 1 ticket
2025-10-16 16:23:19 <@humaton:fedora.im> maybe we will just close 3 because we find out they are EOL or something. 
2025-10-16 16:23:46 <@humaton:fedora.im> Or spen 15 talking about  1. Its more about the regular cadence, than anything else.
2025-10-16 16:24:23 <@zlopez:fedora.im> Let me continue than
2025-10-16 16:24:33 <@humaton:fedora.im> sorry for the derail
2025-10-16 16:24:38 <@zlopez:fedora.im> !topic Oncall
2025-10-16 16:24:38 <@zlopez:fedora.im> !info on call from 2025-10-10 to 2025-10-16 - nirik
2025-10-16 16:24:38 <@zlopez:fedora.im> !info on call from 2025-10-17 to 2025-10-23 - ???
2025-10-16 16:24:38 <@zlopez:fedora.im> !info https://docs.fedoraproject.org/en-US/infra/day_to_day_fedora/#_the_oncall_role_in_our_team
2025-10-16 16:24:49 <@zlopez:fedora.im> Any news on getting rid of oncall?
2025-10-16 16:25:06 <@nirik:matrix.scrye.com> no, I have had no time to start that discussion. I will try if fires go down
2025-10-16 16:25:19 <@nirik:matrix.scrye.com> also no oncall pings this last week either
2025-10-16 16:26:09 <@zlopez:fedora.im> Let me take the next week than
2025-10-16 16:26:19 <@zlopez:fedora.im> !info on call from 2025-10-17 to 2025-10-23 - zlopez
2025-10-16 16:26:42 <@zlopez:fedora.im> !topic Monitoring discussion [nirik]
2025-10-16 16:26:42 <@zlopez:fedora.im> !info https://nagios.fedoraproject.org/nagios 
2025-10-16 16:26:42 <@zlopez:fedora.im> !info Go over existing items and fix them
2025-10-16 16:27:14 <@nirik:matrix.scrye.com> it's pretty much the same as last week...
2025-10-16 16:27:18 <@nirik:matrix.scrye.com> nothing too new.
2025-10-16 16:27:20 <@zlopez:fedora.im> !oncall
2025-10-16 16:27:23 <@zodbot:fedora.im> If they do not respond, please file a ticket (https://pagure.io/fedora-infrastructure/issues)
2025-10-16 16:27:23 <@zodbot:fedora.im> ● @Zlopez:matrix.org (zlopez) Current Time for them: 18:27 (Europe/Prague)
2025-10-16 16:27:23 <@zodbot:fedora.im> 
2025-10-16 16:27:23 <@zodbot:fedora.im> The following people are oncall:
2025-10-16 16:27:27 <@zlopez:fedora.im> OK, set
2025-10-16 16:27:39 <@nirik:matrix.scrye.com> There are 2 openshift certs to renew... I can do that sometime, and/or train someone else to.
2025-10-16 16:27:44 <@zlopez:fedora.im> What about the fedorainfracloud machines
2025-10-16 16:27:46 <@zlopez:fedora.im> ?
2025-10-16 16:28:05 <@nirik:matrix.scrye.com> yes, that was there last week too. We can talk about them again...
2025-10-16 16:28:36 <@nirik:matrix.scrye.com> datanommer01.fedorainfracloud.org I still cannot seem to login to (it's in aws). It's unclear to me if it's going to be needed still.
2025-10-16 16:29:03 <@nirik:matrix.scrye.com> logdetective01.fedorainfracloud.org is being worked on... it has some different firewall rules needs as it runs a bunch of containers I think
2025-10-16 16:29:29 <@zlopez:fedora.im> I have a question respins01.fedorainfracloud.org, that is in AWS too?
2025-10-16 16:29:42 <@nirik:matrix.scrye.com> yep.
2025-10-16 16:30:04 <@nirik:matrix.scrye.com> we provide that to respins sig... but don't monitor it, or do anything much with it. They manage it for the most part.
2025-10-16 16:32:00 <@zlopez:fedora.im> OK
2025-10-16 16:32:12 <@nirik:matrix.scrye.com> I think James said it got rebooted yesterday, but I did not do it...
2025-10-16 16:32:12 <@zlopez:fedora.im> Let's go to backlog refinement than
2025-10-16 16:33:27 <@zlopez:fedora.im> !info Refine oldest tickets on Fedora Infra tracker
2025-10-16 16:33:27 <@zlopez:fedora.im> !link https://pagure.io/fedora-infrastructure/report/weekly_refinement
2025-10-16 16:33:27 <@zlopez:fedora.im> !topic Fedora Infra backlog refinement https://pagure.io/fedora-infrastructure/report/weekly_refinement
2025-10-16 16:34:01 <@zlopez:fedora.im> !ticket 7361
2025-10-16 16:34:02 <@zodbot:fedora.im> ● **Assignee:** kevin
2025-10-16 16:34:02 <@zodbot:fedora.im> ● **Last Updated:** 3 weeks ago
2025-10-16 16:34:02 <@zodbot:fedora.im> ● **Opened:** 6 years ago by pbrobinson
2025-10-16 16:34:02 <@zodbot:fedora.im> 
2025-10-16 16:34:02 <@zodbot:fedora.im> **fedora-infrastructure #7361** (https://pagure.io/fedora-infrastructure/issue/7361):**Signing infrastructure for aarch64**
2025-10-16 16:34:32 <@nirik:matrix.scrye.com> yes, this is in my current sprint, but I have had no time to actually work on it.
2025-10-16 16:34:39 <@nirik:matrix.scrye.com> I hope to someday
2025-10-16 16:35:36 <@nirik:matrix.scrye.com> Well, the part of this I want to work on is getting out secure boot signing moved to the new setup, then we can move forward on adding aarch64 (but there will be more work on it too)
2025-10-16 16:35:54 <@smooge:fedora.im> this is the sprint task that never ends.. it just goes on and on my friend
2025-10-16 16:37:09 <@humaton:fedora.im> hm
2025-10-16 16:37:30 <@nirik:matrix.scrye.com> if we have time at the end I would like to talk about the timeout issue. ;)
2025-10-16 16:37:41 <@zlopez:fedora.im> Should we move to next ticket?
2025-10-16 16:37:47 <@humaton:fedora.im> yup
2025-10-16 16:37:50 <@zlopez:fedora.im> !ticket 10241
2025-10-16 16:37:51 <@zodbot:fedora.im> **fedora-infrastructure #10241** (https://pagure.io/fedora-infrastructure/issue/10241):**Port apps to OIDC**
2025-10-16 16:37:51 <@zodbot:fedora.im> ● **Assignee:** zlopez
2025-10-16 16:37:51 <@zodbot:fedora.im> ● **Last Updated:** a week ago
2025-10-16 16:37:51 <@zodbot:fedora.im> ● **Opened:** 4 years ago by zlopez
2025-10-16 16:37:51 <@zodbot:fedora.im> 
2025-10-16 16:37:58 <@zlopez:fedora.im> This is going forward
2025-10-16 16:38:31 <@humaton:fedora.im> what are we missing?
2025-10-16 16:38:40 <@zlopez:fedora.im> I have ipsilon instance for OpenID separated in staging, there is issue with certificate to test everything, but I should be able to do the same on production after freeze
2025-10-16 16:39:15 <@nirik:matrix.scrye.com> Michal Konečný: I sent you email on certs, you should be able to get a new one for *.id.stg.fedoraproject.org. Or I can walk you through it sometime...
2025-10-16 16:39:49 <@zlopez:fedora.im> We can then look at Keycloak as replacement and just sunset the OpenID after some time, right now the page shows that the support will end at 1st May 2026, I assume that is enough time for anybody to switch to OIDC
2025-10-16 16:40:12 <@zlopez:fedora.im> I noticed it, maybe we can set some meeting for next week to go through it
2025-10-16 16:40:15 <@nirik:matrix.scrye.com> Aurélien B is setting up keycloak / ipa-tura in staging
2025-10-16 16:40:29 <@nirik:matrix.scrye.com> sure, fine with me
2025-10-16 16:41:44 <@zlopez:fedora.im> OK, let's go to next one
2025-10-16 16:41:47 <@nirik:matrix.scrye.com> then we can hopefully roll to prod after f43 is out the door
2025-10-16 16:42:15 <@zlopez:fedora.im> !ticket 10458
2025-10-16 16:42:16 <@zodbot:fedora.im> 
2025-10-16 16:42:16 <@zodbot:fedora.im> **fedora-infrastructure #10458** (https://pagure.io/fedora-infrastructure/issue/10458):**Help me move my discourse bots to production?**
2025-10-16 16:42:16 <@zodbot:fedora.im> ● **Assignee:** ryanlerch
2025-10-16 16:42:16 <@zodbot:fedora.im> ● **Last Updated:** 3 months ago
2025-10-16 16:42:16 <@zodbot:fedora.im> ● **Opened:** 3 years ago by mattdm
2025-10-16 16:42:31 <@zlopez:fedora.im> I think this is still waiting for @mattdm
2025-10-16 16:42:39 <@nirik:matrix.scrye.com> yeah, I was gonna ask mattdm about this... but keep forgetting.
2025-10-16 16:42:55 <@nirik:matrix.scrye.com> I wonder if we shouldn't just close it and say 'open when you have time/are ready to move forward'?
2025-10-16 16:43:16 <@humaton:fedora.im> I was just going to type ^^
2025-10-16 16:43:44 <@nirik:matrix.scrye.com> I think thats fine... since it's been sitting there for so long
2025-10-16 16:44:09 <@zlopez:fedora.im> I'm +1 for closing it as there is no response
2025-10-16 16:44:25 <@humaton:fedora.im> Yeah we might even get more strict. 2-3 pings in 3 months and closing
2025-10-16 16:44:57 <@humaton:fedora.im> It does not have to sit there if the requester is not interested or available.
2025-10-16 16:45:19 <@nirik:matrix.scrye.com> sure, I don't think we need a hard rule, but this one is clearly just not happening.
2025-10-16 16:45:28 <@nirik:matrix.scrye.com> close and move on?
2025-10-16 16:45:33 <@humaton:fedora.im> yup
2025-10-16 16:45:45 <@humaton:fedora.im> and I think we can close the refinement for today
2025-10-16 16:46:02 <@humaton:fedora.im> we did 3 tickets in 10mins have some updates and 1 down.
2025-10-16 16:46:14 <@zlopez:fedora.im> I switch to open floor, do you want me to close the ticket?
2025-10-16 16:46:21 <@nirik:matrix.scrye.com> sure, or I can if you prefer?
2025-10-16 16:46:31 <@zlopez:fedora.im> I will do it
2025-10-16 16:46:34 <@nirik:matrix.scrye.com> I have actually 2 items for open floor/general discussion
2025-10-16 16:46:46 <@zlopez:fedora.im> jednorozec was quicker 😄
2025-10-16 16:46:47 <@zlopez:fedora.im> !topic Open Floor
2025-10-16 16:47:50 <@nirik:matrix.scrye.com> first. I'd like us to move to forge after f43 is out the door. We should do any planning we need beforehand to be ready for it. ie, what goes where, how we redirect people, docs updates, etc. We have a ticket on this, but just wanted to mention it...
2025-10-16 16:48:28 <@nirik:matrix.scrye.com> I'll try and check the ticket / update it if things every quiet down.
2025-10-16 16:48:29 <@zlopez:fedora.im> That sounds like a good way to do it
2025-10-16 16:48:58 <@nirik:matrix.scrye.com> we need to be good about telling people about the new place since we get so many people filing infra tickets.
2025-10-16 16:49:23 <@nirik:matrix.scrye.com> and since so many people work on infra ansible
2025-10-16 16:50:08 <@humaton:fedora.im> Visibility is the key in this case
2025-10-16 16:50:20 <@humaton:fedora.im> should we start talking about it on discussions?
2025-10-16 16:50:39 <@nirik:matrix.scrye.com> https://pagure.io/fedora-infrastructure/issue/12737 I guess is the one I was thinking of?
2025-10-16 16:50:40 <@humaton:fedora.im> the audience there seems to be bigger
2025-10-16 16:50:49 <@nirik:matrix.scrye.com> yeah. Sounds good.
2025-10-16 16:51:08 <@humaton:fedora.im> Yup that one and releng have their own 
2025-10-16 16:51:29 <@nirik:matrix.scrye.com> we have a number of forge requests tickets... I don't know if ryan is handling those or what...
2025-10-16 16:51:58 <@humaton:fedora.im> I am 
2025-10-16 16:52:05 <@james:fedora.im> Yeh, I don't think it was running at one point. It then booted, but the person in the ticket asked for a reboot ... so I did that this morning.
2025-10-16 16:52:06 <@zlopez:fedora.im> How it will be with the mirroring to batcave?
2025-10-16 16:52:11 <@humaton:fedora.im> created bunch of the groups and ryan did some as well
2025-10-16 16:52:14 <@nirik:matrix.scrye.com> I suppose we could do a seperate ticket for infra and use that for our stuff since that ticket above is more general?
2025-10-16 16:52:44 <@humaton:fedora.im> yes, that one was more intended to discuss the new structure but there are some links and lists.
2025-10-16 16:52:45 <@james:fedora.im> Sorry for the delay ... also in the FPC meeting, and that's going great today ;)
2025-10-16 16:53:10 <@nirik:matrix.scrye.com> https://pagure.io/fedora-infrastructure/issue/12806 and https://pagure.io/fedora-infrastructure/issue/12793 and https://pagure.io/fedora-infrastructure/issue/12793 and https://pagure.io/fedora-infrastructure/issue/12409 and https://pagure.io/fedora-infrastructure/issue/12755
2025-10-16 16:53:23 <@nirik:matrix.scrye.com> (perhaps we should add a forge tag. ;)
2025-10-16 16:53:52 <@nirik:matrix.scrye.com> good question. we will need to figure that out. If it emits fedora-messaging messages we could use the same listener perhaps with minor adjustments.
2025-10-16 16:54:34 <@nirik:matrix.scrye.com> also there's https://pagure.io/fedora-infrastructure/issue/12558 about commops
2025-10-16 16:55:23 <@zlopez:fedora.im> I'm not sure how it is handled right now, I remember there were some issues during the datacenter move
2025-10-16 16:56:14 <@nirik:matrix.scrye.com> ok, we only have a few min left, I wanted to mention the gigantic pain timeout issue ( https://pagure.io/fedora-infrastructure/issue/12814 ). Folks are welcome to look and see if they can find it... I mentioned some more invasive things I could try (reinstalling proxies or virthosts ). We are of course in freeze, but this is kinda a outage too...
2025-10-16 16:57:24 <@nirik:matrix.scrye.com> I think Akashdeep Dhar was working on webhook fedora messaging for forge. I am not sure the status tho.
2025-10-16 16:57:31 <@zlopez:fedora.im> We tried to investigate it a little with nils, but didn't noticed the ticket that is already open
2025-10-16 16:58:11 <@nirik:matrix.scrye.com> I could have missed something for sure, but I have tried everything I could think of over the last few weeks.
2025-10-16 16:58:20 <@humaton:fedora.im> we can emit messages on any event from the new forge.
2025-10-16 16:58:58 <@nirik:matrix.scrye.com> we just need a 'ansible git repo updated' message to trigger a git mirror on. Should be hopefully easy to adjust.
2025-10-16 16:59:01 <@zlopez:fedora.im> I don't think we tried something you didn't
2025-10-16 17:00:25 <@zlopez:fedora.im> We can definitely try that before moving completely
2025-10-16 17:00:35 <@zlopez:fedora.im> We are at end of our time now
2025-10-16 17:00:41 <@zlopez:fedora.im> Let me end the meeting
2025-10-16 17:00:50 <@zlopez:fedora.im> !endmeeting