2025-07-17 17:01:14 <@Zlopez:matrix.org> !startmeeting Infrastructure (2025-07-10)
2025-07-17 17:01:17 <@meetbot:fedora.im> Meeting started at 2025-07-17 17:01:14 UTC
2025-07-17 17:01:18 <@meetbot:fedora.im> The Meeting name is 'Infrastructure (2025-07-10)'
2025-07-17 17:01:21 <@Zlopez:matrix.org> !info About our team: https://docs.fedoraproject.org/en-US/cle/
2025-07-17 17:01:21 <@Zlopez:matrix.org> !meetingname infrastructure
2025-07-17 17:01:21 <@Zlopez:matrix.org> !chair @nirik:matrix.scrye.com @zlopez:matrix.org @nb:fedora.im bodanel @dtometzki:fedora.im @jnsamyak:matrix.org @lenkaseg:fedora.im @patrikp:matrix.org
2025-07-17 17:01:21 <@Zlopez:matrix.org> !info Agenda is at: https://board.net/p/fedora-infra
2025-07-17 17:01:21 <@Zlopez:matrix.org> !topic Hola y bienvenido
2025-07-17 17:01:21 <@Zlopez:matrix.org> !info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
2025-07-17 17:01:22 <@meetbot:fedora.im> The Meeting Name is now infrastructure
2025-07-17 17:01:25 <@nirik:matrix.scrye.com> morning
2025-07-17 17:01:30 <@Zlopez:matrix.org> !hi
2025-07-17 17:01:32 <@zodbot:fedora.im> Michal Konecny (zlopez)
2025-07-17 17:01:53 <@Zlopez:matrix.org> Welcome on the Fedora Infrastructure meeting :-)
2025-07-17 17:03:23 <@Zlopez:matrix.org> Let's start
2025-07-17 17:03:29 <@Zlopez:matrix.org> !topic New folks introductions
2025-07-17 17:03:29 <@Zlopez:matrix.org> !info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
2025-07-17 17:03:29 <@Zlopez:matrix.org> !info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
2025-07-17 17:03:41 <@Zlopez:matrix.org> Do we have anybody new on the meeting?
2025-07-17 17:03:57 <@nirik:matrix.scrye.com> Hopefully some others besides us two are here today...
2025-07-17 17:04:03 <@nirik:matrix.scrye.com> but it is still summer I guess.
2025-07-17 17:04:39 <@james:fedora.im> I am here, just ending my previous meeting too
2025-07-17 17:04:43 <@Zlopez:matrix.org> It is
2025-07-17 17:05:29 <@Zlopez:matrix.org> So let's go to next one
2025-07-17 17:05:59 <@Zlopez:matrix.org> !info magic eight ball says:
2025-07-17 17:05:59 <@Zlopez:matrix.org> !info char 2025-07-24 - ???
2025-07-17 17:05:59 <@Zlopez:matrix.org> !topic Next chair
2025-07-17 17:05:59 <@Zlopez:matrix.org> !info char 2025-07-17 - zlopez
2025-07-17 17:07:07 <@nirik:matrix.scrye.com> I guess I could take it...
2025-07-17 17:07:09 <@Zlopez:matrix.org> Anybody wants to take chair for next week?
2025-07-17 17:08:52 <@Zlopez:matrix.org> So it's yours nirik 
2025-07-17 17:09:06 <@Zlopez:matrix.org> !info chair 2025-07-24 - nirik
2025-07-17 17:09:26 <@Zlopez:matrix.org> !info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1900 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-07-17 17:09:26 <@Zlopez:matrix.org> !info DC move is over, just cleanup and bringing up shipped machines left
2025-07-17 17:09:26 <@Zlopez:matrix.org> !topic announcements and information
2025-07-17 17:09:26 <@Zlopez:matrix.org> !info CLE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0815 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-07-17 17:10:16 <@Zlopez:matrix.org> Anything else to announce?
2025-07-17 17:11:04 <@nirik:matrix.scrye.com> nothing off hand...
2025-07-17 17:11:40 <@Zlopez:matrix.org> !topic Oncall
2025-07-17 17:11:40 <@Zlopez:matrix.org> !info https://docs.fedoraproject.org/en-US/infra/day_to_day_fedora/#_the_oncall_role_in_our_team
2025-07-17 17:11:40 <@Zlopez:matrix.org> !info on call from 2025-07-04 to 2025-07-10 - nirik
2025-07-17 17:11:40 <@Zlopez:matrix.org> !info on call from 2025-07-10 to 2025-07-17 - ?
2025-07-17 17:11:54 <@Zlopez:matrix.org> Hm, we didn't set any oncall last week
2025-07-17 17:12:00 <@nirik:matrix.scrye.com> ha, we didn't do this last time... yeah...
2025-07-17 17:12:12 <@nirik:matrix.scrye.com> I actually didn't notice any that I can recall.
2025-07-17 17:12:18 <@Zlopez:matrix.org> I can take it now
2025-07-17 17:12:36 <@Zlopez:matrix.org> There was a lot of post dc move things, so I think there wasn't much oncall
2025-07-17 17:13:05 <@nirik:matrix.scrye.com> yeah.
2025-07-17 17:13:20 <@nirik:matrix.scrye.com> folks were pretty nice about not pinging while we were moving / fixing things.
2025-07-17 17:13:22 <@Zlopez:matrix.org> !info on call from 2025-07-18 to 2025-07-24 - zlopez
2025-07-17 17:14:15 <@Zlopez:matrix.org> It seems that I was actually oncall :-D
2025-07-17 17:14:19 <@Zlopez:matrix.org> !oncall
2025-07-17 17:14:20 <@zodbot:fedora.im> The following people are oncall:
2025-07-17 17:14:20 <@zodbot:fedora.im> If they do not respond, please file a ticket (https://pagure.io/fedora-infrastructure/issues)
2025-07-17 17:14:20 <@zodbot:fedora.im> ● @Zlopez:matrix.org (zlopez) Current Time for them: 19:14 (Europe/Prague)
2025-07-17 17:14:20 <@zodbot:fedora.im> 
2025-07-17 17:14:34 <@nirik:matrix.scrye.com> ha. whoops
2025-07-17 17:14:45 <@nirik:matrix.scrye.com> It may have been the bot wasn't working when I was supposed to be taking it
2025-07-17 17:14:53 <@Zlopez:matrix.org> That was set on 26th June :-D
2025-07-17 17:15:08 <@james:fedora.im> Oof
2025-07-17 17:15:48 <@Zlopez:matrix.org> I must say I didn't notice any pings during DC move
2025-07-17 17:16:16 <@Zlopez:matrix.org> So I can stay for another week
2025-07-17 17:16:49 <@Zlopez:matrix.org> !info on call from 2025-07-25 to 2025-07-31 - ???
2025-07-17 17:16:51 <@nirik:matrix.scrye.com> ok. Or I could take it to make up for not the last few weeks.
2025-07-17 17:16:57 <@Zlopez:matrix.org> Does anybody wants the next one
2025-07-17 17:18:32 <@Zlopez:matrix.org> Does anybody wants the next one?
2025-07-17 17:18:41 <@nirik:matrix.scrye.com> I can, but I will be away the 31st...
2025-07-17 17:18:55 <@nirik:matrix.scrye.com> (at flossy 31st,1st,2nd)
2025-07-17 17:19:38 <@Zlopez:matrix.org> In that case, let's decide on next week
2025-07-17 17:19:59 <@Zlopez:matrix.org> !info Summary of last week: (from current oncall)
2025-07-17 17:20:14 <@Zlopez:matrix.org> I didn't remember any call for the last few weeks
2025-07-17 17:20:34 <@Zlopez:matrix.org> I remember people complaining about things not working after DC move, but no oncall pings
2025-07-17 17:20:50 <@Zlopez:matrix.org> !info Go over existing items and fix them
2025-07-17 17:20:50 <@Zlopez:matrix.org> !topic Monitoring discussion [nirik]
2025-07-17 17:20:50 <@Zlopez:matrix.org> !info https://nagios.fedoraproject.org/nagios 
2025-07-17 17:21:06 <@nirik:matrix.scrye.com> There's still a few items...
2025-07-17 17:21:56 <@nirik:matrix.scrye.com> 3 listed as down. One is our friend logdetective02... not sure whats wrong with kernel02 (it should be up / working), compose-x86-riscv01 I never deployed yet. I can do that
2025-07-17 17:22:37 <@nirik:matrix.scrye.com> certgetter http... just needs something adjusted to return a 200 instead of a 403
2025-07-17 17:22:46 <@Zlopez:matrix.org> The kernel02 looks similar to what I was solving today with copr hypervisor
2025-07-17 17:22:52 <@nirik:matrix.scrye.com> db01.stg.rdu3.fedoraproject.org disk... probibly we should make the disk bigger there.
2025-07-17 17:23:17 <@nirik:matrix.scrye.com> registry disk. I guess I can just grow it again... we need to move to quay and avoid the problem someday
2025-07-17 17:23:22 <@Zlopez:matrix.org> It should be good to check if there isn't any excessive logging on db01
2025-07-17 17:23:32 <@Zlopez:matrix.org> I hit that on ipa01
2025-07-17 17:23:43 <@nirik:matrix.scrye.com> and the datanommer checks should be fixed sometime once packages are thru
2025-07-17 17:24:06 <@nirik:matrix.scrye.com> could be...
2025-07-17 17:24:33 <@Zlopez:matrix.org> Waiting for the epel9 compose right now :-)
2025-07-17 17:24:33 <@nirik:matrix.scrye.com> anyhow, if anyone wants to fix any of those, feel free... do announce that you are working on it tho, so others don't do so also
2025-07-17 17:24:59 <@Zlopez:matrix.org> I'm trying to fix what I can, but the amount of work is piling up
2025-07-17 17:25:42 <@nirik:matrix.scrye.com> sure, none of the nagios things are urgent
2025-07-17 17:25:57 <@Zlopez:matrix.org> And I found this one today for example https://pagure.io/fedora-infrastructure/issue/12620#comment-978914
2025-07-17 17:26:07 <@nirik:matrix.scrye.com> I plan to go thru the 'post dc move' ticket again, and refile things still outstanding as seperate tickets and close that one...
2025-07-17 17:26:26 <@Zlopez:matrix.org> That would be probably great
2025-07-17 17:26:44 <@Zlopez:matrix.org> I should probably file tickets for all the bigger things I'm working on
2025-07-17 17:26:51 <@nirik:matrix.scrye.com> the biggest thing I think... is the 500's from src.fedoraproject.org for some people.
2025-07-17 17:27:10 <@nirik:matrix.scrye.com> I thought it was related to the core dumps on pkgs01, but now I am not sure at all... could be something else.
2025-07-17 17:28:21 <@Zlopez:matrix.org> It doesn't seem like that, I hit it yesterday and it seems that this is cached somewhere as the requests weren't even in dist-git access log
2025-07-17 17:28:42 <@Zlopez:matrix.org> I will try to investigate more next time I will see it
2025-07-17 17:29:01 <@nirik:matrix.scrye.com> yeah, it's anoying a lot of users. ;(
2025-07-17 17:29:32 <@nirik:matrix.scrye.com> I wonder if the 500 is coming from the proxy for some reason
2025-07-17 17:29:48 <@Zlopez:matrix.org> I think so, but I need to check
2025-07-17 17:31:00 <@Zlopez:matrix.org> So what we want to do next? Give thumbs up to vote
2025-07-17 17:31:05 <@Zlopez:matrix.org> Backlog refinement
2025-07-17 17:31:18 <@Zlopez:matrix.org> Open floor
2025-07-17 17:32:07 <@nirik:matrix.scrye.com> lets backlog/see if we can close anything. ;)
2025-07-17 17:32:18 <@Zlopez:matrix.org> You got it
2025-07-17 17:32:33 <@Zlopez:matrix.org> !topic Revisit blocked tickets
2025-07-17 17:32:33 <@Zlopez:matrix.org> !info Check if any blocked ticket is unblocked
2025-07-17 17:32:33 <@Zlopez:matrix.org> !link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status=
2025-07-17 17:32:57 <@Zlopez:matrix.org> !ticket 11377
2025-07-17 17:33:01 <@zodbot:fedora.im> 
2025-07-17 17:33:01 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-07-17 17:33:01 <@zodbot:fedora.im> ● **Opened:** 2 years ago by kevin
2025-07-17 17:33:01 <@zodbot:fedora.im> ● **Last Updated:** 5 months ago
2025-07-17 17:33:01 <@zodbot:fedora.im> **fedora-infrastructure #11377** (https://pagure.io/fedora-infrastructure/issue/11377):**Install AWX for testing proof of concept**
2025-07-17 17:34:04 <@Zlopez:matrix.org> I don't think we can move this forward now
2025-07-17 17:34:50 <@nirik:matrix.scrye.com> I want to revisit this, but I am not sure I want to go with awx anymore.
2025-07-17 17:35:04 <@Zlopez:matrix.org> Do we have better option?
2025-07-17 17:35:06 <@nirik:matrix.scrye.com> perhaps we should just close this and open a new spike when we are ready to revisit it?
2025-07-17 17:35:13 <@nirik:matrix.scrye.com> There's some other options...
2025-07-17 17:35:52 <@nirik:matrix.scrye.com> semaphore ... and some other one I am not recalling
2025-07-17 17:36:09 <@Zlopez:matrix.org> I don't mind closing it
2025-07-17 17:36:44 <@nirik:matrix.scrye.com> we could also look at perhaps forgejo actions/ci stuff...
2025-07-17 17:37:38 <@Zlopez:matrix.org> Maybe doing ARC investigation for it?
2025-07-17 17:38:20 <@nirik:matrix.scrye.com> possibly.
2025-07-17 17:38:30 <@nirik:matrix.scrye.com> I know Gwmngilfen had some ideas too.
2025-07-17 17:38:39 <@Zlopez:matrix.org> Ok, let me close it
2025-07-17 17:39:59 <@Zlopez:matrix.org> !ticket 7361
2025-07-17 17:40:01 <@zodbot:fedora.im> ● **Last Updated:** 4 months ago
2025-07-17 17:40:01 <@zodbot:fedora.im> ● **Opened:** 6 years ago by pbrobinson
2025-07-17 17:40:01 <@zodbot:fedora.im> **fedora-infrastructure #7361** (https://pagure.io/fedora-infrastructure/issue/7361):**Signing infrastructure for aarch64**
2025-07-17 17:40:01 <@zodbot:fedora.im> 
2025-07-17 17:40:01 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-07-17 17:40:46 <@nirik:matrix.scrye.com> getting so close... but not there yet.
2025-07-17 17:40:52 <@nirik:matrix.scrye.com> I can update it.
2025-07-17 17:41:20 <@nirik:matrix.scrye.com> basically we need to move to the new signing setup, get that all working (it should be close) and then we can look at it, but... there's some kernel issues
2025-07-17 17:41:26 <@nirik:matrix.scrye.com> and those need to get sorted out too.
2025-07-17 17:41:57 <@Zlopez:matrix.org> Sounds good
2025-07-17 17:43:10 <@Zlopez:matrix.org> That were all blocked tickets
2025-07-17 17:43:12 <@Zlopez:matrix.org> !info Refine oldest tickets on Fedora Infra tracker
2025-07-17 17:43:12 <@Zlopez:matrix.org> !topic Fedora Infra backlog refinement
2025-07-17 17:43:12 <@Zlopez:matrix.org> !link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc
2025-07-17 17:43:27 <@Zlopez:matrix.org> !ticket 12246
2025-07-17 17:43:28 <@zodbot:fedora.im> **fedora-infrastructure #12246** (https://pagure.io/fedora-infrastructure/issue/12246):**deploy a Draupnir instance for our matrix servers**
2025-07-17 17:43:28 <@zodbot:fedora.im> 
2025-07-17 17:43:28 <@zodbot:fedora.im> ● **Opened:** 9 months ago by kevin
2025-07-17 17:43:28 <@zodbot:fedora.im> ● **Last Updated:** 9 months ago
2025-07-17 17:43:28 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-07-17 17:43:59 <@nirik:matrix.scrye.com> we are using a community member run one now, so it's not super urgent
2025-07-17 17:44:12 <@nirik:matrix.scrye.com> we might want to wait until we move to our own server (there's another ticket on that)
2025-07-17 17:44:37 <@Zlopez:matrix.org> OK, do we want to mark this as blocked by another ticket?
2025-07-17 17:45:02 <@nirik:matrix.scrye.com> https://pagure.io/fedora-infrastructure/issue/12323
2025-07-17 17:45:07 <@nirik:matrix.scrye.com> yeah, seems reasonable
2025-07-17 17:46:56 <@Zlopez:matrix.org> Done
2025-07-17 17:47:09 <@Zlopez:matrix.org> !ticket 12323
2025-07-17 17:47:10 <@zodbot:fedora.im> 
2025-07-17 17:47:10 <@zodbot:fedora.im> ● **Opened:** 7 months ago by kevin
2025-07-17 17:47:10 <@zodbot:fedora.im> ● **Last Updated:** Never
2025-07-17 17:47:10 <@zodbot:fedora.im> ● **Assignee:** kevin
2025-07-17 17:47:10 <@zodbot:fedora.im> **fedora-infrastructure #12323** (https://pagure.io/fedora-infrastructure/issue/12323):**Deploy Element Server Suite operator in staging**
2025-07-17 17:47:18 <@Zlopez:matrix.org> That is the next one :-D
2025-07-17 17:48:14 <@nirik:matrix.scrye.com> ha.
2025-07-17 17:48:32 <@nirik:matrix.scrye.com> So, talked with Gwmngilfen about this one the other day... I think he may want to work on it.
2025-07-17 17:48:54 <@nirik:matrix.scrye.com> We are kinda waiting for element to move us to MAS (matrix authentication server), which will make this easier to migrate.
2025-07-17 17:49:04 <@nirik:matrix.scrye.com> which they are supposed to do pretty soon
2025-07-17 17:49:13 <@nirik:matrix.scrye.com> so, I guess mention him in that ticket?
2025-07-17 17:49:29 <@Zlopez:matrix.org> Let me update the ticket
2025-07-17 17:51:04 <@Zlopez:matrix.org> Updated
2025-07-17 17:51:14 <@Zlopez:matrix.org> !ticket 10458
2025-07-17 17:51:15 <@zodbot:fedora.im> **fedora-infrastructure #10458** (https://pagure.io/fedora-infrastructure/issue/10458):**Help me move my discourse bots to production?**
2025-07-17 17:51:15 <@zodbot:fedora.im> ● **Assignee:** ryanlerch
2025-07-17 17:51:15 <@zodbot:fedora.im> ● **Last Updated:** 7 months ago
2025-07-17 17:51:15 <@zodbot:fedora.im> ● **Opened:** 3 years ago by mattdm
2025-07-17 17:51:15 <@zodbot:fedora.im> 
2025-07-17 17:51:48 <@Zlopez:matrix.org> Last time I checked this was waiting for @mattdm
2025-07-17 17:52:02 <@nirik:matrix.scrye.com> yeah. I have a meetup with him in a few I can mention it. ;)
2025-07-17 17:53:35 <@Zlopez:matrix.org> We have 7 minutes left, so let's go to open floor and see if we have something to discuss there
2025-07-17 17:53:47 <@Zlopez:matrix.org> !topic Open Floor
2025-07-17 17:53:47 <@Zlopez:matrix.org> !endmeeting