2025-01-09 17:00:06 <@phsmoura:fedora.im> !startmeeting Infrastructure (2025-01-09)
2025-01-09 17:00:07 <@meetbot:fedora.im> Meeting started at 2025-01-09 17:00:06 UTC
2025-01-09 17:00:07 <@meetbot:fedora.im> The Meeting name is 'Infrastructure (2025-01-09)'
2025-01-09 17:00:14 <@phsmoura:fedora.im> !info About our team: https://docs.fedoraproject.org/en-US/cpe/
2025-01-09 17:00:14 <@phsmoura:fedora.im> !meetingname infrastructure
2025-01-09 17:00:14 <@phsmoura:fedora.im> !chair nirik zlopez nb bodanel dtometzki jnsamyak lenkaseg patrikp
2025-01-09 17:00:14 <@phsmoura:fedora.im> !info Agenda is at: https://board.net/p/fedora-infra
2025-01-09 17:00:14 <@phsmoura:fedora.im> !topic ahoy 
2025-01-09 17:00:14 <@phsmoura:fedora.im> !info Fedora Infra documentation: https://docs.fedoraproject.org/en-US/infra
2025-01-09 17:00:15 <@meetbot:fedora.im> The Meeting Name is now infrastructure
2025-01-09 17:00:22 <@phsmoura:fedora.im> hello
2025-01-09 17:00:38 <@nirik:matrix.scrye.com> morning
2025-01-09 17:01:51 <@Zlopez:matrix.org> !hi
2025-01-09 17:01:54 <@zodbot:fedora.im> Michal Konecny (zlopez)
2025-01-09 17:02:03 <@james:fedora.im> !hi
2025-01-09 17:02:04 <@zodbot:fedora.im> James Antill (james)
2025-01-09 17:02:38 <@phsmoura:fedora.im> !info This is a place where people who are interested in Fedora Infrastructure can introduce themselves
2025-01-09 17:02:38 <@phsmoura:fedora.im> !info Getting Started Guide: https://docs.fedoraproject.org/en-US/infra/gettingstarted/
2025-01-09 17:02:38 <@phsmoura:fedora.im> !topic New folks introductions
2025-01-09 17:02:52 <@phsmoura:fedora.im> do we have new people around?
2025-01-09 17:03:05 <@phsmoura:fedora.im> feel free to introduce yourselves
2025-01-09 17:04:10 <@nirik:matrix.scrye.com> It's a new year, does that count? 😄
2025-01-09 17:04:39 <@Zlopez:matrix.org> You think the 2025 will be the year of Fedora?
2025-01-09 17:04:58 <@Zlopez:matrix.org> With Fedora 42 it's possible :-D
2025-01-09 17:05:25 <@phsmoura:fedora.im> !info chair 2025-01-16 - saibug
2025-01-09 17:05:25 <@phsmoura:fedora.im> !info chair 2025-01-23 - Yaash
2025-01-09 17:05:25 <@phsmoura:fedora.im> !topic Next chair
2025-01-09 17:05:25 <@phsmoura:fedora.im> !info magic eight ball says:
2025-01-09 17:05:25 <@phsmoura:fedora.im> !info chair 2025-01-30 - ???
2025-01-09 17:05:53 <@phsmoura:fedora.im> we are good for the next 2 wks, but is there any volunteer for 2025-01-30?
2025-01-09 17:06:36 <@Zlopez:matrix.org> Let's see next week
2025-01-09 17:07:04 <@phsmoura:fedora.im> ok, moving on then
2025-01-09 17:07:09 <@phsmoura:fedora.im> !info CLE Infra&Releng NA-hours team has a Monday through Thursday 30 minute meeting going through tickets at 1800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-01-09 17:07:09 <@phsmoura:fedora.im> !topic announcements and information
2025-01-09 17:07:09 <@phsmoura:fedora.im> !info CLE Infra&Releng EU-hours team has a Monday through Thursday 30 minute meeting going through tickets at 0800 UTC in https://matrix.to/#/#meeting-3:fedoraproject.org
2025-01-09 17:07:24 <@phsmoura:fedora.im> any other announcement?
2025-01-09 17:08:13 <@nirik:matrix.scrye.com> !info mass rebuild is next wed
2025-01-09 17:09:16 <@Zlopez:matrix.org> I completely forgot about that one
2025-01-09 17:10:46 <@nirik:matrix.scrye.com> coming up fast this time.
2025-01-09 17:11:36 <@nirik:matrix.scrye.com> I'm hoping to improve ppc64le builders before then... but we will see if I can. Probibly wouldn't hurt to update/reboot all the builders before then too...
2025-01-09 17:12:49 <@nirik:matrix.scrye.com> no other announcements I can think of
2025-01-09 17:13:04 <@phsmoura:fedora.im> lets see oncall
2025-01-09 17:13:23 <@Zlopez:matrix.org> I don't think we had any during shutdown
2025-01-09 17:13:58 <@nirik:matrix.scrye.com> we didn't. but... I forgot to remove myself... so I guess I was.
2025-01-09 17:14:19 <@nirik:matrix.scrye.com> but only one oncall that I recall. Someone with questions about one of the blogs
2025-01-09 17:15:06 <@james:fedora.im> Yeh, I remember one instance of nirik running !oncall to show nobody was ... and it returned nirik is on call. Was kind of amusing, while slightly drunk.
2025-01-09 17:15:55 <@nirik:matrix.scrye.com> heh, yeah, thats the one.
2025-01-09 17:16:03 <@nirik:matrix.scrye.com> The user DM'ed me their question...
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info ??? is on call from 2025-01-23 to 2025-01-30
2025-01-09 17:16:42 <@phsmoura:fedora.im> !topic Oncall
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info https://docs.fedoraproject.org/en-US/infra/day_to_day_fedora/#_the_oncall_role_in_our_team
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info nirik is on call from 2024-12-06 to 2024-12-12
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info NOBODY is on call from 2024-12-13 to 2025-01-09
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info ??? is on call from 2025-01-09 to 2025-01-16
2025-01-09 17:16:42 <@phsmoura:fedora.im> !info ??? is on call from 2025-01-16 to 2025-01-23
2025-01-09 17:17:07 <@phsmoura:fedora.im> volunteers for next wks tobe oncall?
2025-01-09 17:17:19 <@phsmoura:fedora.im> volunteers for next wks to be oncall?
2025-01-09 17:17:25 <@Zlopez:matrix.org> I will not be here next week, but I can take the one from 16th January
2025-01-09 17:17:36 <@nirik:matrix.scrye.com> I guess I could just keep it another week. ;)
2025-01-09 17:17:51 <@nirik:matrix.scrye.com> It doesn't seem like people use it as much on matrix vs irc. I am not sure why.
2025-01-09 17:18:02 <@Zlopez:matrix.org> And I remember there was the proposal to rotate the oncall and chairing, not sure if we got to any conclusion
2025-01-09 17:18:42 <@nirik:matrix.scrye.com> I'm fine doing that... as long as we allow new folks to take it if they want to....
2025-01-09 17:18:54 <@Zlopez:matrix.org> That was the idea
2025-01-09 17:19:09 <@nirik:matrix.scrye.com> but we might ping back in that discssion and make sure everyone is ok and then start doing it next week or two?
2025-01-09 17:19:31 <@Zlopez:matrix.org> We need to clear the chair list first
2025-01-09 17:19:47 <@Zlopez:matrix.org> And then use it for the rotation
2025-01-09 17:20:41 <@Zlopez:matrix.org> But I'm for it to start use it soon
2025-01-09 17:20:41 <@nirik:matrix.scrye.com> ok
2025-01-09 17:21:01 <@nirik:matrix.scrye.com> I don't think anyone really objected
2025-01-09 17:21:31 <@Zlopez:matrix.org> I don't remember any response in the discussion that would be against it, or here on the meeting
2025-01-09 17:22:43 <@phsmoura:fedora.im> 
2025-01-09 17:22:43 <@phsmoura:fedora.im> !info nirik is on call from 2025-01-09 to 2025-01-16
2025-01-09 17:22:43 <@phsmoura:fedora.im> !info zlopez is on call from 2025-01-16 to 2025-01-23
2025-01-09 17:22:43 <@phsmoura:fedora.im> so we have this for the next weeks:
2025-01-09 17:23:05 <@seddik:fedora.im> !hi
2025-01-09 17:23:06 <@zodbot:fedora.im> seddik alaouiismaili (seddik)
2025-01-09 17:23:20 <@nirik:matrix.scrye.com> 👍
2025-01-09 17:23:31 <@nirik:matrix.scrye.com> hey saibug. Hope you had a good holidays.
2025-01-09 17:24:15 <@seddik:fedora.im> Yes it was great
2025-01-09 17:24:48 <@phsmoura:fedora.im> !info Summary of last week: (@nirik)
2025-01-09 17:24:48 <@phsmoura:fedora.im> there was nobody oncall during the holidays and the last person oncall was nirik.. 
2025-01-09 17:25:08 <@phsmoura:fedora.im> there was nobody oncall during the holidays and the last person oncall was nirik.. 
2025-01-09 17:25:08 <@phsmoura:fedora.im> !info Summary of last week: ( nirik  )
2025-01-09 17:25:32 <@nirik:matrix.scrye.com> just that one oncall that I recall... nothing else. Was pretty quiet... just a few alerts and such...
2025-01-09 17:26:14 <@phsmoura:fedora.im> !info Go over existing items and fix them
2025-01-09 17:26:14 <@phsmoura:fedora.im> !info https://nagios.fedoraproject.org/nagios 
2025-01-09 17:26:14 <@phsmoura:fedora.im> !topic Monitoring discussion [nirik]
2025-01-09 17:26:48 <@nirik:matrix.scrye.com> lets see...
2025-01-09 17:27:22 <@nirik:matrix.scrye.com> we have proxy37 thats not behaving (I think due to network issues between iad2 and there)
2025-01-09 17:27:45 <@nirik:matrix.scrye.com> for logdetective02 I think there was a PR from saibug. Should look back at it
2025-01-09 17:28:09 <@nirik:matrix.scrye.com> there's a copr-dist-git, but it's acked, so I assume they know about it.
2025-01-09 17:28:15 <@nirik:matrix.scrye.com> otherwise, looking good. ;)
2025-01-09 17:28:33 <@nirik:matrix.scrye.com> oh, one other thing to note:
2025-01-09 17:29:23 <@nirik:matrix.scrye.com> we have been getting mailman alerts most days... when the updates pushes are happening. I looked slightly at it, and it looks like it's when sending to the package-announce list its getting a lot of bounces. But I am not sure why. We should look into it and fix it, getting daily alerts on it is anoying.
2025-01-09 17:30:16 <@nirik:matrix.scrye.com> alerts are like: PROBLEM [noc01]: mail_queue on mailman01.iad2.fedoraproject.org is CRITICAL 1014 mail(s) in queue
2025-01-09 17:31:00 <@nirik:matrix.scrye.com> If someone would like to investgate, feel free. ;)
2025-01-09 17:31:09 <@nirik:matrix.scrye.com> I don't think there's anything else nagios wise.
2025-01-09 17:31:24 <@phsmoura:fedora.im> maybe I can take a look while I dont have access to work on issue 12118?
2025-01-09 17:32:20 <@nirik:matrix.scrye.com> sure, it's probibly looking over logs from that time to see whats going on...
2025-01-09 17:33:13 <@phsmoura:fedora.im> ok, lets see blocked tickets
2025-01-09 17:33:26 <@phsmoura:fedora.im> !topic Revisit blocked tickets
2025-01-09 17:33:26 <@phsmoura:fedora.im> !info Check if any blocked ticket is unblocked
2025-01-09 17:33:26 <@phsmoura:fedora.im> !link https://pagure.io/fedora-infrastructure/issues?status=Open&tags=blocked&priority=0&close_status=
2025-01-09 17:34:20 <@nirik:matrix.scrye.com> still blocked I guess... although OIDC isn't really... blocked so much as needs some last few things done
2025-01-09 17:34:52 <@nirik:matrix.scrye.com> copr and anitya are left?
2025-01-09 17:35:58 <@Zlopez:matrix.org> I need to release new Anitya version, it has OIDC support
2025-01-09 17:36:51 <@Zlopez:matrix.org> Only COPR will be left
2025-01-09 17:37:51 <@nirik:matrix.scrye.com> I'll mention that on the copr bug.
2025-01-09 17:39:03 <@phsmoura:fedora.im> !topic Fedora Infra backlog refinement
2025-01-09 17:39:03 <@phsmoura:fedora.im> !link https://pagure.io/fedora-infrastructure/issues?status=Open&order_key=last_updated&order=asc
2025-01-09 17:39:03 <@phsmoura:fedora.im> !info Refine oldest tickets on Fedora Infra tracker
2025-01-09 17:40:22 <@nirik:matrix.scrye.com> ok, we haven't done this in a while... shall we take them one by one (we do only have ~20min tho)
2025-01-09 17:40:51 <@Zlopez:matrix.org> I'm for going through few of them
2025-01-09 17:41:25 <@nirik:matrix.scrye.com> go for it then...
2025-01-09 17:44:10 <@Zlopez:matrix.org> !ticket 7361
2025-01-09 17:44:12 <@zodbot:fedora.im> **fedora-infrastructure #7361** (https://pagure.io/fedora-infrastructure/issue/7361):**Signing infrastructure for aarch64**
2025-01-09 17:44:12 <@zodbot:fedora.im> 
2025-01-09 17:44:12 <@zodbot:fedora.im> ● **Opened:** 6 years ago by pbrobinson
2025-01-09 17:44:12 <@zodbot:fedora.im> ● **Last Updated:** 5 months ago
2025-01-09 17:44:12 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-01-09 17:44:30 <@Zlopez:matrix.org> That is one of the blocked ones, I see
2025-01-09 17:44:31 <@nirik:matrix.scrye.com> this is... being worked on, but is complicated.
2025-01-09 17:45:00 <@Zlopez:matrix.org> !ticket 12144
2025-01-09 17:45:02 <@zodbot:fedora.im> ● **Last Updated:** 5 months ago
2025-01-09 17:45:02 <@zodbot:fedora.im> **fedora-infrastructure #12144** (https://pagure.io/fedora-infrastructure/issue/12144):**Add support for creating RDS instances under `testing-farm-` prefix**
2025-01-09 17:45:02 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-01-09 17:45:02 <@zodbot:fedora.im> 
2025-01-09 17:45:02 <@zodbot:fedora.im> ● **Opened:** 5 months ago by mvadkert
2025-01-09 17:45:25 <@nirik:matrix.scrye.com> hum, must have missed this one when i landed. ;(
2025-01-09 17:45:44 <@nirik:matrix.scrye.com> just needs doing. I can or perhaps David Kirwan might ?
2025-01-09 17:45:56 <@Zlopez:matrix.org> We didn't even responded to this one and it's sitting there for 5 months already
2025-01-09 17:46:05 <@nirik:matrix.scrye.com> yeah, not sure why I missed it.
2025-01-09 17:46:16 <@nirik:matrix.scrye.com> it did get triaged.
2025-01-09 17:46:54 <@nirik:matrix.scrye.com> proposal: say we are sorry in ticket, cc David Kirwan and if he can't get to it, I can try in the next few days.
2025-01-09 17:47:39 <@Zlopez:matrix.org> I did that already
2025-01-09 17:48:04 <@nirik:matrix.scrye.com> ok, cool
2025-01-09 17:48:20 <@Zlopez:matrix.org> !ticket 12146
2025-01-09 17:48:21 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-01-09 17:48:21 <@zodbot:fedora.im> **fedora-infrastructure #12146** (https://pagure.io/fedora-infrastructure/issue/12146):**Manage our new testing.farm domain via AWS Route53**
2025-01-09 17:48:21 <@zodbot:fedora.im> 
2025-01-09 17:48:21 <@zodbot:fedora.im> ● **Opened:** 5 months ago by mvadkert
2025-01-09 17:48:21 <@zodbot:fedora.im> ● **Last Updated:** 5 months ago
2025-01-09 17:48:33 <@Zlopez:matrix.org> Another one for testing.farm
2025-01-09 17:48:49 <@nirik:matrix.scrye.com> I can do this one.
2025-01-09 17:48:54 <@Zlopez:matrix.org> And another one without reply for 5 months :-/
2025-01-09 17:49:04 <@nirik:matrix.scrye.com> again, not sure why it was missed. ;(
2025-01-09 17:49:07 <@Zlopez:matrix.org> They were probably opened at the same time
2025-01-09 17:49:20 <@nirik:matrix.scrye.com> but also... wonder why they didn't ping back to ask us?
2025-01-09 17:49:46 <@Zlopez:matrix.org> Maybe they solved that differently in the meantime and forgot about the ticket
2025-01-09 17:50:30 <@nirik:matrix.scrye.com> humm... .
2025-01-09 17:50:34 <@nirik:matrix.scrye.com> it seems to be done already
2025-01-09 17:51:01 <@nirik:matrix.scrye.com> So, let me ask them and close it? or just ask?
2025-01-09 17:51:27 <@nirik:matrix.scrye.com> I wonder if that other one is already done also
2025-01-09 17:51:30 <@Zlopez:matrix.org> If it's done let's ask and close it, they can reopen if something is still missing
2025-01-09 17:51:57 <@Zlopez:matrix.org> Probably somebody did this with AWS rights and just forgot about the ticket
2025-01-09 17:52:44 <@nirik:matrix.scrye.com> yeah, I'd like to add more auditing so we can see things like this, but haven't gotten to it.
2025-01-09 17:53:27 <@Zlopez:matrix.org> At least we see that we didn't do a backlog refinement for some time
2025-01-09 17:53:37 <@nirik:matrix.scrye.com> yeah
2025-01-09 17:54:07 <@Zlopez:matrix.org> !ticket 11804
2025-01-09 17:54:09 <@zodbot:fedora.im> **fedora-infrastructure #11804** (https://pagure.io/fedora-infrastructure/issue/11804):**rhel7 EOL - github2fedmsg**
2025-01-09 17:54:09 <@zodbot:fedora.im> 
2025-01-09 17:54:09 <@zodbot:fedora.im> ● **Opened:** 11 months ago by zlopez
2025-01-09 17:54:09 <@zodbot:fedora.im> ● **Last Updated:** 5 months ago
2025-01-09 17:54:09 <@zodbot:fedora.im> ● **Assignee:** abompard
2025-01-09 17:55:01 <@nirik:matrix.scrye.com> yeah, I don't know where we are here... but we should retire it soon
2025-01-09 17:55:02 <@Zlopez:matrix.org> We need to announce the sunset and replacement
2025-01-09 17:55:21 <@Zlopez:matrix.org> Replacement is already in place, people just need to migrate
2025-01-09 17:55:24 <@nirik:matrix.scrye.com> The replacement was announced... I think tickets were opened for all existing ones.
2025-01-09 17:55:52 <@nirik:matrix.scrye.com> trying to recall where that tracker was
2025-01-09 17:56:00 <@Zlopez:matrix.org> I will need to push this a little, as it's the last thing blocking both fedmsg and rhel7 EOL
2025-01-09 17:56:57 <@nirik:matrix.scrye.com> yeah, I can't seem to find that tracker, I know there was one... at least I think there was.
2025-01-09 17:57:10 <@Zlopez:matrix.org> Found it https://pagure.io/fedora-infra/w2fm-migration/
2025-01-09 17:57:37 <@nirik:matrix.scrye.com> ah yep.
2025-01-09 17:57:40 <@nirik:matrix.scrye.com> I just got there too.
2025-01-09 17:57:55 <@Zlopez:matrix.org> I went through the announcement blog post :-D
2025-01-09 17:57:57 <@nirik:matrix.scrye.com> so yeah, we should really push on getting this done
2025-01-09 17:58:12 <@Zlopez:matrix.org> I think this is the last one for backlog refinement
2025-01-09 17:58:20 <@james:fedora.im> https://pagure.io/fedora-infrastructure/issue/8213
2025-01-09 17:58:47 <@james:fedora.im> I think that's the tracker
2025-01-09 17:59:42 <@nirik:matrix.scrye.com> one of them. ;) Thats the fedmsg->fedora-messaging, but the last one of that is the github2fedmsg -> webhook2fedoramessaging (which is that other tracker)
2025-01-09 18:00:05 <@nirik:matrix.scrye.com> so, once we finish this we can close a bunch of things and shut off a bunch of vm's and remove a bunch of ansible
2025-01-09 18:00:36 <@Zlopez:matrix.org> I replied in the github2fedmsg ticket, if @abompard:fedora.im doesn't have time to work on that I will do the announcement
2025-01-09 18:00:55 <@nirik:matrix.scrye.com> I assume we want to retire fedmsg at the same time?
2025-01-09 18:00:55 <@Zlopez:matrix.org> Let's give users a month till we pull the plug
2025-01-09 18:01:01 <@Zlopez:matrix.org> Yes
2025-01-09 18:01:17 <@nirik:matrix.scrye.com> I'd say a few weeks... but I just want it done. ;)
2025-01-09 18:02:07 <@Zlopez:matrix.org> Let's go with three weeks than
2025-01-09 18:02:59 <@nirik:matrix.scrye.com> sure
2025-01-09 18:03:07 <@Zlopez:matrix.org> I think we are already overtime
2025-01-09 18:03:18 <@Zlopez:matrix.org> We can discuss that in the ticket itself
2025-01-09 18:03:19 <@nirik:matrix.scrye.com> yep
2025-01-09 18:05:21 <@phsmoura:fedora.im> we can talk about anything else on noc
2025-01-09 18:05:34 <@phsmoura:fedora.im> thanks everyone for joinning today :)
2025-01-09 18:05:49 <@phsmoura:fedora.im> !endmeeting