2025-03-04 20:02:41 <@james:fedora.im> !startmeeting Fedora Infrastructure Ops Daily Standup Meeting

2025-03-04 20:02:42 <@meetbot:fedora.im> Meeting started at 2025-03-04 20:02:41 UTC
2025-03-04 20:02:43 <@meetbot:fedora.im> The Meeting name is 'Fedora Infrastructure Ops Daily Standup Meeting'
2025-03-04 20:02:45 <@james:fedora.im> !chair nirik phsmoura james
2025-03-04 20:02:49 <@james:fedora.im> !meetingname fedora_infrastructure_ops_daily_standup_meeting
2025-03-04 20:02:50 <@meetbot:fedora.im> The Meeting Name is now fedora_infrastructure_ops_daily_standup_meeting
2025-03-04 20:02:53 <@james:fedora.im> !info meeting is 30 minutes MAX. At the end of 30, its stops
2025-03-04 20:02:55 <@smilner:fedora.im> !hi
2025-03-04 20:02:56 <@james:fedora.im> !info agenda is at https://board.net/p/fedora-infra-daily
2025-03-04 20:02:56 <@zodbot:fedora.im> None (smilner)
2025-03-04 20:02:58 <@nirik:matrix.scrye.com> moring to all.
2025-03-04 20:03:00 <@james:fedora.im> !info reminder: speak up if you want to work on a ticket!
2025-03-04 20:03:14 <@james:fedora.im> !topic Tickets needing review
2025-03-04 20:03:18 <@james:fedora.im> !info https://pagure.io/fedora-infrastructure/issues?status=Open&priority=1
2025-03-04 20:03:36 <@james:fedora.im> !ticket 12435 
2025-03-04 20:03:37 <@zodbot:fedora.im> Issue querying Pagure: 403: FORBIDDEN
2025-03-04 20:03:59 <@smilner:fedora.im> 12436?
2025-03-04 20:04:06 <@james:fedora.im> Closed => Wait for pagure to go away ;)
2025-03-04 20:04:21 <@james:fedora.im> No, it's marked as private
2025-03-04 20:04:26 <@nirik:matrix.scrye.com> thats a private ticket I think, we just skip those here.
2025-03-04 20:04:35 <@james:fedora.im> Ahh, fair enough
2025-03-04 20:04:43 <@james:fedora.im> !ticket 12436
2025-03-04 20:04:43 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-03-04 20:04:43 <@zodbot:fedora.im> ● **Opened:** 3 hours ago by mattia
2025-03-04 20:04:43 <@zodbot:fedora.im> 
2025-03-04 20:04:43 <@zodbot:fedora.im> **fedora-infrastructure #12436** (https://pagure.io/fedora-infrastructure/issue/12436):**Inactive provenpackagers for the F42 cycle**
2025-03-04 20:04:43 <@zodbot:fedora.im> ● **Last Updated:** Never
2025-03-04 20:04:54 <@nirik:matrix.scrye.com> med gain, low trouble, ops?
2025-03-04 20:05:25 <@nirik:matrix.scrye.com> we usually wait a few days on these in case someone pops up at the last minute to say they are still around. ;)
2025-03-04 20:05:50 <@james:fedora.im> Sure. I know I can't do it anyway ;)
2025-03-04 20:06:09 <@james:fedora.im> Maybe just you and Fabian?
2025-03-04 20:06:30 <@nirik:matrix.scrye.com> yeah, needs accounts access. Should be sysadmin-main only...
2025-03-04 20:06:52 <@nirik:matrix.scrye.com> but it's super easy. Just ipa remove-account-from-group in a loop or whatever.
2025-03-04 20:07:09 <@james:fedora.im> !info https://pagure.io/releng/issues?status=Open
2025-03-04 20:07:22 <@james:fedora.im> So technically no new tickets, but...
2025-03-04 20:07:24 <@james:fedora.im> !releng 12595
2025-03-04 20:07:25 <@zodbot:fedora.im> **releng #12595** (https://pagure.io/releng/issue/12595):**Rawhide netinstall broken.**
2025-03-04 20:07:25 <@zodbot:fedora.im> 
2025-03-04 20:07:25 <@zodbot:fedora.im> ● **Opened:** 2 weeks ago by eddy02
2025-03-04 20:07:25 <@zodbot:fedora.im> ● **Last Updated:** a week ago
2025-03-04 20:07:25 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-03-04 20:07:40 <@james:fedora.im> We never tagged this one ... med/med?
2025-03-04 20:08:06 <@nirik:matrix.scrye.com> I guess. It's still unclear to me if it's something with the infra or something in the contents...
2025-03-04 20:08:38 <@smilner:fedora.im> From my very quick scan it looks more like a distro or compose bug
2025-03-04 20:08:39 <@nirik:matrix.scrye.com> oh, although...
2025-03-04 20:08:39 <@james:fedora.im> Yeh, I'd lean heavily towards contents ... so we could close.
2025-03-04 20:09:13 <@nirik:matrix.scrye.com> I was not able to boot rawhide kernels here form rc3->rc4... but then it's been working again. Perhaps this was the same issue... and it will work for them now?
2025-03-04 20:09:36 <@nirik:matrix.scrye.com> we could ask them to retest with current?
2025-03-04 20:10:24 <@smilner:fedora.im> We could. But what can we do about it if they figure out the issue? Should this be a Fedora bug report?
2025-03-04 20:11:20 <@nirik:matrix.scrye.com> well, if it works now, it's fixed? but if it still doesn't... I don't know... yeah, should probibly be a bug, but against what?
2025-03-04 20:11:21 <@james:fedora.im> I made a comment
2025-03-04 20:11:44 <@james:fedora.im> And tagged it, so we won't see it again anyway ;)
2025-03-04 20:12:08 <@nirik:matrix.scrye.com> dracut perhaps, systemd perhaps... dunno. I guess we can see if it's still happening for them
2025-03-04 20:12:13 <@smilner:fedora.im> Not exactly sure either. Maybe anaconda? Dracut?
2025-03-04 20:12:23 <@james:fedora.im> !topic Planning, Upcoming work and Open floor
2025-03-04 20:12:34 <@nirik:matrix.scrye.com> !info nirik is out on PTO thursday and friday
2025-03-04 20:12:46 <@nirik:matrix.scrye.com> !info f42 go/nogo is on thursday
2025-03-04 20:13:19 <@nirik:matrix.scrye.com> I thought I would mention that https://pagure.io/fedora-infrastructure/issue/12427 continues to vex me... if anyone has cycles to look it over and see if they can see anything that would be great
2025-03-04 20:13:42 <@james:fedora.im> I did patch and a PR based on yesterday's nftable deployment ... so can try deploying to another koji builder today and it "should work perfectly". https://pagure.io/fedora-infra/ansible/pull-request/2497
2025-03-04 20:13:56 <@nirik:matrix.scrye.com> (sporadic for some small number of people ostree getting 502's....)
2025-03-04 20:14:12 <@nirik:matrix.scrye.com> cool! sounds good. Thanks for driving that forward.
2025-03-04 20:14:59 <@nirik:matrix.scrye.com> other random news: DC folks fully power cycled bvmhost-p09-03 and it's back up! Also, ordered a replacement drive for the drive it has thats wonky...
2025-03-04 20:15:10 <@james:fedora.im> Yeh, when I first saw the 502 thing I wondered if it was related to the koji s390 problems ... but that seems to have been solved by a kernel change.
2025-03-04 20:15:56 <@nirik:matrix.scrye.com> yeah, Its the same machines, so it could be related...
2025-03-04 20:16:33 <@nirik:matrix.scrye.com> ¯\\\_(ツ)\_/¯ but it's basically that some of the things people are requesting don't exist. They are 404 on the kojipkgs end (which cloudfront returns as a 502)
2025-03-04 20:16:53 <@nirik:matrix.scrye.com> I don't know enough about how ostree requests things/repo is setup. ;(
2025-03-04 20:17:12 <@james:fedora.im> We could ping ostree people?
2025-03-04 20:17:21 <@nirik:matrix.scrye.com> Bad disk on bvmhost-p09-03 has last error at: Error 2474 occurred at disk power-on lifetime: 51455 hours (2143 days + 23 hours)
2025-03-04 20:17:25 <@smilner:fedora.im> James Antill: they opened the issue
2025-03-04 20:17:34 <@nirik:matrix.scrye.com> I did ping a bunch of people in the ticket. ;(
2025-03-04 20:17:49 <@smilner:fedora.im> But they still may be able to help look through logs or have more ideas as to what could have happened.
2025-03-04 20:17:54 <@nirik:matrix.scrye.com> https://mirrors.fedoraproject.org/metalink?repo=fedora-cisco-openh264-$releasever&arch=$basearch
2025-03-04 20:17:56 <@nirik:matrix.scrye.com> oops
2025-03-04 20:18:05 <@nirik:matrix.scrye.com> https://mirrors.fedoraproject.org/metalink?repo=fedora-cisco-openh264-$releasever&arch=$basearch
2025-03-04 20:18:09 <@nirik:matrix.scrye.com> dammit paste
2025-03-04 20:18:21 <@nirik:matrix.scrye.com> https://pagure.io/fedora-infrastructure/issue/12427#comment-958728
2025-03-04 20:18:34 <@james:fedora.im> Might be worth asking if there's some kind of repochecker script they could create which we could run in nagios?
2025-03-04 20:18:54 <@smilner:fedora.im> Oddly enough I can hit the url they reference without a 502
2025-03-04 20:18:59 <@smilner:fedora.im> I get a valid dirtree
2025-03-04 20:19:50 <@smilner:fedora.im> I'll poke around a little nirik and then I'll see if I can't get someone from team CoreOS team to help review
2025-03-04 20:20:10 <@nirik:matrix.scrye.com> I think it's sporadic... but no idea why it would be.
2025-03-04 20:20:26 <@nirik:matrix.scrye.com> there's a lot of example urls there, but all different kinds...
2025-03-04 20:20:32 <@nirik:matrix.scrye.com> that would be great if you could.
2025-03-04 20:21:25 <@nirik:matrix.scrye.com> I don't have much else I don't think.
2025-03-04 20:22:05 <@james:fedora.im> Yeh, only other thing is booking 20m after nirik 's lunch to make sure the second PR/deployment is good.
2025-03-04 20:22:20 <@nirik:matrix.scrye.com> yep. I can do that. ;) can ping once I am back at it.
2025-03-04 20:22:27 <@james:fedora.im> Sounds good.
2025-03-04 20:22:38 <@james:fedora.im> And on that note...
2025-03-04 20:22:40 <@james:fedora.im> !endmeeting