2025-01-27 09:31:42 <@meetbot:fedora.im> HTML Log: https://meetbot.fedoraproject.org/meeting-3_matrix_fedoraproject-org/2025-01-27/cpe-infra-releng-daily-standup.2025-01-27-09.00.log.html
2025-01-27 09:31:42 <@meetbot:fedora.im> Text Minutes: https://meetbot.fedoraproject.org/meeting-3_matrix_fedoraproject-org/2025-01-27/cpe-infra-releng-daily-standup.2025-01-27-09.00.txt
2025-01-27 16:00:13 <@patrikp:matrix.org> !startmeeting RELENG (2025-01-27)
2025-01-27 16:00:16 <@meetbot:fedora.im> Meeting started at 2025-01-27 16:00:13 UTC
2025-01-27 16:00:16 <@meetbot:fedora.im> The Meeting name is 'RELENG (2025-01-27)'
2025-01-27 16:00:21 <@patrikp:matrix.org> !info Agenda is at https://hackmd.io/vm6biLBcTYKtkQUH5kQkmw.
2025-01-27 16:00:21 <@patrikp:matrix.org> !meetingname releng
2025-01-27 16:00:21 <@patrikp:matrix.org> !chair nirik jnsamyak patrikp amedvede
2025-01-27 16:00:21 <@patrikp:matrix.org> !info Meeting is 60 minutes MAX. At the end of 60, it stops.
2025-01-27 16:00:22 <@meetbot:fedora.im> The Meeting Name is now releng
2025-01-27 16:00:29 <@patrikp:matrix.org> Hi.
2025-01-27 16:00:30 <@nirik:matrix.scrye.com> morning
2025-01-27 16:01:57 <@nirik:matrix.scrye.com> wonder if it's just us two today? :)
2025-01-27 16:02:01 <@patrikp:matrix.org> Not sure if Samyak is around, he rescheduled a 1:1 today because of noise so maybe he's travelling.
2025-01-27 16:02:06 <@patrikp:matrix.org> Could be yes.
2025-01-27 16:02:32 <@nirik:matrix.scrye.com> yeah, I think he said he might not be here today...
2025-01-27 16:03:22 <@patrikp:matrix.org> Let's freestyle it then. I'm looking at the tracker, there seems to be one new issue.
2025-01-27 16:03:28 <@patrikp:matrix.org> !releng 12547
2025-01-27 16:03:29 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-01-27 16:03:29 <@zodbot:fedora.im> ● **Last Updated:** 2 days ago
2025-01-27 16:03:29 <@zodbot:fedora.im> **releng #12547** (https://pagure.io/releng/issue/12547):**Updating flatpak-module-tools to 1.1**
2025-01-27 16:03:29 <@zodbot:fedora.im> ● **Opened:** 3 days ago by yselkowitz
2025-01-27 16:03:29 <@zodbot:fedora.im> 
2025-01-27 16:04:03 <@patrikp:matrix.org> Updating the builders? That means running a playbook?
2025-01-27 16:04:31 <@nirik:matrix.scrye.com> well, I am planning a update/reboot cycle for wed... it can just be done then as part of that if it's not needed sooner.
2025-01-27 16:05:11 <@nirik:matrix.scrye.com> or we can do it sooner, but might be disruptive if kojid needs restarting
2025-01-27 16:05:27 <@nirik:matrix.scrye.com> so anyhow, how about I ask in ticket about that?
2025-01-27 16:05:59 <@jnsamyak:matrix.org> hello hello
2025-01-27 16:06:06 <@patrikp:matrix.org> It says before beta freeze which is well off. Says "preferably sooner" but sounds like Wednesday is OK...
2025-01-27 16:06:31 <@nirik:matrix.scrye.com> hey jnsamyak!
2025-01-27 16:06:33 <@jnsamyak:matrix.org> I'm here :3 trying to manage timings
2025-01-27 16:06:41 <@patrikp:matrix.org> Is there more information about the update/reboot cycle? An SOP somewhere I could take a look like?
2025-01-27 16:07:43 <@patrikp:matrix.org> https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/massupgrade/
2025-01-27 16:07:45 <@patrikp:matrix.org> This one?
2025-01-27 16:08:48 <@jnsamyak:matrix.org> I would just say to do it either this week - or next to next week before branching 
2025-01-27 16:08:53 <@jnsamyak:matrix.org> or after
2025-01-27 16:09:12 <@nirik:matrix.scrye.com> yeah, I think thats the one...
2025-01-27 16:09:19 <@nirik:matrix.scrye.com> well, it will be done on wed.
2025-01-27 16:09:26 <@nirik:matrix.scrye.com> I am not going to exclude that one package. ;)
2025-01-27 16:09:30 <@jnsamyak:matrix.org> awesome 💃
2025-01-27 16:10:19 <@patrikp:matrix.org> Basically it's about running a bunch of playbooks and observing and making sure they don't fail?
2025-01-27 16:11:00 <@nirik:matrix.scrye.com> in the right order and making sure applications don't break after. :)
2025-01-27 16:12:04 <@patrikp:matrix.org> Excuse my ignorance but do you usually talk about it in the releng channel as you do those? I'd like to tag along/watch, looks like a practical thing to go through.
2025-01-27 16:12:23 <@nirik:matrix.scrye.com> In the #noc:fedoraproject.org channel usually.
2025-01-27 16:12:41 <@nirik:matrix.scrye.com> There's already an outage ticket and hackmd doc. ;)
2025-01-27 16:13:20 <@patrikp:matrix.org> Thanks, I'll take a look. I suppose this was the init? We kind of went into it freestyle but as Samyak is here I suppose let's move to the next section properly.
2025-01-27 16:13:26 <@patrikp:matrix.org> !info Here we list/discuss anything about items that are due to be done in the next week.
2025-01-27 16:13:26 <@patrikp:matrix.org> !link https://fedorapeople.org/groups/schedule/f-41/f-41-releng-tasks.html
2025-01-27 16:13:26 <@patrikp:matrix.org> !topic Scheduled actions coming up in the next week.
2025-01-27 16:13:26 <@patrikp:matrix.org> !link https://fedorapeople.org/groups/schedule/f-42/f-42-releng-tasks.html
2025-01-27 16:13:49 <@patrikp:matrix.org> Mass branching next week.
2025-01-27 16:14:34 <@jnsamyak:matrix.org> hehe I'm free for freestyling as well
2025-01-27 16:14:38 <@nirik:matrix.scrye.com> time goes by so fast. ;(
2025-01-27 16:14:48 <@jnsamyak:matrix.org> yeahhhhhhh ://///
2025-01-27 16:15:01 <@jnsamyak:matrix.org> I thought i would do something about it and it is here again
2025-01-27 16:15:12 <@jnsamyak:matrix.org> but nvm i had one quick thing about filing bugs 
2025-01-27 16:15:23 <@patrikp:matrix.org> Sure, go ahead. 👍️
2025-01-27 16:15:39 <@jnsamyak:matrix.org> !releng 12545
2025-01-27 16:15:40 <@zodbot:fedora.im> 
2025-01-27 16:15:40 <@zodbot:fedora.im> ● **Opened:** 4 days ago by churchyard
2025-01-27 16:15:40 <@zodbot:fedora.im> ● **Assignee:** jnsamyak
2025-01-27 16:15:40 <@zodbot:fedora.im> ● **Last Updated:** 4 days ago
2025-01-27 16:15:40 <@zodbot:fedora.im> **releng #12545** (https://pagure.io/releng/issue/12545):**300+ F42FTBFS bugzillas block the F41FTBFS tracker**
2025-01-27 16:16:00 <@jnsamyak:matrix.org> I feel I'm stuck at figuring bugzilla api
2025-01-27 16:16:12 <@jnsamyak:matrix.org> Last two comments, I have left my findings
2025-01-27 16:16:22 <@nirik:matrix.scrye.com> yeah, I am not sure who would know it better...
2025-01-27 16:17:07 <@nirik:matrix.scrye.com> how many bugs is it?
2025-01-27 16:17:18 <@nirik:matrix.scrye.com> ah, 334...
2025-01-27 16:18:22 <@jnsamyak:matrix.org> yeah, I could do it manually
2025-01-27 16:18:27 <@jnsamyak:matrix.org> but automation would be better
2025-01-27 16:18:28 <@nirik:matrix.scrye.com> can you just do a advanced query on the web interface and mass edit them there?
2025-01-27 16:18:39 <@jnsamyak:matrix.org> anyways I will figure something out
2025-01-27 16:18:43 <@jnsamyak:matrix.org> yeah selenium 
2025-01-27 16:18:46 <@jnsamyak:matrix.org> is my last resort
2025-01-27 16:20:24 <@nirik:matrix.scrye.com> well, I meant I think there's some 'edit' thing for lists of bugs... can do it in one go... but I might be misremembering
2025-01-27 16:20:41 <@jnsamyak:matrix.org> I will check 
2025-01-27 16:20:54 <@jnsamyak:matrix.org> thank you didnt know that
2025-01-27 16:21:21 <@jnsamyak:matrix.org> brb in 5 but that's all i have from my end on this topic
2025-01-27 16:21:57 <@patrikp:matrix.org> !link https://docs.fedoraproject.org/en-US/infra/release_guide/sop_mass_branching/
2025-01-27 16:22:38 <@nirik:matrix.scrye.com> there's also a checklist now I think? we should definitely use that
2025-01-27 16:22:50 <@patrikp:matrix.org> Ugh, already looking at the SOP...
2025-01-27 16:23:00 <@nirik:matrix.scrye.com> mass branching is easily the most complex part of the release cycle IMHO. So many things need changing
2025-01-27 16:23:19 <@nirik:matrix.scrye.com> https://docs.fedoraproject.org/en-US/infra/release_guide/mass_branching_checklist/
2025-01-27 16:23:19 <@jnsamyak:matrix.org> Yes there is, and I’ll open the tracker for that as well
2025-01-27 16:23:44 <@nirik:matrix.scrye.com> and... it would be great depending on time of course to actually start making pr's and getting everything lined up
2025-01-27 16:25:07 <@jnsamyak:matrix.org> I plan to start working on this week, so we can have all of then verified before next week 
2025-01-27 16:25:16 <@jnsamyak:matrix.org> All helps are welcome 🤗 
2025-01-27 16:25:19 <@jnsamyak:matrix.org> Hehe 
2025-01-27 16:25:34 <@patrikp:matrix.org> Let's talk about this more about tomorrow's 1:1.
2025-01-27 16:25:46 <@jnsamyak:matrix.org> Sure thing 
2025-01-27 16:25:47 <@patrikp:matrix.org> more during*
2025-01-27 16:26:10 <@patrikp:matrix.org> Alright. So that's it for upcoming actions and we went through the tickets... Let's see.
2025-01-27 16:26:19 <@patrikp:matrix.org> !topic Choose next chair.
2025-01-27 16:26:35 <@patrikp:matrix.org> Anybody want to host it? I can take it too, I don't mind it.
2025-01-27 16:26:54 <@jnsamyak:matrix.org> I can chair next week hopefullly if branching doesn’t bite my add
2025-01-27 16:27:09 <@jnsamyak:matrix.org> I can chair next week hopefullly if branching doesn’t bite my ss 😂
2025-01-27 16:27:15 <@patrikp:matrix.org> !info Next chair Feb 3rd: Samyak
2025-01-27 16:27:39 <@patrikp:matrix.org> !topic Discussion / Any other business.
2025-01-27 16:27:49 <@patrikp:matrix.org> We still have quite a bit of time. Anything else on your minds?
2025-01-27 16:28:51 <@patrikp:matrix.org> OK I have something then.
2025-01-27 16:30:43 <@patrikp:matrix.org> I'm playing with the OpenH composes, already talked to nirik about it, but now we can generate the repos, e.g.:
2025-01-27 16:30:43 <@patrikp:matrix.org> What exactly does it involve for it to be considered "multilib"? Wouldn't it only include adding 32 bit architectures? I'm missing the bigger picture.
2025-01-27 16:30:43 <@patrikp:matrix.org> 
2025-01-27 16:30:43 <@patrikp:matrix.org> https://kojipkgs.fedoraproject.org/repos-dist/f42-openh264/latest/
2025-01-27 16:31:01 <@patrikp:matrix.org> I made some attempts at the config file but unsuccessful.
2025-01-27 16:31:03 <@nirik:matrix.scrye.com> the koji dist-repo multilib is currently broken. ;(
2025-01-27 16:31:15 <@patrikp:matrix.org> But if it's just specifying arches in the config file anyway.....
2025-01-27 16:31:17 <@nirik:matrix.scrye.com> yeah, it's including i686 packages in the x86_64 repo
2025-01-27 16:31:30 <@nirik:matrix.scrye.com> (multilib that is)
2025-01-27 16:31:59 <@patrikp:matrix.org> So what does multilib mean in this context? I tried looking it up in a general sense but I'm not sure I get it.
2025-01-27 16:32:06 <@nirik:matrix.scrye.com> https://pagure.io/releng/issue/12473
2025-01-27 16:32:26 <@nirik:matrix.scrye.com> ^ this is the request to do that (our current ones are not multilib)
2025-01-27 16:32:50 <@nirik:matrix.scrye.com> so, IMHO, we should not block on that... koji upstream is working on fixing it (I hope), so we can enable it later once thats fixed.
2025-01-27 16:33:59 <@patrikp:matrix.org> So we can ignore the multilib part? That makes it easier. Alright, in that case don't we have everything we need?
2025-01-27 16:34:26 <@patrikp:matrix.org> We have the rpms.
2025-01-27 16:34:28 <@nirik:matrix.scrye.com> yep. I think we do. as long as the repo looks/contains what the old one did (which it should).
2025-01-27 16:34:48 <@nirik:matrix.scrye.com> so, if the packages look right/good against the old one, I'd say we could send to cisco...
2025-01-27 16:35:06 <@nirik:matrix.scrye.com> at least for rawhide at first?
2025-01-27 16:35:11 <@patrikp:matrix.org> I will take a closer look, I have the older compose saved locally (odcs).
2025-01-27 16:35:22 <@patrikp:matrix.org> And write up how to actually get to this point...
2025-01-27 16:35:39 <@patrikp:matrix.org> Now the actual rpms should be available anywhere Koji is mounted?
2025-01-27 16:35:42 <@nirik:matrix.scrye.com> Yep. will be great to get it done/moving again
2025-01-27 16:35:50 <@nirik:matrix.scrye.com> yep.
2025-01-27 16:36:02 <@jnsamyak:matrix.org> +100
2025-01-27 16:36:23 <@jnsamyak:matrix.org> But I’m happy to see progress on this as well
2025-01-27 16:36:51 <@patrikp:matrix.org> I can see the rpms. 👍️
2025-01-27 16:37:25 <@patrikp:matrix.org> Again, for reference the SOP is here.
2025-01-27 16:37:25 <@patrikp:matrix.org> !link https://docs.fedoraproject.org/en-US/infra/releng_misc_guide/sop_generating_openh264_composes/
2025-01-27 16:37:25 <@patrikp:matrix.org> So hypothetical next steps once I confirm that we have everything and it can be shipped to Cisco...
2025-01-27 16:37:59 <@patrikp:matrix.org> And we are at the step where we would like to make the tarballs.
2025-01-27 16:37:59 <@patrikp:matrix.org> https://docs.fedoraproject.org/en-US/infra/releng_misc_guide/sop_generating_openh264_composes/#_make_the_changes_needed
2025-01-27 16:38:05 <@jnsamyak:matrix.org> Corrrect
2025-01-27 16:38:50 <@patrikp:matrix.org> And once Cisco publish them, what would be the next steps? Just sync to sundries which should be without issue and sync to mm-backend which now runs in OpenShift?
2025-01-27 16:39:14 <@patrikp:matrix.org> Or am I missing something?
2025-01-27 16:40:17 <@jnsamyak:matrix.org> No yes yes 
2025-01-27 16:40:52 <@jnsamyak:matrix.org> Yes the last steps to be sync to sundries 
2025-01-27 16:41:32 <@patrikp:matrix.org> The SOP is outdated, we ran into issues with mm being in OpenShift now but I'm pretty sure we overcame it so there should be a way to do it.
2025-01-27 16:42:36 <@patrikp:matrix.org> And dist-repo stores each individual compose so even a rollback shouldn't be difficult at all if something goes wrong, which was not the case before.
2025-01-27 16:42:57 <@patrikp:matrix.org> Overall these realizations make me very happy.
2025-01-27 16:44:52 <@patrikp:matrix.org> That's all I have on this topic.
2025-01-27 16:49:59 <@patrikp:matrix.org> I try to do it via this command:
2025-01-27 16:49:59 <@patrikp:matrix.org> I have one more thing, real quick. Ran into an issue doing package unretirements. When I try to push the unretirement commit I get:
2025-01-27 16:49:59 <@patrikp:matrix.org> `error: %patchN is obsolete, use %patch N (or %patch -P N): %patch0 -p1 -b .r
2025-01-27 16:49:59 <@patrikp:matrix.org> 
2025-01-27 16:49:59 <@patrikp:matrix.org> Could not execute pre_push_check: Failed to execute command.
2025-01-27 16:49:59 <@patrikp:matrix.org> error: failed to push some refs to 'ssh://pkgs.fedoraproject.org/rpms/tachyon'`
2025-01-27 16:49:59 <@patrikp:matrix.org> 
2025-01-27 16:49:59 <@patrikp:matrix.org> `GIT_SSH=/usr/local/bin/relengpush fedpkg --user releng push`
2025-01-27 16:49:59 <@patrikp:matrix.org> 
2025-01-27 16:49:59 <@patrikp:matrix.org> I thought it was a fedpkg thing but even if I try to push it directly via git it still fails.
2025-01-27 16:49:59 <@patrikp:matrix.org> `GIT_SSH=/usr/local/bin/relengpush git push origin`
2025-01-27 16:49:59 <@patrikp:matrix.org> 
2025-01-27 16:49:59 <@patrikp:matrix.org> What do I do? Should I fix the spec file and amend it to the unretirement commit? I don't want to mess with people's packages needlessly but it's a small enough change I guess.
2025-01-27 16:51:23 <@nirik:matrix.scrye.com> sorry, took longer than I thought it would... reading back
2025-01-27 16:52:03 <@nirik:matrix.scrye.com> pushing to sundries should be enough, mm now reads from that same volume
2025-01-27 16:52:37 <@nirik:matrix.scrye.com> On the unretire, the problem is not in your command, it's rpm changes... there might be a way to disable that precheck
2025-01-27 16:53:09 <@nirik:matrix.scrye.com> fedpkg has a --no-verify argument, passing that I think will bypass it.
2025-01-27 16:55:02 <@patrikp:matrix.org> When I grep for it (fedpkg --help | grep verify) I don't see such an option.
2025-01-27 16:56:03 <@patrikp:matrix.org> Maybe "pre-push-check" to NO?
2025-01-27 16:56:50 <@nirik:matrix.scrye.com> might depend on the fedpkg version? I was looking at rawhide here.
2025-01-27 16:57:16 <@nirik:matrix.scrye.com> oh, it's position dependent
2025-01-27 16:57:23 <@nirik:matrix.scrye.com> fedpkg push --help | grep verify
2025-01-27 16:59:07 <@patrikp:matrix.org> That did the trick!
2025-01-27 16:59:10 <@patrikp:matrix.org> Thanks!
2025-01-27 16:59:16 <@patrikp:matrix.org> And we are at time.
2025-01-27 16:59:25 <@patrikp:matrix.org> !info Thank you all for coming!
2025-01-27 16:59:25 <@patrikp:matrix.org> !endmeeting