2025-02-17 16:00:15 <@patrikp:matrix.org> !startmeeting RELENG (2025-02-17)
2025-02-17 16:00:17 <@meetbot:fedora.im> Meeting started at 2025-02-17 16:00:15 UTC
2025-02-17 16:00:17 <@meetbot:fedora.im> The Meeting name is 'RELENG (2025-02-17)'
2025-02-17 16:00:21 <@patrikp:matrix.org> !info Agenda is at https://hackmd.io/vm6biLBcTYKtkQUH5kQkmw.
2025-02-17 16:00:21 <@patrikp:matrix.org> !meetingname releng
2025-02-17 16:00:21 <@patrikp:matrix.org> !chair nirik jnsamyak patrikp amedvede
2025-02-17 16:00:21 <@patrikp:matrix.org> !info Meeting is 60 minutes MAX. At the end of 60, it stops.
2025-02-17 16:00:22 <@meetbot:fedora.im> The Meeting Name is now releng
2025-02-17 16:00:48 <@patrikp:matrix.org> Good day to all people of good faith.
2025-02-17 16:00:56 <@nirik:matrix.scrye.com> morning
2025-02-17 16:02:14 <@jnsamyak:matrix.org> hello hello
2025-02-17 16:02:31 <@jnsamyak:matrix.org> how's everyone doing? I guess one less monday  this month :P
2025-02-17 16:02:33 <@smilner:fedora.im> !hi
2025-02-17 16:02:34 <@zodbot:fedora.im> None (smilner)
2025-02-17 16:03:26 <@nirik:matrix.scrye.com> its mondays all the way down. ;)
2025-02-17 16:03:51 <@jnsamyak:matrix.org> For the init, I just want to inform to have atleast 30 mins today for the mass branching retro in the end 
2025-02-17 16:04:05 <@patrikp:matrix.org> Alright, let's go to init then.
2025-02-17 16:04:10 <@patrikp:matrix.org> !topic Init process.
2025-02-17 16:04:10 <@patrikp:matrix.org> Do we have anything for the init? Any blockers/tasks/issues/requests/features that need releng intervention?
2025-02-17 16:04:15 <@jnsamyak:matrix.org> so let's get started and the last topic will be Mass Branching Retro
2025-02-17 16:04:20 <@patrikp:matrix.org> Anything else apart from what Samyak has said?
2025-02-17 16:04:41 <@patrikp:matrix.org> Moving on to scheduled actions...
2025-02-17 16:04:43 <@patrikp:matrix.org> !link https://fedorapeople.org/groups/schedule/f-41/f-41-releng-tasks.html
2025-02-17 16:04:43 <@patrikp:matrix.org> !topic Scheduled actions coming up in the next week.
2025-02-17 16:04:43 <@patrikp:matrix.org> !link https://fedorapeople.org/groups/schedule/f-42/f-42-releng-tasks.html
2025-02-17 16:04:43 <@patrikp:matrix.org> !info Here we list/discuss anything about items that are due to be done in the next week.
2025-02-17 16:04:52 <@jnsamyak:matrix.org> EPEL mass branching will start once we have the snapshot for it ready
2025-02-17 16:05:15 <@jnsamyak:matrix.org> they will be filing a ticket for it, we need to keep an eye for that, those are generally high priority ticket
2025-02-17 16:05:31 <@nirik:matrix.scrye.com> f42 beta freeze is tomorrow. ;)
2025-02-17 16:06:06 <@patrikp:matrix.org> Talked to Samyak earlier and looks like the changes are ready to go just need to be pushed.
2025-02-17 16:06:08 <@jnsamyak:matrix.org> yes!
2025-02-17 16:06:22 <@patrikp:matrix.org> How will we do it tomorrow? Hop on a call or in a Matrix channel?
2025-02-17 16:06:28 <@jnsamyak:matrix.org> I'll link the PR, as soon as I push those changes 
2025-02-17 16:06:34 <@patrikp:matrix.org> Freeze is fairly straightforward I suppose.
2025-02-17 16:07:30 <@jnsamyak:matrix.org> I'll try to talk about that in the channel unless you folks prefer to jump on the call, by talking in channel we have a variety of other community folks seeing what we do, so I prefer it doing there
2025-02-17 16:07:34 <@jnsamyak:matrix.org> yes
2025-02-17 16:07:52 <@nirik:matrix.scrye.com> channel >> call (ie, please channel)
2025-02-17 16:08:02 <@patrikp:matrix.org> Works for me.
2025-02-17 16:08:07 <@nirik:matrix.scrye.com> note that adamw has a pr updating some docs:
2025-02-17 16:08:17 <@jnsamyak:matrix.org> any blockers before the freeze that you see on hand nirik ?
2025-02-17 16:08:28 <@nirik:matrix.scrye.com> https://pagure.io/infra-docs-fpo/pull-request/361#
2025-02-17 16:08:49 <@nirik:matrix.scrye.com> this will update that freeze doc a bit.
2025-02-17 16:09:50 <@nirik:matrix.scrye.com> I don't think it's anything too big... some less playbooks, the email template into the docs, a bunch of clarifications.
2025-02-17 16:10:01 <@nirik:matrix.scrye.com> but hopefully we can merge that today and use it tomorrow?
2025-02-17 16:10:40 <@nirik:matrix.scrye.com> but yeah, beta freeze / bodhi activation is much less compelx than branching. ;)
2025-02-17 16:10:55 <@patrikp:matrix.org> Looking at Adam's comment it looks like he wants to add some final touches before merging?
2025-02-17 16:11:22 <@jnsamyak:matrix.org> interesting I'll have to review it properly will do post call or tomorrow morning
2025-02-17 16:11:42 <@jnsamyak:matrix.org> try to do it post call 
2025-02-17 16:12:06 <@nirik:matrix.scrye.com> yeah... there was some debate about the variable 'frozen'...
2025-02-17 16:12:42 <@patrikp:matrix.org> Alright, anything else regarding the freeze or shall we move on?
2025-02-17 16:13:10 <@patrikp:matrix.org> !link https://pagure.io/releng/issues
2025-02-17 16:13:10 <@patrikp:matrix.org> !topic Tickets that need attention.
2025-02-17 16:13:10 <@patrikp:matrix.org> !info This is the time where you can bring up releng tickets to discuss if there are blockers, etc.
2025-02-17 16:14:51 <@nirik:matrix.scrye.com> I was gonna check in on the openh264 stuff.... thats heating up because we don't have a valid rawhide repo for it currently. ;(
2025-02-17 16:15:46 <@nirik:matrix.scrye.com> https://pagure.io/releng/issues?status=Open&search_pattern=h264&close_status=
2025-02-17 16:15:53 <@patrikp:matrix.org> !releng 12585
2025-02-17 16:15:55 <@zodbot:fedora.im> ● **Last Updated:** 4 days ago
2025-02-17 16:15:55 <@zodbot:fedora.im> ● **Assignee:** Not Assigned
2025-02-17 16:15:55 <@zodbot:fedora.im> **releng #12585** (https://pagure.io/releng/issue/12585):**Please update openh264 to 2.6.0 in rawhide and F42**
2025-02-17 16:15:55 <@zodbot:fedora.im> 
2025-02-17 16:15:55 <@zodbot:fedora.im> ● **Opened:** 4 days ago by catanzaro
2025-02-17 16:16:27 <@nirik:matrix.scrye.com> !infra 12112
2025-02-17 16:16:28 <@zodbot:fedora.im> **Usage:** !infra <subcommand> [...]
2025-02-17 16:16:28 <@zodbot:fedora.im> ● status - get a list of the ongoing and planned outages
2025-02-17 16:16:28 <@zodbot:fedora.im> ● oncall <subcommand> [...] - oncall
2025-02-17 16:16:28 <@zodbot:fedora.im> 
2025-02-17 16:16:35 <@nirik:matrix.scrye.com> !ticket 12112
2025-02-17 16:16:36 <@zodbot:fedora.im> ● **Assignee:** patrikp
2025-02-17 16:16:36 <@zodbot:fedora.im> **fedora-infrastructure #12112** (https://pagure.io/fedora-infrastructure/issue/12112):**The process to update the OpenH264 repos is broken**
2025-02-17 16:16:36 <@zodbot:fedora.im> 
2025-02-17 16:16:36 <@zodbot:fedora.im> ● **Opened:** 6 months ago by abompard
2025-02-17 16:16:36 <@zodbot:fedora.im> ● **Last Updated:** 2 days ago
2025-02-17 16:16:50 <@jnsamyak:matrix.org> I'm looking in the ticket but patrikp: what's the update on the compose, do we have any progress on ithat? Is there anything that blocking you? 
2025-02-17 16:18:29 <@smilner:fedora.im> Or any help needed?
2025-02-17 16:18:40 <@patrikp:matrix.org> I have a draft for the SOP about how we want to do it but I haven't sent the tarball to Cisco yet. Let me assign these new ones to myself as well...
2025-02-17 16:18:40 <@patrikp:matrix.org> Question about Michael Catanzaro's comment in the issue that I linked. Looks like the github issue is still open but a PR fixing it was merged? Not sure what to make of:
2025-02-17 16:18:40 <@patrikp:matrix.org> "We should just skip this release entirely. Upstream is going to have to create a corrected release."
2025-02-17 16:20:13 <@nirik:matrix.scrye.com> but then found there was a one line patch to fix it.
2025-02-17 16:20:38 <@nirik:matrix.scrye.com> but I don't know if thats been built yet
2025-02-17 16:20:42 <@nirik:matrix.scrye.com> might ask in ticket?
2025-02-17 16:21:04 <@jnsamyak:matrix.org> for the ticket we need to check for the build because there is nothing for us to do yet if i get that correctly 
2025-02-17 16:21:07 <@nirik:matrix.scrye.com> hum, it's not
2025-02-17 16:21:35 <@jnsamyak:matrix.org> looking at the upstream ticket the version update etc 
2025-02-17 16:21:47 <@nirik:matrix.scrye.com> oh, I see... we should ask Kalev Lember if he can build it.
2025-02-17 16:22:10 <@nirik:matrix.scrye.com> but in any case we could send the 2.5.0 version for rawhide to cisco... unless 2.6.0 is going to be along pretty fast.
2025-02-17 16:22:40 <@jnsamyak:matrix.org> 
2025-02-17 16:22:40 <@jnsamyak:matrix.org> >  draft for the SOP about how we want to do it
2025-02-17 16:22:40 <@jnsamyak:matrix.org> Do we have the tarballs ready for the previous compose ticket? 
2025-02-17 16:23:04 <@jnsamyak:matrix.org> with the proces you want to propose for sop?
2025-02-17 16:23:32 <@patrikp:matrix.org> 
2025-02-17 16:23:32 <@patrikp:matrix.org> https://pagure.io/releng/issue/12585
2025-02-17 16:23:32 <@patrikp:matrix.org> There's already a ticket for 2.6.0:
2025-02-17 16:23:32 <@patrikp:matrix.org> But I don't see the builds.
2025-02-17 16:24:35 <@patrikp:matrix.org> What about asking Kalev for 2.6.0 builds the relevant builds for Fedora versions and sending that?
2025-02-17 16:24:57 <@nirik:matrix.scrye.com> sure, we could. If he can't do them I guess I could...
2025-02-17 16:25:02 <@patrikp:matrix.org> https://koji.fedoraproject.org/koji/packageinfo?packageID=21431
2025-02-17 16:25:41 <@nirik:matrix.scrye.com> yeah, he did a rawhide build a few days ago... (of 2.5.0)
2025-02-17 16:25:50 <@patrikp:matrix.org> I suppose since F40 is still supported that would be 2.6.0 builds for F40, F41, F42 and F43?
2025-02-17 16:26:14 <@patrikp:matrix.org> Not sure if you want to bring up any other tickets but we have 4 minutes until the retro, time check.
2025-02-17 16:27:27 <@patrikp:matrix.org> OK let me tag Kalev in the ticket if this sounds like a reasonable plan?
2025-02-17 16:27:30 <@nirik:matrix.scrye.com> yes, and epel.
2025-02-17 16:28:02 <@patrikp:matrix.org> So: F40, F41, F42, F43, EPEL9 and EPEL10? (Or 10.0?)
2025-02-17 16:28:07 <@nirik:matrix.scrye.com> +1 to the plan
2025-02-17 16:28:26 <@patrikp:matrix.org> And EPEL8 too?
2025-02-17 16:28:29 <@nirik:matrix.scrye.com> I'm not sure how big a change it is. we could also get a 2.6.0 for rawhide first
2025-02-17 16:28:36 <@carlwgeorge:fedora.im> iirc the plan for epel10 openh264 was to have one build we tagged multiple times for each minor version
2025-02-17 16:28:59 <@carlwgeorge:fedora.im> https://pagure.io/releng/issue/12334
2025-02-17 16:29:11 <@jnsamyak:matrix.org> patrikp: https://pagure.io/releng/issue/12466
2025-02-17 16:29:17 <@nirik:matrix.scrye.com> right no epel8.
2025-02-17 16:30:41 <@patrikp:matrix.org> OK, I suppose it's time for the retrospective then.
2025-02-17 16:31:06 <@patrikp:matrix.org> !topic Branch F43 from Rawhide - Mass Branching Retrospective
2025-02-17 16:32:01 <@jnsamyak:matrix.org> Okay so I'll let first 5 minute for everyone to lookinto this hackmd document: https://hackmd.io/@2se5GAvIQGSWX9IGCuz2Tw/massbranchingretrof42/edit
2025-02-17 16:32:01 <@jnsamyak:matrix.org> And, then let me know if you want me to add something or you can add something directly by yourself if you feel like I missed something?
2025-02-17 16:32:01 <@jnsamyak:matrix.org> 
2025-02-17 16:32:25 <@jnsamyak:matrix.org> And then I'll start to discuss each topic 1:1 in 5-10 min frame
2025-02-17 16:33:14 <@patrikp:matrix.org> I don't have access to the doc.
2025-02-17 16:33:41 <@jnsamyak:matrix.org> are you a signed in user?
2025-02-17 16:33:59 <@jnsamyak:matrix.org> read is open for everyone
2025-02-17 16:33:59 <@patrikp:matrix.org> No.
2025-02-17 16:34:08 <@smilner:fedora.im> I removed `edit` and can now see it https://hackmd.io/DykaMNJ1TXGjilEOokTp1A
2025-02-17 16:34:59 <@jnsamyak:matrix.org> write is for signed in user - it's easier for us to see who is writing and if they want to say something people can also use [name] before there point in case they want to discuss that issue
2025-02-17 16:35:07 <@jnsamyak:matrix.org> does this work for everyone?
2025-02-17 16:35:54 <@patrikp:matrix.org> Yes, I can see it now.
2025-02-17 16:36:21 <@jnsamyak:matrix.org> Any suggestions before we start?
2025-02-17 16:36:47 <@jnsamyak:matrix.org> we will start in 4 mins, leaving for everyone to add things/seethings if they wants
2025-02-17 16:36:49 <@jnsamyak:matrix.org> etc
2025-02-17 16:37:52 <@patrikp:matrix.org> In the meantime let's choose the next chair?
2025-02-17 16:37:54 <@smilner:fedora.im> seethings ... see things?
2025-02-17 16:38:19 <@jnsamyak:matrix.org> hehe i mean review them :P
2025-02-17 16:38:34 <@patrikp:matrix.org> It's acceptable to seethe during retrospectives.
2025-02-17 16:38:58 <@jnsamyak:matrix.org> i can do that
2025-02-17 16:39:15 <@patrikp:matrix.org> !info Next chair Feb 24: Samyak
2025-02-17 16:41:03 <@jnsamyak:matrix.org> okay 
2025-02-17 16:41:38 <@jnsamyak:matrix.org> # Fedora 42 Mass Branching Retrospective
2025-02-17 16:41:45 <@jnsamyak:matrix.org> ## 🔍 Discussion Points
2025-02-17 16:41:55 <@jnsamyak:matrix.org> !topic 🔍 Discussion Points
2025-02-17 16:42:07 <@jnsamyak:matrix.org> !info 1️⃣ Signing Process Considerations
2025-02-17 16:43:02 <@jnsamyak:matrix.org> So before branching we have to sing all the rawhide packages with the new rawhide +1 key, that means before branching the rawhide has everything signed with 42 keys and just before the branching we need to sign it with f43 keys
2025-02-17 16:44:00 <@jnsamyak:matrix.org> we usually get the packages with unsigned sigul script and then sign them manually using robosignatory 
2025-02-17 16:44:34 <@jnsamyak:matrix.org> this time the first new rawhide compose failed because we had some unsigned packages
2025-02-17 16:45:08 <@jnsamyak:matrix.org> Is there anything better we can do to avoid those cases?
2025-02-17 16:47:08 <@nirik:matrix.scrye.com> The packages that were not signed were all ones where sigul gets stuck... I guess we just need to make sure we iterate over it until there's 0 listed
2025-02-17 16:47:25 <@nirik:matrix.scrye.com> alternately, it would sure be helpfull if robosignatory had ability to do this.
2025-02-17 16:47:36 <@jnsamyak:matrix.org> One thing I got to learn this time to also check things from the pungi logs and get the list of packages from there and not just keep waiting for the sigul script to run
2025-02-17 16:48:02 <@jnsamyak:matrix.org> is there anything we can do nirik to automate this or fasten the script somehow, anyideas?
2025-02-17 16:48:33 <@nirik:matrix.scrye.com> Well, we could ask Aurélien B if it could be implemented in robosignatory...
2025-02-17 16:49:12 <@nirik:matrix.scrye.com> https://pagure.io/robosignatory
2025-02-17 16:49:53 <@jnsamyak:matrix.org> Okay action item is to brainstorm with Aurélien B on this, and get back to this
2025-02-17 16:50:13 <@jnsamyak:matrix.org> Anyone who can help in capturing these in the document? OR i can do that after the call as well
2025-02-17 16:50:19 <@abompard:fedora.im> Happy to discuss it when you can :-)
2025-02-17 16:50:34 <@jnsamyak:matrix.org> thanks++
2025-02-17 16:50:35 <@zodbot:fedora.im> Sorry, but Fedora Accounts user 'thanks' does not exist
2025-02-17 16:50:59 <@jnsamyak:matrix.org> Okay let's move on to the second topic
2025-02-17 16:51:41 <@nirik:matrix.scrye.com> Not sure how it would be implemented, but perhaps a new command that says 'check all packages in this tag, any that aren't signed queue a message to sign them at lower priority'
2025-02-17 16:51:59 <@nirik:matrix.scrye.com> there's a number of tickets in that project that could be closed... to do with module signing. ;)
2025-02-17 16:52:10 <@nirik:matrix.scrye.com> but yeah, lets move on.
2025-02-17 16:52:32 <@jnsamyak:matrix.org> !info 2️⃣ Transition from Toddlers to Poddlers
2025-02-17 16:52:39 <@jnsamyak:matrix.org> - Update documentation reflecting SLA changes.
2025-02-17 16:52:44 <@jnsamyak:matrix.org> - Restart poddlers playbook and pod from OpenShift.
2025-02-17 16:53:19 <@jnsamyak:matrix.org> These things were missing from the documentation since these got activate after last release
2025-02-17 16:53:29 <@jnsamyak:matrix.org> so we need to make sure to have these changes in the documentation
2025-02-17 16:53:36 <@jnsamyak:matrix.org> I can take this up as an action item
2025-02-17 16:53:47 <@jnsamyak:matrix.org> Anything else regarding this?
2025-02-17 16:54:02 <@james:fedora.im> FWIW I wrote code to do the first half of this for CentOS (outputs a list of all packages that are unsigned in a tag).
2025-02-17 16:54:04 <@nirik:matrix.scrye.com> Cool. If you can, might be good to look for 'toddlers' in any other docs that might need updating too
2025-02-17 16:54:22 <@jnsamyak:matrix.org> sure
2025-02-17 16:55:14 <@nirik:matrix.scrye.com> yeah, we have a script to do that already, but then you have to feed them in a loop one at a time to robosignatory. It would be nice to just say 'robosignatory mass-sign f43 f43-key' and have it do it. ;)
2025-02-17 16:55:23 <@jnsamyak:matrix.org> Let's move on to the next item then
2025-02-17 16:55:37 <@jnsamyak:matrix.org> We can continue the retro in the next meeting I see there are lot of points 
2025-02-17 16:55:46 <@jnsamyak:matrix.org> !info 3️⃣ Koji Builders and Ansible Changes
2025-02-17 16:56:00 <@jnsamyak:matrix.org>   - Introduce a separate Ansible commit for manual updates.
2025-02-17 16:56:00 <@jnsamyak:matrix.org> - **Automation Ideas:** Identify repetitive tasks that can be automated.
2025-02-17 16:56:00 <@jnsamyak:matrix.org>   - Run playbook first, then apply manual changes. (is this correct way?)
2025-02-17 16:56:00 <@jnsamyak:matrix.org>   - Leave as is.
2025-02-17 16:56:00 <@jnsamyak:matrix.org> - Options to handle Koji changes in Ansible:
2025-02-17 16:56:00 <@jnsamyak:matrix.org> - Avoid running `koji-hub.yaml` playbook after turning off Koji builders to prevent overwriting manual changes.
2025-02-17 16:56:11 <@jnsamyak:matrix.org> These are the things i captured 
2025-02-17 16:57:22 <@nirik:matrix.scrye.com> so, a seperate commit that just added the blocking might be the way to go. Then we can run the playbook and it will do the right thing. we will need then to revert it and re-run the playbook to open things up tho
2025-02-17 16:58:08 <@jnsamyak:matrix.org> okay we need to then define an ansible of disabling it from the iptables 
2025-02-17 16:59:04 <@nirik:matrix.scrye.com> yeah, should just be 2 lines... but it makes it more complex having to commit, run, revert, run
2025-02-17 16:59:27 <@nirik:matrix.scrye.com> what changes are made then to koji...
2025-02-17 16:59:42 <@jnsamyak:matrix.org> but do you think this will be the right way Run playbook first, then apply manual changes. (is this correct way?)
2025-02-17 17:00:24 <@patrikp:matrix.org> After this point we are at time.
2025-02-17 17:01:02 <@jnsamyak:matrix.org> atleast this way we can be sure to not let it go online because of a playbook
2025-02-17 17:01:16 <@jnsamyak:matrix.org> Should we continue this next time then?
2025-02-17 17:01:51 <@nirik:matrix.scrye.com> ok, so either way... yeah, we could
2025-02-17 17:01:53 <@nirik:matrix.scrye.com> time flies.
2025-02-17 17:02:21 <@patrikp:matrix.org> Let's continue next week.
2025-02-17 17:02:26 <@patrikp:matrix.org> !endmeeting
2025-02-17 17:02:26 <@patrikp:matrix.org> !info Thank you all for coming!