#meeting-3:fedoraproject.org log

<@jnsamyak:matrix.org>

15:06:03

!startmeeting RELENG (2025-09-15)

<@meetbot:fedora.im>

15:06:04

Meeting started at 2025-09-15 15:06:03 UTC

<@meetbot:fedora.im>

15:06:05

The Meeting name is 'RELENG (2025-09-15)'

<@jnsamyak:matrix.org>

15:06:09

!meetingname releng

<@meetbot:fedora.im>

15:06:10

The Meeting Name is now releng

<@jnsamyak:matrix.org>

15:06:12

!chair nirik jnsamyak patrikp amedvede

<@jnsamyak:matrix.org>

15:06:18

!info Meeting is 60 minutes MAX. At the end of 60, it STOPS.

<@jnsamyak:matrix.org>

15:06:26

!info Agenda is at https://hackmd.io/vm6biLBcTYKtkQUH5kQkmw.

<@jnsamyak:matrix.org>

15:06:26

<@jnsamyak:matrix.org>

15:06:31

!topic Init process.

<@jnsamyak:matrix.org>

15:06:35

<@jnsamyak:matrix.org>

15:06:35

Do we have anything for the init? Any blockers/tasks/issues/requests/features that need releng intervention?

<@jnsamyak:matrix.org>

15:06:49

Okay we are all set, I forgot I had to chair today

<@jnsamyak:matrix.org>

15:06:58

Hello again!

<@nirik:matrix.scrye.com>

15:07:13

morning again

<@patrikp:matrix.org>

15:07:18

👋

<@jnsamyak:matrix.org>

15:07:29

What do we have for init?

<@amedvede:fedora.im>

15:07:44

guten morgen

<@amedvede:fedora.im>

15:08:35

nothing from me

<@jnsamyak:matrix.org>

15:09:20

releng #12934 and #12935

<@jnsamyak:matrix.org>

15:09:20

<@jnsamyak:matrix.org>

15:09:20

I have one thing to bring up - reviews are open for the tickets:

<@naraiank:fedora.im>

15:09:46

!hi

<@zodbot:fedora.im>

15:09:48

Naraian K (naraiank) - he / him / his

<@jnsamyak:matrix.org>

15:10:09

both Anton Medvedev and patrikp are done with there investigations about where to move the scripts, and remove if needed. I'm doing my reviews gradualy but dealing with beta release first.

<@jnsamyak:matrix.org>

15:10:25

both Anton Medvedev and patrikp are done with their investigations about where to move the scripts, and remove if needed. I'm doing my reviews gradually, but dealing with the beta release first.

<@jnsamyak:matrix.org>

15:10:43

cf: https://pagure.io/releng/issue/12935 and https://pagure.io/releng/issue/12934

<@jnsamyak:matrix.org>

15:11:32

Once, these are closed, I'll start with moving and closing https://pagure.io/releng/issue/12925 in order to start working on https://pagure.io/releng/issue/12931

<@nirik:matrix.scrye.com>

15:11:46

I had no time to look friday. I can try again today...

<@patrikp:matrix.org>

15:11:56

I left some comments about the structure on your ticket. I was also wondering if it would be worthwhile having a `tests` subdirectory. At this point we only have the one unit test file and it doesn't work, perhaps we can improve that over time.

<@jnsamyak:matrix.org>

15:11:58

Yeah - same!

<@nirik:matrix.scrye.com>

15:12:04

but it's also fine if we just do it, we can always adjust more later if something ends up in a non ideal place.

<@jnsamyak:matrix.org>

15:12:22

let me see, unit tests which script?

<@patrikp:matrix.org>

15:12:57

Right now we only have `mass-rebuild-test.py`.

<@jnsamyak:matrix.org>

15:13:14

I really want to atleast do one review so it always two pair of eyes rather than one, I will finish it by wednesday if all goes clear tomorrow :D

<@amedvede:fedora.im>

15:13:30

I took a look on sheet that you sent jnsamyak , looks correct

<@jnsamyak:matrix.org>

15:13:38

Anton Medvedev: thanks man!

<@amedvede:fedora.im>

15:14:36

I think after you finish your review, it would be nice if all of us will jump to meet to just get through list and confirm opinion on each script

<@patrikp:matrix.org>

15:14:54

And in an ideal scenario we would have tests for all scripts that are critical for the release process.

<@jnsamyak:matrix.org>

15:15:11

<@jnsamyak:matrix.org>

15:15:11

│ ├│ ├── tests/

<@jnsamyak:matrix.org>

15:15:11

│ ├── mass-rebuilds/

<@jnsamyak:matrix.org>

15:15:11

release_process/

<@jnsamyak:matrix.org>

15:15:11

Right if required -

<@jnsamyak:matrix.org>

15:15:11

we can move it like that

<@jnsamyak:matrix.org>

15:15:11

<@nirik:matrix.scrye.com>

15:15:21

yeah, although some things are hard to test.

<@jnsamyak:matrix.org>

15:15:30

correct!

<@jnsamyak:matrix.org>

15:15:46

that is the goal :)

<@jnsamyak:matrix.org>

15:16:11

I mentioned it before once the review happens, we can always do a hour call to settle on anf finalise the moving details

<@patrikp:matrix.org>

15:16:17

Some are but some are not. We may at least go for the low hanging fruit. But I'm just brainstorming here.

<@jnsamyak:matrix.org>

15:16:28

I mentioned it before, once the review happens, we can always do an hour call to settle on and finalise the moving details

<@nirik:matrix.scrye.com>

15:16:42

sure

<@jnsamyak:matrix.org>

15:17:04

sure but that can be another independent post migration work to bring up

<@patrikp:matrix.org>

15:17:14

Yes, just wanted to mention it.

<@jnsamyak:matrix.org>

15:17:18

Alright anything else?

<@patrikp:matrix.org>

15:18:02

Is there any particular reason why `release_process` uses an underscore but nothing else does?

<@amedvede:fedora.im>

15:18:33

if we want to cover scripts by tests, the best point to start is from scripts connected to SOP's imho

<@jnsamyak:matrix.org>

15:18:42

it was a typo

<@jnsamyak:matrix.org>

15:18:45

I'll fix it

<@jnsamyak:matrix.org>

15:18:52

adding a comment wait

<@jnsamyak:matrix.org>

15:21:17

Sure, added a comment.

<@jnsamyak:matrix.org>

15:21:23

Anything ekse?

<@jnsamyak:matrix.org>

15:21:27

Anything else?

<@patrikp:matrix.org>

15:21:31

Nothing from me.

<@patrikp:matrix.org>

15:22:04

Will we do the rest of the retrospective today? Seems like all issues are assigned.

<@jnsamyak:matrix.org>

15:22:13

yes

<@jnsamyak:matrix.org>

15:22:31

<@jnsamyak:matrix.org>

15:22:31

> Seems like all issues are assigned.

<@jnsamyak:matrix.org>

15:22:31

in what terms?

<@patrikp:matrix.org>

15:25:06

In that I don't see any new issue on the issue tracker that is not assigned.

<@jnsamyak:matrix.org>

15:25:13

if you mean from new issues, we have a backlog 102 issues pick which are open

<@jnsamyak:matrix.org>

15:25:23

pick it up groom it

<@jnsamyak:matrix.org>

15:25:32

start working

<@jnsamyak:matrix.org>

15:25:49

oh you meant in term of issues being discussed?

<@jnsamyak:matrix.org>

15:25:57

today in meeting?

<@patrikp:matrix.org>

15:26:09

Yeah, I know that we have a backlog but I don't see any new ones, that's all I meant.

<@jnsamyak:matrix.org>

15:26:17

sure

<@patrikp:matrix.org>

15:26:21

Didn't have some deep insight.

<@jnsamyak:matrix.org>

15:26:21

!topic Scheduled actions coming up in the next week.

<@jnsamyak:matrix.org>

15:26:27

!info Here we list/discuss anything about items that are due to be done in the next week.

<@jnsamyak:matrix.org>

15:26:33

<@jnsamyak:matrix.org>

15:26:33

!link https://fedorapeople.org/groups/schedule/f-43/f-43-releng-tasks.html

<@jnsamyak:matrix.org>

15:26:43

F43 Beta Release is scheduled for tomorrow

<@jnsamyak:matrix.org>

15:27:40

Most of the things are done, stage release and checksum synced; Torrents are also

<@jnsamyak:matrix.org>

15:27:45

done

<@nirik:matrix.scrye.com>

15:28:04

excellent

<@amedvede:fedora.im>

15:28:43

So there are left only run a playbooks and post beta cleanup?

<@jnsamyak:matrix.org>

15:29:07

F40 torrents were still there - so i cleaned up those

<@nirik:matrix.scrye.com>

15:29:45

so... there's something I see thats not there, but not sure we want to add it.

<@jnsamyak:matrix.org>

15:29:56

playbooks were done last friday and releng lift was lifted, I did this discussion on releng there

<@jnsamyak:matrix.org>

15:30:06

playbooks were done last friday and releng lift was lifted, I did this discussion on releng channel there

<@jnsamyak:matrix.org>

15:31:01

shoot, I see there was -t roll out changes from bodhi, torrents creation mention was missing.

<@jnsamyak:matrix.org>

15:31:01

> so... there's something I see thats not there, but not sure we want to add it.

<@jnsamyak:matrix.org>

15:31:01

<@jnsamyak:matrix.org>

15:31:04

anything else?

<@jnsamyak:matrix.org>

15:31:47

Can we look into adam's ticket and review

<@jnsamyak:matrix.org>

15:31:47

https://pagure.io/fedora-infra/ansible/pull-request/2841 and https://pagure.io/infra-docs-fpo/pull-request/410

<@nirik:matrix.scrye.com>

15:32:20

The night before a release we do a 'bit flip' and make permissions readable on the various release dirs on the mirrors. We do this in advance of the release because if we just switched it right at release time many non tier1 mirrors may not have synced the content and all of them will hit at once with all the users. So we do it the night before so mirrors have a chance to sync up... but we don't want to really advertise it or tell users it's there until the actual release...

<@jnsamyak:matrix.org>

15:32:53

so 12 hours before the release?

<@patrikp:matrix.org>

15:33:28

What is it? Or what is not where?

<@jnsamyak:matrix.org>

15:33:34

yeah but if it is not readable people can't access it right?

<@nirik:matrix.scrye.com>

15:33:34

yeah, something like that. In the past I have done it before going to bed... but someone else could do it their morning.

<@nirik:matrix.scrye.com>

15:34:09

it will be readable? not sure I follow

<@jnsamyak:matrix.org>

15:34:19

done as part of stage-release script

<@jnsamyak:matrix.org>

15:34:19

The dl.alt.fpo has staged rc that are released

<@nirik:matrix.scrye.com>

15:35:07

yes, with mode 750 on the directories so only tier1 mirrors can read them

<@nirik:matrix.scrye.com>

15:35:14

or see them

<@jnsamyak:matrix.org>

15:35:40

I mean, in terms of access, even if we don't make it 750, and it is there in SOP public viewer should not be able to see it right but internally it can pick up?

<@nirik:matrix.scrye.com>

15:35:42

drwxr-x---. 15 263 263 4096 Sep 12 17:59 43_Beta-1.3/

<@jnsamyak:matrix.org>

15:35:53

and we can open it at the time of release

<@nirik:matrix.scrye.com>

15:36:03

but then we have the problem I described.

<@jnsamyak:matrix.org>

15:36:04

and we can open it at the time of release (I mean 12 hour before)

<@jnsamyak:matrix.org>

15:36:27

yeah :/

<@nirik:matrix.scrye.com>

15:36:38

a mass of NON tier1 mirrors will try and sync all the content that suddenly appeared ALONG with all the users who read the announcement.

<@jnsamyak:matrix.org>

15:37:13

I think we can just add a generic note to make it accessible at 12 hour before? Not exactly mentioning paths?

<@nirik:matrix.scrye.com>

15:37:23

so to mitigate that we have opened it before to allow for mirrors (who have crons and automation) to get it before folks see an announcement

<@nirik:matrix.scrye.com>

15:37:52

sure...

<@jnsamyak:matrix.org>

15:38:35

yeah or just remember, I know I had to do it tomorrow morning, I remember me and you asks this every cycle at the time of your night or my morning whoever want's do it just do it and get it done with it

<@nirik:matrix.scrye.com>

15:38:36

I guess if people start pointing at it we can try and do something else.

<@nirik:matrix.scrye.com>

15:39:10

sure.

<@patrikp:matrix.org>

15:39:13

I understand what you guys are saying but I'm still not sure I get what you meant in the beginning Kevin. Something that is not there and not sure if we want to add it. Do you mean a step in the SOP? And since people can read the docs they would know?

<@nirik:matrix.scrye.com>

15:39:32

yes, exactly.

<@nirik:matrix.scrye.com>

15:39:56

we could add it and adjust if it becomes a problem I guess

<@patrikp:matrix.org>

15:40:15

They can read the release engineering Matrix channel too though. I think all should be in SOPs.

<@jnsamyak:matrix.org>

15:41:17

Let's get in so we can see what happens.

<@jnsamyak:matrix.org>

15:41:22

Let's get in sop we can see what happens.

<@jnsamyak:matrix.org>

15:41:37

anything else for release process?

<@jnsamyak:matrix.org>

15:41:40

Or we can move on?

<@patrikp:matrix.org>

15:42:23

Retro time!

<@jnsamyak:matrix.org>

15:43:07

before that let's just quickly choose chair so we dont loose it in the end

<@jnsamyak:matrix.org>

15:43:22

!topic Choose next chair.

<@jnsamyak:matrix.org>

15:43:37

!info Next chair September 22nd: ??

<@patrikp:matrix.org>

15:43:40

🖐️

<@jnsamyak:matrix.org>

15:43:45

Any volunteers?

<@jnsamyak:matrix.org>

15:43:47

Okay

<@jnsamyak:matrix.org>

15:43:53

you are first :D

<@jnsamyak:matrix.org>

15:44:06

!info Next chair September 22nd: patrikp

<@jnsamyak:matrix.org>

15:44:20

!topic Fedora 43 Retrospective part 2

<@jnsamyak:matrix.org>

15:44:50

I think we covered on Communication with Website Team and OpenH264 Repository Missing

<@jnsamyak:matrix.org>

15:46:04

For the openh264 as an action item I did update the documentation: https://docs.fedoraproject.org/en-US/infra/release_guide/sop_mass_branching/#_openh264_repository_update_after_branching But not sure if we can close it now, it has been unresponsive for couple of days

<@jnsamyak:matrix.org>

15:46:19

The ticket: https://pagure.io/releng/issue/12880

<@jnsamyak:matrix.org>

15:48:51

Any thoughts? Or should we jump into the second item?

<@patrikp:matrix.org>

15:49:25

Looks good to me and I think the ticket can be closed.

<@jnsamyak:matrix.org>

15:49:48

+1 okay

<@jnsamyak:matrix.org>

15:49:57

!info MirrorManager Updates

<@jnsamyak:matrix.org>

15:50:07

Needed to manually set ctime=0, update the fullfiletimelist, and trigger the usual mirror updates for the changes to reflect.

<@jnsamyak:matrix.org>

15:51:09

Because this time the mirrormanager did not pick up with the repository updates when we did those

<@jnsamyak:matrix.org>

15:51:28

<@jnsamyak:matrix.org>

15:51:28

My Recommendation: Document this as a standard post-branching step.

<@jnsamyak:matrix.org>

15:51:56

What are your opinions?

<@patrikp:matrix.org>

15:52:13

+1, for sure it should be in the docs.

<@amedvede:fedora.im>

15:52:50

+1, yep lest update teh doc

<@amedvede:fedora.im>

15:53:17

should we create ticket on it, or quick PR will be enough?

<@nirik:matrix.scrye.com>

15:53:31

yeah. +1

<@jnsamyak:matrix.org>

15:53:34

quick pr for this is okay there is mass branching ticket is on

<@jnsamyak:matrix.org>

15:53:40

Post-Branch Monitoring

<@jnsamyak:matrix.org>

15:54:02

- Monitor the branched compose status.

<@jnsamyak:matrix.org>

15:54:02

- Fire off the first compose as soon as the branch opens.

<@jnsamyak:matrix.org>

15:54:02

We need better-structured post-branch steps:

<@jnsamyak:matrix.org>

15:54:14

This will help identify any repository gaps or configuration mismatches earlier.

<@jnsamyak:matrix.org>

15:54:21

We need to add it to post branching work

<@jnsamyak:matrix.org>

15:55:14

So, one more thing to add is documentation.

<@jnsamyak:matrix.org>

15:56:18

So, one more thing to add to documentation.

<@jnsamyak:matrix.org>

15:56:36

this sums up the retro stuff, but if folks want to bring something else in terms of retro

<@jnsamyak:matrix.org>

15:56:53

please bring up

<@patrikp:matrix.org>

15:59:17

I was on PTO so nothing to add but I would like to say that I found the retro useful and am glad the mass branching itself went relatively smoothly. Good job.

<@patrikp:matrix.org>

15:59:57

And looks like we're at time, thanks for running the meeting.

<@jnsamyak:matrix.org>

16:00:49

My network just went-out agian

<@jnsamyak:matrix.org>

16:01:12

Sigh it has been an irritating day with Internet

<@nirik:matrix.scrye.com>

16:01:16

yeah, retros are good...

<@jnsamyak:matrix.org>

16:01:32

!info thank you everyone for coming

<@jnsamyak:matrix.org>

16:01:44

Thanks patrikp nirik

<@jnsamyak:matrix.org>

16:01:53

!endmeeting