<@jnsamyak:matrix.org>
16:01:08
!startmeeting RELENG (2024-03-26)
<@meetbot:fedora.im>
16:01:11
Meeting started at 2024-03-26 16:01:08 UTC
<@meetbot:fedora.im>
16:01:11
The Meeting name is 'RELENG (2024-03-26)'
<@jnsamyak:matrix.org>
16:01:17
!meetingname releng
<@jnsamyak:matrix.org>
16:01:24
!chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec jnsamyak patrikp
<@nirik:matrix.scrye.com>
16:01:25
morning.
<@jnsamyak:matrix.org>
16:01:29
!info Meeting is 60 minutes MAX. At the end of 60, it stop
<@jnsamyak:matrix.org>
16:01:36
!info agenda is at https://hackmd.io/vm6biLBcTYKtkQUH5kQkmw
<@jnsamyak:matrix.org>
16:01:41
!topic init process
<@nirik:matrix.scrye.com>
16:01:47
happy f40 beta release morning.
<@jnsamyak:matrix.org>
16:02:01
Hello everyone, release day yayyyy 🎉
<@jnsamyak:matrix.org>
16:02:13
how's everyone doing today?
<@jnsamyak:matrix.org>
16:02:33
!info Fedora Linux 40 Beta has been released today at 14:00 UTC!
<@nirik:matrix.scrye.com>
16:03:04
so far so good
<@jnsamyak:matrix.org>
16:04:02
nice I have just one thing for init process, we have only one week for the final freeze to start; so anything that is left post beta freeze that needs to go on this is the time :/
<@nirik:matrix.scrye.com>
16:04:33
yep. I also mailed out a note about that to devel-announce to remind everyone
<@jnsamyak:matrix.org>
16:04:43
it happened because the beta release target got shifted, and fesco agreed on the final freeze and release shouldn't get affted due to beta
<@jnsamyak:matrix.org>
16:05:04
nirik: noice, thanks as usual you are awesome!
<@jnsamyak:matrix.org>
16:05:16
Anything else, or can we move on to our ticket dumps?
<@nirik:matrix.scrye.com>
16:05:28
sure, we can move to tickets.
<@jnsamyak:matrix.org>
16:06:01
!topic Tickets needing attention
<@jnsamyak:matrix.org>
16:06:08
!info this is the time where you can bring releng tickets to discuss if there are blockers etc
<@jnsamyak:matrix.org>
16:06:34
!releng 12039
<@zodbot:fedora.im>
16:06:35
**releng #12039** (https://pagure.io/releng/issue/12039):**libva-2.21.0-2.fc40 doesn't appear in repo**
● **Opened:** 7 hours ago by kwizart
● **Last Updated:** Never
● **Assignee:** Not Assigned
<@jnsamyak:matrix.org>
16:07:06
nirik: I want to work on this I know it seems simple enough but i need to understand quick debugging terminology for this
<@nirik:matrix.scrye.com>
16:07:50
So... we have a script to look for this...
<@jnsamyak:matrix.org>
16:07:51
so i did ` koji list-history --package=libva`
<@jnsamyak:matrix.org>
16:07:56
to debug what happened there
<@nirik:matrix.scrye.com>
16:08:20
check-latest-build.py f40
<@jnsamyak:matrix.org>
16:08:35
oh
<@nirik:matrix.scrye.com>
16:09:26
what happened is likely what often happens: Someone submits an update during freeze, it is pending stable, but the freeze blocks it from going stable. Then they submit a new update and forget about the old one, and then when the freeze is over, both of them go stable. Which one is in the compose depends on which one was tagged last. If it's the older one... then this happens.
<@jnsamyak:matrix.org>
16:09:50
aha
<@jnsamyak:matrix.org>
16:10:05
by looking at the history libva-2.21.0-3.fc40 I see a new one
<@nirik:matrix.scrye.com>
16:10:22
There's also other cases similar, like someone submits an update with a,b,c in it, someone then later submits a newer c... and the other one goes stable after the single one
<@jnsamyak:matrix.org>
16:10:52
so what's the best approach to it? figure out what's needed and untag rest updates?
<@nirik:matrix.scrye.com>
16:10:55
anyhow, we can fix this by running the above script and looking at the output then retagging things that we need to fix.
<@jnsamyak:matrix.org>
16:11:01
from stable? (f40 tag i mean)
<@nirik:matrix.scrye.com>
16:11:23
theres often some broken ones we don't want to fix... like:
<@nirik:matrix.scrye.com>
16:11:30
calendar-1.37-6.20240209cvs.fc40 < calendar-1.37-7.20211220cvs.fc37
<@nirik:matrix.scrye.com>
16:11:49
The old f37 update there is 'newer' than the f40 one... but we don't want that one.
<@jnsamyak:matrix.org>
16:12:23
makes sense need to have eyes and ears open while tagging stuff in and out
<@nirik:matrix.scrye.com>
16:12:31
so, process would be: run script, look at output, saneify it, and then retag the newer ones for the ones we want to fix.
<@jnsamyak:matrix.org>
16:13:06
I'll try out
<@nirik:matrix.scrye.com>
16:13:28
sounds good. :) Let me know if you have any questions.
<@nirik:matrix.scrye.com>
16:13:47
and... we should put this in our release schedule. Do it right after the unfrozen updates push
<@jnsamyak:matrix.org>
16:14:15
sure thing, once I get an idea, I'll try to put a draft in lay man terms
<@jnsamyak:matrix.org>
16:14:27
Next one, 🥁
<@jnsamyak:matrix.org>
16:14:34
!releng 12037
<@zodbot:fedora.im>
16:14:36
**releng #12037** (https://pagure.io/releng/issue/12037):**Fix F40 flatpak build tags**
● **Opened:** 22 hours ago by yselkowitz
● **Last Updated:** 20 hours ago
● **Assignee:** Not Assigned
<@nirik:matrix.scrye.com>
16:16:00
yeah, we made adjustments on the fly to the f39 ones when we were bringing it up... we need to make those to f40 and then make sure they are set moving forward.
<@nirik:matrix.scrye.com>
16:16:15
you want to take this one? or want me to?
<@jnsamyak:matrix.org>
16:16:15
we already have a pr for this
<@jnsamyak:matrix.org>
16:16:25
https://pagure.io/releng/pull-request/12038
<@nirik:matrix.scrye.com>
16:16:56
yeah, for f41+... we still need to manually fix f40 tho
<@jnsamyak:matrix.org>
16:17:55
so we want to just review I guess for f40+ but need to make changes for f40 for sure, can you take this one? I'm off for next few days :/
<@nirik:matrix.scrye.com>
16:18:25
sure thing. feel free to assign to me
<@jnsamyak:matrix.org>
16:18:51
thanks, one quick request tho, can you add those commands it will be easier for me to review the PR for sure
<@jnsamyak:matrix.org>
16:18:57
12308 sold to you
<@nirik:matrix.scrye.com>
16:19:18
you mean the f40 fix commands? yeah, I'll draft up a set of commands and add to the ticket.
<@jnsamyak:matrix.org>
16:19:26
yes
<@jnsamyak:matrix.org>
16:20:18
!releng 12036
<@zodbot:fedora.im>
16:20:19
**releng #12036** (https://pagure.io/releng/issue/12036):**Warning 'Cannot find relevant tag for build' in bodhi**
● **Opened:** a day ago by jplesnik
● **Last Updated:** 7 hours ago
● **Assignee:** Not Assigned
<@jnsamyak:matrix.org>
16:20:45
do we put this for investigation?
<@nirik:matrix.scrye.com>
16:20:46
I guess we just fix these as we find em
<@nirik:matrix.scrye.com>
16:21:37
so I think I have a theory...
<@jnsamyak:matrix.org>
16:21:51
I checked these three stable updates went to stable should we close this for now?
<@jnsamyak:matrix.org>
16:22:02
nirik: shoot!
<@nirik:matrix.scrye.com>
16:22:21
well, there's at least one more to fix (see last comment)
<@jnsamyak:matrix.org>
16:22:34
:) yes
<@nirik:matrix.scrye.com>
16:23:10
I think what happened is that these failed gating for a bit... then when they were passing gating, it was time to stable, so bodhi requested stable... without ever requesting testing.
<@nirik:matrix.scrye.com>
16:23:31
I'll add that to the ticket and see if mattia can think of a way for bodhi to handle that.
<@jnsamyak:matrix.org>
16:23:54
+1
<@jnsamyak:matrix.org>
16:24:16
btw you might have tagged them to f40-updates-testing tag again right? and wait for bodhi to do it's job!
<@nirik:matrix.scrye.com>
16:25:56
right, I just tagged them into testing...
<@nirik:matrix.scrye.com>
16:26:06
and then bodhi was fine with them on the next push.
<@jnsamyak:matrix.org>
16:26:14
yep
<@jnsamyak:matrix.org>
16:26:27
!releng 12032
<@zodbot:fedora.im>
16:26:28
**releng #12032** (https://pagure.io/releng/issue/12032):**The aarch64 minimal synced to the wrong location for releases**
● **Opened:** 4 days ago by pbrobinson
● **Last Updated:** 19 hours ago
● **Assignee:** Not Assigned
<@jnsamyak:matrix.org>
16:26:49
I just read the comment, it's waiting on me to update the sop i guess, I'll assign it to myself!
<@jnsamyak:matrix.org>
16:27:12
Also, we are discussing this at releng channel as well, this has website impact as well!
<@nirik:matrix.scrye.com>
16:27:28
yes, this is tricky.. so don't actually do it yet. ;)
<@jnsamyak:matrix.org>
16:27:36
yeah
<@jnsamyak:matrix.org>
16:27:47
I am reading at both places
<@jnsamyak:matrix.org>
16:27:51
¯\_(ツ)_/¯
<@nirik:matrix.scrye.com>
16:28:01
so, needs more discussion I'd say
<@jnsamyak:matrix.org>
16:28:49
Do you think a devel email will bring more eyes to it?
<@nirik:matrix.scrye.com>
16:29:13
lets continue in ticket for now? then we can bring it up if we can't come to some solution
<@jnsamyak:matrix.org>
16:29:59
that makes sense1
<@jnsamyak:matrix.org>
16:30:03
that makes sense!
<@jnsamyak:matrix.org>
16:30:17
moving on to one more item
<@jnsamyak:matrix.org>
16:30:21
!releng 12030
<@zodbot:fedora.im>
16:30:23
**releng #12030** (https://pagure.io/releng/issue/12030):**adjust torrent_hashes.py to non bittorrent world**
● **Opened:** 4 days ago by kevin
● **Last Updated:** 7 hours ago
● **Assignee:** jnsamyak
<@jnsamyak:matrix.org>
16:30:52
so i have a pr ready to merge this on releng: https://pagure.io/releng/pull-request/12027
<@nirik:matrix.scrye.com>
16:30:53
yeah, the torrent scripts are... old
<@nirik:matrix.scrye.com>
16:31:28
anyhow, yes, I can merge that one.
<@jnsamyak:matrix.org>
16:31:33
I need to check ansible but it should be same ini python script there as well?
<@jnsamyak:matrix.org>
16:32:40
For torrent-hashes.py script, as mentioned it worked locally to generate whitelist when we switched to python-fastbencode with this script: https://pagure.io/releng/issue/12030#comment-901508
<@nirik:matrix.scrye.com>
16:32:46
k, I was confused there.
<@nirik:matrix.scrye.com>
16:32:51
I meant:
<@nirik:matrix.scrye.com>
16:33:01
roles/torrent/files/torrent-generator/torrent-generator
<@nirik:matrix.scrye.com>
16:33:31
that one is the one that looks at all the *.torrent files and makes a .ini file
<@nirik:matrix.scrye.com>
16:33:45
which is used to make the website, etc.
<@jnsamyak:matrix.org>
16:33:53
aha this is another file for websited and stuff
<@jnsamyak:matrix.org>
16:34:31
okay this needs working; I can take a look at it; and will hopefully open a PR next week, or do we want this before that?
<@nirik:matrix.scrye.com>
16:34:56
it's not urgent.. I just hacked up a ini for beta. ;)
<@nirik:matrix.scrye.com>
16:35:05
but we will need it for final.
<@jnsamyak:matrix.org>
16:35:35
thanks; I'll keep this in mind yes ofcourse; need less broken and more testable stuff in f40 for sure that's the agenda
<@jnsamyak:matrix.org>
16:35:36
:D
<@nirik:matrix.scrye.com>
16:36:25
Sorry torrents are so rough... moving from 7 to 9 was a big jump. ;)
<@jnsamyak:matrix.org>
16:36:30
also i need to opena pr for maketorrent script i updated it directly to the machine but never open a pr on ansible,w ill do that so in future we wont loose it
<@nirik:matrix.scrye.com>
16:36:47
yes please do. ;)
<@jnsamyak:matrix.org>
16:36:53
No need of sorry, I actually had some hacking fun that day!
<@jnsamyak:matrix.org>
16:37:07
🕺
<@jnsamyak:matrix.org>
16:37:39
!releng 11673
<@zodbot:fedora.im>
16:37:40
**releng #11673** (https://pagure.io/releng/issue/11673):**Fixes for release-candidate.sh in pungi-fedora**
● **Opened:** 6 months ago by kevin
● **Last Updated:** 5 months ago
● **Assignee:** patrikp
<@nirik:matrix.scrye.com>
16:37:45
👍
<@jnsamyak:matrix.org>
16:38:14
This is for patrikp; if there are any blockers regarding this?
<@nirik:matrix.scrye.com>
16:38:31
I think there was some last round of comments for him...
<@jnsamyak:matrix.org>
16:38:34
we might be needing this to test it out for final
<@nirik:matrix.scrye.com>
16:39:04
yeah. we can try... if something comes up we can always revert. but I think it's in pretty reasonable shape now.
<@jnsamyak:matrix.org>
16:39:08
yes but the pr seems not modified since the comment so i was worried if there are some more clarifications that are needed.
<@nirik:matrix.scrye.com>
16:39:53
dunno, we should ping patrikp when he's next around to look. ;)
<@jnsamyak:matrix.org>
16:40:11
but that's mostly all from my side; do you have some tickets that you want to bring up?
<@nirik:matrix.scrye.com>
16:40:27
lets see...
<@nirik:matrix.scrye.com>
16:40:52
!releng 12001
<@zodbot:fedora.im>
16:40:53
**releng #12001** (https://pagure.io/releng/issue/12001):**Getting Rawhide + 1 keys at least 2 weeks before branching**
● **Opened:** 2 weeks ago by praiskup
● **Last Updated:** 18 hours ago
● **Assignee:** Not Assigned
<@nirik:matrix.scrye.com>
16:41:19
proposing to make the N+1 key eariler... if that looks good we can get it added to the schedule?
<@jnsamyak:matrix.org>
16:41:38
+1 to that i dont see any issues for it
<@jnsamyak:matrix.org>
16:41:56
should i bring it to next fesco meeting?
<@jnsamyak:matrix.org>
16:42:05
or can you since you are the fesco ;) :P
<@nirik:matrix.scrye.com>
16:42:26
naw, I don't think this needs fesco approval, it's a releng thing... but we need to ask it be added to the official schedule...
<@jnsamyak:matrix.org>
16:42:50
yep i meant for the schedule only; I can put my +1 there in the comment
<@nirik:matrix.scrye.com>
16:43:01
I can't recall where that repo is, but we should file a ticket or bug aoife
<@jnsamyak:matrix.org>
16:43:33
I can ping Aoife for the details on that ;)
<@nirik:matrix.scrye.com>
16:43:53
thanks
<@nirik:matrix.scrye.com>
16:44:02
!releng 12028
<@zodbot:fedora.im>
16:44:02
**releng #12028** (https://pagure.io/releng/issue/12028):**Package retirements are broken in rawhide**
● **Opened:** 5 days ago by gotmax23
● **Last Updated:** 4 days ago
● **Assignee:** Not Assigned
<@nirik:matrix.scrye.com>
16:44:39
so, we need to figure out whats going on here...
<@nirik:matrix.scrye.com>
16:44:53
the nightly.sh for rawhide/branched runs the retire script...
<@nirik:matrix.scrye.com>
16:45:07
./releng/scripts/block_retired.py --profile compose_koji
<@nirik:matrix.scrye.com>
16:45:13
but I don't see any output from it.
<@nirik:matrix.scrye.com>
16:45:48
I guess we should run it manually and try and see whats going wrong. ;)
<@jnsamyak:matrix.org>
16:46:45
sorry i was putting a comment on the last ticket reading this one, can i ask from where are you able to dig those logs as my first question while i go through the ticket?
<@nirik:matrix.scrye.com>
16:47:41
yeah, all our stuff (well, most of it) goes to the releng-cron mailing list.
<@nirik:matrix.scrye.com>
16:47:57
https://lists.fedoraproject.org/archives/list/releng-cron@lists.fedoraproject.org
<@nirik:matrix.scrye.com>
16:48:49
so, you can see the output from the composes... but note that it's large. ;)
<@jnsamyak:matrix.org>
16:48:51
need to go throught the block_retired script to see what is happening there i guess for starters
<@jnsamyak:matrix.org>
16:49:16
yep manual debugging options seems to be a valid starting point
<@nirik:matrix.scrye.com>
16:49:21
basically it's asking pdc what packages are eol... and then making sure they are blocked in koji
<@nirik:matrix.scrye.com>
16:50:06
yeah, welcome to drinking from the firehose. ;)
<@nirik:matrix.scrye.com>
16:50:17
anyhow, I can look at this if you will be out?
<@jnsamyak:matrix.org>
16:50:19
nirik: if i remeber correctly Anton Medvedev was working on this retiring stuff for PDC and this script
<@jnsamyak:matrix.org>
16:51:10
nirik: if you can great, but if i am just in my bed tomorrow morning with no plans i can try to hack and put a comment otherwise you take over
<@nirik:matrix.scrye.com>
16:51:23
yeaah, that would replace this.
<@nirik:matrix.scrye.com>
16:51:34
but this is what we have now. ;)
<@nirik:matrix.scrye.com>
16:51:47
I'll try and look at it...
<@nirik:matrix.scrye.com>
16:51:59
I had just one more general note today:
<@nirik:matrix.scrye.com>
16:52:38
We have a ton of old scripts in releng/scripts... we may want to consider removing ones we don't use... and reorganizing things better in there...
<@nirik:matrix.scrye.com>
16:52:54
some things are in subdirs, but a lot is just scattered around.
<@nirik:matrix.scrye.com>
16:53:00
anyhow, just something to think about.
<@jnsamyak:matrix.org>
16:53:12
this is something i have to discuss with you all but i was waiting for the release cycle to finish or get less hectic
<@nirik:matrix.scrye.com>
16:53:17
yep.
<@nirik:matrix.scrye.com>
16:53:26
it gets less hectic? ;)
<@nirik:matrix.scrye.com>
16:53:52
anyhow, thats all I had off hand... and I have another meeting in a few.
<@jnsamyak:matrix.org>
16:53:55
haha hey i know i have been here since 38 but this is my first time sinking it all in :P
<@jnsamyak:matrix.org>
16:54:25
nirik: thanks for joining this meeting, and helping out! Much much appreciated! 🍪
<@nirik:matrix.scrye.com>
16:54:38
thank you for running it!
<@jnsamyak:matrix.org>
16:54:47
I'll see you in the next one!
<@jnsamyak:matrix.org>
16:55:19
!info Thank you all for coming.
<@jnsamyak:matrix.org>
16:55:25
!endmeeting