2024-03-26 16:01:08 <@jnsamyak:matrix.org> !startmeeting RELENG (2024-03-26)
2024-03-26 16:01:11 <@meetbot:fedora.im> Meeting started at 2024-03-26 16:01:08 UTC
2024-03-26 16:01:11 <@meetbot:fedora.im> The Meeting name is 'RELENG (2024-03-26)'
2024-03-26 16:01:17 <@jnsamyak:matrix.org> !meetingname releng

2024-03-26 16:01:24 <@jnsamyak:matrix.org> !chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec jnsamyak patrikp
2024-03-26 16:01:25 <@nirik:matrix.scrye.com> morning.
2024-03-26 16:01:29 <@jnsamyak:matrix.org> !info Meeting is 60 minutes MAX. At the end of 60, it stop

2024-03-26 16:01:36 <@jnsamyak:matrix.org> !info agenda is at https://hackmd.io/vm6biLBcTYKtkQUH5kQkmw

2024-03-26 16:01:41 <@jnsamyak:matrix.org> !topic init process

2024-03-26 16:01:47 <@nirik:matrix.scrye.com> happy f40 beta release morning.
2024-03-26 16:02:01 <@jnsamyak:matrix.org> Hello everyone, release day yayyyy 🎉
2024-03-26 16:02:13 <@jnsamyak:matrix.org> how's everyone doing today?
2024-03-26 16:02:33 <@jnsamyak:matrix.org> !info Fedora Linux 40 Beta has been released today at 14:00 UTC!
2024-03-26 16:03:04 <@nirik:matrix.scrye.com> so far so good
2024-03-26 16:04:02 <@jnsamyak:matrix.org> nice I have just one thing for init process, we have only one week for the final freeze to start; so anything that is left post beta freeze that needs to go on this is the time :/
2024-03-26 16:04:33 <@nirik:matrix.scrye.com> yep. I also mailed out a note about that to devel-announce to remind everyone
2024-03-26 16:04:43 <@jnsamyak:matrix.org> it happened because the beta release target got shifted, and fesco agreed on the final freeze and release shouldn't get affted due to beta
2024-03-26 16:05:04 <@jnsamyak:matrix.org> nirik: noice, thanks as usual you are awesome!
2024-03-26 16:05:16 <@jnsamyak:matrix.org> Anything else, or can we move on to our ticket dumps?
2024-03-26 16:05:28 <@nirik:matrix.scrye.com> sure, we can move to tickets.
2024-03-26 16:06:01 <@jnsamyak:matrix.org> !topic Tickets needing attention

2024-03-26 16:06:08 <@jnsamyak:matrix.org> !info this is the time where you can bring releng tickets to discuss if there are blockers etc

2024-03-26 16:06:34 <@jnsamyak:matrix.org> !releng 12039
2024-03-26 16:06:35 <@zodbot:fedora.im> **releng #12039** (https://pagure.io/releng/issue/12039):**libva-2.21.0-2.fc40 doesn't appear in repo**

● **Opened:** 7 hours ago by kwizart
● **Last Updated:** Never
● **Assignee:** Not Assigned
2024-03-26 16:07:06 <@jnsamyak:matrix.org> nirik: I want to work on this I know it seems simple enough but i need to understand quick debugging terminology for this
2024-03-26 16:07:50 <@nirik:matrix.scrye.com> So... we have a script to look for this...
2024-03-26 16:07:51 <@jnsamyak:matrix.org> so i did ` koji list-history --package=libva`
2024-03-26 16:07:56 <@jnsamyak:matrix.org> to debug what happened there
2024-03-26 16:08:20 <@nirik:matrix.scrye.com> check-latest-build.py f40
2024-03-26 16:08:35 <@jnsamyak:matrix.org> oh
2024-03-26 16:09:26 <@nirik:matrix.scrye.com> what happened is likely what often happens: Someone submits an update during freeze, it is pending stable, but the freeze blocks it from going stable. Then they submit a new update and forget about the old one, and then when the freeze is over, both of them go stable. Which one is in the compose depends on which one was tagged last. If it's the older one... then this happens.
2024-03-26 16:09:50 <@jnsamyak:matrix.org> aha
2024-03-26 16:10:05 <@jnsamyak:matrix.org> by looking at the history libva-2.21.0-3.fc40 I see a new one
2024-03-26 16:10:22 <@nirik:matrix.scrye.com> There's also other cases similar, like someone submits an update with a,b,c in it, someone then later submits a newer c... and the other one goes stable after the single one
2024-03-26 16:10:52 <@jnsamyak:matrix.org> so what's the best approach to it? figure out what's needed and untag rest updates?
2024-03-26 16:10:55 <@nirik:matrix.scrye.com> anyhow, we can fix this by running the above script and looking at the output then retagging things that we need to fix.
2024-03-26 16:11:01 <@jnsamyak:matrix.org> from stable? (f40 tag i mean)
2024-03-26 16:11:23 <@nirik:matrix.scrye.com> theres often some broken ones we don't want to fix... like:
2024-03-26 16:11:30 <@nirik:matrix.scrye.com> calendar-1.37-6.20240209cvs.fc40 < calendar-1.37-7.20211220cvs.fc37
2024-03-26 16:11:49 <@nirik:matrix.scrye.com> The old f37 update there is 'newer' than the f40 one... but we don't want that one.
2024-03-26 16:12:23 <@jnsamyak:matrix.org> makes sense need to have eyes and ears open while tagging stuff in and out
2024-03-26 16:12:31 <@nirik:matrix.scrye.com> so, process would be: run script, look at output, saneify it, and then retag the newer ones for the ones we want to fix.
2024-03-26 16:13:06 <@jnsamyak:matrix.org> I'll try out
2024-03-26 16:13:28 <@nirik:matrix.scrye.com> sounds good. :) Let me know if you have any questions.
2024-03-26 16:13:47 <@nirik:matrix.scrye.com> and... we should put this in our release schedule. Do it right after the unfrozen updates push
2024-03-26 16:14:15 <@jnsamyak:matrix.org> sure thing, once I get an idea, I'll try to put a draft in lay man terms
2024-03-26 16:14:27 <@jnsamyak:matrix.org> Next one, 🥁
2024-03-26 16:14:34 <@jnsamyak:matrix.org> !releng 12037
2024-03-26 16:14:36 <@zodbot:fedora.im> **releng #12037** (https://pagure.io/releng/issue/12037):**Fix F40 flatpak build tags**

● **Opened:** 22 hours ago by yselkowitz
● **Last Updated:** 20 hours ago
● **Assignee:** Not Assigned
2024-03-26 16:16:00 <@nirik:matrix.scrye.com> yeah, we made adjustments on the fly to the f39 ones when we were bringing it up... we need to make those to f40 and then make sure they are set moving forward.
2024-03-26 16:16:15 <@nirik:matrix.scrye.com> you want to take this one? or want me to?
2024-03-26 16:16:15 <@jnsamyak:matrix.org> we already have a pr for this
2024-03-26 16:16:25 <@jnsamyak:matrix.org> https://pagure.io/releng/pull-request/12038
2024-03-26 16:16:56 <@nirik:matrix.scrye.com> yeah, for f41+... we still need to manually fix f40 tho
2024-03-26 16:17:55 <@jnsamyak:matrix.org> so we want to just review I guess for f40+ but need to make changes for f40 for sure, can you take this one? I'm off for next few days :/
2024-03-26 16:18:25 <@nirik:matrix.scrye.com> sure thing. feel free to assign to me
2024-03-26 16:18:51 <@jnsamyak:matrix.org> thanks, one quick request tho, can you add those commands it will be easier for me to review the PR for sure
2024-03-26 16:18:57 <@jnsamyak:matrix.org> 12308 sold to you
2024-03-26 16:19:18 <@nirik:matrix.scrye.com> you mean the f40 fix commands? yeah, I'll draft up a set of commands and add to the ticket.
2024-03-26 16:19:26 <@jnsamyak:matrix.org> yes
2024-03-26 16:20:18 <@jnsamyak:matrix.org> !releng 12036
2024-03-26 16:20:19 <@zodbot:fedora.im> **releng #12036** (https://pagure.io/releng/issue/12036):**Warning 'Cannot find relevant tag for build' in bodhi**

● **Opened:** a day ago by jplesnik
● **Last Updated:** 7 hours ago
● **Assignee:** Not Assigned
2024-03-26 16:20:45 <@jnsamyak:matrix.org> do we put this for investigation?
2024-03-26 16:20:46 <@nirik:matrix.scrye.com> I guess we just fix these as we find em
2024-03-26 16:21:37 <@nirik:matrix.scrye.com> so I think I have a theory...
2024-03-26 16:21:51 <@jnsamyak:matrix.org> I checked these three stable updates went to stable should we close this for now?
2024-03-26 16:22:02 <@jnsamyak:matrix.org> nirik: shoot!
2024-03-26 16:22:21 <@nirik:matrix.scrye.com> well, there's at least one more to fix (see last comment)
2024-03-26 16:22:34 <@jnsamyak:matrix.org> :) yes
2024-03-26 16:23:10 <@nirik:matrix.scrye.com> I think what happened is that these failed gating for a bit... then when they were passing gating, it was time to stable, so bodhi requested stable... without ever requesting testing.
2024-03-26 16:23:31 <@nirik:matrix.scrye.com> I'll add that to the ticket and see if mattia can think of a way for bodhi to handle that.
2024-03-26 16:23:54 <@jnsamyak:matrix.org> +1
2024-03-26 16:24:16 <@jnsamyak:matrix.org> btw you might have tagged them to f40-updates-testing tag again right? and wait for bodhi to do it's job!
2024-03-26 16:25:56 <@nirik:matrix.scrye.com> right, I just tagged them into testing...
2024-03-26 16:26:06 <@nirik:matrix.scrye.com> and then bodhi was fine with them on the next push.
2024-03-26 16:26:14 <@jnsamyak:matrix.org> yep
2024-03-26 16:26:27 <@jnsamyak:matrix.org> !releng 12032
2024-03-26 16:26:28 <@zodbot:fedora.im> **releng #12032** (https://pagure.io/releng/issue/12032):**The aarch64 minimal synced to the wrong location for releases**

● **Opened:** 4 days ago by pbrobinson
● **Last Updated:** 19 hours ago
● **Assignee:** Not Assigned
2024-03-26 16:26:49 <@jnsamyak:matrix.org> I just read the comment, it's waiting on me to update the sop i guess, I'll assign it to myself!
2024-03-26 16:27:12 <@jnsamyak:matrix.org> Also, we are discussing this at releng channel as well, this has website impact as well!
2024-03-26 16:27:28 <@nirik:matrix.scrye.com> yes, this is tricky.. so don't actually do it yet. ;)
2024-03-26 16:27:36 <@jnsamyak:matrix.org> yeah
2024-03-26 16:27:47 <@jnsamyak:matrix.org> I am reading at both places
2024-03-26 16:27:51 <@jnsamyak:matrix.org> ¯\_(ツ)_/¯
2024-03-26 16:28:01 <@nirik:matrix.scrye.com> so, needs more discussion I'd say
2024-03-26 16:28:49 <@jnsamyak:matrix.org> Do you think a devel email will bring more eyes to it?
2024-03-26 16:29:13 <@nirik:matrix.scrye.com> lets continue in ticket for now? then we can bring it up if we can't come to some solution
2024-03-26 16:29:59 <@jnsamyak:matrix.org> that makes sense1
2024-03-26 16:30:03 <@jnsamyak:matrix.org> that makes sense!
2024-03-26 16:30:17 <@jnsamyak:matrix.org> moving on to one more item 
2024-03-26 16:30:21 <@jnsamyak:matrix.org> !releng 12030
2024-03-26 16:30:23 <@zodbot:fedora.im> **releng #12030** (https://pagure.io/releng/issue/12030):**adjust torrent_hashes.py to non bittorrent world**

● **Opened:** 4 days ago by kevin
● **Last Updated:** 7 hours ago
● **Assignee:** jnsamyak
2024-03-26 16:30:52 <@jnsamyak:matrix.org> so i have a pr ready to merge this on releng: https://pagure.io/releng/pull-request/12027
2024-03-26 16:30:53 <@nirik:matrix.scrye.com> yeah, the torrent scripts are... old
2024-03-26 16:31:28 <@nirik:matrix.scrye.com> anyhow, yes, I can merge that one.
2024-03-26 16:31:33 <@jnsamyak:matrix.org> I need to check ansible but it should be same ini python script there as well?
2024-03-26 16:32:40 <@jnsamyak:matrix.org> For torrent-hashes.py script, as mentioned it worked locally to generate whitelist when we switched to python-fastbencode with this script: https://pagure.io/releng/issue/12030#comment-901508
2024-03-26 16:32:46 <@nirik:matrix.scrye.com> k, I was confused there.
2024-03-26 16:32:51 <@nirik:matrix.scrye.com> I meant:
2024-03-26 16:33:01 <@nirik:matrix.scrye.com> roles/torrent/files/torrent-generator/torrent-generator
2024-03-26 16:33:31 <@nirik:matrix.scrye.com> that one is the one that looks at all the *.torrent files and makes a .ini file
2024-03-26 16:33:45 <@nirik:matrix.scrye.com> which is used to make the website, etc.
2024-03-26 16:33:53 <@jnsamyak:matrix.org> aha this is another file for websited and stuff
2024-03-26 16:34:31 <@jnsamyak:matrix.org> okay this needs working; I can take a look at it; and will hopefully open a PR next week, or do we want this before that?
2024-03-26 16:34:56 <@nirik:matrix.scrye.com> it's not urgent.. I just hacked up a ini for beta. ;)
2024-03-26 16:35:05 <@nirik:matrix.scrye.com> but we will need it for final.
2024-03-26 16:35:35 <@jnsamyak:matrix.org> thanks; I'll keep this in mind yes ofcourse; need less broken and more testable stuff in f40 for sure that's the agenda
2024-03-26 16:35:36 <@jnsamyak:matrix.org> :D
2024-03-26 16:36:25 <@nirik:matrix.scrye.com> Sorry torrents are so rough... moving from 7 to 9 was a big jump. ;)
2024-03-26 16:36:30 <@jnsamyak:matrix.org> also i need to opena pr for maketorrent script i updated it directly to the machine but never open a pr on ansible,w ill do that so in future we wont loose it 
2024-03-26 16:36:47 <@nirik:matrix.scrye.com> yes please do. ;)
2024-03-26 16:36:53 <@jnsamyak:matrix.org> No need of sorry, I actually had some hacking fun that day!
2024-03-26 16:37:07 <@jnsamyak:matrix.org> 🕺
2024-03-26 16:37:39 <@jnsamyak:matrix.org> !releng 11673
2024-03-26 16:37:40 <@zodbot:fedora.im> **releng #11673** (https://pagure.io/releng/issue/11673):**Fixes for release-candidate.sh in pungi-fedora**

● **Opened:** 6 months ago by kevin
● **Last Updated:** 5 months ago
● **Assignee:** patrikp
2024-03-26 16:37:45 <@nirik:matrix.scrye.com> 👍
2024-03-26 16:38:14 <@jnsamyak:matrix.org> This is for patrikp; if there are any blockers regarding this? 
2024-03-26 16:38:31 <@nirik:matrix.scrye.com> I think there was some last round of comments for him...
2024-03-26 16:38:34 <@jnsamyak:matrix.org> we might be needing this to test it out for final
2024-03-26 16:39:04 <@nirik:matrix.scrye.com> yeah. we can try... if something comes up we can always revert. but I think it's in pretty reasonable shape now.
2024-03-26 16:39:08 <@jnsamyak:matrix.org> yes but the pr seems not modified since the comment so i was worried if there are some more clarifications that are needed.
2024-03-26 16:39:53 <@nirik:matrix.scrye.com> dunno, we should ping patrikp when he's next around to look. ;)
2024-03-26 16:40:11 <@jnsamyak:matrix.org> but that's mostly all from my side; do you have some tickets that you want to bring up?
2024-03-26 16:40:27 <@nirik:matrix.scrye.com> lets see...
2024-03-26 16:40:52 <@nirik:matrix.scrye.com> !releng 12001
2024-03-26 16:40:53 <@zodbot:fedora.im> **releng #12001** (https://pagure.io/releng/issue/12001):**Getting Rawhide + 1 keys at least 2 weeks before branching**

● **Opened:** 2 weeks ago by praiskup
● **Last Updated:** 18 hours ago
● **Assignee:** Not Assigned
2024-03-26 16:41:19 <@nirik:matrix.scrye.com> proposing to make the N+1 key eariler... if that looks good we can get it added to the schedule?
2024-03-26 16:41:38 <@jnsamyak:matrix.org> +1 to that i dont see any issues for it
2024-03-26 16:41:56 <@jnsamyak:matrix.org> should i bring it to next fesco meeting? 
2024-03-26 16:42:05 <@jnsamyak:matrix.org> or can you since you are the fesco ;) :P
2024-03-26 16:42:26 <@nirik:matrix.scrye.com> naw, I don't think this needs fesco approval, it's a releng thing... but we need to ask it be added to the official schedule...
2024-03-26 16:42:50 <@jnsamyak:matrix.org> yep i meant for the schedule only; I can put my +1 there in the comment
2024-03-26 16:43:01 <@nirik:matrix.scrye.com> I can't recall where that repo is, but we should file a ticket or bug aoife
2024-03-26 16:43:33 <@jnsamyak:matrix.org> I can ping Aoife for the details on that ;)
2024-03-26 16:43:53 <@nirik:matrix.scrye.com> thanks
2024-03-26 16:44:02 <@nirik:matrix.scrye.com> !releng 12028
2024-03-26 16:44:02 <@zodbot:fedora.im> **releng #12028** (https://pagure.io/releng/issue/12028):**Package retirements are broken in rawhide**

● **Opened:** 5 days ago by gotmax23
● **Last Updated:** 4 days ago
● **Assignee:** Not Assigned
2024-03-26 16:44:39 <@nirik:matrix.scrye.com> so, we need to figure out whats going on here...
2024-03-26 16:44:53 <@nirik:matrix.scrye.com> the nightly.sh for rawhide/branched runs the retire script...
2024-03-26 16:45:07 <@nirik:matrix.scrye.com> ./releng/scripts/block_retired.py --profile compose_koji
2024-03-26 16:45:13 <@nirik:matrix.scrye.com> but I don't see any output from it.
2024-03-26 16:45:48 <@nirik:matrix.scrye.com> I guess we should run it manually and try and see whats going wrong. ;)
2024-03-26 16:46:45 <@jnsamyak:matrix.org> sorry i was putting a comment on the last ticket reading this one, can i ask from where are you able to dig those logs as my first question while i go through the ticket?
2024-03-26 16:47:41 <@nirik:matrix.scrye.com> yeah, all our stuff (well, most of it) goes to the releng-cron mailing list.
2024-03-26 16:47:57 <@nirik:matrix.scrye.com> https://lists.fedoraproject.org/archives/list/releng-cron@lists.fedoraproject.org
2024-03-26 16:48:49 <@nirik:matrix.scrye.com> so, you can see the output from the composes... but note that it's large. ;)
2024-03-26 16:48:51 <@jnsamyak:matrix.org> need to go throught the block_retired script to see what is happening there i guess for starters
2024-03-26 16:49:16 <@jnsamyak:matrix.org> yep manual debugging options seems to be a valid starting point
2024-03-26 16:49:21 <@nirik:matrix.scrye.com> basically it's asking pdc what packages are eol... and then making sure they are blocked in koji
2024-03-26 16:50:06 <@nirik:matrix.scrye.com> yeah, welcome to drinking from the firehose. ;)
2024-03-26 16:50:17 <@nirik:matrix.scrye.com> anyhow, I can look at this if you will be out?
2024-03-26 16:50:19 <@jnsamyak:matrix.org> nirik: if i remeber correctly Anton Medvedev was working on this retiring stuff for PDC and this script
2024-03-26 16:51:10 <@jnsamyak:matrix.org> nirik: if you can great, but if i am just in my bed tomorrow morning with no plans i can try to hack and put a comment otherwise you take over
2024-03-26 16:51:23 <@nirik:matrix.scrye.com> yeaah, that would replace this.
2024-03-26 16:51:34 <@nirik:matrix.scrye.com> but this is what we have now. ;)
2024-03-26 16:51:47 <@nirik:matrix.scrye.com> I'll try and look at it...
2024-03-26 16:51:59 <@nirik:matrix.scrye.com> I had just one more general note today:
2024-03-26 16:52:38 <@nirik:matrix.scrye.com> We have a ton of old scripts in releng/scripts... we may want to consider removing ones we don't use... and reorganizing things better in there...
2024-03-26 16:52:54 <@nirik:matrix.scrye.com> some things are in subdirs, but a lot is just scattered around.
2024-03-26 16:53:00 <@nirik:matrix.scrye.com> anyhow, just something to think about.
2024-03-26 16:53:12 <@jnsamyak:matrix.org> this is something i have to discuss with you all but i was waiting for the release cycle to finish or get less hectic
2024-03-26 16:53:17 <@nirik:matrix.scrye.com> yep.
2024-03-26 16:53:26 <@nirik:matrix.scrye.com> it gets less hectic? ;)
2024-03-26 16:53:52 <@nirik:matrix.scrye.com> anyhow, thats all I had off hand... and I have another meeting in a few.
2024-03-26 16:53:55 <@jnsamyak:matrix.org> haha hey i know i have been here since 38 but this is my first time sinking it all in :P
2024-03-26 16:54:25 <@jnsamyak:matrix.org> nirik: thanks for joining this meeting, and helping out! Much much appreciated! 🍪
2024-03-26 16:54:38 <@nirik:matrix.scrye.com> thank you for running it!
2024-03-26 16:54:47 <@jnsamyak:matrix.org> I'll see you in the next one!
2024-03-26 16:55:19 <@jnsamyak:matrix.org> !info Thank you all for coming.

2024-03-26 16:55:25 <@jnsamyak:matrix.org> !endmeeting