16:00:11 <jednorozec> #startmeeting RELENG (2023-06-27)
16:00:11 <zodbot> Meeting started Tue Jun 27 16:00:11 2023 UTC.
16:00:11 <zodbot> This meeting is logged and archived in a public location.
16:00:11 <zodbot> The chair is jednorozec. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:00:11 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:11 <zodbot> The meeting name has been set to 'releng_(2023-06-27)'
16:00:11 <jednorozec> #meetingname releng
16:00:11 <zodbot> The meeting name has been set to 'releng'
16:00:11 <jednorozec> #chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec
16:00:11 <jednorozec> #topic init process
16:00:11 <zodbot> Current chairs: dustymabe jednorozec nirik pbrobinson phsmoura sharkcz
16:00:28 <nirik> good morning
16:00:45 <sharkcz> hi
16:00:59 <jednorozec> hello all
16:01:03 <nirik> I had a few things to discuss... (I should have written them down. oh well, hopefully I will remember)
16:02:44 <jednorozec> So I have some updates
16:03:03 <jednorozec> .releng 11399
16:03:04 <zodbot> jednorozec: Issue #11399: Add container/fedora-toolbox as a Fedora Linux 39 release-blocking deliverable - releng - Pagure.io - https://pagure.io/releng/issue/11399
16:03:13 <jednorozec> things are in place as of few moments ago
16:03:29 <nirik> hopefully it all works. ;)
16:03:32 <jednorozec> we should get first toolbox container or traceback from the build with new nightly
16:04:01 <jednorozec> well even if it does not work at first its in place and we will make it work :D
16:04:40 <jednorozec> .releng 11089
16:04:41 <zodbot> jednorozec: Issue #11089: Please, rename Fedora-Server-KVM Checksum file for F37 to be different from ISO - releng - Pagure.io - https://pagure.io/releng/issue/11089
16:04:50 <nirik> I guess it's blocking too, so if it doesn't work... no rawhide. ;(
16:04:53 <jednorozec> I would like to merge the PR changing this
16:05:11 <jednorozec> I know its not a priority but its entry level issue for one of the juniors
16:05:28 <jednorozec> and he opened pr for it
16:05:43 <nirik> ok. I didn't see that? was it against pungi-fedora?
16:05:51 <jednorozec> https://pagure.io/pungi-fedora/pull-request/1172
16:06:10 <jednorozec> I think this might break some tests
16:06:11 <nirik> oh yeah, huh, must have missed it.
16:06:37 <nirik> yeah, can you ping adamw on the pr?
16:06:42 <jednorozec> should I ping adam in the ticket before merging it?
16:06:45 <jednorozec> heh
16:06:57 <jednorozec> yeah, writing comment
16:07:06 <nirik> +1
16:09:07 <jednorozec> done
16:09:30 <jednorozec> one more update
16:10:05 <jednorozec> toddlers for package unretirement are deployed in staging, its in rough shape but there is update PR to fix things
16:10:36 <nirik> progress is good!
16:11:13 <jednorozec> I am thinking about announcing the change and deployment in prod with next branching
16:11:49 <jednorozec> we will use scm-requests tracker for that...
16:12:05 <jednorozec> hopefully soon also for stalled epel packages
16:12:10 <nirik> so how does it all work? is there a fedpkg unretire?
16:12:18 <jednorozec> in progress
16:12:22 <jednorozec> but yes
16:12:30 <jednorozec> fedpk clone && fedpk unretire
16:12:37 * nirik hasn't followed it much. As long as it does all the checks.
16:12:42 <jednorozec> should be the workflow
16:12:58 * nirik nods
16:12:58 <jednorozec> yes same things as I am doing manually but in toddler
16:13:14 <nirik> will be nice to automate those.
16:13:18 <jednorozec> we have some weeks ahead to get it working correctly
16:14:21 <nirik> sure, it will get there. ;)
16:15:20 <jednorozec> sure it will
16:15:22 <jednorozec> so
16:15:43 <jednorozec> I bet there are more updates but I cant remember right now
16:15:52 <jednorozec> oh yeah
16:16:07 <jednorozec> the silverblue install media
16:17:13 <jednorozec> I tried to create the task few times but it kept hanging in open state
16:18:08 <nirik> might be related to the lovely systemd bug I have been fighting (depending on when you did it)
16:18:56 <jednorozec> oh last week
16:19:19 <nirik> ok, then thats not it.
16:19:24 <jednorozec> So I still think I dont exactly understand the way the ostree is made
16:19:47 * jednorozec looks for the koji command in notes
16:19:52 <nirik> I updated builders this weekend... and got a new systemd that causes havok with libvirt, so image builds have been failing so no rawhide
16:20:06 <jednorozec> it actuall broke my history I remember that very well
16:20:57 <nirik> systemd is now downgraded on all of them and they are rebooted into that... so... fingers crossed.
16:21:12 <jednorozec> so am I doing the right thing running this?
16:21:13 <jednorozec> https://paste.centos.org/view/65233847
16:21:45 <jednorozec> or is that just the repo for ostree that is composed by runroot
16:23:48 <nirik> no, I think the installer images are also runroot like that... looking
16:25:18 <nirik> I'd say perhaps change the paths so it doesn't land in the old compose... like /mnt/koji/compose/branched/oneoff-aarch64-ostree-image? but you need the paths for some of it to find the packages/ostrees...
16:25:27 <nirik> so that just hung?
16:25:34 <jednorozec> yup
16:25:37 <jednorozec> in open state
16:25:48 <jednorozec> its basically the same command that failed in RC compose
16:26:00 <jednorozec> but yeah I forgot about the output dir
16:26:07 <nirik> if you can run it again sometime when I am around I can dig around and see if I can see where it's getting stuck...?
16:26:24 <jednorozec> the thing is that it lands in compose chanell when running by the compose
16:26:43 <jednorozec> but mine landedn in different chanell and just hang there
16:26:47 <nirik> might be the buildroot changed and something is messing it up there?
16:26:55 <jednorozec> ack
16:27:19 <nirik> could run it as releng user too I suppose... but that shouldn't matter I would not think....
16:28:16 <jednorozec> yeah
16:28:28 <jednorozec> just wanted to bring it up, I didnt forget about that!
16:28:36 <jednorozec> that is a rarity for my brain
16:29:11 <jednorozec> I will give it little bit more poking once the systemd bug is resolved
16:29:31 <nirik> might update the ticket too so people know it's still being worked on
16:29:35 <jednorozec> and if nothing will stick around in upcoming days
16:30:23 * jednorozec updates the ticket
16:32:16 <jednorozec> now that all from me
16:32:33 <nirik> I had a few things... lets see.
16:32:51 <nirik> .releng 11422
16:32:53 <zodbot> nirik: Issue #11422: Please send openh264-2.3.1-1.el9 to Cisco - releng - Pagure.io - https://pagure.io/releng/issue/11422
16:33:04 <nirik> can we move that repo to match what mm wants?
16:33:20 <nirik> I don't know if that needs a new compose or just moving the repo...
16:33:34 * jednorozec checks
16:33:58 <jednorozec> oh
16:34:04 <nirik> basically we have epel9 and it should be epel/9/
16:34:08 <jednorozec> that shoudl be just copy
16:34:15 <jednorozec> right
16:34:31 <jednorozec> I think this is just about the final rsync that puts it in place
16:34:48 <nirik> well, it has to be changed on mm-backend01 too so it can detect it right.
16:35:07 <nirik> and... possibly the cisco end needs changing?
16:35:09 <jednorozec> yeah that is where the final rsync is taking place
16:35:19 <jednorozec> hm
16:35:23 <jednorozec> I dont think so
16:35:37 <nirik> ok. Can you look into this? :)
16:35:44 <jednorozec> I thinkthis is just the dir struct for the mm
16:35:55 <jednorozec> not really the structure in the compose data
16:36:02 * jednorozec checks right now
16:36:27 <nirik> well, but mm will make metalinks pointing to a specific dir... if thats not there on the cisco end it won't work
16:38:06 <nirik> dunno. Might be safest to just make it over and resend it and tell them to nuke the old one and make the new one? but might not be needed... not sure.
16:39:31 <nirik> but if you could investigate that would be great. I could try and look too, but not sure when I will get to it. ;)
16:39:46 <jednorozec> yeah
16:39:46 <nirik> I had 2 more...
16:39:52 <nirik> .releng 11479
16:39:53 <zodbot> nirik: Issue #11479: Create tag structure for non-modular Flatpaks - releng - Pagure.io - https://pagure.io/releng/issue/11479
16:39:55 <jednorozec> so we just send the rpms in a tarball to cisco
16:40:04 <jednorozec> nothing more no dir structure nothing
16:40:19 <jednorozec> but yeah move on to the next issue
16:40:43 <nirik> I think this will all work... otaylor said he could try and do a pr for our branching script... but if you or someone else wants to before then we could land this.
16:41:09 <nirik> I'm happy to have less mbs
16:41:35 <jednorozec> for sure
16:41:44 <jednorozec> I am also happy about modularity retirement
16:41:59 <pcreech_> "someone else wants to before" -> would this be a good for someone looking to get involved help out with? ;)
16:42:13 <jednorozec> anyhow this looks exactly like work for our morning meeting with guys
16:42:33 <jednorozec> pcreech_, sure!
16:42:43 <nirik> sure. I think this is just translating the initial setup in that ticket to a version for branching...
16:42:51 <jednorozec> exactly
16:43:11 <pcreech_> yeah, looked like a bunch of koji tags/commands. figured it' help with learning fedora koji stuff
16:43:40 <nirik> .releng 11499
16:43:41 <zodbot> nirik: Issue #11499: Fedora 34 updates repo on archives.fedoraproject.org is incorrect - releng - Pagure.io - https://pagure.io/releng/issue/11499
16:43:48 <jednorozec> right
16:43:50 <jednorozec> so
16:44:06 <nirik> this is a fun one. ;) So our archived f34 updates is wrong.
16:44:15 <jednorozec> I just few minutes ago found the command that is nowhere to be found to update filelists
16:44:41 <nirik> yeah, it's in the cron jobs for rawhide/updatespushing etc
16:44:57 <nirik> anyhow, I think it might be nice to fix this... but we need to be careful.
16:45:24 <jednorozec> I think this again should be just about updating mm data
16:45:34 <jednorozec> the latest things are in place in archive
16:45:37 * nirik wonders how many people running f34 will be surprised to see updates appear. ;)
16:45:41 <jednorozec> they are just not live
16:45:42 <nirik> nope
16:45:48 <nirik> it's not.
16:45:54 <nirik> this is what happened as I understand it.
16:46:24 <jednorozec> ah
16:46:54 <nirik> we were mid way through the cycle with f34... smooge archived f32 f33 f34... but of course f34 updates were still getting updated. Then, f34 went eol and we said "oh, its already archived, great, done". But it wasn't it has a updates/updates-testing from back months before.
16:47:08 <nirik> then we deleted the non archived version.
16:47:26 <nirik> we still do have the composes/updates/ version of the last updates/updates-testing pushes.
16:47:43 <nirik> so we could sync those to updates/updates-testing (just like new-updates-sync does) and it would be right.
16:48:14 <nirik> so right now the archives have those repos from a few months before eold
16:48:16 <nirik> eol
16:48:33 <jednorozec> I see
16:49:01 <nirik> So, to fix it we would need to carefully sync that. I'd advise a script we can run with -n first to see what it will change...
16:49:47 <jednorozec> so we are talking about https://kojipkgs.fedoraproject.org/compose/updates/Fedora-34-updates-testing-20220607.0/ and https://kojipkgs.fedoraproject.org/compose/updates/Fedora-34-updates-20220607.0/
16:49:48 <jednorozec> ?
16:50:37 <nirik> yes, vs https://dl.fedoraproject.org/pub/archive/fedora/linux/updates/34/Everything ... and updates/testing/34/
16:50:44 <jednorozec> right
16:50:58 <jednorozec> ok, we can write some script with guys in the morning
16:51:04 <nirik> look at the repodata there and you can see its from may 12th... not june 7th
16:51:19 <jednorozec> right
16:51:22 <nirik> so, a month or so of updates missing.
16:51:29 <jednorozec> that can be a lot
16:51:34 <jednorozec> but srsly f34
16:51:39 <jednorozec> why?
16:51:41 <jednorozec> :D
16:51:41 <nirik> also, fedora-secondary for it I guess.
16:51:55 <nirik> yeah, it's eol, but... it's anoying to be wrong. ;)
16:52:22 <nirik> I think I had one more, looking
16:52:37 <jednorozec> sure thing, we should have things right
16:52:54 <nirik> ah yes...
16:52:58 <nirik> .releng 11500
16:53:08 <zodbot> nirik: Issue #11500: Changes/FedoraWorkstationImageBuilder: switch pungi config to use image builder - releng - Pagure.io - https://pagure.io/releng/issue/11500
16:53:18 <nirik> This isn't approved yet...
16:53:32 <nirik> but if/when it is, we probibly need a new repo for the new format files.
16:53:42 <jednorozec> is the fesco voting today?
16:53:51 <nirik> and there will be some learning curve on how to debug things, etc.
16:54:40 <jednorozec> I am all in for any alternative to image-factory
16:54:58 <nirik> no, it's next week, it's still being discussed on list. ;)
16:55:26 <nirik> I'm a little worried that they only want to do workstation... but I guess we could push more of them ourselves.
16:55:44 <jednorozec> well we have to start somewhere
16:56:13 <nirik> sure, but I don't want "hey, we moved the important one, bye!" to happen. ;)
16:56:29 <nirik> not that they would, but...
16:56:33 <nirik> anyhow, thats all I had.
16:56:44 <jednorozec> and if it works for workstation I will volunteer to move stuff to imagebuilder
16:57:02 <jednorozec> some of it not all of it :)
16:57:09 <jednorozec> so we are ner the end
16:57:12 <jednorozec> oh
16:57:45 <jednorozec> I am on PTO next week, will be back for mass rebuild and gone again back for branching
16:58:04 <nirik> pcreech_: BTW, welcome... and do chime in if you want to work on something or have questions. :)
16:58:23 <pcreech_> :D Will do
16:58:48 <jednorozec> pcreech_, I am also available but in CET timezone
16:59:03 <jednorozec> #endmeeting