2024-02-20 09:01:49 <@meetbot:fedora.im> HTML Minutes: https://meetbot.fedoraproject.org/meeting-3_matrix_fedoraproject-org/2024-02-20/cpe-infra-releng-daily-standup.2024-02-20-08.34.html
2024-02-20 17:00:18 <@humaton:fedora.im> !startmeeting RELENG (2024-02-20)
!meetingname releng
2024-02-20 17:00:19 <@meetbot:fedora.im> Meeting started at 2024-02-20 17:00:18 UTC
2024-02-20 17:00:19 <@meetbot:fedora.im> The Meeting name is 'RELENG (2024-02-20) !meetingname releng'
2024-02-20 17:00:28 <@humaton:fedora.im> !chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec
2024-02-20 17:00:32 <@humaton:fedora.im> !topic init process
2024-02-20 17:00:54 <@nirik:matrix.scrye.com> morning
2024-02-20 17:01:10 <@humaton:fedora.im> welcome back sir
2024-02-20 17:01:12 <@patrikp:matrix.org> Good evening to all.
2024-02-20 17:01:24 <@humaton:fedora.im> howdy
2024-02-20 17:04:15 <@jnsamyak:matrix.org> o/ I'm here
2024-02-20 17:04:24 <@jnsamyak:matrix.org> but sleeping a little :) :P
2024-02-20 17:04:45 <@humaton:fedora.im> nice
2024-02-20 17:05:31 <@humaton:fedora.im> I have just one thing for today
2024-02-20 17:05:49 <@humaton:fedora.im> !releng 11970
2024-02-20 17:05:50 <@zodbot:fedora.im> **releng #11970** (https://pagure.io/releng/issue/11970):**Create detached signatures for the butane 0.20.0 release**
● **Opened:** an hour ago by ydesouza
● **Last Updated:** 18 seconds ago
● **Assignee:** Not Assigned
2024-02-20 17:06:00 <@jnsamyak:matrix.org> I just tagged it
2024-02-20 17:06:38 <@humaton:fedora.im> to be able to work on this the person need to setup sigul client and get access to the relevant release keys
2024-02-20 17:06:57 <@humaton:fedora.im> does any of you have the sigul setup done? jnsamyak patrikp
2024-02-20 17:07:09 <@jnsamyak:matrix.org> also this need to be run as root of some machine?
2024-02-20 17:07:16 <@humaton:fedora.im> nope
2024-02-20 17:07:29 <@jnsamyak:matrix.org> jednorozec: nope tried but i dont remember why i was not able to do it
2024-02-20 17:08:57 <@zodbot:fedora.im> jnsamyak has already given cookies to kevin during the F39 timeframe
2024-02-20 17:09:35 <@patrikp:matrix.org> I don't have it set up.
2024-02-20 17:09:49 <@patrikp:matrix.org> Do we have some SOP with steps?
2024-02-20 17:10:06 <@nirik:matrix.scrye.com> There's an sop in the old docs, but not sure it was moved over yet to new one.
2024-02-20 17:10:35 <@nirik:matrix.scrye.com> https://docs.pagure.org/releng/sop_sigul_client_setup.html
2024-02-20 17:10:40 <@jnsamyak:matrix.org> https://docs.fedoraproject.org/en-US/infra/releng_misc_guide/sop_sigul_client_setup/#_sigul_client_setup
2024-02-20 17:10:52 <@jnsamyak:matrix.org> this is here^
2024-02-20 17:11:00 <@nirik:matrix.scrye.com> ok, great.
2024-02-20 17:11:00 <@jnsamyak:matrix.org> but i remember now
2024-02-20 17:11:07 <@jnsamyak:matrix.org> what was the problem
2024-02-20 17:11:22 <@jnsamyak:matrix.org> getting these:
~/.fedora.cert
~/.fedora-server-ca.cert
~/.fedora-upload-ca.cert
2024-02-20 17:11:24 <@zodbot:fedora.im> patrikp gave a cookie to kevin. They now have 654 cookies, 17 of which were obtained in the Fedora 39 release cycle
2024-02-20 17:11:30 <@nirik:matrix.scrye.com> yeah, that part is wrong. ;)
2024-02-20 17:11:46 <@nirik:matrix.scrye.com> we have a seperate ca for sigul... we need to issue you a cert from it.
2024-02-20 17:12:17 <@nirik:matrix.scrye.com> (also the phx2 in those hostnames is... really wrong. ;)
2024-02-20 17:12:22 <@jnsamyak:matrix.org> ahaa i think that's why I tried this with jednorozec but didnt go anywhere
2024-02-20 17:12:32 <@humaton:fedora.im> yeah soe hostnames are still decade old
2024-02-20 17:13:02 <@humaton:fedora.im> nope jnsamyak I gave you ca cert not mine cert
2024-02-20 17:13:05 <@jnsamyak:matrix.org> yeah hostname are obsolete because of migrating it from old docs i'm updating most of them as i go through the release process but all doc help welcome :D
2024-02-20 17:13:19 <@nirik:matrix.scrye.com> also, usually you just run it on bodhi-backend01... since the client is installed there and setup, etc... I don't know if it could work from a client machine.
2024-02-20 17:13:40 <@jnsamyak:matrix.org> yep^ that
2024-02-20 17:13:52 <@humaton:fedora.im> It can but its complicated and not worthed
2024-02-20 17:14:17 <@humaton:fedora.im> what I mean by setup the client is to have the certs and everything in ~ on bodhi-backend
2024-02-20 17:15:54 <@humaton:fedora.im> anyhow can you issue certs for guys nirik ? or can you show me how?
2024-02-20 17:17:03 <@nirik:matrix.scrye.com> sure... either. ;) It's in ansible-private/files/sigul-ca/production (and staging) and it's a normal easyrsa setup...
2024-02-20 17:17:14 <@humaton:fedora.im> ahh
2024-02-20 17:17:36 <@humaton:fedora.im> let tomorrow be the certification day!
2024-02-20 17:17:42 <@nirik:matrix.scrye.com> ha
2024-02-20 17:18:32 <@humaton:fedora.im> ok I have one more
2024-02-20 17:18:38 <@nirik:matrix.scrye.com> let me know if you run into any problems with it...
2024-02-20 17:18:47 <@humaton:fedora.im> sure will do
2024-02-20 17:18:50 <@humaton:fedora.im> !releng 11969
2024-02-20 17:18:51 <@zodbot:fedora.im> **releng #11969** (https://pagure.io/releng/issue/11969):**Fedora Atomic Desktops composes are failing for F40 & Rawhide as they include "external" repos**
● **Opened:** 6 hours ago by siosm
● **Last Updated:** Never
● **Assignee:** Not Assigned
2024-02-20 17:19:20 <@humaton:fedora.im> some branching cleanup needed maybe?
2024-02-20 17:20:53 <@nirik:matrix.scrye.com> could be.
2024-02-20 17:23:27 <@nirik:matrix.scrye.com> I do have a few smaller items tho
2024-02-20 17:23:42 <@humaton:fedora.im> go on nirik
2024-02-20 17:23:51 <@humaton:fedora.im> we will look into the composes tmrw
2024-02-20 17:24:06 <@nirik:matrix.scrye.com> ok. I have scheduled an outage tomorrow for upgrading koji to 1.34.0... this brings in a lot of changes.
2024-02-20 17:24:46 <@nirik:matrix.scrye.com> The scheduler is completely re-implemented. There's 'draft builds' (which I plan on denying being tagged into anything until we can decide what we want to use them for).
2024-02-20 17:25:17 <@nirik:matrix.scrye.com> I have added a upstream patch to allow noarch builds only on specific arches... so we can say 'no i686 noarch builds' which has been hitting a lot of people.
2024-02-20 17:25:41 <@nirik:matrix.scrye.com> The kiwi plugin enablement I was planning to do at the same time. (Cloud wants to use that)
2024-02-20 17:26:01 <@nirik:matrix.scrye.com> A small patch from adamw for size on images.
2024-02-20 17:26:20 <@humaton:fedora.im> amazing!
2024-02-20 17:27:20 <@humaton:fedora.im> hum the draft build are interesting https://docs.pagure.org/koji/draft_builds/
2024-02-20 17:27:29 <@nirik:matrix.scrye.com> So, hopefully it goes well...
2024-02-20 17:27:47 <@nirik:matrix.scrye.com> yeah, but I don't see any use for us... but the ci people did have one, but I am not sure what it is.
2024-02-20 17:28:34 <@nirik:matrix.scrye.com> Also, in other koji news, FINALY!!!! tracked down the move builds issues I have been hitting for like the last 6 months.
2024-02-20 17:29:13 <@nirik:matrix.scrye.com> Turns out we had switched our httpd to multithreaded for better performance. The koji move build call is... not thread safe. ;( (well, there's now a fix upstream)
2024-02-20 17:29:32 <@nirik:matrix.scrye.com> So I moved a bunch of f32 builds off and freed up space.
2024-02-20 17:30:00 <@humaton:fedora.im> nice job
2024-02-20 17:30:16 <@nirik:matrix.scrye.com> it was all mikem really figuring it out. ;)
2024-02-20 17:30:52 <@nirik:matrix.scrye.com> I think I had something else, but trying to remember what it was...
2024-02-20 17:31:30 <@nirik:matrix.scrye.com> oh, I just got access to our new s390x lpars. Will be working on configuring/spinning up builders there this week...
2024-02-20 17:33:03 <@nirik:matrix.scrye.com> And reminder that friday is a RH recharge day... and I will be out on PTO again next monday... and next tuesday is... freeze day!
2024-02-20 17:35:29 <@jnsamyak:matrix.org> i'm also out thu hopefully no fire
2024-02-20 17:35:37 <@jnsamyak:matrix.org> can i go through some tickets real quick/
2024-02-20 17:35:38 <@jnsamyak:matrix.org> ?
2024-02-20 17:35:52 <@humaton:fedora.im> shoot
2024-02-20 17:36:05 <@jnsamyak:matrix.org> !releng 11888
2024-02-20 17:36:06 <@zodbot:fedora.im> **releng #11888** (https://pagure.io/releng/issue/11888):**Fedora 40 Mass Rebuild Tracker**
● **Opened:** a month ago by jnsamyak
● **Last Updated:** a day ago
● **Assignee:** jnsamyak
2024-02-20 17:36:10 <@jnsamyak:matrix.org> Can we close this now?
2024-02-20 17:36:43 <@jnsamyak:matrix.org> the last comment i see is an enhancement ticket which was raised back in the days
2024-02-20 17:37:09 <@nirik:matrix.scrye.com> yeah, I think we can close it... and ask anyone who has anything outstanding to file new on it?
2024-02-20 17:37:47 <@jnsamyak:matrix.org> awesome will do, but the issue is being worked on my jantil for packages that are failed to get reported
2024-02-20 17:39:14 <@jnsamyak:matrix.org> !releng 11939
2024-02-20 17:39:15 <@zodbot:fedora.im> **releng #11939** (https://pagure.io/releng/issue/11939):**Fedora 40 Mass Branching Tracker**
● **Opened:** a week ago by jnsamyak
● **Last Updated:** 5 days ago
● **Assignee:** jnsamyak
2024-02-20 17:39:29 <@jnsamyak:matrix.org> I think we can close this one as well
2024-02-20 17:39:59 <@jnsamyak:matrix.org> my action item from that ticket is to update the docs with the details missed from branching
2024-02-20 17:40:27 <@jnsamyak:matrix.org> and btw thanks folks for the effort
2024-02-20 17:40:27 <@nirik:matrix.scrye.com> oh... I noticed one other thing we missed... but it's unclear if we can just fix it.
2024-02-20 17:40:43 <@jnsamyak:matrix.org> what was it?
2024-02-20 17:41:16 <@nirik:matrix.scrye.com> https://pagure.io/pungi-fedora/blob/f40/f/fedora.conf#_1053 (and also rawhide)
2024-02-20 17:41:45 <@humaton:fedora.im> huh
2024-02-20 17:42:01 <@nirik:matrix.scrye.com> that needs to be fedora-40 / fedora-41... but I am not 100% sure that it's all... or if osbuild folks have to do something from their end to enable.
2024-02-20 17:42:44 <@jnsamyak:matrix.org> huh i think that's why that ticket issue, I will update it after the meeting :3
2024-02-20 17:42:53 <@nirik:matrix.scrye.com> I think thats causing https://bugzilla.redhat.com/show_bug.cgi?id=2264795
2024-02-20 17:43:10 <@jnsamyak:matrix.org> apologies for missing out
2024-02-20 17:43:14 <@humaton:fedora.im> Yeah I think that is the cause
2024-02-20 17:43:27 <@nirik:matrix.scrye.com> well, I mean we can change it and see I guess. ;)
2024-02-20 17:43:52 <@humaton:fedora.im> it wont change anything about the images but respins should pick it up
2024-02-20 17:44:07 <@humaton:fedora.im> oh
2024-02-20 17:44:08 <@humaton:fedora.im> nope
2024-02-20 17:44:11 <@humaton:fedora.im> we are not there yet
2024-02-20 17:44:15 <@nirik:matrix.scrye.com> hum? rawhide and branched do those every night. ;)
2024-02-20 17:44:16 <@humaton:fedora.im> so it will be fixed
2024-02-20 17:44:29 <@humaton:fedora.im> yup nevermind
2024-02-20 17:44:44 <@humaton:fedora.im> I thought we are already at rc stage
2024-02-20 17:45:12 <@nirik:matrix.scrye.com> but yeah, osbuild is a bit opaque it seems like. The plan was for some api where we could specify all the stuff we wanted to... but not sure where that is.
2024-02-20 17:45:34 <@humaton:fedora.im> I can write that down actually
2024-02-20 17:45:55 <@humaton:fedora.im> will have some meeting with guys who work on it
2024-02-20 17:46:36 <@nirik:matrix.scrye.com> They have a matrix channel here too... #image-builder:fedoraproject.org
2024-02-20 17:47:55 <@jnsamyak:matrix.org> ok looking at conf i think we need to update these references for rawhide/branched for all the artifacts right for ex here: https://pagure.io/pungi-fedora/blob/main/f/fedora.conf#_868?
2024-02-20 17:48:50 <@nirik:matrix.scrye.com> nope.
2024-02-20 17:48:56 <@humaton:fedora.im> we did the flatpaks later didnt we?
2024-02-20 17:49:12 <@nirik:matrix.scrye.com> flatpaks are special... they build against a specific platform until they are ready to move to the next one nearer release.
2024-02-20 17:49:23 <@humaton:fedora.im> ^ this
2024-02-20 17:49:39 <@jnsamyak:matrix.org> :o
2024-02-20 17:49:51 <@nirik:matrix.scrye.com> so, all the flatpaks are using f39 right now... no matter for what release... then near release they will move to f40 and we will switch that then
2024-02-20 17:50:09 <@nirik:matrix.scrye.com> (and... you know we could add some comments about all this so it's more clear in the config)
2024-02-20 17:50:44 <@jnsamyak:matrix.org> yeah +1
2024-02-20 17:51:19 <@humaton:fedora.im> also to the SOP
2024-02-20 17:51:34 <@humaton:fedora.im> this is one of the things that i remember but was never writen anywhere :/
2024-02-20 17:51:37 <@humaton:fedora.im> sorry about that
2024-02-20 17:51:51 <@jnsamyak:matrix.org> yep hopefully i'll not forget it
2024-02-20 17:51:53 <@jnsamyak:matrix.org> oh wait
2024-02-20 17:52:04 <@humaton:fedora.im> and ofc I remember it because I did the change and switched the platform after branching once :D
2024-02-20 17:52:26 <@jnsamyak:matrix.org> !action jnsamyak to add comment about flatpaks version moment in sop and pungi fedora conf
2024-02-20 17:52:43 <@jnsamyak:matrix.org> ok i have one more
2024-02-20 17:52:48 <@jnsamyak:matrix.org> !releng 11965
2024-02-20 17:52:49 <@zodbot:fedora.im> **releng #11965** (https://pagure.io/releng/issue/11965):**Collect Atomic Desktop artifacts under single directory**
● **Opened:** 2 days ago by alebastr
● **Last Updated:** a day ago
● **Assignee:** jnsamyak
2024-02-20 17:53:41 <@jnsamyak:matrix.org> so the request is to have all the variants in one directory instead of having it seperately
2024-02-20 17:53:44 <@humaton:fedora.im> I think this can be fixed by modifying the sync scripts
2024-02-20 17:54:05 <@humaton:fedora.im> there is the sync part in SOP and stage_release script
2024-02-20 17:54:22 <@humaton:fedora.im> also the rsync params are simillar in the compose script
2024-02-20 17:54:28 <@nirik:matrix.scrye.com> this also needs awareness/sign off from QE and websites.
2024-02-20 17:54:34 <@nirik:matrix.scrye.com> because they will be broken by moving things.
2024-02-20 17:54:43 <@humaton:fedora.im> sure
2024-02-20 17:55:03 <@jnsamyak:matrix.org> https://docs.fedoraproject.org/en-US/infra/release_guide/beta_RC_compose/ this is what we are doing at,
2024-02-20 17:55:05 <@jnsamyak:matrix.org> https://docs.fedoraproject.org/en-US/infra/release\_guide/beta\_RC\_compose/ this is what we are doing atm
2024-02-20 17:55:26 <@humaton:fedora.im> and branched and nightly script are doing simillar syncs
2024-02-20 17:55:37 <@nirik:matrix.scrye.com> yeah, that would need adjusting, nightly.sh would need adjusting and any release.sh would need adjusting.
2024-02-20 17:55:47 <@jnsamyak:matrix.org> https://docs.fedoraproject.org/en-US/infra/release_guide/final_RC_compose/and finally for the final compose we do
2024-02-20 17:55:48 <@humaton:fedora.im> yup
2024-02-20 17:55:59 <@jnsamyak:matrix.org> https://docs.fedoraproject.org/en-US/infra/release\_guide/final\_RC\_compose/finally for the final compose we do
2024-02-20 17:56:53 <@jnsamyak:matrix.org> what should be the first step to it? Talking to adam/qa folks and website folks for the confirmation?
2024-02-20 17:57:33 <@humaton:fedora.im> ping people in the ticket not sure whom to ping for websites
2024-02-20 17:57:47 <@humaton:fedora.im> to make sure they are aware of the changes
2024-02-20 17:58:02 <@jnsamyak:matrix.org> i tagged it as med gain/med trouble hope that is right not sure
2024-02-20 17:58:07 <@nirik:matrix.scrye.com> probibly need to schedule when it happens too.
2024-02-20 17:58:41 <@humaton:fedora.im> sooner the better so we dont have to get freezeexceptions to fix things
2024-02-20 17:59:03 <@jnsamyak:matrix.org> the beta freeze is next week 27/02 so we might need to expedite this
2024-02-20 17:59:09 <@jnsamyak:matrix.org> :3
2024-02-20 18:00:28 <@humaton:fedora.im> Yeah but its basicaly pointing it to different dir
2024-02-20 18:00:48 <@humaton:fedora.im> I think QA will have more problems with this than we
2024-02-20 18:01:40 <@nirik:matrix.scrye.com> actually... it may also need pungi.conf changes?
2024-02-20 18:02:01 <@humaton:fedora.im> I dont think so because we move stuff after the compose into places
2024-02-20 18:02:21 <@jnsamyak:matrix.org> I tagged adam, will confirm tomorrow with folks regarding the website poc
2024-02-20 18:02:53 <@nirik:matrix.scrye.com> well, ideally all of them would be composed to a subdir and we could just copy that.
2024-02-20 18:03:02 <@nirik:matrix.scrye.com> instead of copying each one to a subdir
2024-02-20 18:03:55 <@nirik:matrix.scrye.com> but we can figure it
2024-02-20 18:04:03 <@humaton:fedora.im> hm yes ideally but that means we have maybe 5-6 composes to do this and fix issues
2024-02-20 18:04:08 <@humaton:fedora.im> I am not sure about that
2024-02-20 18:04:32 <@nirik:matrix.scrye.com> well, things take as long as they do... if they miss the boat they get done later.
2024-02-20 18:04:47 <@nirik:matrix.scrye.com> anyhow, we are over time. ;)
2024-02-20 18:04:53 <@humaton:fedora.im> yep
2024-02-20 18:05:00 <@humaton:fedora.im> !endmeeting