#meeting-3:fedoraproject.org log

<@meetbot:fedora.im>

09:01:49

HTML Minutes: https://meetbot.fedoraproject.org/meeting-3_matrix_fedoraproject-org/2024-02-20/cpe-infra-releng-daily-standup.2024-02-20-08.34.html

<@humaton:fedora.im>

17:00:18

!startmeeting RELENG (2024-02-20) !meetingname releng

<@meetbot:fedora.im>

17:00:19

Meeting started at 2024-02-20 17:00:18 UTC

<@meetbot:fedora.im>

17:00:19

The Meeting name is 'RELENG (2024-02-20) !meetingname releng'

<@humaton:fedora.im>

17:00:28

!chair nirik sharkcz pbrobinson phsmoura dustymabe jednorozec

<@humaton:fedora.im>

17:00:32

!topic init process

<@nirik:matrix.scrye.com>

17:00:54

morning

<@humaton:fedora.im>

17:01:10

welcome back sir

<@patrikp:matrix.org>

17:01:12

Good evening to all.

<@humaton:fedora.im>

17:01:24

howdy

<@jnsamyak:matrix.org>

17:04:15

o/ I'm here

<@jnsamyak:matrix.org>

17:04:24

but sleeping a little :) :P

<@humaton:fedora.im>

17:04:45

nice

<@humaton:fedora.im>

17:05:31

I have just one thing for today

<@humaton:fedora.im>

17:05:49

!releng 11970

<@zodbot:fedora.im>

17:05:50

**releng #11970** (https://pagure.io/releng/issue/11970):**Create detached signatures for the butane 0.20.0 release** ● **Opened:** an hour ago by ydesouza ● **Last Updated:** 18 seconds ago ● **Assignee:** Not Assigned

<@jnsamyak:matrix.org>

17:06:00

I just tagged it

<@humaton:fedora.im>

17:06:38

to be able to work on this the person need to setup sigul client and get access to the relevant release keys

<@humaton:fedora.im>

17:06:57

does any of you have the sigul setup done? jnsamyak patrikp

<@jnsamyak:matrix.org>

17:07:09

also this need to be run as root of some machine?

<@humaton:fedora.im>

17:07:16

nope

<@jnsamyak:matrix.org>

17:07:29

jednorozec: nope tried but i dont remember why i was not able to do it

<@zodbot:fedora.im>

17:08:57

jnsamyak has already given cookies to kevin during the F39 timeframe

<@patrikp:matrix.org>

17:09:35

I don't have it set up.

<@patrikp:matrix.org>

17:09:49

Do we have some SOP with steps?

<@nirik:matrix.scrye.com>

17:10:06

There's an sop in the old docs, but not sure it was moved over yet to new one.

<@nirik:matrix.scrye.com>

17:10:35

https://docs.pagure.org/releng/sop_sigul_client_setup.html

<@jnsamyak:matrix.org>

17:10:40

https://docs.fedoraproject.org/en-US/infra/releng_misc_guide/sop_sigul_client_setup/#_sigul_client_setup

<@jnsamyak:matrix.org>

17:10:52

this is here^

<@nirik:matrix.scrye.com>

17:11:00

ok, great.

<@jnsamyak:matrix.org>

17:11:00

but i remember now

<@jnsamyak:matrix.org>

17:11:07

what was the problem

<@jnsamyak:matrix.org>

17:11:22

getting these: ~/.fedora.cert ~/.fedora-server-ca.cert ~/.fedora-upload-ca.cert

<@zodbot:fedora.im>

17:11:24

patrikp gave a cookie to kevin. They now have 654 cookies, 17 of which were obtained in the Fedora 39 release cycle

<@nirik:matrix.scrye.com>

17:11:30

yeah, that part is wrong. ;)

<@nirik:matrix.scrye.com>

17:11:46

we have a seperate ca for sigul... we need to issue you a cert from it.

<@nirik:matrix.scrye.com>

17:12:17

(also the phx2 in those hostnames is... really wrong. ;)

<@jnsamyak:matrix.org>

17:12:22

ahaa i think that's why I tried this with jednorozec but didnt go anywhere

<@humaton:fedora.im>

17:12:32

yeah soe hostnames are still decade old

<@humaton:fedora.im>

17:13:02

nope jnsamyak I gave you ca cert not mine cert

<@jnsamyak:matrix.org>

17:13:05

yeah hostname are obsolete because of migrating it from old docs i'm updating most of them as i go through the release process but all doc help welcome :D

<@nirik:matrix.scrye.com>

17:13:19

also, usually you just run it on bodhi-backend01... since the client is installed there and setup, etc... I don't know if it could work from a client machine.

<@jnsamyak:matrix.org>

17:13:40

yep^ that

<@humaton:fedora.im>

17:13:52

It can but its complicated and not worthed

<@humaton:fedora.im>

17:14:17

what I mean by setup the client is to have the certs and everything in ~ on bodhi-backend

<@humaton:fedora.im>

17:15:54

anyhow can you issue certs for guys nirik ? or can you show me how?

<@nirik:matrix.scrye.com>

17:17:03

sure... either. ;) It's in ansible-private/files/sigul-ca/production (and staging) and it's a normal easyrsa setup...

<@humaton:fedora.im>

17:17:14

ahh

<@humaton:fedora.im>

17:17:36

let tomorrow be the certification day!

<@nirik:matrix.scrye.com>

17:17:42

<@humaton:fedora.im>

17:18:32

ok I have one more

<@nirik:matrix.scrye.com>

17:18:38

let me know if you run into any problems with it...

<@humaton:fedora.im>

17:18:47

sure will do

<@humaton:fedora.im>

17:18:50

!releng 11969

<@zodbot:fedora.im>

17:18:51

**releng #11969** (https://pagure.io/releng/issue/11969):**Fedora Atomic Desktops composes are failing for F40 & Rawhide as they include "external" repos** ● **Opened:** 6 hours ago by siosm ● **Last Updated:** Never ● **Assignee:** Not Assigned

<@humaton:fedora.im>

17:19:20

some branching cleanup needed maybe?

<@nirik:matrix.scrye.com>

17:20:53

could be.

<@nirik:matrix.scrye.com>

17:23:27

I do have a few smaller items tho

<@humaton:fedora.im>

17:23:42

go on nirik

<@humaton:fedora.im>

17:23:51

we will look into the composes tmrw

<@nirik:matrix.scrye.com>

17:24:06

ok. I have scheduled an outage tomorrow for upgrading koji to 1.34.0... this brings in a lot of changes.

<@nirik:matrix.scrye.com>

17:24:46

The scheduler is completely re-implemented. There's 'draft builds' (which I plan on denying being tagged into anything until we can decide what we want to use them for).

<@nirik:matrix.scrye.com>

17:25:17

I have added a upstream patch to allow noarch builds only on specific arches... so we can say 'no i686 noarch builds' which has been hitting a lot of people.

<@nirik:matrix.scrye.com>

17:25:41

The kiwi plugin enablement I was planning to do at the same time. (Cloud wants to use that)

<@nirik:matrix.scrye.com>

17:26:01

A small patch from adamw for size on images.

<@humaton:fedora.im>

17:26:20

amazing!

<@humaton:fedora.im>

17:27:20

hum the draft build are interesting https://docs.pagure.org/koji/draft_builds/

<@nirik:matrix.scrye.com>

17:27:29

So, hopefully it goes well...

<@nirik:matrix.scrye.com>

17:27:47

yeah, but I don't see any use for us... but the ci people did have one, but I am not sure what it is.

<@nirik:matrix.scrye.com>

17:28:34

Also, in other koji news, FINALY!!!! tracked down the move builds issues I have been hitting for like the last 6 months.

<@nirik:matrix.scrye.com>

17:29:13

Turns out we had switched our httpd to multithreaded for better performance. The koji move build call is... not thread safe. ;( (well, there's now a fix upstream)

<@nirik:matrix.scrye.com>

17:29:32

So I moved a bunch of f32 builds off and freed up space.

<@humaton:fedora.im>

17:30:00

nice job

<@nirik:matrix.scrye.com>

17:30:16

it was all mikem really figuring it out. ;)

<@nirik:matrix.scrye.com>

17:30:52

I think I had something else, but trying to remember what it was...

<@nirik:matrix.scrye.com>

17:31:30

oh, I just got access to our new s390x lpars. Will be working on configuring/spinning up builders there this week...

<@nirik:matrix.scrye.com>

17:33:03

And reminder that friday is a RH recharge day... and I will be out on PTO again next monday... and next tuesday is... freeze day!

<@jnsamyak:matrix.org>

17:35:29

i'm also out thu hopefully no fire

<@jnsamyak:matrix.org>

17:35:37

can i go through some tickets real quick/

<@jnsamyak:matrix.org>

17:35:38

<@humaton:fedora.im>

17:35:52

shoot

<@jnsamyak:matrix.org>

17:36:05

!releng 11888

<@zodbot:fedora.im>

17:36:06

**releng #11888** (https://pagure.io/releng/issue/11888):**Fedora 40 Mass Rebuild Tracker** ● **Opened:** a month ago by jnsamyak ● **Last Updated:** a day ago ● **Assignee:** jnsamyak

<@jnsamyak:matrix.org>

17:36:10

Can we close this now?

<@jnsamyak:matrix.org>

17:36:43

the last comment i see is an enhancement ticket which was raised back in the days

<@nirik:matrix.scrye.com>

17:37:09

yeah, I think we can close it... and ask anyone who has anything outstanding to file new on it?

<@jnsamyak:matrix.org>

17:37:47

awesome will do, but the issue is being worked on my jantil for packages that are failed to get reported

<@jnsamyak:matrix.org>

17:39:14

!releng 11939

<@zodbot:fedora.im>

17:39:15

**releng #11939** (https://pagure.io/releng/issue/11939):**Fedora 40 Mass Branching Tracker** ● **Opened:** a week ago by jnsamyak ● **Last Updated:** 5 days ago ● **Assignee:** jnsamyak

<@jnsamyak:matrix.org>

17:39:29

I think we can close this one as well

<@jnsamyak:matrix.org>

17:39:59

my action item from that ticket is to update the docs with the details missed from branching

<@jnsamyak:matrix.org>

17:40:27

and btw thanks folks for the effort

<@nirik:matrix.scrye.com>

17:40:27

oh... I noticed one other thing we missed... but it's unclear if we can just fix it.

<@jnsamyak:matrix.org>

17:40:43

what was it?

<@nirik:matrix.scrye.com>

17:41:16

https://pagure.io/pungi-fedora/blob/f40/f/fedora.conf#_1053 (and also rawhide)

<@humaton:fedora.im>

17:41:45

huh

<@nirik:matrix.scrye.com>

17:42:01

that needs to be fedora-40 / fedora-41... but I am not 100% sure that it's all... or if osbuild folks have to do something from their end to enable.

<@jnsamyak:matrix.org>

17:42:44

huh i think that's why that ticket issue, I will update it after the meeting :3

<@nirik:matrix.scrye.com>

17:42:53

I think thats causing https://bugzilla.redhat.com/show_bug.cgi?id=2264795

<@jnsamyak:matrix.org>

17:43:10

apologies for missing out

<@humaton:fedora.im>

17:43:14

Yeah I think that is the cause

<@nirik:matrix.scrye.com>

17:43:27

well, I mean we can change it and see I guess. ;)

<@humaton:fedora.im>

17:43:52

it wont change anything about the images but respins should pick it up

<@humaton:fedora.im>

17:44:07

<@humaton:fedora.im>

17:44:08

nope

<@humaton:fedora.im>

17:44:11

we are not there yet

<@nirik:matrix.scrye.com>

17:44:15

hum? rawhide and branched do those every night. ;)

<@humaton:fedora.im>

17:44:16

so it will be fixed

<@humaton:fedora.im>

17:44:29

yup nevermind

<@humaton:fedora.im>

17:44:44

I thought we are already at rc stage

<@nirik:matrix.scrye.com>

17:45:12

but yeah, osbuild is a bit opaque it seems like. The plan was for some api where we could specify all the stuff we wanted to... but not sure where that is.

<@humaton:fedora.im>

17:45:34

I can write that down actually

<@humaton:fedora.im>

17:45:55

will have some meeting with guys who work on it

<@nirik:matrix.scrye.com>

17:46:36

They have a matrix channel here too... #image-builder:fedoraproject.org

<@jnsamyak:matrix.org>

17:47:55

ok looking at conf i think we need to update these references for rawhide/branched for all the artifacts right for ex here: https://pagure.io/pungi-fedora/blob/main/f/fedora.conf#_868?

<@nirik:matrix.scrye.com>

17:48:50

nope.

<@humaton:fedora.im>

17:48:56

we did the flatpaks later didnt we?

<@nirik:matrix.scrye.com>

17:49:12

flatpaks are special... they build against a specific platform until they are ready to move to the next one nearer release.

<@humaton:fedora.im>

17:49:23

^ this

<@jnsamyak:matrix.org>

17:49:39

<@nirik:matrix.scrye.com>

17:49:51

so, all the flatpaks are using f39 right now... no matter for what release... then near release they will move to f40 and we will switch that then

<@nirik:matrix.scrye.com>

17:50:09

(and... you know we could add some comments about all this so it's more clear in the config)

<@jnsamyak:matrix.org>

17:50:44

yeah +1

<@humaton:fedora.im>

17:51:19

also to the SOP

<@humaton:fedora.im>

17:51:34

this is one of the things that i remember but was never writen anywhere :/

<@humaton:fedora.im>

17:51:37

sorry about that

<@jnsamyak:matrix.org>

17:51:51

yep hopefully i'll not forget it

<@jnsamyak:matrix.org>

17:51:53

oh wait

<@humaton:fedora.im>

17:52:04

and ofc I remember it because I did the change and switched the platform after branching once :D

<@jnsamyak:matrix.org>

17:52:26

!action jnsamyak to add comment about flatpaks version moment in sop and pungi fedora conf

<@jnsamyak:matrix.org>

17:52:43

ok i have one more

<@jnsamyak:matrix.org>

17:52:48

!releng 11965

<@zodbot:fedora.im>

17:52:49

**releng #11965** (https://pagure.io/releng/issue/11965):**Collect Atomic Desktop artifacts under single directory** ● **Opened:** 2 days ago by alebastr ● **Last Updated:** a day ago ● **Assignee:** jnsamyak

<@jnsamyak:matrix.org>

17:53:41

so the request is to have all the variants in one directory instead of having it seperately

<@humaton:fedora.im>

17:53:44

I think this can be fixed by modifying the sync scripts

<@humaton:fedora.im>

17:54:05

there is the sync part in SOP and stage_release script

<@humaton:fedora.im>

17:54:22

also the rsync params are simillar in the compose script

<@nirik:matrix.scrye.com>

17:54:28

this also needs awareness/sign off from QE and websites.

<@nirik:matrix.scrye.com>

17:54:34

because they will be broken by moving things.

<@humaton:fedora.im>

17:54:43

sure

<@jnsamyak:matrix.org>

17:55:03

https://docs.fedoraproject.org/en-US/infra/release_guide/beta_RC_compose/ this is what we are doing at,

<@jnsamyak:matrix.org>

17:55:05

https://docs.fedoraproject.org/en-US/infra/release\_guide/beta\_RC\_compose/ this is what we are doing atm

<@humaton:fedora.im>

17:55:26

and branched and nightly script are doing simillar syncs

<@nirik:matrix.scrye.com>

17:55:37

yeah, that would need adjusting, nightly.sh would need adjusting and any release.sh would need adjusting.

<@jnsamyak:matrix.org>

17:55:47

https://docs.fedoraproject.org/en-US/infra/release_guide/final_RC_compose/and finally for the final compose we do

<@humaton:fedora.im>

17:55:48

yup

<@jnsamyak:matrix.org>

17:55:59

https://docs.fedoraproject.org/en-US/infra/release\_guide/final\_RC\_compose/finally for the final compose we do

<@jnsamyak:matrix.org>

17:56:53

what should be the first step to it? Talking to adam/qa folks and website folks for the confirmation?

<@humaton:fedora.im>

17:57:33

ping people in the ticket not sure whom to ping for websites

<@humaton:fedora.im>

17:57:47

to make sure they are aware of the changes

<@jnsamyak:matrix.org>

17:58:02

i tagged it as med gain/med trouble hope that is right not sure

<@nirik:matrix.scrye.com>

17:58:07

probibly need to schedule when it happens too.

<@humaton:fedora.im>

17:58:41

sooner the better so we dont have to get freezeexceptions to fix things

<@jnsamyak:matrix.org>

17:59:03

the beta freeze is next week 27/02 so we might need to expedite this

<@jnsamyak:matrix.org>

17:59:09

<@humaton:fedora.im>

18:00:28

Yeah but its basicaly pointing it to different dir

<@humaton:fedora.im>

18:00:48

I think QA will have more problems with this than we

<@nirik:matrix.scrye.com>

18:01:40

actually... it may also need pungi.conf changes?

<@humaton:fedora.im>

18:02:01

I dont think so because we move stuff after the compose into places

<@jnsamyak:matrix.org>

18:02:21

I tagged adam, will confirm tomorrow with folks regarding the website poc

<@nirik:matrix.scrye.com>

18:02:53

well, ideally all of them would be composed to a subdir and we could just copy that.

<@nirik:matrix.scrye.com>

18:03:02

instead of copying each one to a subdir

<@nirik:matrix.scrye.com>

18:03:55

but we can figure it

<@humaton:fedora.im>

18:04:03

hm yes ideally but that means we have maybe 5-6 composes to do this and fix issues

<@humaton:fedora.im>

18:04:08

I am not sure about that

<@nirik:matrix.scrye.com>

18:04:32

well, things take as long as they do... if they miss the boat they get done later.

<@nirik:matrix.scrye.com>

18:04:47

anyhow, we are over time. ;)

<@humaton:fedora.im>

18:04:53

yep

<@humaton:fedora.im>

18:05:00

!endmeeting