#meeting:fedoraproject.org log

<@sgallagh:fedora.im>

17:01:06

!startmeeting Fedora ELN (2024-02-09)

<@meetbot:fedora.im>

17:01:07

Meeting started at 2024-02-09 17:01:06 UTC

<@meetbot:fedora.im>

17:01:07

The Meeting name is 'Fedora ELN (2024-02-09)'

<@sgallagh:fedora.im>

17:01:09

!meetingname eln

<@sgallagh:fedora.im>

17:01:48

!topic Init Process

<@sgallagh:fedora.im>

17:01:50

!hi

<@zodbot:fedora.im>

17:01:51

Stephen Gallagher (sgallagh) - he / him / his

<@smooge:fedora.im>

17:03:24

hello

<@tdawson:fedora.im>

17:03:30

Howdy

<@nhanlon:beeper.com>

17:04:19

!hi

<@zodbot:fedora.im>

17:04:21

Neil Hanlon (neil) - he / him / his

<@nhanlon:beeper.com>

17:04:34

g'day folks

<@sgallagh:fedora.im>

17:05:11

Hello my friends!

<@jonathanspw:fedora.im>

17:05:21

!hi

<@yselkowitz:fedora.im>

17:05:22

!hi

<@zodbot:fedora.im>

17:05:22

Jonathan Wright (jonathanspw)

<@zodbot:fedora.im>

17:05:23

Yaakov Selkowitz (yselkowitz)

<@sgallagh:fedora.im>

17:05:23

And welcome to another edition of "What could possibly go wrong?"

<@sgallagh:fedora.im>

17:05:59

So, I'll jump right into our main topic for today

<@sgallagh:fedora.im>

17:06:54

!topic Forking EL 10

<@sgallagh:fedora.im>

17:07:23

(For those of you who watched "The Good Place"... that phrasing was intentional.)

<@sgallagh:fedora.im>

17:08:29

!info As previously announced, Tuesday February 13th marks the end of Fedora ELN syncing to CentOS Stream 10

<@sgallagh:fedora.im>

17:09:19

From the ELN side, we have a number of steps that will be taken to move on towards RHEL 11

<@sgallagh:fedora.im>

17:09:48

!link https://docs.fedoraproject.org/en-US/eln/branching/

<@sgallagh:fedora.im>

17:10:40

We are currently in the process of cleaning up the last remaining fallout from the F40 mass-rebuild.

<@sgallagh:fedora.im>

17:11:14

There are two main tickets for tracking these issues:

<@sgallagh:fedora.im>

17:11:20

!link https://github.com/fedora-eln/eln/issues/180

<@sgallagh:fedora.im>

17:11:36

!link https://github.com/fedora-eln/eln/issues/179

<@sgallagh:fedora.im>

17:11:54

Sorry, three:

<@sgallagh:fedora.im>

17:11:57

!link https://github.com/fedora-eln/eln/issues/176

<@sgallagh:fedora.im>

17:12:29

Those are "Packages in the ELN set that are FTBFS in Rawhide", "Packages in the ELN set that are FTBFS in CentOS Stream 10" and "Packages in the ELN set that are FTBFS in ELN",

<@sgallagh:fedora.im>

17:12:34

(respectively)

<@sgallagh:fedora.im>

17:13:05

In an ideal world, tickets 176 and 180 will be finished by Monday.

<@sgallagh:fedora.im>

17:13:15

In our world... we need help.

<@nhanlon:beeper.com>

17:15:57

i volunteer Troy

<@sgallagh:fedora.im>

17:16:03

yselkowitz: Could you run us through the non-leaf remainder?

<@yselkowitz:fedora.im>

17:16:16

from #176:

<@yselkowitz:fedora.im>

17:16:33

asio: waiting on a gcc update now in testing

<@yselkowitz:fedora.im>

17:16:55

atlas: is a mess with the new gcc, but we're looking to make it a leaf

<@yselkowitz:fedora.im>

17:17:04

gtk-doc: needs help

<@yselkowitz:fedora.im>

17:17:16

fortunately that's not gcc related :-)

<@yselkowitz:fedora.im>

17:17:27

https://gitlab.gnome.org/GNOME/gtk-doc/-/issues/150

<@yselkowitz:fedora.im>

17:17:45

openblas: https://github.com/OpenMathLib/OpenBLAS/issues/4475

<@yselkowitz:fedora.im>

17:18:38

perl-Tk: also a mess with the new gcc, and upstream is dead

<@yselkowitz:fedora.im>

17:19:07

squid is a leaf, so worst case we skip it

<@yselkowitz:fedora.im>

17:19:24

zziplib: https://bugzilla.redhat.com/show_bug.cgi?id=2256917 with links to github inside

<@yselkowitz:fedora.im>

17:20:11

from #180

<@yselkowitz:fedora.im>

17:20:37

jackson-*: something weird going on with (x)mvn there

<@sgallagh:fedora.im>

17:20:53

Do we have anyone in attendance who will admit to understanding maven?

<@sgallagh:fedora.im>

17:21:06

Because it's black magic as far as I'm concerned

<@yselkowitz:fedora.im>

17:21:18

jss has a PR, and the rrdtool issue is explained in the ticket, I won't rehash here

<@nhanlon:beeper.com>

17:21:36

I can try and poke at gtk-doc. it sounds like my brand of self flagellation.

<@yselkowitz:fedora.im>

17:21:52

from #179:

<@yselkowitz:fedora.im>

17:22:03

thanks Neil Hanlon

<@yselkowitz:fedora.im>

17:22:09

from #179:

<@sgallagh:fedora.im>

17:22:15

!action Neil Hanlon to look into gtk-doc

<@yselkowitz:fedora.im>

17:22:34

gmp, libkcapi, and nettle have very odd fips-related errors only in c10s, hasn't been an issue in eln

<@yselkowitz:fedora.im>

17:22:55

libecpg and tftp appear to be race conditions in the makefiles

<@sgallagh:fedora.im>

17:23:07

Our suspicion on those is that it may be related to the kernel running on the build hosts being RHEL 8 rather than recent Fedora kernels

<@sgallagh:fedora.im>

17:23:20

those being gmp, libkcapi and nettle

<@sgallagh:fedora.im>

17:23:56

I'll come back around to that later; I have some thoughts for ELN11

<@yselkowitz:fedora.im>

17:23:58

most of the rest are test failures due to infrastructure pecularities (e.g. rhel 8 kernel) or flakiness, although retries haven't helped yet

<@yselkowitz:fedora.im>

17:24:25

tracker-miners needs a meson configure patch: https://gitlab.gnome.org/GNOME/tracker-miners/-/issues/300

<@yselkowitz:fedora.im>

17:24:49

I think that pretty much covers it, happy to answer any questions

<@sgallagh:fedora.im>

17:27:11

If anyone has cycles they can dig out of the couch cushions to help push on any of these, that would be immensely helpful

<@sgallagh:fedora.im>

17:30:19

So it goes...

<@sgallagh:fedora.im>

17:30:47

OK, I have one other minor topic that's mostly directed at the folks here who know something about Fedora Infrastructure

<@sgallagh:fedora.im>

17:31:01

!topic RHEL-like Koji Builders

<@sgallagh:fedora.im>

17:31:42

What would it take to add some Koji builder instances that share a configuration with CentOS Stream/RHEL for ELN?

<@sgallagh:fedora.im>

17:32:27

In the interest of "shifting left" and identifying failures earlier (such as those where tests might fail if the kernel on the builder is too old)

<@sgallagh:fedora.im>

17:33:02

I'm kind of directing this question at Stephen J Smoogen, who is no longer on that team but who I hope still remembers a few things about it :)

<@smooge:fedora.im>

17:33:44

ugh. I am in a meeting but can help in a bit

<@sgallagh:fedora.im>

17:34:19

It's not urgent, just something I want to explore after we fork and can start looking forward to RHEL 11

<@sgallagh:fedora.im>

17:35:30

!topic Open Floor

<@smooge:fedora.im>

17:36:26

so the main issues would be needed to work through is that the RHEL builders are running RHEL in kernel land but the Fedora builders are running the Fedora kernel.

<@smooge:fedora.im>

17:37:28

if we skip a 1:1 kernel issue, what would be needed to put into koji configs outside of what is done already?

<@sgallagh:fedora.im>

17:38:13

Sorry, I don't understand "if we skip a 1:1 kernel issue".

<@smooge:fedora.im>

17:38:29

so I may have misread by what you meant as a kernel too old

<@sgallagh:fedora.im>

17:39:28

Our issue is that some packages have test suites (or `%configure` suites) that require kernel features that work on Fedora, but once we sync to CentOS Stream they fail

<@smooge:fedora.im>

17:39:39

are you meaning you would want specific builders which were RHEL only and a chain of builds (like koschei) done on them?

<@sgallagh:fedora.im>

17:40:33

We'd like to be able to catch those issues earlier (in Fedora ELN) since we won't turn on the sync for CentOS Stream 11 for at least a year.

<@sgallagh:fedora.im>

17:42:08

Also the feedback loop is much tigher when things fail in ELN vs. after the syncing to CS

<@smooge:fedora.im>

17:42:08

So I think you would need to work out if Fedora has the resources for builders which are not 'the same' for whatever arches you needed this built on. And i mean in physical hardware and mental space for kevin :)

<@nhanlon:beeper.com>

17:42:36

surely we can download more ram for kevin? :)

<@sgallagh:fedora.im>

17:43:27

That's not how this works. That's not how _any_ of this works!

<@sgallagh:fedora.im>

17:44:54

OK, as previously noted, this is just an exploratory discussion right now.

<@smooge:fedora.im>

17:45:03

understood.

<@sgallagh:fedora.im>

17:45:53

And maybe the answer ends up being "turn on the CS sync earlier".

<@smooge:fedora.im>

17:46:04

I would say as an explatory discussion.. why use koji and why use Fedora space but instead use something in the cloud which rebuilds all the items in ELN for maybe one or two arches in a continuous method.

<@smooge:fedora.im>

17:47:03

a dedicated COPR or something similar tied with something like Koshei might get a proof of concept going

<@sgallagh:fedora.im>

17:47:37

I don't think we need a continuous rebuild. That seems like overkill (and a lot of electricity)

<@smooge:fedora.im>

17:47:44

longer term something better in koji could be worked out as resources are made available

<@smooge:fedora.im>

17:47:52

well even a once a week build

<@smooge:fedora.im>

17:49:15

you can then set things up in the 'hardware' for what you want really tested as in 'my build system's libs needs work'

<@sgallagh:fedora.im>

17:49:31

I do have one more thing to bring up before we run out of time.

<@sgallagh:fedora.im>

17:49:39

!topic OpenSSL 3.2

<@sgallagh:fedora.im>

17:50:15

As some of you may have seen, OpenSSL 3.2 hit Fedora Rawhide yesterday. We already have at least two reports of breakage there.

<@sgallagh:fedora.im>

17:51:10

I'd like to recruit the members of this SIG to try and test as many things as possible in Rawhide that rely on OpenSSL today.

<@sgallagh:fedora.im>

17:52:04

Currently, we've blocked it from landing in Fedora ELN out of an abundance of caution right before the fork, but if we can get to a high level of confidence in it by Monday, we might consider letting it in

<@sgallagh:fedora.im>

17:53:54

Can each of you spare 30 minutes or more today to poke at things in Rawhide?

<@sgallagh:fedora.im>

17:55:41

OK, last item of business: next meeting.

<@sgallagh:fedora.im>

17:55:53

!topic Next Meeting

<@sgallagh:fedora.im>

17:56:48

I will be unable to attend the next meeting (two weeks from today), but given the big happenings over that period, I'd appreciate if someone else could step up and hold it in my absence.

<@tdawson:fedora.im>

17:57:13

The next meeting is Feb. 23, correct?

<@sgallagh:fedora.im>

17:57:22

Correct

<@tdawson:fedora.im>

17:57:27

That is a Red Hat Recharge day

<@yselkowitz:fedora.im>

17:57:54

oh that's funny timing

<@smooge:fedora.im>

17:58:08

I would say do the next week

<@tdawson:fedora.im>

17:58:10

I'm not saying we need to cancel it ... I just noticed it last night.

<@smooge:fedora.im>

17:58:20

I think this is a definite 'cancel'

<@smooge:fedora.im>

17:58:33

but I have the luxury of being able to say that :)

<@sgallagh:fedora.im>

17:59:47

Hmm, I need to fix my PTO; I must have put it in before they picked that date because it's costing my a PTO day...

<@sgallagh:fedora.im>

18:00:09

Anyway, I guess we'll skip and consider having one the following week based on interest

<@sgallagh:fedora.im>

18:00:49

!info There will be no meeting on Feb. 23rd. We may have an out-of-schedule meeting on Mar. 1 if interest is high.

<@sgallagh:fedora.im>

18:00:58

Alright, we're at time. Thanks for coming, folks!

<@sgallagh:fedora.im>

18:01:03

!endmeeting