#meeting:fedoraproject.org log

<@yselkowitz:fedora.im>

19:01:00

!startmeeting ELN SIG 27 March '25

<@meetbot:fedora.im>

19:01:02

Meeting started at 2025-03-27 19:01:00 UTC

<@meetbot:fedora.im>

19:01:02

The Meeting name is 'ELN SIG 27 March '25'

<@yselkowitz:fedora.im>

19:01:09

!meetingname eln

<@meetbot:fedora.im>

19:01:10

The Meeting Name is now eln

<@yselkowitz:fedora.im>

19:01:16

!topic Init process

<@tdawson:fedora.im>

19:01:27

!hi

<@zodbot:fedora.im>

19:01:28

Troy Dawson (tdawson)

<@davide:cavalca.name>

19:01:39

!hi

<@zodbot:fedora.im>

19:01:41

Davide Cavalca (dcavalca) - he / him / his

<@salimma:fedora.im>

19:01:41

!hi

<@zodbot:fedora.im>

19:01:43

Michel Lind (salimma) - he / him / his

<@conan_kudo:matrix.org>

19:03:53

!hi

<@zodbot:fedora.im>

19:03:55

Neal Gompa (ngompa) - he / him / his

<@rcallicotte:fedora.im>

19:04:11

!hi

<@zodbot:fedora.im>

19:04:12

Robby Callicotte (rcallicotte) - he / him / his

<@yselkowitz:fedora.im>

19:04:41

welcome Robby Callicotte do you want to introduce yourself and your interest in ELN?

<@salimma:fedora.im>

19:04:54

ohai Robby

<@rcallicotte:fedora.im>

19:06:19

Thank you yselkowitz . I just dropped by to listen in. I'm a member of CentOS Alt Images SIG and at my $DAYJOB I handle the OS stuff (planning etc)

<@yselkowitz:fedora.im>

19:06:51

np welcome

<@yselkowitz:fedora.im>

19:06:54

let's get started

<@yselkowitz:fedora.im>

19:07:03

!topic New Business

<@yselkowitz:fedora.im>

19:07:40

!link https://src.fedoraproject.org/rpms/python-zope-i18nmessageid/pull-request/3

<@yselkowitz:fedora.im>

19:08:10

thanks for merging Michel Lind UTC-6

<@yselkowitz:fedora.im>

19:08:18

oh and there's a first build

<@gmoro:matrix.org>

19:08:20

!hi

<@zodbot:fedora.im>

19:08:22

Guilherme Moro (guilhermemoro)

<@yselkowitz:fedora.im>

19:08:38

that leaves only 3 ELN and 1 ELN Extras F42FTBFS left

<@salimma:fedora.im>

19:08:51

yup, I was just building it. do we need it for F42 too or just rawhide?

<@yselkowitz:fedora.im>

19:09:41

*I* only need it for rawhide and ELN, but it didn't rebuild for F42

<@sgallagh:fedora.im>

19:09:52

!hi

<@zodbot:fedora.im>

19:09:53

Stephen Gallagher (sgallagh) - he / him / his

<@yselkowitz:fedora.im>

19:10:10

!link https://github.com/fedora-eln/eln/issues/238

<@sgallagh:fedora.im>

19:10:10

I'm sort of here, but busy with other things. Please ping directly if you need my input.

<@yselkowitz:fedora.im>

19:10:53

Michel Lind UTC-6 Davide Cavalca this involves you iirc

<@davide:cavalca.name>

19:11:26

So this one is needed for Chef, among other things

<@davide:cavalca.name>

19:11:43

I agree that we should probably drop the obsoletes on the RHEL side

<@yselkowitz:fedora.im>

19:12:17

can we bump the release to 100+, and is the Requires: libxcrypt needed?

<@davide:cavalca.name>

19:12:54

I'm pretty sure the requires is needed, but it's been a while since I've poked at it. [@michel:one.ems.host](https://matrix.to/#/@michel:one.ems.host) do you remember?

<@conan_kudo:matrix.org>

19:13:09

it is needed

<@davide:cavalca.name>

19:13:16

I don't like the idea of bumping the release. It's kind of a bad precedent and would just paper over the problem

<@davide:cavalca.name>

19:13:27

This does need to be kept in sync with libscrypt

<@davide:cavalca.name>

19:13:38

This does need to be kept in sync with libxcrypt

<@conan_kudo:matrix.org>

19:13:50

the compat library is built on top of the regular one

<@conan_kudo:matrix.org>

19:13:59

IIRC

<@davide:cavalca.name>

19:14:03

Yeah

<@salimma:fedora.im>

19:14:23

one sec

<@yselkowitz:fedora.im>

19:14:42

but e.g. https://koji.fedoraproject.org/koji/rpminfo?rpmID=41827501 shows no dep on `libxcrypt.so.2`?

<@yselkowitz:fedora.im>

19:15:06

but e.g. https://koji.fedoraproject.org/koji/rpminfo?rpmID=41827501 shows no dep on `libcrypt.so.2`?

<@conan_kudo:matrix.org>

19:15:07

it's been a while since I looked at it, but IIRC it's a split out of some functions that still may call into the main library

<@salimma:fedora.im>

19:15:14

I think we try to keep it in sync with the RHEL package because the spec is based on it, not sure if it can be made independent -- I'll have to take a look

<@conan_kudo:matrix.org>

19:15:21

that might have changed in newer versions though

<@conan_kudo:matrix.org>

19:15:49

if it's done through dlopen then it won't show up that way

<@conan_kudo:matrix.org>

19:16:05

same reason I have to depend on sdl3 for sdl2-compat manually

<@salimma:fedora.im>

19:16:08

it's basically a standard '-epel' package - stream dropped the compat stuff, we ship only the compat stuff. like iptables-legacy being shipped in iptables-epel

<@yselkowitz:fedora.im>

19:16:16

ugh ok but is that what it's doing?

<@yselkowitz:fedora.im>

19:16:23

dlopen I mean

<@salimma:fedora.im>

19:16:28

I'll need to check again, I don't think any of us remember

<@conan_kudo:matrix.org>

19:16:35

I think that's what it did originally, I don't know if it still does now

<@salimma:fedora.im>

19:16:39

fwiw we've been trying to get Chef to fix this since... 2019. almost 6 years now

<@conan_kudo:matrix.org>

19:16:46

they may have fully split things apart since

<@salimma:fedora.im>

19:16:53

so ... I also would dearly love this to die. as I do *all* my -epel packages

<@davide:cavalca.name>

19:17:08

It's not just chef to be clear, tons of 3rd party stuff needs this

<@conan_kudo:matrix.org>

19:17:11

wow that's... a long time

<@yselkowitz:fedora.im>

19:17:57

yeah well the libxcrypt maintainer really didn't want -compat in RHEL 10 even, but VDDK is in the same boat

<@salimma:fedora.im>

19:18:13

yeah. the company imploded post acquisition... you're familiar with how that goes

<@davide:cavalca.name>

19:18:26

https://github.com/chef/omnibus/issues/890 fyi

<@salimma:fedora.im>

19:18:34

also they do those clowny omnibus style RPMs and pretended that insulates them from system dependencies, which is a fiction

<@salimma:fedora.im>

19:18:48

and turned off RPM requires generator so we find out the hard way, the RPM happily installs

<@davide:cavalca.name>

19:18:54

yeah I was about to say, the "right" solution is to grudgingly accept this is still needed and put it back into RHEL

<@conan_kudo:matrix.org>

19:19:43

the juice isn't really worth the squeeze here

<@rcallicotte:fedora.im>

19:19:50

seems to be the norm now with some automation tools

<@yselkowitz:fedora.im>

19:19:50

and that's what happened for 10, but they really really seriously this time don't want it in 11, you know how it goes

<@conan_kudo:matrix.org>

19:20:05

GitLab made this a popular approach

<@davide:cavalca.name>

19:20:06

yep, I know

<@conan_kudo:matrix.org>

19:20:10

it's super annoying

<@conan_kudo:matrix.org>

19:20:46

this whole thing reminds me that Puppet still requires `lsb_release(1)`

<@conan_kudo:matrix.org>

19:21:00

it's not my problem anymore, but I feel bad for everyone who still has to deal with it

<@yselkowitz:fedora.im>

19:21:17

can you guys look into the -compat->libxcrypt dependency side of this at least?

<@davide:cavalca.name>

19:21:29

yeah, we'll follow up on that

<@salimma:fedora.im>

19:21:35

yeah but if Chef does not need it we can orphan it and let someone who needs it more own it

<@salimma:fedora.im>

19:21:41

will do

<@yselkowitz:fedora.im>

19:21:47

!action Davide and Michel to follow up on libxcrypt-compat

<@salimma:fedora.im>

19:21:54

Salt just went this route too, which is sad

<@yselkowitz:fedora.im>

19:22:49

!link https://github.com/fedora-eln/eln/issues/239

<@yselkowitz:fedora.im>

19:23:44

this came up a couple weeks ago but didn't find a quick resolution so here we are

<@conan_kudo:matrix.org>

19:24:08

the only fix is to adjust upstream code to handle nmap-ncat

<@yselkowitz:fedora.im>

19:24:09

cloud-init 25.1 added a dep on netcat's `nc` which is incompatible with the nmap-ncat `nc` in RHEL and ELN

<@conan_kudo:matrix.org>

19:24:32

there is no quick resolution if nmap-ncat is missing something that cloud-init requires

<@conan_kudo:matrix.org>

19:24:36

but I can go look

<@yselkowitz:fedora.im>

19:25:36

I would think that there has got to be another way to do it that doesn't specifically require netcat

<@conan_kudo:matrix.org>

19:26:04

if new code needs to be written, this will need to be taken up with the RHEL cloud-init maintainers

<@yselkowitz:fedora.im>

19:27:13

unfortunately we're probably the only ones who care about one nc over the other

<@salimma:fedora.im>

19:27:15

this .. seems like something that should be solved internally at RH?

<@yselkowitz:fedora.im>

19:27:27

annoying that they're not fully compatible with each other though

<@salimma:fedora.im>

19:27:30

either fix the code to use the nc that's available or allow the nc that is needed in

<@salimma:fedora.im>

19:27:31

yeah

<@conan_kudo:matrix.org>

19:27:40

I am actually surprised the RHEL maintainers didn't chime in during the PR review upstream

<@conan_kudo:matrix.org>

19:27:52

they are supposed to be tracking this

<@yselkowitz:fedora.im>

19:28:24

well, here we are, we caught it

<@sgallagh:fedora.im>

19:28:52

Conan Kudo: It's pretty easy to miss that they aren't 100% compatible with one another.

<@sgallagh:fedora.im>

19:29:10

For most common actions, they are.

<@conan_kudo:matrix.org>

19:30:07

I guess I'll poke upstream and ask about this :/

<@salimma:fedora.im>

19:30:12

(aside: commented on the libxcrypt task. I can file an MR fixing the obsolete, which should be pinned to a version and not float since... *why*)

<@yselkowitz:fedora.im>

19:30:22

!action Yaakov to remind the RHEL maintainer

<@yselkowitz:fedora.im>

19:30:57

I would think the obsolete should not happen e.g. `%if 0%{?rhel} >= 11`

<@yselkowitz:fedora.im>

19:31:19

because they shouldn't be blocking EPEL from providing libxcrypt-compat

<@yselkowitz:fedora.im>

19:31:32

and RHEL major version upgrades don't rely on Provides/Obsoletes

<@conan_kudo:matrix.org>

19:31:42

the obsolete could be dropped entirely if it's been in place long enough

<@salimma:fedora.im>

19:31:44

that too. allow one release since it was obsoleted and no more

<@salimma:fedora.im>

19:31:56

oh right

<@conan_kudo:matrix.org>

19:32:01

obsoletes are only supposed to exist for 26 months

<@salimma:fedora.im>

19:32:07

anyway, I don't want to derail this

<@salimma:fedora.im>

19:32:16

wait... 26 months is very specific. how do we get there?

<@salimma:fedora.im>

19:32:26

oh Fedora

<@conan_kudo:matrix.org>

19:32:29

yup

<@conan_kudo:matrix.org>

19:32:40

the lifetimes for two fedora releases is 26 months

<@conan_kudo:matrix.org>

19:32:56

it actually is probably shorter due to overlapping lifecycles

<@conan_kudo:matrix.org>

19:33:02

but the maximum should be 26 months

<@conan_kudo:matrix.org>

19:33:53

that said, there has been some discussion about maintaining obsoletes for rhel upgrades to reduce the complexity of in-place upgrades

<@conan_kudo:matrix.org>

19:34:05

last I heard, no firm policy was in place for that yet

<@yselkowitz:fedora.im>

19:34:28

regardless I don't think it applies here though

<@yselkowitz:fedora.im>

19:34:45

anything further on those points?

<@conan_kudo:matrix.org>

19:35:05

yeah libxcrypt was introduced in f28

<@conan_kudo:matrix.org>

19:35:38

the compat library was introduced in f30

<@conan_kudo:matrix.org>

19:35:49

so now it's safe to drop across the board

<@salimma:fedora.im>

19:36:32

what matters is when the compat library was first dropped surely - looking at when it was first in Fedora does not really matter

<@salimma:fedora.im>

19:36:41

but anyway, we'll sort this out, no need to derail the meeting further

<@salimma:fedora.im>

19:37:26

oh... it's still in the fedora package so it's probably easier to fix right now

<@conan_kudo:matrix.org>

19:37:39

that's why I explained all this :)

<@conan_kudo:matrix.org>

19:37:54

it's easier to push a fix into rhel if it also applies to fedora

<@yselkowitz:fedora.im>

19:38:41

!topic Old business

<@salimma:fedora.im>

19:38:51

I mean, c11s does not exist yet so if we fix this in Fedora (and we have enough provenpackagers) c11s will inherit by defualt

<@salimma:fedora.im>

19:39:03

interesting timing

<@yselkowitz:fedora.im>

19:39:43

we're 2+ months since the F42 mass rebuild and I'm still running into packages which are F42FTBFS, so I'm not sure there are enough PPs...

<@yselkowitz:fedora.im>

19:39:47

but anyway...

<@yselkowitz:fedora.im>

19:40:06

!link https://github.com/fedora-eln/eln/issues/229

<@yselkowitz:fedora.im>

19:40:35

background: a different team is using ELN to start assigning packages for RHEL 11 already based on ELN

<@salimma:fedora.im>

19:40:39

speaking of myself this PP does not have enough time to fix other people's packages this cycle

<@salimma:fedora.im>

19:40:42

too many brokenness

<@conan_kudo:matrix.org>

19:41:00

same statement

<@conan_kudo:matrix.org>

19:41:08

I've been firefighting almost continuously since February

<@yselkowitz:fedora.im>

19:41:37

but they didn't understand to use CR, so they used the composes to try to determine ELN content

<@yselkowitz:fedora.im>

19:42:00

in the process they found some issues that were fixable, and some known issues

<@salimma:fedora.im>

19:42:02

is this just a documentation issue?

<@salimma:fedora.im>

19:42:04

or a training issue

<@salimma:fedora.im>

19:42:14

but if they find issues that's good I guess

<@yselkowitz:fedora.im>

19:42:30

yeah I'm working with them on it now

<@yselkowitz:fedora.im>

19:43:09

and while I'm happy to fix issues in the composes, that's not blocking ultimately because CR is the source of truth for ELN

<@yselkowitz:fedora.im>

19:43:53

our composes are in a bit better shape now, and more of them even pass repoclosure (a separate issue)

<@yselkowitz:fedora.im>

19:43:57

but HA is still a mess

<@yselkowitz:fedora.im>

19:44:41

because the bulk of HA is supposed to be placeholders, for packages which exist in Fedora but are built with vendored deps in RHEL (and therefore the specs are quite different)

<@yselkowitz:fedora.im>

19:45:36

if we actually needed the content then we'd use an eln branch, but since this is HA then the thought was placeholders would suffice

<@salimma:fedora.im>

19:45:50

oof, that's tricky

<@yselkowitz:fedora.im>

19:46:01

and while placeholders aren't normally supposed to be built, in this case they are because of ELN Extras

<@salimma:fedora.im>

19:46:13

seems like... seeding ELN with the package specs that would end up in EL11 is a good idea indeed

<@salimma:fedora.im>

19:46:23

can eln extras use HA? that seems odd

<@yselkowitz:fedora.im>

19:46:34

so in short, the packages are in HA but as they are in Fedora but not as in RHEL, and therefore there's a bunch of extra deps there

<@salimma:fedora.im>

19:46:41

or rather, should they be able to? I thought ELN Extras are meant for things that could end up in EPEL, and EPEL does not use HA

<@yselkowitz:fedora.im>

19:47:03

it doesn't affect the ELN package list but it does affect anyone *using* the ELN HA repo

<@yselkowitz:fedora.im>

19:47:29

so, first off, does anybody using ELN repos care?

<@tdawson:fedora.im>

19:48:26

Not I

<@nhanlon:beeper.com>

19:48:29

nor I

<@nhanlon:beeper.com>

19:48:36

(hi, super late, sorry)

<@salimma:fedora.im>

19:48:44

not really

<@yselkowitz:fedora.im>

19:48:53

but EPEL can have packages which are in HA, and that's what's happening here, and hence they're being built, but shipped in HA because that's how they are in RHEL

<@yselkowitz:fedora.im>

19:49:01

it's a weird corner case

<@salimma:fedora.im>

19:49:11

ah right

<@salimma:fedora.im>

19:49:15

so the leakage goes the other way

<@davide:cavalca.name>

19:49:48

This is going to happen for all the layered products

<@yselkowitz:fedora.im>

19:50:12

well, not really

<@davide:cavalca.name>

19:50:19

Should we loop in whoever owns these on the RH side? Like, I personally don't care, but I assume they probably might?

<@yselkowitz:fedora.im>

19:50:23

assuming you mean RHEL addons such as HA

<@davide:cavalca.name>

19:50:33

Yeah, that's what I meant

<@yselkowitz:fedora.im>

19:51:04

so SAP/SAPHANA are mostly RHEL-specific packages which are *not* in Fedora, and therefore the placeholders act as expected

<@yselkowitz:fedora.im>

19:51:32

but placeholders for packages which *are* in Fedora but very different in RHEL, well, those can end up getting built

<@salimma:fedora.im>

19:51:35

yeah. the issue is with packages that overlap with EPEL - this has been discussed in EPEL meetings too fwiw

<@salimma:fedora.im>

19:51:51

and the consensus is "since EPEL does not target HA if you enable both at the same time you get to keep both pieces"

<@salimma:fedora.im>

19:52:07

I'm sure Troy Dawson put it more delicately than I just did :/

<@conan_kudo:matrix.org>

19:52:55

but yeah, EPEL historically doesn't care about addons since they aren't generally available

<@yselkowitz:fedora.im>

19:53:02

1. Maintain an eln branch (possibly based on c10s) of these (supposed) placeholder packages, and build those for ELN (and convert the placeholders to real packages)

<@yselkowitz:fedora.im>

19:53:02

so the possible "fixes" enumerated in the ticket are:

<@yselkowitz:fedora.im>

19:53:02

3. OR leave the status quo as a known "issue" (there would at least be content in HA, and the package list in CR is correct, but it's confusing).

<@yselkowitz:fedora.im>

19:53:02

2. OR exclude these packages from the HA compose, which wouldn't leave much left there (they would then move to ELN Extras, as they are listed there)

<@tdawson:fedora.im>

19:53:26

I believe I put it less delicately :) but yes, you are correct in what you say.

<@yselkowitz:fedora.im>

19:53:42

(2) would essentially be the EPEL approach, kick them out of ELN HA and let them land in ELN Extras

<@yselkowitz:fedora.im>

19:54:34

or (3) we just don't care, and there's no issue with HA vs Extras because it's all one compose

<@salimma:fedora.im>

19:54:49

I lean towards 1) but the HA engineers at RH need to maintain them

<@davide:cavalca.name>

19:55:06

Yes, this

<@salimma:fedora.im>

19:55:10

but I would point out that with option 1) we have to make sure ELN Extras workload cannot inherit from them

<@salimma:fedora.im>

19:55:26

because... I don't want to accidentally have my dependencies resolved by HA - it won't be correct for EPEL

<@tdawson:fedora.im>

19:55:39

Although (1) would be the "best" thing to do ... I don't see the HA maintainers doing it.

<@yselkowitz:fedora.im>

19:55:50

well that's the other thing, since it's all one compose then you can't have the same package both ways

<@conan_kudo:matrix.org>

19:56:02

option 2 is probably the reasonable choice

<@tdawson:fedora.im>

19:56:12

I'm leaning towards (2) until someone actually comes to us and says they are using the HA repo and need the packages in there.

<@salimma:fedora.im>

19:56:33

without a way to isolate HA builds I think (2) is more realistic

<@tdawson:fedora.im>

19:56:51

I didn't even know we HAD a real HA repo in ELN. I thought it was pretty much empty, like the other ones.

<@yselkowitz:fedora.im>

19:57:23

the more you know 😁

<@salimma:fedora.im>

19:57:39

oof, almost time

<@yselkowitz:fedora.im>

19:57:52

oh wow thanks for pointing that out

<@davide:cavalca.name>

19:58:10

I will reiterate that I think we should be having this conversation with the HA maintainers in RHEL

<@salimma:fedora.im>

19:59:15

quick follow up from last meeting - I just filed the PR tracking conda

<@salimma:fedora.im>

19:59:15

https://github.com/minimization/content-resolver-input/pull/1403

<@salimma:fedora.im>

19:59:15

<@yselkowitz:fedora.im>

19:59:41

ok we'll follow up next week

<@yselkowitz:fedora.im>

19:59:45

unless there's a conflict?

<@tdawson:fedora.im>

20:00:13

No conflict for me

<@salimma:fedora.im>

20:00:24

not that I know of for me

<@yselkowitz:fedora.im>

20:00:34

!info Next meeting on Thursday 03 April 15:00 EDT

<@yselkowitz:fedora.im>

20:00:43

timezones will be back in sync next week

<@davide:cavalca.name>

20:00:46

Should be fine

<@salimma:fedora.im>

20:00:50

oh right

<@yselkowitz:fedora.im>

20:00:53

and we're at time

<@yselkowitz:fedora.im>

20:01:09

thank you all for the lively discussions, see you in channel

<@zodbot:fedora.im>

20:01:12

salimma has already given cookies to yselkowitz during the F41 timeframe

<@yselkowitz:fedora.im>

20:01:13

!endmeeting