#meeting-1:fedoraproject.org log

<@tdawson:fedora.im>

18:00:30

!startmeeting EPEL (2024-10-23)

<@meetbot:fedora.im>

18:00:31

Meeting started at 2024-10-23 18:00:30 UTC

<@meetbot:fedora.im>

18:00:32

The Meeting name is 'EPEL (2024-10-23)'

<@tdawson:fedora.im>

18:00:37

!topic aloha

<@tdawson:fedora.im>

18:00:37

!meetingname epel

<@meetbot:fedora.im>

18:00:37

The Meeting Name is now epel

<@dherrera:fedora.im>

18:01:29

!hi

<@zodbot:fedora.im>

18:01:29

Diego Herrera (dherrera) - he / him / his

<@tdawson:fedora.im>

18:01:55

Hi Diego Herrera

<@salimma:fedora.im>

18:02:14

!hi

<@zodbot:fedora.im>

18:02:16

Michel Lind (salimma) - he / him / his

<@salimma:fedora.im>

18:02:25

glad to be back!

<@davide:cavalca.name>

18:03:18

!hi

<@zodbot:fedora.im>

18:03:19

Davide Cavalca (dcavalca) - he / him / his

<@tdawson:fedora.im>

18:03:32

Hi Michel Lind 🎩 UTC-5 and Davide Cavalca

<@carlwgeorge:matrix.org>

18:03:44

!hi

<@zodbot:fedora.im>

18:03:46

Carl George (carlwgeorge) - he / him / his

<@music:fedora.im>

18:03:53

!hi

<@zodbot:fedora.im>

18:03:55

Benjamin Beasley (music) - he / him / his

<@tdawson:fedora.im>

18:04:21

Hi Carl George and music

<@tdawson:fedora.im>

18:04:51

music: Are you here about the llhttp item? Or are there other things you wanted to talk about?

<@tdawson:fedora.im>

18:06:00

!topic EPEL Issues https://pagure.io/epel/issues

<@tdawson:fedora.im>

18:06:00

!link https://pagure.io/epel/issues?tags=meeting&status=Open

<@music:fedora.im>

18:06:04

I am here just in case someone wants to ask me about llhttp, although I don’t expect to have much to say since the issue at hand has more to do with EPEL than llhttp itself.

<@sgallagh:fedora.im>

18:06:24

!hi

<@zodbot:fedora.im>

18:06:25

Stephen Gallagher (sgallagh) - he / him / his

<@tdawson:fedora.im>

18:06:26

OK. Thank you for being here.

<@tdawson:fedora.im>

18:06:34

Hi Stephen Gallagher

<@sgallagh:fedora.im>

18:06:35

I'm here about rpmautospec

<@carlwgeorge:matrix.org>

18:06:41

we appreciate your experience on the matter

<@tdawson:fedora.im>

18:06:49

Luckily the llhttp thing is first on the agenda

<@tdawson:fedora.im>

18:06:55

!epel 302

<@zodbot:fedora.im>

18:06:56

● **Opened:** 45 minutes ago by carlwgeorge

<@zodbot:fedora.im>

18:06:56

● **Last Updated:** a minute ago

<@zodbot:fedora.im>

18:06:56

● **Assignee:** carlwgeorge

<@zodbot:fedora.im>

18:06:56

**epel #302** (https://pagure.io/epel/issue/302):**llhttp rebase**

<@zodbot:fedora.im>

18:06:56

<@carlwgeorge:matrix.org>

18:06:58

thanks music and Stephen Gallagher both for being here

<@nhanlon:beeper.com>

18:07:00

!hi

<@zodbot:fedora.im>

18:07:02

Neil Hanlon (neil) - he / him / his

<@tdawson:fedora.im>

18:07:07

Hi Neil Hanlon

<@sgallagh:fedora.im>

18:07:10

(You say that now... :))

<@tdawson:fedora.im>

18:07:26

Carl George: Did you want to lead the discussion on this issue?

<@xavierb:bachelot.org>

18:07:33

!hi

<@carlwgeorge:matrix.org>

18:07:33

sure

<@zodbot:fedora.im>

18:07:36

None (xavierb)

<@nhanlon:beeper.com>

18:07:47

/me in sunny Santa Clara for riscv summit today

<@rcallicotte:fedora.im>

18:07:54

!hi

<@zodbot:fedora.im>

18:07:56

Robby Callicotte (rcallicotte) - he / him / his

<@tdawson:fedora.im>

18:08:02

Hi Robby Callicotte

<@carlwgeorge:matrix.org>

18:08:05

oh neat, this format works now, instead of `!link ...`

<@carlwgeorge:matrix.org>

18:08:31

the full details are in that issue, but here's a tldr for anyone that didn't get a chance to read it yet

<@carlwgeorge:matrix.org>

18:09:36

there is an incoming llhttp rebase in c10, and i think it will break the epel10 buildroot when it lands. builds need rpmautospec installed really early in the process, which has a dependency chain that includes libgit2 which links against llhttp. so if libgit2 becomes uninstallable, all builds will fail.

<@nirik:matrix.scrye.com>

18:09:48

morning

<@salimma:fedora.im>

18:10:29

oh fun, rpmautospec bootstrapping again :(

<@salimma:fedora.im>

18:10:53

I saw Carl's bump on the email asking to get access to libgit2... it's still unresolved huh?

<@carlwgeorge:matrix.org>

18:10:56

basically, or rather, how do we avoid the bootstrap pain

<@carlwgeorge:matrix.org>

18:11:19

thinking about this last night, i created a compat package, llhttp9.1, which i think will be a good workaround for the issue. the problem is that is has a file conflict with the current llhttp 9.1. so it's technically not allowed by policy currently.

<@tdawson:fedora.im>

18:11:39

Well, I like you idea of having the compat-llhttp9.1 package. I think it's quick, simple, and doesn't mess with any other packages.

<@sgallagh:fedora.im>

18:11:40

So, to be clear, the current rebase was triggered by a CVE fix. The actual CVE fixes can be backported, so we could kick this can down the road.

<@sgallagh:fedora.im>

18:12:27

But long-term maintenance of this package suggests that it will bump soname semi-regularly, and future backports may become harder

<@conan_kudo:matrix.org>

18:12:35

!hi

<@zodbot:fedora.im>

18:12:38

Neal Gompa (ngompa) - he / him / his

<@carlwgeorge:matrix.org>

18:13:03

if we wait for llhttp9.1 to be allowed by policy after the llhttp 9.2 rebase, then it won't be able to be built because of the buildroot instantiation problem. so i'm thinking we do an exception to policy now, allow llhttp and llhttp9.1 to coexist, and then after the rebase lands rebuild libgit2 against llhttp 9.2, and retire llhttp9.1

<@tdawson:fedora.im>

18:13:09

Ugg ... so we might have to do this every six months to a year :(

<@nirik:matrix.scrye.com>

18:13:11

Does some decision need to be made here? or can it just get discussed in ticket/etc and decided out of meeting?

<@carlwgeorge:matrix.org>

18:13:27

depends on how quickly llhttp 9.2 is landing

<@nirik:matrix.scrye.com>

18:13:32

I haven't read the ticket as I was in a meeting before this one.

<@salimma:fedora.im>

18:13:59

if llhttp9.1 gets released the same time llhttp is bumped, this becomes a non-issue right?

<@tdawson:fedora.im>

18:14:02

Carl George: What about just leaving it in the side-tag? Thus it never makes it into EPEL proper.

<@salimma:fedora.im>

18:14:09

ideally the same person maintains both and they get released in lockstep

<@carlwgeorge:matrix.org>

18:14:15

for clarity, if i put a space inbetween the name and version, i'm talking about the regular package, and if it's run together i'm talking about the compat package

<@conan_kudo:matrix.org>

18:14:20

Can we make c10s package more tolerant of alternative versions?

<@sgallagh:fedora.im>

18:14:45

Troy Dawson: That's one of the reasons I'm wondering if it would be saner for us to have libgit2 bundle llhttp. Then an update of it in RHEL/CS proper won't trigger the rpmautospec bootstrapping problem

<@carlwgeorge:matrix.org>

18:14:55

yes but that's impossible, because llhttp is coming in via the external repo

<@nirik:matrix.scrye.com>

18:15:30

you should be able to build in a side tag, then when the external repo updates, update the buildroot in the side tag

<@carlwgeorge:matrix.org>

18:15:48

that was part of the alternative i brought up in the issue, however that would leave a period of time where builds are broken. i think it would be harmless for llhttp and llhttp9.1 to temporarily coexist.

<@tdawson:fedora.im>

18:16:07

Yep, even if I tried my hardest to time a CS10 release, just the mechanism of releases is going to have them not match.

<@music:fedora.im>

18:16:49

Upstream has backported some fixes to older branches in the past, but some of the security fixes so far have inherently required API changes. It’s hard to predict whether this will continue, or the package will become more and more quiescent in the future.

<@carlwgeorge:matrix.org>

18:17:28

yup, but this is the classic bundling problem. any llhttp rebase would likely be justified by a significant enough cve, which would also mean updating the bundled one in libgit2. no bootstrap problem, just double the work for the rhel maintainers.

<@salimma:fedora.im>

18:17:34

oh right :(

<@sgallagh:fedora.im>

18:17:41

I suppose another option is to remove rpmautospec macros from everything in the rpmautospec dep chain

<@sgallagh:fedora.im>

18:18:07

Carl George: I am the RHEL maintainer of libgit2. I am aware of what I'd be signing up for :)

<@salimma:fedora.im>

18:18:17

yeah, Google does this with their distributed SQL database iirc. if you're one of its dependencies you can't use it

<@tdawson:fedora.im>

18:18:32

My thoughts are that if anything should bundle it's dependencies, it would be rpmautospec.

<@sgallagh:fedora.im>

18:19:16

That would be nice, but not realistic since we can't really statically-link a python program. Or if we can, it'll be needlessly complex.

<@salimma:fedora.im>

18:19:17

if we ban some packages from using rpmautospec we should comment their specs to explain why though, or some well-meaning packager might convert the package again later

<@carlwgeorge:matrix.org>

18:19:43

does anyone have concerns with the solution of publishing llhttp9.1, allowing the temporary conflict with a package in the target base, so we don't have a time period of broken epel10 builds?

<@davide:cavalca.name>

18:19:47

and ideally enforce it with a push-time check

<@tdawson:fedora.im>

18:19:59

I did like the one solution that was said. Write rpmautospec in rust, so it's a static binary.

<@sgallagh:fedora.im>

18:20:21

Troy Dawson: Go sit in the corner.

<@nhanlon:beeper.com>

18:20:36

no one puts troy in the corner

<@tdawson:fedora.im>

18:20:46

I'm good with it.

<@tdawson:fedora.im>

18:20:58

That's because I sit in a round room. :)

<@nhanlon:beeper.com>

18:21:03

i'm cool w/ it

<@nhanlon:beeper.com>

18:21:11

padded?

<@nirik:matrix.scrye.com>

18:21:26

I think that would be ok.

<@carlwgeorge:matrix.org>

18:21:33

in my testing, libgit2 still resolves to the base llhttp with llhttp9.1 present, and then once the llhttp rebase is present it resolves to llhttp9.1 instead

<@tdawson:fedora.im>

18:21:36

Very padded.

<@sgallagh:fedora.im>

18:21:45

Even if we rewrite rpmautospec in a compiled language (which I would love to see), it is likely still going to need to link to libgit2 (either dynamically or statically), which doesn't solve this problem

<@nhanlon:beeper.com>

18:22:42

could we dlopen() it?

<@nhanlon:beeper.com>

18:23:01

(may be a dumb question and feel free to tell me so ;))

<@sgallagh:fedora.im>

18:23:15

Neil Hanlon: How does that help?

<@davide:cavalca.name>

18:23:38

there are rust git implementations that don't use libgit2 iirc

<@music:fedora.im>

18:23:49

(Ideally using rust-gix rather than git2-rs so it doesn’t still end up linking libgit2, I guess!)

<@sgallagh:fedora.im>

18:23:53

Implementations of what?

<@carlwgeorge:matrix.org>

18:24:05

while interesting, i think we should punt the long term rust ideas till after the meeting

<@nhanlon:beeper.com>

18:24:10

like i said, could be a dumb question! :D

<@sgallagh:fedora.im>

18:24:21

Right, let's ignore the theoretical rewrite of rpmautospec for now

<@carlwgeorge:matrix.org>

18:24:27

do we want to vote on the issue or just call it good since no one opposed?

<@conan_kudo:matrix.org>

18:24:51

I'm unopposed

<@sgallagh:fedora.im>

18:24:51

Carl George: What happens if we decide not to do the 9.2 rebase in CS/RHEL?

<@sgallagh:fedora.im>

18:24:55

That's on the table

<@davide:cavalca.name>

18:25:08

I was thinking stuff like https://github.com/GitoxideLabs/gitoxide but yeah, let's punt this thread for now

<@sgallagh:fedora.im>

18:25:18

We may just backport to 9.1 for now in order to buy time to figure out a long-term solution

<@carlwgeorge:matrix.org>

18:25:26

then the llhttp9.1 package can be retired and we pretend like nothing happened

<@carlwgeorge:matrix.org>

18:25:39

and delete the side tag

<@sgallagh:fedora.im>

18:25:57

"Nothing to see here. Move along. All is well..."

<@salimma:fedora.im>

18:25:59

I need to check how the libgit2 bindings work in Rust before I can vouch that that is safe

<@carlwgeorge:matrix.org>

18:26:08

well, the side tag would already be gone if we merge it

<@salimma:fedora.im>

18:26:12

unless Fabio Valentini is here and can just answer that since he has to deal with that mess :)

<@carlwgeorge:matrix.org>

18:28:08

ok, i'll merge that side tag, and keep an eye on how this plays out in centos to determine the next steps

<@tdawson:fedora.im>

18:28:24

Carl George: Sounds good. Thank you.

<@music:fedora.im>

18:28:37

(I don’t do downstream backports in llhttp in Fedora because I’m not highly confident analyzing them quickly with the typescript-compiled-to-C nature of the package. I can believe a backport could be fine if someone took the time to think it through, though.)

<@nirik:matrix.scrye.com>

18:28:37

I can do one-off syncs too if desired.

<@tdawson:fedora.im>

18:28:51

I think by not having an official vote, it makes it easier for us to back it out if the update doesn't happen.

<@salimma:fedora.im>

18:29:15

that seems safe, bat requires rust-git2 which requires rust-libgit2-sys and it does not have install-time dependency on libgit2

<@carlwgeorge:matrix.org>

18:29:22

yeah, just blame it on that rogue guy who built it 😀

<@tdawson:fedora.im>

18:30:05

Anything else on this subject before we move on?

<@music:fedora.im>

18:30:09

git2-rs links the system libgit2, gix/gitoxide is pure rust

<@salimma:fedora.im>

18:30:48

I wonder if we defauled to the vendored libgit2 and that's why we don't see this

<@salimma:fedora.im>

18:31:05

but anyway

<@tdawson:fedora.im>

18:31:18

I think we'll let Michel Lind 🎩 UTC-5 and music continue to muse on that while we move on.

<@tdawson:fedora.im>

18:31:34

!topic EPEL 10

<@tdawson:fedora.im>

18:31:34

https://hackmd.io/q6TNkYjJT82EtzhlyPGpog

<@tdawson:fedora.im>

18:32:13

We're up to 2730+ source packages in EPEL10.

<@carlwgeorge:matrix.org>

18:33:08

we have officially surpassed the count of epel9 packages at the time of the rhel9 launch, and we haven't even launched epel10 yet

<@sgallagh:fedora.im>

18:33:12

Now exceeding RHEL 10 itself!

<@tdawson:fedora.im>

18:33:47

Ya ... this is sorta crazy.

<@tdawson:fedora.im>

18:34:13

It sorta feels like the release of RHEL 10 is going to be anti-climatic, because everyone will have been using it for so long.

<@carlwgeorge:matrix.org>

18:34:30

speaking of, in case i hadn't mentioned it in this meeting yet, we're tentatively looking at november 19th for the epel10 launch

<@nirik:matrix.scrye.com>

18:34:36

yeah, lets make things more interesting and add riscv into the mix! :)

<@carlwgeorge:matrix.org>

18:35:02

i wouldn't post that date everywhere in case we need to slip it, but i figured the people here would like the heads up

<@davide:cavalca.name>

18:35:12

much appreciated, thanks

<@carlwgeorge:matrix.org>

18:35:40

i'm in the process of writing the announcement, once i get further along with it i'll share it in one of these meetings for feedback

<@tdawson:fedora.im>

18:36:01

Someone asked me "What will be different at the release?" and my answer was "We'll have announced it."

<@carlwgeorge:matrix.org>

18:36:38

i would also answer "normal process of a week in testing before going to stable"

<@nirik:matrix.scrye.com>

18:36:50

so we switch to 'composed by bodhi' then?

<@carlwgeorge:matrix.org>

18:36:57

yes

<@carlwgeorge:matrix.org>

18:36:59

speaking of...

<@carlwgeorge:matrix.org>

18:37:03

!epel 300

<@zodbot:fedora.im>

18:37:05

<@zodbot:fedora.im>

18:37:05

**epel #300** (https://pagure.io/epel/issue/300):**EPEL 10 launch**

<@zodbot:fedora.im>

18:37:05

● **Opened:** a day ago by carlwgeorge

<@zodbot:fedora.im>

18:37:05

● **Last Updated:** Never

<@zodbot:fedora.im>

18:37:05

● **Assignee:** carlwgeorge

<@tdawson:fedora.im>

18:37:13

Correct, and that is true. We will go from releases "wheneever we want" to "Once a week, unless there is an emergency"

<@carlwgeorge:matrix.org>

18:37:47

we have the big hackmd of all the epel10 stuff, but i figured we needed a more narrow list of things blocking the release

<@conan_kudo:matrix.org>

18:37:58

Still missing the important one: KDE 🤪

<@nirik:matrix.scrye.com>

18:38:11

whats the 'bugzilla exception' ?

<@carlwgeorge:matrix.org>

18:38:34

the thing where bodhi ignores attaching bugzillas because it was grabbing old ones

<@carlwgeorge:matrix.org>

18:38:58

https://pagure.io/fedora-infra/ansible/pull-request/2206

<@tdawson:fedora.im>

18:39:03

Yep ... Today and tomorrow I'm going to see what happens if I pull ffmpeg out of the dependencies for some critical packages ... see if I can get KDE going ... without sound or video.

<@carlwgeorge:matrix.org>

18:39:43

i would love to have a working kde stack at launch, but i don't think it is release blocking

<@nirik:matrix.scrye.com>

18:39:57

ah, due to the merging in from fedora... makes sense I guess.

<@conan_kudo:matrix.org>

18:40:38

We should be able to get it before the launch

<@carlwgeorge:matrix.org>

18:40:46

most of this checklist can be one big pr to the fedora-infra/ansible repo, that we review carefully ahead of time and then merge closer to the launch

<@tdawson:fedora.im>

18:41:06

I believe it's possible to have ffmpeg before the 19th ... we're getting very close.

<@carlwgeorge:matrix.org>

18:42:03

i was thinking of trying to have everything buttoned up about a week before the launch, just to give us a short period to make sure the normal workflow through the testing repo works as expected

<@tdawson:fedora.im>

18:42:16

For those of you that have "long time" CS10 installs. Note that between now and the release there will be a mass rebuild. So every package is going to be updated.

<@davide:cavalca.name>

18:42:53

do you have a sense if that'll be this year or next year?

<@conan_kudo:matrix.org>

18:43:10

It's a shame no rpm 4.20 for c10s though

<@tdawson:fedora.im>

18:43:33

Oh, when I said "release" I meant EPEL10 release.

<@davide:cavalca.name>

18:43:45

ah perfect, that answers my question :)

<@davide:cavalca.name>

18:43:46

thanks

<@tdawson:fedora.im>

18:43:56

As in very close, but not this week.

<@nirik:matrix.scrye.com>

18:44:03

should epel do anything with that? I guess in theory nothing should break right?

<@tdawson:fedora.im>

18:44:15

In theory, nothing should break

<@davide:cavalca.name>

18:44:31

yeah, that works; I'm gearing up to kick off the internal mass migration to c10s at Meta and wanted to make sure we did after the mass rebuild

<@tdawson:fedora.im>

18:44:39

If your packages depend on ocaml ... watch them ... cuz ocaml is like tumbling towers.

<@nirik:matrix.scrye.com>

18:45:14

🗼

<@tdawson:fedora.im>

18:46:24

Carl George: Is the epel-release in CentOS's extras-common repo something you can do? Or is that something Johnny needs to do?

<@tdawson:fedora.im>

18:46:55

I'm really fuzzy on how that get's in there.

<@davide:cavalca.name>

18:47:09

if it's the same as sig release packages all sig leads can push to that

<@tdawson:fedora.im>

18:47:35

Oh ... so even I could do that ... ok. Good to know.

<@davide:cavalca.name>

18:48:13

https://sigs.centos.org/guide/delivery/#centos-release-package see the box at the bottom

<@conan_kudo:matrix.org>

18:48:33

I'm pretty sure I updated it last time

<@carlwgeorge:matrix.org>

18:49:18

the last time i touched that stuff it was still just extras, not extras-common, so i'm out of the loop

<@tdawson:fedora.im>

18:49:21

Yep, just looked. Carl did it first, then Conan.

<@carlwgeorge:matrix.org>

18:50:00

i would super appreciate if someone else can take that on and just let me know when it's done

<@tdawson:fedora.im>

18:50:15

I can do that.

<@carlwgeorge:matrix.org>

18:50:40

i have this checklist issue assigned to me, but that's more for oversight and not necessarily saying i'm doing every step myself

<@tdawson:fedora.im>

18:50:50

If it turns out I don't have permissions, I'll ping around until it get's done.

<@carlwgeorge:matrix.org>

18:51:05

oh look at the time

<@tdawson:fedora.im>

18:51:18

Ya, I was about to comment ... time to move on.

<@carlwgeorge:matrix.org>

18:51:18

that's probably everything for epel10 right now, lets have an open floor

<@tdawson:fedora.im>

18:51:38

!topic General Issues / Open Floor

<@tdawson:fedora.im>

18:51:51

Does anyone have anything for open floor?

<@carlwgeorge:matrix.org>

18:52:55

if you're in the u.s., please go vote ☑️

<@dherrera:fedora.im>

18:53:27

a thing, installability checks are now working on epel10 PRs on pagure :)

<@carlwgeorge:matrix.org>

18:54:05

after working through that, do you have any idea about the path forward to get that enabled for bodhi updates too?

<@salimma:fedora.im>

18:54:17

my sympathies for those who can only vote same day

<@salimma:fedora.im>

18:54:17

<@salimma:fedora.im>

18:54:17

I just voted this morning (3rd day early voting is available in my county)

<@zodbot:fedora.im>

18:54:38

tdawson gave a cookie to dherrera. They now have 22 cookies, 8 of which were obtained in the Fedora 40 release cycle

<@dherrera:fedora.im>

18:55:22

no, but I opened an issue to keep track on that work here https://pagure.io/epel/issue/301

<@zodbot:fedora.im>

18:55:24

carlwgeorge gave a cookie to dherrera. They now have 23 cookies, 9 of which were obtained in the Fedora 40 release cycle

<@dherrera:fedora.im>

18:55:28

!link https://pagure.io/epel/issue/301

<@nirik:matrix.scrye.com>

18:55:35

my mailin ballot is ready to send in. ;)

<@dherrera:fedora.im>

18:58:38

also, some weeks ago I noticed that the automatic workflow for willit was stuck cos epel9+epel9next took too much time to build, so I removed epel9next from it for now

<@dherrera:fedora.im>

18:59:03

https://dherrerace.github.io/willit-result/ has been working fine besides that

<@carlwgeorge:matrix.org>

18:59:18

repoclosure output is also looking much better recently

<@carlwgeorge:matrix.org>

18:59:47

several maintainers took me up on the offer of "i can untag this for you if you don't think you'll get around to fixing it for a bit" in bz comments

<@tdawson:fedora.im>

19:00:05

Our time is up. Thank you all for coming, for the good discussions, and good resolutions to problems.

<@salimma:fedora.im>

19:00:14

Troy Dawson++

<@zodbot:fedora.im>

19:00:15

salimma has already given cookies to tdawson during the F40 timeframe

<@dherrera:fedora.im>

19:00:25

Thanks Troy ^^

<@nirik:matrix.scrye.com>

19:00:28

thanks Troy Dawson as always!

<@tdawson:fedora.im>

19:00:30

Carl George: Ya, I have a couple KDE ones that if I don't fix it in a couple weeks, I'll untag them.

<@tdawson:fedora.im>

19:00:41

Talk to you all next week, if not sooner.

<@tdawson:fedora.im>

19:00:56

!endmeeting