<@salimma:fedora.im>
16:01:38
!startmeeting CentOS Hyperscale SIG
<@meetbot:fedora.im>
16:01:39
Meeting started at 2026-01-14 16:01:38 UTC
<@t0xic0der:fedora.im>
16:01:40
Justin Wheeler++, see ya folks around!
<@meetbot:fedora.im>
16:01:40
The Meeting name is 'CentOS Hyperscale SIG'
<@zodbot:fedora.im>
16:01:43
t0xic0der has already given cookies to jflory7 during the F43 timeframe
<@salimma:fedora.im>
16:01:52
!topic Roll call
<@conan_kudo:matrix.org>
16:02:07
!hi
<@zodbot:fedora.im>
16:02:10
Neal Gompa (ngompa) - he / him / his
<@salimma:fedora.im>
16:02:22
since we don't really want to ping the overly large group yet... ah Neal is here already
<@gmambro:fedora.im>
16:02:25
!hi
<@zodbot:fedora.im>
16:02:27
Gabriele Mambrini (gmambro) - he / him / his
<@salimma:fedora.im>
16:02:35
Davide said he'll be absent, but summoning pzmarzly
<@pzmarzly:matrix.org>
16:02:54
i am here, how do i use zodbot though?
<@salimma:fedora.im>
16:03:47
if you have an account in accounts.fedoraproject.org, and your matrix ID is listed, you can do ! hi
<@salimma:fedora.im>
16:03:48
!hi
<@zodbot:fedora.im>
16:03:50
Michel Lind (salimma) - he / him / his
<@rcolebaugh:matrix.org>
16:03:59
!hi
<@salimma:fedora.im>
16:04:02
it is just nice for roll call as it pulls in the bio you entered in the account
<@zodbot:fedora.im>
16:04:04
Raymond Colebaugh (rcolebaugh) - he / him / his
<@pzmarzly:matrix.org>
16:04:29
!hi
<@zodbot:fedora.im>
16:04:32
Pawel Zmarzly (pawo2500)
<@salimma:fedora.im>
16:06:35
let's give people a couple more minutes and then we can start looking at follow ups
<@rcallicotte:fedora.im>
16:07:46
!hi
<@zodbot:fedora.im>
16:07:48
Robby Callicotte (rcallicotte) - he / him / his
<@salimma:fedora.im>
16:08:33
hi folks!
<@salimma:fedora.im>
16:08:38
ok let's carry on
<@salimma:fedora.im>
16:08:45
!topic Followups
<@salimma:fedora.im>
16:09:21
Conan Kudois double booked, but let's see if he has time right now - in the previous meeting you mentioned updated kernels
<@conan_kudo:matrix.org>
16:09:43
Yeah. So I stopped working on 6.17 once Fedora rebased to 6.18
<@conan_kudo:matrix.org>
16:10:06
and now that 6.18 has been released to Fedora users, I've started looking at this for CentOS Hyperscale
<@salimma:fedora.im>
16:10:07
the upgrade treadmill :/
<@conan_kudo:matrix.org>
16:10:31
it's fine, it's just that the build->fix loop in CBS is slow
<@conan_kudo:matrix.org>
16:10:37
that's what ultimately led to this
<@salimma:fedora.im>
16:11:54
yeah... local builds are fast though right?
<@salimma:fedora.im>
16:12:17
maybe we should follow up with discussing if we should just dedicate an EC2 machine with persistent storage for kernel work
<@salimma:fedora.im>
16:12:34
I wonder how useful ccache is for kernel builds
<@conan_kudo:matrix.org>
16:12:58
yes
<@salimma:fedora.im>
16:12:59
we only care about x86_64 and aarch64 anyway, and those we can get in the cloud
<@conan_kudo:matrix.org>
16:13:16
local builds are fast-ish
<@conan_kudo:matrix.org>
16:13:20
they take about an hour apiece
<@conan_kudo:matrix.org>
16:13:24
for x86_64
<@conan_kudo:matrix.org>
16:13:34
for aarch64 it's a lot worse because it's actually multiple kernel builds
<@salimma:fedora.im>
16:13:35
the curse of bullding all those kernel modules
<@conan_kudo:matrix.org>
16:13:44
nah, it's the kernel variants
<@salimma:fedora.im>
16:14:00
and there's more variants for aarch64 I guess
<@conan_kudo:matrix.org>
16:14:01
the modules are fast, it's building all the different permutations that slow it down
<@salimma:fedora.im>
16:14:07
do we care about all the variants Fedora produce?
<@conan_kudo:matrix.org>
16:14:08
3x
<@conan_kudo:matrix.org>
16:14:25
we already follow the RHEL config for variants, which means we have _more_ than Fedora
<@salimma:fedora.im>
16:14:27
I wonder if those can be done in parallel
<@salimma:fedora.im>
16:14:43
do we ... care about all those variants? or can we trim them down
<@conan_kudo:matrix.org>
16:14:46
with buildsystem support, yes (like what SUSE's OBS does for their kernel package)
<@conan_kudo:matrix.org>
16:15:12
unfortunately I don't think we can trim them down since they are intended to overshadow the ones the stock kernel has
<@conan_kudo:matrix.org>
16:15:26
we actually add one more flavor for Asahi, so that kind of makes things worse
<@salimma:fedora.im>
16:15:40
ah... well. yeah we can just throw some resources at development I guess
<@salimma:fedora.im>
16:15:55
maybe we can short-circuit for testing and only do one variant
<@conan_kudo:matrix.org>
16:16:00
I was really hoping the upstream Arm patchset for boot-time configured page sizes would land
<@salimma:fedora.im>
16:16:10
and once everything is solid, do one proper build with all the variants enabled
<@conan_kudo:matrix.org>
16:16:13
that would allow us to drop half of the variants
<@conan_kudo:matrix.org>
16:16:22
that's my current loop yes :)
<@conan_kudo:matrix.org>
16:16:29
there are bconds to toggle them
<@salimma:fedora.im>
16:17:00
do you know what the blocker is for that? I wonder if anyone at work doing ARM enablement can help push it (of course it might not affect internal use as much)
<@conan_kudo:matrix.org>
16:17:15
!link https://lore.kernel.org/all/20241014105514.3206191-1-ryan.roberts@arm.com/
<@conan_kudo:matrix.org>
16:17:29
I think someone who cares needs to poke Arm to prioritize it
<@salimma:fedora.im>
16:17:36
15 months ago :O
<@salimma:fedora.im>
16:17:49
yeah let's try and follow up on that later on the side
<@conan_kudo:matrix.org>
16:18:21
it would affect your internal use too
<@salimma:fedora.im>
16:18:22
our Connect talk on improving kernel tooling packaging work by detaching them from the kernel is related to this
<@conan_kudo:matrix.org>
16:18:31
since it means you don't need to figure out page size at compile time for workloads
<@salimma:fedora.im>
16:18:38
imagine having to wait that 1 hour or twiddle with bconds just to change the packaging for, say, perf
<@conan_kudo:matrix.org>
16:18:51
e.g. NVIDIA Grace Hopper preferring 64k vs others preferring 16k and whatnot
<@salimma:fedora.im>
16:19:11
it will still not help get rid of an Asahi variant, right? page size is just one of the many differences it has
<@conan_kudo:matrix.org>
16:19:21
actually it would
<@salimma:fedora.im>
16:19:33
but it might help keep the compiler cache warm if code does not need to be recompiled for different page sizes
<@salimma:fedora.im>
16:19:45
oh!
<@conan_kudo:matrix.org>
16:19:48
the drivers are already built for all aarch64 flavors
<@conan_kudo:matrix.org>
16:20:01
it's just they're broken without 16k page size in practice because of hardware constraints
<@salimma:fedora.im>
16:20:27
nice, so driver binaries won't change with this patchset and don't have to be recompiled
<@conan_kudo:matrix.org>
16:20:32
it would be a massive improvement to _everyone_ if this landed
<@salimma:fedora.im>
16:20:48
ok let's continue this later, or we'll be discussing this for the whole hour :)
<@conan_kudo:matrix.org>
16:20:50
and I won't hate myself for the nvidia driver packaging for aarch64 in almalinux either :)
<@salimma:fedora.im>
16:22:10
my quick follow up related to this - perf 6.18 in Hyperscale seems to be working well, and we have ynl in hsx - I'll rebase that to 6.19 pre-release this week as one of the upstream devs wanted his new tool shipped out
<@salimma:fedora.im>
16:22:36
it's nice being able to decouple kernel tooling packaging and move faster than the kernel package itself :)
<@salimma:fedora.im>
16:22:45
anyone else has follow ups?
<@salimma:fedora.im>
16:23:21
I'll have a last one before we move to announcements, but that will probably trigger a few mins of discussions so I'll go last
<@conan_kudo:matrix.org>
16:23:43
nothing else from me followup wise
<@salimma:fedora.im>
16:24:26
ok... let's do this then.... we talked previously about finding a different time for our monthly video hangout
<@salimma:fedora.im>
16:24:54
right now it's 3rd Wednesday of the month, which... uh the Fedora calendar system does not support, so it goes out of sync everytime there's a month with 5 Wednesdays
<@salimma:fedora.im>
16:25:24
the time is also EU-unfriendly (and now we have james who is based in India so the current time is even worse for him)
<@salimma:fedora.im>
16:25:43
I am going to set up a whenisgood but... we probably need at least two different times
<@conan_kudo:matrix.org>
16:26:03
We could move it to the US morning but the days to choose are pretty difficult
<@salimma:fedora.im>
16:26:11
should we just do one US afternoon time every four weeks and one EU afternoon time every four weeks?
<@salimma:fedora.im>
16:26:31
yeah, the US morning / EU afternoon slots would be tough
<@salimma:fedora.im>
16:26:56
if we want to pin it to the month, we should probably abandon fedocal for this meeting, updating the calendar by hand each time it broke is tedious
<@conan_kudo:matrix.org>
16:27:04
we could split to twice a month alternating times
<@conan_kudo:matrix.org>
16:27:22
so every two weeks, but have alternating windows
<@conan_kudo:matrix.org>
16:27:44
that way it's once a month and we are able to support folks in APAC
<@conan_kudo:matrix.org>
16:27:56
that way it's once a month per slot and we are able to support folks in APAC
<@conan_kudo:matrix.org>
16:28:17
it's essentially the approach that the Fedora Social Hour uses
<@salimma:fedora.im>
16:28:24
yup
<@salimma:fedora.im>
16:28:46
do we still want twice a month, or do we want to just say every four weeks?
<@conan_kudo:matrix.org>
16:28:48
that's easier to rationalize, works with fedocal, and I don't have to keep editing it
<@conan_kudo:matrix.org>
16:29:11
we would say every two weeks and stagger the windows so they are different times
<@salimma:fedora.im>
16:29:12
if we want twice a month we should use either Google Calendar (ugh) or frama or something else that supports "nth week of the month"
<@salimma:fedora.im>
16:29:38
right, so for fedocal we just program in two meetings and they're every 4 weeks each
<@salimma:fedora.im>
16:29:41
that works for me
<@conan_kudo:matrix.org>
16:29:48
yep
<@conan_kudo:matrix.org>
16:29:53
just offset by two weeks
<@conan_kudo:matrix.org>
16:30:06
so in practice we have every two weeks, but each meeting is every four weeks
<@salimma:fedora.im>
16:30:11
so set up two whenisgood, and the EU friendly one, US folks can add the times where they *might* be available but don't worry if they can't attend each one
<@conan_kudo:matrix.org>
16:30:11
so in practice we have every two weeks, but each meeting window is every four weeks
<@conan_kudo:matrix.org>
16:30:17
yup
<@conan_kudo:matrix.org>
16:30:51
who wants to do the whenisgood?
<@salimma:fedora.im>
16:30:56
and likewise with the US friendly meeting time, EU/APAC folks feel free to chime in for the time that you might be available - we know where people are and can bias the selection to those actually in those areas
<@salimma:fedora.im>
16:30:59
I can do that
<@salimma:fedora.im>
16:31:14
!action Michel will set up whenisgood polls for US and EU HS video calls
<@conan_kudo:matrix.org>
16:31:18
awesome
<@salimma:fedora.im>
16:31:23
let's move on to...
<@salimma:fedora.im>
16:31:26
!topic Announcements
<@salimma:fedora.im>
16:31:51
I don't have anything - I think we covered FOSDEM/CentOS Connect already previously
<@salimma:fedora.im>
16:32:13
anyone else?
<@salimma:fedora.im>
16:33:38
ok, let's move on
<@salimma:fedora.im>
16:33:44
!topic Tickets
<@salimma:fedora.im>
16:34:35
we really should move our tickets to GitLab ... I forgot if anyone signed up for doing it already
<@salimma:fedora.im>
16:34:36
https://pagure.io/centos-sig-hyperscale/sig/issues
<@salimma:fedora.im>
16:34:59
I don't see anything new apart from membership, which we'll cover next - anyone has an update for the open tickets?
<@salimma:fedora.im>
16:35:50
the zstd / erofs one seems stalled Conan Kudohttps://pagure.io/centos-sig-hyperscale/sig/issue/191
<@conan_kudo:matrix.org>
16:36:15
this probably requires poking kernel people again
<@salimma:fedora.im>
16:36:57
istr there were some ... heated discussion between the erofs maintainer and devs of competing solutions, but I can't remember the details right now
<@salimma:fedora.im>
16:37:35
fedora /rh kernel people or upstream kernel people?
<@conan_kudo:matrix.org>
16:37:43
rh kernel people
<@salimma:fedora.im>
16:38:01
so hopefully we can find some of them at connect and/or at flock
<@conan_kudo:matrix.org>
16:38:09
this is in demand by other rhel product groups, so I just think it's a question of time
<@salimma:fedora.im>
16:38:14
or even devconf.cz though that conference is looking very impractical
<@salimma:fedora.im>
16:38:17
ah good
<@conan_kudo:matrix.org>
16:38:19
yeah...
<@salimma:fedora.im>
16:38:23
might even be worth asking in FRCL
<@conan_kudo:matrix.org>
16:38:28
yup
<@salimma:fedora.im>
16:38:31
but let's move on
<@salimma:fedora.im>
16:38:34
!topic Membership
<@pzmarzly:matrix.org>
16:38:37
qq: do we want to centralize all issues? i opened one at https://pagure.io/centos-sig-hyperscale/containers-releng/issue/6, now i wonder if i should have done it in main tracker instead
<@pzmarzly:matrix.org>
16:38:42
sorry, posted too late
<@salimma:fedora.im>
16:38:45
sorry to keep you waiting pzmarzly
<@salimma:fedora.im>
16:39:05
for open floor we should discuss moving membership processing to earlier in the meeting :)
<@salimma:fedora.im>
16:39:13
argh what on earth happened to that formatting
<@conan_kudo:matrix.org>
16:39:18
probably before tickets
<@salimma:fedora.im>
16:39:24
yeah
<@salimma:fedora.im>
16:39:53
I think if there is a separate subproject it makes sense to create an issue there
<@salimma:fedora.im>
16:41:03
but we should move that repo to gitlab too. even the one for the main SIG tracker - we probably don't need to port all the old issues, right? just start fresh and drop the old ones that are stalled anyway
<@salimma:fedora.im>
16:41:34
so anyway, Paveł, want to introduce yourself?
<@pzmarzly:matrix.org>
16:41:41
sure
<@pzmarzly:matrix.org>
16:41:57
it's Paweł with w, BTW (Polish alphabet is weird)
<@salimma:fedora.im>
16:42:10
argh I mised the two up
<@pzmarzly:matrix.org>
16:43:37
Hi everyone. I'm Pawel, Production Engineer at Meta, joining the Linux team. I'll be trying my best to help Michel and Davide with maintaining Hyperscale packages, hence my request for membership. I'll be attending FOSDEM and Flock later this year, hoping to meet some of you in person.
<@salimma:fedora.im>
16:44:24
Pawel already has a PR bringing our documentation up to date, which I'll try and merge today
<@salimma:fedora.im>
16:44:59
any questions / objections?
<@rcolebaugh:matrix.org>
16:45:30
Awesome, welcome Pawel! 😃
<@rcolebaugh:matrix.org>
16:45:37
No objections from me!
<@pzmarzly:matrix.org>
16:45:44
thanks!
<@salimma:fedora.im>
16:47:31
alrighty, we'll process this after this (sadly those of us who can process it have a back to back meeting after this, but later today anyway). welcome Pawel!
<@salimma:fedora.im>
16:47:43
and with that done, I guess it's time for...
<@salimma:fedora.im>
16:47:45
!topic Open floor
<@conan_kudo:matrix.org>
16:48:06
it's great to me too welcome :)
<@salimma:fedora.im>
16:48:10
a.k.a. Miscellaneous, we should finally do another PR to properly rename it
<@conan_kudo:matrix.org>
16:48:18
yaaaasssss
<@conan_kudo:matrix.org>
16:48:21
no more misc
<@salimma:fedora.im>
16:48:26
!action Michel will put up a PR moving up membership processing and renaming Miscellaneous
<@salimma:fedora.im>
16:48:47
I like spelling bees but even I hate typing that on the Mac keyboard
<@conan_kudo:matrix.org>
16:49:07
will Pawel be at our Hyperscale meetup at Connect?
<@salimma:fedora.im>
16:49:18
for those going to FOSDEM and CentOS Connect, we have an unofficial day out on Tuesday if you want to join
<@salimma:fedora.im>
16:49:39
should be I think - pzmarzlywhen do you arrive in Brussels?
<@salimma:fedora.im>
16:49:52
the meetup is... day 1 (Thursday) right? the morning if I recall
<@pzmarzly:matrix.org>
16:50:00
good question, let me check cal
<@conan_kudo:matrix.org>
16:50:27
yes
<@salimma:fedora.im>
16:51:08
Conan Kudoyou and Davide really need to find out where that place that serves the goat leg is
<@conan_kudo:matrix.org>
16:51:15
I know!
<@pzmarzly:matrix.org>
16:51:17
i'm arriving in brussels wednesday morning, so i'll be there on thursday
<@salimma:fedora.im>
16:51:17
would be a fun team activity
<@conan_kudo:matrix.org>
16:51:20
we'll need to find the old recipts
<@salimma:fedora.im>
16:51:35
but so far my favorite dish is the rabbit stew in Bruges ... don't think we'll end up there this year though
<@salimma:fedora.im>
16:52:22
anyone else has a topic? otherwise I will end the meeting a bit early
<@salimma:fedora.im>
16:54:15
alright ... nice seeing everyone after almost a month!
<@salimma:fedora.im>
16:54:27
and see some of you at CentOS Connect / FOSDEM in a few weeks... oh wait
<@salimma:fedora.im>
16:54:35
do we hold our next meeting in two weeks?
<@salimma:fedora.im>
16:55:15
I guess that's the day before Connect technically starts, so we could but we have the in-person hangout the day after (and ... time difference... 5 pm CET, iirc we normally end up helping folks set up / tour the conference venue
<@salimma:fedora.im>
16:55:25
so ... cancel next meeting?
<@davide:cavalca.name>
16:55:27
!hi
<@zodbot:fedora.im>
16:55:30
Davide Cavalca (dcavalca) - he / him / his
<@salimma:fedora.im>
16:55:40
you're back early!
<@davide:cavalca.name>
16:55:50
yeah, made it right in time
<@salimma:fedora.im>
16:56:51
I hear no objection to canceling the next meeting
<@davide:cavalca.name>
16:57:43
yeah that's fine by me
<@salimma:fedora.im>
16:57:46
!info the next Hyperscale meeting will be canceled as it will have taken place the day before CentOS Connect's Hyperscale meetup
<@salimma:fedora.im>
16:58:02
we should add that to the calendar for folks who are not attending in person - we're streaming it like last year, I guess?
<@salimma:fedora.im>
16:58:07
with that owl
<@davide:cavalca.name>
16:58:26
I'm not sure
<@davide:cavalca.name>
16:58:37
we've streamed in the past but remote participation was basically nil
<@davide:cavalca.name>
16:58:55
so I was inclined to just made this in person, unless someone is remote and really wants to attend
<@salimma:fedora.im>
16:59:01
yeah... good point
<@salimma:fedora.im>
16:59:29
that makes it easier to discuss sensitive topics too (though that generally does not come up)
<@davide:cavalca.name>
16:59:34
we do take notes, so folks shouldn't miss out too much either way
<@salimma:fedora.im>
16:59:36
one less gadget to pack for the trip then
<@davide:cavalca.name>
16:59:45
yeah that thing is huge lol
<@salimma:fedora.im>
17:00:10
I guess we can do it halfway and update the Matrix channel if there are interesting developments
<@salimma:fedora.im>
17:00:26
alright... looks like we're out of time. thanks everyone for coming!
<@salimma:fedora.im>
17:00:27
!endmeeting