<@tflink:fedora.im>
17:30:21
!startmeeting fedora-ai-ml-sig
<@meetbot:fedora.im>
17:30:22
Meeting started at 2026-01-15 17:30:21 UTC
<@meetbot:fedora.im>
17:30:23
The Meeting name is 'fedora-ai-ml-sig'
<@tflink:fedora.im>
17:30:36
!topic roll call and welcome
<@tflink:fedora.im>
17:30:39
!hi
<@zodbot:fedora.im>
17:30:39
Tim Flink (tflink)
<@tflink:fedora.im>
17:30:46
who all is here for the ai-ml sig meeting?
<@xanderlent:fedora.im>
17:31:22
!hi
<@zodbot:fedora.im>
17:31:23
Alexander Lent (xanderlent) - he / him / his
<@tflink:fedora.im>
17:32:57
tom said that he might be a bit late to the meeting today
<@tflink:fedora.im>
17:33:23
is it just the two of us otherwise?
<@tflink:fedora.im>
17:36:06
Alexander Lent: should your topic about non-x86_64 wait for tom to be here? I haven't been following 100% but it seems like there has been some ... discussion around non-x86 arches lately and I'm not sure what exactly you were looking to talk about
<@xanderlent:fedora.im>
17:38:09
Yeah, I think we'd best wait a little.
<@tflink:fedora.im>
17:38:35
ok, we'll wait a bit. tom didn't say how late he'd be, hopefully it isn't until the top of the hour
<@conan_kudo:matrix.org>
17:39:03
!hi
<@zodbot:fedora.im>
17:39:05
Neal Gompa (ngompa) - he / him / his
<@tflink:fedora.im>
17:41:25
hi, we're waiting a bit for tom. he said that he'd be a bit late today and most if not all of the topics today involve him
<@conan_kudo:matrix.org>
17:42:00
this is the first one I've been able to make in a while because of some fortuitous schedule juggling
<@conan_kudo:matrix.org>
17:42:06
normally I'm triple booked 😅
<@tflink:fedora.im>
17:44:19
there are a limited number of meeting times that more or less work for both NA (especially western NA) and EMEA (not to mention APAC) so that does lead to lots of stuff at the same time slots :)
<@conan_kudo:matrix.org>
17:45:31
yeah I don't expect my ability to come to this to be a regular thing
<@trix:fedora.im>
17:45:49
hey, sorry i am late. another meeting.
<@conan_kudo:matrix.org>
17:46:15
but I did want to register that there are people interested in KDE software leveraging ollama and the AI/ML stack on x86_64, aarch64, and riscv64
<@trix:fedora.im>
17:46:39
yes.
<@conan_kudo:matrix.org>
17:46:51
specifically Alpaka is being actively developed in KDE and we'd like to eventually bring it into Fedora
<@conan_kudo:matrix.org>
17:47:03
!link https://apps.kde.org/alpaka/
<@xanderlent:fedora.im>
17:47:08
Conan Kudo: Ollama upstream only supports x86_64 and aarch64, they strip out sources for all other arches in the bundled dependencies.
<@trix:fedora.im>
17:47:09
do we want to put that up as a feature folks work towards ?
<@conan_kudo:matrix.org>
17:47:25
Alexander Lent: yes I know, riscv64 is a "down the road" thing
<@xanderlent:fedora.im>
17:47:28
I'm happy to help with this, though I have not yet engaged upstream
<@conan_kudo:matrix.org>
17:47:33
this is about registering interest
<@conan_kudo:matrix.org>
17:47:41
and asking for help :)
<@tflink:fedora.im>
17:47:52
cool, we can get started
<@conan_kudo:matrix.org>
17:48:07
davdunc has also been interested in building AI things on top of Fedora KDE for local and cloud stuff
<@conan_kudo:matrix.org>
17:48:24
I don't know if he's around but I figured I'd bring that up
<@conan_kudo:matrix.org>
17:48:40
just so everyone is aware
<@trix:fedora.im>
17:49:12
do we want a general ollama addons feature ?
<@tflink:fedora.im>
17:49:46
!info there are KDE folks interested in leveraging ollama and the AI/ML stack on x86_64, aarch64 and riscv64
<@trix:fedora.im>
17:50:23
i just don't want a list of things and no one doing the work. i do want to see LLM going throughout fedora.
<@conan_kudo:matrix.org>
17:50:26
I think tosky is one of the folks contributing in upstream too, so there's at least a familiar face :)
<@mystro256:fedora.im>
17:50:43
!hi
<@zodbot:fedora.im>
17:50:44
None (mystro256)
<@mystro256:fedora.im>
17:50:51
Sorry im late
<@tflink:fedora.im>
17:51:03
no worries, we're a bit late getting started
<@conan_kudo:matrix.org>
17:51:08
Tom Rix: oh sure, I'm not saying there won't be help to get things in place, but I want people to be aware so nobody is surprised
<@conan_kudo:matrix.org>
17:51:32
I expect that folks from Fedora ARM, Fedora RISC-V, and Fedora KDE will show up and want to help
<@conan_kudo:matrix.org>
17:51:48
and may even need guidance so they can help
<@conan_kudo:matrix.org>
17:51:58
and may even need guidance so they can help effectively
<@trix:fedora.im>
17:52:02
let me go over the state of ollama.
<@tflink:fedora.im>
17:52:09
!topic state of ollama
<@trix:fedora.im>
17:53:08
golang reorged its best practices to make bundling go in packages . so all of the go code is in ollama now.
<@trix:fedora.im>
17:54:03
this is better in that build works without a lot of hacking and dependencies so I was able to do the update for fedora AND get it going on EPEL.
<@trix:fedora.im>
17:54:40
the downside is now looking after all the go depenencies. so pluses and minuses.
<@trix:fedora.im>
17:55:25
Alexander Lent: enabled aarch64 and has offered to maintain it.
<@trix:fedora.im>
17:56:06
backends we only have ROCm. we could i believe easily have opencl, vulkan etc.
<@trix:fedora.im>
17:56:45
if we want to go into riscv, or s390 etc. we need some help.
<@conan_kudo:matrix.org>
17:56:58
I know the Fedora RISC-V SIG is interested in helping
<@tflink:fedora.im>
17:57:03
!info due to changes in golang packaging policies, the go bits of ollama are now being bundled which has made an update to Fedora and EPEL easier
<@conan_kudo:matrix.org>
17:57:06
I don't know anything about IBM Z
<@tflink:fedora.im>
17:57:30
!info aarch64 support has been enabled and Alexander Lent will be maintaining that
<@xanderlent:fedora.im>
17:57:37
libGGML supports basically anything you can think of, the issue is mostly wiring that up in Ollama
<@trix:fedora.im>
17:57:39
i also have python-ollama package. and have been trying to get emacs+ollama going but that did not get very far.
<@trix:fedora.im>
17:58:16
there is a huge list of integrations on the ollama site i loving look at but have no time to do, alpaka was one of them.
<@trix:fedora.im>
17:59:16
and i believe ollama is more useful day-to-day than any other ai thingie.
<@conan_kudo:matrix.org>
17:59:32
I think Steve Cossette (Farchord) was interested in bringing alpaka in, and maybe he already made a package review request for it?
<@conan_kudo:matrix.org>
17:59:40
I vaguely recall a spec already
<@tflink:fedora.im>
18:00:24
I don't see anything with a quick search in rhbz
<@conan_kudo:matrix.org>
18:00:31
ah no, but we have a ticket about it, and I just reopened it
<@conan_kudo:matrix.org>
18:00:32
!link https://pagure.io/fedora-kde/SIG/issue/451
<@farchord:fedora.im>
18:00:35
Mmmm I swear I made something for this
<@farchord:fedora.im>
18:00:55
Oh
<@conan_kudo:matrix.org>
18:00:58
I thought you did too 😅
<@farchord:fedora.im>
18:01:14
Technically we can get it in.... But Fedora keeps a very (ish) old version of ollama in their repos
<@tflink:fedora.im>
18:01:15
my faith in my quick rhbz search is not terribly high, I could have easily missed it :)
<@conan_kudo:matrix.org>
18:01:50
it probably makes sense tflink to add an %ollama_arches thing so that we can lock dependents to it?
<@conan_kudo:matrix.org>
18:01:59
oof I meant Tom Rix
<@trix:fedora.im>
18:03:01
sure, let's add that to the feature.
<@trix:fedora.im>
18:03:23
not sure how to do that for backends.
<@trix:fedora.im>
18:04:57
ah.. forgot sysctl is working now.
<@tflink:fedora.im>
18:06:24
is there anything else to say on this topic?
<@tflink:fedora.im>
18:06:34
it sounds like we can move on
<@tflink:fedora.im>
18:08:03
Alexander Lent: has this covered the bits you wanted to bring up WRT non-x86 support/landscape?
<@xanderlent:fedora.im>
18:08:44
Yes, I think we've overlapped sufficiently. Let's move on.
<@tflink:fedora.im>
18:08:52
ok, making sure :)
<@tflink:fedora.im>
18:08:56
!topic F44 branch readiness
<@tflink:fedora.im>
18:09:19
!info F44 branch will be on 2026-02-03
<@trix:fedora.im>
18:10:54
All my ai things are updated and i am in the process of testing them. this is only on x86 and with rocm backend. concurrently this is also close to epel 10.2
<@tflink:fedora.im>
18:11:12
are there any packages that we're worried about WRT being ready for F44 branch?
<@trix:fedora.im>
18:11:37
onnx and onnxruntime i believe are still not building for anyone.
<@trix:fedora.im>
18:11:41
let me check..
<@trix:fedora.im>
18:13:28
and i am not sure if openvino is.
<@tflink:fedora.im>
18:13:34
!info onnx and onnxruntime are currently not building and will need help
<@trix:fedora.im>
18:14:03
maybe a hangover from gcc 16 update. since these aren't my packages, i don't know for sure.
<@xanderlent:fedora.im>
18:15:24
Tom Rix: Is PyTorch ready to go in Rawhide?
<@xanderlent:fedora.im>
18:15:46
I saw an FTI yesterday on x86_64 related to MIOpen, but I'm not sure if that was related to the ongoing work.
<@trix:fedora.im>
18:15:52
the release is set to 2.9.1 and is what i am testing now.
<@trix:fedora.im>
18:16:32
miopen failure is the boost update to 1.90, i tried rebuilding it yesterday and still got 1.83 in mock. i will try again today.
<@tflink:fedora.im>
18:16:48
!info python-torch will be updated to 2.9.1 soon, currently being tested
<@xanderlent:fedora.im>
18:17:04
Ah, thank you!
<@trix:fedora.im>
18:17:11
boost update is also effect some other packages ..
<@trix:fedora.im>
18:17:24
yeah late updates!
<@tflink:fedora.im>
18:18:18
!info the boost 1.90 update is also affecting some other packages, that is being sorted out
<@trix:fedora.im>
18:19:13
my testing is around some containers i can share with folks offline. i am working on moving away from manual and its take a little longer. if folks have some expample of workload you want to see tested. give it to me an i will work it into a container like the rocm_check from yesterday.
<@dherrera:fedora.im>
18:19:40
I blocked the rest of my day to look into this btw
<@dherrera:fedora.im>
18:19:58
this is on rawhide, right?
<@trix:fedora.im>
18:20:04
yes.
<@trix:fedora.im>
18:20:35
i use this copr https://copr.fedorainfracloud.org/coprs/g/rocm-packagers-sig/RH/packages/ to track things
<@trix:fedora.im>
18:21:11
onnxruntime has a rocm integration and pytorch has a onnx,onnxruntime requires so it get covered.
<@tflink:fedora.im>
18:21:49
anything else for F44 prep?
<@tflink:fedora.im>
18:22:52
Tom Rix: I doubt that we have time for both of the remaining topics, do you want to do your rocm compat proposal or the ai packagers sig stuff?
<@trix:fedora.im>
18:23:06
lets do rocm-compat its the oldest.
<@tflink:fedora.im>
18:23:14
!topic F45 ROCm compatiblity poposal
<@tflink:fedora.im>
18:23:20
!link https://fedoraproject.org/wiki/SIGs/HC#Compatibility_Proposal
<@trix:fedora.im>
18:24:50
this is technical proposal on how multiple rocm's could be delivered . I am looking for feed back, to firm it up into something real enough to figure out the effort envolved. then see if we have folks enough to do the work.
<@trix:fedora.im>
18:25:34
baked in here is an LTS ROCm version that i think would be useful.
<@tflink:fedora.im>
18:25:37
ie, having multiple major versions of rocm installed at the same time in the same release instead of the current approach of limiting rocm to one version per release
<@trix:fedora.im>
18:25:46
yes.
<@trix:fedora.im>
18:26:09
ex/ we have ROCm 7.2.x LTS while ROCm 8's are rolling in
<@tflink:fedora.im>
18:26:15
so it would probably affect anything that uses ROCm but hopefully not in a horrible way :)
<@trix:fedora.im>
18:27:34
yes it would. it would allow integrations to pin to older version while they work out how to move to recent versions. it is a real PITA when there is a major update in ROCm that does work with the recent public version of pytorch (for example)
<@tflink:fedora.im>
18:28:12
it does look like the rate of change in ROCm is going to be accelerating starting with ROCm 8 so having multiple versions co-installable would make the maintenance issues less for the things using ROCm
<@trix:fedora.im>
18:28:59
yes. this is also a mitegration for that. I would have expected ROCm 7 to have 2 years, not 6 months of life.
<@mystro256:fedora.im>
18:31:00
Is that a wish or an actual commitment from amd?
<@mystro256:fedora.im>
18:31:25
as far as I know, AMD only supports latest
<@trix:fedora.im>
18:31:36
it is more looking at the past. 6.x was ok for a couple of years.
<@mystro256:fedora.im>
18:31:58
so basically unsupported upstream, supported in Fedora?
<@trix:fedora.im>
18:32:59
The nature of what LTS means would need to be worked out with AMD. I do not want to get into a bind supporting things ex/ 10 years when AMD only does 2.
<@mystro256:fedora.im>
18:33:27
The problem is that we might end up with majors every year or something
<@mystro256:fedora.im>
18:33:43
and I doubt AMD will support multiple majors at all.
<@trix:fedora.im>
18:33:55
yes. the proposal only have a single old major.
<@tflink:fedora.im>
18:34:28
!info there is a proposal (linked above) to enable having multiple versions of ROCm installed at the same time in the same release. This would be a departure from how ROCm has been packaged in the past and feedback on this proposal would be appreciated
<@trix:fedora.im>
18:35:03
the intent is to help people move to the newest rocm without requiring it to happen immediately.
<@mystro256:fedora.im>
18:35:14
well if we had 8 this year, and 9 next, I think that would mean only really a year of LTS in the current model, so having one old major would mean 2?
<@mystro256:fedora.im>
18:35:38
just conjecture of course, I have no idea what's actually commited by AMD
<@trix:fedora.im>
18:35:40
once 9 comes in, 7 goes out.
<@trix:fedora.im>
18:35:53
if that is this year or 10yr
<@tflink:fedora.im>
18:35:57
!info the intent of the ROCm compatibility proposal is to make it easier for other packages to use the latest ROCm without requiring them to move at the pace of upstream ROCm
<@mystro256:fedora.im>
18:36:26
ok gotcha, so if it's yearly, it's 2 years, if its more, it's more
<@trix:fedora.im>
18:36:37
support 1 old major will be hard. i don't want to sign up for N of them.
<@trix:fedora.im>
18:37:43
this at least doubles the work on rocm side. but reduces friction on the user side.
<@trix:fedora.im>
18:38:58
the mechanics i think are 50+ compat repo's based off of rawhide everytime we want to pull the trigger.
<@tflink:fedora.im>
18:39:19
yeah, we're going to have to invest in more scripting to pull that off, I think
<@trix:fedora.im>
18:39:21
that is also itself a lot of overhead.
<@tflink:fedora.im>
18:39:49
we're almost 10 minutes over time, anything else on this topic?
<@trix:fedora.im>
18:39:59
thanks for the extra time!
<@trix:fedora.im>
18:40:12
we can talk more about this as F45 rolls out.
<@tflink:fedora.im>
18:40:17
sounds good
<@tflink:fedora.im>
18:40:21
!topic open floor
<@carlwgeorge:fedora.im>
18:40:29
!hi
<@tflink:fedora.im>
18:40:29
anything else that needs to be brought up this week?
<@zodbot:fedora.im>
18:40:30
Carl George (carlwgeorge) - he / him / his
<@carlwgeorge:fedora.im>
18:41:08
python3-torch and magma are busted in epel 10.2, it seems like they got missed in the rocm 7 rebuild
<@carlwgeorge:fedora.im>
18:41:14
!link https://bugzilla.redhat.com/show_bug.cgi?id=2428183
<@carlwgeorge:fedora.im>
18:41:20
!link https://bugzilla.redhat.com/show_bug.cgi?id=2428182
<@trix:fedora.im>
18:42:00
yes. the pytorch build fails, i am not sure why, it looks like something python related changed in epel.
<@trix:fedora.im>
18:42:41
magma i think i just missed in the 7.1 update, that will be covered in the 7.1.1 i am doing now.
<@farchord:fedora.im>
18:43:10
Tom Rix Is there a plan to update ollama a bit more often? Not pushy or anything, but I checked recently and noticed it was a decent ways behind upstream's versioning
<@carlwgeorge:fedora.im>
18:43:15
in the future, can the rocm major rebases not be pushed to stable if there are build failures? ideally epel should have repoclosure (i.e. all packages successfully install)
<@trix:fedora.im>
18:43:38
so a couple of questions.
<@trix:fedora.im>
18:44:00
i think with the bundling of the ollama code, we are in a lot better shape.
<@farchord:fedora.im>
18:44:12
Ah, ok
<@trix:fedora.im>
18:44:56
i just dinked it recently because i did not want to big change to upset the sysctl work i just did.
<@trix:fedora.im>
18:45:17
and i don't have time to go back and retest that.
<@farchord:fedora.im>
18:45:32
Are you looking for help in updating it? If it's not too elaborate I'd be happy to help if I can.
<@trix:fedora.im>
18:45:34
once F44 branch we can jump to what ever is the latest and try that.
<@trix:fedora.im>
18:45:47
similar for pytorch going to 2.10
<@trix:fedora.im>
18:47:31
Steve Cossette (Farchord): yes. let get that going, in Feb. if you want to try sooner, there is an option to build newer versions baked into ollama and pytorch. use the with gitcommit logic to put your changes in
<@trix:fedora.im>
18:48:39
if we really want to do this, maybe we have an ollama-next copr ?
<@farchord:fedora.im>
18:49:26
You can. In this case, I'll start looking at ollama again. But man I wish it was as good as that one GTK app that works with ollama....
<@farchord:fedora.im>
18:49:30
Trying to remember the name
<@trix:fedora.im>
18:49:56
those apps i would like to get into the copr.
<@tflink:fedora.im>
18:51:57
!info there is interest in getting ollama updates to land faster and create a new COPR for development and update purposes
<@carlwgeorge:fedora.im>
18:52:40
i think this got missed, but i am looking for an answer here. the approval to do the major rebase of rocm in epel10 (https://pagure.io/epel/issue/354) was based on the assumption that all the packages would be rebuilt. pushing updates in epel that break the installation of other packages is not permissible. if there was a known pytorch failure then the update should have been delayed until it was sorted, and if it can't get sorted then pytorch should be retired from epel10.
<@trix:fedora.im>
18:54:07
if folks want to help sort that out, that would be good. keep in mind this is just pytorch. getting the other torch things in aren't there so its not that useful right now.
<@trix:fedora.im>
18:54:56
i will be trying to sort that out as part of the rocm update. from what i can tell it is not specifically a rocm problem, so it is not related.
<@tflink:fedora.im>
18:55:28
I think that the request is that if there are build failures in the future to not push stuff stable until that is dealt with even if it isn't directly rocm related
<@tflink:fedora.im>
18:55:46
ie, don't push anything that breaks repoclosure
<@trix:fedora.im>
18:56:46
it fails on strsight cpu builds.
<@trix:fedora.im>
18:57:29
sure. i though i would have time to sort it out, i did not.
<@farchord:fedora.im>
18:58:23
Tom Rix do you know if there's a plan to package open-webui?
<@farchord:fedora.im>
18:58:52
For those that don't work, it's a really nice webui for managing ais, going from Stable Diffusion to Ollama
<@farchord:fedora.im>
18:59:12
For those that don't know, it's a really nice webui for managing ais, going from Stable Diffusion to Ollama
<@trix:fedora.im>
18:59:39
this is what i mean about needing help .. i am using open-webui now in a container for my ollama testing, so it is great! but doing the work to package it is not going to be easy, not something i have time for.
<@tflink:fedora.im>
18:59:40
!info concerns about the recent ROCm 7 update for EPEL breaking repoclosure were raised and clarifications have been made
<@farchord:fedora.im>
19:00:10
Yeah I get that. Maybe we could look at a cockpit module for it?
<@trix:fedora.im>
19:00:37
it does not build on rawhide because iirc pytorch 3.14 is not supported.
<@xanderlent:fedora.im>
19:01:19
Last I looked into Open-WebUI, you effectively have to run it in containers or with ancient versions of python, so it won't be possible to package it as a system package without some unusual exceptions.
<@trix:fedora.im>
19:01:20
fighting with bleeding python is an ongoing issue for anything ai related
<@trix:fedora.im>
19:02:03
pytorch 2.9.1 is not supported on 3.14, yet we have it.
<@tflink:fedora.im>
19:03:03
ok, anything else for open floor?
<@tflink:fedora.im>
19:03:14
we're more than 30 minutes over time at this point
<@tflink:fedora.im>
19:04:25
thanks for coming, everyone
<@tflink:fedora.im>
19:04:33
!endmeeting