#meeting-2:fedoraproject.org log

<@tflink:fedora.im>

17:30:04

!startmeeting fedora-ai-ml-sig

<@meetbot:fedora.im>

17:30:05

Meeting started at 2025-02-13 17:30:04 UTC

<@meetbot:fedora.im>

17:30:05

The Meeting name is 'fedora-ai-ml-sig'

<@tflink:fedora.im>

17:30:08

!topic welcome

<@tflink:fedora.im>

17:30:09

!hello

<@zodbot:fedora.im>

17:30:10

Tim Flink (tflink)

<@man2dev:fedora.im>

17:30:21

!hi

<@zodbot:fedora.im>

17:30:22

Mohammadreza Hendiani (man2dev)

<@tflink:fedora.im>

17:30:23

who all's here for the ai-ml sig meeting?

<@tflink:fedora.im>

17:33:28

let's get started and hopefully a few more folks will find their way here in the meantime

<@tflink:fedora.im>

17:33:41

!topic previous meeting follow-up

<@tflink:fedora.im>

17:33:50

!link https://discussion.fedoraproject.org/t/figuring-out-npu-support-in-fedora/143717

<@ludiusvox:fedora.im>

17:34:22

Hey Tflink how are you

<@tflink:fedora.im>

17:34:43

!info last meeting we talked about NPU support in Fedora. There has been one update to the discourse thread about what is needed for Intel's NPUs but more discussion will be needed

<@mystro256:fedora.im>

17:34:55

!hi

<@tflink:fedora.im>

17:34:57

anyone have anything to add on the NPU discussion from last week?

<@zodbot:fedora.im>

17:35:00

None (mystro256)

<@ludiusvox:fedora.im>

17:35:00

Tom Rix: I showed up I know my attendance been spotty

<@tflink:fedora.im>

17:35:28

I assume not for the moment or it'll come up in today's topics :)

<@trix:fedora.im>

17:35:29

!hi

<@zodbot:fedora.im>

17:35:31

Tom Rix (trix)

<@tflink:fedora.im>

17:36:03

moving on to today's topics

<@tflink:fedora.im>

17:36:10

!topic Laptop testing for F42

<@tflink:fedora.im>

17:36:20

Tom Rix: this one is yours, take it away

<@ludiusvox:fedora.im>

17:36:22

I wasn't here last time and I am not going to ask to be caught up, so it's fine.

<@ludiusvox:fedora.im>

17:36:22

The last meeting I went to we were planning on training granite for use of documentation assistance and I demonstrated RAG-bot LLM code with langchain.

<@ludiusvox:fedora.im>

17:36:22

Can I ask as to the status of the Granite LLM?

<@ludiusvox:fedora.im>

17:36:22

<@ludiusvox:fedora.im>

17:36:22

<@tflink:fedora.im>

17:36:54

Aaron Linder: would it be ok to leave that for open floor?

<@ludiusvox:fedora.im>

17:37:01

Yes sir

<@trix:fedora.im>

17:37:48

I enabled a lot of AMD laptops i don't have have. anyone want to poke at ai in the F42 testing now ?

<@xanderlent:fedora.im>

17:37:53

!hi

<@zodbot:fedora.im>

17:37:54

Alexander Lent (xanderlent) - he / him / his

<@tflink:fedora.im>

17:38:05

which ones were enabled?

<@trix:fedora.im>

17:38:28

680M, 780M, strix*

<@trix:fedora.im>

17:38:50

folks likely have 680M or 780M

<@tflink:fedora.im>

17:38:51

!info several AMD laptops have been enabled for the AI stack (680M, 780M, strix*) and testing would be appreciated if folks have the hardware and time

<@tflink:fedora.im>

17:39:20

I have a 780M and will try to do some testing over the weekend

<@trix:fedora.im>

17:39:26

thanks!

<@tflink:fedora.im>

17:39:37

are you looking for anything in particular? just some basic poke-at-pytorch?

<@trix:fedora.im>

17:39:54

pytorch would be good.

<@trix:fedora.im>

17:40:09

i only run the unit tests, i don't really do much else.

<@tflink:fedora.im>

17:41:24

is there anything else that folks wanted to add on this?

<@ludiusvox:fedora.im>

17:41:51

I appreciate the ability and ease of use and performance of my AMD 6800 on fedora workstation

<@trix:fedora.im>

17:42:16

that also needs testing, i don't have a 6800.

<@tflink:fedora.im>

17:42:29

yeah, we have almost no representation for gfx10

<@ludiusvox:fedora.im>

17:43:03

I would need assistance on running the rest because I don't have F42 rawhide installed to be able to do experimental testing

<@trix:fedora.im>

17:43:24

i have container thingie..

<@ludiusvox:fedora.im>

17:43:36

Okay I we can figure it out later

<@tflink:fedora.im>

17:44:00

it might be interesting to see containerized test results. for better or worse the kernels tend to be similar if not the same in fedora releases

<@trix:fedora.im>

17:44:09

https://github.com/trixirt/rocm-distro-containers/blob/main/fedora/f42/pytorch/Dockerfile

<@trix:fedora.im>

17:44:28

lets talk about container later, i just cooked it up yesterday for someone else.

<@ludiusvox:fedora.im>

17:44:29

I'll put this in Google keep

<@tflink:fedora.im>

17:44:33

but yeah, we can talk about containerized testing outside of the meeting

<@man2dev:fedora.im>

17:44:41

Mock v 6 support container builds now

<@tflink:fedora.im>

17:45:04

anything else on this topic or shall we move on?

<@trix:fedora.im>

17:45:11

move on.

<@tflink:fedora.im>

17:45:33

!topic non x86 handing

<@trix:fedora.im>

17:45:50

pytorch is also build for aarch64.

<@tflink:fedora.im>

17:45:52

the perpetual topic that ends in "nobody has HW" :)

<@trix:fedora.im>

17:46:01

i don't have that hw.

<@tflink:fedora.im>

17:46:16

does rocm even build for aarch64?

<@ludiusvox:fedora.im>

17:46:22

Okay I made a sticky note for later

<@tflink:fedora.im>

17:46:32

or is that cpu only?

<@trix:fedora.im>

17:46:35

not really, that is another reason it has no love from me.

<@trix:fedora.im>

17:46:41

yes, cpu only.

<@mystro256:fedora.im>

17:47:30

rocm is spotty on nonx86

<@tflink:fedora.im>

17:47:32

!info testing is needed for cpu-only build of pytorch on aarch64

<@mystro256:fedora.im>

17:47:39

right now it doesn't build for abig chunk

<@trix:fedora.im>

17:47:43

this is also a warning, if it seriously breaks on aarch64, i will not fix it.

<@mystro256:fedora.im>

17:48:13

basically if it compiles, it's use at your own risk

<@mystro256:fedora.im>

17:48:35

I mean you could run pytorch on cpu no?

<@mystro256:fedora.im>

17:48:40

not sure if it makes sense though

<@ludiusvox:fedora.im>

17:48:56

Okay

<@tflink:fedora.im>

17:49:13

it works, it just tends to take forever on anything non-trivial :)

<@tflink:fedora.im>

17:49:40

the only accelerator I know of right now that works with aarch64 is some nvidia stuff

<@trix:fedora.im>

17:50:09

anyone have time & interested ?

<@tflink:fedora.im>

17:50:29

!info if you do end up testing pytorch with aarch64 HW, please report your findings in #ai-ml:fedoraproject.org

<@tflink:fedora.im>

17:50:53

I don't have any aarch64 HW available for testing, unfortunately

<@xanderlent:fedora.im>

17:51:04

I know of some other folks working on aarch64 AI/ML accelerators, (for example Mesa libTeflon or the Asahi ANE project) but nothing w.r.t. pytorch specifically.

<@man2dev:fedora.im>

17:51:16

Im looking into how to add aarch53 without breaking x86

<@man2dev:fedora.im>

17:51:34

Im looking into how to add aarch64 without breaking x86

<@trix:fedora.im>

17:51:44

any way.. i think we can move on.

<@tflink:fedora.im>

17:51:46

yeah, I think there are more accelerators coming but nothing else is working and available right now that I know of

<@tflink:fedora.im>

17:51:48

yep

<@ludiusvox:fedora.im>

17:51:56

I have a mediatek Chromebook, but I don't get a distrochoice in android as to what container I install and I heard there will be changed with android and ChromeOS in general

<@man2dev:fedora.im>

17:51:58

But ihaven't find anytging conclusive to testout

<@tflink:fedora.im>

17:52:13

!topic Granite and documentation assistance

<@tflink:fedora.im>

17:52:31

Aaron Linder: I wasn't sure what to call this but it's your topic from earlier

<@ludiusvox:fedora.im>

17:52:55

Thanks, as I was saying earlier what's the status granite is coming along, I have seen some users on social media worried about telemetry and privacy

<@tflink:fedora.im>

17:53:24

I'm not sure there has been any progress in Fedora but to be honest, I haven't been paying a ton of attention

<@ludiusvox:fedora.im>

17:53:40

I think that IBM has been working on it for red hat

<@tflink:fedora.im>

17:53:48

in Fedora?

<@ludiusvox:fedora.im>

17:54:16

I am not able to locate this article or actually it was a YouTube podcast but I read the general headline about it and I posted my 02 cents

<@tflink:fedora.im>

17:54:31

I know that there was a new version of the granite models released in the last week or so. I think they're available on HF

<@ludiusvox:fedora.im>

17:54:38

I can go look on YouTube rq with a query

<@ludiusvox:fedora.im>

17:54:51

Let me go look brb

<@xanderlent:fedora.im>

17:57:07

https://blogs.gnome.org/uraeus/2025/02/03/looking-ahead-at-2025-and-fedora-workstation-and-jobs-on-offer/

<@xanderlent:fedora.im>

17:57:07

There is apparently work going on at IBM/RH on Granite and AI in general in Fedora, including accelerated workloads, according to this blog post: (first heading, "Artificial Intelligence")

<@ludiusvox:fedora.im>

17:57:23

https://www.fudzilla.com/news/60487-red-hat-plans-to-integrate-ai-with-ibm-s-granite-engine#:~:text=The%20Red%20Hat%20team%20has%20announced%20plans%20to,environments%20%28IDEs%29%20and%20create%20an%20AI-powered%20Code%20Assistant.

<@ludiusvox:fedora.im>

17:57:34

And it's something with fedora also

<@ludiusvox:fedora.im>

17:57:56

This is not the original article but I have heard some buzz about this

<@tflink:fedora.im>

17:58:15

honestly, I'm not sure what exactly is planned. I think that's all workstation stuff

<@tflink:fedora.im>

17:58:59

it'll be interesting to see what they have planned but I'll bet it centers around ramallama and maybe toolbx

<@ludiusvox:fedora.im>

17:59:11

But nobody in here knew about this and I don't know who to ask

<@tflink:fedora.im>

17:59:14

I don't recall if the ramallama review passed or not

<@tflink:fedora.im>

18:00:02

the fedora workstation room would be a good place to start, I think

<@man2dev:fedora.im>

18:00:08

Ramallama is in

<@tflink:fedora.im>

18:00:30

cool, thanks for the update

<@xanderlent:fedora.im>

18:01:00

<@xanderlent:fedora.im>

18:01:00

Seems exactly that:

<@xanderlent:fedora.im>

18:01:00

"We been brainstorming various ideas in the team for how we can make use of AI to provide improved or new features to users of GNOME and Fedora Workstation. This includes making sure Fedora Workstation users have access to great tools like RamaLama, that we make sure setting up accelerated AI inside Toolbx is simple, that we offer a good Code Assistant based on Granite and that we come up with other cool integration points."

<@tflink:fedora.im>

18:01:27

which would run locally but then HW enablement is an issue

<@ludiusvox:fedora.im>

18:02:09

Which I am okay with it, if the person who knew about this was in here I would only request model variability if running off of an ollama compatible system to have model quantization for lower performance machines.

<@ludiusvox:fedora.im>

18:02:09

But I have a feeling that this will be a telemetry based system with an API, I guess we would have to ask the system architect who I met at flock he probably knows something about this

<@ludiusvox:fedora.im>

18:02:09

<@tflink:fedora.im>

18:02:28

ramallama is local only AFAIK

<@tflink:fedora.im>

18:02:42

its roughly equivalent to ollama and vllm in terms of functionality

<@tflink:fedora.im>

18:03:27

I'd be a bit surprised if the AI stuff was anything other than local just due to cost

<@man2dev:fedora.im>

18:03:52

Its just llama-cpp and vllm with multiple backends based on how you setup your Container

<@ludiusvox:fedora.im>

18:03:52

Okay I will have to go test ramallama I have gotten Ollama to work with a custom install.sh to make compatible with AMD GPU which when I get home I can post somewhere the custom install.sh

<@tflink:fedora.im>

18:04:34

ah, ok. I haven't gotten around to actually looking at it yet, I've just been hearing about it :)

<@tflink:fedora.im>

18:04:59

was rocm enabled for ramallama?

<@man2dev:fedora.im>

18:05:26

I Don't remmber

<@ludiusvox:fedora.im>

18:06:31

I am not sure I don't know enough about ramallama, but the increased capabilities I have had in python packages for ollama I found a 1/2 way done repository which I am talking to the author, and I got ollama working for F41 for ROCm and it's a package to apply for jobs for me a robot it's funny it's just WIP work in progress

<@man2dev:fedora.im>

18:06:55

I know vllm had cuda image

<@ludiusvox:fedora.im>

18:07:11

Yeah I think that some custom work needs to be done for ROCm

<@tflink:fedora.im>

18:07:24

vllm can be built with rocm support but I think there are patches required from amd

<@ludiusvox:fedora.im>

18:08:00

So I think a ramallama option based modification to the install.sh would need to be done

<@ludiusvox:fedora.im>

18:08:19

Or it is an rpm package

<@tflink:fedora.im>

18:08:46

it sounds like ramallama was approved as a fedora package

<@ludiusvox:fedora.im>

18:09:22

So an rpm auto detect system hardware I have no idea how to inspect packages I just can do install shell scripts

<@man2dev:fedora.im>

18:10:27

It mostly seems to use podman functionality

<@tflink:fedora.im>

18:10:30

Aaron Linder: have your questions been somewhat answered? at least to the point where you know where to ask for more info?

<@ludiusvox:fedora.im>

18:12:13

Yes sir thank you

<@ludiusvox:fedora.im>

18:12:20

I will get on ramallama

<@tflink:fedora.im>

18:13:03

cool, then we can move on to

<@tflink:fedora.im>

18:13:06

!topic open floor

<@tflink:fedora.im>

18:13:13

any other things that folks want to bring up?

<@ludiusvox:fedora.im>

18:14:16

I somehow got mediapipe working on local machine F41 I think it's tensorflow lite I have no idea how it works because I know we haven't successfully compiled tensorflow yet

<@ludiusvox:fedora.im>

18:14:39

Outside of containers I got working

<@tflink:fedora.im>

18:14:41

yeah, tensorflow is a daunting task

<@ludiusvox:fedora.im>

18:15:04

I got a container working g with NVIDIA I haven't tried ROCm

<@ludiusvox:fedora.im>

18:15:57

Media pipe I was using it for biomedical data collection from MP4 files of distinguishing between faces and hands and it works somehow

<@ludiusvox:fedora.im>

18:16:34

Maybe it doesn't need a GPU not sure

<@ludiusvox:fedora.im>

18:17:41

Let me show mediapipe library

<@ludiusvox:fedora.im>

18:19:26

https://github.com/google-ai-edge/mediapipe

<@xanderlent:fedora.im>

18:20:54

A quick status update on my work on the Intel NPU stack:

<@xanderlent:fedora.im>

18:20:54

- I've also been working in parallel on getting more of the high-level stuff packaged; This is things like the Audacity and GIMP AI plugins that use the NPU through OpenVINO. Still very much WIP, though.

<@xanderlent:fedora.im>

18:20:54

- Except for the big part of the driver, the compiler-in-driver; I'm still wrestling that ball of code into an RPM. Luckily it is semi-separable from the rest.

<@xanderlent:fedora.im>

18:20:54

- The core driver/firmware stuff is getting closer to ready for upstreaming. (Even just getting the firmware into Fedora would be useful because then you could run Intel's Ubuntu user-space bits in a container.)

<@xanderlent:fedora.im>

18:20:54

<@tflink:fedora.im>

18:21:49

sounds like progress, though. and that's not a small task

<@zodbot:fedora.im>

18:21:59

tflink gave a cookie to xanderlent. They now have 1 cookie, 1 of which was obtained in the Fedora 41 release cycle

<@ludiusvox:fedora.im>

18:22:56

Okay I look at NPU manufacturers I no longer own an NPU

<@xanderlent:fedora.im>

18:23:15

Thanks. If I had AMD hardware I'd also be looking at the XDNA stack, but I'm currently focused on what I can test. 🙂

<@tflink:fedora.im>

18:23:42

yeah, that's a limitation we all have - hardware is expensive and constantly changing. money and time are finite

<@tflink:fedora.im>

18:23:53

anyhow, we're pretty much out of time for today

<@tflink:fedora.im>

18:24:14

if there are no other topics, I'll close out today's meeting and we can move conversation to #ai-ml:fedoraproject.org

<@tflink:fedora.im>

18:24:49

thanks for coming, everyone

<@tflink:fedora.im>

18:24:58

!endmeeintg

<@tflink:fedora.im>

18:25:04

!endmeeting