#meeting-2:fedoraproject.org log

<@tflink:fedora.im>

16:30:44

!startmeeting fedora-ai-ml-sig

<@meetbot:fedora.im>

16:30:45

Meeting started at 2024-09-12 16:30:44 UTC

<@meetbot:fedora.im>

16:30:45

The Meeting name is 'fedora-ai-ml-sig'

<@tflink:fedora.im>

16:31:00

who all is here for some ai-ml meeting fun times?

<@trix:fedora.im>

16:31:13

not me

<@tflink:fedora.im>

16:31:53

!hi

<@zodbot:fedora.im>

16:31:54

Tim Flink (tflink)

<@tflink:fedora.im>

16:32:30

Tom Rix: that would remove some of the agenda items if you're not here ... :-D

<@trix:fedora.im>

16:33:04

i gots stuff to talk about as aways

<@tflink:fedora.im>

16:33:08

although, if it's just the 2 of us it would probably go more quickly

<@trix:fedora.im>

16:34:37

assign action items to everyone _not_ here, starting with arnold a. aardvark

<@tflink:fedora.im>

16:34:57

heh

<@man2dev:fedora.im>

16:35:17

!hi

<@zodbot:fedora.im>

16:35:18

Mohammadreza Hendiani (man2dev)

<@trix:fedora.im>

16:35:33

let's get started. if that is ok.. my rocm/pt stuff is mostly an update.

<@trix:fedora.im>

16:35:48

i expect chatbot to be interesting and longer.

<@tflink:fedora.im>

16:35:52

yeah, I was waiting until :35

<@tflink:fedora.im>

16:36:03

!topic F41 Update

<@tflink:fedora.im>

16:36:59

llvm18 compat packages are still not in F41, apparently they broke the rawhide compose when they were added while llvm was still at 18. this means that we're going to have a short-turnaround rebuild of the ROCm stack ... again

<@trix:fedora.im>

16:37:27

aaaah how about we pass on 19 in f41

<@tflink:fedora.im>

16:37:59

I think it's too late for that but I'm planning to object more for what I assume will be 20 in F42

<@trix:fedora.im>

16:38:01

since when things break we do what we said if we won't make it..

<@tflink:fedora.im>

16:38:20

maybe we can get the copr stuff to work but that's more the next topic

<@tflink:fedora.im>

16:38:34

any other updates on F41? has the pytorch stuff been sorted?

<@trix:fedora.im>

16:39:15

in rawhide all bit are there for 6.2 and pytorch has been updated to 2.4.1 and all the other torch* things.

<@tflink:fedora.im>

16:39:55

!info llvm18 compat packages are still not in rawhide or F41, rocm rebuilds will be required once they actually land

<@trix:fedora.im>

16:40:05

builds for f41 have been done but seem to be stalled out on the update to the compose ex/ miopen is still on 6.1 yet i built 6.2 last week.

<@tflink:fedora.im>

16:40:12

!info rocm 6.2 and pytorch 2.4.1 are in rawhide now

<@trix:fedora.im>

16:41:02

there were significant fixes made for the compiler usage in miopen.. so not happy to see compiler change.

<@tflink:fedora.im>

16:41:04

we are in beta freeze. is the build just in updates-testing?

<@trix:fedora.im>

16:41:23

not sure.

<@tflink:fedora.im>

16:41:26

in theory, there shouldn't be much of a change when we switch to llvm18

<@tflink:fedora.im>

16:41:53

huh, the fc41 build doesn't show up in bodhi.

<@trix:fedora.im>

16:41:59

i am tired of folks saying that. in practice the stack need a validation cycle.

<@tflink:fedora.im>

16:42:03

we can chase this down after the meeting, though

<@tflink:fedora.im>

16:42:21

I did say "in theory"

<@trix:fedora.im>

16:42:39

validation is build everythign and see if pytorch coughs up a problem.

<@trix:fedora.im>

16:43:21

we can move onto copr..

<@tflink:fedora.im>

16:43:22

!info f41 rocm and pytorch builds are a bit stuck right now, will need some poking beyond the impending llvm19 stuff

<@tflink:fedora.im>

16:43:43

!topic ROCm 6.2 and COPR

<@trix:fedora.im>

16:44:10

thanks for tflink for getting copr started!

<@tflink:fedora.im>

16:44:45

so, we've been doing some experiments to see if we can start building ROCm in COPR. the immediate goal is to test against the llvm team's llvm18 compat packages which are in COPR

<@tflink:fedora.im>

16:44:50

https://copr.fedorainfracloud.org/coprs/g/rocm-packagers-sig/rocm-devel/

<@tflink:fedora.im>

16:45:33

I'm going to get the builds against the llvm18 compat packages done in the next several days. once they're done, testing would be very much appreciated

<@tflink:fedora.im>

16:46:17

!info we've started doing rocm builds in COPR to start testing against the llvm18 compat packages which are yet to land in rawhide or f41. once the rebuild is done, testing would be very helpful

<@trix:fedora.im>

16:46:25

do you want me to add the pytorch dependencies to this ?

<@tflink:fedora.im>

16:47:12

it might make sense to add the things that require rocm to the copr. blender and pytorch are the first that come to mind

<@trix:fedora.im>

16:47:36

do we have someone to test blender ?

<@tflink:fedora.im>

16:48:17

not that I'm aware of. there were some emails from the blender folks recently asking questions about rocm related debugging but I don't think that we responded to them

<@trix:fedora.im>

16:48:41

please forward those to me in a dm.

<@tflink:fedora.im>

16:48:52

will do

<@trix:fedora.im>

16:48:59

since part of 6.2 is adding the --with debug

<@tflink:fedora.im>

16:49:45

which is the second part of this

<@tflink:fedora.im>

16:50:11

!info options are being added to ROCm packages to enable testing and debug options

<@tflink:fedora.im>

16:50:54

for now, the intent is to test those options in COPR. in their current state, most self-test bits for ROCm are not suitable for inclusion in the Fedora repos

<@trix:fedora.im>

16:50:57

yes. found both useful in my miopen bughunt. having a copr of --with debug would be very helpful.

<@trix:fedora.im>

16:51:37

so i am reworking the stack.

<@tflink:fedora.im>

16:51:39

I wonder if we could kill two birds with one stone and have a copr with debug and test enabled unless those options would conflict with eachother

<@trix:fedora.im>

16:53:00

that would be fine to.. i notice there is a handy 'clone' button so in theory we could create a lot of these pretty easily once we have the base.

<@trix:fedora.im>

16:53:22

i am working on the base with

<@trix:fedora.im>

16:53:32

https://copr.fedorainfracloud.org/coprs/g/rocm-packagers-sig/ROCm6.2/

<@tflink:fedora.im>

16:54:18

that's mostly for centos stream, right? with rawhide to make sure stuff doesn't break?

<@trix:fedora.im>

16:54:49

yes. but we could add f41 pretty easy here too.

<@trix:fedora.im>

16:55:07

cs is for rhat folks

<@tflink:fedora.im>

16:55:28

rawhide and f41 should be covered pretty well by the other copr for now but we can continue to figure out this stuff as we go

<@trix:fedora.im>

16:55:36

the limiter is i believe llvm.

<@tflink:fedora.im>

16:56:16

is there anything else that we should cover WRT ROCm or COPR? we're already halfway through the meeting

<@trix:fedora.im>

16:56:34

all done, thanks!

<@tflink:fedora.im>

16:56:39

thanks for the info and the updates

<@tflink:fedora.im>

16:56:48

!topic community chatbot proposals

<@tflink:fedora.im>

16:57:18

davdunc: are you around for updates on this?

<@tflink:fedora.im>

16:57:38

!info this is a continuation of the conversation from the last meeting about some proposals for Fedora-related chatbots

<@tflink:fedora.im>

16:58:02

!info a fedora chatbot has been proposed to the Fedora council

<@tflink:fedora.im>

16:58:13

!link https://discussion.fedoraproject.org/t/fedora-council-tickets-ticket-503-initiative-fedora-ai-chatbot-development/130370

<@tflink:fedora.im>

16:59:02

it seems like there has been discussion there and as I understand it, the two proposals are still looking to combine, adding a self-hosted option to the slightly newer hosted-proposal

<@tflink:fedora.im>

16:59:19

but I don't know much more than that

<@tflink:fedora.im>

16:59:41

and the folks who have been involved don't seem to be present for the meeting today so I think that's about all

<@tflink:fedora.im>

17:00:03

if anyone has thoughts etc. to add to the proposal, please join in on the discourse thread I linked above

<@man2dev:fedora.im>

17:00:07

daMaestro: had a conflict

<@man2dev:fedora.im>

17:01:05

we can push this back and see if he can make it later

<@tflink:fedora.im>

17:01:23

yeah, we'll have to continue this next time or in another venue

<@tflink:fedora.im>

17:01:40

moving on to the last topic

<@tflink:fedora.im>

17:01:57

!topic generative models and packaging in Fedora

<@man2dev:fedora.im>

17:02:08

<@man2dev:fedora.im>

17:02:08

d users, respectively. For example, according to current Fedora packaging guidelines, where should Ollama, which downloads generative models, place the LLMs? While there has been some discussion on this from Ollama's side, I am unsure how the current packaging guidelines apply in this case.

<@man2dev:fedora.im>

17:02:08

<@man2dev:fedora.im>

17:02:08

There are some interesting ideas going around.

<@man2dev:fedora.im>

17:02:08

For example, changing the download location based on environment variables.

<@man2dev:fedora.im>

17:02:08

<@man2dev:fedora.im>

17:02:08

Some relevant discussions about this (haven't gone through all of it):

<@man2dev:fedora.im>

17:02:08

!link: https://github.com/ollama/ollama/pull/897

<@man2dev:fedora.im>

17:02:08

!link: https://github.com/ollama/ollama/issues/228

<@man2dev:fedora.im>

17:02:08

!idea: I would like to discuss where generative models should be placed when packaged or downloaded by developers an

<@tflink:fedora.im>

17:02:36

canonically, it should be /var/lib, no?

<@man2dev:fedora.im>

17:03:04

that my thought on it too

<@tflink:fedora.im>

17:03:04

well, /var/lib or /home/*

<@man2dev:fedora.im>

17:04:11

im very against putting it in home dir because we need to isolate the models

<@tflink:fedora.im>

17:05:01

yeah, I meant more that downloaded/generated data can live in either place in the general case but it does depend on the details

<@man2dev:fedora.im>

17:05:27

my leading idea on isolation that has worked in my tests is just isolating them though methods like unix group and user permission and systemd permistions

<@tflink:fedora.im>

17:06:10

the primary argument for /home is that it works better with the atomic stuff, toolbox etc.

<@tflink:fedora.im>

17:06:47

what's wrong with selinux? is there a reason not to have selinux policy to isolate the weights, assuming that's what you're talking about

<@man2dev:fedora.im>

17:07:59

thats my worry too but i just don't think its good idea so i wanted to get opinion

<@trix:fedora.im>

17:08:09

ahhh selinux.. i just turn that off

<@tflink:fedora.im>

17:08:39

if I had one of those "setenforce on" tshirts, I would wear it :)

<@man2dev:fedora.im>

17:09:52

i have no opinion on the weights and biases i don't even think we can ship any of them since they mostly come with terms and conditions

<@tflink:fedora.im>

17:09:57

getting back to where the data lives, I'm not sure which is the right answer mostly because I'm not terribly familiar with the atomic stuff and how much of a problem data outside /home is

<@tflink:fedora.im>

17:10:37

but it seems like containers are being used heavily for ai related bits - stuff like podman desktop

<@tflink:fedora.im>

17:11:49

my instinct is that if the model files are supposed to be used for multiple users and it's downloaded by a packaged program, that data belongs in /var/lib

<@tflink:fedora.im>

17:12:59

the only time that would affect atomic users is if they installed ollama using rpm-ostree which isn't the use case that they push. the common use case would be to use containers or flatpaks which don't affect the atomic storage

<@man2dev:fedora.im>

17:13:40

its based on use case if for example there was this really heavy img that basically could do every thing because it shipped with everything and could basicly read any file in home and use if as input

<@tflink:fedora.im>

17:13:45

there's a word for the style of setups that include atomic desktops that I just can't seem to remember ATM - apologies for substituting atomic for that word

<@man2dev:fedora.im>

17:14:21

ollama case it can execute bash commands in it as to read files

<@tflink:fedora.im>

17:15:33

this sounds like a job for selinux and maybe some user/group management

<@man2dev:fedora.im>

17:15:36

its immutable

<@tflink:fedora.im>

17:15:42

thank you. that's the word

<@tflink:fedora.im>

17:15:57

man2dev++

<@zodbot:fedora.im>

17:15:58

tflink gave a cookie to man2dev. They now have 1 cookie, 1 of which was obtained in the Fedora 40 release cycle

<@man2dev:fedora.im>

17:16:04

is there a selinux team?

<@tflink:fedora.im>

17:16:45

yeah, there are folks working on selinux policy. I'm not entirely sure what the usual process is to contact them for changes is outside of filing rhbz tickets, though

<@man2dev:fedora.im>

17:17:24

i'll ask around i guess

<@man2dev:fedora.im>

17:18:15

is there another topic

<@tflink:fedora.im>

17:18:20

sounds like a plan. let us know if you run into problems

<@tflink:fedora.im>

17:18:31

that's all the topics I have

<@tflink:fedora.im>

17:19:05

Mohammadreza Hendiani: is there anything else you wanted to cover with this topic?

<@man2dev:fedora.im>

17:20:15

no i think /var/lib makes sense but isolation stuff is out of scope of here

<@man2dev:fedora.im>

17:20:24

no i think /var/lib makes sense and isolation stuff is out of scope of here

<@tflink:fedora.im>

17:20:34

ok, sounds good.

<@tflink:fedora.im>

17:20:37

we can move on to

<@tflink:fedora.im>

17:20:41

!topic open floor

<@tflink:fedora.im>

17:20:49

any other things that folks want to bring up for the meeting today?

<@tflink:fedora.im>

17:26:12

thanks for coming, everyone

<@tflink:fedora.im>

17:26:21

!endmeeting