2024-09-12 16:30:44 <@tflink:fedora.im> !startmeeting fedora-ai-ml-sig
2024-09-12 16:30:45 <@meetbot:fedora.im> Meeting started at 2024-09-12 16:30:44 UTC
2024-09-12 16:30:45 <@meetbot:fedora.im> The Meeting name is 'fedora-ai-ml-sig'
2024-09-12 16:31:00 <@tflink:fedora.im> who all is here for some ai-ml meeting fun times?
2024-09-12 16:31:13 <@trix:fedora.im> not me
2024-09-12 16:31:53 <@tflink:fedora.im> !hi
2024-09-12 16:31:54 <@zodbot:fedora.im> Tim Flink (tflink)
2024-09-12 16:32:30 <@tflink:fedora.im> Tom Rix: that would remove some of the agenda items if you're not here ... :-D
2024-09-12 16:33:04 <@trix:fedora.im> i gots stuff to talk about as aways
2024-09-12 16:33:08 <@tflink:fedora.im> although, if it's just the 2 of us it would probably go more quickly
2024-09-12 16:34:37 <@trix:fedora.im> assign action items to everyone _not_ here, starting with arnold a. aardvark
2024-09-12 16:34:57 <@tflink:fedora.im> heh
2024-09-12 16:35:17 <@man2dev:fedora.im> !hi
2024-09-12 16:35:18 <@zodbot:fedora.im> Mohammadreza Hendiani (man2dev)
2024-09-12 16:35:33 <@trix:fedora.im> let's get started. if that is ok.. my rocm/pt stuff is mostly an update.
2024-09-12 16:35:48 <@trix:fedora.im> i expect chatbot to be interesting and longer.
2024-09-12 16:35:52 <@tflink:fedora.im> yeah, I was waiting until :35
2024-09-12 16:36:03 <@tflink:fedora.im> !topic F41 Update
2024-09-12 16:36:59 <@tflink:fedora.im> llvm18 compat packages are still not in F41, apparently they broke the rawhide compose when they were added while llvm was still at 18. this means that we're going to have a short-turnaround rebuild of the ROCm stack ... again
2024-09-12 16:37:27 <@trix:fedora.im> aaaah how about we pass on 19 in f41
2024-09-12 16:37:59 <@tflink:fedora.im> I think it's too late for that but I'm planning to object more for what I assume will be 20 in F42
2024-09-12 16:38:01 <@trix:fedora.im> since when things break we do what we said if we won't make it..
2024-09-12 16:38:20 <@tflink:fedora.im> maybe we can get the copr stuff to work but that's more the next topic
2024-09-12 16:38:34 <@tflink:fedora.im> any other updates on F41? has the pytorch stuff been sorted?
2024-09-12 16:39:15 <@trix:fedora.im> in rawhide all bit are there for 6.2 and pytorch has been updated to 2.4.1 and all the other torch* things.
2024-09-12 16:39:55 <@tflink:fedora.im> !info llvm18 compat packages are still not in rawhide or F41, rocm rebuilds will be required once they actually land
2024-09-12 16:40:05 <@trix:fedora.im> builds for f41 have been done but seem to be stalled out on the update to the compose ex/ miopen is still on 6.1 yet i built 6.2 last week.
2024-09-12 16:40:12 <@tflink:fedora.im> !info rocm 6.2 and pytorch 2.4.1 are in rawhide now
2024-09-12 16:41:02 <@trix:fedora.im> there were significant fixes made for the compiler usage in miopen.. so not happy to see compiler change.
2024-09-12 16:41:04 <@tflink:fedora.im> we are in beta freeze. is the build just in updates-testing?
2024-09-12 16:41:23 <@trix:fedora.im> not sure.
2024-09-12 16:41:26 <@tflink:fedora.im> in theory, there shouldn't be much of a change when we switch to llvm18
2024-09-12 16:41:53 <@tflink:fedora.im> huh, the fc41 build doesn't show up in bodhi.
2024-09-12 16:41:59 <@trix:fedora.im> i am tired of folks saying that. in practice the stack need a validation cycle. 
2024-09-12 16:42:03 <@tflink:fedora.im> we can chase this down after the meeting, though
2024-09-12 16:42:21 <@tflink:fedora.im> I did say "in theory"
2024-09-12 16:42:39 <@trix:fedora.im> validation is build everythign and see if pytorch coughs up a problem.
2024-09-12 16:43:21 <@trix:fedora.im> we can move onto copr..
2024-09-12 16:43:22 <@tflink:fedora.im> !info f41 rocm and pytorch builds are a bit stuck right now, will need some poking beyond the impending llvm19 stuff
2024-09-12 16:43:43 <@tflink:fedora.im> !topic ROCm 6.2 and COPR
2024-09-12 16:44:10 <@trix:fedora.im> thanks for tflink for getting copr started!
2024-09-12 16:44:45 <@tflink:fedora.im> so, we've been doing some experiments to see if we can start building ROCm in COPR. the immediate goal is to test against the llvm team's llvm18 compat packages which are in COPR
2024-09-12 16:44:50 <@tflink:fedora.im> https://copr.fedorainfracloud.org/coprs/g/rocm-packagers-sig/rocm-devel/
2024-09-12 16:45:33 <@tflink:fedora.im> I'm going to get the builds against the llvm18 compat packages done in the next several days. once they're done, testing would be very much appreciated
2024-09-12 16:46:17 <@tflink:fedora.im> !info we've started doing rocm builds in COPR to start testing against the llvm18 compat packages which are yet to land in rawhide or f41. once the rebuild is done, testing would be very helpful
2024-09-12 16:46:25 <@trix:fedora.im> do you want me to add the pytorch dependencies to this ?
2024-09-12 16:47:12 <@tflink:fedora.im> it might make sense to add the things that require rocm to the copr. blender and pytorch are the first that come to mind
2024-09-12 16:47:36 <@trix:fedora.im> do we have someone to test blender ?
2024-09-12 16:48:17 <@tflink:fedora.im> not that I'm aware of. there were some emails from the blender folks recently asking questions about rocm related debugging but I don't think that we responded to them
2024-09-12 16:48:41 <@trix:fedora.im> please forward those to me in a dm.
2024-09-12 16:48:52 <@tflink:fedora.im> will do
2024-09-12 16:48:59 <@trix:fedora.im> since part of 6.2 is adding the --with debug
2024-09-12 16:49:45 <@tflink:fedora.im> which is the second part of this
2024-09-12 16:50:11 <@tflink:fedora.im> !info options are being added to ROCm packages to enable testing and debug options
2024-09-12 16:50:54 <@tflink:fedora.im> for now, the intent is to test those options in COPR. in their current state, most self-test bits for ROCm are not suitable for inclusion in the Fedora repos
2024-09-12 16:50:57 <@trix:fedora.im> yes. found both useful in my miopen bughunt.  having a copr of --with debug would be very helpful.
2024-09-12 16:51:37 <@trix:fedora.im> so i am reworking the stack.
2024-09-12 16:51:39 <@tflink:fedora.im> I wonder if we could kill two birds with one stone and have a copr with debug and test enabled unless those options would conflict with eachother
2024-09-12 16:53:00 <@trix:fedora.im> that would be fine to.. i notice there is a handy 'clone' button so in theory we could create a lot of these pretty easily once we have the base.
2024-09-12 16:53:22 <@trix:fedora.im> i am working on the base with 
2024-09-12 16:53:32 <@trix:fedora.im> https://copr.fedorainfracloud.org/coprs/g/rocm-packagers-sig/ROCm6.2/
2024-09-12 16:54:18 <@tflink:fedora.im> that's mostly for centos stream, right? with rawhide to make sure stuff doesn't break?
2024-09-12 16:54:49 <@trix:fedora.im> yes. but we could add f41 pretty easy here too.
2024-09-12 16:55:07 <@trix:fedora.im> cs is for rhat folks
2024-09-12 16:55:28 <@tflink:fedora.im> rawhide and f41 should be covered pretty well by the other copr for now but we can continue to figure out this stuff as we go
2024-09-12 16:55:36 <@trix:fedora.im> the limiter is i believe llvm.
2024-09-12 16:56:16 <@tflink:fedora.im> is there anything else that we should cover WRT ROCm or COPR? we're already halfway through the meeting
2024-09-12 16:56:34 <@trix:fedora.im> all done, thanks!
2024-09-12 16:56:39 <@tflink:fedora.im> thanks for the info and the updates
2024-09-12 16:56:48 <@tflink:fedora.im> !topic community chatbot proposals
2024-09-12 16:57:18 <@tflink:fedora.im> davdunc: are you around for updates on this?
2024-09-12 16:57:38 <@tflink:fedora.im> !info this is a continuation of the conversation from the last meeting about some proposals for Fedora-related chatbots
2024-09-12 16:58:02 <@tflink:fedora.im> !info a fedora chatbot has been proposed to the Fedora council
2024-09-12 16:58:13 <@tflink:fedora.im> !link https://discussion.fedoraproject.org/t/fedora-council-tickets-ticket-503-initiative-fedora-ai-chatbot-development/130370
2024-09-12 16:59:02 <@tflink:fedora.im> it seems like there has been discussion there and as I understand it, the two proposals are still looking to combine, adding a self-hosted option to the slightly newer hosted-proposal
2024-09-12 16:59:19 <@tflink:fedora.im> but I don't know much more than that
2024-09-12 16:59:41 <@tflink:fedora.im> and the folks who have been involved don't seem to be present for the meeting today so I think that's about all
2024-09-12 17:00:03 <@tflink:fedora.im> if anyone has thoughts etc. to add to the proposal, please join in on the discourse thread I linked above
2024-09-12 17:00:07 <@man2dev:fedora.im> daMaestro: had a conflict
2024-09-12 17:01:05 <@man2dev:fedora.im> we can push this back and see if he can make it later
2024-09-12 17:01:23 <@tflink:fedora.im> yeah, we'll have to continue this next time or in another venue
2024-09-12 17:01:40 <@tflink:fedora.im> moving on to the last topic
2024-09-12 17:01:57 <@tflink:fedora.im> !topic generative models and packaging in Fedora
2024-09-12 17:02:08 <@man2dev:fedora.im> ~
2024-09-12 17:02:08 <@man2dev:fedora.im> d users, respectively. For example, according to current Fedora packaging guidelines, where should Ollama, which downloads generative models, place the LLMs? While there has been some discussion on this from Ollama's side, I am unsure how the current packaging guidelines apply in this case.
2024-09-12 17:02:08 <@man2dev:fedora.im> 
2024-09-12 17:02:08 <@man2dev:fedora.im> There are some interesting ideas going around.
2024-09-12 17:02:08 <@man2dev:fedora.im> For example, changing the download location based on environment variables.
2024-09-12 17:02:08 <@man2dev:fedora.im> 
2024-09-12 17:02:08 <@man2dev:fedora.im> Some relevant discussions about this (haven't gone through all of it):
2024-09-12 17:02:08 <@man2dev:fedora.im> !link: https://github.com/ollama/ollama/pull/897
2024-09-12 17:02:08 <@man2dev:fedora.im> !link: https://github.com/ollama/ollama/issues/228
2024-09-12 17:02:08 <@man2dev:fedora.im> !idea: I would like to discuss where generative models should be placed when packaged or downloaded by developers an
2024-09-12 17:02:36 <@tflink:fedora.im> canonically, it should be /var/lib, no?
2024-09-12 17:03:04 <@man2dev:fedora.im> that my thought on it too
2024-09-12 17:03:04 <@tflink:fedora.im> well, /var/lib or /home/*
2024-09-12 17:04:11 <@man2dev:fedora.im> im very against putting it in home dir because we need to isolate the models 
2024-09-12 17:05:01 <@tflink:fedora.im> yeah, I meant more that downloaded/generated data can live in either place in the general case but it does depend on the details
2024-09-12 17:05:27 <@man2dev:fedora.im> my leading idea on isolation that has worked in my tests is just isolating them though methods like unix group and user permission and systemd permistions
2024-09-12 17:06:10 <@tflink:fedora.im> the primary argument for /home is that it works better with the atomic stuff, toolbox etc.
2024-09-12 17:06:47 <@tflink:fedora.im> what's wrong with selinux? is there a reason not to have selinux policy to isolate the weights, assuming that's what you're talking about
2024-09-12 17:07:59 <@man2dev:fedora.im> thats my worry too but i just don't think its good idea so i wanted to get opinion
2024-09-12 17:08:09 <@trix:fedora.im> ahhh selinux.. i just turn that off
2024-09-12 17:08:39 <@tflink:fedora.im> if I had one of those "setenforce on" tshirts, I would wear it :)
2024-09-12 17:09:52 <@man2dev:fedora.im> i have no opinion on the weights and biases i don't even think we can ship any of them since they mostly come with terms and conditions
2024-09-12 17:09:57 <@tflink:fedora.im> getting back to where the data lives, I'm not sure which is the right answer mostly because I'm not terribly familiar with the atomic stuff and how much of a problem data outside /home is
2024-09-12 17:10:37 <@tflink:fedora.im> but it seems like containers are being used heavily for ai related bits - stuff like podman desktop
2024-09-12 17:11:49 <@tflink:fedora.im> my instinct is that if the model files are supposed to be used for multiple users and it's downloaded by a packaged program, that data belongs in /var/lib
2024-09-12 17:12:59 <@tflink:fedora.im> the only time that would affect atomic users is if they installed ollama using rpm-ostree which isn't the use case that they push. the common use case would be to use containers or flatpaks which don't affect the atomic storage
2024-09-12 17:13:40 <@man2dev:fedora.im> its based on use case if for example there was this really heavy img that basically could do every thing because it shipped with everything and could basicly read any file in home and use if as input
2024-09-12 17:13:45 <@tflink:fedora.im> there's a word for the style of setups that include atomic desktops that I just can't seem to remember ATM - apologies for substituting atomic for that word
2024-09-12 17:14:21 <@man2dev:fedora.im> ollama case it can execute bash commands in it as to read files
2024-09-12 17:15:33 <@tflink:fedora.im> this sounds like a job for selinux and maybe some user/group management
2024-09-12 17:15:36 <@man2dev:fedora.im> its immutable
2024-09-12 17:15:42 <@tflink:fedora.im> thank you. that's the word
2024-09-12 17:15:57 <@tflink:fedora.im> man2dev++
2024-09-12 17:15:58 <@zodbot:fedora.im> tflink gave a cookie to man2dev. They now have 1 cookie, 1 of which was obtained in the Fedora 40 release cycle
2024-09-12 17:16:04 <@man2dev:fedora.im> is there a selinux team?
2024-09-12 17:16:45 <@tflink:fedora.im> yeah, there are folks working on selinux policy. I'm not entirely sure what the usual process is to contact them for changes is outside of filing rhbz tickets, though
2024-09-12 17:17:24 <@man2dev:fedora.im> i'll ask around i guess
2024-09-12 17:18:15 <@man2dev:fedora.im> is there another topic
2024-09-12 17:18:20 <@tflink:fedora.im> sounds like a plan. let us know if you run into problems
2024-09-12 17:18:31 <@tflink:fedora.im> that's all the topics I have
2024-09-12 17:19:05 <@tflink:fedora.im> Mohammadreza Hendiani: is there anything else you wanted to cover with this topic?
2024-09-12 17:20:15 <@man2dev:fedora.im> no i think /var/lib makes sense but isolation stuff is out of scope of here 
2024-09-12 17:20:24 <@man2dev:fedora.im> no i think /var/lib makes sense and isolation stuff is out of scope of here 
2024-09-12 17:20:34 <@tflink:fedora.im> ok, sounds good.
2024-09-12 17:20:37 <@tflink:fedora.im> we can move on to
2024-09-12 17:20:41 <@tflink:fedora.im> !topic open floor
2024-09-12 17:20:49 <@tflink:fedora.im> any other things that folks want to bring up for the meeting today?
2024-09-12 17:26:12 <@tflink:fedora.im> thanks for coming, everyone
2024-09-12 17:26:21 <@tflink:fedora.im> !endmeeting