16:31:10 <dustymabe> #startmeeting fedora_coreos_meeting 16:31:10 <zodbot> Meeting started Wed Aug 16 16:31:10 2023 UTC. 16:31:10 <zodbot> This meeting is logged and archived in a public location. 16:31:10 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 16:31:10 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:31:10 <zodbot> The meeting name has been set to 'fedora_coreos_meeting' 16:31:15 <dustymabe> #topic roll call 16:31:27 <travier> .hello siosm 16:31:33 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com> 16:31:37 <gursewak> .hi 16:31:41 <zodbot> gursewak: gursewak 'Gursewak Singh' <gurssing@redhat.com> 16:31:48 <ravanelli> .hi 16:31:52 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com> 16:32:23 <dustymabe> #chair travier gursewak ravanelli 16:32:23 <zodbot> Current chairs: dustymabe gursewak ravanelli travier 16:33:38 <spresti> .hi 16:33:40 <zodbot> spresti: spresti 'Steven Presti' <spresti@redhat.com> 16:33:48 <dustymabe> #chair spresti 16:33:48 <zodbot> Current chairs: dustymabe gursewak ravanelli spresti travier 16:35:04 <jmarrero> .hi 16:35:05 <zodbot> jmarrero: jmarrero 'Joseph Marrero' <jmarrero@redhat.com> 16:35:29 <marmijo> .hello marmijo 16:35:30 <zodbot> marmijo: marmijo 'Michael Armijo' <marmijo@redhat.com> 16:36:21 <dustymabe> #chair jmarrero marmijo 16:36:21 <zodbot> Current chairs: dustymabe gursewak jmarrero marmijo ravanelli spresti travier 16:36:32 <dustymabe> #topic Action items from last meeting 16:36:46 <dustymabe> IIUC the last meeting was canceled 16:37:23 <dustymabe> well - it was started but then not enough people were here 16:37:36 <dustymabe> so we'll move on 16:37:57 <dustymabe> #topic New Package Request: AWS amazon-ssm-agent for AWS EC2 AMI 16:38:03 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1537 16:38:44 <dustymabe> travier: you already had a reply in the ticket 16:39:20 <dustymabe> I also linked to https://github.com/coreos/fedora-coreos-tracker/issues/95 where we had discussions about not including cloud agents in the base images 16:39:39 <travier> dustymabe: there was one action item from last meeting 16:39:45 <travier> I re-actionned it 16:39:47 <dustymabe> travier: oh? 16:39:59 <dustymabe> ahh I see it now 16:40:03 <aaradhak85> .hello aaradhak85 16:40:04 <zodbot> aaradhak85: Sorry, but user 'aaradhak85' does not exist 16:40:06 <dustymabe> yes that action was completed 16:40:11 <travier> ok good 16:40:12 <dustymabe> #chair aaradhak85 16:40:12 <zodbot> Current chairs: aaradhak85 dustymabe gursewak jmarrero marmijo ravanelli spresti travier 16:42:01 <travier> Overall I don't like adding agents that do more than a simple health check. 16:42:16 <spresti> So from my reading of the issue, we really should not include the package. It feels like its opening a box of worms 16:42:17 <travier> like a "ping: I'm up" check 16:42:38 <dustymabe> travier: agreed 16:43:05 <dustymabe> I do wish we had an answer for people like this. even if it was in the FAQ 16:43:27 <dustymabe> i.e. I think I'd be happy linking from the FAQ to a community supported method for getting the agent installed 16:43:35 <travier> As it's an external package, layering is a fine option if they want to take the risk 16:43:45 <dustymabe> travier: agree 16:43:51 <travier> we don't have to make all our users take the risk 16:44:41 <spresti> +1 16:45:16 <dustymabe> #proposed We will not include the amazon-ssm-agent in Fedora CoreOS. We have generally not shipped cloud agents in the past and the package is only available from third party repos. If users want to layer this package they have that option. 16:45:24 <travier> +1 16:45:32 <spresti> +1 16:46:27 <jmarrero> +1 16:46:41 <dustymabe> #agreed We will not include the amazon-ssm-agent in Fedora CoreOS. We have generally not shipped cloud agents in the past and the package is only available from third party repos. If users want to layer this package they have that option. 16:46:51 <dustymabe> (I'll count myself as +1 there) 16:47:07 <gursewak> +1 to that as well. 16:47:20 <dustymabe> gursewak: 👍 16:47:29 <dustymabe> #topic low memory systems with large disks fail to provision 16:47:32 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1535 16:47:43 * dustymabe gives a moment to read the ticket 16:49:47 <dustymabe> TL;DR people can't really boot small memory systems any longer with a large disk 16:49:58 <dustymabe> unless they set up a separate mount for /var 16:50:02 <spresti> So why do we reprovision root at 200gb? i was under the impression it was a larger amount where the issue was more prominant? 16:50:12 <dustymabe> spresti: we do it at 100g+ 16:51:08 <dustymabe> yeah the performance regressions get worse the larger it gets, but we were operating off of the recommendations from the XFS team and they basically recommended anything above a certain agcount should get repaved (which turned out to be right around 100G) 16:53:11 <spresti> Ah ok, that makes sense. I like the option 1 you recommended seems like an all rounder 16:53:38 <travier> I'm OK with disableing the reprovision if memory is low, so option 1 16:53:41 <spresti> I wonder though should we add a part 2 into that fix, where we also bail if ram is < 3gb and the hdd is 12432143? 16:54:12 <spresti> ignore my numbers on hdd size, just trying to indicate too large 16:54:44 <ravanelli> if we go for 1, should we add some message to let the users know about the issues with large disks? 16:55:34 <travier> Usually the use case for low memory systems is Raspberry boards or small cloud instance. Those don't usually have large disks 16:55:36 <dustymabe> spresti: that's the behavior today IIUC. if HDD size is large then autosave-xfs/reprovisioning will be requested. when the reprovisioning code starts up it checks memory and if it's too low it bails out. 16:56:07 <dustymabe> travier: think backup server 16:56:34 <dustymabe> memory requirements: low 16:56:38 <dustymabe> disk requirements: high 16:56:39 <travier> would it be the root partition that is large and not a set of RAID disks? 16:56:49 <spresti> I think there is a diffrence. Today we would bail at 100g but instead we bail when it becomes too long I thought we encountered timeouts in the past 16:56:59 <dustymabe> travier: good point. it depends on how you architected it 16:57:14 <travier> but yes, there are always cases 16:57:46 <dustymabe> spresti: so in that case we'd have to find another threshold in which we'd decide to hard fail 16:57:55 <spresti> ie if we have 3gb of ram we repro at 100g but if we dont we try and ride with the old implemnetation and warn about slowness, and then finaly fail at some threshold. 16:58:36 <travier> If you have more than 100GB, I'd say that you can afford the /var split 16:58:46 <dustymabe> travier: indeed 16:58:56 <dustymabe> I think the hard part is getting the user to understand the problem 16:59:06 <spresti> Ah thats fair travier 16:59:11 <dustymabe> all they know is that their instance no longer gets provisioned 16:59:23 <dustymabe> i.e. we implemented autosave-xfs without them ever requesting it 16:59:41 <dustymabe> so their system that provisioned fine 6 months ago, now fails 16:59:51 <travier> agree it's not create that it's failing the boot where it was working before 17:00:42 <spresti> Sorry for waffling so much I think I am all in on option 1. (ignore my idea about another threshold) 17:00:49 <dustymabe> spresti: :) 17:01:03 <travier> which is why I like the "don't do anything if we can't, but warn using CLHM" 17:01:15 <dustymabe> so in option 1. we decide that on low memory systems we won't request autosave-xfs 17:01:21 <travier> We can warn about memory and /var split 17:01:33 <ravanelli> +1 17:01:37 <dustymabe> and then later (an independent process) drop down a CLHM helper 17:02:15 <dustymabe> travier: i'm thinking the CLHM helper can be implemented completely independent of the "memory checking" - WDYT? 17:02:25 <travier> agree 17:02:36 <dustymabe> i.e. it's useful in all cases 17:02:41 <travier> it was like that before 17:02:49 <travier> so the CLHM addition is a "feature" 17:03:05 <dustymabe> right.. now what it will cause (probably) is existing systems to start "warning" to their owners 17:03:14 <dustymabe> but raising awareness of this problem is probably a good thing 17:03:24 <travier> how long have we shipped the transpose xfs workaround in releases so far? 17:03:33 <travier> since how long* 17:03:43 <dustymabe> I think it landed in like june(ish) 17:04:20 <travier> not that long ago so that should be fine 17:04:38 <dustymabe> yeah 17:05:26 <dustymabe> #proposed We will attempt to make the autosave-xfs code not request a reprovision if the memory in the system is too low. We will also develop a CLHM helper script that checks agcount in the root filesystem (if it is XFS) and give the user a proper message if it's too low. 17:05:43 <ravanelli> +1 17:06:06 <spresti> +1 17:06:08 <aaradhak85> +1 17:06:10 <marmijo> +1 17:06:18 <mnguyen> +1 17:06:20 <jmarrero> +1 17:06:32 <travier> +1 17:06:37 <gursewak> +1 17:06:48 <dustymabe> #agreed We will attempt to make the autosave-xfs code not request a reprovision if the memory in the system is too low. We will also develop a CLHM helper script that checks agcount in the root filesystem (if it is XFS) and give the user a proper message if it's too low. 17:07:16 <dustymabe> #topic Low memory machines fails to intialize/boot fcos 17:07:20 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1540 17:07:32 <dustymabe> If this issue seems like the last one, I assure you they are different :) 17:07:48 <travier> :) 17:07:57 <dustymabe> in this case someone is trying to boot a 512M AWS instance and it can't even load the initramfs 17:08:23 <dustymabe> I guess the open question I have is.. do we care? 17:08:37 <dustymabe> also do we have any docs anywhere (I couldn't find any) that specifies minimum requirements 17:09:06 <travier> https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/#hardware_overview-specs 17:09:36 <travier> I don't think we have FCOS specific requirements 17:10:01 <spresti> wow thats like a pi zero 17:10:26 <dustymabe> travier: nice - I didn't find that when I searched before 17:10:49 <dustymabe> honestly I think that bit of documentation is enough 17:10:54 <travier> We could add a small note in https://docs.fedoraproject.org/en-US/fedora-coreos/platforms/ maybe? 17:11:15 <dustymabe> TBH I'm completely surprised 512M was working at all before 17:11:44 <dustymabe> thought it's nice to know we were fitting in that footprint OK and still able to update (though I guarantee no packages were layered) 17:11:52 <dustymabe> up until recently 17:12:20 <dustymabe> #info we have documentation for minimum system requirements at https://docs.fedoraproject.org/en-US/fedora/latest/release-notes/welcome/Hardware_Overview/#hardware_overview-specs 17:12:36 <dustymabe> I don't think we need a #agreed for this one. Do you? 17:13:42 <jmarrero> I think that makes sense. 17:13:56 <travier> I think we could go as low as 1CPU 1GB RAM 15GB disk but that's so small you would not run that many containers on it 17:14:23 <dustymabe> yeah, in this case the user was using the host as a bastion (so just ssh running) 17:14:50 <travier> or we default to the Fedora ones and users are free to try below, but we're not obligated to make sure it works 17:15:30 <dustymabe> #topic tracker: Fedora 39 changes considerations 17:15:35 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1491 17:15:51 <dustymabe> ok there are a few we haven't covered yet 17:16:05 <dustymabe> subtopic 126. Deprecating libuser and removing passwd package from Fedora 17:16:12 <dustymabe> #link https://fedoraproject.org/wiki/Changes/LibuserDeprecation 17:16:28 <dustymabe> travier: jmarrero: I was going to lean on you here to tell me if rpm-ostree was OK with this change 17:16:48 <travier> I don't think that should impact us 17:17:11 <dustymabe> travier: any other supported context I should add to the NOTE about it? 17:17:17 <dustymabe> supporting* 17:18:33 <travier> rpm-ostree and ostree do not link to it 17:18:51 <travier> as far as I can see 17:18:52 <dustymabe> #info This should not impact us. rpm-ostree and ostree do not link to libuser. 17:19:00 <dustymabe> travier: that's good enough for me 17:19:13 <dustymabe> subtopic 128. LLVM 17 17:19:20 <dustymabe> #link https://fedoraproject.org/wiki/Changes/LLVM-17 17:19:54 <dustymabe> #info This should not affect us or should be transparent. 17:20:16 <dustymabe> subtopic 129. Migrate NetworkManager ifcfg profiles to keyfile 17:20:25 <dustymabe> #link https://fedoraproject.org/wiki/Changes/MigrateIfcfgToKeyfile 17:20:42 <dustymabe> #info This should not affect us. Fedora CoreOS never supported ifcfg style network configuration files. 17:20:43 <travier> we've already done that haven't we? 17:20:50 <travier> +1 17:20:59 <dustymabe> travier: yep. We never supported the old style from the beginning (we actively removed the plugin) 17:21:11 <dustymabe> subtopic 130. IBus 1.5.29 17:21:18 <dustymabe> #link https://fedoraproject.org/wiki/Changes/IBus_1.5.29 17:21:30 <travier> does not impact us 17:21:56 <dustymabe> #info Nothing to do. We don't ship IBUS. 17:22:11 <dustymabe> subtopic 131. Use Noto fonts for Indic (Indian language) scripts 17:22:17 <dustymabe> #link https://fedoraproject.org/wiki/Changes/Indic_Noto_fonts 17:22:19 <travier> nothing to do 17:22:27 <dustymabe> #info We don't ship fonts. 17:22:40 <dustymabe> subtopic 132. Anaconda WebUI for Fedora Workstation by default 17:22:49 <travier> we don't ship anaconda 17:22:50 <dustymabe> #link https://fedoraproject.org/wiki/Changes/AnacondaWebUIforFedoraWorkstation 17:22:58 <dustymabe> it's interesting to me that this is a system wide change 17:23:22 <dustymabe> #info Nothing for us to do. We don't ship Anaconda and this change is targetted at Workstation. 17:23:35 <dustymabe> subtopic 133. Build JDKs once, repack everywhere 17:23:41 <dustymabe> #link https://fedoraproject.org/wiki/Changes/BuildJdkOncePackEverywhere 17:23:54 <dustymabe> #info Nothing to do. We don't ship Java. 17:24:06 <dustymabe> subtopic 134. GNU Toolchain Update (gcc 13.2, binutils 2.40, glibc 2.38, gdb 13.2) 17:24:12 <dustymabe> #link https://fedoraproject.org/wiki/Changes/GNUToolchainF39 17:24:39 <dustymabe> #info This would should be transparent to us. We'll pick up packages as they get rebuilt and actively test them in our `rawhide` stream. 17:25:01 <dustymabe> ok now moving on to the self contained changes 17:25:12 <dustymabe> someone please stop me if there is anything to bring up! 17:25:23 <dustymabe> subtopic 220. Clean Systemd-boot installs 17:25:30 <dustymabe> #link https://fedoraproject.org/wiki/Changes/cleanup_systemd_install 17:26:17 <travier> does not impact us 17:26:30 <dustymabe> This one is interesting though 17:26:51 * dustymabe often wonders if eventually GRUB should be replaced 17:27:07 <dustymabe> #info Nothing for us to do. We will continue to use GRUB for now. 17:27:22 <dustymabe> subtopic 221. No custom Qt theming for Fedora Workstation 17:27:29 <dustymabe> #link https://fedoraproject.org/wiki/Changes/NoCustomQtThemingForWorkstation 17:27:40 <dustymabe> #info Nothing for us to do. This one is Workstation specific. 17:27:50 <dustymabe> subtopic 222. Retire Modularity 17:27:56 <dustymabe> #link https://fedoraproject.org/wiki/Changes/RetireModularity 17:28:20 <dustymabe> Well I can't say I didn't see this one coming ever since modularity was introduced :) 17:28:55 <dustymabe> We've already done some work to remove the modular repos from F39+ so that should be all. 17:29:12 <dustymabe> though I think maybe a little more work to make sure updating systems don't have issues 17:29:37 <dustymabe> this was/is the issue for that https://github.com/coreos/fedora-coreos-tracker/issues/1513 17:30:03 <dustymabe> #info We've already done some work in https://github.com/coreos/fedora-coreos-tracker/issues/1513 to remove modular repos from F39+. 17:30:24 <dustymabe> subtopic 223. Sericea and Sway Spin Xorg-less 17:30:32 <dustymabe> #link https://fedoraproject.org/wiki/Changes/sericea-xorgless 17:30:45 <dustymabe> #info Nothing for us to do. This is Sericea and Sway spin specific. 17:30:58 <dustymabe> subtopic 224. Further reduce Fedora-specific build flags in non-RPM Python extensions 17:31:04 <dustymabe> #link https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction 17:31:24 <dustymabe> #info Nothing for us to do. We don't ship python. 17:31:35 <dustymabe> subtopic 225. Passkey authentication for centrally managed users 17:31:45 <dustymabe> #link https://fedoraproject.org/wiki/Changes/Passkey_authentication_centrally_managed_users 17:32:12 <dustymabe> hmm. I don't know if anyone has tried to register an FCOS system with AD, but I guess it would work 17:32:36 <dustymabe> #info No work for us to do here. If there are AD managed FCOS systems I imagine the change could be picked up by them. 17:32:44 <dustymabe> Not sure if that ^^ is completely accurate, but let me know 17:32:55 <dustymabe> subtopic 226. ibus-anthy 1.5.15 17:33:03 <dustymabe> #link https://fedoraproject.org/wiki/Changes/ibus-anthy_1.5.15 17:33:12 <dustymabe> #info Nothing for us to do. We don't ship ibus. 17:33:22 <dustymabe> subtopic 227. Enable auto-updates by default in Fedora Kinoite 17:33:29 <dustymabe> #link https://fedoraproject.org/wiki/Changes/KDEKinoiteAutoUpdateByDefault 17:33:31 <travier> nothing to do :) 17:33:42 <dustymabe> #info Nothing for us to do. This is kinoite specific. 17:33:49 <dustymabe> #topic open floor 17:34:04 <travier> thanks for going through this 17:34:06 <dustymabe> travier: though I do wonder - you are enabling auto updates on kinoite but not silverblue? 17:34:20 <travier> it's "already" enabled on Silverblue :) 17:34:25 <dustymabe> oh, nice 17:34:30 <dustymabe> didn't know that 17:34:42 <travier> This is GUI specific. Silverblue does it in GNOME Software, Kinoite in Discover 17:34:45 <dustymabe> but I do run SB - so it's weird that I didn't know that 17:34:50 <dustymabe> ahh ok that's why 17:34:56 <dustymabe> I run i3wm and not Gnome 17:35:04 <dustymabe> anyone with any topics for open floor? 17:35:24 <dustymabe> we do need someone to start working on https://github.com/coreos/fedora-coreos-tracker/issues/1502 soon - beta is right around the corner 17:35:43 <travier> indeed 17:35:51 <dustymabe> oh and I guess we could use a volunteer for the work we discussed earlier about https://github.com/coreos/fedora-coreos-tracker/issues/1535 17:36:21 <dustymabe> (sorry for letting the meeting run long) 17:36:53 <travier> well, we skipped a week 17:37:03 <dustymabe> i'll end the meeting soon unless new discussion starts 17:37:36 <dustymabe> #endmeeting