16:30:07 #startmeeting fedora_coreos_meeting 16:30:07 Meeting started Wed Jul 31 16:30:07 2019 UTC. 16:30:07 This meeting is logged and archived in a public location. 16:30:07 The chair is ajeddeloh. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:30:07 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:30:07 The meeting name has been set to 'fedora_coreos_meeting' 16:30:18 #topic roll call 16:30:21 .hello2 16:30:23 ajeddeloh: ajeddeloh 'Andrew Jeddeloh' 16:30:33 .hello lucab 16:30:34 kaeso[m]: lucab 'Luca Bruno' 16:30:40 .hello2 16:30:41 slowrie: slowrie 'Stephen Lowrie' 16:30:51 .hello2 16:30:52 jlebon: jlebon 'None' 16:31:14 #chair ajeddeloh kaeso[m] slowrie jlebon 16:31:14 Current chairs: ajeddeloh jlebon kaeso[m] slowrie 16:31:18 .hello sinnykumari 16:31:19 ksinny: sinnykumari 'Sinny Kumari' 16:32:01 #chair ksinny 16:32:01 Current chairs: ajeddeloh jlebon kaeso[m] ksinny slowrie 16:32:10 geoff-: 'Geoff Levand' 16:32:42 #chair geoff- 16:32:42 Current chairs: ajeddeloh geoff- jlebon kaeso[m] ksinny slowrie 16:33:53 #topic Action items from last meeting 16:34:35 ajeddeloh to look into removing the hard /boot dependency for ignition-dracut 16:34:56 jlebon to email coreos-status about upgrade workaround once the next release is out 16:35:37 #info jlebon emailed coreos-status about upgrade workaround: https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/thread/NQ3GEHAMZC2CCCAND2JRVU6UD3D2JCVT/ 16:35:46 I've looked at https://github.com/coreos/ignition-dracut/pull/80, have some thoughts on it but haven't typed up a response 16:36:04 though the email subject line could've been more helpful :) 16:36:31 #action ajeddeloh to respond to https://github.com/coreos/ignition-dracut/pull/80 16:36:49 we can discuss that more at the open floor if folks want 16:37:04 ajeddeloh: Thanks; I'll be on vacation next week, so depending on when you add your comments I won't be able to respond any more. 16:37:34 sounds good. I'll bring it up at open floor too 16:37:48 #topic Support for reconfiguring the root storage 16:37:48 I've marked the first release in the Cincinnati graph as a "dead-end", meaning it won't have an update edge for auto-updates. 16:38:06 #link https://github.com/coreos/fedora-coreos-tracker/issues/94 16:39:19 This issue is basically "what if we want to move root somewhere else, or reformat root" 16:39:35 there's basically two parts: first boot and subsequent boots 16:40:06 walters and I have been mostly talking about the first boot side, and I think we have a decent plan, though we haven't tried it 16:40:47 I wanted to mostly talk about the second half: how do we find root if it's not trivial (e.g. on raid where we need to start mdadm) 16:41:33 I see three main options: 16:42:07 1) Do something like CL did with GPT type_guids and mark partitions that contain things containing root 16:42:36 2) generate a new initramfs that knows how to find root client side 16:43:12 3) have some custom tool in the initramfs that knows how to read a config file spelling out how to find root 16:43:20 thoughts? 16:44:53 do i recall correctly you had some hesitations about using GPT guids in the past for some other reasons? 16:45:45 yes, it prevents things like using a whole device for raid (have to do RAID on GPT instead of GPT on RAID) 16:46:15 generating a new initramfs... i think that'd mean running rpm-ostree from the initrd on first boot, right? 16:46:16 also means we need to allocate more type guids for different types of complex devices (which isn't too bad, they're freeeeee) 16:46:28 if we do 3 that would make a place for stuffing the parameters for "encrypted rootfs" 16:46:35 and it means we can't encode any configuration other than "this is raid" 16:47:10 jlebon: yeah, or from the real root but initramfs would be better 16:47:25 hmm, and i guess traditional Fedora systems deal with this by just regenerating the grub cfg, right? 16:47:34 kaeso[m]: right, I was actually thinking of the work you did for CL as inspiration for that 16:47:45 jlebon: the grub config? 16:48:05 grub config shouldn't need to change (well it would need to point at a new initramfs) 16:49:16 for 3 there is also the question of "do we try to figure out the root config in ignition or let the user write it by hand" 16:49:34 or let FCCT try to figure it out 16:49:48 if we don't care about supporting RO initramfs, I would strike out 1. I don't know whether suse folks ends up in that scenario or not 16:50:46 I'm in favor of not picking 1 as well 16:51:40 this might be a silly question: can't we just edit the BLS config to add a root karg and reboot? 16:51:56 jlebon: some things can't be done with kargs 16:52:18 .hello2 16:52:18 encryption, raid on raid (at least I think we can't do that with kargs) 16:52:18 bgilbert_: Sorry, but you don't exist 16:52:23 .hello2 16:52:23 bgilbert: bgilbert 'Benjamin Gilbert' 16:52:29 #chair bgilbert 16:52:29 Current chairs: ajeddeloh bgilbert geoff- jlebon kaeso[m] ksinny slowrie 16:52:40 ajeddeloh: actually, yeah, why couldn't everything be done with kargs? 16:52:59 ajeddeloh: hmm, though isn't e.g. traditional Fedora dealing with this through kargs? 16:53:31 for RAID, we need a list of array IDs to activate 16:53:50 most of the other scenarios require some configuration. Traditional distros deal with that by embedding the config in the client-generated initramfs 16:55:07 for raid on raid would we list the intermediate raid devices as kargs too? 16:55:22 We do care about ro initramfs, but as far as I understand this is only for the case of configuring the root device itself - in that case that's not a scenare we are currently thinking of. 16:55:45 ajeddeloh: sure 16:55:47 yeah, everything not-root can be handled "normally" in the real root 16:56:45 I like kargs if it can handle all cases (incl. encryption) 16:57:09 but like option 3 there's still the question of how do we determine the kargs 16:57:09 fos: is encrypted rootfs somewhere on the radar or not interesting at all? 16:58:18 It is, but we are thinking about using Ignition to set up a basic system configuration. The file systems will always have been created by some other mechanism. 16:59:34 one thought for both the config-file and kargs is have extra fields in FCCL like "contains_root: true" that are used to generate the config file/kargs 16:59:36 So the ignition disks stage is basically unused 16:59:56 * ajeddeloh 's favor stage is the disks stage 16:59:59 :( 17:00:06 sorry :-( 17:00:17 haha it's all good 17:01:04 We could reconsider it if way more filesystems including subvolume layout creation was implemented, but I guess that's totally out of scope :-P 17:01:31 fos: as in btrfs subvolumes? 17:01:39 one the tricky part in this is that there is an hand-over process required between ignition on first-boot (which creates stuff) and non-ignition on non-first-boot (which unlocks on further reboots) 17:01:39 exactly 17:02:00 yeah those are tricky 17:02:23 Another thing would be overlayfs, which is not supported at all currently, and which would require dynamic configuration. I gave up at that point. 17:03:09 kaeso[m]: would that case be different than something like raid where we want to set extra option? 17:03:23 options* 17:04:27 I think it boils to a very similar logic, just the complexity of the config itself can vary 17:05:19 right 17:05:52 any other thoughts? I can write this up in the ticket for more discussion there 17:07:20 #action ajeddeloh to summarize the discussion on reconfiguring root storage in the issue tracker 17:07:39 #topic open floor 17:07:48 if we plan to re-generate the initramfs to embed the config, when do we plan to re-generate it? 17:08:43 not sure 17:08:55 I'll leave that as an open question in the writeup 17:10:06 fos: regarding weakening the requirement on /boot, in a nutshell my thought was to split the ignition-setup into two parts 17:10:23 1) grabbing the platform-specific base config from the initramfs 17:10:30 2) grabbing the user config 17:10:50 and moving 2 to fedora-coreos-config isntead of ignition-dracut 17:11:14 and having a target they both use 17:11:46 ajeddeloh: does that address the PXE case? 17:11:55 so each distro would implement their own way of getting the user config to the correct place in the initramfs 17:12:07 bgilbert: which PXE case? 17:13:01 FCOS live PXE not reading from /boot 17:13:33 we could add either ConditionXXXX= or checks in the script for that 17:13:59 and that would only affect 2, 1 would be there always 17:14:29 in the case of live PXE would it not be expected that `ignition.config.url` would be specified on the kcmdline? 17:15:17 it would, but we need to not try waiting for a device that will never appear 17:16:01 fos: thoughts on that proposal? 17:16:46 ajeddeloh: Would be fine with me, too. My current pull request would just have the advantage that there would be some generic option for all distributions (and even customized user defined locations) 17:16:55 IIRC live-PXE has the exact same problem of trying to look for a /boot partition that does not exist 17:17:36 so I guess it unblocks that too 17:18:04 weren't we going to mark live-pxe with it's own kcmdline flag (or am I misremembering) 17:18:55 ajeddeloh: that's the plan 17:19:14 I guess you are not considering bare metal installation scenarios for Ignition, do you? 17:19:36 fos: I don't follow 17:19:56 we do, that's the whole reason to copy in the user config from /boot or in your case / 17:20:36 Having a configurable user config location would make it possible to just attach a USB flash drive - that's something which is quite common in our case. 17:21:05 if you're PXE booting FCOS to install you're either using the coreos.inst.* kcmdline options (in which case ignition doesn't run), or you're booting live pxe wiht an ignition config that installs to disk with a different ignition config 17:21:26 fos: is the USB flash drive the boot disk in this case? 17:21:52 When you tag the USB flash drive "boot", then yes ;-) 17:22:38 With FCOS, one would have to put the configuration on the /boot partition during image creation, wouldn't you? 17:23:04 nope, the installer mounts the /boot partition after installing and adds it there at install time 17:23:22 you _could_ do it at image creation time I suppose, but that seems slower/harder 17:23:53 it's also possible to do "layered" initramfs where we don't truly regenerate, just take the built image and append more stuff to it 17:25:11 right, you can actually include an ignition config that way too (could be useful for live pxe) 17:25:39 ok, i see. 17:26:17 so any objections to splitting out the base config fetching and the user config fetching into their own units, then moving user config fetching to fcos-config? 17:26:41 no objection to the splitting 17:26:52 nope 17:26:54 I'd rather see something overridable/disablable stay in ignition-dracut 17:27:07 to reduce the number of pieces a distro has to implement separately 17:28:26 bgilbert: as in like keep the user config fetching in ignition-dracut but have a way of disabling it? 17:28:32 yeah 17:29:01 I suppose you can do that with dracut modules, can't you (have one override the other) 17:29:26 I think fetching from /boot should be the default path for consistency, but let distros override if they know what they're doing 17:29:35 I'm fine with that. fos? 17:30:48 yes, fine with me, too. Like I said I'd prefer a generic way to configure it, but that's the deluxe variant. 17:30:55 We're about out of time. I've already action'd myself to reply to the ticket. Any last minute things? 17:32:12 #endmeeting