16:30:12 <dustymabe> #startmeeting fedora_coreos_meeting 16:30:12 <zodbot> Meeting started Wed Oct 5 16:30:12 2022 UTC. 16:30:12 <zodbot> This meeting is logged and archived in a public location. 16:30:12 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions. 16:30:12 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:30:12 <zodbot> The meeting name has been set to 'fedora_coreos_meeting' 16:30:16 <dustymabe> #topic roll call 16:30:18 <bgilbert> .hi 16:30:18 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net> 16:30:18 <dustymabe> .hi 16:30:21 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com> 16:30:34 <gursewak> .hi 16:30:35 <zodbot> gursewak: gursewak 'Gursewak Singh' <gurssing@redhat.com> 16:30:39 <ravanelli> .hi 16:30:40 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com> 16:31:06 <jlebon> .hello2 16:31:07 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com> 16:31:49 <dustymabe> #chair bgilbert gursewak ravanelli jlebon 16:31:49 <zodbot> Current chairs: bgilbert dustymabe gursewak jlebon ravanelli 16:33:04 <dustymabe> welcome all 16:33:07 <dustymabe> #chair jbrooks 16:33:07 <zodbot> Current chairs: bgilbert dustymabe gursewak jbrooks jlebon ravanelli 16:33:13 <davdunc> .hello2 16:33:15 <zodbot> davdunc: davdunc 'David Duncan' <davdunc@amazon.com> 16:33:17 <dustymabe> #chair davdunc 16:33:17 <zodbot> Current chairs: bgilbert davdunc dustymabe gursewak jbrooks jlebon ravanelli 16:33:26 <dustymabe> #topic Action items from last meeting 16:33:31 <dustymabe> Looks like there were no action items from last meeting! 16:33:36 <dustymabe> \o/ 16:34:33 <dustymabe> #topic Fedora CoreOS 37 test day 16:34:37 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1225 16:35:16 <dustymabe> Thanks everyone who helped us test F37 based CoreOS for the test day last week 16:35:35 <dustymabe> Thank you to jbrooks for organizing the test day! 16:35:40 <dustymabe> jbrooks++ 16:35:45 <jbrooks> .hello jasonbrooks 16:35:46 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com> 16:36:02 <jlebon> jbrooks++ 16:36:10 <dustymabe> looks like we didn't find any major issues - but did have some docs updates (always welcome) 16:36:24 <dustymabe> fifofonix also added some new documentation for proxying! 16:36:26 <dustymabe> fifofonix++ 16:36:57 <jlebon> nice! 16:37:16 <dustymabe> #info we had a successful test day. No major issues were found with F37 based FCOS on the `next` stream. We had several documentation updates that were proposed and some new documentation that was created as well! 16:37:18 <jbrooks> fifofonix++ 16:38:12 <dustymabe> if it turns out that DigitalOcean doesn't work in the end we can blame jdoss 16:38:18 <dustymabe> :) 16:38:38 <dustymabe> #topic Supporting FIPS mode in FCOS 16:38:47 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/302 16:39:06 <dustymabe> jlebon: i'm going to let you drive this topic 16:39:21 <fifofonix> hi 16:39:29 <jlebon> sure 16:39:31 <fifofonix> .hello 16:39:31 <zodbot> fifofonix: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1". 16:39:54 <dustymabe> #chair fifofonix 16:39:54 <zodbot> Current chairs: bgilbert davdunc dustymabe fifofonix gursewak jbrooks jlebon ravanelli 16:40:53 <jlebon> so FIPS originally used to work in FCOS when we initially added support for it in RHCOS. then it broke at some point. fifofonix mentioned that it works again now in f37. we don't really document it as supported anywhere but some FCOS users have chimed in saying it's a use case they're interested in. 16:41:22 <jlebon> so we need to decide if we want to officially support FIPS mode, and if so, figure out what the UX for it should be 16:41:32 <jlebon> (and actually test it in CI) 16:42:15 <jlebon> for comparison, in RHCOS there's glue code that makes it painless for customers. they just need to enable a knob in the MachineConfig. 16:42:34 <jlebon> some of that glue code should go away I think now that we have Ignition kargs but I suspect we'll still need some 16:42:51 <dustymabe> I don't have a strong opinion here 16:43:04 <bgilbert> if we care about FIPS 140-2 and not just 140-3, then Butane needs to know that FIPS is enabled when configuring disk encryption 16:43:21 <dustymabe> considering it is working now it would be nice to have a test for it, but I don't really know how much time we can spend on it 16:43:37 <bgilbert> the Butane openshift variant handles that with the openshift.fips flag but there's currently no equivalent for FCOS 16:44:15 <fifofonix> well, as a consumer i would be happy to ditch the systemd unit i am using to enable this currently. 16:44:34 <bgilbert> there's an argument that FCOS tries to be modern and so should just support 140-3 16:44:45 <bgilbert> but there's also an argument that FCOS tries to be modern and so should completely ignore FIPS :-) 16:44:49 <walters> does seem like a common butane field makes sense 16:44:53 <jlebon> bgilbert: i.e. with openshift, if the knob is on, Butane auto-enables encryption and adds some extra cryptsetup options? 16:44:58 <bgilbert> jlebon: yup 16:45:04 <bgilbert> well 16:45:04 <jlebon> cool, didn't know that 16:45:08 <bgilbert> well, it doesn't auto-enable encryption 16:45:17 <bgilbert> but it adds the cryptsetup options if encryption is also enabled 16:45:18 <dustymabe> bgilbert: if we were to add "sugar" to enhance the UX then maybe butane could just do the right thing if that "sugar" was enabled 16:45:20 <jlebon> ahh gotcha 16:45:43 <bgilbert> dustymabe: indeed 16:46:09 <bgilbert> on the openshift side we'd have the same problem we have with kernel arguments, which is that there'd be two ways to enable FIPS and we'd probably want to omit one 16:47:24 <bgilbert> in general, since we don't certify for FIPS and have no intention of certifying for FIPS, I'm a bit uncomfortable claiming to support it 16:47:51 <jlebon> bgilbert: i wonder what the overall Fedora stance is there 16:48:03 <dustymabe> yeah, though I do see value in testing upstream something that we need downstream 16:48:07 <bgilbert> dustymabe: indeed 16:48:22 <jlebon> if it's "supported but not officially certified" then matching that seems reasonable 16:48:31 <dustymabe> so for example.. the sugar,etc,documentation could be a nice to have 16:48:41 <dustymabe> but could come with a large disclaimer 16:48:54 * walters has a flashback to https://bugzilla.redhat.com/show_bug.cgi?id=1380866 16:48:57 <dustymabe> but that has its own drawbacks 16:49:37 <jlebon> walters: oh yeah, that's a classic :) 16:50:08 <dustymabe> walters: IOW - that's an example where we caught something early by testing it upstream and saved us pain later? 16:50:45 <walters> dustymabe: in theory...yes? I think we got that fixed before rhel8 16:50:47 <bgilbert> also also, we've generally been pretty good about making reasonable security tradeoffs in FCOS, and I generally consider FIPS to be snake oil 16:51:00 <dustymabe> bgilbert: :) 16:51:15 <fifofonix> fifofonix tries to find a comparison table of CURRENT, FUTURE, FIPS 16:51:37 <dustymabe> bgilbert: yeah, I don't know if I know enough to have an opinion on that one 16:52:47 <dustymabe> so.. my perspective here: 1) sugar/docs/tests would help us find issues before they land downstream 2) I think we would have to make certain disclaimers about users relying on this 3) not sure how high of a priority it would be to do the work to implement `1)` 16:53:20 <bgilbert> yeah, IMO the main argument is to reduce divergence with downstream 16:53:25 <walters> bgilbert: not going to fully disagree, I personally feel the value in having shared infrastructure and just the bare minimum of "OS boots in fips" is enough to start 16:54:45 <dustymabe> does the `1)` `2)` `3)` reflect the sentiment of most here? 16:54:58 <bgilbert> +1 to those 16:55:02 <bgilbert> though one approach is to not support FIPS sugar in Butane for FCOS 16:55:05 <jlebon> i can't find an official Fedora stance on this, but clearly we do support a crypto-policy called FIPS, and so I think it makes sense to support putting FCOS in that mode too, with the same caveats that would apply to traditional 16:55:36 <bgilbert> reason being: it doesn't help reduce divergence with downstream, since downstream would continue to use the openshift.fips flag for compatibility 16:55:58 <jlebon> i initially didn't think we'd need sugar at all for it and just document `fips=1` as a karg, but hadn't thought about the encryption bit 16:56:15 <dustymabe> bgilbert: IOW - maybe add just a test and not sugar or documentation? 16:56:18 <fifofonix> i'm interested in snakeoil comment, and whether really there should be sugar for enabling FUTURE crypto? 16:56:19 <bgilbert> and it's okay for the UX to be a little rough if we don't really want to encourage use of the feature 16:56:47 <jlebon> bgilbert: but maybe have butane warn about it if the karg is there and an encrypted device is defined 16:56:49 <bgilbert> dustymabe: well, we could add docs, and the docs could say to manually set the LUKS parameters if you want 140-2 compat 16:57:03 <bgilbert> jlebon: hmm, yeah, that's possible 16:57:22 <bgilbert> though the warning would be spurious if you only want 140-3 16:57:33 <jlebon> gotcha 16:57:57 <jlebon> so 140-2 dictates a specific algo to use for disk encryption, but not 140-3? 16:58:11 <bgilbert> jlebon: no, the issue is that 140-2 doesn't allow XTS 16:58:18 <bgilbert> so we have to fall back to CBC 16:59:42 <bgilbert> fifofonix: sugar for enabling FUTURE could be interesting 17:00:00 <jlebon> bgilbert: gotcha. so in that case, probably better to leave it as a documentation item 17:00:35 <jlebon> i think there's an issue for better UX for crypto policies in general 17:00:51 <jlebon> where Butane sugar to manually set the symlinks was discussed 17:01:07 <dustymabe> jlebon: i.e. https://github.com/coreos/fedora-coreos-tracker/issues/607 17:01:36 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/607 17:01:38 <fifofonix> As a first step maybe another PR on docs covering some of: https://discussion.fedoraproject.org/t/strong-crypto-settings-how-best-to-define-apply/22600/15 17:01:42 <jlebon> yeah, I don't think we ever added the docs for that 17:02:32 <dustymabe> anyone want to sign up for pushing that `pending-action` ticket forward? 17:02:47 <walters> (one thing to note here is that in a layering world, the python dependency is OK if one can also remove it after a build is done) 17:03:43 <dustymabe> let's see if we can move forward on this conversation 17:03:46 <bgilbert> fifofonix: re snake oil, basically a really heavyweight certification process is orthogonal to security 17:03:52 <dustymabe> #proposed We do think documentation and tests for FIPS in FCOS would be useful for catching issues before the land downstream, but because we're not confident in the ability to keep FIPs mode working we're not sure about encouraging users to use it. At this time we're also not sure if adding "sugar" to butane for this would be necessary given the above position. 17:04:35 <dustymabe> maybe s/necessary/desired/ 17:04:52 <walters> that seems like a non-conclusion? 17:05:04 <bgilbert> fifofonix: the standard has some good rules in it, but also ends up pushing distros to maintain special code paths in order to run less-maintained crypto code implementing obsolete algorithms 17:05:31 <fifofonix> bgilbert: 🙏 17:05:33 <dustymabe> walters: the discussion started off at "should we do anything for this"? 17:05:53 <dustymabe> and I think we agreed that having a test for it would help at least for preventing issues leaking into downstream 17:06:23 <dustymabe> walters: want to attempt a #proposed? 17:06:45 <jlebon> how about "we will add documentation and tests for FIPS in FCOS but will not add sugar for it in Butane at this time" ? 17:06:56 <jlebon> (this is orthogonal to prioritizing it) 17:07:29 <dustymabe> yeah - that second part is the open question 17:07:44 <walters> #proposed adding a fips=1 kernel argument via Ignition becomes the canonical low level API to signal desire for FIPS mode, FCOS will add glue to do the remaining userspace portions on firstboot, and we will add a minimal kola test that verifies this works; no butane sugar yet 17:07:48 <dustymabe> i get caught up when we say `we will` and then have no plans for it to happen anytime soon 17:08:19 <dustymabe> ^^ response to jlebon 17:08:19 <bgilbert> dustymabe: yeah 17:08:36 <jlebon> agreed this is a long-running problem, but at least we declare our intent 17:08:41 <jlebon> +1 to proposed 17:08:49 <cyberpear> +1 17:09:12 <bgilbert> walters: should we mention docs? 17:09:58 <dustymabe> +1 docs would be nice 17:10:42 <walters> #proposed adding a fips=1 kernel argument via Ignition becomes the canonical low level API to signal desire for FIPS mode, FCOS will add glue to do the remaining userspace portions on firstboot, and we will add a minimal kola test that verifies this works; no butane sugar yet, and documentation that briefly describes this state 17:11:14 <jlebon> ack 17:11:16 <dustymabe> +1 17:11:28 <bgilbert> +1 17:12:15 <dustymabe> #chair walters 17:12:15 <zodbot> Current chairs: bgilbert davdunc dustymabe fifofonix gursewak jbrooks jlebon ravanelli walters 17:12:22 <jlebon> i think possible butane sugar to set the crypto-policy is strongly related to this since it'd allow us to drop a chunk of the userspace portion 17:12:37 <dustymabe> any other votes? 17:12:47 <fifofonix> +! 17:12:56 <fifofonix> +1 even 17:13:28 <dustymabe> #agreed adding a fips=1 kernel argument via Ignition becomes the canonical low level API to signal desire for FIPS mode, FCOS will add glue to do the remaining userspace portions on firstboot, and we will add a minimal kola test that verifies this works; no butane sugar yet, and documentation that briefly describes this state 17:13:46 * dustymabe wonders about priority - but maybe a discussion for another time 17:14:02 <dustymabe> #topic OstreeNativeContainerStable proposal 17:14:09 <dustymabe> #link https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable 17:14:37 <dustymabe> This is a proposal by walters to Fedora 17:14:51 <dustymabe> which has implications on Fedora CoreOS 17:15:06 <walters> (yes, though there are two other names there too, happy to gather more 😄) 17:15:22 <dustymabe> I'm interested in discussing this proposal in the context of this group 17:15:37 <bgilbert> walters: does the switch away from ostree repos substantially increase the download size of updates? 17:15:52 <dustymabe> ^^ that's one bit that has improved 17:16:18 <dustymabe> The containers are now "chunked" mostly based at the RPM level IIUC 17:16:25 <dustymabe> and you only download the chunks that you don't have 17:16:55 <walters> bgilbert: see https://github.com/ostreedev/ostree-rs-ext/issues/69 - you can actually try this today by pulling `quay.io/fedora/fedora-coreos:stable` and then pulling `quay.io/fedora/fedora-coreos:testing-devel` and seeing the shared chunks 17:17:07 <bgilbert> +1 17:17:22 <dustymabe> walters: when you say "pulling" do we need to be more specific? 17:17:32 <dustymabe> i.e. `podman pull` wouldn't do the right thing would it? Or maybe it would 17:17:53 <walters> it works exactly the same in podman (and docker, and kubelet) 17:18:30 <dustymabe> +1 17:19:06 <walters> but we are missing a section on this, actually the delta support was a big concern of Fedora IoT, will flesh that out there 17:19:09 <dustymabe> at a high level I think this proposal is large and has several pieces that aren't necessary to couple. it might be good to break them out into separate proposals 17:19:21 <bgilbert> I'll note for the record that I strongly disagree with https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable#Encouraging_building_derivative_containers in an FCOS context 17:19:29 <walters> but very briefly, we want https://github.com/containers/image/pull/902 17:20:06 <walters> bgilbert: can you elaborate why? 17:20:28 <bgilbert> it makes sense in RHCOS for a variety of customer reasons ("run all your code in containers" + "use Ignition" is a big shift for some customers) 17:20:44 <jlebon> walters: would it make sense to rework this proposal to only set the stage so people can opt-in and have changing defaults (and changing default behaviours) be a separate proposal, possibly f39? 17:20:55 <jlebon> otherwise, it's quite a lot to take in in one shot 17:21:56 <bgilbert> but for FCOS I think we should try to maintain the core premise that you shouldn't install arbitrary software in the host, and shouldn't use FCOS as an "OS construction kit". users who want arbitrarily customizable OSes can use other editions; a big part of the FCOS value prop is that we don't do that. 17:22:20 <dustymabe> bgilbert: I think your concern is mostly with how much we encourage people to create derivatives? 17:22:58 <bgilbert> dustymabe: yup, where "create derivatives" is read to include "customize the host software for the user's own purposes" 17:23:45 <jlebon> bgilbert: i think there's some shades there, just like there is today with layering in FCOS. some software make sense to install/layer because they're core host services 17:24:08 <bgilbert> jlebon: there are indeed shades :-) 17:25:01 <walters> one big, big thing here is that it gives an efficient and sane model for "day 2" config changes that aren't "reprovision the machine" 17:25:37 <bgilbert> walters: yes, and that's out of scope for FCOS :-) 17:25:40 <dustymabe> I think some of the nuances of the postivies and negatives of "CoreOS Layering" is covered in https://github.com/coreos/fedora-coreos-tracker/issues/1219 17:26:11 <dustymabe> bgilbert: I'm not sure I 100% agree. I think it's worth exploring our position on that 17:26:31 <bgilbert> walters: I'd rather see us improve the reprovisioning UX instead 17:26:32 <dustymabe> though.. what I do think this discussion is proving. is that the proposal is too large 17:27:09 <dustymabe> for example.. there's switching to container registry + CoreOS Layering + dnf cliwrap + rebranding - all thrown in there 17:27:30 <dustymabe> all of those could be considered separately IIUC 17:27:58 <dustymabe> well s/IIUC/IMO 17:28:00 <walters> well...dnf cliwrap makes less sense without the rest 17:28:13 <jlebon> bgilbert: i mean, it's out of scope today because we can't really support it with our current tech, so we ended up building our philosophy around that. but if the tech could support it, we might've come to a different conclusion 17:28:57 <walters> dustymabe: I'm kind of trying to actively stop saying "coreos layering" FWIW because it actually has nothing to do with coreos other than it happens to be a place where I'm trying to drive this to happen first 17:29:12 <dustymabe> walters: sorry - what's the new term? 17:29:18 <dustymabe> I'll try to use that 17:29:38 <walters> there's "ostree native container" but...see, I think the actual succinct term should be "dnf image" 17:30:22 <dustymabe> To me "ostree native container" is just an ostree in a container.. but "ostree native container builds" could be what we are referring to as CoreOS Layering today 17:30:46 <jlebon> "ostree container layering" ? 17:30:57 <walters> hmm, I don't quite get what the difference between those two is 17:31:00 <dustymabe> semantics :) jlebon +1 17:31:28 <walters> the current change says: "A top level goal is that users should not need to know or understand ostree (or rpm-ostree); they only need to know containers and most key functionality is exposed via the yum/dnf frontend with a few extensions. " 17:31:47 <dustymabe> walters: "ostree native container" could just be a container we put in a registry that you could pull that didn't support Dockerfile style building on top 17:31:47 <walters> if you need to know ostree to customize the OS, this effort has failed 17:32:06 <bgilbert> jlebon: partially, but I'm not sure completely, e.g. moving back toward imperative doesn't seem great 17:32:20 <bgilbert> (and also, our users signed up for the path we're on, and I'm not wild about dragging them in a different direction) 17:32:27 <walters> dustymabe: true...but the value in doing the former is quite low without the latter 17:33:01 <jlebon> bgilbert: agreed 17:33:01 <dustymabe> still has "distribution" benefits (just container registry to deliver OS updates) 17:33:15 <dustymabe> anyway that's a tangent 17:33:21 <walters> bgilbert: bear in mind one can use any container build system on top of this, you are not required to use Dockerfile - and in fact, i'm going to investigate having the existing `rpm-ostree compose image` learn to build derivative images (we *already have* a declarative interface there) 17:33:48 <bgilbert> walters: +1 17:34:12 <jlebon> where i was driving this before was to a place where you still use Ignition to configure your hosts, but now it's just about whether you apply it to the image or to a specific node 17:34:35 <dustymabe> considering we've discussedf like 4 different topics here.. does anyone disagree that it would be good to break up this proposal (how do you eat a cow? one bite at a time.)? 17:35:15 <walters> dustymabe: note that the current proposal does link to github issues for the individual topics already 17:35:19 <walters> or at least, some of them 17:35:20 <bgilbert> jlebon: yeah, though I've seen discussion lately that deemphasizes Ignition quite a bit 17:35:46 <dustymabe> walters: yes, but it's hard to propose and discuss things as a community and at a project level that have so many subtopics 17:36:02 <dustymabe> kind of like a code review that has 100 commits - sometimes we break those off into smaller reviews 17:36:03 <bgilbert> (note: Ignition has plenty of problems of its own, but it's the UX we've built expectations around) 17:37:07 <bgilbert> dustymabe: +1 to splittin 17:37:09 <bgilbert> g 17:38:33 <jlebon> i see at least the "changing over-the-wire format" as a good chunk we could easily split out 17:38:44 <walters> hmm, splitting how exactly? 17:39:07 <dustymabe> indeed, which has it's own lot of details that will need investigation/discussion 17:39:14 <jlebon> as a separate change proposal 17:40:15 <walters> you're right that technically we could do everything else without changing the default wire transport, but...that seems weird 17:40:40 <dustymabe> or we could change the default wire transport without doing everything else 17:40:44 <dustymabe> they're independent 17:40:49 <walters> (Note that https://github.com/coreos/coreos-assembler/pull/2523 when we flip it on will make the initial pull efficient for derived images too) 17:40:53 <jlebon> yeah, i more meant the latter :) 17:41:16 <dustymabe> #info Summary: there are a lot of subtopics within this proposal and we think it would be useful to split up the proposal into smaller proposals in order to focus discussion and help drive to conclusions and actions. 17:41:31 <dustymabe> I can undo ^^ if needed 17:42:44 <dustymabe> we're out of time already - move to open floor? 17:43:19 <jlebon> it's good to discuss it as a whole too of course to see the big picture 17:43:57 <jlebon> definitely interested to see what the dnf folks and the rest of Fedora think about `dnf image`. that's a bit i don't quite agree with :) 17:43:58 <dustymabe> I'm OK with that too. though might be good at a video session? 17:44:33 <jlebon> to go over how to split it? 17:45:11 <walters> bgilbert: just to have something actually change as a result of this discussion I edited the page to s/Encourage/Support/ in "building derivative containers" 17:45:12 <dustymabe> no.. to talk about the high level goal - here we keep getting caught up in the details, which are important, but gets in the way of the high level outcome discussion I think 17:45:30 <jlebon> dustymabe: that makes sense, yeah 17:45:39 <bgilbert> walters: +1 to "Support", thanks! 17:45:49 <dustymabe> #topic open floor 17:45:55 <dustymabe> anybody with anything for open floor? 17:46:11 <dustymabe> sorry for the meeting running late, though I'm always happy to have a lot of discussion with a lot of participants 17:46:29 <jlebon> walters: should the bullet under https://fedoraproject.org/wiki/Changes/OstreeNativeContainerStable#Detailed_Description be changed too? 17:47:21 <dustymabe> #info Fedora Final Freeze is now in effect and we will start building `next` once a week to increase use/feedbacak of F37 in during this time 17:47:23 <walters> sure, done 17:47:31 <dustymabe> #undo 17:47:31 <zodbot> Removing item from minutes: INFO by dustymabe at 17:47:21 : Fedora Final Freeze is now in effect and we will start building `next` once a week to increase use/feedbacak of F37 in during this time 17:47:34 <dustymabe> #info Fedora Final Freeze is now in effect and we will start building `next` once a week to increase use/feedback of F37 in during this time 17:47:48 <bgilbert> #info The new serial console defaults landed in `next`. Please test! 17:47:52 <jlebon> walters: +1 17:47:56 <dustymabe> bgilbert++ 17:48:04 <bgilbert> #link https://lists.fedoraproject.org/archives/list/coreos-status@lists.fedoraproject.org/message/GHLXX4MXNHUEAXQLK6BZN45IQYHRVQB4/ 17:48:33 <jlebon> bgilbert: awesome! really nice to finally see it out 17:48:42 * dustymabe will close out the meeting in 60s if no other discussion occurs 17:49:37 <fifofonix> busy meeting. thanks for running dustymabe. ostree native containers fascinating. scary user optionality. 17:49:41 <dustymabe> #endmeeting