16:30:51 <dustymabe> #startmeeting fedora_coreos_meeting
16:30:51 <zodbot> Meeting started Wed Aug  1 16:30:51 2018 UTC.
16:30:51 <zodbot> This meeting is logged and archived in a public location.
16:30:51 <zodbot> The chair is dustymabe. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:51 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:51 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:55 <dustymabe> #topic roll call
16:30:58 <ajeddeloh> .hello2
16:30:59 <zodbot> ajeddeloh: ajeddeloh 'Andrew Jeddeloh' <andrew.jeddeloh@redhat.com>
16:31:09 <jbrooks> .fas jasonbrooks
16:31:10 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:31:17 <bgilbert> .hello2
16:31:18 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:31:33 <rfairley> .hello rfairleyredhat
16:31:34 <zodbot> rfairley: rfairleyredhat 'Robert Fairley' <rfairley@redhat.com>
16:31:44 <geoff-> Geoff Levand <geoff@infradead.org>
16:32:00 <lorbus[m]> .hello lorbus
16:32:03 <zodbot> lorbus[m]: lorbus 'Christian Glombek' <c@petersen-glombek.de>
16:32:28 <slowrie> !hello2
16:32:34 <slowrie> .hello2
16:32:35 <zodbot> slowrie: slowrie 'Stephen Lowrie' <slowrie@redhat.com>
16:32:48 <ksinny> .hello sinnykumari
16:32:49 <zodbot> ksinny: sinnykumari 'Sinny Kumari' <ksinny@gmail.com>
16:34:24 <dustymabe> #chair ajeddeloh jbrooks rfairley lorbus[m] slowrie ksinny
16:34:24 <zodbot> Current chairs: ajeddeloh dustymabe jbrooks ksinny lorbus[m] rfairley slowrie
16:34:40 <dustymabe> hope everyone is having a great wednesday!
16:34:53 <dustymabe> welcome jbrooks
16:34:55 <dustymabe> jberkus:
16:35:02 <jberkus> .hello jberkus
16:35:05 <zodbot> jberkus: jberkus 'Josh Berkus' <josh@agliodbs.com>
16:35:07 <jbrooks> o/
16:35:09 <dustymabe> #chair geoff- jberkus
16:35:09 <zodbot> Current chairs: ajeddeloh dustymabe geoff- jberkus jbrooks ksinny lorbus[m] rfairley slowrie
16:35:31 <dustymabe> ok we'll get started here with previous meeting action items
16:35:55 <jdoss> .hello jdoss
16:35:56 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
16:36:06 <dustymabe> #chair jdoss
16:36:06 <zodbot> Current chairs: ajeddeloh dustymabe geoff- jberkus jbrooks jdoss ksinny lorbus[m] rfairley slowrie
16:36:14 <dustymabe> #topic Action items from last meeting
16:36:31 <dustymabe> only had one from last meeting
16:36:36 <dustymabe> * dustymabe update the issue/PRD with distinctions for the different
16:36:37 <dustymabe> categories
16:37:15 <dustymabe> #info dusty added results from last meeting about use case distinctions to #7
16:37:19 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/7#issuecomment-408454966
16:37:47 <dustymabe> which means we can dig into the issues now
16:38:08 <dustymabe> #topic Partition Layout
16:38:15 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/18
16:39:09 * dustymabe waits while people read - also walters and ajeddeloh feel free to drive the discussion
16:40:06 <walters> I don't have much to add beyond what has been said so far
16:41:10 <bgilbert> walters: I know there are some security hardening specs that want /var, /var/log, and /var/log/audit all on separate partitions.  is that case handled at all in Atomic now?
16:41:11 <dustymabe> I think I missed #chair on bgilbert
16:41:18 <dustymabe> #chair bgilbert
16:41:18 <zodbot> Current chairs: ajeddeloh bgilbert dustymabe geoff- jberkus jbrooks jdoss ksinny lorbus[m] rfairley slowrie
16:41:23 <bgilbert> .chair bgilbert
16:41:23 <zodbot> bgilbert is seated in a chair with a nice view of a placid lake, unsuspecting that another chair is about to be slammed into them.
16:41:44 <dustymabe> damn zodbot
16:41:47 <dustymabe> #chair bgilbert
16:41:47 <zodbot> Current chairs: ajeddeloh bgilbert dustymabe geoff- jberkus jbrooks jdoss ksinny lorbus[m] rfairley slowrie
16:41:50 <jdoss> chair as a service
16:42:20 <ajeddeloh> So one new thing to consider: 4k sectors
16:42:30 <walters> bgilbert: yeah, was the rationale behind  https://bugzilla.redhat.com/show_bug.cgi?id=1459623 AKA https://github.com/ostreedev/ostree/issues/855
16:42:52 <ajeddeloh> CL cannot install on disks with 4k sectors since the GPT partition layout is different.
16:43:02 <walters> ajeddeloh: yeah I'm curious about that...I think `mkfs` will try to detect and optimize these things too right?
16:43:07 <bgilbert> https://github.com/coreos/bugs/issues/2109 for a bit more context on the 4k issue
16:43:35 <walters> ah, `-global virtio-blk-device.logical_block_size=4096` is nice and easy
16:43:44 <dustymabe> bgilbert: ajeddeloh: so that is an argument for *not* shipping a dd-able disk image ?
16:43:53 <bgilbert> when I looked at that issue before, I had the impression that it was fixable
16:44:00 <bgilbert> the partitions themselves are already aligned properly
16:44:06 <bgilbert> and we just need to rewrite the GPT.
16:44:11 <ajeddeloh> not necesarily, but its something that needs some work to support
16:44:20 <ajeddeloh> yeah
16:44:30 <dustymabe> would that be a lot of work?
16:44:46 <bgilbert> coreos-install is in bash so it wasn't obvious how to do it there
16:44:51 <ajeddeloh> rewriting gpt: probably not
16:45:13 <ajeddeloh> if we had some slightly better tool than bash to handle it
16:45:18 <bgilbert> at this point IMO rewriting coreos-install in not-bash is a good idea, but we could also keep a separate 4K GPT somewhere that it dd's down
16:45:50 <bgilbert> I'm not aware that CL ever had a user complain about it.  it only affects the boot drive
16:46:03 <bgilbert> and probably servers are all SSD-root by now
16:46:13 <dustymabe> ok so let me make a statement and see if anyone disagrees
16:46:40 <dustymabe> 1. we'd like to strive for a fixed partition layout for our shipped image artifacts
16:46:44 <walters> one thing that isn't clear to me; e.g. `mkfs.xfs` defaults to a sector size of 512, but clearly on a 4k disk one would want that to be 4k right?  is that normally the responsibility of a higher level tool?  would nee
16:46:58 * walters glances through google results for "xfs 4k sector"
16:46:59 <dustymabe> 2. the goal here is to have an image that is dd-able to any harddrive and can be booted
16:47:38 <dustymabe> 3. other mounts (like /var/) can be added at runtime on first boot
16:48:20 <ajeddeloh> There's really two issues: gpt and whatever the root fs is
16:48:33 <ajeddeloh> gpt is easy: just blat downt the correct table
16:48:54 <ajeddeloh> assuming we can support root deployed between ignition stages, we could fix the fs issue too
16:49:37 <dustymabe> anyone want to comment on 1/2/3 above ?
16:50:09 <walters> agreed on those
16:50:21 <bgilbert> +1
16:50:55 <ajeddeloh> worth noting #2 is the sole advantage of a dd'able image, but yeah +1
16:51:15 <bgilbert> walters: looks like the default XFS block size is 4KB
16:51:43 <walters> bgilbert: yeah, block size vs sector size: https://www.spinics.net/lists/linux-xfs/msg10526.html
16:52:00 <dustymabe> ok I'll update the issue with the results of this conversation
16:52:10 <dustymabe> we ready to move to next topic ?
16:52:31 <bgilbert> walters: not immediately obvious from that thread why XFS cares about sector size once the block size is set
16:53:16 <dustymabe> #topic arm64 / aarch64 support for Fedora CoreOS
16:53:22 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/13
16:53:26 <walters> yeah let's file this as "to determine later", I think we can plow forward with a dd-able image and deal with the 4k drive later
16:53:31 <bgilbert> +1
16:53:38 <dustymabe> ed-packet: around today ?
16:54:02 <geoff-> ed-packet: is traveling
16:54:08 <dustymabe> geoff-: +1
16:54:16 <dustymabe> geoff-: do you have anything for this topic ?
16:54:24 <geoff-> will the build system do cross compiles?
16:54:40 <geoff-> or will it only be 'self-hosted'?
16:54:44 <lorbus[m]> how does this relate to the Fedora IoT edition?
16:54:47 <dustymabe> cross compile.. meaning compiling for aarch64 on an x86_64 host ?
16:54:57 <geoff-> dustymabe: yes
16:55:01 <dustymabe> lorbus[m]: it doesn't (this is mostly aarch64 servers)
16:55:22 <dustymabe> geoff-: no I don't believe so. we typically build aarch64 pieces on aarch64 hosts in Fedora
16:55:36 <dustymabe> walters: ksinny correct me if I'm wrong
16:55:54 <walters> https://blog.verbum.org/2012/10/13/building-everything-from-source-vs-self-hosting/
16:56:00 <ksinny> dustymabe: yes, we build on aarch64
16:56:29 <ksinny> No cross compilation
16:56:54 <geoff-> that was one nice feature of CL, I didn't need to port the build system to arm64
16:56:56 <dustymabe> walters: was there anything in particular you wanted us to glean from that link without reading the whole thing right now in this meeting ?
16:57:08 <walters> fedora is defined to be a self-hosted system (today)
16:57:16 <walters> hence that's how the build system works
16:57:27 <dustymabe> walters: +1
16:57:41 <walters> but that doesn't mean we can't define our own subset that isn't required to be self hosting
16:58:09 <walters> obviously doing so has a whole lot of implications for how much code/infrastructure such a subset would share with current fedora
16:58:21 <dustymabe> walters: true. geoff- this is something we'
16:58:26 <dustymabe> we'll have to address as we go along
16:58:47 <walters> (I personally a m a fan of https://github.com/openembedded/openembedded-core )
16:58:59 <dustymabe> ultimate goal is to have build tools that work on the subset of platforms we support and that each artifact is built in that environment
16:59:56 <dustymabe> geoff-: WDYT?
17:00:21 <ksinny> geoff-: In Fedora build infrastucture, compose process for all architetcures started by pungi and depending upon on what articture we are going to build an artifatcs(image,iso,etc), we send jobs to architecture specific builders
17:00:49 <ksinny> This is my understanding
17:01:10 <dustymabe> ksinny: yep. that's how it is done today. We will probably evaluate what tools we use along this journey
17:01:12 <geoff-> I guess my main concern is a bunch of work is done, only tested on x86, then when I pull that in it breaks horribly on arm64
17:01:26 <ksinny> dustymabe: right
17:01:40 <dustymabe> geoff-: ksinny has helped us enable aarch64 for atomic host in the past
17:02:14 <dustymabe> can we establish you and her as a point team for helping make sure we consider aarch64 when making decisions ?
17:03:07 <ksinny> For interested folks, we can work on providing access to community aarch64 hardware in order to work on adding support/debugging things
17:03:59 <geoff-> dustymabe: OK, that sounds good, I guess ed-packet would be interested also
17:04:47 <dustymabe> indeed. ksinny can you update the ticket to state we established you 3 as the point team for aarch64 considerations during design decisions ? we won't close the ticket yet because we still want to hear from ed-packet
17:05:13 <ksinny> dustymabe: will update
17:05:20 <dustymabe> #action dustymabe to update #18 with results of partition layout discussion and close ticket
17:05:44 <dustymabe> #action ksinny to update #13 to reflect her, ed, geoff are point team for aarch64
17:06:02 <dustymabe> ok will move to next topic in 30 seconds
17:06:31 <dustymabe> #topic flesh out use cases for Fedora CoreOS
17:06:39 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/7
17:07:05 <dustymabe> ok so last meeting we established some more definitions around what each category means
17:07:13 <dustymabe> see https://github.com/coreos/fedora-coreos-tracker/issues/7#issuecomment-408454966
17:07:26 <dustymabe> do we now want to revisit what use cases we have in each category ?
17:07:34 <dustymabe> https://github.com/coreos/fedora-coreos-tracker/issues/7#issuecomment-407532435
17:09:39 * dustymabe checks net connection
17:09:43 <bgilbert> under those definitions, I'm okay with the use cases as defined
17:10:07 <jberkus> it looks fine to me.
17:10:26 <ajeddeloh> +1
17:10:37 <dustymabe> \o/
17:11:07 <jbrooks> It looks good to me
17:11:09 <dustymabe> I know there was at least one person commenting in the ticket about desire for stronger wording around security being a focus
17:11:27 <dustymabe> that seems reasonable to me
17:11:52 <ajeddeloh> To play devils advocate for that one: what actions can we (reasonably) take for security
17:12:18 <ajeddeloh> moreso than Fedora already does
17:12:36 <dustymabe> ajeddeloh: i think maybe the stated goal of having a reasonable small base
17:12:51 <dustymabe> what wording did container linux use to that effect ?
17:12:53 <walters> "Fedora" doesn't ship with auto-updates on by default today
17:13:01 * ajeddeloh is all for as minimal a distro as possible
17:13:17 <ajeddeloh> I think CL's wording changed over time
17:13:31 <geoff-> SELinux was never really done right in CL.  I don't want to see that happen again...
17:13:52 <dustymabe> geoff-: i think there is 0 chance of us shipping FCOS without SELinux enabled by default
17:14:16 <ajeddeloh> the original goal for CL was to not leave people's servers vulnerable. That was goal #1. Auto updates + containers allowed that since the containerization meant your host didn't matter as much
17:14:19 <walters> geoff-: https://github.com/coreos/ignition/pull/569
17:14:24 <ajeddeloh> geoff-: ++
17:14:29 <bgilbert> the security point does seem relevant wrt prioritizing out-of-cycle releases
17:14:52 <bgilbert> the naive approach for e.g. stable is to snapshot Fedora updates at regular intervals
17:15:02 <bgilbert> but we also need to pay attention to CVEs and roll out-of-cycle updates
17:15:05 <ajeddeloh> then docker and k8s started having all sorts of weird compat contrainsts for the host and mucked with that vision
17:15:50 <bgilbert> not sure how aggressively Atomic does that now, but we should probably at least make it an explicit FCOS goal
17:16:07 <bgilbert> s/Atomic/AH/
17:16:48 <dustymabe> ok so from what I'm hearing we can probably add some wording to the PRD (or identify wording that already exists) to state that we consider security important and a few things that we do with FCOS (SELinux enabled, reasonably small base, automatic updates)
17:17:17 <slowrie> +1
17:17:35 <bgilbert> +1
17:18:02 <lorbus[m]> +1
17:18:31 <geoff-> +1, out-of-cycle updates are a necessity
17:18:46 <bgilbert> what is the AH policy onw?
17:18:49 <bgilbert> now
17:19:14 <dustymabe> out-of-cycle meaning what?
17:19:30 <slowrie> outside of the set release cadence
17:19:37 <dustymabe> we currently ship every two weeks. If a heartbleed comes out we'll respin as soon as the update is available
17:20:10 <bgilbert> is there a (rough) sense of the threshold for doing so?
17:20:22 <bgilbert> heartbleed is a pretty high bar :-)
17:20:32 <dustymabe> i.e. what level of security vuln prompts an impromptu release ?
17:20:37 <bgilbert> right
17:21:09 <dustymabe> so far we've mostly gone by judgement.. ratings and such help
17:21:23 <walters> i'd say it hasn't happened often enough to have any kind of true policy
17:21:25 <dustymabe> if we want to establish some metric we can try to do that
17:22:21 <bgilbert> CL doesn't have a crisply defined threshold either
17:22:42 <bgilbert> but it happens a nontrivial number of times per year
17:23:05 <bgilbert> which is mostly what I was wondering about
17:23:48 <dustymabe> +1 either by judgement or established metric work for me
17:24:07 <dustymabe> we can decide going when we start having those problems
17:24:24 <ksinny> I think it will also depend upon how frequently FCOS will recieve automatic updates?
17:24:44 <dustymabe> ok bgilbert jbrooks ajeddeloh jbrooks all: can we get some in-ticket votes for the use cases? https://github.com/coreos/fedora-coreos-tracker/issues/7#issuecomment-409654340
17:24:57 <dustymabe> ksinny: we haven't really decided a cadence yet
17:25:06 <dustymabe> ksinny: do you mind opening a new ticket for that ?
17:25:24 <ksinny> dustymabe: will do it
17:25:35 <jbrooks> ok
17:27:07 <dustymabe> #agree the use cases in #7 seem reasnonable to the members of this meeting. each individual will give their vote in ticket
17:27:21 <dustymabe> ok open floor time
17:27:23 <dustymabe> #topic open floor
17:27:37 * dustymabe waves at everyone and thanks them for waiting patiently
17:27:53 <bgilbert> belated additional thought on partition layout:
17:28:14 <bgilbert> one change vs. CL with everything-on-root is that the user can fill up the disk so we can no longer perform updates.
17:28:22 <dustymabe> lorbus[m]: did you have something for open floor ?
17:28:23 <bgilbert> is that a concern?
17:29:03 * ajeddeloh has to go
17:29:03 <jberkus> jbrooks: so if we're doing kubernetes for coreos, I'm assumign we still use containerized kubeadm?
17:29:06 <dustymabe> bgilbert: is there any way around "filling up root" in either traditional CL, traditional atomic host, or FCOS ?
17:29:12 <ajeddeloh> see y'all next week
17:29:15 <lorbus[m]> dustymabe: it's not ready yet :(
17:29:22 <jbrooks> jberkus, We need to discuss all of that
17:29:25 <dustymabe> lorbus[m]: poo
17:29:30 <bgilbert> dustymabe: there's no way around it happening, but on CL it shouldn't break updates
17:29:35 <bgilbert> separate /boot and /usr
17:29:35 <lorbus[m]> who of you is going to be at Flock next week?
17:29:50 <jbrooks> At the moment, containerized kubeadm isn't even working for me. I'd prefer to ship it in the image
17:30:08 <jbrooks> kubeadm+kubelet+kubectl
17:30:08 <bgilbert> dustymabe: in principle we could do the same here, but I guess it'd require rpm-ostree changes
17:30:12 <walters> bgilbert: i guess a big question is whether we ship with /var as a separate mount by default
17:30:30 <dustymabe> bgilbert: maybe we can pick up that discussion in the ticket ?
17:30:44 <dustymabe> lorbus[m]: I'll be at flock and so will bgilbert and kaeso
17:30:45 <bgilbert> dustymabe: sure
17:30:47 <bgilbert> lorbus[m]: me
17:31:01 <ksinny> me as well
17:31:05 <dustymabe> jbrooks: i think that seems reasonable +cri-o
17:31:15 <jbrooks> Yeah, that too
17:31:27 <jbrooks> But then that raises questions about origin
17:31:34 <dustymabe> but I think practically we need to evaluate what we ship when we start adding it to the image
17:31:35 <lorbus[m]> cool! looking forward to meeting you all!
17:32:59 <dustymabe> jbrooks: does that seem reasonable?
17:33:21 <jbrooks> dustymabe, evaluating what we ship?
17:33:47 <dustymabe> i.e. target those 4 and then evaluate if we want to change that once we actually have an artifact
17:34:09 * dustymabe notes we are over time
17:34:12 <jbrooks> Yes
17:34:21 <dustymabe> cool, anyone else with anything before we break?
17:35:16 <dustymabe> #endmeeting