16:30:22 <lorbus> #startmeeting fedora_coreos_meeting
16:30:22 <zodbot> Meeting started Wed Nov 14 16:30:22 2018 UTC.
16:30:22 <zodbot> This meeting is logged and archived in a public location.
16:30:22 <zodbot> The chair is lorbus. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:22 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:22 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:29 <lorbus> #topic roll call
16:30:33 <slowrie> .hello2
16:30:34 <zodbot> slowrie: slowrie 'Stephen Lowrie' <slowrie@redhat.com>
16:30:42 <rfairley> .hello rfairleyredhat
16:30:43 <zodbot> rfairley: rfairleyredhat 'Robert Fairley' <rfairley@redhat.com>
16:30:55 <miabbott> .hello miabbott
16:30:57 <zodbot> miabbott: miabbott 'Micah Abbott' <miabbott@redhat.com>
16:31:01 <kaeso> .hello lucab
16:31:02 <zodbot> kaeso: lucab 'Luca Bruno' <lucab@redhat.com>
16:31:04 <geoff-> geoff-: Geoff Levand <geoff@infradead.org>
16:31:05 <ajeddeloh> .hello2
16:31:06 <zodbot> ajeddeloh: ajeddeloh 'Andrew Jeddeloh' <andrew.jeddeloh@redhat.com>
16:31:14 <ksinny> .hello sinnykumari
16:31:15 <zodbot> ksinny: sinnykumari 'Sinny Kumari' <ksinny@gmail.com>
16:31:43 <lorbus> #chair slowrie rfairleyredhat miabbott lucab geoff- ajeddeloh ksinny
16:31:43 <zodbot> Current chairs: ajeddeloh geoff- ksinny lorbus lucab miabbott rfairleyredhat slowrie
16:33:48 <lorbus> #topic Action items from last meeting
16:33:59 <lorbus> * lorbus to review gce-oslogin rpm
16:33:59 <lorbus> https://pagure.io/fedora-server/issue/5#comment-538460
16:33:59 <lorbus> * ajeddeloh and lorbus to experiment on grubenv + static grub config
16:33:59 <lorbus> * kumarmn to open a ticket representing ppc64le multi-arch interests in
16:33:59 <lorbus> Fedora CoreOS
16:34:00 <lorbus> * mskarbek to follow up in #76 with more information on suggested
16:34:02 <lorbus> configuration changes for the moby-engine package
16:34:04 <lorbus> * bgilbert to followup on DHCP + private IP addresses problem in Digital
16:34:06 <lorbus> Ocean card
16:34:25 <mnguyen_> .hello mnguyen
16:34:26 <zodbot> mnguyen_: mnguyen 'Michael Nguyen' <mnguyen@redhat.com>
16:34:39 <bgilbert> .hello2
16:34:40 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:34:53 <lorbus> #chair mnguyen bgilbert
16:34:53 <zodbot> Current chairs: ajeddeloh bgilbert geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat slowrie
16:35:15 <ajeddeloh> I think I'm pretty close to getting an image built without using anaconda which would let us install custom grub configs
16:35:38 <lorbus> ajeddeloh++ nice
16:35:38 <zodbot> lorbus: Karma for ajeddeloh changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:35:51 <jbrooks> .fas jasonbrooks
16:35:52 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:36:18 <sanja> .hello2
16:36:19 <zodbot> sanja: sanja 'Sanja Bonic' <sanja@redhat.com>
16:36:32 <ksinny> I think opening ticket for ppc64le on multiarch was done in ticket https://github.com/coreos/fedora-coreos-tracker/issues/78
16:36:32 <kaeso> ajeddeloh: is that a supermin with gpt+mkfs+ostree broadly speaking?
16:36:35 <lorbus> #chair sanja
16:36:35 <zodbot> Current chairs: ajeddeloh bgilbert geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja slowrie
16:36:40 <sanja> ./me waves
16:36:47 <ajeddeloh> no supermin yet, but that's the goal
16:36:50 <kaeso> sanja: o/
16:36:52 <lorbus> jbrooks you want a chair as well?
16:36:56 <lorbus> hey sanja!!
16:37:34 <ajeddeloh> I want to do something like how we do compose where if you have root it uses loop devices and skips the supermin but, but if you dont it does
16:37:43 * dustymabe joins late
16:37:45 <dustymabe> .hello2
16:37:48 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:38:04 <lorbus> #info gce-oslogin rpm review has been picked up by a community member
16:38:06 <sayan> .hello sayanchowdhury
16:38:06 <bgilbert> dustymabe: hello2!
16:38:06 <zodbot> sayan: sayanchowdhury 'Sayan Chowdhury' <sayan.chowdhury2012@gmail.com>
16:38:12 <lorbus> #chair dustymabe
16:38:12 <zodbot> Current chairs: ajeddeloh bgilbert dustymabe geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja slowrie
16:38:24 <lorbus> #chair sayanchowdhury
16:38:24 <zodbot> Current chairs: ajeddeloh bgilbert dustymabe geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja sayanchowdhury slowrie
16:38:37 <dustymabe> ok are we in action items ?
16:38:46 <kaeso> yep
16:39:13 <dustymabe> #info mskarbek did follow up on #76 with suggested configuration changes
16:39:14 <lorbus> #info ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs
16:39:20 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/76#issuecomment-437991748
16:39:49 <lorbus> ahh we messed up the order now :P
16:40:05 <dustymabe> lorbus: you can undo :)
16:40:12 <lorbus> #undo
16:40:12 <zodbot> Removing item from minutes: <MeetBot.items.Link object at 0x7fe503192c90>
16:40:16 <lorbus> #undo
16:40:16 <zodbot> Removing item from minutes: INFO by lorbus at 16:39:14 : ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs
16:40:23 <dustymabe> #undo
16:40:23 <zodbot> Removing item from minutes: INFO by dustymabe at 16:39:13 : mskarbek did follow up on #76 with suggested configuration changes
16:40:28 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/76#issuecomment-437991748
16:40:42 <dustymabe> #info  mskarbek did follow up on #76 with suggested configuration changes
16:40:51 <lorbus> #info ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs
16:41:34 <lorbus> bgilbert do you have an update in the Digital Ocean part?
16:42:00 <bgilbert> whoops, which part?
16:42:22 <bgilbert> ah, yeah
16:42:25 <lorbus> * bgilbert to followup on DHCP + private IP addresses problem in Digital Ocean card
16:42:27 <bgilbert> done
16:42:38 <bgilbert> #link https://github.com/coreos/fedora-coreos-tracker/issues/71#issuecomment-436704433
16:42:49 <bgilbert> no response as of yet
16:43:45 <lorbus> should I re-action this then?
16:44:19 <dustymabe> probably not
16:44:21 <bgilbert> no, I think it's waiting for external feedback at this point
16:44:23 <bgilbert> ^
16:44:25 <dustymabe> +1
16:44:32 <lorbus> +1
16:45:28 <lorbus> #info external feedback is required for the Digital Ocean card
16:46:00 <lorbus> we had two more action items last time
16:46:22 <lorbus> kumarmn, mskarbek aren't here though, are they?
16:46:24 <dustymabe> i can handle one of those
16:46:37 <dustymabe> well actually I already handled one of them with my #info above
16:46:42 <dustymabe> i can handle the other one too
16:47:18 <dustymabe> #info kumarmn opened #78 to document and track down ppc64le multi-arch issues for FCOS
16:47:23 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/78
16:47:44 <dustymabe> I see ksinny mentioned that earlier, thanks ksinny
16:48:28 <lorbus> ok, great! should we move to current meeting topics?
16:48:48 <lorbus> #topic Throttled update rollouts
16:48:56 <lorbus> #link https://github.com/coreos/fedora-coreos-tracker/issues/83
16:49:12 <bgilbert> in some sense, this is the other half of the update streams discussion
16:49:30 <bgilbert> i.e., now that we have a set of streams, how do we roll out updates
16:49:48 <bgilbert> what we don't want to do is just update an rpm-ostree ref.
16:50:06 <bgilbert> since FCOS hosts are supposed to essentially run themselves,
16:50:34 <bgilbert> and since it's impossible to prevent all regressions,
16:50:43 <dustymabe> bgilbert: nice writeup. I'll be honest I haven't had a chance to fully read it yet, but promise I will today
16:50:45 <bgilbert> we need to be able to stop a bad update before we break everybody.
16:50:57 <ajeddeloh> I mean we would still need to update a ref, its just the hosts can't all learn about it at once
16:51:13 <bgilbert> ajeddeloh: well, sure
16:51:22 <bgilbert> ...which implies a gradual controlled rollout, as we do with Container Linux.
16:51:38 <bgilbert> but the tooling will need to be new, because reusing the CL infrastructure is... not a great idea.
16:52:01 <ajeddeloh> yeah, I guess one difference would be that a host could manually force an update, which they can't on CL
16:52:10 <bgilbert> ajeddeloh: sure they can
16:52:14 <ajeddeloh> (if we've paused)
16:52:18 <bgilbert> ah, right
16:52:33 <bgilbert> there's some update tooling being built for OpenShift that we can hopefully reuse, at least in part
16:52:39 <ajeddeloh> whether that's a good idea to re-implement I don't know
16:52:41 <bgilbert> #link https://github.com/openshift/cincinnati
16:53:02 <ajeddeloh> for those not in on the joke, cincinnati replaces omaha
16:53:07 <bgilbert> but we'll need to implement FCOS-specific server bits, plus the client bits.
16:53:46 <kaeso> I did a first pass on cincinnati lib to remove some openshift assumptions here: https://github.com/openshift/cincinnati/pull/7
16:54:04 <bgilbert> unlike with Container Linux, the update server will be public, with the expectation that anyone can easily run it for their cluster if they want.
16:54:05 <kaeso> I haven't looked into the graph and policy parts yet though
16:54:12 <dustymabe> thanks kaeso
16:54:18 <bgilbert> kaeso: nice!
16:54:42 <lorbus> #info kaseo did a first pass on cincinnati lib to remove some openshift assumptions
16:54:48 <lorbus> #link https://github.com/openshift/cincinnati/pull/7
16:55:12 <kaeso> what's not clear to me right now is from where we discover new refs
16:55:25 <dustymabe> kaeso: define "discover"
16:55:35 <kaeso> in openshift they currently do that by scanning docker registry
16:55:58 <dustymabe> kaeso: we'll have an ostree repo for now
16:56:15 <bgilbert> there will also be some sort of release metadata repository
16:56:17 <bgilbert> with things like AMI IDs
16:56:30 <bgilbert> ...if we don't have a bug for that, I should file one
16:56:37 <dustymabe> bgilbert: actually it would be cool if that could be embedded in the ostree repo
16:56:38 <kaeso> dustymabe: as in, there is a new release "foo" that corresponds to ostree ref "x" and can be reached from previous releases "bar" and "qux"
16:56:56 <dustymabe> bgilbert: i'll try to find the discussion and link you to it
16:57:31 <bgilbert> kaeso: my thinking was that every release can be reached from every other release, absent an explicit update barrier (which would be in metadata)
16:57:55 <kaeso> bgilbert: fair
16:58:14 <kaeso> bgilbert: there is still how to discover new release and map them to refs
16:58:31 <bgilbert> kaeso: yeah, was thinking release metadata, however that's implemented
16:58:42 <kaeso> walking a metadata repo/bucket seems fine
16:58:58 <dustymabe> kaeso: what we do today is just manipulate the refs in the ostree repo - we can add a thin layer on top of that and perform throttling etc..
16:59:11 <dustymabe> i'd say we just need to set up some time for a brainstorming session on the topic
16:59:26 <lorbus> action anything?
16:59:34 <kaeso> ack
17:00:15 <dustymabe> lorbus: probably action us getting together and discussing some mechanics and updating #83
17:00:57 <ajeddeloh> sgtm
17:00:57 <dustymabe> though it probably won't happen before next wednesday (we should also check who all is going to be around next wednesday and consider cancelling the meeting)
17:01:06 * ajeddeloh wont be here next week
17:01:19 * lorbus will be here
17:01:58 <lorbus> then we can probably ommit an action item and just put it on the agenda again next time?
17:02:28 <dustymabe> sure
17:02:58 <lorbus> ok, anybody want to add anything here? otherwise let's move on
17:03:13 <lorbus> #topic Version numbering for OS releases
17:03:23 <lorbus> #link https://github.com/coreos/fedora-coreos-tracker/issues/81
17:03:49 <bgilbert> #info BIKESHED
17:03:52 <bgilbert> #undo
17:03:52 <zodbot> Removing item from minutes: INFO by bgilbert at 17:03:49 : BIKESHED
17:04:02 <ajeddeloh> bgilbert++
17:04:02 <zodbot> ajeddeloh: Karma for bgilbert changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:04:05 <kaeso> bikeshed <3
17:04:39 <dustymabe> haha
17:04:43 <kaeso> (I already did my part there)
17:04:50 <bgilbert> I'm interested in input here
17:04:58 <bgilbert> though maybe not an enormous unending discussion :-)
17:05:12 <dustymabe> can you define "patch release" ?
17:05:39 <bgilbert> any followup release that doesn't involve a fresh snapshot
17:06:05 <bgilbert> i.e., maybe a few packages change for specific reasons
17:06:14 <ajeddeloh> on CL we've typically used them for "we messed up the build in some way and need to re-tag"
17:06:15 <bgilbert> but not routine version bumps
17:06:28 <dustymabe> zeroing in on proposal 2, for now
17:06:30 <bgilbert> ajeddeloh: yeah, I was thinking of the equivalent of CL minor versions
17:06:48 <ajeddeloh> oh gotcha
17:06:50 <dustymabe> so today in Atomic Host we have something like proposal 2
17:06:51 <bgilbert> ajeddeloh: because the proposal uses the major version for the Fedora release
17:07:05 <dustymabe> i.e. 31.20191015.0
17:07:16 <ajeddeloh> I agree with kaeso though, regardless of the numbering having some indicator of the stream would be nice
17:07:19 <kaeso> ajeddeloh: *cough*grub*cough*
17:07:34 <dustymabe> but the .0 at the end indicates the number of builds for that day
17:07:44 <dustymabe> .1 would indicate the 2nd run for the day
17:07:50 <dustymabe> etc..
17:08:02 <ajeddeloh> kaeso: oh god, giving me flashbacks to "perturb until working"
17:08:29 <dustymabe> ajeddeloh: it's similar to git. the "branch" you are following can be seen by looking at the ref
17:08:56 <ajeddeloh> ah right, that works then
17:09:09 <bgilbert> ajeddeloh++
17:09:09 <zodbot> bgilbert: Karma for ajeddeloh changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:09:36 <bgilbert> dustymabe: the ref is external state, though
17:09:40 <dustymabe> bgilbert: so I like proposal 2, just need to think through the mechanics of what <patch release> means and how we would go about doing it
17:10:07 <dustymabe> also interested in how we version a multiple builds happening on the same day
17:10:10 <bgilbert> dustymabe: IOW, yeah, you can look somewhere on the system and see what ref you're on, but it's separate from the name people will actually talk about
17:10:15 <bgilbert> dustymabe: which could be good or bad
17:10:33 <dustymabe> bgilbert: yep. so this can work to your advantage or against you
17:10:50 <dustymabe> for Atomic Host what we do is we only actually have two "branches" that get built against
17:10:58 <dustymabe> 1 - updates-testing
17:11:00 <dustymabe> 2 - updates
17:11:29 <dustymabe> when we do a "release" we just update the stable ref that we have to point to a specific commit in the updates stream
17:11:54 <dustymabe> so in that case, not embedding the "stream info" in the version, works to our advantage
17:12:08 <dustymabe> but it all depends on how we want to use it in the future
17:12:15 <dustymabe> so i'm open to anything
17:12:40 <bgilbert> sure
17:12:46 <bgilbert> dustymabe: I added proposal 2 because there was a request for including dates, but I prefer proposal 1.  "thirty-one dot two thousand eighteen ten twenty-five dot three" is a lot more unwieldy than "thirty-one dot seven dot three".
17:13:27 <dustymabe> yep. I do like including dates :) - but that is one opinion
17:13:28 <bgilbert> admittedly I work on the OS, and thus talk about versions a lot :-)
17:13:55 <dustymabe> i'll ask these questions in the ticket, once I've thought through the "patch release" part
17:14:07 <bgilbert> dustymabe: anything I can clarify there?
17:14:23 <bgilbert> dustymabe: coming from the CL side, the "patch release" mechanic feels pretty straightforward to me
17:14:28 <bgilbert> (i.e. I'm used to it)
17:14:43 <dustymabe> bgilbert: i think it's mostly just working through an example
17:15:00 <dustymabe> i'll ask a question in the ticket
17:15:09 <dustymabe> so that we can move on to other meeting items
17:15:11 <lorbus> #action dustymabe to add questions on the ticket
17:15:18 <bgilbert> +1
17:15:23 <lorbus> #topic where to move ignition dracut modules source code
17:15:30 <lorbus> #link https://github.com/coreos/fedora-coreos-tracker/issues/16
17:16:04 <lorbus> I think there is two parts to this: under what org do we want this to live?
17:16:10 <lorbus> and what should the name of the repo be?
17:16:25 <lorbus> a few suggestions have been made on the ticket
17:16:55 <dustymabe> lorbus: thanks for adding the meeting label to that one
17:17:09 <dustymabe> ajeddeloh: I think I had an outstanding question to you in the ticket
17:17:25 <lorbus> dustymabe, thanks for the permissions :)
17:17:56 <dustymabe> lorbus: thanks for volunteering to run the meeting
17:18:04 * ajeddeloh looks
17:18:23 <bgilbert> I agree with the concern that a common repo may not even make sense
17:18:31 <lorbus> is the repo going to be distro-agnostic or specific to FCOS?
17:18:35 <lorbus> so no?
17:18:35 <ajeddeloh> yeah I think that's a valid test
17:18:48 <kaeso> (the easy part: under coreos org is fine)
17:19:00 <lorbus> kaeso +1
17:19:16 <ajeddeloh> I'm still skeptical that we can get one set of dracut modules to work across a bunch of different distros
17:19:25 <ajeddeloh> but yeah, coreos org is good
17:19:39 <dustymabe> yeah. I think that's reasonable skeptisism
17:19:45 <lorbus> #info repo to be moved to the coreos org
17:19:46 * dustymabe doesn't know how to spell that word ^^
17:20:03 <dustymabe> bgilbert: i think we might be able to do something similar to dracut modules upstream
17:20:26 <dustymabe> https://github.com/dracutdevs/dracut/
17:20:43 <dustymabe> i think most distros share those common modules and then I'm sure they tweak them for their needs
17:21:13 <dustymabe> so if this endeavor proves complicated we can have a "common ignition" repo and then fork/tweak for each distro
17:21:17 <bgilbert> ¯\_(ツ)_/¯
17:21:23 <dustymabe> either way, future bridges to cross ?
17:21:26 <bgilbert> sure
17:21:37 <dustymabe> cool
17:21:55 <dustymabe> maybe one day my optimism will wear off
17:22:11 <kaeso> dustymabe: I already had this discussion with haraldh and we kinda agreed to disagree
17:22:26 <kaeso> (re CL dracut modules and the upstream repo)
17:22:29 <ksinny> Good to keep it with wherever ignition repo lives
17:23:23 <dustymabe> next topic?
17:23:35 <lorbus> so what's the name for the repo going to be?
17:23:55 <dustymabe> github.com/coreos/ignition-dracut ?
17:23:56 <lorbus> ccoreos/ignition-dracut-modules?
17:24:04 <dustymabe> that works too
17:24:13 <ajeddeloh> I'd prefer the first
17:24:20 <dustymabe> k
17:24:22 <lorbus> I wouldn't mind the shorter one either
17:24:23 <ajeddeloh> modules is somewhat redundant
17:24:41 <lorbus> #undo
17:24:41 <zodbot> Removing item from minutes: <MeetBot.items.Link object at 0x7fe5062b2f50>
17:24:53 <lorbus> #info repo to move to github.com/coreos/ignition-dracut
17:25:12 <bgilbert> +1
17:25:18 <lorbus> #topic open floor
17:25:37 <kaeso> #link https://developer.gnome.org/NetworkManager/stable/nm-initrd-generator.html
17:26:02 <kaeso> this is recent in NM-land and targeted at F30
17:26:16 <dustymabe> thanks for the link
17:26:31 <ajeddeloh> useful
17:26:44 <bgilbert> just had an update in the ticket: DigitalOcean still uses the metadata service for setting up private IPs.
17:27:08 * dustymabe may have poked ryan for that info
17:27:20 <kaeso> I've some more items for a later time on NM, I'm waiting for some references/links
17:27:59 <kaeso> in general NM is moving toward networkd direction
17:28:14 <bgilbert> dustymabe++
17:28:14 <zodbot> bgilbert: Karma for dustymabe changed to 4 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:28:22 <lorbus> kaeso: what exactly does that mean?
17:28:23 <ajeddeloh> kaeso: what do you mean by that specifically
17:28:33 <kaeso> usr-run-etc split
17:28:47 <bgilbert> nice
17:28:55 <kaeso> non-persistent runtime profiles
17:29:02 <kaeso> and initramfs integration
17:29:17 <ajeddeloh> cool
17:29:25 <dustymabe> kaeso++ thanks for helping us run some of those issues to ground
17:29:25 <zodbot> dustymabe: Karma for lucab changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:29:29 <lorbus> nice
17:29:41 <kaeso> (for what we are concerned, tons of other differences standing of course)
17:30:09 <lorbus> do we want to info a short summary here?
17:30:43 <lorbus> otherwise let's end the meeting :)
17:30:54 <lorbus> thanks for coming everyone!
17:30:55 <kaeso> lorbus: I'll update the ticket on -tracker as soon as I get all the references I asked to NM folks
17:31:03 <dustymabe> lorbus: thanks for running
17:31:04 <ajeddeloh> kaeso++
17:31:04 <zodbot> ajeddeloh: Karma for lucab changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:31:09 <ajeddeloh> lorbus++
17:31:12 <zodbot> ajeddeloh: Karma for lorbus changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:31:13 <lorbus> #endmeeting