16:30:22 #startmeeting fedora_coreos_meeting 16:30:22 Meeting started Wed Nov 14 16:30:22 2018 UTC. 16:30:22 This meeting is logged and archived in a public location. 16:30:22 The chair is lorbus. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:30:22 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:30:22 The meeting name has been set to 'fedora_coreos_meeting' 16:30:29 #topic roll call 16:30:33 .hello2 16:30:34 slowrie: slowrie 'Stephen Lowrie' 16:30:42 .hello rfairleyredhat 16:30:43 rfairley: rfairleyredhat 'Robert Fairley' 16:30:55 .hello miabbott 16:30:57 miabbott: miabbott 'Micah Abbott' 16:31:01 .hello lucab 16:31:02 kaeso: lucab 'Luca Bruno' 16:31:04 geoff-: Geoff Levand 16:31:05 .hello2 16:31:06 ajeddeloh: ajeddeloh 'Andrew Jeddeloh' 16:31:14 .hello sinnykumari 16:31:15 ksinny: sinnykumari 'Sinny Kumari' 16:31:43 #chair slowrie rfairleyredhat miabbott lucab geoff- ajeddeloh ksinny 16:31:43 Current chairs: ajeddeloh geoff- ksinny lorbus lucab miabbott rfairleyredhat slowrie 16:33:48 #topic Action items from last meeting 16:33:59 * lorbus to review gce-oslogin rpm 16:33:59 https://pagure.io/fedora-server/issue/5#comment-538460 16:33:59 * ajeddeloh and lorbus to experiment on grubenv + static grub config 16:33:59 * kumarmn to open a ticket representing ppc64le multi-arch interests in 16:33:59 Fedora CoreOS 16:34:00 * mskarbek to follow up in #76 with more information on suggested 16:34:02 configuration changes for the moby-engine package 16:34:04 * bgilbert to followup on DHCP + private IP addresses problem in Digital 16:34:06 Ocean card 16:34:25 .hello mnguyen 16:34:26 mnguyen_: mnguyen 'Michael Nguyen' 16:34:39 .hello2 16:34:40 bgilbert: bgilbert 'Benjamin Gilbert' 16:34:53 #chair mnguyen bgilbert 16:34:53 Current chairs: ajeddeloh bgilbert geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat slowrie 16:35:15 I think I'm pretty close to getting an image built without using anaconda which would let us install custom grub configs 16:35:38 ajeddeloh++ nice 16:35:38 lorbus: Karma for ajeddeloh changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:35:51 .fas jasonbrooks 16:35:52 jbrooks: jasonbrooks 'Jason Brooks' 16:36:18 .hello2 16:36:19 sanja: sanja 'Sanja Bonic' 16:36:32 I think opening ticket for ppc64le on multiarch was done in ticket https://github.com/coreos/fedora-coreos-tracker/issues/78 16:36:32 ajeddeloh: is that a supermin with gpt+mkfs+ostree broadly speaking? 16:36:35 #chair sanja 16:36:35 Current chairs: ajeddeloh bgilbert geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja slowrie 16:36:40 ./me waves 16:36:47 no supermin yet, but that's the goal 16:36:50 sanja: o/ 16:36:52 jbrooks you want a chair as well? 16:36:56 hey sanja!! 16:37:34 I want to do something like how we do compose where if you have root it uses loop devices and skips the supermin but, but if you dont it does 16:37:43 * dustymabe joins late 16:37:45 .hello2 16:37:48 dustymabe: dustymabe 'Dusty Mabe' 16:38:04 #info gce-oslogin rpm review has been picked up by a community member 16:38:06 .hello sayanchowdhury 16:38:06 dustymabe: hello2! 16:38:06 sayan: sayanchowdhury 'Sayan Chowdhury' 16:38:12 #chair dustymabe 16:38:12 Current chairs: ajeddeloh bgilbert dustymabe geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja slowrie 16:38:24 #chair sayanchowdhury 16:38:24 Current chairs: ajeddeloh bgilbert dustymabe geoff- ksinny lorbus lucab miabbott mnguyen rfairleyredhat sanja sayanchowdhury slowrie 16:38:37 ok are we in action items ? 16:38:46 yep 16:39:13 #info mskarbek did follow up on #76 with suggested configuration changes 16:39:14 #info ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs 16:39:20 #link https://github.com/coreos/fedora-coreos-tracker/issues/76#issuecomment-437991748 16:39:49 ahh we messed up the order now :P 16:40:05 lorbus: you can undo :) 16:40:12 #undo 16:40:12 Removing item from minutes: 16:40:16 #undo 16:40:16 Removing item from minutes: INFO by lorbus at 16:39:14 : ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs 16:40:23 #undo 16:40:23 Removing item from minutes: INFO by dustymabe at 16:39:13 : mskarbek did follow up on #76 with suggested configuration changes 16:40:28 #link https://github.com/coreos/fedora-coreos-tracker/issues/76#issuecomment-437991748 16:40:42 #info mskarbek did follow up on #76 with suggested configuration changes 16:40:51 #info ajeddeloh is close to getting an image built without using anaconda which would let us install custom grub configs 16:41:34 bgilbert do you have an update in the Digital Ocean part? 16:42:00 whoops, which part? 16:42:22 ah, yeah 16:42:25 * bgilbert to followup on DHCP + private IP addresses problem in Digital Ocean card 16:42:27 done 16:42:38 #link https://github.com/coreos/fedora-coreos-tracker/issues/71#issuecomment-436704433 16:42:49 no response as of yet 16:43:45 should I re-action this then? 16:44:19 probably not 16:44:21 no, I think it's waiting for external feedback at this point 16:44:23 ^ 16:44:25 +1 16:44:32 +1 16:45:28 #info external feedback is required for the Digital Ocean card 16:46:00 we had two more action items last time 16:46:22 kumarmn, mskarbek aren't here though, are they? 16:46:24 i can handle one of those 16:46:37 well actually I already handled one of them with my #info above 16:46:42 i can handle the other one too 16:47:18 #info kumarmn opened #78 to document and track down ppc64le multi-arch issues for FCOS 16:47:23 #link https://github.com/coreos/fedora-coreos-tracker/issues/78 16:47:44 I see ksinny mentioned that earlier, thanks ksinny 16:48:28 ok, great! should we move to current meeting topics? 16:48:48 #topic Throttled update rollouts 16:48:56 #link https://github.com/coreos/fedora-coreos-tracker/issues/83 16:49:12 in some sense, this is the other half of the update streams discussion 16:49:30 i.e., now that we have a set of streams, how do we roll out updates 16:49:48 what we don't want to do is just update an rpm-ostree ref. 16:50:06 since FCOS hosts are supposed to essentially run themselves, 16:50:34 and since it's impossible to prevent all regressions, 16:50:43 bgilbert: nice writeup. I'll be honest I haven't had a chance to fully read it yet, but promise I will today 16:50:45 we need to be able to stop a bad update before we break everybody. 16:50:57 I mean we would still need to update a ref, its just the hosts can't all learn about it at once 16:51:13 ajeddeloh: well, sure 16:51:22 ...which implies a gradual controlled rollout, as we do with Container Linux. 16:51:38 but the tooling will need to be new, because reusing the CL infrastructure is... not a great idea. 16:52:01 yeah, I guess one difference would be that a host could manually force an update, which they can't on CL 16:52:10 ajeddeloh: sure they can 16:52:14 (if we've paused) 16:52:18 ah, right 16:52:33 there's some update tooling being built for OpenShift that we can hopefully reuse, at least in part 16:52:39 whether that's a good idea to re-implement I don't know 16:52:41 #link https://github.com/openshift/cincinnati 16:53:02 for those not in on the joke, cincinnati replaces omaha 16:53:07 but we'll need to implement FCOS-specific server bits, plus the client bits. 16:53:46 I did a first pass on cincinnati lib to remove some openshift assumptions here: https://github.com/openshift/cincinnati/pull/7 16:54:04 unlike with Container Linux, the update server will be public, with the expectation that anyone can easily run it for their cluster if they want. 16:54:05 I haven't looked into the graph and policy parts yet though 16:54:12 thanks kaeso 16:54:18 kaeso: nice! 16:54:42 #info kaseo did a first pass on cincinnati lib to remove some openshift assumptions 16:54:48 #link https://github.com/openshift/cincinnati/pull/7 16:55:12 what's not clear to me right now is from where we discover new refs 16:55:25 kaeso: define "discover" 16:55:35 in openshift they currently do that by scanning docker registry 16:55:58 kaeso: we'll have an ostree repo for now 16:56:15 there will also be some sort of release metadata repository 16:56:17 with things like AMI IDs 16:56:30 ...if we don't have a bug for that, I should file one 16:56:37 bgilbert: actually it would be cool if that could be embedded in the ostree repo 16:56:38 dustymabe: as in, there is a new release "foo" that corresponds to ostree ref "x" and can be reached from previous releases "bar" and "qux" 16:56:56 bgilbert: i'll try to find the discussion and link you to it 16:57:31 kaeso: my thinking was that every release can be reached from every other release, absent an explicit update barrier (which would be in metadata) 16:57:55 bgilbert: fair 16:58:14 bgilbert: there is still how to discover new release and map them to refs 16:58:31 kaeso: yeah, was thinking release metadata, however that's implemented 16:58:42 walking a metadata repo/bucket seems fine 16:58:58 kaeso: what we do today is just manipulate the refs in the ostree repo - we can add a thin layer on top of that and perform throttling etc.. 16:59:11 i'd say we just need to set up some time for a brainstorming session on the topic 16:59:26 action anything? 16:59:34 ack 17:00:15 lorbus: probably action us getting together and discussing some mechanics and updating #83 17:00:57 sgtm 17:00:57 though it probably won't happen before next wednesday (we should also check who all is going to be around next wednesday and consider cancelling the meeting) 17:01:06 * ajeddeloh wont be here next week 17:01:19 * lorbus will be here 17:01:58 then we can probably ommit an action item and just put it on the agenda again next time? 17:02:28 sure 17:02:58 ok, anybody want to add anything here? otherwise let's move on 17:03:13 #topic Version numbering for OS releases 17:03:23 #link https://github.com/coreos/fedora-coreos-tracker/issues/81 17:03:49 #info BIKESHED 17:03:52 #undo 17:03:52 Removing item from minutes: INFO by bgilbert at 17:03:49 : BIKESHED 17:04:02 bgilbert++ 17:04:02 ajeddeloh: Karma for bgilbert changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:04:05 bikeshed <3 17:04:39 haha 17:04:43 (I already did my part there) 17:04:50 I'm interested in input here 17:04:58 though maybe not an enormous unending discussion :-) 17:05:12 can you define "patch release" ? 17:05:39 any followup release that doesn't involve a fresh snapshot 17:06:05 i.e., maybe a few packages change for specific reasons 17:06:14 on CL we've typically used them for "we messed up the build in some way and need to re-tag" 17:06:15 but not routine version bumps 17:06:28 zeroing in on proposal 2, for now 17:06:30 ajeddeloh: yeah, I was thinking of the equivalent of CL minor versions 17:06:48 oh gotcha 17:06:50 so today in Atomic Host we have something like proposal 2 17:06:51 ajeddeloh: because the proposal uses the major version for the Fedora release 17:07:05 i.e. 31.20191015.0 17:07:16 I agree with kaeso though, regardless of the numbering having some indicator of the stream would be nice 17:07:19 ajeddeloh: *cough*grub*cough* 17:07:34 but the .0 at the end indicates the number of builds for that day 17:07:44 .1 would indicate the 2nd run for the day 17:07:50 etc.. 17:08:02 kaeso: oh god, giving me flashbacks to "perturb until working" 17:08:29 ajeddeloh: it's similar to git. the "branch" you are following can be seen by looking at the ref 17:08:56 ah right, that works then 17:09:09 ajeddeloh++ 17:09:09 bgilbert: Karma for ajeddeloh changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:09:36 dustymabe: the ref is external state, though 17:09:40 bgilbert: so I like proposal 2, just need to think through the mechanics of what means and how we would go about doing it 17:10:07 also interested in how we version a multiple builds happening on the same day 17:10:10 dustymabe: IOW, yeah, you can look somewhere on the system and see what ref you're on, but it's separate from the name people will actually talk about 17:10:15 dustymabe: which could be good or bad 17:10:33 bgilbert: yep. so this can work to your advantage or against you 17:10:50 for Atomic Host what we do is we only actually have two "branches" that get built against 17:10:58 1 - updates-testing 17:11:00 2 - updates 17:11:29 when we do a "release" we just update the stable ref that we have to point to a specific commit in the updates stream 17:11:54 so in that case, not embedding the "stream info" in the version, works to our advantage 17:12:08 but it all depends on how we want to use it in the future 17:12:15 so i'm open to anything 17:12:40 sure 17:12:46 dustymabe: I added proposal 2 because there was a request for including dates, but I prefer proposal 1. "thirty-one dot two thousand eighteen ten twenty-five dot three" is a lot more unwieldy than "thirty-one dot seven dot three". 17:13:27 yep. I do like including dates :) - but that is one opinion 17:13:28 admittedly I work on the OS, and thus talk about versions a lot :-) 17:13:55 i'll ask these questions in the ticket, once I've thought through the "patch release" part 17:14:07 dustymabe: anything I can clarify there? 17:14:23 dustymabe: coming from the CL side, the "patch release" mechanic feels pretty straightforward to me 17:14:28 (i.e. I'm used to it) 17:14:43 bgilbert: i think it's mostly just working through an example 17:15:00 i'll ask a question in the ticket 17:15:09 so that we can move on to other meeting items 17:15:11 #action dustymabe to add questions on the ticket 17:15:18 +1 17:15:23 #topic where to move ignition dracut modules source code 17:15:30 #link https://github.com/coreos/fedora-coreos-tracker/issues/16 17:16:04 I think there is two parts to this: under what org do we want this to live? 17:16:10 and what should the name of the repo be? 17:16:25 a few suggestions have been made on the ticket 17:16:55 lorbus: thanks for adding the meeting label to that one 17:17:09 ajeddeloh: I think I had an outstanding question to you in the ticket 17:17:25 dustymabe, thanks for the permissions :) 17:17:56 lorbus: thanks for volunteering to run the meeting 17:18:04 * ajeddeloh looks 17:18:23 I agree with the concern that a common repo may not even make sense 17:18:31 is the repo going to be distro-agnostic or specific to FCOS? 17:18:35 so no? 17:18:35 yeah I think that's a valid test 17:18:48 (the easy part: under coreos org is fine) 17:19:00 kaeso +1 17:19:16 I'm still skeptical that we can get one set of dracut modules to work across a bunch of different distros 17:19:25 but yeah, coreos org is good 17:19:39 yeah. I think that's reasonable skeptisism 17:19:45 #info repo to be moved to the coreos org 17:19:46 * dustymabe doesn't know how to spell that word ^^ 17:20:03 bgilbert: i think we might be able to do something similar to dracut modules upstream 17:20:26 https://github.com/dracutdevs/dracut/ 17:20:43 i think most distros share those common modules and then I'm sure they tweak them for their needs 17:21:13 so if this endeavor proves complicated we can have a "common ignition" repo and then fork/tweak for each distro 17:21:17 ¯\_(ツ)_/¯ 17:21:23 either way, future bridges to cross ? 17:21:26 sure 17:21:37 cool 17:21:55 maybe one day my optimism will wear off 17:22:11 dustymabe: I already had this discussion with haraldh and we kinda agreed to disagree 17:22:26 (re CL dracut modules and the upstream repo) 17:22:29 Good to keep it with wherever ignition repo lives 17:23:23 next topic? 17:23:35 so what's the name for the repo going to be? 17:23:55 github.com/coreos/ignition-dracut ? 17:23:56 ccoreos/ignition-dracut-modules? 17:24:04 that works too 17:24:13 I'd prefer the first 17:24:20 k 17:24:22 I wouldn't mind the shorter one either 17:24:23 modules is somewhat redundant 17:24:41 #undo 17:24:41 Removing item from minutes: 17:24:53 #info repo to move to github.com/coreos/ignition-dracut 17:25:12 +1 17:25:18 #topic open floor 17:25:37 #link https://developer.gnome.org/NetworkManager/stable/nm-initrd-generator.html 17:26:02 this is recent in NM-land and targeted at F30 17:26:16 thanks for the link 17:26:31 useful 17:26:44 just had an update in the ticket: DigitalOcean still uses the metadata service for setting up private IPs. 17:27:08 * dustymabe may have poked ryan for that info 17:27:20 I've some more items for a later time on NM, I'm waiting for some references/links 17:27:59 in general NM is moving toward networkd direction 17:28:14 dustymabe++ 17:28:14 bgilbert: Karma for dustymabe changed to 4 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:28:22 kaeso: what exactly does that mean? 17:28:23 kaeso: what do you mean by that specifically 17:28:33 usr-run-etc split 17:28:47 nice 17:28:55 non-persistent runtime profiles 17:29:02 and initramfs integration 17:29:17 cool 17:29:25 kaeso++ thanks for helping us run some of those issues to ground 17:29:25 dustymabe: Karma for lucab changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:29:29 nice 17:29:41 (for what we are concerned, tons of other differences standing of course) 17:30:09 do we want to info a short summary here? 17:30:43 otherwise let's end the meeting :) 17:30:54 thanks for coming everyone! 17:30:55 lorbus: I'll update the ticket on -tracker as soon as I get all the references I asked to NM folks 17:31:03 lorbus: thanks for running 17:31:04 kaeso++ 17:31:04 ajeddeloh: Karma for lucab changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:31:09 lorbus++ 17:31:12 ajeddeloh: Karma for lorbus changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:31:13 #endmeeting