16:30:04 <kaeso> #startmeeting fedora_coreos_meeting
16:30:04 <zodbot> Meeting started Wed Oct 17 16:30:04 2018 UTC.
16:30:04 <zodbot> This meeting is logged and archived in a public location.
16:30:04 <zodbot> The chair is kaeso. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:04 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:04 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:15 <kaeso> .hello lucab
16:30:15 <zodbot> kaeso: lucab 'Luca Bruno' <lucab@redhat.com>
16:30:30 <kaeso> #topic roll call
16:30:44 <jligon> .hello
16:30:44 <zodbot> jligon: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
16:30:46 <miabbott> .hellomynameis miabbott
16:30:48 <zodbot> miabbott: miabbott 'Micah Abbott' <miabbott@redhat.com>
16:30:50 <dustymabe> .hello2
16:30:51 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:30:56 <jlebon> .hello2
16:30:56 <ajeddeloh> .hello2
16:30:56 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:30:59 <zodbot> ajeddeloh: ajeddeloh 'Andrew Jeddeloh' <andrew.jeddeloh@redhat.com>
16:31:04 <jligon> .hello2
16:31:05 <zodbot> jligon: jligon 'Jeff Ligon' <jligon@redhat.com>
16:31:30 <jlebon> thankfully, my middle name actually *is* "None"
16:31:42 <bgilbert[1]> .hello bgilbert
16:31:43 <zodbot> bgilbert[1]: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:32:09 <jligon> lol
16:32:35 <kaeso> #chair jligon miabbott jlebon ajeddeloh bgilbert[1] dustymabe
16:32:35 <zodbot> Current chairs: ajeddeloh bgilbert[1] dustymabe jlebon jligon kaeso miabbott
16:33:06 * dustymabe added a few more topics with meeting labels
16:33:16 <jligon> offtopic: why is bgilbert[1] not zero indexed?
16:33:19 <kaeso> miabbott: https://github.com/fedora-infra/supybot-fedora/blob/173bbd8a3ac4cb672b11b2c9c6aba4f4501e5ab6/plugin.py#L502
16:33:26 <dustymabe> haha.. I think he is using matrix ??
16:33:29 <miabbott> yeah, i was just hunting for that
16:33:39 <kaeso> jligon: today is the evil twin
16:33:40 <bgilbert[1]> jligon: bgilbert[0] == *bgilbert
16:33:49 <dustymabe> haha
16:33:54 <jligon> we need a goatee index
16:34:22 <dustymabe> but does bgilbert[1] == *(bgilbert + 1) ?
16:34:52 <jligon> been too long since i did pointer arithmetic
16:34:55 <kaeso> ok, I'd say let's start
16:34:59 <bhavin192> .hello2
16:35:00 <zodbot> bhavin192: bhavin192 'Bhavin Gandhi' <bhavin7392@gmail.com>
16:35:01 <jligon> +1
16:35:11 <kaeso> #topic Action items from last meeting
16:35:17 <ajeddeloh> nah, thats how we get a grub bug, he really wanted *((*(char)bgilbert)+1)
16:35:17 <geoff-> geoff-: Geoff Levand <geoff@infradead.org>
16:35:32 <kaeso> last meeting -> https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-06-27-16.30.html
16:35:42 <ajeddeloh> err *((char*)bgilbert + 1)
16:35:51 <kaeso> according to that, no action time from last time
16:35:54 <slowrie> .hello2
16:35:55 <zodbot> slowrie: slowrie 'Stephen Lowrie' <slowrie@redhat.com>
16:35:57 <kaeso> *items
16:35:59 <bgilbert[1]> dustymabe ajeddeloh: indeed
16:36:01 <dustymabe> HMM
16:36:11 <mnguyen_> .hello mnguyen
16:36:16 <walters> .hello2
16:36:17 <dustymabe> kaeso: that link looks old
16:36:22 <zodbot> mnguyen_: mnguyen 'Michael Nguyen' <mnguyen@redhat.com>
16:36:29 <zodbot> walters: walters 'Colin Walters' <walters@redhat.com>
16:36:31 <miabbott> kaeso: that was from june...look here https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-10-10-16.30.txt
16:36:44 <kaeso> indeed, sorry
16:36:49 <kaeso> (wrong tab)
16:36:55 <kaeso> https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-10-10-16.30.html
16:36:55 <miabbott> np, we're all here to help  :)
16:37:13 <dustymabe> so:
16:37:22 <dustymabe> oops, wrong channel
16:37:39 <miabbott> #chair bhavin192 geoff- slowrie mnguyen_ walters
16:37:39 <zodbot> Current chairs: ajeddeloh bgilbert[1] bhavin192 dustymabe geoff- jlebon jligon kaeso miabbott mnguyen_ slowrie walters
16:37:58 <kaeso> dustymabe: first item was your, re. legal and channel logging
16:38:05 <dustymabe> yep
16:38:17 <dustymabe> * dustymabe  start an email thread with bgilbert + ajeddeloh and legal
16:38:19 <dustymabe> about logging #fedora-coreos in fedora infra or otherwise
16:38:43 <dustymabe> #info I started an email to legal list and got back some promising news. https://github.com/coreos/fedora-coreos-tracker/issues/11#issuecomment-429329611
16:39:27 <dustymabe> hopefully we don't hear any other dissenting opinions and we can move forward with trying to get fedora infra to set us up an instance of botbot.me
16:39:45 <kaeso> dustymabe: so for that one we wait on https://pagure.io/fedora-infrastructure/issue/7306 right?
16:40:01 <ligon> internet flaking today
16:40:02 <dustymabe> if fedora infra says `can you provide us a config for running botbot.me in openshift` is there anyone who could work on that ?
16:40:12 <dustymabe> kaeso: right
16:40:21 <dustymabe> kaeso: I don't expect much until f29 is out the door
16:41:13 <dustymabe> i guess we can cross that bridge when we get there
16:41:18 <kaeso> ack
16:42:14 <kaeso> bgilbert[1]: the other action item was yours
16:42:21 <kaeso> bgilbert[1]: I think that is https://github.com/coreos/fedora-coreos-tracker/issues/22#issuecomment-430513322
16:42:47 <bgilbert[1]> yup
16:42:48 <kaeso> bgilbert[1]: anything else to note on that?
16:42:54 <bgilbert[1]> nope
16:43:30 <kaeso> that same ticket is also marked for meeting discussion
16:43:57 <dustymabe> yeah we can start that topic now ?
16:44:24 <kaeso> #topic Major release and update cycle for Fedora CoreOS
16:44:54 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/22
16:44:56 <bgilbert[1]> I've updated the proposal at v
16:44:56 <bgilbert[1]> https://github.com/coreos/fedora-coreos-tracker/issues/22#issuecomment-427218716
16:45:03 <dustymabe> thanks bgilbert[1] !
16:45:16 <bgilbert[1]> in that version the `testing` channel is still called `testing`
16:45:27 <dustymabe> yeah I think that is fine
16:45:30 <bgilbert[1]> since we didn't come up with a name that everyone was happier with
16:45:57 <dustymabe> yeah to avoid the bikeshed and now that updates-testing is clarified as bodhi-updates-testing, i think we are good
16:46:09 <dustymabe> so remaining items I see here are:
16:46:09 <bgilbert[1]> I'm interested in any additional comments on the proposal.  I think the next step is to PR the design doc with that text
16:46:11 <kaeso> do we have sanything else controversial to decide there?
16:46:36 <dustymabe> kaeso: I don't think so. I'm sure we'll make tweaks as we get closer to implementation if we need to
16:46:47 <dustymabe> but for now I think this is a solid plan
16:47:04 <kaeso> let's go for a PR before next meeting and we can comment directly on that then?
16:47:10 <dustymabe> +1
16:47:19 <bgilbert[1]> dustymabe: what were the remaining items?
16:47:22 <dustymabe> I think a PR on the design doc is a good thing to have
16:47:37 <bgilbert[1]> I'll also note that the proposal completely ignores any interactions with the update mechanism for now
16:47:48 <lorbus> .hello2
16:47:49 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:47:57 <dustymabe> bgilbert[1]: the only other thing I think is to socialize the "possible backporting" of kernel fixes with the kernel team
16:47:57 <bgilbert[1]> (for centrally controlled gradual rollouts)
16:48:08 * lorbus is sorry he's late
16:48:13 * dustymabe waves at lorbus
16:48:18 <dustymabe> we assigned you all the action items!
16:48:31 <lorbus> yay! :D
16:48:36 <bgilbert[1]> dustymabe: sure
16:48:48 <bgilbert[1]> dustymabe: primarily at this point I think the important thing is to have a mechanism for shipping variant packages
16:48:57 <dustymabe> bgilbert[1]: indeed
16:49:14 <dustymabe> maybe we should open an issue specifically for that
16:49:19 <ajeddeloh> dustymabe: +
16:49:31 <dustymabe> one way I can think of is that we have our own koji tag we pull from and control
16:49:50 <dustymabe> the other is if we want to build different versions of packages then we'll need to be able to have a branch in git for that
16:50:15 <dustymabe> but I think that is good to bring up in another ticket
16:50:24 <bgilbert[1]> dustymabe: ...wouldn't we need three koji tags?
16:50:46 <dustymabe> yeah if we wanted to control each stream independently
16:50:51 <walters> personally i'd like to have a not-koji-dist-git buildsystem
16:51:11 <dustymabe> there are positives and negatives to that
16:51:27 <dustymabe> walters: maybe we can add arguments to the new ticket ?
16:51:41 <bgilbert[1]> +1
16:51:42 <kaeso> bgilbert[1]: would you copy into a PR and open tickets for the remaining TODO details?
16:51:45 <ajeddeloh> +1 to moving discussion to a ticket
16:52:45 <bgilbert[1]> #action bgilbert to PR the design doc and open tickets for backport-related questions
16:53:07 <kaeso> ack
16:53:14 <kaeso> let's proceed
16:53:27 <kaeso> #topic Determine how to handle automatic rollback
16:54:01 <kaeso> for that, there has been some offline discussion and a rough proposal
16:54:07 <kaeso> #link https://github.com/coreos/fedora-coreos-tracker/issues/47#issuecomment-430353782
16:54:28 <dustymabe> +1 thanks to ajeddeloh and lorbus for driving the discussion there
16:54:39 <lorbus> ajeddeloh++
16:54:39 <zodbot> lorbus: Karma for ajeddeloh changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
16:55:49 <kaeso> ajeddeloh, dustymabe: I think there was some mis-alignment on the definition/usage of slots
16:56:20 <dustymabe> yeah i was wondering the motivation specifically
16:56:30 <dustymabe> i.e. what is a slot and why is "3" the number
16:56:47 <dustymabe> ajeddeloh: explained that a slot is basically something that is defined in a "static" grub config
16:56:59 <dustymabe> and we can simply place files into those slots in order to use them
16:57:05 <dustymabe> which means we don't need to update the grub config
16:57:06 <ajeddeloh> yup
16:57:13 <lorbus> did we want to call them 1..3 or A..C?
16:57:19 <dustymabe> which makes sense if we want a static grub config
16:57:23 <kaeso> my understanding is that the slots are really just 2, the third one is to fake an atomic-swap
16:57:29 <ajeddeloh> yup
16:58:15 <dustymabe> yeah. I was trying to think of a scenario where if someone did use "pinning" to get more deployments we wouldn't error and still do the right thing
16:58:20 <ajeddeloh> I'd vote to not bikeshed the 1-3/A-C right now, but if I we're going to I'd vote A-C since they aren't ordered
16:58:29 <bgilbert[1]> I'm not sure I'm comfortable with treating package layering like an upgrade, since it'll clobber the previous OS release.  fixing that would require more slots
16:58:56 <dustymabe> bgilbert[1]: there will still be two deployments though
16:59:12 <lorbus> ajeddeloh: agreed
16:59:16 <dustymabe> one with the layered package (the candidate for next boot)  and one without (the known good bood)
16:59:26 <ajeddeloh> bgilbert[1]: what if we wait for successful boot to delete the 3rd slot
16:59:27 <dustymabe> s/bood/boot
16:59:27 <bgilbert[1]> dustymabe: right, but no previous release
16:59:51 <dustymabe> right, but do we care about that.. we've already booted (known good) and did an `rpm-ostree install`
16:59:56 <bgilbert[1]> ajeddeloh: not sure that changes anything?
17:00:12 <dustymabe> at the point we have a known good, do we care about going back?
17:00:16 <bgilbert[1]> dustymabe: yeah, I'm not sure.  how much do we trust that the "known good" OS release is actually good?
17:00:33 <dustymabe> bgilbert[1]: we have to rely on the checks we've defined, or the user has defined
17:00:38 <bgilbert[1]> reaching a systemd target is not always the same as working
17:00:59 <dustymabe> the other thing to note is that because ostree is like git for your OS, the user can theoretically `rpm-ostree deploy <previous commit>` and go back that way
17:01:25 <dustymabe> so there's an emergency hatch if we need it
17:01:46 <dustymabe> ORR we just say "we really care about this scenario" and we store N, N-1, and N-2
17:02:08 <bgilbert[1]> I only bring it up because increasing the slot count will be difficult after the fact
17:02:12 <dustymabe> there will be more requirements on disk space for the rootfs, but if we state that upfront then maybe it's OK
17:02:49 <bgilbert[1]> if no one else is especially concerned about it, I'm okay proceeding
17:03:18 <walters> one thing the Cockpit designers pushed hard for is `rpm-ostree deploy <version>`
17:03:35 <walters> the problem they had with rollback is its unpredictable nature
17:03:35 <kaeso> I think we are framing it as: 2(+1) auto-rolling deployments, N manually pinned/deployed deployment
17:04:13 <walters> ostree has a nice sophisticated versioning/history model...but as soon as package layering is involved it's messier since yum/rpm-md doesn't really have that
17:04:26 <dustymabe> kaeso: yeah, but bgilbert[1]'s concern was related to package layering and it taking up both deployments in that case
17:04:32 <bgilbert[1]> kaeso: if we have a static grub config, can there be any manual deployments?
17:04:49 <ajeddeloh> quick ostree question: the commit hash to deploy, thats on the kernel command line, yes?
17:04:50 <kaeso> dustymabe: yes, that's why I'm saying deployments, not releases
17:05:11 <dustymabe> kaeso: +1
17:05:14 <jlebon> is there a way to make the number of slots dynamic? e.g. 2 slots, but pinned deployments take up slots on the side
17:05:16 <ajeddeloh> or is that baked into the initramfs
17:05:37 <kaeso> bgilbert[1]: good question, I'm not familiar with that enough to answer
17:05:38 <walters> ajeddeloh: the kernel commandline points to a symlink that points to the deployment
17:06:10 <ajeddeloh> gotcha
17:06:13 <walters> (this is related to the optimization to only touch the bootloader config when necessary)
17:06:38 <jlebon> (also note, since it's super confusing, that it's not exactly the commit hash there, but a hash of the kernel+initrd)
17:06:45 <walters> another reason for this is that one can have the same commit deployed multiple times, so there's a "deployserial"
17:06:55 <walters> that's the .0
17:08:24 <kaeso> it looks like the slotting thing may benefit from a bit of experimenting with static grub config + grubenv first
17:08:44 <kaeso> while the releases vs deployments
17:08:47 <lorbus> kaeso +1
17:08:48 <ajeddeloh> re pinning: what if we require you layer in grub for pinning and if you pin you generate a combined grub config
17:08:51 <ajeddeloh> kaeso: ++
17:09:10 <dustymabe> yeah definitely need some research here
17:09:25 <dustymabe> i like the "ordered list" model we have today, but also like the idea of a static grub config
17:09:32 <kaeso> I think I agree with dustymabe on considering plain deployments, even it could mean "a single release with different package layering"
17:10:05 <kaeso> *even if
17:10:50 <kaeso> action ajeddeloh and lorbus to experiment on static + grubenv?
17:11:04 <lorbus> SGTM
17:11:44 <kaeso> #action ajeddeloh and lorbus to experiment on grubenv + static grub config
17:11:55 <kaeso> anything else on this?
17:12:16 <kaeso> nope
17:12:24 <kaeso> #topic Set up POC pipeline for Fedora CoreOS
17:12:32 <kaeso> #link https://github.com/coreos/fedora-coreos-tracker/issues/49
17:12:58 <ajeddeloh> dustymabe: +1 for setting that up!
17:13:11 <kaeso> dustymabe got the POC running, right?
17:13:39 <kaeso> (or at least I've seen a qcow this morning)
17:13:43 <dustymabe> yep
17:13:51 <lorbus> dustymabe++
17:13:53 <dustymabe> i'm about to push up some configs and docs
17:14:14 <dustymabe> to this repo
17:14:16 <dustymabe> https://github.com/dustymabe/fcos-ci
17:14:25 <dustymabe> so don't look at it yet, but maybe in an hour :)
17:14:43 <dustymabe> the question is pretty much.. where should we put these files?
17:15:14 <kaeso> dustymabe: is it the equivalent of https://github.com/coreos/jenkins-os?
17:15:19 <dustymabe> so maybe vote in the ticket
17:15:37 <dustymabe> kaeso: looks similar
17:15:51 <jlebon> kaeso: yeah, I think that's where it'll likely go
17:15:54 <jlebon> + openshift manifests
17:16:06 <dustymabe> although, up until this point I hadn't considered making it "user friedly"
17:16:18 <dustymabe> i.e. I was just thinking this would be where we store "our configs"
17:16:20 <jlebon> i'd agree that starting with a separate repo is likely a better idea because it's easier to merge later than to split later
17:16:23 <kaeso> IIRC geoff- mirrors the CL one on his own infra
17:16:37 <dustymabe> not necessarily "here's how you take this repo and make your own"
17:16:44 <jlebon> dustymabe: i think we should make it as `oc cluster up` friendly as we can
17:16:47 <dustymabe> but that's something we could consider
17:17:05 <dustymabe> jlebon: yeah i'm not opposed
17:17:18 <geoff-> Yes, I have the whole CI setup locally
17:17:20 <dustymabe> although I will say that we need /dev/kvm, and right now you need oci-kvm-hook rpm for that
17:17:30 <dustymabe> so we'll have to add that to requirements
17:17:40 <kaeso> dustymabe: it looks like there is a general agreement on dedicated repo + a repos list somewhere prominent
17:17:41 <dustymabe> either way.. we're making progress
17:17:50 <kaeso> dustymabe: fcos-ci sounds also ok
17:17:59 <kaeso> yay
17:17:59 <dustymabe> kaeso: regarding repos list.. i think fedora-coreos-tracker could be a good place for that?
17:18:04 <ajeddeloh> yup
17:18:25 <ajeddeloh> it's basically the entrypoint to FCOS dev so it makes sense to put it there
17:18:31 <dustymabe> kaeso: yeah I just called it fcos-ci because I needed somewhere to post up a file that could be retrieved from github
17:18:36 <kaeso> I know I always repeat myself, but at some we'll likely need a repo manager too
17:18:42 <dustymabe> maybe fedora-coreos-ci would be better
17:19:07 <dustymabe> to model the other repos we have
17:19:25 <dustymabe> will suggest in the ticket.. i think we can move forward to other topics
17:19:50 <kaeso> acl
17:19:52 <kaeso> *ack
17:19:56 <kaeso> last one
17:20:01 <kaeso> #topic meeting times and daylight savings
17:20:09 <kaeso> #link https://github.com/coreos/fedora-coreos-tracker/issues/57
17:20:56 <kaeso> (I'm actually checking when the DST switch is over here)
17:20:59 <ajeddeloh> so for those of us with DST, when we change time in a week or two, the meeting will move to an earlier hour, yes?
17:21:28 <slowrie> I'm fine with it either way personally
17:21:34 * ajeddeloh is bad at time math
17:21:40 <lorbus> daylight saving may be going away in the EU from next year
17:21:49 <ajeddeloh> same with CA
17:21:55 <ajeddeloh> prop 7
17:22:15 <kaeso> so, in EU on 28th Oct we go back an hour
17:22:34 <kaeso> I think in US the switch is on a different date
17:22:38 <ajeddeloh> I think US is early november?
17:22:54 <slowrie> Nov 5
17:23:07 <ajeddeloh> 4th
17:23:16 <ajeddeloh> 5th is a monday
17:23:28 <dustymabe> so all of these confusions are why sticking with UTC is helpful. there is no confusion about when the meeting is
17:23:30 <slowrie> ah whoops, I found the 2017 date
17:23:37 <dustymabe> it does suck because the meeting "moves"
17:23:56 <kaeso> dustymabe: I'd stuck with UTC for clarity too
17:23:57 <dustymabe> but each locale does their conversion independently and there is no confusion about when the meeting is or when it moves
17:24:04 <ajeddeloh> it moves to an earlier hour, yes?
17:24:18 <mnguyen_> i think later?
17:24:19 <kaeso> only question is if becomes too early for west coast
17:24:39 <ajeddeloh> it'd move to 8:30am west coast
17:24:42 <ajeddeloh> which is fine
17:24:57 <kaeso> ok
17:24:58 <geoff-> https://www.timeanddate.com/worldclock/meetingtime.html?month=11&day=14&year=2018&p1=283&p2=48&iv=0
17:25:12 <dustymabe> i honestly wish we could just stick to "summer time" year round
17:25:16 <dustymabe> but I know that's controversial
17:25:23 <dustymabe> depends on who you are
17:25:24 <lorbus> dustymabe +1
17:25:35 <ajeddeloh> yeah summertime ftw
17:25:50 <ajeddeloh> what time is the meeting currently at utc?
17:25:52 <dustymabe> so. summary.. maybe add your vote in the ticket? and we'll regroup next week
17:26:07 <dustymabe> ajeddeloh: 16:30
17:26:27 <ajeddeloh> yeah, it'd be 830, which is fine
17:26:41 <kaeso> dustymabe: I think you can directly skip to "keep 16:30 UTC"
17:26:51 <dustymabe> kaeso: :) ok
17:27:03 <dustymabe> I can add that the sentiment from the meeting was "keep 16:30 UTC"
17:27:13 <dustymabe> and we'll see if we get any comments from people who couldn't make it to the meeting
17:27:27 <kaeso> yup
17:27:28 <dustymabe> i don't see sinny/sayan around and I'd be interested in their thoughts
17:27:34 <dustymabe> although I don't think IST moves
17:27:39 <dustymabe> so would probably be better for them
17:28:03 <bhavin192> dustymabe, yep, IST doesn't change :)
17:28:09 <dustymabe> +1
17:28:35 <kaeso> ack
17:28:40 <kaeso> that was it I guess
17:28:42 <kaeso> #topic Open Floor
17:28:53 <kaeso> any other random items?
17:29:48 <kaeso> (come on, not even a funny trivia?)
17:29:59 <dustymabe> :)
17:30:17 <dustymabe> i noticed the greenboot RPM made it into Fedora!
17:30:21 <dustymabe> nice work lorbus !
17:30:39 <lorbus> I think its in "batched" mode :)
17:30:44 <lorbus> another week til stable
17:30:48 <dustymabe> will get there eventually
17:31:00 <kaeso> I'll cut this here, thanks everybody!
17:31:02 <kaeso> #endmeeting