16:30:04 #startmeeting fedora_coreos_meeting 16:30:04 Meeting started Wed Oct 17 16:30:04 2018 UTC. 16:30:04 This meeting is logged and archived in a public location. 16:30:04 The chair is kaeso. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:30:04 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:30:04 The meeting name has been set to 'fedora_coreos_meeting' 16:30:15 .hello lucab 16:30:15 kaeso: lucab 'Luca Bruno' 16:30:30 #topic roll call 16:30:44 .hello 16:30:44 jligon: (hello ) -- Alias for "hellomynameis $1". 16:30:46 .hellomynameis miabbott 16:30:48 miabbott: miabbott 'Micah Abbott' 16:30:50 .hello2 16:30:51 dustymabe: dustymabe 'Dusty Mabe' 16:30:56 .hello2 16:30:56 .hello2 16:30:56 jlebon: jlebon 'None' 16:30:59 ajeddeloh: ajeddeloh 'Andrew Jeddeloh' 16:31:04 .hello2 16:31:05 jligon: jligon 'Jeff Ligon' 16:31:30 thankfully, my middle name actually *is* "None" 16:31:42 .hello bgilbert 16:31:43 bgilbert[1]: bgilbert 'Benjamin Gilbert' 16:32:09 lol 16:32:35 #chair jligon miabbott jlebon ajeddeloh bgilbert[1] dustymabe 16:32:35 Current chairs: ajeddeloh bgilbert[1] dustymabe jlebon jligon kaeso miabbott 16:33:06 * dustymabe added a few more topics with meeting labels 16:33:16 offtopic: why is bgilbert[1] not zero indexed? 16:33:19 miabbott: https://github.com/fedora-infra/supybot-fedora/blob/173bbd8a3ac4cb672b11b2c9c6aba4f4501e5ab6/plugin.py#L502 16:33:26 haha.. I think he is using matrix ?? 16:33:29 yeah, i was just hunting for that 16:33:39 jligon: today is the evil twin 16:33:40 jligon: bgilbert[0] == *bgilbert 16:33:49 haha 16:33:54 we need a goatee index 16:34:22 but does bgilbert[1] == *(bgilbert + 1) ? 16:34:52 been too long since i did pointer arithmetic 16:34:55 ok, I'd say let's start 16:34:59 .hello2 16:35:00 bhavin192: bhavin192 'Bhavin Gandhi' 16:35:01 +1 16:35:11 #topic Action items from last meeting 16:35:17 nah, thats how we get a grub bug, he really wanted *((*(char)bgilbert)+1) 16:35:17 geoff-: Geoff Levand 16:35:32 last meeting -> https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-06-27-16.30.html 16:35:42 err *((char*)bgilbert + 1) 16:35:51 according to that, no action time from last time 16:35:54 .hello2 16:35:55 slowrie: slowrie 'Stephen Lowrie' 16:35:57 *items 16:35:59 dustymabe ajeddeloh: indeed 16:36:01 HMM 16:36:11 .hello mnguyen 16:36:16 .hello2 16:36:17 kaeso: that link looks old 16:36:22 mnguyen_: mnguyen 'Michael Nguyen' 16:36:29 walters: walters 'Colin Walters' 16:36:31 kaeso: that was from june...look here https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-10-10-16.30.txt 16:36:44 indeed, sorry 16:36:49 (wrong tab) 16:36:55 https://meetbot-raw.fedoraproject.org/teams/fedora_coreos_meeting/fedora_coreos_meeting.2018-10-10-16.30.html 16:36:55 np, we're all here to help :) 16:37:13 so: 16:37:22 oops, wrong channel 16:37:39 #chair bhavin192 geoff- slowrie mnguyen_ walters 16:37:39 Current chairs: ajeddeloh bgilbert[1] bhavin192 dustymabe geoff- jlebon jligon kaeso miabbott mnguyen_ slowrie walters 16:37:58 dustymabe: first item was your, re. legal and channel logging 16:38:05 yep 16:38:17 * dustymabe start an email thread with bgilbert + ajeddeloh and legal 16:38:19 about logging #fedora-coreos in fedora infra or otherwise 16:38:43 #info I started an email to legal list and got back some promising news. https://github.com/coreos/fedora-coreos-tracker/issues/11#issuecomment-429329611 16:39:27 hopefully we don't hear any other dissenting opinions and we can move forward with trying to get fedora infra to set us up an instance of botbot.me 16:39:45 dustymabe: so for that one we wait on https://pagure.io/fedora-infrastructure/issue/7306 right? 16:40:01 internet flaking today 16:40:02 if fedora infra says `can you provide us a config for running botbot.me in openshift` is there anyone who could work on that ? 16:40:12 kaeso: right 16:40:21 kaeso: I don't expect much until f29 is out the door 16:41:13 i guess we can cross that bridge when we get there 16:41:18 ack 16:42:14 bgilbert[1]: the other action item was yours 16:42:21 bgilbert[1]: I think that is https://github.com/coreos/fedora-coreos-tracker/issues/22#issuecomment-430513322 16:42:47 yup 16:42:48 bgilbert[1]: anything else to note on that? 16:42:54 nope 16:43:30 that same ticket is also marked for meeting discussion 16:43:57 yeah we can start that topic now ? 16:44:24 #topic Major release and update cycle for Fedora CoreOS 16:44:54 #link https://github.com/coreos/fedora-coreos-tracker/issues/22 16:44:56 I've updated the proposal at v 16:44:56 https://github.com/coreos/fedora-coreos-tracker/issues/22#issuecomment-427218716 16:45:03 thanks bgilbert[1] ! 16:45:16 in that version the `testing` channel is still called `testing` 16:45:27 yeah I think that is fine 16:45:30 since we didn't come up with a name that everyone was happier with 16:45:57 yeah to avoid the bikeshed and now that updates-testing is clarified as bodhi-updates-testing, i think we are good 16:46:09 so remaining items I see here are: 16:46:09 I'm interested in any additional comments on the proposal. I think the next step is to PR the design doc with that text 16:46:11 do we have sanything else controversial to decide there? 16:46:36 kaeso: I don't think so. I'm sure we'll make tweaks as we get closer to implementation if we need to 16:46:47 but for now I think this is a solid plan 16:47:04 let's go for a PR before next meeting and we can comment directly on that then? 16:47:10 +1 16:47:19 dustymabe: what were the remaining items? 16:47:22 I think a PR on the design doc is a good thing to have 16:47:37 I'll also note that the proposal completely ignores any interactions with the update mechanism for now 16:47:48 .hello2 16:47:49 lorbus: lorbus 'Christian Glombek' 16:47:57 bgilbert[1]: the only other thing I think is to socialize the "possible backporting" of kernel fixes with the kernel team 16:47:57 (for centrally controlled gradual rollouts) 16:48:08 * lorbus is sorry he's late 16:48:13 * dustymabe waves at lorbus 16:48:18 we assigned you all the action items! 16:48:31 yay! :D 16:48:36 dustymabe: sure 16:48:48 dustymabe: primarily at this point I think the important thing is to have a mechanism for shipping variant packages 16:48:57 bgilbert[1]: indeed 16:49:14 maybe we should open an issue specifically for that 16:49:19 dustymabe: + 16:49:31 one way I can think of is that we have our own koji tag we pull from and control 16:49:50 the other is if we want to build different versions of packages then we'll need to be able to have a branch in git for that 16:50:15 but I think that is good to bring up in another ticket 16:50:24 dustymabe: ...wouldn't we need three koji tags? 16:50:46 yeah if we wanted to control each stream independently 16:50:51 personally i'd like to have a not-koji-dist-git buildsystem 16:51:11 there are positives and negatives to that 16:51:27 walters: maybe we can add arguments to the new ticket ? 16:51:41 +1 16:51:42 bgilbert[1]: would you copy into a PR and open tickets for the remaining TODO details? 16:51:45 +1 to moving discussion to a ticket 16:52:45 #action bgilbert to PR the design doc and open tickets for backport-related questions 16:53:07 ack 16:53:14 let's proceed 16:53:27 #topic Determine how to handle automatic rollback 16:54:01 for that, there has been some offline discussion and a rough proposal 16:54:07 #link https://github.com/coreos/fedora-coreos-tracker/issues/47#issuecomment-430353782 16:54:28 +1 thanks to ajeddeloh and lorbus for driving the discussion there 16:54:39 ajeddeloh++ 16:54:39 lorbus: Karma for ajeddeloh changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 16:55:49 ajeddeloh, dustymabe: I think there was some mis-alignment on the definition/usage of slots 16:56:20 yeah i was wondering the motivation specifically 16:56:30 i.e. what is a slot and why is "3" the number 16:56:47 ajeddeloh: explained that a slot is basically something that is defined in a "static" grub config 16:56:59 and we can simply place files into those slots in order to use them 16:57:05 which means we don't need to update the grub config 16:57:06 yup 16:57:13 did we want to call them 1..3 or A..C? 16:57:19 which makes sense if we want a static grub config 16:57:23 my understanding is that the slots are really just 2, the third one is to fake an atomic-swap 16:57:29 yup 16:58:15 yeah. I was trying to think of a scenario where if someone did use "pinning" to get more deployments we wouldn't error and still do the right thing 16:58:20 I'd vote to not bikeshed the 1-3/A-C right now, but if I we're going to I'd vote A-C since they aren't ordered 16:58:29 I'm not sure I'm comfortable with treating package layering like an upgrade, since it'll clobber the previous OS release. fixing that would require more slots 16:58:56 bgilbert[1]: there will still be two deployments though 16:59:12 ajeddeloh: agreed 16:59:16 one with the layered package (the candidate for next boot) and one without (the known good bood) 16:59:26 bgilbert[1]: what if we wait for successful boot to delete the 3rd slot 16:59:27 s/bood/boot 16:59:27 dustymabe: right, but no previous release 16:59:51 right, but do we care about that.. we've already booted (known good) and did an `rpm-ostree install` 16:59:56 ajeddeloh: not sure that changes anything? 17:00:12 at the point we have a known good, do we care about going back? 17:00:16 dustymabe: yeah, I'm not sure. how much do we trust that the "known good" OS release is actually good? 17:00:33 bgilbert[1]: we have to rely on the checks we've defined, or the user has defined 17:00:38 reaching a systemd target is not always the same as working 17:00:59 the other thing to note is that because ostree is like git for your OS, the user can theoretically `rpm-ostree deploy ` and go back that way 17:01:25 so there's an emergency hatch if we need it 17:01:46 ORR we just say "we really care about this scenario" and we store N, N-1, and N-2 17:02:08 I only bring it up because increasing the slot count will be difficult after the fact 17:02:12 there will be more requirements on disk space for the rootfs, but if we state that upfront then maybe it's OK 17:02:49 if no one else is especially concerned about it, I'm okay proceeding 17:03:18 one thing the Cockpit designers pushed hard for is `rpm-ostree deploy ` 17:03:35 the problem they had with rollback is its unpredictable nature 17:03:35 I think we are framing it as: 2(+1) auto-rolling deployments, N manually pinned/deployed deployment 17:04:13 ostree has a nice sophisticated versioning/history model...but as soon as package layering is involved it's messier since yum/rpm-md doesn't really have that 17:04:26 kaeso: yeah, but bgilbert[1]'s concern was related to package layering and it taking up both deployments in that case 17:04:32 kaeso: if we have a static grub config, can there be any manual deployments? 17:04:49 quick ostree question: the commit hash to deploy, thats on the kernel command line, yes? 17:04:50 dustymabe: yes, that's why I'm saying deployments, not releases 17:05:11 kaeso: +1 17:05:14 is there a way to make the number of slots dynamic? e.g. 2 slots, but pinned deployments take up slots on the side 17:05:16 or is that baked into the initramfs 17:05:37 bgilbert[1]: good question, I'm not familiar with that enough to answer 17:05:38 ajeddeloh: the kernel commandline points to a symlink that points to the deployment 17:06:10 gotcha 17:06:13 (this is related to the optimization to only touch the bootloader config when necessary) 17:06:38 (also note, since it's super confusing, that it's not exactly the commit hash there, but a hash of the kernel+initrd) 17:06:45 another reason for this is that one can have the same commit deployed multiple times, so there's a "deployserial" 17:06:55 that's the .0 17:08:24 it looks like the slotting thing may benefit from a bit of experimenting with static grub config + grubenv first 17:08:44 while the releases vs deployments 17:08:47 kaeso +1 17:08:48 re pinning: what if we require you layer in grub for pinning and if you pin you generate a combined grub config 17:08:51 kaeso: ++ 17:09:10 yeah definitely need some research here 17:09:25 i like the "ordered list" model we have today, but also like the idea of a static grub config 17:09:32 I think I agree with dustymabe on considering plain deployments, even it could mean "a single release with different package layering" 17:10:05 *even if 17:10:50 action ajeddeloh and lorbus to experiment on static + grubenv? 17:11:04 SGTM 17:11:44 #action ajeddeloh and lorbus to experiment on grubenv + static grub config 17:11:55 anything else on this? 17:12:16 nope 17:12:24 #topic Set up POC pipeline for Fedora CoreOS 17:12:32 #link https://github.com/coreos/fedora-coreos-tracker/issues/49 17:12:58 dustymabe: +1 for setting that up! 17:13:11 dustymabe got the POC running, right? 17:13:39 (or at least I've seen a qcow this morning) 17:13:43 yep 17:13:51 dustymabe++ 17:13:53 i'm about to push up some configs and docs 17:14:14 to this repo 17:14:16 https://github.com/dustymabe/fcos-ci 17:14:25 so don't look at it yet, but maybe in an hour :) 17:14:43 the question is pretty much.. where should we put these files? 17:15:14 dustymabe: is it the equivalent of https://github.com/coreos/jenkins-os? 17:15:19 so maybe vote in the ticket 17:15:37 kaeso: looks similar 17:15:51 kaeso: yeah, I think that's where it'll likely go 17:15:54 + openshift manifests 17:16:06 although, up until this point I hadn't considered making it "user friedly" 17:16:18 i.e. I was just thinking this would be where we store "our configs" 17:16:20 i'd agree that starting with a separate repo is likely a better idea because it's easier to merge later than to split later 17:16:23 IIRC geoff- mirrors the CL one on his own infra 17:16:37 not necessarily "here's how you take this repo and make your own" 17:16:44 dustymabe: i think we should make it as `oc cluster up` friendly as we can 17:16:47 but that's something we could consider 17:17:05 jlebon: yeah i'm not opposed 17:17:18 Yes, I have the whole CI setup locally 17:17:20 although I will say that we need /dev/kvm, and right now you need oci-kvm-hook rpm for that 17:17:30 so we'll have to add that to requirements 17:17:40 dustymabe: it looks like there is a general agreement on dedicated repo + a repos list somewhere prominent 17:17:41 either way.. we're making progress 17:17:50 dustymabe: fcos-ci sounds also ok 17:17:59 yay 17:17:59 kaeso: regarding repos list.. i think fedora-coreos-tracker could be a good place for that? 17:18:04 yup 17:18:25 it's basically the entrypoint to FCOS dev so it makes sense to put it there 17:18:31 kaeso: yeah I just called it fcos-ci because I needed somewhere to post up a file that could be retrieved from github 17:18:36 I know I always repeat myself, but at some we'll likely need a repo manager too 17:18:42 maybe fedora-coreos-ci would be better 17:19:07 to model the other repos we have 17:19:25 will suggest in the ticket.. i think we can move forward to other topics 17:19:50 acl 17:19:52 *ack 17:19:56 last one 17:20:01 #topic meeting times and daylight savings 17:20:09 #link https://github.com/coreos/fedora-coreos-tracker/issues/57 17:20:56 (I'm actually checking when the DST switch is over here) 17:20:59 so for those of us with DST, when we change time in a week or two, the meeting will move to an earlier hour, yes? 17:21:28 I'm fine with it either way personally 17:21:34 * ajeddeloh is bad at time math 17:21:40 daylight saving may be going away in the EU from next year 17:21:49 same with CA 17:21:55 prop 7 17:22:15 so, in EU on 28th Oct we go back an hour 17:22:34 I think in US the switch is on a different date 17:22:38 I think US is early november? 17:22:54 Nov 5 17:23:07 4th 17:23:16 5th is a monday 17:23:28 so all of these confusions are why sticking with UTC is helpful. there is no confusion about when the meeting is 17:23:30 ah whoops, I found the 2017 date 17:23:37 it does suck because the meeting "moves" 17:23:56 dustymabe: I'd stuck with UTC for clarity too 17:23:57 but each locale does their conversion independently and there is no confusion about when the meeting is or when it moves 17:24:04 it moves to an earlier hour, yes? 17:24:18 i think later? 17:24:19 only question is if becomes too early for west coast 17:24:39 it'd move to 8:30am west coast 17:24:42 which is fine 17:24:57 ok 17:24:58 https://www.timeanddate.com/worldclock/meetingtime.html?month=11&day=14&year=2018&p1=283&p2=48&iv=0 17:25:12 i honestly wish we could just stick to "summer time" year round 17:25:16 but I know that's controversial 17:25:23 depends on who you are 17:25:24 dustymabe +1 17:25:35 yeah summertime ftw 17:25:50 what time is the meeting currently at utc? 17:25:52 so. summary.. maybe add your vote in the ticket? and we'll regroup next week 17:26:07 ajeddeloh: 16:30 17:26:27 yeah, it'd be 830, which is fine 17:26:41 dustymabe: I think you can directly skip to "keep 16:30 UTC" 17:26:51 kaeso: :) ok 17:27:03 I can add that the sentiment from the meeting was "keep 16:30 UTC" 17:27:13 and we'll see if we get any comments from people who couldn't make it to the meeting 17:27:27 yup 17:27:28 i don't see sinny/sayan around and I'd be interested in their thoughts 17:27:34 although I don't think IST moves 17:27:39 so would probably be better for them 17:28:03 dustymabe, yep, IST doesn't change :) 17:28:09 +1 17:28:35 ack 17:28:40 that was it I guess 17:28:42 #topic Open Floor 17:28:53 any other random items? 17:29:48 (come on, not even a funny trivia?) 17:29:59 :) 17:30:17 i noticed the greenboot RPM made it into Fedora! 17:30:21 nice work lorbus ! 17:30:39 I think its in "batched" mode :) 17:30:44 another week til stable 17:30:48 will get there eventually 17:31:00 I'll cut this here, thanks everybody! 17:31:02 #endmeeting