16:30:21 #startmeeting fedora_coreos_meeting 16:30:21 Meeting started Wed Aug 26 16:30:21 2020 UTC. 16:30:21 This meeting is logged and archived in a public location. 16:30:21 The chair is cverna. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:30:21 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:30:21 The meeting name has been set to 'fedora_coreos_meeting' 16:30:30 .hello2 16:30:33 bgilbert: bgilbert 'Benjamin Gilbert' 16:30:38 #topic roll call 16:30:42 .hello2 16:30:43 dustymabe: dustymabe 'Dusty Mabe' 16:31:11 .hello2 16:31:12 lorbus: lorbus 'Christian Glombek' 16:31:22 .ello2 16:31:22 #chair bgilbert dustymabe lorbus 16:31:22 Current chairs: bgilbert cverna dustymabe lorbus 16:31:25 .hello2 16:31:26 darkmuggle: darkmuggle 'None' 16:31:32 .hello2 16:31:34 davdunc: davdunc 'David Duncan' 16:31:59 .hello2 16:32:00 jlebon: jlebon 'None' 16:32:27 #chair darkmuggle davdunc jlebon 16:32:27 Current chairs: bgilbert cverna darkmuggle davdunc dustymabe jlebon lorbus 16:32:50 .hello2 16:32:51 aoei: aoei 'Joanna Janet Zaitseva-Doyle' 16:32:53 .hello sohank2602 16:32:53 .hello2 16:32:54 skunkerk: sohank2602 'Sohan Kunkerkar' 16:32:55 let's give another minutes to see if anyone else will be joining 16:32:57 cyberpear: cyberpear 'James Cassell' 16:33:36 #chair skunkerk cyberpear aoei 16:33:36 Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe jlebon lorbus skunkerk 16:34:33 #topic Looking at what's next for FCOS 16:34:53 .hello2 16:34:54 lucab: lucab 'Luca Bruno' 16:35:25 So today's meeting is a bit particular we wanted to take the opportunity to look at what are the issues we would like to focus on 16:36:12 you can find a list of the some of the issues in this hackmd https://hackmd.io/aVJiL9DUQpCDSafSBcs6ZQ?edit 16:36:18 #chair lucab 16:36:18 Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe jlebon lorbus lucab skunkerk 16:37:08 Note the items are labeled with a letter so we can easily refer to them here in discussion 16:37:13 and this meeting is an opportunity to discuss these tickets and try to come up with a rough ideas of priority 16:37:27 .hello miabbott 16:37:28 miabbott: miabbott 'Micah Abbott' 16:37:44 .hello imcleod 16:37:45 imcleod: imcleod 'Ian McLeod' 16:37:51 ideally we would have this kind of meeting on a regular basis and re-evaluate these priorities 16:38:04 #chair miabbott imcleod 16:38:04 Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe imcleod jlebon lorbus lucab miabbott skunkerk 16:38:25 any questions ? or anything not clear with what we trying to achieve today ? 16:38:51 cverna: if I see an item missing from the list, what should I do as a user? 16:40:12 dustymabe: add it to the hackmd :) then we will discuss each category (In progress, To do, etc ..) 16:40:47 +1 16:41:31 ok then let's look at the In Progress tickets 16:41:56 in this list is there anything we want to discuss in particular ? 16:42:08 anything/any ticket ? 16:42:28 i think they're all important and it's good they're in progress :) 16:42:39 I assume that if these tickets are in progress it would make sense to complete them before starting something else 16:43:05 +1 16:43:08 no questions from me.. I will point out that the mulit-arch thing is a POC. really hoping it yields good results and we can add other architectures to our output 16:43:35 i haven't looked at the related ticket in a while, but what's the priority of the multi-arch item? is this something we need to get done sooner than later or, is it more best effort? 16:44:20 miabbott: we have people asking it for us often and parallel efforts within the community to produce multi-arch artifacts 16:44:25 i don't think there's any hard timeline. but the sooner the better :) 16:44:39 bringing it under the production pipeline umbrella mostly helps us to save everyone a lot of wasted effort 16:45:05 dustymabe: what happen once the POC is completed ? do we have another ticket to implement the solution if viable ? 16:45:40 cverna: correct. once we have sufficient investigation there we'll know what it will take to implement 16:45:49 it might be a lot of work, it might be a little 16:45:57 +1 16:46:01 so we'll have to re-evaluate in a bit 16:46:04 .hello2 16:46:05 jdoss: jdoss 'Joe Doss' 16:46:19 Sorry I am late. Double booked hard on Wednesdays. 16:46:20 #chair jdoss 16:46:20 Current chairs: aoei bgilbert cverna cyberpear darkmuggle davdunc dustymabe imcleod jdoss jlebon lorbus lucab miabbott skunkerk 16:46:51 📖 📖 16:47:07 ok so let's move to the issues that are in the "To Do" category and try to find which one are the top 2 or 3 16:47:46 Do we want to talk about item E - Move `next` stream to Fedora 33 https://github.com/coreos/fedora-coreos-tracker/issues/609 16:47:57 since there is a time constraint there ? 16:48:14 I think it's just something we'll have to do. Not sure if there is much to discuss 16:48:26 I think F and G sound potentially quite nice for users- tho perhaps not super high priority 16:48:34 but nice things are also good 16:49:03 yeah, agreed we should prep for f33 soon. the mechanical work shouldn't be hard. likely the hard part will be dealing with whatever fallout 16:49:17 jlebon: indeed 16:49:54 maybe we should make concrete proposals 16:50:03 do we have anything that we thing should come before F and G ? 16:50:03 i propose we drop H out of consideration. 16:50:17 having a rawhide stream will be useful, but not necessary right now 16:50:36 dustymabe: we can move it to the bottom of the list :) 16:50:41 I think F is still blocked on rpm-ostree supporting that in a declarative way, no? 16:50:45 I think K is valuable though also 16:50:50 i reworded F a bit to be about the larger issue instead of focusing on the final bit 16:50:56 anything that helps you troubleshoot and fix stuff is a Good Thing 16:51:12 lucab: indeed, there's a lot of design work there 16:51:12 lucab: right. it's kind of a high level "goal" - plubming required 16:51:26 K and L are valuable. M would help us. 16:51:41 rawhide stream seems like it'd be very low maintenance, and be a "super sensitive canary" 16:51:53 IMO G is not a priority, since we've consistently discouraged layering. 16:52:09 N would be nice, but is probably not in the top 3 16:52:44 bgilbert, good point. I like F but G is not necessary 16:52:48 bgilbert: I think G will be useful (layering sugar), but I think the kargs sugar is way more useful short term (F) 16:52:53 G is important, IMO 16:53:19 so let's talk about F, lucab raised a point about a dep on rpm-ostree is that blocking that ticket ? or part of the ticket ? 16:53:23 cyberpear: let's say you had to choose between F and G 16:53:43 F is more time-pressing, I'd think 16:53:49 cverna: i'd say that's part of the ticket. i rephrased it 16:53:49 cverna, yes i think I saw that, someone is waiting on a fix in external software I think 16:54:13 jlebon: cool just saw that :) 16:54:20 but B and G are almost the same 16:54:49 I see F and G as part of the same "pipeline" so I think it would be safe to eliminate G and keep F in consideration for now 16:55:13 I feel that we have a strong interest about F and it would make sense to keep it as the first ticket on the list 16:55:36 +1 to move G down for now ? 16:55:39 +1 16:55:41 +1 16:55:42 +1 16:55:47 SGTM 16:56:05 +1 16:56:06 +- 0 16:56:31 then after F we have K how does that sounds ? 16:56:33 bgilbert: i think eventually there'll be extensions (layered pkgs) which we'll consider reasonable. but first we need to make it solid 16:56:38 one sec 16:56:44 cyberpear: yeah B and G have overlap 16:56:45 the most important IMO would be B, C, E, F, K, I 16:56:47 I'm all for K 16:56:47 -1 because missing packages is currently a big pain point for me 16:57:04 right now I'm limiting B to "the ability to package layer even on older content" 16:57:13 and I'm real close to having that problem solved 16:57:22 G is more the "now let's make it ergonomic" step 16:57:34 but if B handles G in the short term then +1 16:57:39 sorry catching up. 16:58:00 i think R is really important 16:58:04 would G also need changes to the Ignition config spec? 16:58:26 as in fcct sugar translated to what exactly? 16:58:27 lorbus: not sure exactly.. it needs to go through some design 16:58:48 in MCO this is done with a field on the MachineConfig right now. 16:58:54 we need to nail down our promise re. root disk size otherwise we risk making people angry in the best case, or lose data in the worse case 16:58:59 true its a somewhat vague proposal as it currently stands 16:59:01 sorry, don't want to diverge here. keep going all :) 16:59:15 lorbus: units, presumably 16:59:19 current discussion topic is R Stable root image size 16:59:34 jlebon: would you consider R to be in the top 3 ? 16:59:38 earlier jlebon suggested I just use envsubst which works pretty well for some things 16:59:47 jlebon: I think I agree and I think the work effort there is pretty light.. bgilbert can confirm 16:59:48 no fcct sugar needed 16:59:49 top 3 of the TO Do 16:59:51 bgilbert: ah yeah that should work 17:00:03 cverna: yes, i think so 17:00:23 effort for R might be a bit more substantial if we need Ign spec changes for partition resizing 17:00:27 not sure if we do 17:00:49 how much headroom do your images typically have? 17:01:25 proposal: drop M from consideration, since Q was created 17:01:33 bgilbert: would you agree we should address it Very Soon though? 17:01:39 jlebon: yes 17:01:45 when I created M I was mostly referring to the work described by Q 17:01:49 dustymabe: +1 17:02:05 aoei: headroom isn't the issue, it's that we set the size dynamically at build time, which means we risk clobbering user data when reprovisioning 17:02:10 +1 to axing M 17:02:14 bgilbert, ahh okay 17:02:16 SGTM re. moving M down 17:02:42 bgilbert, could you add headroom to the build time size to mitigate? 17:02:57 So it looks like we have a top 4 for now with F, K, R and L 17:03:14 aoei: probably not worth getting into it right now. let's chat in the ticket 17:03:15 anyone disagree with that top 4 ? 17:03:20 aoei: see https://github.com/coreos/fedora-coreos-tracker/issues/586 for details 17:03:21 jlebon, ok fair 17:03:29 I've been chopping at K in the last two sprints together with abai 17:04:13 lucab: how would you evaluate K and L? 17:04:15 lucab: nice. how far is that from being over the finish line 17:05:12 fwiw, once Q is done, jlebon has a WIP that would make M done, I think 17:05:17 jlebon: I think K is way simpler than L 17:05:28 L sounds nice to have, but it's not a deficiency relative to regular Fedora 17:05:35 is L really that high of a prio? 17:05:42 darkmuggle: indeed, that'll be fun :) 17:05:53 proposal replace L with P in consideration 17:06:07 for P we have a cloud env ready to tap into - we just need to wire things up 17:06:17 that makes sense dusty 17:06:25 agreed, P sounds much easier than L to solve 17:06:25 and hope our openstack provider in mantle works (not sure how long it's been since it was touched) 17:06:33 K is both simpler and arguably more important than P or L 17:06:34 fwiw greenboot is now getting some real testing with Fedora IoT and RHEL Edge, and I suspect it will play a part in L, but I'd leave it to bake for a little longer 17:06:41 dustymabe: not too far, but there is still a bit of design work to be done 17:06:51 lucab: +1 17:07:32 my biggest unknown on K is: how to setup rpm-ostree to use offline repos 17:07:35 L is one of those things that sound super cool, but it's not clear how impactful it'll actually be in practice 17:07:37 ok so F.K.R.P. are the top contenders right now? 17:07:54 lucab: update the ostree remote ? 17:08:03 so close to W.K.R.P 17:08:10 it's pretty easy to do 17:08:43 N has been an issue that has been punted due to low prio for ages....would be great to move that forward not too far into the future 17:08:50 * dustymabe assumes "offline" means "disconnected" 17:08:52 it depends on the definition of "offline", but yeah there are different approaches 17:09:00 dustymabe: well yes, the whole end-to-end thing, both for repo-mirroring and client-setup 17:09:26 I can share some info on that. The rpm-ostree side is easy (IIUC). 17:09:38 ok coming back to having the next 3-4 most wanted items in the To Do section 17:09:54 it looks like we have F, K, R and P 17:10:30 sounds right to me 17:10:38 i like it too :) 17:10:48 I don't think it is worth looking at the rest now, we should probably do this exercise in a month or so and see if we want to update that order or not 17:10:51 as much as I hate to kick the systemd-sysusers stuff down the road (again) :) 17:11:01 K is actually two very different subpoints, which one we care the most? 17:11:02 +1 to FKRP as top 4 17:11:07 +1 to FKRP 17:11:08 aw poor systemd 17:11:11 +1 FKRP 17:11:35 aoei :P 17:11:42 lucab: so Mirrored/Proxied vs Air-gapped ? 17:12:00 should it be Proxied vs Mirrored/Air-Gapped ? 17:12:13 dustymabe: yes. I've been mostly working on the former so far 17:12:32 agreed, proxied v/ mirror/air-gapped seems more accurate 17:12:32 I guess I'd need to understand the nuances between the two 17:12:55 maybe let's chat about that at some point? interested as well 17:13:08 generally, I've needed a mirror to use in the air-gapped environment 17:13:11 uhm no, air-gapped is basically "no network, somebody manually goes with an update disk" 17:13:34 ahh. hmm 17:13:37 that sounds like something you want to never happen 17:13:42 yeah I don't think we need to go into the details now, let's keep K there and we can look at it further when we come close to implement it 17:13:51 in my world, air-gapped is "a network of systems not connected to the internet or any other network" 17:13:53 even tho lucab is already working on it :) 17:14:10 lucab: definitely want to dig in to that soon so we can define the individual use cases and choose the ones that are highest priority 17:14:12 (but sometimes could be "a single standalone system with no network at all") 17:14:29 the former is "auto-update on, but from local infra", the latter is "auto-updates off, manual intervention with disks" 17:14:43 cyberpear, more than one otherwise isolated systems sounds substantially less painful than a single unetworked machine lol 17:14:53 air gapped sound fancier than "my network card driver is not working" 17:15:00 :) 17:15:04 cyberpear: https://github.com/coreos/fedora-coreos-tracker/issues/261 is the latter 17:15:08 misc, xD 17:15:22 lucab: maybe we can try to define this topic in more detail and go over it next meeting? 17:15:31 misc: apparently there are people with purposedly broken network cards ;) 17:15:31 dustymabe: +1 17:15:56 misc: agreed... I don't often have to deal with one-off standalone systems, but networked non-internet systems are quite common 17:15:59 lucab, :o 17:16:04 pending that discussion we can break out K into the one we care most about at this point 17:16:04 dustymabe: sure, it's already two separate tickets on -tracker 17:16:34 I think we have done a pretty good job at prioritizing these issues :) do we want to discuss anything else today ? 17:17:03 yeah I vote to change all the .ign files to .json 17:17:15 i keep spelling it wrong D: 17:17:21 cverna: one sec 17:17:53 aoei: :D 17:17:58 #info We have determined that F.K.R.P are the items with the most interest in completing first 17:18:10 #info - F. Add coherent support for modifying kernel arguments, and provide fcct sugar 17:18:18 #info - K. Updates in offline/disconnected environment 17:18:28 #info - R. [Stable root image size](https://github.com/coreos/fedora-coreos-tracker/issues/586) 17:18:39 #info - P. [Adding OpenStack to our CI](https://github.com/coreos/fedora-coreos-tracker/issues/612) 17:19:03 #info (after the items already in progress) 17:19:09 correct 17:19:16 should we open F and K for discussion as tickets as well? 17:19:42 aoei: both already have tickets? 17:19:42 aoei: it looks like it would be useful for a high level ticket to summarize the goals of F 17:19:43 yeah, it's on my plate to open a top-level tracker ticket for F 17:20:14 for K it looks like we already have two tickets and we'll use next week's meeting to try to focus on which one to work on 17:20:14 bgilbert, I assumed there was a reason they were some of the few without links to the tracker 17:20:24 ah ok i see 17:21:00 jlebon: F: https://github.com/coreos/fedora-coreos-tracker/issues/318 17:21:32 bgilbert: nice, thanks for adding that 17:21:47 will add a comment there 17:22:27 #topic Open Floor 17:22:55 thanks for kicking off this discussion, cverna! 17:23:02 PXE boot on the `next` stream now requires the separate rootfs image 17:23:05 cverna++ 17:23:05 lorbus: Karma for cverna changed to 21 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:23:08 I have a little question, how did you find that type of session ? 17:23:18 yes thank you cverna and everyone else, it has been educational 17:23:22 cverna++ 17:23:23 bgilbert: Karma for cverna changed to 22 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:23:26 cverna++ 17:23:28 re K: 240 seems to be about offline mirroring, to be also used with disconnected networks - 261 seems to be about a literally standalone system, bringing updates to that system on a disc or similar, or "how to ship an update for an FCOS-based appliance" 17:23:47 lorbus: np, dustymabe has been helping a lot behind the scene :) 17:23:55 dustymabe++ 17:24:00 cverna: i'm a bit biased based on my $DAYJOB, but prioritizing the future work is super helpful 17:24:02 🤗 17:24:08 dustymabe++ 17:24:20 cverna, I think I'm in agreement with yuo that it's a good idea to do fairly often 17:24:23 cyberpear: if that's the case then I agree with Luca that they are very different :) 17:25:09 cverna: yeah, it was definitely helpful to see it all laid out like this! 17:25:21 good to have an overview 17:25:27 cverna: yup, super helpful IMO 17:25:31 cool, let's try to do this maybe once a month and see how that goes, if it is too often we can slow down the cadence 17:25:32 cyberpear: yes, I think we should update 240 to avoid the term air-gapped 17:25:58 #info PXE boot on the `next` stream (as of 32.20200824.1.0) now requires the separate rootfs image 17:26:39 dustymabe++ 17:26:53 cverna: +1 for once a month.. or maybe 6 week cadence 17:27:07 either way.. we'll figure it out as we go.. like everything else 17:27:29 +1 17:27:36 I can't wait to knock some of these items out 17:27:42 dustymabe, :D 17:28:46 dustymabe: that's the hard part, planning is easy :P 17:29:07 feels good when you close the things tho 17:30:48 Thanks cverna for running the meeting today 17:30:48 ok thanks everyone for joining and participating 17:30:56 I really enjoyed it 17:31:01 #endmeeting