16:31:36 <travier> #startmeeting fedora_coreos_meeting
16:31:36 <zodbot> Meeting started Wed Feb 24 16:31:36 2021 UTC.
16:31:36 <zodbot> This meeting is logged and archived in a public location.
16:31:36 <zodbot> The chair is travier. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:31:36 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:31:36 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:31:39 <jlebon> .hello2
16:31:40 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:31:41 <travier> .hello siosm
16:31:42 <PanGoat> .hello jaimelm
16:31:43 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com>
16:31:46 <zodbot> PanGoat: jaimelm 'Jaime Magiera' <jaimelm@umich.edu>
16:31:46 <dustymab1> .hello2
16:31:46 <slowrie> .hello2
16:31:49 <zodbot> dustymab1: Sorry, but you don't exist
16:31:49 * dustymab1 lurking
16:31:52 <zodbot> slowrie: slowrie 'Stephen Lowrie' <slowrie@redhat.com>
16:31:56 <dustymab1> oh hmm
16:32:05 <travier> #chair slowrie jlebon PanGoat
16:32:05 <zodbot> Current chairs: PanGoat jlebon slowrie travier
16:32:17 <dustymabe> .hello2
16:32:18 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:32:21 <bgilbert> .hello2
16:32:21 <travier> #chair dustymabe
16:32:21 <zodbot> Current chairs: PanGoat dustymabe jlebon slowrie travier
16:32:22 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:32:28 <travier> #chair bgilbert
16:32:28 <zodbot> Current chairs: PanGoat bgilbert dustymabe jlebon slowrie travier
16:33:50 <lorbus> .hello2
16:33:51 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:34:00 <travier> #chair lorbus
16:34:00 <zodbot> Current chairs: PanGoat bgilbert dustymabe jlebon lorbus slowrie travier
16:34:00 <lorbus> thanks for the pointer travier :)
16:34:25 <travier> #topic roll call
16:35:12 <jlebon> travier: i guess we did that already :)
16:35:29 <travier> oups indeed, missed the command at the beginning
16:35:32 <travier> #topic Action items from last meeting
16:35:54 <travier> bgilbert to investigate FCCT check for too-small rootfs
16:35:58 <travier> miabbott to file a ticket for Outreachy project ideas
16:36:06 <bgilbert> #action bgilbert to investigate FCCT check for too-small rootfs
16:36:29 <travier> #action miabbott to file a ticket for Outreachy project ideas
16:37:01 <jlebon> oh we do have a ticket for that last one
16:37:03 <jlebon> #undo
16:37:03 <zodbot> Removing item from minutes: ACTION by travier at 16:36:29 : miabbott to file a ticket for Outreachy project ideas
16:37:16 <jlebon> #info miabbott filed https://github.com/coreos/fedora-coreos-tracker/issues/748
16:37:26 <travier> Oups. Thanks
16:37:59 <travier> So looks like we are good here
16:38:37 <jlebon> 👍
16:38:41 <travier> #topic Ideas for Outreachy 2021 internships
16:38:47 <travier> #link Ideas for Outreachy 2021 internships
16:38:53 <travier> (To stay on topic :))
16:39:20 <travier> #link https://github.com/coreos/fedora-coreos-tracker/issues/748
16:39:45 <jlebon> i can speak to this one a bit
16:40:30 <jlebon> we may be able to have an outreachy internship for FCOS in the next cycle, which would be cool
16:40:54 <jlebon> we've been discussing different project ideas in that github issue
16:41:27 <jlebon> i think the current one is "integrate FCOS testing into Bodhi"
16:42:14 <jlebon> does anyone have reservations about that, or other ideas that would better fit a 3-month internship or higher benefit/cost ratio ?
16:42:31 <jlebon> the deadline for having something ready is friday
16:42:45 <jlebon> and we can of course just skip this cycle if we feel like we don't have anything good right now
16:43:11 <walters> (I have a semi-related question: Is it correct to say Outreachy is independent of Red Hat and so e.g. the intern wouldn't have access to any internal RH mail/chat/build systems and hence would also be a forcing function for us to ensure everything they need is upstream/public right?)
16:43:12 <travier> Who would be ready to do the mentoring?
16:43:27 <dustymabe> walters: yeah I think that's right
16:43:31 <walters> cool
16:43:39 <dustymabe> similar to when lorbus was an intern through GSoC
16:44:39 <jlebon> travier: i'd be happy to help, but would be nice to have at least two :).  i think cverna is also interested
16:44:45 <lorbus> yes, that'd be without any internal access
16:45:18 <jlebon> i guess it depends on the project in question. if it's something narrow enough, one mentor may be enough
16:46:38 <jlebon> the "FCOS testing in Bodhi" idea for example requires knowledge i think our team doesn't have, which is a con.  we'd want someone from Fedora CI willing to help mentoring too
16:46:39 <travier> I'm +1 Bodhi but don't feel like I would have the time to mentor nor would be a good mentor for that so I don't want to force that on someone else.
16:47:54 <jlebon> does anyone else want to propose alternative project ideas?
16:48:11 <dustymabe> I have so many ideas during the normal course of work
16:48:19 <dustymabe> but tend to draw blanks when someone asks me
16:48:34 <jlebon> dustymabe: heh, i know what you mean :)
16:48:40 <dustymabe> I'm sure there are specific features in ostree/rpm-ostree or ignition/fcct we've been wanting for a while that we've been putting off
16:49:25 <jlebon> a good rpm-ostree one would be RPM overrides from repos
16:49:30 <travier> we have sysusers support in rpm-ostree / fcos :)
16:49:39 <lorbus> module support as well as sysusers support come to mind
16:49:43 <jlebon> https://github.com/coreos/rpm-ostree/issues/1265
16:49:59 <dustymabe> could we propose it as "various enhancements to rpm-ostree" and bullet point them out
16:50:33 <jlebon> dustymabe: having a "kitchen sink" type of proposal might not look good
16:50:45 <jlebon> we could have a primary one, and then stretch goals related to that primary one
16:51:12 <dustymabe> yeah, the problem is I feel our projects are either small or large (longer than normal internship). Not much medium sized in there.
16:51:21 <dustymabe> jlebon: +1
16:53:03 <jlebon> hmm, sysusers support or the overrides one i think could be about medium size (maybe sysusers being a bit larger)
16:53:09 <travier> SO I guess we have a bunch of option. Who wants to take action to submit one / mentor one?
16:53:12 <travier> So*
16:54:36 <dustymabe> I might be able to help mentor for the bodhi stuff
16:54:55 <dustymabe> so someone can add me as a helper on the submission
16:55:06 <jlebon> i'm open to work on submitting and mentoring for the repo overrides one
16:55:09 <dustymabe> assuming the internship doesn't start til after march
16:55:15 <jbrooks> .hello jasonbrooks
16:55:15 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:55:20 <dustymabe> 👋
16:55:35 <travier> #chair jbrooks
16:55:35 <zodbot> Current chairs: PanGoat bgilbert dustymabe jbrooks jlebon lorbus slowrie travier
16:56:14 <travier> jlebon: I can co-mentor rpm-ostree ones if needed
16:56:25 <jlebon> dustymabe: runs from May 24 to August 24
16:56:31 <dustymabe> jlebon: +1
16:57:32 <jlebon> one concern with the Bodhi one is that we should probably socialize it more before basing a project on it
16:58:26 <jlebon> not sure if it'd require a change proposal, but e.g. don't want maintainers to just start seeing red checkmarks out of the blue
16:58:46 <travier> indeed
16:58:52 <dustymabe> yeah, would need some socialization
16:59:18 <travier> I don't have any specific action to point out but this needs to be figured out before the end of the week.
16:59:39 <travier> I'll move to another topic otherwise we won't be able to discuss everything
16:59:42 <jlebon> ok how about: travier and I will take an action to submit a proposal for one of the two rpm-ostree ones (TBD) ?
17:00:03 <travier> #action jlebon and travier will take an action to submit a proposal for one of the two rpm-ostree ones (TBD)
17:00:04 <dustymabe> jlebon: maybe see if cverna was interested in running with the bodhi one, if not we'll do it next time
17:00:21 <jlebon> dustymabe: ack
17:00:45 <travier> #topic New Package Request: stalld
17:00:50 <travier> #link https://github.com/coreos/fedora-coreos-tracker/issues/753
17:02:10 * walters added feedback in the ticket already
17:02:47 <dustymabe> i have no opposition
17:03:00 <walters> I think there's also a looming eventual need to have FCOS RT
17:03:09 <travier> This is about inclusion not enabled by default right?
17:03:55 <dustymabe> travier: that's my understanding, included but no change from current
17:04:07 <cverna> o/
17:04:09 <cverna> kind of around
17:04:18 <dustymabe> looks like the deps should be already included in FCOS. /me checks
17:04:19 <walters> yeah, units will only be enabled if explicitly done in the presets, I don't think that is
17:04:30 <jlebon> no strong opposition, though feels like the extensions/pkglayered side wasn't answered thouroughly
17:05:06 <walters> I think down the line what they want is ConditionRTKernel=yes
17:05:10 <travier> I'd say this should move into the RT extension that we don't have on fcos
17:05:18 <travier> but we don't have that*
17:06:03 <travier> is this useful outside of rt kernels?
17:07:26 <travier> No strong opinions expressed. Should I call a vote?
17:07:50 <jlebon> SGTM
17:09:01 <travier> #proposal In favor of stalld inclusion
17:09:28 <travier> (looking for how to do voting...)
17:09:35 <PanGoat> :)
17:09:55 <jlebon> (normally #proposed :) )
17:10:00 <jlebon> ack
17:10:11 <travier> #proposed In favor of stalld inclusion
17:10:26 <PanGoat> ack
17:11:40 <travier> I'll ack too so that make 3
17:11:59 <dustymabe> +1
17:12:09 <travier> Moving to next topic
17:12:34 <travier> #topic Dynamically sized bare-metal image can clobber data partition on reprovision
17:12:41 <travier> #link https://github.com/coreos/fedora-coreos-tracker/issues/586
17:12:52 <travier> bgilbert: Do you want to take that?
17:13:35 <travier> or jlebon?
17:13:40 <jlebon> sure, i can do that
17:14:22 <jlebon> TL;DR: we want to specify a minimum rootfs size, and a recommended rootfs size. right now leaning on 5 GiB for the former, 8 GiB for the latter
17:14:39 <jlebon> with the caveat for the latter being "You should allocate more space above this based on the requirements of the intended workload."
17:15:19 <jlebon> the minimum size is crucial: we'll be making it a hard error if the user traps their rootfs to a size smaller than this
17:16:11 <dustymabe> are we recommending people use a separate FS for /var ?
17:16:17 <jlebon> so basically: if anyone feels like either of those values need tweaking, speak now :)
17:16:28 <travier> dustymabe: I don't think we are
17:16:30 <jlebon> dustymabe: not necessarily
17:17:02 <jlebon> the messaging will strictly point out that the 8 GiB should be dedicated for the OS itself
17:17:10 <dustymabe> so things like container images and logs and stuff go in there?
17:17:20 <jlebon> no
17:18:17 <jlebon> (in the same partition, yes. but not conceptually when users are sizing their partitions.)
17:19:06 <dustymabe> so you're saying if I specify a size for my partition then I should know what I'm doing and set up container image storage and logs etc.. appropriately
17:19:31 <walters> seems OK, I can't imagine we ever go close to 5G but it's tricky because we're making a promise about the distant future
17:20:20 <jlebon> dustymabe: right, either you size the rootfs partition accordingly to account for those (on top of the recommended), or make separate partitions
17:20:21 <walters> i mean, if more things start to be written in Go/Rust or maybe there's some driving need for us to e.g. ship WASM tooling as well or more Kube deps or way more kernel drivers or ...I dunno
17:20:41 <jlebon> walters: i think we're ok to break that promise in the future if needed
17:20:46 <PanGoat> ^^
17:20:52 <dustymabe> jlebon: i think we need to be clear with the wording and maybe even give an example
17:21:02 <PanGoat> Invariably, there will be changes that require it
17:21:20 <dustymabe> the problem is that it's easy to put things of variable size into your 'rootfs' filesystem
17:21:28 <jlebon> as long as there's enough advance notice. the important thing is that we have a specific minimum with some amount of builtin future-proofing, and if needed we bump it up once in a blue moon
17:21:44 <walters> are we?  if the base ostree commit approaches say 3G, and we have *large* binaries that won't share storage with the previous commit...then we could hit a case where a user with 5G can no longer upgrade
17:22:05 <travier> I think it's better to have recommendations and limits and move them from time to time if needed that to get hard to debug issues.
17:22:57 <jlebon> right, clearly whatever number we choose won't be perfect forever.  but some guidelines (and enforcements) is way better than the status quo
17:23:41 <PanGoat> proposal?
17:23:44 <jlebon> people are already running into this issue: https://github.com/coreos/fedora-coreos-tracker/issues/731
17:23:46 <dustymabe> +1 - let's just make sure we have an example to help the user set up separate filesystems for the directories that "grow". It would be nice to have the OS bits (/usr/) fully isolated.
17:23:51 <walters> right now we're at 1.9G in `/usr`
17:24:03 <PanGoat> sounds like everyone is on the same-ish page
17:24:05 <walters> wait, is that right?  I may have an override in there =)
17:24:23 <travier> Forgot to action for the stalld package. Who wants to take that?
17:24:30 <dustymabe> walters: on my system: 1.7G /usr/
17:24:32 <walters> i'll do it
17:24:48 <jlebon> ok, i'll throw out a proposal with those numbers :)
17:24:59 <dustymabe> 5G seems a little low considering where we currently are
17:25:43 <jlebon> hmm, maybe
17:25:44 <dustymabe> 5G/8G -> 8G/10G ?
17:25:50 <PanGoat> for human memory purposes, you want it to be an even number. So, going up would be 8 and 10
17:26:11 <PanGoat> yes
17:26:18 <travier> 6/10?
17:26:19 <jlebon> 1.7G... let's do x2 for a major ver rebase with not a lot of sharing, + various layered packages
17:26:24 <walters> what I circle back to here is for a server-oriented OS I think we can assume two cases: 1) For "smaller" nodes like say some ephemeral-ish kube worker, don't split `/var`.  For larger baremetal nodes, I think we can assume you have at least a 512GB disk, I don't feel bad taking 8GB or 10 etc.
17:26:31 <jlebon> and possibly pinned deployments
17:27:01 <jlebon> the hard error would only happen in the case where you're trapping the rootfs with a follow-up partition
17:27:10 <travier> indeed, so 8/10 would be better
17:27:18 <jlebon> so i think we're OK increasing the minimum without affecting the small node case
17:27:29 <dustymabe> WFM
17:27:46 <dustymabe> thanks for brainstorming on this and coming up with a way forward
17:27:53 <walters> I think I'd say: warn at 8GB or less
17:27:59 <jlebon> i'm ok with 8/10
17:28:30 <jlebon> ok, let me do a proposal:
17:28:33 <walters> having two numbers feels like it just makes things more confusing/bikeshed-worthy
17:28:38 <travier> isn't 8 the error limit?
17:29:32 <PanGoat> walters: do you mean error at 8 or warn at 8?
17:29:34 <jlebon> walters: hmm maybe.  it made more sense at 5
17:30:18 <walters> i think I'm proposing: hard error when you completely trap the rootfs entirely with no extra space - that seems like it's always an accident
17:30:24 <travier> Because <5 error and <10 warn would make more sense
17:30:39 <walters> i'd lean towards warn at 8 but am fine with 5
17:31:18 <walters> hmm OK I see I forgot there are inherently two numbers today because we have the base rootfs size
17:31:40 <walters> and fcct is unaware of that hardcoding in cosa
17:32:03 <walters> but couldn't we just hard error if the rootfs size isn't specified?
17:32:15 <travier> (warning: we're over time)
17:32:29 <walters> travier: not a hard error =)
17:32:46 <jlebon> leaning now towards just one 8 GiB number to keep it simple:  error if trapped with less than that, warn if less than that but no following partition
17:32:48 <walters> (should it be?  let's debate)
17:33:03 <PanGoat> is that an error or a warning that we're over time?
17:33:22 <PanGoat> Perhaps table this until next week while folks think a bit more
17:33:49 <walters> jlebon: seems OK to me.  Honestly I think my strong feelings on this aren't so much the exact numbers so much as "let's keep it simple"
17:33:50 <jlebon> hmm, guess so. i think we're overthinking this :)
17:34:18 <travier> is that an error or a warning that we're over time? > This is about meeting time, just a notice
17:34:37 <jlebon> walters: yeah, simplicity is important
17:34:39 <PanGoat> :)
17:34:42 <travier> i like last jlebon proposal
17:35:02 <PanGoat> It's simple
17:35:15 <jlebon> #proposed Set a single limit of 8 GiB: we error if the rootfs is trapped with less than that, but only warn if  there are no following partition
17:35:18 <PanGoat> A little easier to explain to users, which is the real concern
17:35:35 <dustymabe> ack
17:35:37 <PanGoat> ack
17:35:49 <travier> ack
17:36:41 <jlebon> SHIPIT
17:36:51 <travier> Alright, we have other issues but they are less important so I'd say keep them for next week?
17:36:58 <PanGoat> +1
17:37:02 <jlebon> #agreed Set a single limit of 8 GiB: we error if the rootfs is trapped with less than that, but only warn if  there are no following partition
17:37:06 <jlebon> travier: agreed
17:37:14 <jlebon> we're way over anyway :)
17:37:19 <jlebon> we've been doing that a lot recently
17:37:23 <travier> #agreed In favor of stalld inclusion
17:37:49 <PanGoat> There's lots of brainstorming on the spot, which is taking more time than expected.
17:37:49 <travier> #topic Open Floor
17:38:02 <dustymabe> Thanks all for working hard on FCOS!
17:38:16 <travier> jlebon: do you want to take an action for the rootfs limit?
17:38:35 <jlebon> travier: nah, already working on it. i'll post the outcome there :)
17:38:45 <travier> 👍
17:38:47 <jlebon> dustymabe: looking forward to have you back :)
17:38:57 <jbrooks> I'll just say that the twitter handle poll is live at: https://twitter.com/coreos/status/1364006780796170242
17:39:22 <travier> Currently at 223 votes, 84% in favor :)
17:39:40 <dustymabe> jbrooks: nice!
17:40:30 <dustymabe> Thanks for posting that up
17:40:46 <travier> jbrooks==
17:40:48 <travier> jbrooks++
17:40:48 <zodbot> travier: Karma for jasonbrooks changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:41:13 <jlebon> very cool!
17:41:14 <jlebon> jbrooks++
17:41:14 <zodbot> jlebon: Karma for jasonbrooks changed to 3 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:41:16 <dustymabe> jbrooks++
17:41:25 <travier> I'll close in 5 min if nothing else gets posted :)
17:41:27 <dustymabe> no we just need to make jlebon a twitter account so he can vote
17:41:37 <travier> dustymabe: :D
17:41:39 <jbrooks> :)
17:42:04 <PanGoat> heads up that I'm participating in an OKD "hackathon" type thing that Diane is setting up on March 20. lorbus will be there I think as well. The idea is to demo automated pulling of FCOS images and OKD installs to simplify testing of new OKD releases. I'll be posting some of my FCOS testing scripts and a write-up in preparation for that. Feel free to join us for the event.
17:42:25 <jlebon> dustymabe: hah! don't hold your breath :)
17:42:28 <dustymabe> PanGoat++
17:42:41 <lorbus> PanGoat++
17:42:58 <jlebon> nice!
17:43:13 <travier> Nice! Might join too
17:44:01 <jlebon> travier: i think you can end it :)
17:44:08 <smooge> dustymabe, are you back from PTO ?
17:44:15 <dustymabe> thanks travier - thanks all!
17:44:23 <dustymabe> smooge: end of march
17:44:30 <dustymabe> i'm just lurking in the community meeting
17:44:37 <dustymabe> need anything?
17:44:45 <smooge> ok thanks. there were some questions for cloud / server sigs that came up
17:44:45 <travier> #endmeeting