16:29:06 <dustymabe> #startmeeting fedora_coreos_meeting
16:29:06 <zodbot> Meeting started Wed Aug 30 16:29:06 2023 UTC.
16:29:06 <zodbot> This meeting is logged and archived in a public location.
16:29:06 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:29:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:29:06 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:29:11 <dustymabe> #topic roll call
16:29:15 <dustymabe> .hi
16:29:16 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:30:28 <travier> .hello siosm
16:30:29 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com>
16:31:20 <aaradhak> .hello aaradhak
16:31:21 <zodbot> aaradhak: aaradhak 'Aashish Radhakrishnan' <aaradhak@redhat.com>
16:31:36 <marmijo> .hi marmijo
16:31:36 <zodbot> marmijo: marmijo 'Michael Armijo' <marmijo@redhat.com>
16:31:41 <dustymabe> #chair travier aaradhak marmijo spresti
16:31:41 <zodbot> Current chairs: aaradhak dustymabe marmijo spresti travier
16:32:06 <ravanelli> .hi
16:32:07 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com>
16:33:28 <dustymabe> #chair ravanelli
16:33:28 <zodbot> Current chairs: aaradhak dustymabe marmijo ravanelli spresti travier
16:34:11 <dustymabe> did I miss anyone on #chair?
16:34:25 <dustymabe> Guidon: want to `.hi` yourself?
16:34:43 <spresti> .hi
16:34:44 <zodbot> spresti: spresti 'Steven Presti' <spresti@redhat.com>
16:34:51 <spresti> Dont think I did either.
16:34:55 <travier> Let's start with #1557 then maybe #1558?
16:35:09 <travier> (unless something else is more important)
16:35:48 <gursewak> .hi
16:35:49 <zodbot> gursewak: gursewak 'Gursewak Singh' <gurssing@redhat.com>
16:36:04 <dustymabe> #chair gursewak
16:36:04 <zodbot> Current chairs: aaradhak dustymabe gursewak marmijo ravanelli spresti travier
16:36:06 <dustymabe> #topic Action items from last meeting
16:36:23 <dustymabe> We had one from last meeting:
16:36:52 <dustymabe> * travier will bring that issue to the rpm-ostree / ostree team
16:37:05 <dustymabe> this was in reference to ConditionNeedsUpdate in https://github.com/coreos/fedora-coreos-tracker/issues/1538
16:37:06 <travier> I've asked those folks to look at it
16:37:50 <dustymabe> #info travier has asked the rpm-ostree/ostree folks to take a look
16:37:54 <dustymabe> thanks travier
16:38:01 <travier> 👍
16:38:17 <dustymabe> #topic sshd.socket going away in Fedora 39
16:38:22 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1558
16:39:04 <dustymabe> was this a Fedora level change?
16:39:38 <travier> it was not
16:39:56 <dustymabe> interesting..
16:40:00 <travier> https://bugzilla.redhat.com/show_bug.cgi?id=2025716#c14
16:40:40 <smooge> the sshd.socket item was discussed on devel list I believe
16:40:47 <dustymabe> probably
16:40:50 <smooge> but no change announcement I remember
16:40:57 <dustymabe> either way - either now or later, we'd need to deal with this
16:41:21 <travier> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/O6V7ZH3BMCWOREDWXOIAPUG7PGMWIBVR/#BQPRQLXPBKPQL7KR62GUR5H7EEAS2CUN
16:41:22 <dustymabe> so in the non OSTree case the sshd.socket file would continue to exist on the system?
16:41:28 <smooge> and yes that is the ticket in the thread
16:42:11 <dustymabe> or would it get cleaned up even in a pure rpm/dnf upgrade from 38->39 ?
16:42:23 <travier> It would be remove as it's part of /usr
16:42:25 <smooge> i think it gets cleaned up
16:42:55 <dustymabe> ok so the problem isn't ostree specific where the socket will go away - that always helps when framing the discussion
16:42:56 <travier> Commiting this without a change or a migration plan is not OK from my perspective.
16:43:20 <dustymabe> yeah, I mean, could people lose access to their systems after upgrade?
16:43:25 <travier> (I requested the change but did not act on it exactly for this reason. It's not easy to do the transition)
16:43:40 <travier> very much yes if they rely on this for ssh
16:44:09 <dustymabe> I guess we'd need to take our concerns to FESCO to get the change rejected and ask to be resubmitted for F40?
16:45:46 <dustymabe> does anyone know the process for something like this?
16:46:08 <dustymabe> FESCO meets on thursdays IIUC
16:46:28 * dustymabe checks his microphone
16:46:53 <travier> we can file a bug about it and mark it as a blocker for F39
16:46:55 <smooge> dustymabe: open a ticket with FESCO and put a flag it needs to be discussed
16:47:09 <dustymabe> smooge: flag == label? do you know what label to use?
16:47:11 <smooge> and propose it as a blocker bug
16:47:17 <dustymabe> travier: could you file that ticket?
16:47:53 <smooge> dustymabe: looks like the Tags: meeting
16:48:04 <travier> Sure, will do. The irony of me fillig the first bug and the ticket likely won't be lost :)
16:48:08 <dustymabe> in our communications here we can be positive - i.e. "we think it would be better if" and then see how the discussion goes
16:48:12 <dustymabe> travier: haha
16:48:13 <smooge> https://pagure.io/fesco/issues
16:48:21 <dustymabe> i can file it if you think that would detract from the goal
16:48:39 <travier> I'll write it and let you file it :)
16:48:44 <dustymabe> or maybe it would actually help.. i.e. "I requested this, but this isn't the way"
16:48:52 <travier> sure, both works
16:49:12 <smooge> I requested this and have learned once again the path to broken systems is paved with good intentions
16:49:13 <dustymabe> #action dustymabe or travier to file a fesco ticket to see if the change should be moved to F40
16:49:20 <dustymabe> smooge: :)
16:49:31 <travier> smooge: +1
16:49:33 <dustymabe> travier: let's coordinate after the meeting
16:49:43 <travier> 👍
16:50:04 <dustymabe> so if we move past "when" - this change will likely come to either F39 or F40 - what should we do?
16:50:34 <dustymabe> I feel like we can't solve every problem created by package maintainers
16:50:40 <travier> For F40, we can announce it or make a CLHM to check for the unit going away being enabled and not the service
16:50:48 <dustymabe> +1
16:51:03 <travier> for F39 that's short notice / short development time
16:51:04 <dustymabe> i.e. we don't try to actually fix anything or keep the old functionality, but we give ample warning
16:51:26 <dustymabe> right. if it did land with f39 I guess we could keep the socket for a notice period and then drop
16:51:50 <travier> yes, would be fairly easy to keep it for a release and announce the change
16:52:10 <travier> (It's already "Fedora" specifc)
16:52:46 <dustymabe> #proposed we will open a ticket with FESCO to see if this change would be better suited for Fedora 40. If it lands in F40 then we'll publish a CLHM helper to let people know the change is coming. If it lands in F39 we'll keep the socket for a notice period and warn people of the coming change.
16:53:00 <travier> +
16:53:02 <travier> +1
16:53:05 <spresti> +1
16:53:10 <marmijo> `+1
16:53:11 <aaradhak> +1
16:53:16 <ravanelli> +1
16:53:25 <dustymabe> #agreed we will open a ticket with FESCO to see if this change would be better suited for Fedora 40. If it lands in F40 then we'll publish a CLHM helper to let people know the change is coming. If it lands in F39 we'll keep the socket for a notice period and warn people of the coming change.
16:53:42 <dustymabe> #topic New Package Request: gvisor-tap-vsock
16:53:46 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1557
16:54:03 <dustymabe> looks like the podman folks requested this one
16:54:48 <dustymabe> travier: how big were those files in https://github.com/coreos/fedora-coreos-tracker/issues/1557#issuecomment-1699368752 ?
16:55:46 <travier> I've not extracted them. Doing it now
16:56:57 <travier> https://github.com/coreos/fedora-coreos-tracker/issues/1557#issuecomment-1699535692
16:57:05 <travier> big :(
16:57:22 <dustymabe> the download was `Will download: 1 package (5.4 MB)`
16:57:34 <travier> │       ├── [-rwxr-xr-x tim      tim       5.6M]  gvforwarder
16:57:34 <travier> │       └── [-rwxr-xr-x tim      tim        12M]  gvproxy
16:58:00 <spresti> So whats the context here. what is good and bad in these types of circumstance
16:58:10 <spresti> size wise
16:58:46 <travier> In general we try to limit the size of the image as much as possible
16:59:19 <travier> if I understand correctly as this is needed for podman to support hyper-v
16:59:20 <dustymabe> yeah, man golang and rust are going to be the death of antyhing small
16:59:34 <travier> so the size is not really an criteria :/
17:00:00 <dustymabe> travier: well. it's something to consider - i.e. how can we improve in the future?
17:00:40 <travier> sure, but it's not much up to us
17:01:27 <dustymabe> i'm kinda of tired of various networking pieces getting moved around in podman land (and I'm sure they are tired of it too)
17:02:33 <dustymabe> i guess I have some questions for the maintainers about how we can reduce the size, but probably shouldn't gum this meeting up with that
17:03:19 <dustymabe> anybody with any other questions?
17:03:53 <dustymabe> I guess a question is "what functionality is missing today"?
17:04:13 <dustymabe> i think the answer is `podman-remote` like functionality on HyperV and AppleHV
17:04:29 <dustymabe> within the VM everything should work fine IIUC
17:04:38 <dustymabe> travier: does that match your understanding?
17:07:11 <travier> whoops sorry, reading
17:07:35 <dustymabe> #proposed In order to enable podman-remote like use cases on HyperV and AppleHV we will include the gvisor-tap-vsock package. The size of 5.6M for gvforwarder and 12M for gvproxy is a concern, though.
17:07:44 <travier> +1
17:08:03 <spresti> +1
17:08:22 <gursewak> +1
17:08:25 <ravanelli> +1
17:08:29 <travier> For usermode networking, this binary allows us to take traffic bound for a tap device and push it across a vsock to the host.
17:09:38 <travier> agree that I don't understand either why this is needed
17:09:50 <marmijo> +1
17:09:56 <dustymabe> travier: but this is a podman specific networking thing? the image (VM) pulls things from the network even if podman wasn't installed or used.
17:09:57 <travier> why can't https://github.com/containers/netavark do it?
17:10:21 <dustymabe> I asked a question over in https://github.com/coreos/fedora-coreos-tracker/issues/1557#issuecomment-1699551691
17:10:46 <dustymabe> I think this is probably so `podman-machine` from outside of the VM can talk to the inside of the VM somehow.
17:11:27 <dustymabe> should I proceed with the proposed -> agreed ? or should we get more information first?
17:11:33 <travier> or https://passt.top/passt/about/ that we already have
17:11:42 <dustymabe> travier: right
17:11:54 <dustymabe> i'm guessing those other solutions don't work with a vsock
17:12:28 <dustymabe> unfortunately this is the world we live in where Apple/Windows/Linux all have slightly different solutions for things :(
17:12:44 <dustymabe> #agreed In order to enable podman-remote like use cases on HyperV and AppleHV we will include the gvisor-tap-vsock package. The size of 5.6M for gvforwarder and 12M for gvproxy is a concern, though.
17:13:00 <dustymabe> ok I stamped ^^ but we'll continue discussion in the ticket for clarity
17:13:06 <travier> let's not block it but still ask for more details
17:13:26 <dustymabe> #topic LUKS Encryption fails after upgrade to 38.20230819.2.0 testing stream on bare metal installation
17:13:30 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1555
17:13:40 <dustymabe> ok I added the meeting label here
17:14:06 <dustymabe> IIUC what we have is a kernel regression that causes peoples encrypted disks to not be unencrypted on boot.
17:14:20 <spresti> Yikes
17:14:41 <travier> this is really not great
17:15:00 <dustymabe> I chimed in the upstream thread to link to our BZ where Fedora users are reporting the issue: https://lore.kernel.org/linux-kernel/20230818181516.19167-1-mario.limonciello@amd.com/T/#m2dbbe85190b5fb5c2a6e062bf6b8ba7f2934c310
17:15:12 <spresti> How can we test this in the future?
17:15:15 <dustymabe> was hoping a fix would have been published that we could have picked up
17:15:25 <dustymabe> spresti: unfortunately this one is hardware specific
17:15:40 <dustymabe> this is why it's useful for actual users to run `testing`/`next`
17:15:53 <travier> This is an important regression, we should notify Thorsten
17:16:03 <dustymabe> I did already :)
17:16:09 <travier> 👍
17:16:12 <dustymabe> it's on his radar
17:16:31 <spresti> Thank you for being on top of that dustymabe
17:16:44 <dustymabe> so.. I'm guessing we should actually revert the kernel in `testing`
17:16:49 <dustymabe> and `next`
17:17:16 <dustymabe> alternatively we don't revert in testing/next but we do prevent it from updating in stable
17:17:53 <dustymabe> this is further complicated by the fact that for a cycle we had a different kernel on ppc64le than the other arches
17:18:14 <dustymabe> so options:
17:18:37 <dustymabe> 1. revert kernel in testing/next and reintroduce the delta between ppc64le and others - ship new testing/next update
17:19:11 <dustymabe> 2. keep updated kernel in testing/next; do normal promotions next week, but DON'T promote the kernel in `stable`
17:20:00 <dustymabe> I think it's fairly safe to allow stable to differ here because we're freezing the kernel (i.e. not updating it), so if it was already running and people were happy then it's probably safe
17:20:04 <travier> I'd prefer we don't ship something in stable we did not have in testing
17:20:20 <travier> ah, it's the same version?
17:20:50 <dustymabe> yeah, in an ideal world, I agree. but.. doing a new set of releases is work AND this is probably fairly safe (keep in mind it will still have to pass CI when stable is built)
17:21:20 <ravanelli> What is the cons for 1? This delta case only?
17:21:39 <dustymabe> travier: `2.` is basically "normal promotions" but we add a commit to the promotion PR to revert the kernel update to what is currently in stable.
17:21:56 <travier> ravanelli: 1 is more work
17:22:07 <travier> as we need to do new testing/next releases
17:22:10 <dustymabe> the cons for 1. is more work basically - and we have to reintroduce the delta for ppc64le
17:22:56 <travier> aren't we going to re-introduce the delta if we revert the kernel in stable?
17:23:20 <dustymabe> well. the delta is currently there for stable - so "reintroduce" is a strong word
17:23:24 <travier> ok
17:23:38 <spresti> Additonally with keeping it in testing/next we get bennifit of other bugs that might be a problem
17:23:51 <travier> I'm good with 2. We'll have to be super careful when doing the release however :/
17:24:04 <spresti> Me to my vote is 2.
17:24:14 <spresti> two *cringe
17:24:23 <ravanelli> I'm ok with 2 as well
17:24:29 <dustymabe> yeah. in most cases I would go with 1. but in this one I say 2.
17:25:10 <spresti> ** too **omega cringe grammar
17:25:24 <travier> let's hope we don't have a critical fix in the kernel between the one we have in stable and the one in testing
17:26:04 <dustymabe> yeah, there's always some downside to freezing the kernel
17:26:12 <dustymabe> #proposed We won't revert the kernel in our testing/next streams but we will be careful and not introduce this regression to our stable stream for the next round of releases. We will monitor the upstream discussion surrounding the bug and try to deliver a fixed kernel in the next set of testing/next releases.
17:26:29 <travier> +1
17:26:31 <aaradhak> +1
17:26:41 <spresti> +1
17:26:45 <ravanelli> +1
17:27:16 <dustymabe> #agreed We won't revert the kernel in our testing/next streams but we will be careful and not introduce this regression to our stable stream for the next round of releases. We will monitor the upstream discussion surrounding the bug and try to deliver a fixed kernel in the next set of testing/next releases.
17:27:58 <dustymabe> #topic Handling of default users/groups
17:28:02 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/155
17:28:10 <dustymabe> travier: sorry it took me a while to get to this one
17:28:23 <dustymabe> two questions:
17:28:33 <dustymabe> 1. is it time sensitive?
17:28:46 <dustymabe> 2. do you think we can discuss it quickly?
17:28:55 <travier> next week, no problem
17:29:03 <dustymabe> ok we'll discuss it next week
17:29:06 <dustymabe> #topic open floor
17:29:14 <dustymabe> anyone with anything for open floor?
17:29:39 <travier> (the other issues were more of a priority, this last one isn't)
17:30:18 <dustymabe> I'll note we need to create a ticket for f39 test day/week and schedule/organize it. See first few bullet points in https://github.com/coreos/fedora-coreos-tracker/issues/1490
17:30:42 <dustymabe> anybody willing to help organize it? sumatro will usually help too
17:31:38 * dustymabe will close out the meeting in 45s unless discussion continues
17:32:20 <dustymabe> #endmeeting