16:30:42 <dustymabe> #startmeeting fedora_coreos_meeting
16:30:42 <zodbot> Meeting started Wed Jul 27 16:30:42 2022 UTC.
16:30:42 <zodbot> This meeting is logged and archived in a public location.
16:30:42 <zodbot> The chair is dustymabe. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
16:30:42 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:42 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:47 <dustymabe> #topic roll call
16:30:47 <travier> .hello siosm
16:30:50 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com>
16:30:50 <dustymabe> .hi
16:30:52 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:02 <travier> 👋👋👋
16:31:05 <lorbus> .hi
16:31:06 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:31:10 <mnguyen_> .hi
16:31:11 <zodbot> mnguyen_: Sorry, but user 'mnguyen_' does not exist
16:31:16 <mnguyen_> .hi mnguyen
16:31:17 <zodbot> mnguyen_: Sorry, but user 'mnguyen_' does not exist
16:31:21 <mnguyen_> .hello mnguyen
16:31:22 <ravanelli> .hi
16:31:22 <zodbot> mnguyen_: mnguyen 'Michael Nguyen' <mnguyen@redhat.com>
16:31:25 <zodbot> ravanelli: ravanelli 'Renata Ravanelli' <renata.ravanelli@gmail.com>
16:31:32 <jlebon> .hello2
16:31:33 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:31:57 <mnguyen_> i went to jlebon.com and got 404
16:32:29 <bgilbert> .hi
16:32:30 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:32:34 <dustymabe> #chair travier lorbus mnguyen_ ravanelli jlebon
16:32:34 <zodbot> Current chairs: dustymabe jlebon lorbus mnguyen_ ravanelli travier
16:32:58 <dustymabe> #chair bgilbert
16:32:58 <zodbot> Current chairs: bgilbert dustymabe jlebon lorbus mnguyen_ ravanelli travier
16:33:13 <lucab> .hi
16:33:14 <zodbot> lucab: lucab 'Luca BRUNO' <lucab@redhat.com>
16:33:24 <dustymabe> #chair lucab
16:33:24 <zodbot> Current chairs: bgilbert dustymabe jlebon lorbus lucab mnguyen_ ravanelli travier
16:33:49 <gursewak> .hi
16:33:50 <zodbot> gursewak: gursewak 'Gursewak Singh' <gurssing@redhat.com>
16:34:20 <dustymabe> #chair gursewak
16:34:20 <zodbot> Current chairs: bgilbert dustymabe gursewak jlebon lorbus lucab mnguyen_ ravanelli travier
16:34:23 <dustymabe> welcome all
16:34:30 <dustymabe> will get started in just a moment
16:34:54 <jdoss> .hi
16:34:55 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
16:35:03 <dustymabe> #chair jdoss
16:35:03 <zodbot> Current chairs: bgilbert dustymabe gursewak jdoss jlebon lorbus lucab mnguyen_ ravanelli travier
16:35:09 <dustymabe> #topic Action items from last meeting
16:35:18 <dustymabe> #info no action items from last meeting!
16:35:21 <dustymabe> \o/
16:35:34 <jdoss> hurray
16:35:36 <travier> 🎉
16:36:13 <dustymabe> moving on to meeting tickets. reminder to add the `meeting` label to tickets that need discussion
16:36:26 <dustymabe> #topic New Package Request: mstflint
16:36:32 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/1264
16:37:06 <dustymabe> this is a fairly new ticket.. we might not have enough information to make a decision today but I figured it was worth bringing up. walters is having some good back and forth in the ticket with the requestors
16:37:40 <dustymabe> one thing I notice in the dep list is that it pulls in python
16:38:16 <lucab> the package is bundling a few binaries, some in C and some in Python
16:39:53 <dustymabe> it seems like the software is used to "unlock" advanced functionality for the NIC cards but it's not required in order for the machine to come up and access network to begin with
16:39:56 <dustymabe> #chair aaradhak
16:39:56 <zodbot> Current chairs: aaradhak bgilbert dustymabe gursewak jdoss jlebon lorbus lucab mnguyen_ ravanelli travier
16:39:57 <aaradhak> .hi
16:39:59 <zodbot> aaradhak: aaradhak 'Aashish Radhakrishnan' <aaradhak@redhat.com>
16:41:06 <dustymabe> combined that with the fact that it only affects specific brand of hardware, this kind of seems like a candidate for package layering or run from container IMO
16:41:20 <jdoss> Can't they just pull a container with it and use it after the fact? I have been doing a lot of pull a container and copy the binaries out of it that I need to run on the host node.
16:41:39 <jlebon> dustymabe: +1
16:42:28 <travier> +1
16:42:30 <lucab> (the binary they mention in the snippet is a C one)
16:42:51 <jlebon> running from a container might be more acceptable to upstream than package layering (to not have CoreOS-specific code)
16:42:51 <dustymabe> Previously we had a request for installing "wifi support" in the base image and we declined on that for now. I think this would arguably help much fewer users so if we said no to wifi we should probably say no to this too
16:43:42 <jdoss> Wifi makes more sense than this.
16:44:40 <dustymabe> any more thoughts on this?
16:45:08 <bgilbert> I'm disinclined
16:45:32 <jlebon> i think we can let the discussions simmer in the ticket for now and see what they say re. running from a container
16:45:35 <bgilbert> +1
16:45:42 <dustymabe> the issue is pretty fresh so it might be premature to make a decision while new information is still coming in, but I think a summary of our discussion here in the ticket and maybe a vote next week could be useful?
16:45:48 <dustymabe> jlebon: +1 ^^
16:46:06 <travier> +1
16:46:36 <lucab> I'm unsure why not letting NetworkManager/systemd-networkd taking care of card setup, instead of doing it manually in a script
16:47:01 <jdoss> can those interface with the card correctly?
16:47:17 <jdoss> Sounds like some specialized stuff going on specific to the card.
16:47:18 <dustymabe> lucab: I think the tool unlocks some advanced features of the NIC, which NetworkManager probably doesn't know how to or doesn't want to control.
16:47:35 <jlebon> yeah, might be seen as out of scope for NM
16:48:14 <dustymabe> ok, I'll add a summary to the ticket and mention we'll revisit in our next meeting
16:48:19 <jlebon> +1
16:48:28 <dustymabe> #topic NetworkManager: consider defaulting to EUI-64 for IPv6 SLAAC (at least on OpenStack)
16:48:33 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/907
16:48:56 <dustymabe> new information.. the global configuration knob for this has landed in NM in Fedora 37
16:49:10 <dustymabe> so now we have an opportunity to use it
16:49:48 <dustymabe> Some time ago I had a meeting with systemd and NetworkManager teams: https://github.com/coreos/fedora-coreos-tracker/issues/907#issuecomment-1119911839
16:50:16 <dustymabe> at least some people from NM preferred to suggest a change for all of Fedora to switch to eui64 for server like variants
16:50:28 <dustymabe> preferred to NOT suggest a change*
16:50:43 <dustymabe> so we decided not to do a change proposal for Fedora 37
16:51:04 <dustymabe> but they did implement the configuration knob for the "vendor" (FCOS in this case) to use
16:51:29 <dustymabe> I think the open question is: should we apply this FCOS-wide or should we only do this for our openstack image?
16:51:46 <bgilbert> why would we do it only for one image?
16:51:49 <jlebon> what was the reasoning to not change other server variants?
16:51:53 <lorbus> This has security implications, i.e. devices your network can possibly be tracked if only one device on this LAN uses this setting. I'd vote for doing this only where absolutely necessary.
16:52:17 <dustymabe> lorbus: servers typically don't move very often :)
16:52:52 <dustymabe> bgilbert: openstack is the one where the web interface shows a different address than reality: https://github.com/coreos/fedora-coreos-tracker/issues/907#issuecomment-892889810
16:53:10 <lorbus> right, but they might be collocated with other things on a private network that you don't want exposed (or trackable). There was an article about this on a German news website recently: https://www.heise.de/news/Trotz-Privacy-Extensions-IPv6-Adressen-fuer-andauerndes-Tracking-nutzbar-7186203.html
16:53:21 <dustymabe> jlebon: you can see the reasoning in https://github.com/coreos/fedora-coreos-tracker/issues/907#issuecomment-1119911839
16:53:21 <jdoss> lorbus is tracking me right now...
16:53:40 <lorbus> ;)
16:53:58 <dustymabe> but mostly from the NM person's side it was 1. security (as lorbus mentioned) 2. hardware replaceability issues
16:54:15 <dustymabe> I think those are kind of weak arguments for our use case, but I understand why they didn't want to propose a Fedora Change for it
16:54:49 <travier> How would we even do it only for OpenStack?
16:55:10 <dustymabe> travier: I can think of ways.. some better than others
16:55:19 <dustymabe> most hacky is probably a systemd-generator
16:55:19 <travier> Maybe this will be doc only as it's fairly small
16:55:36 <jlebon> dustymabe: do i understand correctly from https://github.com/coreos/fedora-coreos-tracker/issues/907#issuecomment-895455009 that cloud-init already uses EUI-64 by virtue of using ifcfg?
16:55:45 <dustymabe> do we have the ability to have platform specific ignition fragments yet?
16:56:06 <jlebon> sorry I mean Fedora Cloud
16:56:42 <dustymabe> jlebon: yes.. at the time that was written. Fedora Cloud base did switch to keyfiles recently and I think it needs to be checked to see if that still applies
16:57:33 <jlebon> so it's possible Fedora Cloud in OpenStack today suffers the same issue?
16:57:49 <dustymabe> correct, possible
16:58:21 <bgilbert> dustymabe: yes, we have for a long time
16:58:43 <dustymabe> bgilbert: thanks, good to know
16:59:10 <jlebon> I don't know much about IPv6, but for the OpenStack case, is the expectation that the host gets the floating IP via DHCPv6 or is it in the metadata and they have to self-configure?
16:59:45 <dustymabe> I think it may depend on how the openstack env was set up
17:00:04 <lucab> it's SLAAC, it's auto-configured on the client side
17:00:23 <dustymabe> but in the basic case (no DHCPv6) I think it assumes the VM will just autoconfigure it's address from the MAC and since openstack knows the MAC it can derive the ipv6 addr
17:00:43 <dustymabe> 'stable-privacy' breaks this assumption
17:00:56 <jlebon> lucab: does floating IP in this case work the same way as for IPv4 where it's not actually visible to the host?
17:01:49 <lucab> that I don't know
17:02:18 <dustymabe> there is one more piece of information to consider here
17:02:59 <jlebon> i guess what i'm trying to understand is: is there a way to fix the OpenStack case without turning on EUI-64?
17:03:19 <dustymabe> because of some history in NM, nm-initrd-generator basically produces connection files that will choose eui64 by default. dynamically generated "Wired Connection 1" will use stable-privacy
17:03:54 <dustymabe> so if a user adds some networking configuration via kernel arguments they will get eui64
17:04:05 <dustymabe> whereas if they just go with all defaults they get stable-privacy
17:04:18 <dustymabe> this is true today in FCOS (and has been for some time)
17:04:57 <dustymabe> if we do choose to use this new global configuration knob to set a global default. this inconsistency could be removed
17:05:08 <dustymabe> *I think*
17:05:32 <dustymabe> so our options are:
17:05:37 <dustymabe> 1. do nothing, add docs
17:05:53 <dustymabe> 2. set the global default for openstack
17:05:59 <dustymabe> 3. set the global default for all FCOS
17:06:13 * dustymabe is reminded that this has implications for RHCOS too
17:06:36 <dustymabe> well.. it may take some time for RHCOS to get the new global configuration knob
17:07:35 <jlebon> for docs, could this just be a dropin via Ignition to configure it ?
17:07:51 <dustymabe> yes
17:08:40 <dustymabe> but one could also argue that the user previously had the choice to just write out the nmconnection keyfile and set it there too (so the new knob doesn't really help us much there)
17:08:58 <lucab> jlebon: looking again at openstack docs, I think they don't support floating IPv6. And when using SLAAC (i.e. no DHCPv6 server) they rely on EUI-64 client-side
17:09:25 <jlebon> lucab: heh was looking at the docs too. yes I think I agree.
17:09:51 <jlebon> so this is only required if OpenStack is configured this way
17:10:28 <jlebon> if there's a way to auto-detect when that's the case, then we could turn on eui64 just when needed
17:11:07 <jlebon> because it's being mandated by the platform
17:11:15 <dustymabe> I think there are a few goals here. 1. fix openstack 2. determine what we think is the right default for FCOS 3.possibly resolve the different between nm-initrd-generator and real root dynamically generated profile
17:12:42 <dustymabe> so.. what should we do?
17:13:20 <jlebon> i'd vote for doing this for openstack only but ideally at runtime only when we know it's required
17:14:28 <dustymabe> yeah I'm not entirely sure how we would determine that
17:14:49 <lucab> I don't think the "when we know it's required" part is feasible
17:16:01 <lucab> this gets set before doing DHCPv6 discovery, and infra relies on it after DHCPv6 is not successful
17:16:43 <dustymabe> I guess it's possible there might be some clue in the metadata service that we could trigger on
17:16:49 <dustymabe> barring that I think lucab is right
17:17:09 <jlebon> dustymabe: yeah, that's what i'm trying to figure out
17:17:34 <jlebon> because then we could frame this as "configuring the host based on platform metadata" and shove it in afterburn
17:18:10 <dustymabe> not sure if afterburn is the right place.. I was thinking probably a systemd generator
17:18:29 <dustymabe> either way let's maybe take the conversation back a bit higher level
17:18:36 <jlebon> possibly. but afterburn would be the right place for metadata querying
17:18:45 <jlebon> +1
17:18:50 <dustymabe> is anyone in favor of changing the global default for FCOS for all platforms?
17:19:16 <dustymabe> I don't think I had any strong advocates of that
17:20:00 <dustymabe> ok, so we'll narrow it down to openstack
17:20:17 <dustymabe> it sounds like we have support for changing it on openstack, but preference for changing it only when required
17:20:28 <lucab> I agree with Lubomir's comment, it isn't a great choice for a default even for servers
17:20:47 <dustymabe> lucab: can you be specific about "it"
17:20:52 <dustymabe> eui64 == "it" ?
17:20:59 <lucab> yes, EUI-64
17:21:17 <dustymabe> ok, so no change for all FCOS
17:21:34 <jlebon> +1
17:21:45 <dustymabe> so.. for openstack.. do we want to change it EVEN IF we can't do it only when required ?
17:21:48 <lorbus> +1
17:22:16 <dustymabe> or do we want to gather more information first?
17:22:25 <jlebon> note there's also ironic there where some of the same concerns could apply
17:22:46 <dustymabe> jlebon: i.e. the same assumptions being invalid?
17:22:52 <jlebon> maybe let's reach out to OSP SMEs about the "only when required" part
17:23:26 <jlebon> dustymabe: right. i'm not sure what the networking setup is with ironic-provisioned bare metals, but the "changing NICs" would be relevant there
17:23:59 <dustymabe> #action jlebon to reach out to OpenStack experts to see if we can detect when the platform is expecting machines to do IPV6 network configuration via SLAAC (to get eui64 based IPv6 addresses)
17:24:19 <dustymabe> jlebon: don't hate me!
17:24:31 <jlebon> dustymabe: :)  guess i deserve it
17:24:39 <dustymabe> ok one final piece here
17:24:53 <dustymabe> do we want to address the initrd versus real root discrepancy?
17:25:13 <dustymabe> i.e. we *could* set a global default of "stable privacy"
17:26:04 <dustymabe> and then our initrd generated configuration versus our dynamically generated real root configs would have consistency
17:26:09 <dustymabe> *I think*
17:27:09 <jlebon> that SGTM, though i think that could break some users today so would require documenting & announcing
17:27:56 <dustymabe> yeah. Let me summarize all of this in the ticket since we are getting close to time
17:28:09 <dustymabe> any final thoughts before open floor?
17:28:23 <travier> :D
17:28:29 <lorbus> dustymabe++
17:28:29 <zodbot> lorbus: Karma for dustymabe changed to 4 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:28:36 <dustymabe> #topic open floor
17:28:55 <jlebon> thanks dustymabe for digging into this complex obscure (to me) issue
17:29:06 <travier> dustymabe++
17:29:11 <dustymabe> #info Fedora's Flock/Nest conference is next week: https://communityblog.fedoraproject.org/nest-with-fedora-2022-registration-now-open/
17:30:05 <travier> Reviews for this change are welcomed: https://github.com/coreos/fedora-coreos-tracker/issues/1232
17:30:19 <travier> It's not FCOS but we could (or not) re use the same logic in FCOS
17:31:19 <dustymabe> I think for FCOS it isn't really worth the risk
17:31:31 <jlebon> travier++ cool that we'll have that in FSB
17:31:32 <dustymabe> I'm down for a CLHM helper
17:31:57 <travier> agree that it may not be interesting enough for fcos
17:31:59 <dustymabe> for other ostree variants they don't auto update IIUC, so more opportunity to address problems if they arise
17:32:49 * dustymabe will close out the meeting soon if other topics don't come up
17:33:34 <dustymabe> #endmeeting