2024-11-13 16:30:19 <@gurssing:matrix.org> !startmeeting fedora_coreos_meeting
2024-11-13 16:30:21 <@gurssing:matrix.org> !topic roll call
2024-11-13 16:30:21 <@meetbot:fedora.im> Meeting started at 2024-11-13 16:30:19 UTC
2024-11-13 16:30:21 <@meetbot:fedora.im> The Meeting name is 'fedora_coreos_meeting'
2024-11-13 16:31:00 <@siosm:matrix.org> !hi
2024-11-13 16:31:01 <@zodbot:fedora.im> Timothée Ravier (siosm) - he / him / his
2024-11-13 16:31:03 <@dustymabe:matrix.org> !hi
2024-11-13 16:31:05 <@zodbot:fedora.im> Dusty Mabe (dustymabe) - he / him / his
2024-11-13 16:31:08 <@gurssing:matrix.org> !hi gursewak
2024-11-13 16:31:10 <@zodbot:fedora.im> Gursewak Singh (gursewak)
2024-11-13 16:33:34 <@aaradhak:matrix.org> !hi aaradhak
2024-11-13 16:33:37 <@zodbot:fedora.im> Aashish Radhakrishnan (aaradhak)
2024-11-13 16:33:40 <@marmijo:fedora.im> !hi
2024-11-13 16:33:42 <@jlebon:fedora.im> !hi
2024-11-13 16:33:42 <@zodbot:fedora.im> Michael Armijo (marmijo)
2024-11-13 16:33:44 <@zodbot:fedora.im> None (jlebon)
2024-11-13 16:33:56 <@gurssing:matrix.org> !topic Action items from last meeting
2024-11-13 16:34:58 <@siosm:matrix.org> Hum, I can't find the minutes on the forum: https://discussion.fedoraproject.org/tags/c/project/7/coreos-wg
2024-11-13 16:35:08 <@gurssing:matrix.org> No action times from previous meeting.
2024-11-13 16:35:21 <@gurssing:matrix.org> Last week's meeting was cancelled
2024-11-13 16:35:31 <@gurssing:matrix.org> https://github.com/coreos/fcos-meeting-action/issues/117#issuecomment-2460304957
2024-11-13 16:35:54 <@siosm:matrix.org> https://discussion.fedoraproject.org/t/fedora-coreos-community-meeting-minutes-2024-10-30/134966 👍️
2024-11-13 16:36:07 <@gurssing:matrix.org> !topic FOSDEM 2025 - Brussels 1 & 2 February 2025
2024-11-13 16:36:14 <@gurssing:matrix.org> !link https://github.com/coreos/fedora-coreos-tracker/issues/1829
2024-11-13 16:37:02 <@siosm:matrix.org> This one is just a general reminder if folks want to submit talks to FOSDEM
2024-11-13 16:37:16 <@gurssing:matrix.org> No action items from previous meeting.
2024-11-13 16:37:46 <@siosm:matrix.org> There is a devroom specific for image based systems were we could talk about things happening in FCOS
2024-11-13 16:37:58 <@dustymabe:matrix.org> would definitely be cool to go to it..
2024-11-13 16:38:18 <@siosm:matrix.org> (I don't have an idea yet but I'm likely to go to FOSDEM this (next) year so feel free if you have suggestions
2024-11-13 16:38:26 <@siosm:matrix.org> I don't have an idea yet but I'm likely to go to FOSDEM this (next) year so feel free if you have suggestions
2024-11-13 16:38:54 <@apiaseck:matrix.org> !hi
2024-11-13 16:38:56 <@zodbot:fedora.im> Adam Piasecki (c4rt0) - he / him / his
2024-11-13 16:39:54 <@gurssing:matrix.org> Moving on.
2024-11-13 16:39:58 <@gurssing:matrix.org> !topic Migrate 'coreos/fedora-coreos-tracker' to 'Webhook To Fedora Messaging'
2024-11-13 16:40:06 <@gurssing:matrix.org> !link https://github.com/coreos/fedora-coreos-tracker/issues/1826
2024-11-13 16:41:15 <@dustymabe:matrix.org> I guess this one needs a volunteer?
2024-11-13 16:41:27 <@dustymabe:matrix.org> travier: added the meeting label
2024-11-13 16:42:44 <@jlebon:fedora.im> re. https://github.com/coreos/fedora-coreos-tracker/issues/1826#issuecomment-2470857671 i meant more whether the sync2jira folks will do a bulk request for all the repos hooked into it, but it seems safer to just do it ourselves since we have to request for f-c-c anyway
2024-11-13 16:42:48 <@siosm:matrix.org> yes, mostly need a volunteer to file the issue. At the time I had no idea what to do about it 😅
2024-11-13 16:43:37 <@dustymabe:matrix.org> i wonder if sync2jira can work with the new webhooks?
2024-11-13 16:43:53 <@dustymabe:matrix.org> i.e. if we "migrate" will we lose sync2jira for a period of time
2024-11-13 16:43:57 <@siosm:matrix.org> Should we update the list of repos hooked / add this to our repo-migration-to-coreos checklist?
2024-11-13 16:44:21 <@jlebon:fedora.im> dustymabe: do you have a link to the sync2jira codebase handy?
2024-11-13 16:44:44 <@dustymabe:matrix.org> one sec
2024-11-13 16:44:58 <@dustymabe:matrix.org> https://github.com/release-engineering/Sync2Jira I think
2024-11-13 16:46:20 <@siosm:matrix.org> What I don't understand is how we can get synced issues for repos that we have in the sync2jira config but not in the list here
2024-11-13 16:46:58 <@dustymabe:matrix.org> travier do you have an example?
2024-11-13 16:47:16 <@dustymabe:matrix.org> like ostreedev/ostree ?
2024-11-13 16:47:33 <@siosm:matrix.org> https://issues.redhat.com/browse/COS-2813
2024-11-13 16:47:44 <@jlebon:fedora.im> dustymabe: hmm, ISTM like they're still using fedmsg
2024-11-13 16:47:48 <@siosm:matrix.org> ah no, it's in the list
2024-11-13 16:48:02 <@siosm:matrix.org> https://issues.redhat.com/browse/COS-2813
2024-11-13 16:48:19 <@dustymabe:matrix.org> Jonathan Lebon: exactly.
2024-11-13 16:48:34 <@jlebon:fedora.im> https://github.com/release-engineering/Sync2Jira/blob/6f26d69f013dc617e21efe200d904ec91245a56e/sync2jira/main.py#L34-L35
2024-11-13 16:48:38 <@dustymabe:matrix.org> I honestly think a requirement before they shutdown github2fedmsg is that sync2jira continue to work
2024-11-13 16:48:59 <@jlebon:fedora.im> so yeah, that's kind of a bumper.
2024-11-13 16:48:59 <@jlebon:fedora.im> yeah, agreed
2024-11-13 16:49:28 <@dustymabe:matrix.org> I guess let's take that feedback to the ticket?
2024-11-13 16:50:28 <@dustymabe:matrix.org> the other thing is.. what updates (if any) do we need to make to coreos-koji-tagger before f-c-c gets migrated?
2024-11-13 16:51:44 <@jlebon:fedora.im> AIUI, it should keep working as is
2024-11-13 16:53:22 <@dustymabe:matrix.org> ok done with this topic?
2024-11-13 16:53:41 <@gurssing:matrix.org> !topic Migrate existing systems to iptables-nftand removeiptables-legacy``
2024-11-13 16:53:46 <@gurssing:matrix.org> !link https://github.com/coreos/fedora-coreos-tracker/issues/1818
2024-11-13 16:54:02 <@siosm:matrix.org> This one has some history.
2024-11-13 16:54:56 <@siosm:matrix.org> For a long time, the alternatives command did not work on ostree based systems as part of the config was stored in /var and was thus mounted over when the /var partition was mounted.
2024-11-13 16:56:22 <@siosm:matrix.org> Now, that has been fixed in F41 in the alternatives program, but it's not completely automatic for existing installations. No "migration" was included, i.e. if you ever used the alternative configuration in the old place, it will keep things there.
2024-11-13 16:57:07 <@siosm:matrix.org> So, old Fedora CoreOS nodes are likely still using the iptables-legacy backend and not the nft one
2024-11-13 16:58:00 <@siosm:matrix.org> Ideally we would migrate everyone, and then declare the legacy backend as deprecated, and remove it when we rebase to F42
2024-11-13 17:00:03 <@siosm:matrix.org> EndOfIntro
2024-11-13 17:00:27 <@dustymabe:matrix.org> you want to migrate everyone in the middle of a release?
2024-11-13 17:00:29 <@jlebon:fedora.im> so right now, we manually set the symlinks so that iptables-nft is used. does the new alternatives code understand how things are set up currently?
2024-11-13 17:00:53 <@jlebon:fedora.im> dustymabe: i understood f42
2024-11-13 17:01:03 <@siosm:matrix.org> Yes, I think we should migrate during the F41 cycle and deprecate for F42.
2024-11-13 17:01:26 <@siosm:matrix.org> The iptables change should really be safe. It's been the default for a while everywhere now
2024-11-13 17:02:10 <@siosm:matrix.org> we manually set the symlink in the part that is in /etc/alternatives, which is the "state" and the config is now in /etc/alternatives-admindir when it was in /var/lib/alternatives before
2024-11-13 17:02:35 <@jlebon:fedora.im> basically, would there be any delta at all in "node state" between a new node that started on nft, and one that was migrated
2024-11-13 17:03:19 <@siosm:matrix.org> the trick is that the new alternatives only use the new config admindir if the older /var/lib one does not exists
2024-11-13 17:03:28 <@siosm:matrix.org> and users could have placed configs there manually
2024-11-13 17:03:53 <@siosm:matrix.org> so we would have to `mv /var/lib/alternatives /etc/alternatives-admindir` which is not really "safe"
2024-11-13 17:05:19 <@siosm:matrix.org> Overall, it's very unlikely to be the case (that anyone set a custom alternatives config) so we could also say we don't care and we break it but that means removing /var/lib/alternatives
2024-11-13 17:05:26 <@siosm:matrix.org> should be safe if empty
2024-11-13 17:05:38 <@jlebon:fedora.im> hmm, but the /var one is a symlink to /usr, so they would've had to purposely break the symlink
2024-11-13 17:05:45 <@siosm:matrix.org> Maybe this calls for a clhm
2024-11-13 17:06:02 <@siosm:matrix.org> it's a symlink only on new systems
2024-11-13 17:06:19 <@siosm:matrix.org> on older systems it's a plain dir
2024-11-13 17:06:31 <@jlebon:fedora.im> fun
2024-11-13 17:07:04 <@siosm:matrix.org> It's even a broken symlink on my Kinoite system right now
2024-11-13 17:07:26 <@siosm:matrix.org> (I argued for the migration to happen in the alternatives command but this did not get traction so here we are)
2024-11-13 17:07:51 <@siosm:matrix.org> we'll have the same issue for Atomic Desktops obviously, and IoT, etc.
2024-11-13 17:08:28 <@jlebon:fedora.im> apart from iptables, do we even have other things that use it in the base? ISTM more likely that any use of alternatives by users would be from layered pkgs
2024-11-13 17:08:29 <@dustymabe:matrix.org> so we'd want to ship the migration script in all of those places?
2024-11-13 17:09:53 <@siosm:matrix.org> The main thing for us is iptables. The rest would be layered packages indeed
2024-11-13 17:10:06 <@siosm:matrix.org> > so we'd want to ship the migration script in all of those places?
2024-11-13 17:10:06 <@siosm:matrix.org> yes, ideally
2024-11-13 17:10:14 <@siosm:matrix.org>
2024-11-13 17:10:14 <@siosm:matrix.org> > so we'd want to ship the migration script in all of those places?
2024-11-13 17:10:14 <@siosm:matrix.org> yes, ideally
2024-11-13 17:10:25 <@siosm:matrix.org> yes, ideally
2024-11-13 17:10:25 <@siosm:matrix.org> > so we'd want to ship the migration script in all of those places?
2024-11-13 17:10:33 <@siosm:matrix.org>
2024-11-13 17:10:33 <@siosm:matrix.org> > so we'd want to ship the migration script in all of those places?
2024-11-13 17:10:33 <@siosm:matrix.org> yes, ideally
2024-11-13 17:11:40 <@dustymabe:matrix.org> is there a precedence?
2024-11-13 17:12:04 <@dustymabe:matrix.org> `/var/lib/alternatives` versus `/etc/alternatives-admindir` ?
2024-11-13 17:12:38 <@siosm:matrix.org> /var/lib/alternatives is always used if available
2024-11-13 17:12:53 <@siosm:matrix.org> https://github.com/fedora-sysv/chkconfig/pull/135/files#diff-562b9b19cb1cd12a7343ce5c739745ebc8f363a195276ca58e926f22927238a5R1474
2024-11-13 17:14:31 <@dustymabe:matrix.org> ```
2024-11-13 17:14:31 <@dustymabe:matrix.org> ```
2024-11-13 17:14:31 <@dustymabe:matrix.org> ok. so the migration would be something like:
2024-11-13 17:14:31 <@dustymabe:matrix.org>
2024-11-13 17:14:31 <@dustymabe:matrix.org> mv /var/lib/alternatives/* /etc/alternatives-admindir/
2024-11-13 17:14:31 <@dustymabe:matrix.org> rmdir /var/lib/alternatives
2024-11-13 17:14:31 <@dustymabe:matrix.org> ln -s /var/lib/alternatives
2024-11-13 17:14:55 <@siosm:matrix.org> (I've just verified that iptables is the only command actually using alternatives on FCOS)
2024-11-13 17:15:17 <@siosm:matrix.org> something like that yes
2024-11-13 17:15:34 <@siosm:matrix.org> ln -s /etc/alternatives-admindir/ /var/lib/alternatives
2024-11-13 17:15:38 <@dustymabe:matrix.org> but I guess only do the first step IFF /var/lib/alternatives isn't already a symlink
2024-11-13 17:15:53 <@siosm:matrix.org> yes
2024-11-13 17:16:33 <@siosm:matrix.org> or we don't create the symlink at all in the end
2024-11-13 17:17:08 <@siosm:matrix.org> The risk is that we don't know what users may have placed there
2024-11-13 17:17:10 <@dustymabe:matrix.org>
2024-11-13 17:17:10 <@dustymabe:matrix.org> ```
2024-11-13 17:17:10 <@dustymabe:matrix.org> $ ls -l /var/lib/alternatives
2024-11-13 17:17:10 <@dustymabe:matrix.org> ls: cannot access '/var/lib/alternatives': No such file or directory
2024-11-13 17:17:10 <@dustymabe:matrix.org> ```
2024-11-13 17:17:10 <@dustymabe:matrix.org> fresh FCOS f42 system:
2024-11-13 17:17:28 <@siosm:matrix.org> It would have been a bad idea to place data there but we don't know
2024-11-13 17:18:00 <@siosm:matrix.org> For 99% maybe of the systems it should be empty so we could optimize for this case and "just" delete the folder
2024-11-13 17:18:19 <@siosm:matrix.org> The other systems we could write a CLHM that would check and warn
2024-11-13 17:18:44 <@dustymabe:matrix.org> works for me..
2024-11-13 17:19:04 <@dustymabe:matrix.org> what happens if they ignore the warning? they stay on iptables legacy and eventually when we remove it they are broke?
2024-11-13 17:19:53 <@siosm:matrix.org> yes, it will break the commands as they won't be set to the new backend by the migration script
2024-11-13 17:20:13 <@siosm:matrix.org> or we do a forced manual migration to compensate for that as well
2024-11-13 17:21:18 <@siosm:matrix.org> 3. migrate using alternatives / force migrate systems where alternatives config is broken
2024-11-13 17:21:18 <@siosm:matrix.org> 2. Warn, if not empty
2024-11-13 17:21:18 <@siosm:matrix.org> 1. rmdir /var/lib/alternatives if empty
2024-11-13 17:22:13 <@siosm:matrix.org> Ah, we can force the admindir in the alterantives call so we could use that
2024-11-13 17:22:53 <@siosm:matrix.org> Alright, I'll suggest a plan that should be safe based on the above
2024-11-13 17:23:30 <@jlebon:fedora.im> seems reasonable to me 👍️
2024-11-13 17:23:52 <@dustymabe:matrix.org> sounds good. maybe put it in the ticket and we can discuss more
2024-11-13 17:24:03 <@dustymabe:matrix.org> the more detail the better :)
2024-11-13 17:24:45 <@siosm:matrix.org> Let's move to open floor :)
2024-11-13 17:24:53 <@gurssing:matrix.org> !topic Open Floor
2024-11-13 17:25:19 <@dustymabe:matrix.org> !info FYI stable FCOS nodes are migrating to F41 this week
2024-11-13 17:26:12 <@dustymabe:matrix.org> Any volunteers would be welcome :)
2024-11-13 17:26:12 <@dustymabe:matrix.org> There are some followup items for F41 that need to be completed (see checklist in https://github.com/coreos/fedora-coreos-tracker/issues/1695)
2024-11-13 17:26:12 <@dustymabe:matrix.org>
2024-11-13 17:27:12 <@marmijo:fedora.im> I can take care of closing out the last items
2024-11-13 17:27:39 <@gurssing:matrix.org> Can help out as well.
2024-11-13 17:28:31 <@gurssing:matrix.org> If there isn't anything more, I will close the meeting in about a minute.
2024-11-13 17:29:15 <@gurssing:matrix.org> !endmeeting