<@gurssing:matrix.org>
16:30:19
!startmeeting fedora_coreos_meeting
<@gurssing:matrix.org>
16:30:21
!topic roll call
<@meetbot:fedora.im>
16:30:21
Meeting started at 2024-11-13 16:30:19 UTC
<@meetbot:fedora.im>
16:30:21
The Meeting name is 'fedora_coreos_meeting'
<@siosm:matrix.org>
16:31:00
!hi
<@zodbot:fedora.im>
16:31:01
Timothée Ravier (siosm) - he / him / his
<@dustymabe:matrix.org>
16:31:03
!hi
<@zodbot:fedora.im>
16:31:05
Dusty Mabe (dustymabe) - he / him / his
<@gurssing:matrix.org>
16:31:08
!hi gursewak
<@zodbot:fedora.im>
16:31:10
Gursewak Singh (gursewak)
<@aaradhak:matrix.org>
16:33:34
!hi aaradhak
<@zodbot:fedora.im>
16:33:37
Aashish Radhakrishnan (aaradhak)
<@marmijo:fedora.im>
16:33:40
!hi
<@jlebon:fedora.im>
16:33:42
!hi
<@zodbot:fedora.im>
16:33:42
Michael Armijo (marmijo)
<@zodbot:fedora.im>
16:33:44
None (jlebon)
<@gurssing:matrix.org>
16:33:56
!topic Action items from last meeting
<@siosm:matrix.org>
16:34:58
Hum, I can't find the minutes on the forum: https://discussion.fedoraproject.org/tags/c/project/7/coreos-wg
<@gurssing:matrix.org>
16:35:08
No action times from previous meeting.
<@gurssing:matrix.org>
16:35:21
Last week's meeting was cancelled
<@gurssing:matrix.org>
16:35:31
https://github.com/coreos/fcos-meeting-action/issues/117#issuecomment-2460304957
<@siosm:matrix.org>
16:35:54
https://discussion.fedoraproject.org/t/fedora-coreos-community-meeting-minutes-2024-10-30/134966 👍️
<@gurssing:matrix.org>
16:36:07
!topic FOSDEM 2025 - Brussels 1 & 2 February 2025
<@gurssing:matrix.org>
16:36:14
!link https://github.com/coreos/fedora-coreos-tracker/issues/1829
<@siosm:matrix.org>
16:37:02
This one is just a general reminder if folks want to submit talks to FOSDEM
<@gurssing:matrix.org>
16:37:16
No action items from previous meeting.
<@siosm:matrix.org>
16:37:46
There is a devroom specific for image based systems were we could talk about things happening in FCOS
<@dustymabe:matrix.org>
16:37:58
would definitely be cool to go to it..
<@siosm:matrix.org>
16:38:18
(I don't have an idea yet but I'm likely to go to FOSDEM this (next) year so feel free if you have suggestions
<@siosm:matrix.org>
16:38:26
I don't have an idea yet but I'm likely to go to FOSDEM this (next) year so feel free if you have suggestions
<@apiaseck:matrix.org>
16:38:54
!hi
<@zodbot:fedora.im>
16:38:56
Adam Piasecki (c4rt0) - he / him / his
<@gurssing:matrix.org>
16:39:54
Moving on.
<@gurssing:matrix.org>
16:39:58
!topic Migrate 'coreos/fedora-coreos-tracker' to 'Webhook To Fedora Messaging'
<@gurssing:matrix.org>
16:40:06
!link https://github.com/coreos/fedora-coreos-tracker/issues/1826
<@dustymabe:matrix.org>
16:41:15
I guess this one needs a volunteer?
<@dustymabe:matrix.org>
16:41:27
travier: added the meeting label
<@jlebon:fedora.im>
16:42:44
re. https://github.com/coreos/fedora-coreos-tracker/issues/1826#issuecomment-2470857671 i meant more whether the sync2jira folks will do a bulk request for all the repos hooked into it, but it seems safer to just do it ourselves since we have to request for f-c-c anyway
<@siosm:matrix.org>
16:42:48
yes, mostly need a volunteer to file the issue. At the time I had no idea what to do about it 😅
<@dustymabe:matrix.org>
16:43:37
i wonder if sync2jira can work with the new webhooks?
<@dustymabe:matrix.org>
16:43:53
i.e. if we "migrate" will we lose sync2jira for a period of time
<@siosm:matrix.org>
16:43:57
Should we update the list of repos hooked / add this to our repo-migration-to-coreos checklist?
<@jlebon:fedora.im>
16:44:21
dustymabe: do you have a link to the sync2jira codebase handy?
<@dustymabe:matrix.org>
16:44:44
one sec
<@dustymabe:matrix.org>
16:44:58
https://github.com/release-engineering/Sync2Jira I think
<@siosm:matrix.org>
16:46:20
What I don't understand is how we can get synced issues for repos that we have in the sync2jira config but not in the list here
<@dustymabe:matrix.org>
16:46:58
travier do you have an example?
<@dustymabe:matrix.org>
16:47:16
like ostreedev/ostree ?
<@siosm:matrix.org>
16:47:33
https://issues.redhat.com/browse/COS-2813
<@jlebon:fedora.im>
16:47:44
dustymabe: hmm, ISTM like they're still using fedmsg
<@siosm:matrix.org>
16:47:48
ah no, it's in the list
<@siosm:matrix.org>
16:48:02
<del>https://issues.redhat.com/browse/COS-2813</del>
<@dustymabe:matrix.org>
16:48:19
Jonathan Lebon: exactly.
<@jlebon:fedora.im>
16:48:34
https://github.com/release-engineering/Sync2Jira/blob/6f26d69f013dc617e21efe200d904ec91245a56e/sync2jira/main.py#L34-L35
<@dustymabe:matrix.org>
16:48:38
I honestly think a requirement before they shutdown github2fedmsg is that sync2jira continue to work
<@jlebon:fedora.im>
16:48:59
so yeah, that's kind of a bumper.
<@jlebon:fedora.im>
16:48:59
yeah, agreed
<@dustymabe:matrix.org>
16:49:28
I guess let's take that feedback to the ticket?
<@dustymabe:matrix.org>
16:50:28
the other thing is.. what updates (if any) do we need to make to coreos-koji-tagger before f-c-c gets migrated?
<@jlebon:fedora.im>
16:51:44
AIUI, it should keep working as is
<@dustymabe:matrix.org>
16:53:22
ok done with this topic?
<@gurssing:matrix.org>
16:53:41
!topic Migrate existing systems to iptables-nftand removeiptables-legacy``
<@gurssing:matrix.org>
16:53:46
!link https://github.com/coreos/fedora-coreos-tracker/issues/1818
<@siosm:matrix.org>
16:54:02
This one has some history.
<@siosm:matrix.org>
16:54:56
For a long time, the alternatives command did not work on ostree based systems as part of the config was stored in /var and was thus mounted over when the /var partition was mounted.
<@siosm:matrix.org>
16:56:22
Now, that has been fixed in F41 in the alternatives program, but it's not completely automatic for existing installations. No "migration" was included, i.e. if you ever used the alternative configuration in the old place, it will keep things there.
<@siosm:matrix.org>
16:57:07
So, old Fedora CoreOS nodes are likely still using the iptables-legacy backend and not the nft one
<@siosm:matrix.org>
16:58:00
Ideally we would migrate everyone, and then declare the legacy backend as deprecated, and remove it when we rebase to F42
<@siosm:matrix.org>
17:00:03
EndOfIntro
<@dustymabe:matrix.org>
17:00:27
you want to migrate everyone in the middle of a release?
<@jlebon:fedora.im>
17:00:29
so right now, we manually set the symlinks so that iptables-nft is used. does the new alternatives code understand how things are set up currently?
<@jlebon:fedora.im>
17:00:53
dustymabe: i understood f42
<@siosm:matrix.org>
17:01:03
Yes, I think we should migrate during the F41 cycle and deprecate for F42.
<@siosm:matrix.org>
17:01:26
The iptables change should really be safe. It's been the default for a while everywhere now
<@siosm:matrix.org>
17:02:10
we manually set the symlink in the part that is in /etc/alternatives, which is the "state" and the config is now in /etc/alternatives-admindir when it was in /var/lib/alternatives before
<@jlebon:fedora.im>
17:02:35
basically, would there be any delta at all in "node state" between a new node that started on nft, and one that was migrated
<@siosm:matrix.org>
17:03:19
the trick is that the new alternatives only use the new config admindir if the older /var/lib one does not exists
<@siosm:matrix.org>
17:03:28
and users could have placed configs there manually
<@siosm:matrix.org>
17:03:53
so we would have to `mv /var/lib/alternatives /etc/alternatives-admindir` which is not really "safe"
<@siosm:matrix.org>
17:05:19
Overall, it's very unlikely to be the case (that anyone set a custom alternatives config) so we could also say we don't care and we break it but that means removing /var/lib/alternatives
<@siosm:matrix.org>
17:05:26
should be safe if empty
<@jlebon:fedora.im>
17:05:38
hmm, but the /var one is a symlink to /usr, so they would've had to purposely break the symlink
<@siosm:matrix.org>
17:05:45
Maybe this calls for a clhm
<@siosm:matrix.org>
17:06:02
it's a symlink only on new systems
<@siosm:matrix.org>
17:06:19
on older systems it's a plain dir
<@jlebon:fedora.im>
17:06:31
fun
<@siosm:matrix.org>
17:07:04
It's even a broken symlink on my Kinoite system right now
<@siosm:matrix.org>
17:07:26
(I argued for the migration to happen in the alternatives command but this did not get traction so here we are)
<@siosm:matrix.org>
17:07:51
we'll have the same issue for Atomic Desktops obviously, and IoT, etc.
<@jlebon:fedora.im>
17:08:28
apart from iptables, do we even have other things that use it in the base? ISTM more likely that any use of alternatives by users would be from layered pkgs
<@dustymabe:matrix.org>
17:08:29
so we'd want to ship the migration script in all of those places?
<@siosm:matrix.org>
17:09:53
The main thing for us is iptables. The rest would be layered packages indeed
<@siosm:matrix.org>
17:10:06
> so we'd want to ship the migration script in all of those places?
<@siosm:matrix.org>
17:10:06
yes, ideally
<@siosm:matrix.org>
17:10:14
<@siosm:matrix.org>
17:10:14
> so we'd want to ship the migration script in all of those places?
<@siosm:matrix.org>
17:10:14
yes, ideally
<@siosm:matrix.org>
17:10:25
yes, ideally
<@siosm:matrix.org>
17:10:25
> so we'd want to ship the migration script in all of those places?
<@siosm:matrix.org>
17:10:33
<@siosm:matrix.org>
17:10:33
> so we'd want to ship the migration script in all of those places?
<@siosm:matrix.org>
17:10:33
yes, ideally
<@dustymabe:matrix.org>
17:11:40
is there a precedence?
<@dustymabe:matrix.org>
17:12:04
`/var/lib/alternatives` versus `/etc/alternatives-admindir` ?
<@siosm:matrix.org>
17:12:38
/var/lib/alternatives is always used if available
<@siosm:matrix.org>
17:12:53
https://github.com/fedora-sysv/chkconfig/pull/135/files#diff-562b9b19cb1cd12a7343ce5c739745ebc8f363a195276ca58e926f22927238a5R1474
<@dustymabe:matrix.org>
17:14:31
```
<@dustymabe:matrix.org>
17:14:31
```
<@dustymabe:matrix.org>
17:14:31
ok. so the migration would be something like:
<@dustymabe:matrix.org>
17:14:31
<@dustymabe:matrix.org>
17:14:31
mv /var/lib/alternatives/* /etc/alternatives-admindir/
<@dustymabe:matrix.org>
17:14:31
rmdir /var/lib/alternatives
<@dustymabe:matrix.org>
17:14:31
ln -s <somewhere in /usr> /var/lib/alternatives
<@siosm:matrix.org>
17:14:55
(I've just verified that iptables is the only command actually using alternatives on FCOS)
<@siosm:matrix.org>
17:15:17
something like that yes
<@siosm:matrix.org>
17:15:34
ln -s /etc/alternatives-admindir/ /var/lib/alternatives
<@dustymabe:matrix.org>
17:15:38
but I guess only do the first step IFF /var/lib/alternatives isn't already a symlink
<@siosm:matrix.org>
17:15:53
yes
<@siosm:matrix.org>
17:16:33
or we don't create the symlink at all in the end
<@siosm:matrix.org>
17:17:08
The risk is that we don't know what users may have placed there
<@dustymabe:matrix.org>
17:17:10
<@dustymabe:matrix.org>
17:17:10
```
<@dustymabe:matrix.org>
17:17:10
$ ls -l /var/lib/alternatives
<@dustymabe:matrix.org>
17:17:10
ls: cannot access '/var/lib/alternatives': No such file or directory
<@dustymabe:matrix.org>
17:17:10
```
<@dustymabe:matrix.org>
17:17:10
fresh FCOS f42 system:
<@siosm:matrix.org>
17:17:28
It would have been a bad idea to place data there but we don't know
<@siosm:matrix.org>
17:18:00
For 99% maybe of the systems it should be empty so we could optimize for this case and "just" delete the folder
<@siosm:matrix.org>
17:18:19
The other systems we could write a CLHM that would check and warn
<@dustymabe:matrix.org>
17:18:44
works for me..
<@dustymabe:matrix.org>
17:19:04
what happens if they ignore the warning? they stay on iptables legacy and eventually when we remove it they are broke?
<@siosm:matrix.org>
17:19:53
yes, it will break the commands as they won't be set to the new backend by the migration script
<@siosm:matrix.org>
17:20:13
or we do a forced manual migration to compensate for that as well
<@siosm:matrix.org>
17:21:18
3. migrate using alternatives / force migrate systems where alternatives config is broken
<@siosm:matrix.org>
17:21:18
2. Warn, if not empty
<@siosm:matrix.org>
17:21:18
1. rmdir /var/lib/alternatives if empty
<@siosm:matrix.org>
17:22:13
Ah, we can force the admindir in the alterantives call so we could use that
<@siosm:matrix.org>
17:22:53
Alright, I'll suggest a plan that should be safe based on the above
<@jlebon:fedora.im>
17:23:30
seems reasonable to me 👍️
<@dustymabe:matrix.org>
17:23:52
sounds good. maybe put it in the ticket and we can discuss more
<@dustymabe:matrix.org>
17:24:03
the more detail the better :)
<@siosm:matrix.org>
17:24:45
Let's move to open floor :)
<@gurssing:matrix.org>
17:24:53
!topic Open Floor
<@dustymabe:matrix.org>
17:25:19
!info FYI stable FCOS nodes are migrating to F41 this week
<@dustymabe:matrix.org>
17:26:12
Any volunteers would be welcome :)
<@dustymabe:matrix.org>
17:26:12
There are some followup items for F41 that need to be completed (see checklist in https://github.com/coreos/fedora-coreos-tracker/issues/1695)
<@dustymabe:matrix.org>
17:26:12
<@marmijo:fedora.im>
17:27:12
I can take care of closing out the last items
<@gurssing:matrix.org>
17:27:39
Can help out as well.
<@gurssing:matrix.org>
17:28:31
If there isn't anything more, I will close the meeting in about a minute.
<@gurssing:matrix.org>
17:29:15
!endmeeting