16:30:36 <dustymabe> #startmeeting fedora_coreos_meeting
16:30:36 <zodbot> Meeting started Wed Oct  7 16:30:36 2020 UTC.
16:30:36 <zodbot> This meeting is logged and archived in a public location.
16:30:36 <zodbot> The chair is dustymabe. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:30:36 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:30:36 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:30:45 <dustymabe> #topic roll call
16:30:50 <dustymabe> .hello2
16:30:50 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:22 <lucab> .hello2
16:31:22 <zodbot> lucab: lucab 'Luca Bruno' <lucab@redhat.com>
16:31:34 <skunkerk> .hello sohank2602
16:31:35 <zodbot> skunkerk: sohank2602 'Sohan Kunkerkar' <skunkerk@redhat.com>
16:31:43 <aoei> .hello2
16:31:44 <zodbot> aoei: aoei 'Joanna Doyle' <jjadoyle@gmail.com>
16:32:21 <davdunc> .hello2
16:32:22 <zodbot> davdunc: davdunc 'David Duncan' <davdunc@amazon.com>
16:32:58 <jdoss> .hello2
16:32:59 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
16:33:06 <jlebon> .hello2
16:33:07 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:33:08 <dustymabe> #chair lucab skunkerk aoei davdunc jdoss jlebon
16:33:08 <zodbot> Current chairs: aoei davdunc dustymabe jdoss jlebon lucab skunkerk
16:34:41 <dustymabe> #topic Action items from last meeting
16:34:48 <dustymabe> * jlebon to reach out to the rpm maintainers to see if the relocation of
16:34:50 <dustymabe> the rpmdb path is something their willing to own for F34
16:34:52 <dustymabe> * we'll file a tracker ticket to come up with rough criteria for adding
16:34:54 <dustymabe> packages
16:35:48 <dustymabe> #info miabbott openeed a ticket to discuss criteria for adding new packages to the base layer
16:35:51 <lorbus> .hello2
16:35:53 <zodbot> lorbus: lorbus 'Christian Glombek' <cglombek@redhat.com>
16:35:55 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/641
16:36:19 <bgilbert> hello2
16:36:21 <bgilbert> .hello2
16:36:23 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:36:31 <dustymabe> sigh. i just realized I used the wrong `their` in that action item for jlebon last week
16:36:52 <dustymabe> I blame my brain
16:36:59 <dustymabe> jlebon: any updates for us on that one?
16:37:21 <jlebon> sorry was briefly afk
16:37:37 <jlebon> so... i didn't fully do that, but i did start the discussion on https://github.com/coreos/fedora-coreos-tracker/issues/639 with Neil
16:37:51 <jlebon> Neal*
16:37:59 <jlebon> let's re-action that one
16:38:00 <dustymabe> +1 - re-action or let die?
16:38:03 <dustymabe> will do
16:38:16 <dustymabe> #action jlebon to reach out to the rpm maintainers to see if the relocation of the rpmdb path is something they're willing to own for F34
16:38:24 <dustymabe> see what I did there ^^
16:38:45 <dustymabe> 🤓
16:38:57 <bcotton> dustymabe++
16:39:10 <dustymabe> ok let's move to meeting topics
16:39:15 <jlebon> hehe
16:39:18 <dustymabe> #topic Need dnsmasq for podman to create CNI networks
16:39:24 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/519
16:39:38 <dustymabe> so i'm bringing up this dead horse to beat one more time
16:40:06 <dustymabe> two weeks ago we managed to get the podman team to make podman-plugins a weak dep of podman (makes sense)
16:40:24 <dustymabe> so we were able to drop podman-plugins (and the hard requirement of dnsmasq) from the base layer
16:40:54 <dustymabe> we've had at least one other user come up with another instance of the dnsmasq binary being in place on the host helping
16:41:09 <dustymabe> https://discussion.fedoraproject.org/t/please-consider-reinstating-dnsmasq-in-coreos/23615
16:42:06 <dustymabe> So in the case of `dnsmasq` as a helper tool to other host things already in the base layer (networkmanager or podman) maybe we should revisit cyberpear's suggestion in https://github.com/coreos/fedora-coreos-tracker/issues/519#issuecomment-698597945
16:42:41 <dustymabe> which is: let's include dnsmasq so the binary can be used, but neuter the service
16:43:13 <dustymabe> a user could re-enable the service if they do away with our override, but that typically would require them to look at documentation and understand the downsides
16:43:25 <cyberpear> similar to the earlier arguments for systemd-networkd, but we were able to instead split the package there instead... dnsmasq is already small and can't be split any further
16:43:26 <dustymabe> so that's one option
16:44:12 <dustymabe> another option is to just tell people to package layer dnsmasq (which should now be "more reliable" in the next testing release)
16:44:31 <dustymabe> though, there are benefits to having `podman-plugins` installed, so it's worth a discussion
16:45:55 <dustymabe> ilius: :) - I just laid the floor for the discussion
16:46:22 <dustymabe> ilius: https://paste.centos.org/view/e1f9d5d1
16:46:32 <ilius> Thank you.
16:47:12 <dustymabe> any opinions on the options?
16:47:30 <dustymabe> A. leave dnsmasq out, people can package layer if really desired
16:47:52 <dustymabe> B. include dnsmasq (and podman-plugins, because "why not?") and neuter the dnsmasq service with an override
16:47:58 <ilius> So is Fedora going to convert to the dark side of systemd-networkd?
16:48:06 <cyberpear> ilius: no
16:48:29 <dustymabe> no plans to move away from NetworkManager
16:50:00 <bgilbert> I'd be interested in lucab's opinion, since he's been opposed to including dnsmasq in the past
16:50:30 <dustymabe> 👋 lucab
16:50:31 <ilius> I'm cool with that.  I believe that we need to be consistent.  As I said in that duscussion post, dnsmasq is considered a plugin to NetworkManager and it integrates with it really well, and it is only a few hundred kilobytes big.  It gives so much functionality that a Docker node may need.  I mean, tools like `jq` are in CoreOS, then so should dnsmasq.
16:50:36 <walters> hmm it'd be a bit ironic to start enabling resolved by default and also dnsmasq (they're not the same but)
16:51:12 <jdoss> Can podman use resolved with some tweaks?
16:51:13 <ilius> It doesn't have to be enabled, just available.  200-400KB is such a small price to pay.
16:51:27 <lucab> yeah well, it looks like everything in Fedora wants to use dnsmaqs-the-binary but not dnsmasq-the-service
16:52:50 <jlebon> libvirtd too, which i suspect is a common overlay in the single node case
16:52:56 <ilius> It is my opinion that if systemd-networkd is used, then systemd-resolved should be available.  If NetworkManager is used, then dnsmasq should be available.  Mixing them seems ... weird to me.
16:53:37 <ilius> <= (not a fan of systemd anyway)
16:53:44 <dustymabe> ilius: at the broader Fedora level the choice is NM + systemd-resolved
16:54:27 <ilius> I don't understand why the mixing of systemd's rewrite of Unix services, but I might be the odd man out on that.  :)
16:54:39 <walters> one concern: I don't think we want podman-plugins for the Kubernetes use case; don't want containers that happen to be co-located on a node to see each other, they should use kube services
16:54:53 <bgilbert> lucab: any conclusions based on that?
16:54:59 <dustymabe> walters: IIUC kubernetes doesn't use podman
16:55:14 <lucab> I'd rather not have an OS with dnsmasq-the-service, but RPM packaging is not really suited for that
16:55:16 <walters> openshift uses coredns for cluster dns
16:55:28 * cyberpear o/t, resolved seems to solve the same DNS-splitting problem as dnsmasq, but less well
16:55:56 <lucab> bgilbert: which is quite unlikely that libirtvd/NM/podman start using some proper owned plugin to managed DNS/DHCP
16:56:08 <walters> dustymabe: yeah I think crio e.g. explicitly avoids this
16:56:44 <misc> lucab: then the rpm packaging could be changed, split dnsmasq and dnsmasq binary ?
16:56:49 <dustymabe> lucab: if we neuter the service with an override (which can be overridden by the user, but would require them to look up docs probably), as suggested by cyberpear, would that help?
16:57:17 <misc> or use a preset
16:57:26 <ilius> If people want dnsmasq to be a service, they can make their service definition in ignition, cant't they?  Just having the binary there is so simple.
16:57:40 <jdoss> I would just neuter the service vis making all this work or looping files out from the RPM like systemd-networkd is currently.
16:57:46 <lucab> misc: yeah, but does that make sense in RPM world? I don't think so
16:57:49 <jdoss> lopping*
16:57:55 <dustymabe> jdoss: yeah we don't want to chainsaw anything out
16:58:07 <misc> lucab: well, I think that's heavyweight, and systemd-preset are here for that exact usecase
16:58:32 <dustymabe> so misc, your argument is to not do the override, but just have the service disabled?
16:58:44 <misc> yeah
16:59:01 <dustymabe> I think that is reasonable, but I think we want to go an extra step to communicate to users that it would be an undesired use of a host service
16:59:04 <lucab> thing is, the service comes with default configs, lookup paths, buildtime configuration
16:59:10 <dustymabe> which is why we proposed the override
16:59:32 <lucab> having the service available (even if disabled in preset) has additional "don't break compatibility" edges
17:00:37 <bgilbert> misc, I don't think default-disabled is quite strong enough
17:00:47 <dustymabe> works for me
17:01:06 <dustymabe> so if we do go with B. we'd definitely want an override and accompanying docs, right?
17:01:09 <bgilbert> if the goal is to have an anti-recommendation, "only enable this if you're sure you want it"
17:01:14 <lucab> I don't personally see a good outcome here and I have been persuaded there are too many other components relying on dnsmasq-the-binary which is not realistic to try to change the world
17:02:05 <jlebon> wait, why can't we leave it as a layered package again?
17:02:24 <dustymabe> jlebon: we can. there are positives and negatives
17:02:25 <jlebon> did ilius hit issues with that path or are we just discussing whether it's worth reducing that friction?
17:02:45 <jlebon> if we want *some* friction, that seems like a natural way to do it :)
17:02:52 <dustymabe> jlebon: I don't know if it was explicitly recommended to ilius yet
17:03:20 <dustymabe> but it is option A I mentioned above
17:04:04 <dustymabe> anyone want to argue for one or the other?
17:04:11 <jlebon> ack right. ilius: does dnsmasq as a layered pkg work?
17:04:24 <jlebon> it requires an extra reboot today, but eventually we'd like to optimize that
17:04:27 <lucab> I think jdoss was also mentioning removing files (i.e. anything but the binary) from the package via our treefile
17:04:43 <dustymabe> yeah, we don't want to do that
17:04:56 <dustymabe> for the same problems that we hit with systemd-networkd I believe
17:04:56 <lucab> I'm not sure I'd want to maim the package that way
17:04:59 <ilius> Sorry, this is the first I've heard of layered packages.  Are they just run as containers?
17:05:14 <lucab> dustymabe: yes, I'd share the same feeling
17:05:18 <jlebon> ilius: they're a way to add packages on top of the base OS
17:05:28 <dustymabe> ilius: `sudo rpm-ostree install dnsmasq && sudo reboot`
17:06:01 <ilius> I use iPXE with an immutable root... no install/reboot.
17:06:19 <dustymabe> jlebon: ^^ that's one case I hadn't considered
17:06:34 <jlebon> the nice thing with layered packages is that `rpm-ostree status` will implicitly be telling you you're a bit off the beaten trail
17:06:45 <dustymabe> I guess the livefs stuff would work?
17:06:46 <jlebon> right yeah. that's something we'll need to address eventually
17:07:20 <jlebon> i wouldn't use livefs as it is today
17:07:53 <jdoss> I hope one day we can layer/install packages via Ignition. That would be so cool.
17:07:59 <dustymabe> jlebon: if he runs everything live (never reboot) then livefs should work just fine I would think, but you're the expert :)
17:08:06 <jdoss> (not sure if that is in scope but a guy can dream)
17:08:29 <dustymabe> ok this is running a bit long
17:08:34 <jlebon> one thing that would work today is `rpm-ostree usroverlay && rpm -ivh dnsmasq.rpm` but that's clearly less elegant :)
17:08:52 <dustymabe> does anyone have a sense of the way the room is leaning? A (package layer) or B ?
17:09:42 <jlebon> to clarify, i'm not against adding it to the host in some form.  just wanted to make sure we had discussed the other path
17:09:58 <dustymabe> yep.
17:10:05 <dustymabe> A. leave dnsmasq out, people can package layer if really desired
17:10:13 <dustymabe> 
17:10:15 <dustymabe> B. include dnsmasq (and podman-plugins, because "why not?") and neuter the dnsmasq service with an override
17:10:24 <jdoss> B
17:10:57 <walters> i lean A but will accept B, it's just a constant battle to keep the host small
17:11:00 <jlebon> personally leaning more on A, but ok with B
17:11:23 <dustymabe> luckily this one is tiny (hopefully they don't rewrite dnsmasq in golang)
17:11:50 <dustymabe> I think I lean B, just so we get podman-plugins behavior, it's something baude said they got beat over the head over many times
17:12:24 <dustymabe> cyberpear: bgilbert: skunkerk: lucab: thoughts?
17:12:42 <cyberpear> B
17:12:57 <skunkerk> B
17:13:18 <bgilbert> I'd lean A until lots of things expect the binary to be there, and then I'd lean B.  I think we're at B but I don't know enough
17:13:19 <lucab> no opposition/preference to either A or B from my side
17:13:50 <ilius> B please
17:14:03 <dustymabe> ok I'm going to make a proposal for B. the use case ilius mentioned about not being able to package layer for a live environment I think is something at least I hadn't considered
17:14:36 <bgilbert> dustymabe: the live thing isn't really an argument for dnsmasq though
17:14:41 <walters> as jlebon said we will be fixing that
17:15:27 <cyberpear> live - "in general"
17:15:49 <jlebon> it would be fixed by way of supporting no-reboot layering
17:15:50 <dustymabe> #proposed we will include the dnsmasq rpm in the host to enable integrations with services that utilize the dnsmasq binary. We will add an override to disalbe the dnsmasq service and add documentation that explains how to re-enable it and explain why it's preferred that users not use the service directly.
17:16:03 <dustymabe> ack/nack
17:16:23 <cyberpear> ack
17:16:44 <ilius> ack
17:16:45 <skunkerk> ack
17:16:58 <jlebon> weak ack
17:17:35 <bgilbert> ack
17:17:46 <dustymabe> #agreed we will include the dnsmasq rpm in the host to enable integrations with services that utilize the dnsmasq binary. We will add an override to disalbe the dnsmasq service and add documentation that explains how to re-enable it and explain why it's preferred that users not use the service directly.
17:17:47 <lucab> jlebon: quack?
17:17:50 <dustymabe> :)
17:18:09 <dustymabe> #topic 2020-09-30: gather status update for Fedora Council
17:18:16 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/608
17:18:24 <dustymabe> I was trying to get to this last meeting but ran out of time
17:18:27 <jlebon> lucab: :)
17:18:37 <dustymabe> Let's write down some things we've been up to the past month
17:18:44 <dustymabe> https://hackmd.io/A7VptlnASNy7bf2BZhma1g
17:19:16 <ilius> Thanks so much for reviewing this today!
17:19:28 <dustymabe> ilius: +1
17:20:09 <dustymabe> jlebon: anything with luks we should add to the list in the hackmd ?
17:20:45 <jlebon> dustymabe: probably, i need to get a github search query going
17:20:52 <dustymabe> :)
17:21:12 <dustymabe> lucab: anything from the zincati/afterburn side ?
17:22:07 <lucab> dustymabe: I can check, I think at least a new zincati release
17:22:09 <jlebon> something like this maybe: https://github.com/coreos/fedora-coreos-tracker/issues?q=is%3Aissue+is%3Aclosed+sort%3Aupdated-desc+updated%3A%3E%3D2020-09-01+
17:23:35 <dustymabe> cool. thanks everyone for adding some items to that list. I'll try to clean them up in a bit and add links
17:23:52 <dustymabe> I'll switch to the last topic
17:24:32 <dustymabe> #topic tracker: Fedora 33 rebase work
17:24:38 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/609
17:25:08 <dustymabe> #info the `next-devel` stream is moved over to F33 and we're planning to do a `next` release with f33 content today
17:25:26 <dustymabe> There are some issues. Some failing tests which we've make exceptions for
17:25:30 <dustymabe> #link
17:25:41 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/644
17:25:47 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/643
17:26:05 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/646
17:26:18 <dustymabe> so a few issues to keep working towards
17:26:34 <dustymabe> but we'll at least get this out there in `next` for users to help find issues
17:26:44 <dustymabe> jdoss: FYI systemd-networkd can now be package layered ^^
17:26:57 <jdoss> \o/
17:27:11 <jlebon> nice work on that dustymabe!
17:27:13 <jdoss> I am going to make so many .link files now.
17:27:48 <dustymabe> #topic open floor
17:27:53 <dustymabe> any topics for open floor?
17:29:24 <lucab> (nope)
17:29:39 <dustymabe> :)
17:29:48 <dustymabe> we might end on time
17:29:50 <dustymabe> \o/
17:30:05 <dustymabe> #endmeeting