16:31:11 <dustymabe> #startmeeting fedora_coreos_meeting
16:31:11 <zodbot> Meeting started Wed Jun  9 16:31:11 2021 UTC.
16:31:11 <zodbot> This meeting is logged and archived in a public location.
16:31:11 <zodbot> The chair is dustymabe. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:31:11 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:31:11 <zodbot> The meeting name has been set to 'fedora_coreos_meeting'
16:31:16 <dustymabe> #topic roll call
16:31:17 <travier> .hello siosm
16:31:17 <zodbot> travier: siosm 'Timothée Ravier' <travier@redhat.com>
16:31:18 <dustymabe> .hi
16:31:23 <zodbot> dustymabe: dustymabe 'Dusty Mabe' <dusty@dustymabe.com>
16:31:34 <jdoss> .hi
16:31:35 <jlebon> .hello2
16:31:35 <zodbot> jdoss: jdoss 'Joe Doss' <joe@solidadmin.com>
16:31:37 <bgilbert> .hi
16:31:38 <zodbot> jlebon: jlebon 'None' <jonathan@jlebon.com>
16:31:41 <zodbot> bgilbert: bgilbert 'Benjamin Gilbert' <bgilbert@backtick.net>
16:31:49 <jbrooks> .hello jasonbrooks
16:31:50 <zodbot> jbrooks: jasonbrooks 'Jason Brooks' <jbrooks@redhat.com>
16:33:07 <cyberpear> .hi
16:33:08 <zodbot> cyberpear: cyberpear 'James Cassell' <fedoraproject@cyberpear.com>
16:33:20 <miabbott_> .hello miabbott
16:33:21 <zodbot> miabbott_: miabbott 'Micah Abbott' <miabbott@redhat.com>
16:33:33 <jaimelm> .hello2
16:33:34 <zodbot> jaimelm: jaimelm 'Jaime Magiera' <jaimelm@umich.edu>
16:33:56 <dustymabe> #chair travier jdoss jlebon bgilbert jbrooks cyberpear miabbott jaimelm
16:33:56 <zodbot> Current chairs: bgilbert cyberpear dustymabe jaimelm jbrooks jdoss jlebon miabbott travier
16:34:02 <dustymabe> nice turnout
16:34:27 <jdoss> Who brought the snacks??
16:34:46 <dustymabe> orange slices for everyone!
16:34:56 <travier> 🍬
16:35:03 <travier> 🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬🍬
16:35:19 <dustymabe> #topic action items from last meeting
16:35:26 * jaimelm pulls up his carpet square
16:35:47 <dustymabe> I don't know if we had anything concrete coming out of colin's discussion in last week's video meeting, though I do think I've seen a few PRs from him on that front
16:36:05 <dustymabe> for the f35 changes discussion I created a ticket (and will create some followups)
16:36:08 <dustymabe> #info created top level issue tracker for Fedora 35 changes discussion https://github.com/coreos/fedora-coreos-tracker/issues/856
16:37:13 <darkmuggle> .hello2
16:37:14 <zodbot> darkmuggle: darkmuggle 'None' <me@muggle.dev>
16:37:21 <dustymabe> let's hop right into tickets.. as always, add the meeting label to tickets or comment and say you want to discuss in a meeting if you want to bring up something here
16:37:30 <dustymabe> otherwise I'll just willy/nilly choose topics :)
16:37:34 <dustymabe> #chair darkmuggle
16:37:34 <zodbot> Current chairs: bgilbert cyberpear darkmuggle dustymabe jaimelm jbrooks jdoss jlebon miabbott travier
16:37:37 <jdoss> In Dusty we trust.
16:37:40 <jaimelm> There were some tasks left over
16:37:49 <jaimelm> from several meetings ago
16:37:57 <jlebon> darkmuggle: heh nice, just noticed your domain name now
16:38:01 <dustymabe> jaimelm: yeah let me check those
16:38:18 <darkmuggle> :)
16:38:27 <jaimelm> e.g. system-oomd mention at OKD WG
16:38:45 <dustymabe> * jaimelm to ask the OKD working group if there are any implications
16:38:47 <dustymabe> that systemd-oomd would have on k8s/OKD
16:38:48 <jaimelm> https://github.com/openshift/okd/discussions/663
16:38:49 <dustymabe> * darkmuggle to investigate systemd-oomd (in use without swap) and
16:38:51 <dustymabe> report back next week
16:38:59 <dustymabe> jaimelm: I'll bring up the systemd-oomd topic here momentarily
16:39:08 <dustymabe> so we'll cover it there
16:39:20 <dustymabe> #topic systemd-oomd for Fedora CoreOS
16:39:28 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/840
16:39:33 <dustymabe> jaimelm: ^^
16:39:38 <jaimelm> hehe
16:39:40 <dustymabe> 🤓
16:40:05 <jaimelm> Discussed at the OKD WG yesterday. There is a discussion item in the repo (see above) where Vadim responded with some thoughts.
16:40:16 <jaimelm> We don't anticpate any issues
16:40:42 <darkmuggle> My findings on the `systemd-oomd` is that without having swap, things can get really wonky. I think its likely safe, but for containers, the entire pod will get oom'd
16:41:11 <dustymabe> "don't anticipate issues" as in "we're still using cgroups v1, so systemd-oomd won't run anyway"?
16:41:46 <travier> OKD folks are planning on disabling systemd-oomd
16:41:53 <jaimelm> That's one aspect. The other is that we plan on disabling it.
16:42:00 <darkmuggle> The only potential problem is that some users have reported that the thresholds are aggressive
16:42:11 <jaimelm> Putting that change in the OKD payload.
16:42:28 <dustymabe> darkmuggle: interesting. maybe based on feedback we receive we can lobby for changing the fedora default values
16:43:04 <darkmuggle> Some users have reported that low-memory systems (less than 4gb -- its 2021 folks) are a bit unpredictable.
16:44:01 <dustymabe> darkmuggle: I have a few of those myself
16:44:09 <travier> I have a 4GB system too
16:44:10 <darkmuggle> I didn't find out if there is a server package
16:44:23 <travier> well 3 actualy
16:44:31 <dustymabe> how about we do something like enable it in the `next` stream and see what feedback we get
16:44:42 <travier> I'd advocate for keeping it in the image but disabled by default
16:44:45 <darkmuggle> for the settings, but my $0.02Doge is that we should check and see, and if not, work with the Fedora teams to have a server version that we can us.
16:44:50 <jaimelm> travier: red hat needs to pay you better.
16:45:16 <andi89gi> Hey folks
16:45:20 <dustymabe> $.02Doge goes to travier!
16:45:25 <jaimelm> +1 for disable by default
16:45:26 <travier> jaimelm: well no need to push for bigger than what I need :) climate change and all that
16:45:54 <travier> no point in having a 64GB server 90% unused
16:45:58 <dustymabe> hi andi89gi
16:46:23 <jdoss> what about combo systemd-oomd + zram both installed but disabled on next ;)
16:46:31 <travier> is oomd enabled by default in cloud & server images / installs ?
16:46:42 <dustymabe> travier: regarding putting it in the image but disabled by default.. i think we should really try to keep parity with Fedora on things unless it really really doesn't make sense
16:47:01 <dustymabe> it would be nice to get some user testing on it to have good reasons for keeping it disabled by default
16:47:11 <dustymabe> travier: not currently in cloud, but it's going to be
16:47:21 <dustymabe> it was mostly an oversight that it wasn't done in f34 by default
16:47:37 <dustymabe> #link https://pagure.io/cloud-sig/issue/324
16:48:02 <dustymabe> I'm not sure about server, but can check
16:48:05 <jlebon> dustymabe: +1
16:48:12 <travier> Looking into server
16:48:43 <andi89gi> dustymabe: hey
16:49:42 <dustymabe> travier: should we wait (are you actively looking at it now?) or should we move on and pick up the discussion later
16:50:00 <travier> move on, I'll post results in the ticket
16:50:11 <dustymabe> jdoss: "combo systemd-oomd + zram both installed but disabled" is what we already have
16:50:22 <jdoss> \o/
16:50:41 <dustymabe> ok, let's at least establish some sort of direction and we can ammend it next week based on results
16:50:43 <jdoss> I couldn't remember if it was installed or not.
16:50:59 <travier> my concern is more about enabling it by default for existing installs
16:51:12 <darkmuggle> fwiw, after the deep dive on the subject, we should consider swap because it makes the memory management easier
16:51:13 <travier> because that will create change for existing users
16:51:22 <dustymabe> darkmuggle: indeed
16:51:23 <jdoss> Can we post how to enable both via Ignition for testing in the ticket?
16:51:38 <dustymabe> jdoss: yeah probably should do that
16:51:47 <jaimelm> On thought... create page with some scripts to push memory usage. Show how to enable it. Then, have a quick test day, linking users to the scripts to make their testing easy.
16:51:54 <jdoss> because I will help test with my workloads
16:53:23 <jaimelm> jdoss: I like the idea of showing how to enable via Ignition
16:54:02 <dustymabe> #info after talking to OKD, they plan to disable systemd-oomd. We stil need to talk to typhoon. In order to get more information from actual users we might end up enabling systemd-oomd or swap-on-zram+systemd-oomd in our next stream to get feedback on any potential issues.
16:54:08 <jdoss> It would just help everyone do the same thing and establish a baseline test method. The script for watching memory usage for reporting would be helpful too.
16:54:19 <jaimelm> yeah
16:54:43 <dustymabe> I can add a butane config to the ticket to show how to enable it
16:54:58 <jdoss> +1
16:55:01 <jaimelm> perfect
16:55:18 <dustymabe> #action dustymabe to add butance config to #840 to show how to enable systemd-oomd
16:55:33 <dustymabe> circle back to this topic next week?
16:55:44 <jdoss> +1
16:56:00 <dustymabe> jaimelm: do you want to work on the memory usage script stuff (I'm not going to go that far)
16:56:20 <jaimelm> What is our timeframe for this?
16:56:49 <dustymabe> we don't really have one specifically. other than having many different pans on the stove
16:57:12 <dustymabe> the quicker we can get back in line with Fedora, the sooner we can concentrate on other changes
16:57:45 <jaimelm> right
16:58:08 <travier> I don't like that we feel forced to be using the exact same config as default Fedora
16:58:18 <bgilbert> travier: +1
16:58:29 <travier> I don't think that's the best way to decide wether or not a feature should be enabled in FCOS
16:58:36 <travier> to me that does not make sense
16:58:43 <dustymabe> travier: I don't look at it as much as forcing as I do the associated cost we incur by being different
16:58:57 <jaimelm> What's the cost in this case?
16:59:07 <travier> Having a service be enabled / disabled on a specific variant is perfectly fine and does not make us Not Fedora
16:59:16 <jdoss> I am in the other camp. I want to have no surprises between Fedora and FCOS so having the same fundamentals is helpful.
16:59:25 <jaimelm> I think that should be estimated per-thing
16:59:38 <bgilbert> jdoss: we're optimizing for a different set of use cases, so we should absolutely turn knobs that help us do that optimization
16:59:51 <bgilbert> would it help to have a docs page listing divergences from Server or Cloud?
16:59:51 <dustymabe> I didn't say we weren't "Fedora", I just said that we're running things differently - and we need to keep those differences small if we can
17:00:02 <travier> jdoss: sure, but we also have existing users and breaking their installs on an update is not Fedora CoreOS mission either
17:00:02 <jaimelm> But if FCOS is hoping to be a true release in its own right, I would expect divergence.
17:00:14 <dustymabe> bgilbert: I definitely think that would be beneficial
17:00:34 <dustymabe> jaimelm: correct, but only where it really makes sense IMO
17:01:02 <travier> I'll file a ticket to create a doc page for "differences" so that we can settle this topic
17:01:07 <dustymabe> we also shouldn't be afraid of change - I think what we're doing here where we are proposing to "try it out" and report feedback is great
17:01:15 <jdoss> dustymabe: +1
17:01:23 <travier> Personally, I'll disable oomd first thing on my installs
17:01:29 <travier> do I expect other will do
17:01:36 <bgilbert> we shouldn't have gratuitous divergences, sure.  but sometimes our hand is forced by e.g. kubelet expectations; see also the swap discussions
17:01:46 <dustymabe> travier: I don't think a page of differences is going to settle this topic
17:01:48 <travier> so for me this is a bad change as it forces me to do admin changes on existing servers
17:01:51 <jaimelm> yeah, maybe save the bigger question for another time.
17:01:59 <jaimelm> We have a path forward on this
17:02:01 <dustymabe> we're still going to discuss topics individually and evaluate inclusion or not
17:02:22 <cyberpear> I'd generally want a very good reason to differ from Fedora changes.
17:02:27 <travier> settle the question of which differences are in FCOS (agree that won't settle the topic)
17:02:36 <dustymabe> +1
17:02:50 <dustymabe> ok I think we're good on this (systemd-oomd) for now
17:02:52 <jdoss> and personally I'd turn it on. I think the closer we are to Fedora the better off we are as a whole. I agree there are things that impact discussion, but IMO k8s and Openshift are downstream to this project and they should account for things on their end to fit their projects needs.
17:03:12 <travier> jdoss: I'm not using k8s on my servers
17:03:14 <dustymabe> jdoss: correct, though it's a balance
17:03:23 <jdoss> Neither am I travier
17:03:50 <dustymabe> next topic
17:04:01 <travier> so I'm not optimising for k8s, just considering that may not be best for existing users if we start killing their containers in unexpected ways
17:04:14 <dustymabe> #topic New Package Request: selinuxd
17:04:18 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/854
17:04:36 <jlebon> travier: nothing precludes doing this for new nodes only. we've done that a bunch of times
17:05:16 <dustymabe> does anyone want to introduce this topic. jlebon, you've had some discussion back and forth
17:05:38 <dustymabe> do we think selinuxd will help solve some of our selinux usability problems ?
17:05:41 <jlebon> dustymabe: i can try
17:06:42 <jlebon> the request seems to come from OCP land where they're working on an operator to make it easier to install custom security profiles
17:07:06 <jlebon> in the process, they've built a helper which makes installing selinux modules more declarative to fit the model
17:07:54 <jlebon> there's still a bunch of unknowns but yes, it might help with the selinux usability issues
17:08:30 <cyberpear> go-- but the functionality is needed
17:08:33 <jlebon> at this point, i wouldn't necessarily advise shipping in FCOS until we have more information (there's no RPM anyway)
17:08:53 <jaimelm> a walkthrough/asciinema of where they are now would be nice
17:08:56 <dustymabe> yeah, i'd be interested in some examples
17:09:12 <dustymabe> and also maybe to understand the `d` part of selinuxd
17:09:18 <jaimelm> ^^
17:09:18 <cyberpear> I assume this fixes the "don't change SELinux settings or upgrades break" issue
17:09:23 <dustymabe> daemon? running all the time?
17:09:33 <jdoss> What would be the usecases for workloads that don't use OKD?
17:10:01 <cyberpear> right. daemon seems overkill, but maybe it's a misnomer like realmd?
17:10:05 <jlebon> jdoss: modifying the selinux policy in a declarative/Ignition friendly way
17:10:10 <dustymabe> jdoss: in my mind you'd drop down some plain text files on the filesystem and selinuxd would apply that policy
17:10:28 <dustymabe> so deliver files via Ignition and policy gets applied
17:10:35 <jlebon> cyberpear: it watches the /etc/selinux.d directory and automatically rebuilds when the contents change
17:10:38 <dustymabe> that's just in my mind - I don't know how it actually works
17:10:45 <Eighth_Doctor> that doesn't make sense though
17:10:45 <cyberpear> dustymabe++ drop-in SELinux config is what's missing now
17:10:45 <zodbot> cyberpear: Karma for dustymabe changed to 7 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:11:17 <jdoss> yeah, I have no idea how this works or why you couldn't just apply policy via systemd via custom unit?
17:11:23 <jbrooks> it's described at https://github.com/JAORMX/selinuxd
17:11:56 <Eighth_Doctor> yeah, that makes no sense
17:12:11 <Eighth_Doctor> because SELinux policies are compiled and loaded into the kernel
17:12:17 <Eighth_Doctor> the plain text files are useless afterward
17:12:29 <jlebon> Eighth_Doctor: right. this is recompiling the policy and reloading AFAIK
17:12:43 <Eighth_Doctor> so then how does it undo that?
17:12:48 <Eighth_Doctor> because that would be Hard(tm)
17:12:49 <walters> perhaps analogous to `update-ca-trust`
17:13:24 <jlebon> Eighth_Doctor: (again AFAIU) it monitors /etc/selinux.d and rebuilds the policy from scratch each time
17:13:30 <jbrooks> can't it just do what semodule -r does
17:13:43 <jdoss> Bear with me I am unearthing my inner SELinux trauma but does that mean it recontexts everything on the fly?
17:14:13 <Eighth_Doctor> that would be required, yes
17:14:21 <Eighth_Doctor> which would fail on FCOS, no?
17:14:22 <jlebon> hmm, why?
17:14:53 <jlebon> whether a relabel is required depends on the change
17:15:05 <Eighth_Doctor> because that's a write operation? and you can't relabel on a RO system
17:15:14 <dustymabe> updating the policy is separate from updating file contexts IIUC, but yeah sometimes you might want to trigger a relabel depending on the changes you make
17:15:18 <Eighth_Doctor> I've been painfully aware of that issue
17:15:38 <jaimelm> Interesting. If it fails on them as a group, it iterates through each individually. https://github.com/JAORMX/selinuxd/blob/a748373d23f48245f8db9672fd923c2bdd1859ec/cmd/oneshot.go#L103
17:15:47 <jlebon> the primary case here is enhancing the base policy, not modifying the base rules themselves as I understand it
17:16:14 <dustymabe> makes sense
17:17:13 <jlebon> personally, I think the selinux problem would be better solved in rpm-ostree
17:17:19 <travier> I think we need a demo and a more detailed use case definition
17:17:32 <travier> jlebon: agree too
17:17:33 <dustymabe> #info we're interested to learn more about selinuxd and how it can support some SELinux usability issues on ostree systems. Having an rpm built would enable us to play around with it on an FCOS system.
17:17:35 <jdoss> It's walters' problem Next ticket!
17:17:46 <jaimelm> travier++
17:17:47 <zodbot> jaimelm: Karma for siosm changed to 4 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
17:17:47 <dustymabe> does that #info reflect our current thoughts?
17:17:51 * jdoss hides from walters
17:18:14 <dustymabe> jlebon: in what way? is there an issue describing that?
17:18:14 <jlebon> dustymabe: WFM
17:18:18 <travier> jdoss: anyone can contribute to rpm-ostree :)
17:18:30 <jdoss> travier: ;)
17:18:30 <walters> heh well, I think this is something to be discussed on e.g. fedora-devel list in general, e.g. selinuxd could be a potential Change?
17:18:42 <jdoss> walters: +1
17:18:47 <jlebon> dustymabe: i think i mentioned it somewhere, i'll have to look for it
17:18:58 <dustymabe> walters: "Change" like "Fedora System-wide Change"
17:19:23 <jlebon> but rpm-ostree could easily recompile the policy and apply new labels in a layered commit
17:19:25 <travier> https://github.com/coreos/rpm-ostree/issues/27
17:19:58 <dustymabe> travier: yeah, but that link describes the problem, not the potential solution
17:20:00 <jlebon> that's basically what it does already with RPMs that ship selinux packages
17:20:36 <dustymabe> so it'd just need to learn how to do that without the RPM to deliver the new policy
17:21:05 * dustymabe moves on to next topic soon
17:21:10 <walters> I think this also relates to https://github.com/ostreedev/ostree/issues/2220 - basically supporting making things like selinux policy changes transactional
17:21:39 <dustymabe> walters: +1
17:21:41 <jlebon> dustymabe: we'd need a scheme to make selinux modifications "managed" and e.g. show up in `rpm-ostree status`. probably would be helped by https://github.com/coreos/rpm-ostree/issues/2326
17:22:04 <dustymabe> +1 - ok i'll try to copy this to the ticket
17:22:13 <dustymabe> next topic (time short)
17:22:20 <dustymabe> #topic Differing behavior for aarch64 vs x86_64 disk images
17:22:27 <dustymabe> #link https://github.com/coreos/fedora-coreos-tracker/issues/855
17:22:38 <dustymabe> FYI: I'm working on adding aarch64 support to our pipeline
17:22:50 <copperi> nice
17:22:51 <walters> nice!
17:22:53 <dustymabe> as part of that I got FCOS up and running on my local Raspberry Pi4 and noticed some things
17:22:58 <jdoss> \o/
17:23:16 <dustymabe> one of them being the differing disk layout
17:23:17 <jlebon> woohoo
17:23:25 <darkmuggle> Excellent work, Dusty
17:23:41 <dustymabe> aww shucks :) - you guys
17:23:45 <jdoss> Differing how?
17:23:48 <dustymabe> ❤️
17:23:53 <dustymabe> jdoss: check out the ticket
17:24:17 <darkmuggle> Since aarch64 has a wider platform profile (RPi4 to Cloud) that 130Mb could be more expensive for those on the lower end using SD Cards
17:24:24 <dustymabe> basically the aarch64 images don't have a partition with a `1` index
17:24:49 <dustymabe> darkmuggle: the difference for aarch64 is `1 MB`
17:25:07 <darkmuggle> I must have misready Benjamin's comment then :)
17:25:11 <bgilbert> right, there's more of a space differential on ppc64le and s390x
17:25:13 <darkmuggle> s/misready/misread
17:25:18 <dustymabe> the `124 MB` was for ppc64le and `128 MB` was for s390x
17:25:24 <darkmuggle> ah
17:25:49 <bgilbert> but it does raise the question of how far we're willing to go to align the GPTs across arches
17:25:52 <jlebon> wait do i understand correctly that e.g. /dev/sda1 can refer to blocks that come after /dev/sda4 in GPT?
17:26:09 <bgilbert> jlebon: sure, it's always worked that way
17:26:12 <bgilbert> MBR is the same
17:26:23 <jlebon> bgilbert: ahh interesting. TIL
17:26:24 <dustymabe> I agree it seems non-intuitive :)
17:26:52 <jdoss> I never even knew aarch64 had this difference. Never noticed.
17:26:55 <jlebon> could we just have sgdisk not do that and leave /dev/sda1 missing?
17:26:57 <bgilbert> note that we could add a zero-length (1-length?) partition if we just want to consume the partition numbers
17:27:21 <bgilbert> jlebon: if the user asks for "number: 0" Ignition will take the first free one
17:27:41 <darkmuggle> why not use part labels and forget the numbering?
17:28:00 <dustymabe> yeah, I think I'd prefer (since the "empty storage" cost is low) to match the actual partition offsets too
17:28:18 <bgilbert> darkmuggle: the Ignition spec doesn't currently allow ignoring partition numbers entirely; see https://github.com/coreos/ignition/issues/1219
17:28:35 <dustymabe> that way if you use a start_mib in your ignition config it can apply across architectures (i.e. you could use that same ignition config for x86_64 of aarch64)
17:28:48 <darkmuggle> and _now_ I see why it matters, thanks bgilbert
17:29:00 <bgilbert> dustymabe: honestly this seems like a non-problem
17:29:17 <dustymabe> which part.. the `start_mib` part?
17:29:20 <jlebon> though there's always going to be some architecture differences in the tables
17:29:28 <bgilbert> dustymabe: misalignment across arches
17:29:31 <jlebon> trying to harmonize them seems misguided
17:29:38 <jaimelm> dustymabe: time check
17:29:45 <dustymabe> yeah
17:29:50 <bgilbert> the partition UUIDs necessarily differ across platforms; users can specify partnums explicitly if they care; and the "bad case" doesn't break anythign
17:30:06 <dustymabe> I guess we can revisit next week - sorry for trying to squeeze it ini
17:30:11 <darkmuggle> Reasonably, I think we cannot make implicit or explicit guarantees that Ignition configs will work cross arch, right?
17:30:20 <travier> Hum, can't we have aarch64 start at 1 like the others?
17:30:45 <dustymabe> travier: we made it do that for a reason too
17:30:45 <bgilbert> darkmuggle: agreed.  Butane already requires that you specify the arch if you're using boot_device.mirror
17:31:06 <bgilbert> travier: the decision was made to harmonize on partition 4 for root, since people do currently have to care about that
17:31:06 <dustymabe> ok i'll pin this and bring it up later - sorry for the squeeze
17:31:10 <dustymabe> #topic open floor
17:31:15 <dustymabe> anything quickly for open floor ?
17:31:22 * dustymabe sorry about open floor
17:31:42 <jdoss> open floor had such hopes and dreams today...
17:32:07 * jaimelm pats open floor on the head
17:32:11 <jaimelm> next time. next time.
17:32:15 <dustymabe> :)
17:32:17 <dustymabe> #endmeeting