16:00:24 <sgallagh> #startmeeting Server SIG Weekly Meeting (2015-12-01)
16:00:24 <zodbot> Meeting started Tue Dec  8 16:00:24 2015 UTC.  The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:24 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:24 <zodbot> The meeting name has been set to 'server_sig_weekly_meeting_(2015-12-01)'
16:00:24 <sgallagh> #meetingname ServerSIG
16:00:24 <sgallagh> #chair sgallagh mizmo nirik stefw adamw simo danofsatx mhayden jds2001
16:00:24 <sgallagh> #topic roll call
16:00:24 <zodbot> The meeting name has been set to 'serversig'
16:00:24 <zodbot> Current chairs: adamw danofsatx jds2001 mhayden mizmo nirik sgallagh simo stefw
16:00:31 <adamw> .hello adamwill
16:00:32 <zodbot> adamw: adamwill 'Adam Williamson' <awilliam@redhat.com>
16:00:39 <jds2001> .hello jstanley
16:00:40 <zodbot> jds2001: jstanley 'Jon Stanley' <jonstanley@gmail.com>
16:00:45 <danofsatx> yay me! I'm actually preset!
16:00:49 <nirik> .hello kevin
16:00:49 <danofsatx> present, too...
16:00:50 <zodbot> nirik: kevin 'Kevin Fenzi' <kevin@scrye.com>
16:00:56 <danofsatx> .hello dmossor
16:00:57 <zodbot> danofsatx: dmossor 'Dan Mossor' <danofsatx@gmail.com>
16:01:21 <sgallagh> .hello sgallagh
16:01:22 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
16:01:38 * jds2001 is slightly multitasking perhaps - there's a global townhall happening at $DAYJOB via webcast
16:01:48 <sgallagh> mhayden and mizmo will not be here today.
16:02:16 <jds2001> if it works, that is.
16:02:35 <simo> the topic date is wrong :)
16:02:44 <sgallagh> Oops
16:02:48 <sgallagh> nirik: Can that be retconned?
16:02:52 * adamw wonders what the presets on danofsatx are
16:03:08 <sgallagh> adamw: Hopefully rlogin.service is disabled
16:03:17 <simo> .hello simo
16:03:18 <stefw> .hello stefw
16:03:18 <zodbot> simo: simo 'Simo Sorce' <ssorce@redhat.com>
16:03:21 <zodbot> stefw: stefw 'Stef Walter' <stefw@redhat.com>
16:03:35 <nirik> well, we can edit it after, but that won't change any messages sent, etc.
16:03:41 <sgallagh> ok
16:03:52 <sgallagh> #topic Agenda
16:04:01 <danofsatx> end this one, start new one?
16:04:05 <sgallagh> So we have two follow-up topics today:
16:04:12 <sgallagh> danofsatx: Not worth it
16:04:18 <danofsatx> roger ball
16:04:19 <sgallagh> #info Agenda Topic: Trimming the DVD
16:04:19 <sgallagh> #info Agenda Topic: Server/Cloud Interaction
16:04:27 <sgallagh> and one new item:
16:04:28 <sgallagh> #info Agenda Topic: Containerized Domain Controller
16:04:34 <sgallagh> Any other topics for this week?
16:05:02 <sgallagh> Oh, actually:
16:05:02 <sgallagh> #info Agenda Topic: Planning meeting schedule around the holidays
16:05:54 <sgallagh> OK, then let's proceed
16:06:00 <sgallagh> #topic Trimming the DVD
16:06:21 <sgallagh> I sent out a patch to the mailing list for feedback on our decision last week to cut back the contents of the Server DVD
16:06:36 <sgallagh> I don't have a clear picture of how much space this will save us, but it'll be some.
16:06:51 * danofsatx must have missed that one, goes spelunking
16:07:34 <sgallagh> danofsatx: It was a reply to last week's minutes
16:07:36 <jds2001> the one about not installing docker?
16:07:42 <danofsatx> yup. found it
16:07:58 <sgallagh> jds2001: No, but we might want to put that on the agenda too...
16:08:20 <simo> sgallagh: it looks ok to me, the names of the groups are somewhat misguiding/mysterious at times, but I trust you did yank what needed to be
16:08:22 <sgallagh> That actually may be important
16:08:44 <sgallagh> simo: Thanks
16:09:16 <sgallagh> #topic Docker as a core service
16:09:40 <sgallagh> So, Docker originally kind of organically grew into the default install of Fedora Server.
16:10:06 <sgallagh> We should make a decision as a group whether we want to continue treating Docker as a fundamental piece of Fedora Server or an optional component.
16:10:07 <simo> didn't we decide to yank it last time ?
16:10:18 * simo votes for optional
16:10:24 <jds2001> cockpit has the wherewithal to install it, right?
16:10:41 * adamw also votes optional
16:10:47 * nirik is also fine with optional
16:11:00 <sgallagh> simo: It's not in the minutes anywhere
16:11:07 <jds2001> just like starting it, clicking the button in cockpit would then install it.
16:11:13 <sgallagh> So... I'm actually in favor of making it mandatory.
16:11:16 <danofsatx> optional
16:11:16 <stefw> cockpit doesn't have functionality to install software
16:11:20 * jds2001 is fine with that, so optional is fine.
16:11:25 <stefw> in additional docker is a packaging and deployment mechanism
16:11:41 <stefw> and it's rather strange to install it in order to install something
16:11:48 <stefw> i would vote for mandatory
16:12:02 <simo> sgallagh: docker is still a very fast moving target, I do not see why we would make it mandatory at  this stage
16:12:06 <jds2001> stefw: why is that? We don't ship npm by default?
16:12:21 <jds2001> or pypi or any other the others.....
16:12:23 <stefw> because npm doesn't deploy entire applications, but rather bits and pieces of them
16:12:31 <stefw> docker is akin to rolekit
16:12:32 <sgallagh> simo: I'm not sure I'd fight hard to declare it stable API, but I'm suggesting that the inclusion of a recent version of Docker should be considered a basic feature of Server
16:12:42 <stefw> something that you use to make your server start to do "stuff"
16:13:26 <jds2001> it can do "stuff" just as easily without it, and not everyone wants it.
16:13:36 <jds2001> not to mention the increased attack surface.
16:13:43 <sgallagh> So, here's my concern:
16:13:53 <sgallagh> "Not everyone wants it" is kind of a straw-man argument.
16:14:20 <jds2001> right, but the attack surface is real
16:14:24 <sgallagh> It risks us falling back into the pre-Fedora.next situation where everyone's "Fedora" is unique and indistinguishable from everyone else's
16:14:57 <sgallagh> jds2001: Sure, there's an attack surface. I won't discount that. However, it's fairly contained, given that it's not network-facing.
16:15:02 <simo> sgallagh: I'd rather you explain why you want to make it mandatory :)
16:15:24 <sgallagh> simo: Well, in particular we are building a LOT of add-on pieces atop it now.
16:15:44 <sgallagh> (And this is going to be relevant to the later topic about the Domain Controller).
16:16:09 <simo> then we need to discuss those pieces and then make a determination based on that
16:16:21 <simo> please move this topic to later in the agenda
16:16:26 <sgallagh> I don't really feel comfortable with "It's optional, but only as long as you're not doing any of the cool stuff we've been advertising"
16:16:58 <sgallagh> To me, it's part of defining a basic *platform* above and beyond a distribution
16:18:03 <jds2001> we offer the container platform in Atomic
16:18:26 <jds2001> is Server the right place for it?
16:18:34 <simo> the problem I have with "docker" is that we do not have management tools around it, that is my main reason not to make it mandatory at this stage
16:18:34 <stefw> i think to the point i see a *lot* of duplication between what people are trying to make happen with Atomic and what people are trying to make happen with rolekit
16:18:45 <stefw> and i really want to see the two converge (rolekit use containers)
16:18:48 <sgallagh> jds2001: Atomic is a container-only environment. I don't think that's exclusive.
16:18:51 <stefw> hence my interest in having docker in the default fedora install
16:19:13 <stefw> obviously this won't happen by itself, but hopefully sgallagh agrees here :)
16:19:35 <sgallagh> I'm also concerned that dropping Docker as a base component would lead us down a path of "Well, maybe the default shouldn't have cockpit or rolekit either...)
16:19:40 <sgallagh> *default install
16:19:52 <simo> I have some concerns about rolekit/container so if we could move to the Domain Controller topic and return to the docker topic after that I feel the discussion would have more sense
16:19:57 <sgallagh> Getting us back to "everyone designs their own OS"
16:20:01 <jds2001> sgallagh: but those are essential to the platform, to your earlier point.
16:20:13 * jds2001 knows of nothing that we ship that *requires* Docker.
16:20:24 <jds2001> save for the later discussion topic.
16:20:26 <sgallagh> OK, let's do as simo says
16:20:29 <simo> jds2001: because sgallagh screwed up the agenda ordering :)
16:20:32 <adamw> sgallagh: cockpit and rolekit have always clearly been a part of what we're trying to do with Server, explicitly chosen as such
16:20:33 <adamw> docker is not
16:20:42 <sgallagh> ok
16:20:49 <sgallagh> #info Agenda Topic: Containerized Domain Controller
16:21:09 <simo> thanks
16:21:19 <sgallagh> #undo
16:21:19 <zodbot> Removing item from minutes: INFO by sgallagh at 16:20:49 : Agenda Topic: Containerized Domain Controller
16:21:24 <sgallagh> #topic Containerized Domain Controller
16:21:34 <sgallagh> jds2001: To your point above: In Fedora 23, we started shipping a rolekit-managed role for memcached that relies on Docker
16:21:36 <simo> sgallagh:  do you want to introduce the topic ?
16:22:01 <sgallagh> This was fairly successful and I've started investigating what it would take to move other roles to this format.
16:22:32 <sgallagh> That being said; rolekit is capable of installing docker on demand, so that does not *insist* upon it being part of the platform.
16:22:56 <sgallagh> But as we develop more roles this way, I think it becomes a different question.
16:22:58 <simo> so on Containerized Domain Controller I have two objections at this stage, one is general and one is specific
16:23:01 <sgallagh> Anyway, Domain Controller:
16:23:03 <simo> where do you want mew to start ?
16:23:16 <sgallagh> simo: Let me give a quick overview of the PoC I did this week first.
16:23:22 <simo> ack
16:23:51 <sgallagh> So, I started this week by taking a look at the semi-official Docker container of FreeIPA that Jan Padzioria(sp?) has been working on.
16:24:21 <simo> (pazdziora)
16:24:40 <sgallagh> It's built atop the Fedora 23 docker image and contains a complete copy of the latest stable upstream release of FreeIPA
16:25:07 <sgallagh> As well as docker addons to deal with being in a container so as to have persistent data for a non-persistent image
16:25:50 <sgallagh> Within a day of work, I was able to modify the Domain Controller role to docker-pull this image and deploy it on a local machine so as to be indistinguishable externally from the RPM-based install.
16:26:06 <sgallagh> (Thanks mostly to the groundwork laid by the memcache role)
16:26:15 <jds2001> so adding some duct tape on top of bubble gum....wonderful :/
16:26:36 <sgallagh> jds2001: Sorry, what?
16:27:04 <jds2001> sgallagh: the docker addons seem a bit of a hack first hearing about it.
16:27:16 <jds2001> sgallagh: why not use the right thing, of external volumes?
16:27:31 <sgallagh> jds2001: That's what it's doing.
16:27:36 <sgallagh> I oversimplified for expediency
16:27:41 <jds2001> ahh, ok
16:28:04 <sgallagh> There are advantages and disadvantages to the container-based approach.
16:28:58 <adamw> and they are...
16:29:14 <sgallagh> Advantages: The domain controller is built and deployed as a complete unit, without risk of a package update in the repo breaking it. This is a testable set. Upgrades using the container can be scheduled separately from OS upgrades.
16:30:17 <sgallagh> Disadvantages: In the current implementation, this necessitates network access pulling from a repository (currently Docker Hub, but *could* be a Fedora-maintained registry). The installed size (in both disk space and memory footprint) is a bit larger than the pure RPM approach.
16:30:50 <simo> sgallagh: ok if I interject here ?
16:31:04 <sgallagh> simo: I was just about to cede you the floor
16:31:06 <jds2001> does docker know how to use KSM?
16:31:15 * stefw notes that a goal for Fedora 24 is to have a Fedora hosted Docker image registry
16:31:20 <jds2001> more curiousity than anything
16:31:50 <simo> sgallagh: so let me talk about the general objection (which you partly touched on)
16:31:50 <sgallagh> jds2001: No idea
16:31:56 <sgallagh> simo: Please do
16:32:11 <simo> 1. we do not have a Fedora Registry for images, I think that is a prerequisite for any serious container use
16:32:27 <stefw> goal is for that to arrive with Fedora 24
16:32:29 <simo> 2. we do not have a decent way for user to find out what is in a container and whether it needs updates
16:32:33 <simo> especially security updates
16:32:45 <simo> 3. we do not have a way to timely push these updates for containers
16:33:02 <sgallagh> 2. In the specific case of images deployed by rolekit, we will be monitoring for updates.
16:33:05 <simo> 4. docker is immature, I had a persistent volume wiped because docker ... is stupid
16:33:28 <simo> (it could have been kubernets in (4) but I am almost sure it was docker)
16:33:46 <simo> sgallagh: monitoring how, and who is "we"
16:34:33 <stefw> sgallagh, will both the containerized and non-containerized IPA roles be initially available in tandem?
16:34:37 <simo> going in the specific anyway
16:34:41 <stefw> if so, then maturity is not a blocker
16:34:44 <simo> specific objections
16:34:50 <simo> 1. I see no upgrade path plan
16:34:57 <sgallagh> simo: So, the image is going to be maintained officially (which means security updates will go out). Rolekit will be checking at appropriate intervals for updated versions of the image and will be able to either inform or directly update
16:35:35 <simo> 2. FrereIPA container images are not fully backed yet, and freeipa is monstrounsly complex, I thin it needs more tiem to bake before you can make an official Fedora image for it
16:35:36 <sgallagh> stefw: There's no reason that they could not be.
16:36:04 <jds2001> simo: it is that complexity which makes containers interesting
16:36:10 <simo> 3. I do not know of a resource dedicated to maintain a container image for it with timely updates, so far is more of an experimental thing
16:36:16 <jds2001> simo: it is one testable unit
16:36:23 <adamw> sgallagh: except that multiple choice is unnecessary complexity - more maintenance work, more documentation work, more support work, more user confusion
16:36:37 <simo> jds2001: one testable unit is the only advantage ATM
16:36:50 <sgallagh> adamw: "I see no *technical* reason that they could not be"
16:36:53 <simo> jds2001: but there are a crapton of disadvantages given the missing infrastructure
16:36:57 <adamw> quibble quibble. :P
16:37:08 <simo> adamw: honestly I do not see those as huge issues
16:37:46 <jds2001> also, is this container a systemd based container?
16:37:57 <jds2001> do we want to propgate that bad behavior?
16:38:08 <simo> sgallagh: *who* is going to monitor that 200+ dependencies of the Domain Controller container image and regenerate the image when needed (which is probably going to be at the very least weekly given the humongous number of dependencies
16:38:14 <jds2001> or is it a collection of containers for the various services?
16:38:35 <sgallagh> jds2001: What do you mean by "systemd based container"?
16:38:51 <simo> jds2001: it's a single container with a crapton of services in it, and no SELinux separation between them
16:38:55 <jds2001> sgallagh: a container where the entrypoint is /usr/bin/systemd
16:38:56 <simo> :)
16:40:00 <sgallagh> jds2001: It is not *today*. I am not certain about Jan's plans
16:40:33 <jds2001> sgallagh: how do things get started up in there?
16:40:44 <sgallagh> jds2001: Custom script.
16:40:49 <jds2001> ahh
16:40:57 <simo> sgallagh: anyway I think this is premature because we lack infrastructure for automatic monitoring *in the project* in order to properly regenerate images as needed
16:40:58 <sgallagh> https://github.com/adelton/docker-freeipa
16:41:17 <simo> sgallagh: it is, Jan is using systemd in the container
16:41:26 <simo> he used to use a script that *fakes* systemd
16:41:29 <sgallagh> ...
16:41:45 <simo> but his plan is to move to actual systemd, except he encountered bugs in systemd when used in the container
16:42:10 <sgallagh> jds2001: Can you explain why you see systemd inside a container as a bad behavior?
16:42:11 <adamw> as jds2001 implied, when i looked it up, the standing advice on running systemd in containers was 'don't do that'...but i'm no expert.
16:42:40 <jds2001> sgallagh: it's just that a container is designed to do one thing, run one service
16:42:46 <stefw> depends who you ask
16:42:47 <jds2001> if you need a new KDC, go start one
16:42:54 <jds2001> not the entire stack
16:43:05 <stefw> there are definitely two kinds of containers, and both have valid use cases
16:43:09 <simo> jds2001: it's not so simple
16:43:11 <stefw> this is talked about a lot
16:43:30 <stefw> machine containers vs service containers
16:43:49 <jds2001> machine containers are best called "VM's" and run with kvm :D
16:43:52 <stefw> and there are cases for both of them ... especially when it comes to containerizing already existing applications
16:43:54 * simo thinks machine containers is a VM :)
16:43:59 <stefw> no
16:44:01 <stefw> even systemd
16:44:03 <stefw> has machine containers
16:44:07 <stefw> see nspawn
16:44:13 <stefw> systemd-nspawn
16:44:15 <stefw> and machinectl
16:44:28 <stefw> you can call them system containers if you want
16:44:29 <simo> sure you can use a hammer to push a square peg into a round hole, doesn't mean it is a good idea :)
16:44:41 <stefw> we could all sit around rewriting all our aps
16:44:49 <stefw> for the next 10 years and turn them into micro services
16:44:56 <simo> which won't happen
16:45:00 <stefw> right
16:45:10 <stefw> hence system/machine containers are a pragmatic approach
16:45:23 <simo> I do not think we weant to discuss how the DC Container is internally organized though
16:45:53 <stefw> right, all i meant to say is that we shouldn't trash talk containerized IPA because it's not micro services
16:46:07 <simo> my objections are on the lack of tooling to actually maintain such an architecture at the distro level at this time
16:46:12 <stefw> the approach that has been taken one valid approach
16:46:27 <simo> I think it would be great to have the option and to experiment but not have DC -> Container in F24 officially
16:47:30 <sgallagh> I suppose I could build it as a role implementation that isn't installed by default (maybe part of a rolekit-experimental subpackage)
16:48:03 * simo nods
16:48:10 * jds2001 nods
16:48:14 <adamw> there's a question of, if we make an experimental role, will anyone actually experiment with it
16:48:23 <simo> I think we do need to experiment with this, just we can't make it official yet!
16:48:56 <simo> adamw: at the very least we'll gain the experience needed to make it work in rolekit
16:49:00 <adamw> true.
16:49:00 <sgallagh> adamw is right though... at some point we'll need to have a flag day or it really won't get used by anyone
16:49:10 <simo> until you fully develop the role you won't know what works and what doesn't
16:49:23 <adamw> sgallagh: i wouldn't go that far, i said it was a *question*. people do run wayland, for e.g.
16:49:36 <simo> sgallagh: the flag day will be when we have the necessary infrastructure to maintain the container image
16:49:46 <sgallagh> Yeah, but that's a desktop feature with easy rollback
16:49:48 <simo> right now we have no infrastructure at all
16:49:54 <sgallagh> People tend to be more conservative with their domain controllers :)
16:49:58 <simo> and if we are lucky we'll have a registry with 0 toioling around it
16:50:03 <simo> *tooling
16:50:48 <simo> adamw: wayland crashing won't block a few hundred other services on other machines :)
16:51:31 <simo> but my concern is not even on crashing, it's the update/maintenance process for the images that is not there
16:51:44 <sgallagh> OK, so the general impression I'm getting here is that for F24 we'll subpackage this and treat it as experimental
16:51:55 <sgallagh> Perhaps we can try for F25 for default implementation.
16:52:11 <sgallagh> simo: Right, in the first implementation it's going to be manual.
16:52:30 <simo> (manual is not sustainable)
16:52:43 <sgallagh> simo: I completely agree. But you have to start somewhere
16:54:25 <sgallagh> #info Current plan of record is to have a containerized Domain Controller role available but not the default implementation
16:55:02 <sgallagh> (I might subpackage it or just name the role 'domaincontroller-experimental-container')
16:55:17 <sgallagh> Anything else on this topic?
16:56:17 <simo> not from me
16:56:29 <jds2001> nope
16:56:40 <sgallagh> #topic Docker as a core service (redux)
16:57:04 * jds2001 has a hard stop in 3 minutes :(
16:57:12 <simo> based on the above I am for docker optional
16:57:18 * jds2001 too
16:57:22 <sgallagh> Hang on
16:57:24 <simo> until we have tooling to make avaiulable images that can be maintained :)
16:57:32 <sgallagh> Let's at least get a proposal up to vote on officially
16:57:57 <simo> le tme explain why though
16:57:59 <sgallagh> simo: That's only interesting to the Server Roles, but Docker as a general service has use
16:58:09 <stefw> i don't think making docker optional will have the effect that we gain image maintenance
16:58:23 <simo> currently docker comes with a feature I Call "pull random crap from the internet and run it as root"
16:58:34 <simo> I do not think we can have such a service installed by default
16:58:47 <sgallagh> simo: Oh, you mean like easy_install?
16:58:48 <simo> people would expect it does something sensible and it doesn;t
16:58:59 <stefw> simo, what if it was limited to the Fedora registry?
16:59:01 <simo> (sgallagh: yes, but that is for developers)
16:59:07 <simo> stefw: I am getting there
16:59:09 <stefw> at least by default
16:59:21 <stefw> we have 1 minute left ...
16:59:35 <simo> stefw: once we have the default set to download maintained,. signed content then I would make it an "official" api of server
16:59:38 <sgallagh> Technically we have the room for two hours, but we try to keep it to one
16:59:50 <simo> but again we need the infrastructure to maintain the images
16:59:52 * stefw notes that you guys can keep going ... obviously
17:00:27 <simo> that means scanners that monitor CVEs and automatically rebuild images when security rpm are released and QA is kicked automatically on those images
17:00:31 <sgallagh> simo: My point was that both docker and easy_install can do just about anything as root and both require a user action to do so.
17:00:34 <simo> (or at least we start on that path)
17:00:58 <simo> sgallagh: sure we can debate the fine points
17:01:13 <simo> I maintaine that docker and easy_install are completely different beasts
17:01:40 <simo> as stefw said before doecker is a packaging and deploying system and I add "of RPM level complexity"
17:01:43 <simo> easy_install is not
17:03:30 * stefw says TTFN o/
17:04:07 <sgallagh> OK, just to see if we have five votes in favor:
17:04:07 <sgallagh> Proposal: Drop docker from the default install component and as a core component of Fedora Server
17:04:36 <sgallagh> I'm still -1 here.
17:04:57 <sgallagh> I'm assuming stefw remains the same as well since he hasn't indicated a change of opinion
17:05:14 <danofsatx> +1 drop docker as a core component/default install package
17:05:33 <simo> +1
17:05:34 * nirik is +0.5 I guess. If we are going to just re-add it later when we have tooling it seems kinda silly/shortterm anyhow.
17:06:35 <sgallagh> adamw, jds2001?
17:06:46 <adamw> +1
17:07:27 <sgallagh> nirik: Could you please decide if that's a +1 or a 0? I suspect it's going to matter :)
17:07:45 <nirik> sure, +1 then, we can always re-add it
17:08:09 <sgallagh> OK, and jds2001 appears to have been pretty convinced as well.
17:08:21 <sgallagh> I'm going to assume he didn't change his mind after he left
17:08:37 <sgallagh> #agreed Drop docker from the default install component and as a core component of Fedora Server (+5, 0, -2)
17:08:53 <sgallagh> Mind if I skip Open Floor since we're late?
17:09:38 <sgallagh> #endmeeting