12:31:36 #startmeeting flock2016 12:31:36 Meeting started Tue Aug 2 12:31:36 2016 UTC. The chair is jzb. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:31:36 Useful Commands: #action #agreed #halp #info #idea #link #topic. 12:31:36 The meeting name has been set to 'flock2016' 12:31:51 Dan Walsh doing talk on Containers in Production 12:32:05 put out paper internally, but will be exteral, about containers in production 12:32:33 one thing - talk about devops all the time. New model putting out packages, release cycles, at this point, Docker is concentrated almost exclusively on the developer side. 12:32:51 really concentrating on the developer. Trying to look from the operators point of view, what are operators looking for? 12:33:04 Most container work concentrated on developer, docker-build docker commit 12:33:16 standardization ,new tools can arise 12:33:27 starting to see other groups starting to build container images using other tools 12:33:31 good to have innovation 12:33:34 ansible has playbooks 12:33:42 openshift has a thing called s2i (Source to Image) 12:33:58 talk about how Fedora is doing layered builds, another project going on for a while... 12:34:16 as long as output is the same, we don't have to ship a Docker fully instrumented for building containers in production. 12:34:20 copy on write filesystem 12:34:29 when doing docker builds and commits, what is that? 12:34:44 taking what I had before, COW filesystem 12:35:05 when I'm running in production, I don't want application to be able to modify itself. we want to move towards a state where we're not using a COW filesystem. 12:35:11 COWs have some problems 12:35:22 DeviceMapper, btrfs break shared memor 12:35:25 memory 12:35:49 block based devices, problem with filesystem, kernel figures out that shared memory is the same. 12:36:01 kernel looks @ filesystem, to see block device is the same 12:36:15 as soon as I use DM or btrfs, kernel gets confused, both systems are using same lower-level image 12:36:41 10 containers ... with JRE, on a standard filesystem, you load 1 into memory, with cow you get 10 12:36:49 overlayfs - doesn't have shared memory 12:36:59 working very hard to fix it, SELinux support isn't there 12:37:04 SElinux coming soon 12:37:39 OverlayFS - not posix compliant filesystem 12:37:52 problems based on assumptions with it being posix compliant 12:38:18 open file for write, and open for read only, can wind up with two separate files. 12:38:23 not the same file twice 12:38:55 write performance suffers at larger scales with COW 12:39:04 in production most images should never be modified 12:39:14 don't want applications able to write to /usr when in production 12:39:30 security much better if only can write to certain directories, COW do not support network storage 12:39:34 doesn't work with NFS 12:39:38 lose ability to share images 12:39:52 each container server downloads gigabytes of images 12:41:26 shared images == instant updates - all containers running the same code 12:41:33 get rid of cow filesystems? 12:42:01 in production we want to, but we can't 12:42:05 we have pet filesystems 12:42:11 correction: Pet containers 12:42:29 atomic/openshift registry, taking docker registry service + additional features 12:42:40 when someone does docker push to registry 12:42:45 watches for messages about push 12:42:50 application explodes onto ostree 12:43:03 explodes filesystem to ostree filesystem 12:43:08 first ability to do an atomic scan 12:43:16 this looks for vulnerabilities 12:43:41 approve or block for sharing via docker pull 12:45:55 [video playing] 12:52:59 system containers 12:53:19 on an atomic host, software must be shipped as a container image 12:53:31 how do you handle when software is installed as container? 12:53:41 kubernetes requires etcd and flanneld. 12:53:50 they have to be up before k8s, and docker needs all of those. 12:53:59 these containers can be run with read/only images 12:54:12 docker has problems with container priority 12:54:34 but systemd can give priority to other things, but can't install etcd and flanneld 12:54:37 and need docker 12:54:46 so - came up with concept of system containers 12:54:59 atomic command enhanced to handle system containers 12:55:15 uses skopeo to pull image from registry 12:55:21 use ostree to store image layers on disk 12:55:31 create systemd unit file and uses runc to run container 12:55:47 runc is a standardized part of open container initiative 12:55:57 going to use runc inside systemd unit files 12:56:04 atomic pull --oci etcd 12:56:44 atomic install --oci etcd 12:56:57 systemd enable etc 12:57:01 ... 12:57:11 using runc, systemd, ostree for backing store - but can't update container 12:57:25 when new etcd comes out, need to pull a new container (and build) 12:57:27 simple signing 12:57:32 wanting to have concept of simple signing 12:57:37 similar to what we use with RPM 12:57:47 user can sign document ... "dan walsh trusts this image" 12:58:06 "we dont' trust all images by dan walsh, but we'll sign ones we trust" 12:58:11 so you add signatures 12:58:28 adding concept to these tools, add gpg type signature 12:58:40 allow users to pgp sign a lookaside cache 12:58:46 allow signatures to be stored anywhere 12:59:03 put 'em on a Web site, "there are 40K docker images, but I only trust these 4 - here are their checksums" 12:59:22 really simple stuff, not a hierarchy 12:59:29 letting people work with signing based on this concept 12:59:33 policy rules engines 12:59:37 to say who you trust 12:59:50 set up a policy, only allow docker engine run applications signed by acme inc 13:00:49 introducing the OCID effort 13:01:15 look at new tool building called OCID - it's components needed by kubernetes in production 13:01:25 container image transport - way to get image to local system 13:01:29 need way to store images 13:01:37 open container initiative runtime 13:01:41 a way to start a container 13:01:50 and then I need an api - a wrapper around this application 13:01:58 Skopeo - greek for remote viewing 13:02:29 used by atomic CLI to view container image data on registry (JSON data) 13:02:43 don't have to pull image to look @ json metadata 13:02:59 also figured we could add pull/push registry features. 13:03:14 worked on splitting library from CLI w/CoreOS 13:03:27 skopeo - two parts, cli and library 13:03:34 might eventually be used by RKT 13:04:09 storage - after pull image, need to be able to store on disk, anybody play with Atomic cli 13:04:16 want to look @ content but not run 13:04:34 several releases ago, added ability to mount (e.g., "atomic mount fedora /mnt" 13:04:53 running containers, can do --live + go into container mount point and then go into container and see content show up 13:06:21 graphc, graphtool, cowman, storetool 13:06:40 create container storage ClI & library 13:06:48 https://github.com/containers/storage 13:07:07 tbd - add support ostree, and network storage (Ceph, Gluster, NFS) 13:07:37 CoreOS could use it as its backend 13:08:23 open container initiative , runc developed in OCI, 13:08:39 runc is just an initial implementation of implementation 13:08:51 clear linux is about to release a specification to run machines 13:08:58 they follow oci spec 13:09:12 docker 1.11 uses runc as default backend 13:09:29 oci tool https://github.com/opencontainers/ocitools 13:11:26 OCID container management API 13:11:36 Open Container Initiative Daemon 13:11:40 least developed part of project 13:11:50 impelments kublet container runtime interface 13:12:02 kubernetes / google server interface launching / running pods 13:12:08 kubernetes wants to execute 13:12:15 kubernetes uses skopeo to pull image 13:12:27 ocid stores image on disk using storage 13:12:32 then ocid runs container using runc 13:12:47 standards based alternative to docker and rkt runtimes 13:12:57 want to get into OCI 13:13:04 not trying to replace docker 13:13:16 trying to supplement - 13:13:36 base on kubernetes but implementing same functionality 13:13:39 for atomic 13:13:52 move openshift to OCID by default 13:14:00 and then have that standardized 13:14:18 where docker is constantly changing, we dont' have to swap things out based on changes in Docker upstream 13:14:35 containers in production will focus on system containers for ... in production 13:15:26 non-production container == under development 13:16:56 question: Copr recently sucks 13:17:09 A: we are aware of that and are trying to fix it as hard as we can 13:17:44 huge rebuilds cause problems, some services time out... 13:17:56 we're trying to change the queue logic 13:19:55 #endmeeting