17:04:24 #startmeeting Env and Stacks (2015-04-16) 17:04:24 Meeting started Thu Apr 16 17:04:24 2015 UTC. The chair is hhorak. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:04:24 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:04:25 Bike is nice and w/o traffic :) 17:05:12 #meetingname env-and-stacks 17:05:12 The meeting name has been set to 'env-and-stacks' 17:05:16 #chair bkabrda hhorak juhp ncoghlan vpavlin sicampbell walters ttomecek phracek 17:05:16 Current chairs: bkabrda hhorak juhp ncoghlan phracek sicampbell ttomecek vpavlin walters 17:05:30 * mattdm joins -- sorry, lost track of time 17:05:32 #topic modular fedora next 17:06:08 hello 17:06:19 hello :) 17:06:26 Hi. Interesting topic for fedora and RHEL. 17:06:43 I was thinking how to kick off today's kind of special meeting (because of topic) and honestly don't have anything special :) so how the future will look like? 17:06:58 shiny? 17:07:20 anybody has Crystal ball? 17:07:23 http://mattdm.org/fedora/2015status/rings2015.svg 17:07:36 * langdon distracted by squirrel 17:07:38 mattdm: that seems quite similar to crystal ball 17:07:45 mattdm: Thanks for your email regarding rings. It was helpful for me. 17:07:45 hhorak: exactly! :) 17:08:10 I don't think we know the exact answer, but I think that's a reasonable general picture 17:08:39 What I'd like to see is an Objective proposal for a concrete stab at figuring out an implementation of this 17:08:45 Example objective proposal 17:08:47 https://fedoraproject.org/wiki/Objectives/Fedora_Editions,_Phase_2 17:09:16 since this is different it doesn't have to look exactly like that model... 17:09:23 mattdm: I have a lot of question for discussion. Or maybe better understanding for me. 17:09:28 but the headlines are the key points to fill in. 17:09:47 Let's say that I have installed Workstation. Does it mean that I can switch to another ring like Server? 17:10:14 er, rings aren't workstation, server, etc 17:10:17 One approach might be to make a "skunkworks" project to implement an ostree-based base + cloud/server/workstation 17:10:18 If so then I can see the future. 17:11:09 phracek: cloud/server/workstation are on a different "image layer" above the rings 17:11:10 mattdm: do we have any alternative for ostree? 17:11:10 mattdm: So you see future Fedora versions being ostree-based? 17:11:30 vpavlin: Right now, i see it as the best candidate technology and worth exploring. 17:11:46 i think it helps with a lot of things that just plain RPM doesn't 17:12:06 jwb can you enumerate some? 17:12:06 like shipping OS updates as a cohesive set 17:12:11 (ooh, yes you can) 17:12:12 * vpavlin should have scrolled on when looking at that image:) 17:12:14 rollback of the OS 17:12:15 etc 17:12:40 but it is definitely a departure from today's Fedora delivery and process 17:12:51 jwb, "switch os" too.. e.g. f21->f22alpha->f21 17:13:20 most of my thoughts on this are in https://lists.fedoraproject.org/pipermail/desktop/2014-December/011211.html 17:13:21 langdon, possibly, but i think i'd start just within a release 17:13:26 it also gives an abstraction layer where we're composing something out of the package set, which gives as an ability to that deparature without totally ripping up what we have 17:13:33 maybe ostree doesn't have to be compulsory, just default and recommended way.. we can still get to the same result with rpms if really required (some users may want it) 17:13:44 ok, makes sense..do you see us using this in case of WS: https://wiki.gnome.org/Projects/SandboxedApps ? 17:13:49 #link https://lists.fedoraproject.org/pipermail/desktop/2014-December/011211.html 17:14:01 jwb: you would like to replace of stop RPM? 17:14:12 #info jwb ostree helps with a lot of things that just plain RPM doesn't 17:14:15 vpavlin, imo, xdg-apps would work on top of ostree 17:14:24 phracek, i'm not sure i understand your question 17:14:33 one alt approach is overlayfs (layering), however, that probably has less "testing" (for this use case) than ostree 17:14:40 jwb: I guess they would:) I am just curious if we all think about same technology 17:15:06 phracek, ostree requires RPMs. i don't see it replacing them at all. i see it leveraging them in a better way 17:15:13 jwb: I don't understand this "i think it helps with a lot of things that just plain RPM doesn't" 17:15:17 vpavlin, that, and docker containers, etc 17:15:32 jwb++ 17:15:39 jwb, *rpm*-ostree requires rpms 17:15:52 walters, sorry, correct. i've been meaning rpm-ostree here 17:16:04 I'd like to see more convergence between the desktop containerization use cases and the server/cloud ones, but right now they've got different immediate problems to solve 17:16:16 jwb: Would you like to have several containers above ostree? 17:16:17 can somebody summarize what is xdg-app in one sentence and put it in #info, please? 17:16:32 phracek, rpm-ostree allows us to take a set of packages, create a tree (or image), QA that _set_, and ship it as a single thing. 17:16:44 jwb: like http -> one container, DevAssistant another container , ... 17:16:50 phracek, but before we use that, i think we need some layering capabilities 17:16:58 phracek, not necessarily but it's possible 17:17:22 #info rpm-ostree layering capabilities seem really important to this plan 17:17:34 hhorak: one sentence? i'm not sure I can :) 17:17:36 jwb: Personally I don't think that this could be a way. A lot of containers. But may be it is a feature for any OS. 17:17:41 Like containers. 17:18:00 phracek, yes, i see containers as an add-on feature. 17:18:20 Or better say. The system will be a collection of containers. And any can be turn off on the request. 17:18:48 phracek: I think that's an interesting long-term structure, but I'm not sure how to get from here to there in a straight line 17:18:49 no, i'm not envisioning that as the default 17:19:09 do I understand it correctly that "rpm-ostree layering capabilities" means layers of different rpm-ostree content? 17:19:10 What is the release plan for rings? F23 or later on? 17:20:11 * phracek nods for ostree and layering like ostree and switching to several rings. It would be really interesting for developers and for users. 17:20:14 convo seems to be diverging... and, I, for one, am getting mildly confused 17:20:17 #link https://github.com/alexlarsson/xdg-app 17:20:31 timeline: some aspects in F23. I'm seeing most of this as "demo in F23 timeframe, deliverable in F24/F25" 17:20:45 mattdm, i think that's slightly optimistic 17:20:52 ostree layering usually means "layer from fedora" + "stuff from somewhere else" + "my stuff" as I understand it 17:20:55 hhorak: It's basically a technology to sandbox a gui application 17:21:02 Sometimes I would like to use system as a workstation for standard user work and sometimes I would like to develop under server. 17:21:07 jwb: mockup in F23, demo in F24? :) 17:21:13 mattdm, yes, possibly 17:21:15 vpavlin: you meant xdg-app? 17:21:15 hhorak, my personal opinion on this is that if what's desired is partial filesystems with metadata, RPM is OK (and the right thing is to fix it) - that's the direction i've been going upstream, but certainly the design is open 17:21:23 hhorak: yes, sorry 17:21:55 jwb: it means that after a year (F24) demo is going to be ready. I am not so optimistic. 17:22:02 phracek: that's probably a good use case for a VM 17:22:08 regarding, "everything containers" ... I am +1 on that idea.. i think a small "core" (base-wg?) plus containers for all apps is probably the long term OS.. some of those containers are local and some are remote.. 17:22:24 #info xdg-app is basically a technology to sandbox a gui application (very briefly said) 17:22:52 mattdm: yeah, but I think it is not case for rings, right? 17:23:03 so do things in "Fedora RPM collection" move at different speeds? 17:23:06 langdon, i'm thinking layering in this context is more like... "core" rpm-ostree + gnome rpm-ostree + xdg-apps 17:23:20 or are they all released every 6 months? 17:23:32 langdon, not fedora + 3rd party + my junk 17:23:40 phracek: cloud/server/workstation is a different things from rings 17:23:45 langdon: me not. users should learn a new system capabilities. I am a pretty scared. 17:23:48 walters, now or in the glorious future? 17:24:12 now none of the exists right? 17:24:27 walters: I think we need some specific use cases for things moving at different speeds and a picture of what that'd look like 17:24:27 well the outer ring exists 17:24:30 the Fedora RPM collection exists 17:24:42 right now, I think mostly i have a vague idea that it'd be useful for some things. 17:24:49 phracek, was that in re to container-all-the-things? or ostree-layering? 17:25:03 walters: I see use cases for different speeds, but then we'll break things to users more often :) 17:25:08 langdon: first one. 17:25:11 * vpavlin gets really confused by interleaving conversations:) 17:25:29 * mattdm is going offline for a few minutes, then back, then gone again. 17:25:40 jwb, sure.. i guess I am thinking distinction w/o a difference.. e.g. my "3rd party" might be gnome.. but.. your way is probably "cleaner" in the near term.. 17:25:46 hhorak: That's fine, it's Fedora, we are used to having it broken.. 17:26:03 #info (different speeds): I think we need some specific use cases for things moving at different speeds and a picture of what that'd look like 17:26:17 langdon, true. but i'm taking a "what does _fedora_ deliver 17:26:19 phracek, but this is the crystal ball :) ... the caps of a user's system don't have to be present locally in an "always conneted" world.. :) 17:26:20 er... 17:26:27 "what does fedora deliver" standpoint 17:27:30 vpavlin: I agree. we should discuss one thing and then go to the other. Sorry. It was probably done from my side.. 17:28:19 vpavlin, oddly enough, i'd like to leverage rpm-ostree to make fedora much less broken. we shouldn't be used to it 17:28:34 * phracek is thinking about ostree and use cases. 17:28:50 and to be clear, some of the concepts i'd like to see could be done with other tooling 17:29:05 btrfs snapshots, reworked updates mechanisms, etc 17:29:15 jwb: Yeah, the "image" based distribution would be interesting 17:29:21 but i think rpm-ostree provides the best stability and tradeoffs thus far 17:29:35 * phracek nods 17:29:37 #info "oddly enough i'd like to leverage rpm-ostree to make fedora much less broken. we shouldn't be used to it" -- jwb 17:29:50 assuming we can get some layering in ways that work 17:30:02 jwb: So, should we also include systemd guys into the picture with Lennart's proposal of new OS? 17:30:21 (I know it's also a crystal ball how all that will/could work, just thinking) 17:30:22 vpavlin, i think we should involve them. i don't necessarily think we should focus on Lennart's specific proposal of using btrfs 17:30:29 but the concepts are very similar, yes 17:30:32 * mattdm nods to that 17:32:09 vpavlin, we definitely need some sort of meetings or design groups that involve the systemd developers 17:32:24 vpavlin: "image" like download from a fedora infrastructure? And you only select kind of images which you would like? 17:32:56 we should invite harald hoyer as representative of base design -- and also systemd, so that's convenient 17:33:02 so is the rpm-ostree based system we talk about here just the small ring or does anybody thinks about it like e.g. whole cloud product may be one ostree? (not taking workstation into this thinking) 17:33:03 seems to be good. And really useful. I can imagine it. 17:33:35 hhorak I think the whole thing. 17:33:41 But, I have to go.... 17:33:49 I expect this will be all worked out when I check back in :) 17:33:53 * phracek nods to that. 17:34:49 phracek: by image based I mean you'll get a "minimal install" of Server as an image base on rpm-ostree - and then extend by using containers and/or rpm-ostree layeres as mentiond before 17:35:03 thx. 17:35:12 so is the process here we're looking at this from an env-and-stacks perspective and then the proposal rotates to other groups and is potentially modified by them? 17:35:49 walters: that sounds fine to me 17:35:51 What about to separate whole product from rpm-ostree? Does it make sense? 17:36:26 cloud already has a whole rpm-ostree image. it's already done and was for f21 17:36:34 that is what atomic cloud is 17:36:40 ok 17:36:51 what we're talking about is taking that and extending it to more than just cloud. like workstation. 17:37:21 hhorak, walters i thought the envs&stacks group "set the policy" ... and then the editions implemented their tools based on it.. (obviously, they should be involved in the policy setting, but once it is done, they shouldn't change it unless envs&stacks agrees) 17:38:34 langdon, hmm but...where does the collaboration/discussion happen? 17:38:37 langdon: That does not conflic with walters's expextation? 17:38:38 i think we're talking about it here because we're going to be talking about it with all the groups. how each WG plays into the overall picture is still up for discussion 17:38:54 so Env & Stacks, Base, the product WGs, rel-eng, etc 17:38:54 langdon: thinking what is the policy... tools, technologies? guidelines how to make all the app look similarly.. what else? 17:39:02 how do you imagine that a user adds a single app? like hexchat? :) in the ostree world? do they compose their own tree? where does config live? in the user's tree? local? is a goal that a user can get back to their system "in toto" after a crash? 17:39:50 I would prefer standard installation. By dnf. Of course not container. 17:40:19 phracek: so you're too old-style hacker for the new fedora.. :) 17:40:27 jwb, walters i think i am just clarifying that, like the rpm guidelines, a WG should not modify the guidelines w/o consent of this group (who will ensure the guideline change meets the needs of all constituencies) 17:40:42 Or we can define a group with apps and make container with it. 17:40:56 langdon, sure 17:41:10 phracek, why wouldn't "dnf install my-container-version-of-hexchat" work? that is basically what the xdg-app folks are thinking 17:41:18 btw the rpm-ostree and dnf people do talk somewhat regularly now although mostly around technology sharing, not the "end picture" 17:41:52 phracek: this all will be build above rpm.. and some non-rpm stuff maybe... but the point is you should still be able to install things by yourself..if you want... (right?) 17:42:01 langdon, oh yes! if we can make this mostly seamless from that standpoint, all the better 17:42:15 hhorak, in my head, definitely 17:42:28 langdon: each application will have its own container? 17:42:37 hhorak: ok. 17:42:48 in the gnome-apps (or whatever it is called) model, yes 17:43:00 in my world.. yes.. 17:43:08 and they all communicate over a magic bus :) 17:43:19 like crystal ball:) 17:43:30 * langdon http://i1.kym-cdn.com/entries/icons/original/000/005/896/Magic-School-Bus.jpg 17:43:51 langdon: lol 17:43:55 * vpavlin sees some unicorns here.. 17:43:56 like crystal ball:) 17:44:17 I can see a problem like we are building a new thing above a simple apps. 17:44:28 below actually 17:44:31 langdon: The blue bus is calling us... but that's quite long way to go, right? what will be the route? will just apps run in container work in real use cases? 17:44:33 but yes, we are building a new thing 17:44:38 (without bus) 17:44:39 i basically am thinking of a services oriented desktop/os .. so.. there is a well known way to communicate (e.g. dbus) between things, there is a way to raise "intents" (a la android), and all the things are independent 17:45:04 hhorak, i actually have started using them a ton... and am getting annoyed when I need to do a "real" install 17:45:10 :) 17:45:50 jwb: well I can imagine it. But hopefully system will not be busy with a lot of containers. That's why I am a bit scary. 17:45:51 vpavlin, if you look hard enough, they aren't unicorns. they are horses with big warts that need fixing. then you just have horses. 17:45:56 so it's probably worth looking at this from the "what would break" perspective 17:45:58 so.. it is a "hard" problem when you think about gui apps.. which is what gnome is trying to solve.. but when you are talking about "Server apps" ... like httpd.. it is much simpler 17:46:27 i can certainly tell you that I want rpm-ostree package layering myself after using it a lot 17:46:29 phracek, you are worried about the overhead of the container mechanisms themselves? 17:46:53 another example is around manageability; with both xdg-app and docker, modifying the ca-certificates is now much more painful 17:47:05 no about containers but like a lot of VMs makes the problem with system performance. 17:47:15 But may be I am from old school:) 17:47:17 you might even be using an ubuntu-based app container and not realize it until you go "where's /etc/pki/"? 17:47:25 phracek, i think "performance" (in a lot of definitions of that word) is a real concern... basically interproc comm begins network-comm, always.. but... com/dcom fixed that ~15 yrs ago.. so I think we might have some ideas 17:47:32 phracek, containers and VMs have totally different overhead characteristics. nobody is talking about running VMs here 17:47:46 jwb: But you are right that just install a new container with already setup application is awesome. 17:47:50 *begins = becomes 17:48:13 there will be overhead for sure, but i'm not sure it's enough to make this not worth investigating 17:48:30 langdon: jwb: ok, I will be a more optimistic. 17:48:43 well.. and the possibilities are sooooo interesting ... 17:49:40 walters, langdon: i'd like to start even more modest. like... take Atomic Host, expand it a bit to make it more useful than the docker host thing, put another "desktop" rpm-ostree layer on top, and then see where we are 17:49:58 jwb, +1 17:50:05 you guys are looking at the end game. i'm wanting to start prototyping based on what will be available relatively soon 17:50:20 and if that means helping with the rpm-ostree layering, great. let's find some people to do that 17:50:27 jwb, note that ostree is wildly different from docker in that it doesn't do layering in that way 17:50:44 it's more like git - if you have two branches they transparently share storage, but they're distinct entities 17:50:51 langdon: yes. If I install a container-appl with settings and on the new system I will have the same, that it is really nice. 17:51:05 i would also like to see an "answer" to my q about "where does the user config live".. in the near term, in can be the ostree model of just "on disk in special places" ... but I think in the longer term, I would like to see "my tree" which contains my config as a layer on the rest 17:51:10 so if you compose the same RPMs into a new branch, it will share storage with the atomic host, but it's not a layer 17:51:36 now we could *make* rpm-ostree do that kind of layering if the end goal is really bitwise identical layering 17:51:54 walters, right and i think we do 17:52:19 how do you capture things like workstation doesn't have ssh enabled by default, server does? 17:52:51 with the "separate commits" model, this is very clear in ostree - they have slightly different /usr/etc (or /usr/lib) 17:53:25 whereas if it's layering...then workstation needs to carry a "disable ssh" bit? 17:54:06 * langdon grumbles about why wkstn has ssh-server disabled by default.. 17:54:45 walters, either could work. i mean, if the end goal is to be able to take a core tree, slap the gnome tree on top, and then slap the product differences on top of that... whatever makes sense 17:55:06 walters, but the idea is that core layer is common, the gnome layer depends on core and works on either product. 17:55:34 common between the rest of the layers 17:55:35 jwb: how the layer will communicate? 17:55:48 the terminology is getting slightly confusing. pictures would help 17:56:11 jwb: great idea:) 17:56:19 jwb, actually.. and i am not sure if this is a good point, but I think you point out... how do we decide which way the layers work? I was thinking it was base+wkstn+gnome & base+server+gnome .. and you have it differently 17:57:29 could be just QED, exercise for the reader, (aka the WG decides based on what they need).. I am not sure if it matters 17:57:57 sorry guys kids are fighting. I have to leave. 17:58:12 Thanks for nice conversation. Bye. 17:58:23 langdon, i think it's a good point, but yeah it might not be clear until we try some stuff 17:59:04 * langdon totally thought phracek was referring to us :) 17:59:21 haha 18:00:01 langdon: your way seem reasonable, the other way would be base+gnome+wkstn? that wouldn't make sense.. 18:00:33 hhorak, sure it would, if you think about people that want to not run a productized install 18:00:44 gnome isn't specific to workstation. it's just a DE 18:00:44 hhorak, per jwb, I am not sure we know yet.. 18:01:01 but yeah, we don't really know yet 18:01:17 and none of this is really possible now anyway :) 18:02:10 thought it is not possible to install non-product fedora any more 18:02:34 but yeah, it may be valid use case.. 18:02:37 there's no media, but you can still do non-productized installed via boot.iso and kickstart 18:02:41 hhorak, can i propose that discussion be later? or something else? 18:02:53 i think we're kind of tailing off here anyway 18:03:01 hhorak: You can install non-product Fedora with any spin... 18:03:02 can i ask the group.. have we (technically "you") made any decisions yet? actions? 18:03:11 Or the netinstall media, etc. 18:04:54 langdon: I have the impression we're all on the same page the future is in layered rpm-ostree and containers, right? 18:05:27 action for container part is definitely to be able to deliver containers in the first place. 18:05:30 hhorak, i think i'd phrase it as "a future really worth looking into" 18:05:47 jwb: or the best idea we have now 18:06:04 hhorak, sounds about right to me.. but how do we get from here->there? i assume we need an "objective" (or is it an "initiative"?) or, to jwb's point, maybe some "spikes" (per agile terminology) 18:06:08 hhorak, aside from just keeping on with what we are doing, yeah. 18:06:26 langdon, i think we need people willing to work on it, and a prototype 18:06:56 because trying to have this discussion without something people can play with is going to be a very long and fraught process 18:07:10 jwb, i guess i think we need to write up what the prototype is .. so that people other than those who are here can help and to make sure we all think we are prototyping the same thing :) 18:07:34 yes, exactly. otherwise, take this meeting (which has actually gone really really well!) and try to hold it over devel list and... uuf 18:08:42 walters: do you see any specific actions for now for rpm-ostree part? (I still don't feel experienced enough in that field) 18:09:22 investigating "binary layering" versus "local rpm layering" is one 18:09:36 s/binary/tree/ i guess 18:09:46 or maybe "plus" and not "versus" 18:10:16 right 18:10:18 i'm pretty sure i need to sit down and read more of the rpm-ostree stuff myself anyway 18:11:48 hhorak, beyond that...i think the requirements here mostly align with the existing roadmap 18:12:11 walters: sounds fine 18:13:21 i need to drop off for a bit. i appreciate everyone participating today. i think it's gone well 18:13:28 langdon: as you mentioned deamons are easier for working with them in containers, I'd start with those.. 18:13:37 jwb: thanks too! 18:14:55 hhorak, start? like container-all-the-things? yeah.. i think there are lots of starts all over.. but there is no unified "target" so they all go different directions in scope and goals and the like.. but i am not sure what you mean by "start"... 18:15:45 langdon, start by defining 'fockeret'. the Fedora Docker/Rocket project. ;) 18:16:22 * jwb really leaves now. thanks all! 18:16:29 jwb, ha.. well.. as a web developer (essentially) rocket is too smart for me ... 18:17:58 * vpavlin will have to read through the log as he got distracted a lot, sorry:( 18:18:13 vpavlin: who is there with you? 18:18:42 hhorak: fkluknav... 18:18:43 * mclasen_ reads through the logs as well 18:18:54 which interesting meeting did I miss here ? 18:19:58 mclasen_, envs&stacks 18:20:20 * mclasen_ should come to those, good stuff 18:20:51 langdon: about the start.. I meant first things to containerize (yeah, I mean docker here, since it's most widely used now)... and thinking about some unified approach how to use those without user noticing they are containers actually.. otherwise I don't think I have any nicer answer for you.. 18:21:40 hhorak, ohh.. yeah.. i do that already.. but, as I brought up last time, i am stuggling with where to put them because having them live in f-dockerfiles forever seems like a bad plan... 18:23:53 ah, right. we were talking about upstream being the right place where they should live, right? 18:24:54 hhorak, yeah.. although we need an aggregator.. i think we discussed containers.fp.o or containers.projectatomic.io .. but not sure where that went next.. 18:25:05 jzb, ^^ 18:26:24 isn't that a registry? 18:26:46 walters: can't we use docker hub in fedora? 18:26:58 sure 18:27:34 https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2014-October/msg00002.html 18:28:30 one angle I look at this from is currently there is a clear line for "fedora" or "not fedora" that boils down to "signed by the GPG key" 18:28:36 walters, sorry.. maybe not the right name.. this is meant to be a "website" with look at all the containers built for fed/cent/rhel or whatever... with base image info, maybe best practices, etc 18:29:48 langdon: there is a red hack week the next week (internal, sorry community) and one of the topics is also about trying to go deeper in developer.fedoraproject.org idea.. hopefully somebody picks it and produces some general concept.. 18:30:12 if not, then we'll have to look at it. topic for next week agenda? 18:30:24 hhorak, ack 18:30:32 (it's getting dark here and the streets are getting dangerous..) 18:30:53 hhorak, i want to laugh at that.. but I think you are serious :) .. bike? feet? 18:31:16 langdon: no, that was really a joke, I'm already at home.. 18:31:23 ..but vpavlin is not, right? 18:31:24 hhorak, ha 18:32:01 * vpavlin is still in the office...building rsyslog image.. 18:32:23 * hhorak thinking about closing soon, my mind protests to bring any reasonable ideas anyway.. 18:32:46 vpavlin, where do i find eliska(sp?) she hang out on IRC? I have to find her to talk about vagrant builds one of these days 18:33:05 hhorak, you should do like me and talk crazy all the time... 18:33:16 langdon: #brno? #devexp? 18:33:22 vpavlin, ack 18:33:27 langdon: I'll start if will continue to work 13h a day 18:34:01 anyway, thanks for the discussion! 18:34:26 #endmeeting