13:03:32 <mmaslano> #startmeeting Env and Stacks (2014-09-30)
13:03:32 <zodbot> Meeting started Tue Sep 30 13:03:32 2014 UTC.  The chair is mmaslano. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:03:32 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
13:03:38 <mmaslano> #meetingname env-and-stacks
13:03:38 <zodbot> The meeting name has been set to 'env-and-stacks'
13:03:51 <mmaslano> #chair pkovar tjanez samkottler bkabrda hhorak juhp mmaslano vpavlin sicampbell ncoghlan
13:03:51 <zodbot> Current chairs: bkabrda hhorak juhp mmaslano ncoghlan pkovar samkottler sicampbell tjanez vpavlin
13:04:01 <mmaslano> #topic init process
13:04:08 <juhp_> hi
13:04:14 <bkabrda> hi
13:04:33 <ncoghlan> evening
13:05:00 <mmaslano> bkabrda: let's start with Koschei recommendation
13:05:04 <mmaslano> right?
13:05:07 <bkabrda> yep
13:05:14 <mmaslano> #topic recommendation of Koschei
13:05:29 <mmaslano> you didn't like wording from last week, so what would you prefer?
13:07:05 <bkabrda> mmaslano: sth. like "Env & Stacks WG recommends all Fedora packagers to use Koschei as means of tracking build failures on dependency updates" (although maybe that should be phrased by a native speaker, not sure this is absolutely ok ;))
13:07:53 <mmaslano> sure why not, but such recommendation are like guidelines :) we have hundreds of them
13:08:11 <juhp_> sounds okay to me - dunno if "all" is too strong maybe?
13:08:22 <bkabrda> mmaslano: my idea was just about saying something more specific than "good idea", which was in the original proposal
13:08:42 <mmaslano> okay
13:08:45 <bkabrda> juhp_: we can omit "all" and it will sound just as good :)
13:08:51 <juhp_> yes
13:08:59 <juhp_> or better even :)
13:09:11 <mmaslano> #info Env & Stacks WG recommends Fedora packagers to use Koschei as means of tracking build failures on dependency updates
13:09:11 <ncoghlan> bkabrda: I think what may not have come across was that we weren't entirely convinced Koschei was baked enough yet to recommend it unreservedly
13:09:30 <ncoghlan> in particular, adding packages is still fairly manual
13:10:05 <ncoghlan> I think ^^^ recommendation is the recommendation we *want* to be able to make
13:10:13 <bkabrda> ncoghlan: that is true, however we're recommending the idea of the tool, not the tool in its current form, IIUC
13:10:43 <ncoghlan> perhaps something more like this:
13:11:13 <ncoghlan> Env & Stacks WG recommends adopting Koschei as the recommended means of tracking build failures on dependency updates
13:11:36 <ncoghlan> that blesses Koschei, without actively directing packagers to use it yet
13:11:45 <bkabrda> ncoghlan: good wording! I'm +1 for that
13:12:08 <mmaslano> +1
13:12:10 <mmaslano> #undo
13:12:10 <zodbot> Removing item from minutes: INFO by mmaslano at 13:09:11 : Env & Stacks WG recommends Fedora packagers to use Koschei as means of tracking build failures on dependency updates
13:12:12 <juhp_> +1
13:12:18 <vpavlin> +1
13:12:29 <hhorak> +1
13:12:36 <mmaslano> um I can't vote anyway, when I gave my vote to ncoghlan
13:12:41 <ncoghlan> +1
13:12:45 <pingou> Env & Stacks WG advices adopting Koschei as the recommended means of tracking build failures on dependency updates # to avoid repeating recommend
13:13:02 * pingou nitpicks
13:13:48 <ncoghlan> Env & Stacks WG recommends adopting Koschei as the suggested means of tracking build failures on dependency updates # perhaps?
13:14:05 <pingou> even better :)
13:14:07 <juhp_> okay
13:14:19 <bkabrda> pingou: good catch :) although the last proposal from ncoghlan sounds the best to me
13:14:20 <mmaslano> ncoghlan: pick something what make sense. You are the only native speaker :)
13:14:28 * vpavlin is for ncoghlan's version as it states this is a recommendation
13:15:05 <ncoghlan> aye, the repeated recommends was a good catch, but I think it's better to replace the second occurence
13:15:22 <juhp_> yea agree
13:15:23 <bkabrda> ncoghlan: +1 to last nick's proposal
13:15:40 <hhorak> still +1
13:16:10 <juhp_> another alternative would be s/suggested/default/
13:16:24 <juhp_> but ncoghlan is fine for me
13:17:01 <ncoghlan> it may be the default some day, but I think it's still only a suggestion at this point
13:17:09 <mmaslano> #agreed Env & Stacks WG recommends adopting Koschei as the suggested means of tracking build failures on dependency updates
13:17:20 <juhp_> that's true probably needs more use first
13:18:09 <juhp_> good
13:18:44 <mmaslano> #topic conda - everyone should look at it
13:19:04 <mmaslano> honestly I skipped that because I spoke to bkabrda about it and he didn' persuade me it's cure for everything
13:19:08 <hhorak> bkabrda: anyway, fedoraproject wiki is silent about koschei (https://fedoraproject.org/wiki/Special:Search?search=koschei); do you happen to know if koschei guys plan to prepare some cookbook for koschei? I would rather wait with publishing the "recommendation" until we have some info about it...
13:19:16 <mmaslano> #undo
13:19:16 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x27f15f10>
13:19:28 <ncoghlan> yeah, conda isn't the cure for everything
13:19:39 <mmaslano> hhorak: I asked two weeks ago and it's in todo list
13:19:47 <mmaslano> hhorak: now they are trying to move it to fedora-infra
13:19:55 <ncoghlan> it has some interesting ideas though, specifically with its focus on user level installs
13:20:00 <bkabrda> (hhorak: I'm not koschei developer, I was only asked by koschei developers to promote it. but yeah, I'll ask them to write some docs on fedora wiki)
13:20:31 <mmaslano> #action bkabrda -> koschei developers should write about Koschei on Fedora wiki
13:20:35 <mmaslano> #topic conda - everyone should look at it
13:20:49 <hhorak> bkabrda: thanks
13:21:14 <ncoghlan> I probably should have sent a mail through to the list regarding the aspects of conda that I find interesting
13:21:46 <ncoghlan> the two primary ones are:
13:21:54 <mmaslano> bkabrda: i didn't remember those downsides of conda, but it wasn't very useful for our usecases, right?
13:21:57 <ncoghlan> 1. user level installs by default
13:22:14 <ncoghlan> 2. virtual environments by default
13:22:34 <ncoghlan> oh, and one I forgot: cross platform, so it works on Mac OS X and Windows
13:22:54 <juhp_> ncoghlan, what is 2?
13:23:10 <juhp_> not sandboxing?
13:23:11 <ncoghlan> that last one in particular makes it potentially relevant to Java, Python, et al packaging in a way that an RPM-based solution isn't
13:23:29 <ncoghlan> juhp_: no, more like what you get with Python or Ruby environments
13:24:05 <bkabrda> mmaslano: it's a bit hacky (doing binary replace of builtin RPATHs during installation) and I'm unsure how it would handle e.g. cross-environment dependencies or stuff like systemd units, etc. also, it doesn't track installed environments, so there is no global "yum update"... and more
13:24:09 <ncoghlan> adjusting environment variables and language runtime settings to look in the active environment, rather than at the whole system
13:24:43 <ncoghlan> bkabrda: right, it wouldn't work for things like *services*
13:24:54 <juhp_> ncoghlan, okay thanks
13:25:11 <ncoghlan> but for components loaded by a service, or for pure user space applications, it has a lot to recommend it
13:25:38 <ncoghlan> the original use case Continuum Analytics built it for was data analyst working environments
13:27:15 <ncoghlan> the interest thing about it from my perspective is that it can give you dependency management that works on Windows and other non-RPM environments
13:27:29 <juhp_> it sounds a bit like Haskell's cabal-install but that probably doesn't help many here...
13:27:44 <ncoghlan> and one of the ongoing challenges with cross-platform language devs is getting them interested in platform specific installers
13:28:14 <ncoghlan> while multi-language platform devs are generally less in interested in *language* specific installers
13:28:40 <ncoghlan> conda ends up being a "cross-platform platform" that you can use without admin rights on your machine
13:29:39 <ncoghlan> it certainly has some *very* rough edges, but it's also something I don't think we've really had before
13:30:44 <ncoghlan> I don't think it's a replacement for software collections, but I do think it may be interesting for some cases neither SCLs nor language specific tools handle well
13:30:44 <juhp_> can it be packaged? :)
13:30:55 <mmaslano> ncoghlan: do you have some general recommendation (pros,cons)?
13:31:02 <ncoghlan> it's pip installable already, so probably
13:31:09 <juhp_> cool
13:31:31 <ncoghlan> at this point, I think it's still in "this is potentially interesting" territory
13:32:33 <ncoghlan> near term, looking at things like devpi seems more important
13:33:05 <ncoghlan> I also came across https://github.com/christian-posta/ceposta-devops-ose/blob/master/docs/demo.md
13:33:19 <ncoghlan> which pretty much describes the kind of workflow I'm also aiming for
13:34:24 <ncoghlan> although Christian uses a Java example, there are fairly direct mappings to other languages, like swapping out Nexus for devpi
13:34:53 <mmaslano> this WG should aim on all languages. This seems to be working only on small part of them
13:35:04 <mmaslano> maybe python-devel would be more appropriate audience
13:35:23 <ncoghlan> mmaslano: it's more on the "stacks" side of things
13:35:47 <ncoghlan> the language communities build language specific tools for reasons that actually make sense
13:36:07 <mmaslano> I don't think we have manpower to work on everything. vpavlin proposal sounds like more doable in short time
13:36:32 <ncoghlan> what was vpavlin's proposal?
13:36:49 <mmaslano> it wasn't proposal but pointer to http://blog.docker.com/2014/09/docker-hub-official-repos-announcing-language-stacks/
13:36:54 <mmaslano> #url http://blog.docker.com/2014/09/docker-hub-official-repos-announcing-language-stacks/
13:37:15 <ncoghlan> ah, no, that doesn't solve the problem
13:37:47 <vpavlin> What is the problem we are trying to solve with conda?
13:37:50 <ncoghlan> it just changes it from a "how do we deploy the software?" question to a "how do we define the image?" question
13:38:00 <mmaslano> yes, maybe we should start with what's the problem
13:38:57 <ncoghlan> for me, the problem I'm trying to solve is sustaining engineering of Java, Python, Perl and Ruby web services in a corporate development environment
13:39:16 <ncoghlan> the model in Christian's post is basically perfect for that
13:39:27 <ncoghlan> with language specific artefact repos
13:39:50 <ncoghlan> and an underlying common RHEL/RHSCL/OpenShift infrastructure layer
13:39:53 <vpavlin> Ok, then there are 2 problems - sustaining engineering and how to make it easy for developers to get from idea to writing the code:)
13:41:10 <ncoghlan> I look at it a bit differently: my aim is to allow development and operations to work together to provide a compelling service to their users
13:41:57 <hhorak> yeah, I think we can solve both problems vpavlin mentioned separately, but we cannot solve it together with one solution, which is the problem
13:42:39 <ncoghlan> I don't think it needs to be a problem though, that's where the "handover of responsibility" in the CI/CD pipeline comes in
13:42:56 <ncoghlan> providing the RHEL/RHSCL/OpenShift layer = operations problem
13:43:28 <ncoghlan> the layer above that, the actual custom applications and their immediate dependencies = development problem
13:44:09 <mmaslano> nothing against devops problems, but in Fedora WGs we should focus also on fedora use-cases
13:44:14 <mmaslano> which are missing above
13:44:52 <mmaslano> I guess images could solve both problems too. We would have image for development and for deployment
13:45:23 <ncoghlan> images are more a scalability thing than anything else
13:45:42 <vpavlin> ncoghlan: Not really..it's also an environment for developers
13:45:55 <ncoghlan> the things you need to deploy into a VM are the same things you need to build an image in the first place
13:47:06 <ncoghlan> base platform, language runtime, dependencies, custom code
13:47:36 <ncoghlan> in terms of ensuring things are relevant to Fedora itself, that's where I think the 3-phase review model comes in
13:48:42 <ncoghlan> since artefact repositories like Nexus (java) and devpi (Python) are useful not only for developers, but also as the first gate in package review
13:48:52 <juhp_> ncoghlan, can't an image also provide the base, runtime, and most of dependencies?
13:48:55 <ncoghlan> even before reaching Fedora Playground/COPR
13:49:13 <ncoghlan> juhp_: yes, but how do you define what goes into the image in the first place?
13:49:44 <ncoghlan> downloading a random image of the internet doesn't make any more sense than downloading random Windows executables or source tarballs and running them
13:50:05 <juhp_> ncoghlan, but if we had Fedora Stacks images say :)
13:51:00 <mmaslano> until now I heard people wants from distribution predefined installation where they can just start programming
13:51:01 <juhp_> well anyway I guess there is certainly room for both approaches, but I also see the point about not spreading ourselves too thinly
13:51:11 <mmaslano> Stacks images sounds like reply to that
13:51:32 <ncoghlan> juhp_: that depends - a base Fedora image + software collections + language specific package repos gets you most of the way there anyway
13:51:33 <juhp_> finally we could have some Env'n'Stacks products ;o)
13:51:55 <vpavlin> I probably still don't understand how does conda itself fit into this - we have powerful packaging tool called RPM, right?:)
13:52:08 <juhp_> ncoghlan, sure :)
13:52:19 <ncoghlan> vpavlin: I don't know how conda got onto the Envs&Stacks agenda :)
13:52:29 <ncoghlan> vpavlin: I think it's interesting for user level environments
13:52:29 <mmaslano> you've mentioned it last time
13:52:42 <ncoghlan> ah, OK
13:53:13 <vpavlin> ncoghlan: Sure, but user level environments can be also solved by Docker - which is THE technology right now:)
13:53:23 <ncoghlan> vpavlin: not really
13:53:38 <ncoghlan> sorry, scratch that
13:53:56 <ncoghlan> vpavlin: yes, I suggested to the conda folks they look at using it to build Docker images
13:54:03 <ncoghlan> but Docker isn't magic
13:54:13 <ncoghlan> it's just a packaging format with some neat properties
13:54:24 <vpavlin> Definitely
13:55:31 <ncoghlan> so thinking about it (and this is in real tangent territory), a full IPython Notebook/Scientific Python stack Docker image could be interesting
13:56:14 <ncoghlan> anyway, back to the Fedora Stacks images idea
13:56:27 <ncoghlan> I'm starting to come around to the notion
13:57:58 <ncoghlan> so you might have, for example, a Fedora Cloud + Apache 2.4 + Python 3.3 stack as a Docker image
13:58:29 <mmaslano> #topic Fedora Stacks images
13:58:46 <mmaslano> #url http://blog.docker.com/2014/09/docker-hub-official-repos-announcing-language-stacks/
13:58:49 <vpavlin> I am not sure what you mean by Fedora Cloud
13:59:02 <ncoghlan> the Cloud product in F21
13:59:18 <vpavlin> That's not Docker image
13:59:26 <ncoghlan> #url https://fedoraproject.org/wiki/Cloud
14:00:01 <vpavlin> That's cloud image - bootable Fedora optimized to run in cloud environments, I'd say:)
14:00:22 <ncoghlan> Also Docker images: https://fedoraproject.org/wiki/Cloud/Cloud_RFC_Docker_Trusted_Images_Rebuild_Policy
14:00:43 <juhp_> anyway let's not get too caught up in details :)
14:00:45 <ncoghlan> you need a base runtime for the container
14:01:23 <ncoghlan> lots of the ones on Docker hub either use a full distro base (wasteful) or an unsupported derivative of a normal distro
14:01:45 <ncoghlan> Fedora Cloud is specifically built to be such a minimal base runtime :)
14:02:00 * juhp_ got distracted by hylang when starting to read the blog post ;)
14:02:18 <ncoghlan> <3 hylang
14:02:39 <vpavlin> Fedora Cloud WG no longer maintains Docker base image - it's assigned to Base WG now
14:02:54 <ncoghlan> ah, OK, I'd missed that
14:03:02 <vpavlin> But that really doesn't matter
14:03:30 <ncoghlan> s/Fedora Cloud/Fedora base image/ above :)
14:03:34 <vpavlin> Sure
14:04:33 <vpavlin> There already are some Fedora images https://github.com/fedora-cloud/Fedora-Dockerfiles
14:05:29 <juhp_> Just curious, anyone know what the base image is for the docker announced Language Stacks?
14:05:30 <vpavlin> But I'd like to see maintainers of stacks to go through them and check if they make sense for how the language stacks are used
14:05:40 <vpavlin> debian wheezy
14:05:44 <juhp_> ok
14:05:48 <juhp_> thanks
14:07:21 <mmaslano> vpavlin: I looked at the dockerfile and there was only interpreter. What I missed?
14:07:44 * juhp_ is still a bit unsure how fine-grained stacks can be with docker though?
14:07:57 <vpavlin> mmaslano: Which dockerfile?
14:08:38 <mmaslano> ah you mean these, okay https://github.com/fedora-cloud/Fedora-Dockerfiles
14:08:46 <vpavlin> mmaslano: Yes
14:08:49 <vpavlin> Sorry:)
14:08:59 <juhp_> oh
14:09:19 <mmaslano> vpavlin: for the start I would pick one framework per language and say we will support this
14:09:35 <mmaslano> I don't think more is doable
14:10:12 <ncoghlan> mmaslano: so, for example, Django/Python/Apache, Rails/Ruby/Apache ?
14:10:24 * vpavlin nods
14:10:26 <mmaslano> that's old isn't it? :)
14:10:34 <mmaslano> nginx and something
14:11:19 <ncoghlan> mod_wsgi is still one of the best Python web servers out there
14:11:44 <ncoghlan> just cursed by the bane of Apache's poor documentation and lack of usability :P
14:12:09 <mmaslano> I wouldn't say that outloud, but there are other prefered non Apache services
14:12:19 <ncoghlan> so folks dismiss Apache as a big ball of mud, and proceed to go reinvent all the wheels :P
14:12:29 <ncoghlan> but yeah, nginx has the edge for some use cases
14:12:50 <ncoghlan> it can't host Python applications directly the way Apache can, though
14:13:12 <juhp_> probably I should contribute my dockerfile to Fedora-Dockerfiles
14:13:18 <ncoghlan> you need to pair it up with something like uWSGI or gunicorn
14:14:05 <ncoghlan> anyway, perhaps the discussion of an initial set of images can go to the mailing list?
14:14:19 <ncoghlan> (getting rather late/early here...)
14:14:31 <mmaslano> probably, to think more about the content
14:17:13 <hhorak> btw. I like the Docker's idea with customized stacks using tags (php-5.6-cli, php-5.6-apache) -- seems handy for users
14:17:20 <hhorak> but yeah, we can continue on ML..
14:18:21 <ncoghlan> main thing from my perspective is that we think about sustainability - security bugs = image respins, so automation will be key.
14:18:59 * vpavlin agrees
14:19:37 <juhp_> true, layered images need to be respun right, if the base image is?
14:19:43 <vpavlin> yes
14:19:45 <ncoghlan> yeah
14:19:55 <juhp_> thought so
14:20:03 <ncoghlan> although that reminds me, I think rel-eng have been looking into a bunch of this for Koji as well
14:20:30 <vpavlin> Yes they are, but it's still long way to go
14:21:00 <mmaslano> vpavlin: could you give us some action item? should we review content of fedora-dockerfiles?
14:21:18 <mmaslano> or is it okay to say what we would put there for our preferred language?
14:22:30 <ncoghlan> mmaslano: feel free to give me an action to follow up with stano about mapping CVEs to Docker images that need rebuilding
14:22:35 <vpavlin> mmaslano: I think to take a look what is there and fix it if that doesn't make sense or add whatever we come up with on ML
14:22:52 <mmaslano> #action  follow up with stano about mapping CVEs to Docker images that need rebuilding
14:23:03 <mmaslano> ncoghlan: good point, he still didn't solve it? :)
14:23:12 <mmaslano> vpavlin: okay
14:23:42 <ncoghlan> mmaslano: I think they're making good progress, but I *don't* think we've considered the fact Fedora might need a similar capability
14:23:44 <mmaslano> #action propose reasonable dockerfile for our favourite languages
14:25:06 <juhp_> ok
14:28:13 <mmaslano> #topic chairman
14:28:20 <mmaslano> who will take it?
14:29:40 <hhorak> if nobody else volunteers I can do it
14:32:11 <ncoghlan> and at that point, I'm going to head off folks (just ticked past 12:30 am here). Catch you on list and next week :)
14:32:22 <juhp_> nite!
14:32:28 <mmaslano> night
14:32:32 <mmaslano> #topic Open Floor
14:34:16 <juhp_> mmaslano, last week hhorak said this was your last time to chair the meeting?  Maybe I missed some news? :-|
14:34:35 <mmaslano> juhp_: I gave up all tasks, going to maternity leave son
14:34:42 <mmaslano> soon
14:34:54 <juhp_> ah right!
14:34:58 <mmaslano> now I need to get rid off some Changes which I proposed for F-21 and didn't happened
14:35:23 <mmaslano> I'll offer them on mailing list, but I might be lucky and even find a developer to work on them ;-)
14:35:31 <mmaslano> in that case I would prefer developer with time to do them
14:36:02 <juhp_> all the best with your leave :-)
14:36:05 <mmaslano> thanks
14:36:11 <mmaslano> sounds like going home
14:36:14 <mmaslano> #endmeeting