16:01:35 <mmaslano> #startmeeting env and stacks (2013-12-17)
16:01:35 <zodbot> Meeting started Tue Dec 17 16:01:35 2013 UTC.  The chair is mmaslano. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:01:45 <mmaslano> #meetingname env and stacks
16:01:45 <zodbot> The meeting name has been set to 'env_and_stacks'
16:01:59 <mmaslano> #chair abadger1999 pkovar tjanez samkottler bkabrda handsome_pirate hhorak juhp
16:01:59 <zodbot> Current chairs: abadger1999 bkabrda handsome_pirate hhorak juhp mmaslano pkovar samkottler tjanez
16:02:30 * samkottler waves & will have to leave in about 15 minutes for a doctor's appointment
16:03:01 * pkovar1 is here
16:03:05 <mmaslano> who else is here
16:03:30 <mattf> o/
16:04:05 <willb> I'm here
16:05:28 * tjanez is being late
16:06:17 <drieden> I'm here too
16:06:46 <mmaslano> #topic init process
16:06:55 <mmaslano> great, let's start
16:07:04 <mmaslano> we don't have much time for something like PRD
16:07:17 <hhorak> Hi, sorry for being late..
16:07:38 <mmaslano> so I created something inspired by Cloud PRD https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD
16:07:53 * pingou around (and late, sorry)
16:08:26 <mmaslano> the real tasks are still little fuzzy, but I plan to work on them more if you agree with it
16:08:53 <mmaslano> mattf: great you are here. You can tell us what you need
16:09:07 <mmaslano> mattf is from Big data SIG
16:09:17 <mattf> willb, would you start?
16:10:25 <mattf> mmaslano, willb has been feeling most of the pain dealing w/ non-primary (c/c++,python,perl) language stacks in fedora
16:10:37 <willb> I think a lot of our headaches in Big Data relate to language packaging ecosystems vs Fedora
16:10:46 <willb> in particular, I have been working on Scala lately
16:12:29 <willb> in short, the main issues are widespread Ivy for dependency management (mizdebsk has been working on this) and very brittle versioning (there is the expectation that you'll be able to have particular versions of the runtime, compiler, and libraries to run any nontrivial application)
16:14:22 <willb> like a lot of language stacks (e.g. rubygems, maven, eggs), there is the assumption that any developer would just be using the language stack instead of downstream dependency management, so I've had to work around a lot of those sorts of issues
16:15:15 <mattf> all those places where we have to work around things the language community does are hurdles to eventually automating / reducing maintenance package work
16:16:28 <mmaslano> we were already speaking about automation of packaging (not ready) and hosting such packages in separated repositories
16:16:29 <mattf> we also ran into this around node.js, which has its package/dep management tool available in fedora, but very little of the language ecosystem / base libraries available
16:17:04 <mmaslano> what would be good solution for you?
16:17:18 <mmaslano> SCL, or latest versions from upstream?
16:17:22 <mattf> for us the application is where we want to focus our time, but we find ourselves in the toolchain management and not getting to the application
16:18:07 <tjanez> willb, mattf, I'm glad to hear your perspective w.r.t. Stacks. Problems like you describe are very aligned with the scope of our WG
16:18:30 <mattf> the one dep i've been harping on lately is jetty. the latest from upstream doesn't support the java version that our app upstream wants to use. so having the latest in fedora is a problem.
16:18:59 <willb> jetty seems to touch a staggering number of projects in our SIG, too
16:19:54 <mattf> mmaslano, something that would have helped (and probably will in the future) is a way to provide language ecosystems in fedora in a way that aligns w/ how the language itself is used
16:20:27 <mattf> all the modern languages have their own dep & ver management tooling
16:20:37 <mattf> and most of it doesn't align well w/ fedora
16:21:34 <mattf> mmaslano, as for SCL, they could very likely help. however, they represent an add-on to fedora
16:22:04 <mattf> almost like rpm fusion, or all the extra repos from early fedora version
16:22:09 <sochotni> so what was that about jetty?
16:22:15 <willb> in the scala case, Fedora would allow us to provide scala 2.9 and 2.10 packages in the same release, but to really work with scala, we'd need both of those resolvable via Ivy, and not just "scalac29" and "scalac210" binaries
16:22:30 <mmaslano> mattf: yes, on last fesco meeting was decided, that additional repositories are acceptable, but content has to be reviewed anyway
16:23:53 <mattf> sochotni, rsquared is the best person to talk about the jetty situation. the exec summary is the version of jetty in fedora does not work w/ java 6 and the hadoop upstream isn't ready to abandon java 6. so we end up maintaining a patch to hadoop for jetty 9 that will live purely in fedora for quite a while to come
16:25:02 <mattf> sochotni, i believe there was some chatter about compat packages for jetty libs too. that might be a workaround for f21/22 or so, but ultimately there's a mismatch in expectations between fedora and languages like java / scala / js that should be resolved
16:25:26 <mattf> (i should note, there's been a lot of good work on figuring it out for java)
16:25:29 <sochotni> mattf: we don't jave JDK 6 in Fedora 18+
16:26:08 <mattf> sochotni, i believe the compat lib discussion hinged on fedora policy and the compat library working w/ java 7
16:26:26 <sochotni> mattf: compat lib of what package?
16:26:27 <mattf> of course that workaround might stop working if the only jetty code doesn't work w/ say java 8 or java 9
16:26:58 <mattf> sochotni, we should chat w/ rsquared on the specifics
16:27:03 <tjanez> So, the question is, is WG willing to work towards integrating upstream language packaging systems
16:27:31 <tjanez> This could complement the existing RPM-based packaging
16:28:17 <mmaslano> tjanez: good point
16:28:30 <tjanez> It would certainly benefit the needs of the Big Data SIG
16:28:43 <sochotni> this is probably too specific task
16:28:44 <tjanez> The Big Data SIG use case could also be put in the PRD
16:28:55 <sochotni> might be a good test case but that's probably it
16:29:03 <mattf> i'd argue that steps in that direction will benefit the needs of groups that want to bring applications written in "new" languages to fedora
16:29:52 <mattf> (where "new" == actually really old languages)
16:29:55 <tjanez> mmaslano, do you feel upstream packaging is in our scope or not?
16:30:13 <sochotni> tjanez: that's not something you will solve in a WG
16:30:14 <mmaslano> tjanez: I thought we agreed it could be done automatically
16:30:22 <sochotni> you need someone who will *actually* work on the code
16:30:24 <mmaslano> tjanez: at least for some languages
16:30:41 <willb> again with my specific case:  right now, if you want to use Scala for real work on Fedora, your option is basically to install a bunch of binaries maintained by upstreams.  It would be great if we were able to better support "new" language ecosystems wholly in Fedora.
16:30:43 <sochotni> all you can agree is "yes we want to do that"
16:31:08 <mmaslano> #info Big Data specific case: right now, if you want to use Scala for real work on Fedora, your option is basically to install a bunch of binaries maintained by upstreams.  It would be great if we were able to better support "new" language ecosystems wholly in Fedora.
16:31:26 <mattf> nice
16:31:31 <tjanez> sochotni: well, I hope our WG will start working on things we put in the PRD
16:31:31 <tjanez> and by working I mean coding
16:31:51 <mmaslano> still someone has to write automation for "new" language and maintain it
16:32:07 <mmaslano> also I don't think packages without review can get into Fedora
16:32:12 <mmaslano> legal issues..
16:32:47 <hhorak> mmaslano: reviews could be done the first time and every-time something big changes in the package, but simple rebase could be done outomatically
16:33:07 <tjanez> sochotni: First I would like to see if we at least agree that complementing RPM packaging with upstream packaging is a viable way for the future
16:33:13 <mattf> that's an interesting point. there's always going to be a human component to packaging. however, we could probably write down that small list and try to automate the rest.
16:33:25 <mmaslano> tjanez: I agree
16:33:44 <tjanez> mmaslano: well, we would start with one language
16:33:52 <hhorak> so the packaging tool should be able to say "this rebase is suspiciously serious, we need a manual review"
16:34:13 <tjanez> mmaslano: but the common thing with all languages would be the concept of co-existance with RPM packages
16:34:28 <mmaslano> now I'm confused
16:34:39 <mmaslano> mattf: do you want to package upstream into rpm or not?
16:34:49 <sochotni> what I *could* imagine is some wrapper around pip/rubygems or alternative packaging systems
16:35:11 <hhorak> mmaslano: I think the tool should produce rpms in the end
16:35:18 <tjanez> mmaslano: +1 good question
16:35:25 <sochotni> but policy of updates...my head starts to hurt just thinking about it
16:35:37 <mattf> mmaslano, i want to package upstream into something that fedora will happily include directly in its ecosystem
16:35:59 <mmaslano> mattf: I'd like to see something like that too
16:36:20 <mattf> in other words, the form (rpm, deb, npm, other) isn't as important to me
16:36:41 <hhorak> sochotni: even now many people do big updates that introduce new issues according to guidelines and only if someone notices it it gets resolved..
16:36:48 <mmaslano> #proposal add automatic packaging of upstreams into WG goals
16:36:59 <mattf> hhorak, too true!
16:37:47 <tjanez> mattf, well from the consumer (aka big data user) point-of-view, the form is not important, from the distribution point-of-view it is
16:38:14 <mattf> hhorak, there's little infrastructure that i see for CI around fedora. you're a lib that has 12+ users, well when you update you should maybe rebuild your deps to find breaks.
16:38:24 <mattf> hhorak, fedora has a very reactive model atm
16:38:26 <hhorak> what I can't imagine is including a really new package without a review.. I mean at least initial review would have to be done manually, but with something like fedora-review it shouldn't be too hard
16:38:36 <mattf> tjanez, that's fair
16:40:03 <mmaslano> #proposal  add automatic packaging of upstreams into WG goals. Initial review of packages will be neded.
16:40:03 <tjanez> mmaslano, I'm ok with that, I would just like to clarify a bit more what the final form of that packaging would look like
16:40:03 <tjanez> would it be Fedora-approved subset of upstream packaging repositories
16:40:03 <tjanez> and the users would use tools like pip
16:40:04 <tjanez> or would it be an rpm repository automatically generated with *2rpm tools
16:40:05 <mmaslano> any votes?
16:40:07 <tjanez> and the users use dnf (distro tools)
16:40:18 <hhorak> mmaslano: +1
16:40:29 <mmaslano> tjanez: I would stay with rpm, we alredy know how it works
16:40:33 <mmaslano> (most of the time)
16:41:01 <willb> tjanez, to some extent, the existing processes are important for consumers as well as for the distro.  For any reasonable list of advantages we can list for using Fedora, most of them come from being well-integrated with the Fedora ecosystem and from having packages that follow Fedora processes.
16:42:25 <hhorak> tjanez: I don't think we should provide any bits outside rpms, it would require to solve all the issues that rpm solves today again and again for every language stack
16:42:36 <willb> hhorak, +1
16:43:07 <tjanez> Ok, you convinced me
16:43:22 <tjanez> I just wanted to put it out so we discuss it
16:43:23 <hhorak> tjanez: what I can imagine is creating something that would work like pip, but would actually work with rpms in the background.. (not familiar to pip, so maybe it is not easy)
16:44:01 <willb> IMHO RPM/rubygems integration and xmvn are good examples of language-specific packaging systems working well in Fedora
16:44:15 <mmaslano> hhorak: why? we have dnf/yum already. it's not obvious to me, why create another tool for installation
16:44:18 <tjanez> hhorak: +1
16:44:49 <tjanez> mmaslano: it would be just a conveniece wrapper for dnf/yum
16:45:10 * pingou doesn't see the point of wrapping wrapper
16:45:18 <mmaslano> tjanez: I wouldn't say just in case of dnf
16:46:37 <mmaslano> it seems to me Fedora users need some way how automatically generate packages from upstream. Implementation details can be solved later
16:47:02 <tjanez> well, the use case I can see is to attract developers with different backgrounds
16:47:03 <tjanez> they maybe very familiar with pip/rubygems/...
16:47:19 <tjanez> ok, fair enough
16:48:33 <mmaslano> tjanez: yeah, but which tool would you pick? pip/rubygems? you would probably have to patch all these tools
16:49:12 <tjanez> mmaslano, I would change your proposal to something like: automatic packaging of upstream language repositories into rpm-based repositories. Initial review of distributed packages would still be needed
16:49:17 <hhorak> mmaslano: I understand that as users just don't want to use dnf, because the style of work is way different from their language-specific tools.. that's why they would like to use what they use in other distributions (language specific ways, pip for example).. but command name doesn't matter, it's more about style of work, so if dnf can be as easy to use as these language-specific tools, then we can stay with pure dnf.. Maybe I didn't ca
16:50:14 <mmaslano> hhorak: we are missing end of sentence, not all irc clients can display so long messages ;-)
16:50:18 <mmaslano> tjanez: ok +1
16:50:24 <tjanez> mmaslano: yes, you would have to patch all of them. Or rather, replace them with commands that have the same CLI
16:51:02 <tjanez> mmaslano: but yea, this is an implementation detail that can be added or not later
16:51:08 <mmaslano> tjanez: at this moment it seems to me as far away future, because we don't even have the automatic packaging :)
16:51:10 <hhorak> mmaslano: sorry ;) my point was to try to understand why people don't like dnf/rpm/yum and prefer language specific tools
16:51:32 <Subfusc> the advantage of distribution tools for language specific purposes is not that they can install $random_language_pack but that they work in isolated environments (like with python-virtualenv)
16:51:50 <Subfusc> atleast that is how i see it
16:52:00 <pingou> mmaslano: we do have a number of *spec tools :)
16:52:01 <mmaslano> Subfusc: I guess in near future everything will be in container. So that's solved
16:52:40 <tjanez> #proposal Add to tasks/goals of our WG: Automatic packaging of upstream language repositories into rpm-based repositories. Initial review of distributed packages would still be needed
16:53:46 <tjanez> My proposal would actually just clarify the "The automatic packaging" task in mmaslano's PRD draft
16:53:49 <mmaslano> tjanez: still +1
16:54:46 <hhorak> tjanez: +1
16:55:37 <tjanez> It would be great if we also add a short summary of what mattf and willb said earlier
16:55:39 <hhorak> Subfusc: thanks for that point. After quick reading it seems like virtualenv is kind of a python version of software collections, or do I understood it wrong?
16:56:17 <tjanez> Maybe as a "problem case" the WG is solving with that particular task
16:56:19 <mmaslano> tjanez: do you want to sum it up?
16:56:40 <tjanez> Yes, I can and then put it in the wiki
16:57:11 <willb> tjanez, if you want some more detail, I wrote up some of the Scala-specific issues in August (http://chapeau.freevariable.com/2013/08/making-fedora-a-better-place-for-scala.html) and have been posting updates to the wiki here:  https://fedoraproject.org/wiki/SIGs/bigdata/packaging/Scala
16:57:23 <tjanez> Maybe better if I do it after the meeting
16:57:42 <tjanez> willb, thanks, I will look at it
16:57:49 <willb> thanks
16:58:29 <mmaslano> #action tjanez will sum up Big Data SIG needs
16:58:40 <Subfusc> hhorak: I'm not that familiar with software collections, but it works in complete isolation with every virtualenv having its own libraries and it lives on $HOME so that non-admins can administer it. Software collections does not seem to completely envelop this concept
16:59:20 <Subfusc> the install in home folder is the primary feature that makes virtualenv easy to use for developers
16:59:33 <tjanez> mmaslano: should "problem cases" be a new section or just a sub-bullet under automated packaging task
16:59:41 * mmaslano will need to leave in few minutes
16:59:56 <mmaslano> tjanez: the whole task section is open for editing
17:00:04 <hhorak> Subfusc: right, user-specific installation is missing..
17:00:06 <mmaslano> I guess the rest is okay
17:00:22 <mmaslano> tjanez: I copied topic which were discussed in last few weeks
17:00:36 <tjanez> Ok, are there any takers for writing up other tasks in the PRD?
17:00:59 <hhorak> if the task list gets longer in the future, tasks could be separated into two groups as discussed two weaks back, to primary and secondary (less priority) tasks
17:01:18 <Subfusc> hhorak: its also easy to install and edit libraries without affecting anything else (which a system wide installation could not emulate)
17:01:20 <tjanez> We have to speed things up a bit
17:01:38 <tjanez> Jan 14th is coming very quickly
17:01:40 <mmaslano> tjanez: I'll try to specify some. But it should be short definition up to 6 sentences
17:01:55 <mmaslano> tjanez: yeah, that's the reason why I've started with prd without discussion
17:01:58 <tjanez> When will we have our next meeting?
17:02:06 <hhorak> tjanez: you mean what we discussed so far or some new tasks?
17:02:24 <tjanez> hhorak: the things from our discussions
17:02:30 <mmaslano> tjanez: imho January 7
17:02:54 <mmaslano> not much time, let's add as many tasks as possible now
17:03:01 <mmaslano> and we can review it by email
17:03:04 <tjanez> I think we have to look through the IRC logs and extract the things already said
17:03:41 <mmaslano> I tried to do it on my blog, but only from last 3 meetings
17:04:11 <tjanez> Ok, I'll look at your blog post
17:04:43 <mmaslano> for the record http://mmaslano.livejournal.com/
17:04:51 <tjanez> I'll certainly help, since I'll have more free time around the holidays
17:05:11 <mmaslano> #action everyone will look at https://fedoraproject.org/wiki/User:Mmaslano/Draft:Env_and_Stack_PRD and think/work on tasks.
17:05:32 <mmaslano> #info deadline for PRD is January 17th
17:05:44 <hhorak> mmaslano: ^date -d '2014-01-07' +"%V" says 02 so it should be at 13:00UTC
17:05:45 <mmaslano> or 14th?
17:05:47 <tjanez> I hope others will join to create a better PrD
17:06:02 <mmaslano> tjanez: hopefully
17:07:02 <tjanez> mmaslano: well it's actually Jan 13 (see: https://fedorahosted.org/fesco/ticket/1196)
17:07:10 <mmaslano> #undo
17:07:10 <zodbot> Removing item from minutes: <MeetBot.items.Info object at 0x11d6e7d0>
17:07:16 <mmaslano> #info deadline for PRD is January 14th
17:07:43 <mmaslano> #info next meeting will be on 13:00UTC January 7th
17:07:50 <mmaslano> tjanez: thanks
17:08:01 <mmaslano> if you don't have any other topics, I would like to close the meeting
17:08:09 <mattf> mmaslano, thanks for having us
17:08:24 <tjanez> mmaslano, you still put in the wrong date?
17:08:44 <tjanez> or has it been changed from Jan 13 to Jan 14?
17:09:15 <willb> thanks all
17:09:42 <mmaslano> aaa
17:09:49 <mmaslano> #undo
17:09:49 <zodbot> Removing item from minutes: <MeetBot.items.Info object at 0xf892910>
17:09:55 <hhorak> thank you all and enjoy Chrismas!
17:10:02 <mmaslano> #undo
17:10:02 <zodbot> Removing item from minutes: <MeetBot.items.Info object at 0xf892dd0>
17:10:10 <mmaslano> #info deadline for PRD is January 13th
17:10:17 <mmaslano> #info next meeting will be on 13:00UTC January 7th
17:10:20 <tjanez> mmaslano: sorry for the trouble
17:10:35 <mmaslano> I hope meeting log will be nice :)
17:10:42 <mmaslano> let's go home
17:10:48 <mattf> ciao
17:10:49 <tjanez> Thank you for a good meeting
17:10:52 <mmaslano> #endmeeting