13:08:40 #startmeeting Env and Stacks (2015-02-12) 13:08:40 Meeting started Thu Feb 12 13:08:40 2015 UTC. The chair is hhorak. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:08:40 Useful Commands: #action #agreed #halp #info #idea #link #topic. 13:08:52 #meetingname env-and-stacks 13:08:52 The meeting name has been set to 'env-and-stacks' 13:08:56 #chair bkabrda hhorak juhp ncoghlan vpavlin sicampbell walters ttomecek phracek 13:08:56 Current chairs: bkabrda hhorak juhp ncoghlan phracek sicampbell ttomecek vpavlin walters 13:10:10 #topic Env&Stacks Governance Charter Review as per https://fedoraproject.org/wiki/Env_and_Stacks/Governance_Charter#Changing_These_Rules 13:10:15 #link https://fedoraproject.org/wiki/Env_and_Stacks/Governance_Charter#Changing_These_Rules 13:10:36 I guess this is not much to discuss for today 13:11:23 I just wanted to ask you to read at it and if there are some things we need to change, we can add some specific issues to the agenda for some of the next meetings. 13:11:28 seems fine? 13:12:01 * vpavlin nods 13:12:15 #agreed 13:12:29 #action all to read Governance charter and if there are any issues that should be revisited, let's bring them on ML or add them to agenda for some of the next weeks 13:12:41 so just an update about devpi - I'm not sure if I've already shared this. upstream wanted us to create a formal proposal for the feature we need. I passed the work on to Matej Stuchlik and we now have the proposal for upstream https://groups.google.com/forum/#!topic/devpi-dev/py-B9kwaK5Y and are waiting for comments 13:12:55 #topic Follow-ups -- what has changed since last meeting 13:13:32 #info update about devpi - upstream wanted us to create a formal proposal for the feature we need. I passed the work on to Matej Stuchlik and we now have the proposal for upstream https://groups.google.com/forum/#!topic/devpi-dev/py-B9kwaK5Y and are waiting for comments 13:14:16 bkabrda: so is this the main blocker and we can see some PoC working with this feature implemented? or are there any other blockers? 13:15:00 hhorak: this is the only blocker (that I know of). once it's done, we can go on and deploy devpi testing instance 13:15:28 upstream seems to be very busy, but we'll keep poking them and hopefully they'll be able to approve the proposal soon 13:15:35 bkabrda: only for my understanding devpi is a UI for downloaded and installing PyPi packages to Fedora? Without RPM Packaging? 13:15:50 is there a link for more information on this devpi proposal? 13:16:06 it's the pilot for https://fedoraproject.org/wiki/Env_and_Stacks/Projects/LanguageSpecificRepositories 13:16:17 #info the devpi feature seems to be the only blocker. once it's done, we can go on and deploy devpi testing instance 13:16:22 or is the devpi connected to DevAssistant 13:16:24 if we decide it's a good idea, we'd switch to the Pulp based repos 13:16:25 #link https://fedoraproject.org/wiki/Env_and_Stacks/Projects/LanguageSpecificRepositories 13:16:40 thanks 13:16:47 #link http://pulp-python.readthedocs.org/en/latest/ 13:16:51 phracek: well it's not a UI, it's a transparent PyPI caching server with some nice features :) 13:17:15 ah right, thanks. Excellent wiki pages I have to say 13:17:31 but the Pulp plugins are really new, so we didn't want to be debugging them will working out if we liked the overall workflow 13:17:57 walters: if only the other projects have the same quality pages :) 13:18:16 walters: I still need to explicitly state what we want to do (like what to upload, what to prebuild, etc etc) 13:18:23 * ncoghlan has probably written more PEPs than is healthy 13:18:55 i think my main question is - the package would get into fedora, then its upstream would be mirrored? 13:19:14 I think that's the way to go for the time being 13:19:23 makes sense 13:19:29 longer term, we'd like to explore the idea of "staged inclusion" 13:19:37 where the upstream packaging gets mirrored first 13:19:52 then we get a non-policy compliant RPM into COPR 13:20:03 (potentially non-policy compliant) 13:20:22 and then a package can graduate to the main repos once it's packaging is actually right 13:20:55 that's a lot more working though, since it means proposing changes to the package review process 13:21:10 walters: ncoghlan: I'll try to work with Matej on these and we'll come up with a proposal that tries to answer all of this (including workflows around that etc) 13:21:17 would nice to have a tool, which would autopackage stuff from that devpi instance to rpm 13:21:37 then we'll at least have something to discuss :) 13:21:46 what I've always wanted is *bidirectional* sync between upstream and packages 13:21:52 ttomecek: that becomes feasible once we sort out PEP 426 upstream 13:22:14 at the moment, it isn't really feasible because the upstream metadata is missing stuff Fedora needs 13:22:19 if you autopackage and then git commit the spec file, then you have {Build,}Requires drift over time, etc 13:22:23 and there's no extension system to stash it 13:22:36 has anyone looked at this from the Maven perspective? 13:22:38 #info the plan is the package would get into fedora, then its upstream would be mirrored, at least for the time being 13:22:38 #info longer term, the upstream packaging may be mirrored first and a potentially non-policy compliant RPM got into COPR 13:23:03 walters: I've been nudging some of the Java folks here in BNE about it, but no takers so far 13:23:31 walters: theoretically, you should be able to regenerate the specfile automatically on every rebase, assuming the automatic generation works in the first place, right? 13:23:54 that's one spectacular assumption at this point, though :) 13:24:02 bkabrda, but that means if another packager comes by and adds a missing Requires (e.g. on some external /usr/bin/foo), that gets lost 13:24:35 ncoghlan, what a long pep 13:25:01 ttomecek: and that's *after* I moved big chunks of it out to PEP 440 and PEP 459 :) 13:25:10 walters: true, but I'm sure we could solve these automatically somehow 13:25:48 bkabrda, walters: for Python, this is actually one of my intended use cases for the metadata extensions 13:26:16 #info PEP 426 should allow autopackage stuff from devpi instance to rpm 13:26:16 #link https://www.python.org/dev/peps/pep-0426/ 13:27:40 my hope is that if we can get at least *one* language ecosystem playing nice with distro packaging, we might be able to encourage others to follow suit 13:27:58 ncoghlan: I have to admit that I haven't read the whole pep yet, but I'm still assuming that it won't be possible to just say "generate specfile from foo.tar.gz" - there'll still be a need for some tool that'll unpack the tarball, generate dist_info and then parse it (and actually dist info can theoretically be different for different runtimes, if I'm not mistaken) 13:28:12 I have at time been accused of being overly optimistic :) 13:28:52 bkabrda: aim will be to have a stable JSON description of the source tarball 13:29:17 bkabrda: hence the environment marker support in the dependency specifiers 13:30:03 ncoghlan: ah, ok. not wanting to dive into details, but where will this description live? 13:30:15 either way, pyp2rpm is looking forward to this ;) 13:30:21 bkabrda: in the sdist, so it will still need to be unpacked 13:30:38 although ideally we'd have it available direct from PyPI as well 13:30:53 ncoghlan: ok, thanks. I'll read the whole pep and then continue having stupid questions :) 13:30:59 Donald's currently working on getting the PyPI rewrite into shape 13:31:18 what about with dependencies to another PyPi packages. Is there a plan how to build a dependencies? 13:31:31 https://github.com/pypa/interoperability-peps if you spot any major issues 13:31:47 related to bridging upstream and packaging is https://people.gnome.org/~walters/docs/build-api.txt 13:32:21 although I also have a bunch of open issues at https://bitbucket.org/pypa/pypi-metadata-formats/issues?status=new&status=open 13:33:13 walters: one that came up at LCA2015 last month was Meson 13:33:35 walters: apparently that has a JSON build description format for IDE integration 13:34:16 #link https://people.gnome.org/~walters/docs/build-api.txt 13:34:33 #info it would be nice to have a tool independent "How do we build this?" description format 13:34:41 #link https://github.com/jpakkane/meson/wiki/IDE-integration 13:35:09 bkabrda: pyp2rpm is going to generate devpi stuff to RPM? What about devpi2rpm? 13:35:10 we have a related problem in Python upstream, where we'd like to make distutils/setuptools optional 13:35:15 declarative JSON is good but there are some important things like the "meta-build" system (rpm/dpkg/etc) being able to specify --enable-foo --disable-bar, set CFLAGS etc 13:35:16 ncoghlan: thanks for helping with #infoing :) 13:35:55 walters: yeah, that's why we lobbed it into the "too hard" basket upstream for now, and are sticking with setup.py 13:36:11 phracek: devpi will have the same source package format as PyPI, it's just a mirror 13:36:13 bkabrda: But it is not related to this discussion. THe name is not important 13:36:23 bkabrda: ok 13:36:32 walters: but the core design of distutils was laid down 17 years ago, and it has not aged well :) 13:36:40 the "pyp" in pyp2rpm is for "PYthon Package" :) 13:36:47 make sense to me. thanks for explanation. 13:39:44 either I got offline or we're done with this topic :) 13:40:02 ... for now :) 13:40:13 ncoghlan: right 13:40:37 #topic DevConf legacy -- something interesting from DevConf we should mention here? 13:41:27 Btw. it was very nice to meet you in Brno! 13:42:32 Nice to meet you guys too! 13:42:44 * vpavlin nods 13:42:46 * langdon now lurking here 13:43:17 I didn't make it, but hopefully at least some folks got to meet dcallagh from the Beaker team 13:43:38 I know he delivered some Python propaganda to bkabrda for me :) 13:44:21 ncoghlan: actually he delivered it to a guy who delivered it to me :) and thanks 13:45:36 this isn't DevConf related, but the Beaker connection reminded me: https://bugzilla.redhat.com/show_bug.cgi?id=1183913 13:46:18 we're aiming to add a standard task to check that a batch of SRPMs rebuild properly 13:46:42 the intended use case is to have a way to check that a mass rebuild is going to work before actually doing it 13:47:04 and use Beaker's failure reporting to highlight the SRPMs with problems 13:47:20 aren't there like 5 other implementations of that? 13:47:41 from Koschei to what matt domsch used to do to ... 13:47:48 f.e. Koschei 13:48:06 ah..my irc behaves weirdly.. 13:48:11 walters: probably, but very few of them will let us automatically do it across every arch the distro supports 13:48:57 more importantly, that's only the initial iteration 13:49:18 the next step will be to turn it into a regression test for the build toolchain itself 13:49:38 where we can swap that out (or even just configure it differently) and see what breaks 13:50:40 walters: although if you wanted to send links my way, or add them to the ticket, that would be useful background info 13:53:34 with gnome-continuous when someone introduces a build break we generally fix it to the previous working version; we're constantly *trying* new things but quickly reverting if they break, for a fast iteration cycle. See https://git.gnome.org/browse/gnome-continuous/commit/?id=42187d2e8d5060bbb0ba0d3c82b3e2c1bf25a8ee 13:54:03 but that's not possible with rpm/yum/koji due to the newer-is-always-better semantic 13:54:40 walters: yeah 13:55:16 I think we're getting off-topic for Env & Stacks though :) 13:55:29 (as near and dear as CI is to my heart) 13:55:38 this is CD, not CI 13:56:16 Koschei and your Beaker proposal are CI - after a build the binaries are garbage collected, all you have is the logs 13:56:45 gnome-continuous is CD - after a build, you can atomically upgrade to the result and boot it 13:58:05 walters: interesting, and OSTree based so upgrade/downgrade can be atomic 13:58:32 anyways from Devconf I just got the perspective of a *lot* going on... 14:00:01 As for SCLs in fedora -- I talked with pingou and denis.. we agreed that it makes things simple and more clear to have scls in current dist-git in special branches.. it will be easier to implement in rcm and pkgdb. 14:00:09 But first we need to have the guidelines if this way is fine, some agreement for branches naming, etc.. 14:00:16 So, that's my plan to look at for the following weeks. 14:00:26 #info it makes things simple and more clear to have scls in current dist-git in special branches.. it will be easier to implement in rcm and pkgdb. 14:00:26 #info First, we need to have the guidelines if this way is fine, some agreement for branches naming, etc.. 14:00:35 you're working on SCLs in CentOS too? is that matching Fedora? 14:01:07 walters: yes, I'm involved in anything which is connected to SCLs :) 14:01:45 walters: I think we should use the same approaches in centos as in fedora or the the other way round (providing centos gets scls sooner) 14:02:34 there is also epel in the game, where I'm not actually sure if we're fine with SCLs from centos or we need something more 14:03:02 that's what needs to be cleared 14:04:48 #topic Open Floor 14:05:22 Sorry guys, somebody goes to throw me out, which means I'll lose connection soon. Could somebody close the meeting instead of me after you guys go through Open Floor? (good time for arbitrary questions not only from new members.. basically about anything) 14:06:01 * hhorak going offline.. Thanks everybody!!! 14:07:07 anyone have anything for the open floor? 14:08:45 I guess not :) 14:08:49 i just had one question - my experience (from the rpm-ostree side) is that doing anything that involves touching the fedora releng side is quite difficult. For example with the devpi work, would the mirror set be versioned and released with Fedora X under releng? 14:09:29 walters: we'll run the pilot ourselves and we haven't really thought beyond that, yet 14:10:35 I also think the rel-eng side of things has been identified as an area needing more hands 14:11:00 hence stickster's recent post about hiring into that area 14:12:21 there is an alt.fedoraproject.org which is a staging area of sorts 14:13:02 not very formal but it might make sense to create an intermediate stage like COPR between not-fedora and official-releng 14:13:21 anyways that's all I had myself 14:14:45 closing in 10... 14:15:08 #endmeeting