13:02:08 #startmeeting Env and Stacks (2014-09-23) 13:02:08 Meeting started Tue Sep 23 13:02:08 2014 UTC. The chair is hhorak. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:02:08 Useful Commands: #action #agreed #halp #info #idea #link #topic. 13:02:14 #meetingname env-and-stacks 13:02:14 The meeting name has been set to 'env-and-stacks' 13:02:20 #chair pkovar tjanez samkottler bkabrda hhorak juhp mmaslano vpavlin sicampbell 13:02:20 Current chairs: bkabrda hhorak juhp mmaslano pkovar samkottler sicampbell tjanez vpavlin 13:02:27 #topic init process 13:02:36 Hi guys, so who do we have here today? 13:03:02 hi! 13:03:06 * mizdebsk is here to answer any questions about koschei 13:03:11 * tflink is lurking 13:03:58 mizdebsk: great! 13:04:53 tflink: how the things with Taskotron go? 13:05:31 hhorak: slowly, chasing down some small issues and annoyances as we prepare to move into production and decommision autoqa 13:07:05 * ncoghlan is here, but not fully awake (it's late and I'm not 100% well) 13:07:28 tflink: Ok, keeping fingers crossed for that move, it will be a big leap for Fedora 13:07:42 tflink, hhorak: we're looking at Taskotron for a few things internally 13:08:19 no specific plans yet, but rpmgrill and decoupling Beaker's resultsdb from the main app are ideas being kicked around 13:08:34 ncoghlan: that really makes sense 13:09:10 hhorak: talk to stano for more details :) 13:09:54 ncoghlan: I suppose that explains why rmancy was trying to set up an instance - I thought he was just exploring rpmgrill 13:10:41 yeah, esantiago built a custom app for internal use, but we're not sure that's a good long term plan 13:10:58 taskotron's architecture is far more flexible 13:11:31 we'll need to figure out how to hook it up to AMQP rather than fedmsg, but that should be doable 13:11:39 ncoghlan: Actually I talked to Stano on Thursday, we met at the kitchen also with Kamil, great to hear we're pretty much on the same page about what the things should look like. 13:11:40 hi 13:12:27 hhorak: well, maybe not *all* - there's a few folks in certain divisions that are still a bit "Jenkins for ALL the things" 13:14:08 hhorak: the Taskotron idea makes even sense though that I'm pretty sure we'll end up doing it that way 13:14:57 ncoghlan: actually I understand Jenkins usage for upstream CI, but for integration not that much. But I should learn more about Jenkins .. 13:15:26 right 13:15:49 #topic Taskotron's current status 13:15:54 (haven't changed topic, sorry) 13:15:59 hhorak: yeah, single project CI seems to be where it shines. Some folks see it as a really big hammer for hitting all the nails, though, which can make for some interesting conversations :) 13:16:41 #info Taskotron is close to production, soon replacing autoqa 13:18:52 #info ncoghlan interested in Taskotron vs. rpmgrill integration, it seems to be better and more flexible choice than the custom app around rpmgrill 13:19:38 Ok, that was a bit out of agenda, which is perfectly fine for me, can we move to Koschei? 13:20:51 #topic Recommendation for running Koschei for all Fedora packages 13:21:13 I spoke to guys working on Koschei. They would like to have our recommnedation for using of Koschei 13:21:25 after that they could try to move Koschei into fedora-infra 13:22:05 We do not have majority for official voting, since only three of us are here, but we can still discuss and vote I guess. 13:22:55 not good :( 13:23:13 (and the rest would vote in the ML) 13:24:02 Personally, I don't see any issues, so I'm definitely +1 for recommend Koschei. 13:24:41 given the "only when load is low" behaviour, +1 from me as well 13:26:29 However, not every package has to be suitable for koschei -- that's something we should cover in the recommendation as well. (e.g. long building leaf package with many dependencies..) 13:27:12 +1 for Koschei too 13:28:01 mizdebsk: what happens when for example glibc changes? I suppose not every package using glibc get's rebuild.. 13:28:23 koschei supports per-package options, for example static priority which can be set to low value to prevent too often rebuilds 13:29:10 glibc and any other pkg in @buildsys-build is treated as dependency of every package, so glibc update increases priority of all packages 13:29:43 packages are rebuilt based on priority, not every dep change causes rebuild 13:29:51 aha 13:30:08 and direct dep changes affect priorities more than transitive 13:31:54 mizdebsk: is this magic transparent somewhere? I mean for example is it possible to see the priority for packages, queue for next rebuilds? 13:32:48 not at this point, but it is on todo list 13:32:57 not that it would be crucial feature but could be handy if someone is wondering about when the package will be scheduled.. 13:33:04 mizdebsk: sounds fine 13:33:43 mizdebsk: so we do not have UI for adding a package to Koschei yet, right? 13:34:17 hhorak: not yet, but msimacek is working on it 13:36:20 I'm thinking about that because we should describe the process of adding a package there in the recommendation message. So just thinking if we should wait for having this feature ready or just add some "manual guide" temporarily.. 13:36:36 mizdebsk: do you have any idea if it is a question of days/weeks/more? 13:37:20 i expect it around december/january 13:37:42 until then people can contact me or msimacek to have packages added 13:38:35 so perhaps the initial recommendation is to just say "this is the direction we're heading, start the conversation with infra" 13:38:45 minor question but will there be a way to tell that a koji task is from koschei? 13:39:01 but not really looking to expand the package set until the updated UI is available 13:39:40 I assume answer will be they will be run by koschei user? 13:39:43 juhp_: we asked rel-eng for koschei certificate, once done you'll be able to see the build as requested by user "koschei" 13:39:50 cool 13:40:04 https://fedorahosted.org/rel-eng/ticket/5941 13:40:51 ncoghlan: i was looking for recommendation like "we think it is a good idea and would like to see it as official fedora service" 13:41:04 ^^^ that :) 13:41:20 by no means koschei is done, development will definitely continue 13:44:28 mizdebsk, +1 sounds fine to me 13:45:15 #info Koschei devels would like to have our recommendation for using of Koschei before trying to move it into fedora-infra 13:45:15 #info mmaslano, juhp and hhorak are +1 for that, others are expected to vote on ML 13:45:15 #action All voting members should vote on ML on the proposal: "Env & Stacks will recommend Koschei as a good idea and we would like to see it as official fedora service" 13:45:15 #info UI for adding a package to Koschei is not ready yet, could be ready around December/January; until then people can contact mizdebsk or msimacek to have packages added 13:45:16 #info Koschei builds should be distinguished by user (should be build by koschei user in the future) 13:47:06 mizdebsk: are you fine with final resolution on recommendation the next week so we give some time to other members for voting? 13:47:24 hhorak: yes, of course 13:47:27 thank you 13:47:57 mizdebsk: np, thank you guys for koschei! 13:48:56 so, let's quickly see if there are some news about the exciting projects.. 13:49:05 #topic Follow-up: language specific mirrors for Fedora Playground compliant packages 13:49:20 ncoghlan: anything you'd like to share? 13:49:53 (bkabrda is sick this week, unfortunately) 13:50:14 bkabrda submitted the VM request to infra, but I'm not aware of any further progress 13:50:50 my own availability will be limited until after October 30 due to other priorities 13:51:01 but should pick up after that 13:52:01 ncoghlan: ok, just let us know anytime if anything changes. 13:55:21 oh, missed the most of the meeting :( 13:55:47 I was thinking a bit about a similar effort for other languages, but remembering some gossips that other languages' maintainers are not that much in favor of something like that.. 13:56:30 I'm actually curious what other developers in these languages (not only package maintainers, but also some users/devels) think about it.. 13:57:07 would "src repo" be a better name than "mirror"? 13:57:08 And I haven't think off anyhing better than just to ask publicly on the -devel mailing list for opinions, i.e. to ask actual ruby/perl/php/nodejs developers hanging there what they think about such idea... 13:58:14 I'm a bit afraid to use repo since it reminds RPM repos too much to me. 13:58:18 juhp_: I think mirror is better 13:58:33 okay except it is not a mirror :) 13:58:38 juhp_: or "filtered mirror", since it's not a complete set 13:58:45 yeah that might work 13:59:11 as hhorak says, repo is a problem due to the confusion with yum repos 13:59:15 anyway just a comment, I find "mirror" a bit confusing 13:59:32 yeah, there's only so many words to go around in this space 13:59:42 devpi *is* a caching mirror 13:59:56 right that is why I wrote "source repo" but I can see it is not ideal either 13:59:58 but it runs an opt-in whitelist by default 14:00:04 ok 14:00:14 especially for Python, since wheels are a binary format :) 14:00:36 okay miror probably makes sense for devpi :) 14:00:42 r 14:00:50 I think we should definitely qualify it as "filtered mirror", though 14:00:59 ok 14:02:42 for other languages, it arguably makes sense to stick with one language as a pilot, and then look to see if their are devpi equivalents for other languages if it works out well 14:03:09 right 14:03:39 #info even if it is kind of repo, we should rather use term "filtered mirror" to not be confused with RPM repos and since devpi is actually a caching mirror 14:04:16 ok, sounds fine to me as well 14:05:15 #info we should rather postpone something similar in other languages until we see consequences and results of devpi pilot 14:05:33 one interesting point... we don't have a CDN available to Fedora infra, do we? 14:06:12 mirrormanager? 14:06:37 okay that is not specific to infra though 14:07:12 I don't believe that's quite the same thing - I was talking about a transparent HTTPS CDN, like the one Fastly provide for pypi.python.org 14:07:24 ah 14:08:20 * hhorak haven't heard of anything but also not much educated in that regards 14:08:35 so we may end up needing to find a way to push the filtering to PyPI upstream instead (maybe as a Warehouse feature) for performance reasons 14:08:57 anyway, something to worry about later 14:09:21 we can try out the workflow with devpi first, and look at the performance consequences as part of that 14:09:55 #info we should investigate performance as part of the pilot, as pypi.python.org is behind the Fastly CDN 14:10:37 ncoghlan: would you mind adding this whole "filtered mirror" task to the list at https://fedoraproject.org/wiki/Env_and_Stacks/Tasklist, please ? 14:13:10 well, I can do it as well, but would include you and bkabrda as the owners, hope you're ok with that :) 14:13:20 adding it now :) 14:13:33 all right, thanks 14:14:06 Anyone anything else "filtered mirror" related? 14:16:53 #topic Follow-up: SCLs, building above them and their position in Fedora/EPEL 14:17:19 mmaslano: you wanted to ask open shift about how they use the SCL, any luck with that? 14:18:02 hhorak: yes 14:18:26 hhorak: they have scls in docker images, they are enable by wrapper 14:18:28 https://github.com/openshift/ruby-19-centos/blob/master/ruby/.bashrc 14:18:49 I didn't find anything for Python, no-one cares about it probably ;-) 14:19:46 mmaslano: ouch, that's a nice hack, but still hack :) 14:20:10 I can't remember why we wanted to know that though :) 14:20:11 hhorak: yes, but it's not their priority to make it nice 14:21:46 mmaslano: Python world is still primarily Heroku afaik 14:23:33 and a wrapper itself that is expected to be used in shebang 14:23:45 ...is here: https://github.com/openshift/ruby-19-centos/blob/master/ruby/bin/ruby 14:24:00 anyway, it's definitely another reason why scl-utils should offer persistent enabling, people just need it. 14:24:07 definitely 14:24:39 also worth looking at tools like rbenv/pyenv etc 14:25:11 as well as conda for a cross-platform example 14:25:19 yes, spoke about conda 14:25:25 it has different issues 14:25:42 bkabrda would know more 14:25:59 yeah, there are at least some interesting decisions around dynamic linking that can cause build challenges 14:26:34 still, it mostly works, and they cover over some of the gaps with prebuilt binaries 14:28:07 anyay, my main point was that "environment" level activation is very useful 14:28:23 * hhorak must learn more about conda 14:29:08 (shouldn't be much pain to make it a AI for everybody :) ) 14:29:39 certainly far from perfect, but solves a complex problem (user level installs of scientific software across Windows, Mac OS X and Linux) 14:30:44 for server side dev, things like pip and virtualenv are generally a much simpler option 14:31:25 #action Everyone should look at conda since there are some interesting features solved (beside other challenges brought up) so we won't repeat the same mistakes or will be able to use the usable features 14:32:19 (and if anyone figures out how to bring the pyenv user experience to conda, I suspect a lot of people would thank them, me included...) 14:35:03 We exceeded 90 minutes, so let's move on.. 14:35:10 #topic Picking chairman for the next meeting 14:35:18 anyone interested? 14:35:33 I could do it 14:35:36 but you are FESCo liaison since next time ;-) 14:37:13 Yeah, great challenge for me, will try my best :) 14:37:56 :) 14:38:43 mmaslano: so you're fine with chairing the next week? 14:38:53 (as a bye bye chairing?) 14:39:32 yes exactly 14:39:54 #action mmaslano will chair the next meeting 14:40:04 #topic Open floor 14:40:14 anything else we should discuss today? 14:40:28 I'm going to head out at this point - rather late here :) 14:40:39 thanks for the discussion all 14:41:41 Yup, thank you all! 14:41:56 #endmeeting