19:06:00 #startmeeting core public irc meeting 19:06:00 Meeting started Tue Jul 7 19:06:00 2020 UTC. 19:06:00 This meeting is logged and archived in a public location. 19:06:00 The chair is bcoca. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:06:00 Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:06:00 The meeting name has been set to 'core_public_irc_meeting' 19:06:27 * shertel waves 19:06:27 #topic https://github.com/ansible/proposals/issues/181 19:06:33 \o 19:06:47 I am here :) 19:07:12 first point, 2.10 beta has shipped so no feature changes will make it in for 2.10 .. its too late at this point 19:07:19 any changes will now be looking at 2.11 19:07:28 fine with me 19:07:44 In concept I have no arguments with the proposal 19:08:04 But I don't think you are going to get any more of an agreement than that in this meeting for now 19:08:04 agreed w the concept 19:08:23 We need to draft a complete product requirements doc to outline the behaviors 19:08:26 main reason roles were not updatable was 'no role version scheme' 19:08:44 bcoca: right, I never realized that, but I am fine with doing that just for collections 19:08:45 and then likely assign this work to an internal team to spend a release or part of a release, addressing this 19:09:21 we also didn't have the ability to even compare versions early in the release 19:09:23 my only concern would be to keep a hash of the installed role or collection and only overwrite if not locally modified 19:09:47 It would be nice to have `install --upgrade` or just `upgrade` as a subcommand. 19:09:48 but as I outline above, I think it's premature to really go over this topic now 19:09:50 also we just decided collections require a semantic version for ansible-base also, that opens this up, but also explains why the feature was not thoguht of for 2.10 19:10:17 * bcoca votes for `upgrade/downgrade` 19:10:18 otherwise, require --force to avoid stepping on local ad-hoc changes 19:10:36 bcoca: the reason against another subcommand would be the semantics 19:10:40 cyberpear: we already hase 'verify' so detecting 'dirty' collection should be simple 19:10:44 So we're in favor of the upgrade option, but a lot more design work needs to happen. 19:11:16 I think with -r it should upgrade or downgrade as appropriate 19:11:20 ie install asd:123 would install a collection if not existant, but you'd need update asd:123 to actually update to it? 19:11:29 to match the requirements 19:11:40 cyberpear: same can be said if you include the version on the cmd line 19:11:50 agreed 19:11:53 or how do you see a difference between -r and … 19:11:54 ok 19:11:56 same way pip works 19:12:42 This is the only topic on the agenda, so I won't stop you all from going to town on this. But in concept we are agreed, but I'd ask that no one go run off and work on this 19:12:53 `pip install 'ansible < 2.9'` will downgrade me to 2.8, IIRC 19:12:56 So that we can properly scope this and create a PRD to ensure we cover everything 19:13:01 okay, can you help me a little bit with regard to the next steps 19:13:08 apollo13: just gave them to you 19:13:10 sivel: run off and work on design in proposal ticket .. should be fine 19:13:25 sure, just not actually develop it. 19:13:32 toola .. er nmvd 19:13:38 sivel: sorry, not up2date with the acronyms. PRD is what? 19:13:50 We'll need to put together a team that will work with the community to create a complete product requirements doc 19:14:04 apollo13: ^ PRD == Product Requirements Doc 19:14:16 +1 for proper scoping 19:14:18 But we do understand that this is needed 19:14:29 But it will also not be limited to simply the `install` option 19:14:38 we need to go back and do it completely for the whole CLI 19:15:21 So it's likely that in a future release, we will assemble a project team of employees that will spend a release or part of a release addressing this 19:15:33 sivel: to what extend are you thinking? there appears only to be the install command that seems to be affected by that 19:15:38 (re whole CLI) 19:16:13 apollo13: we also install roles 19:16:18 apollo13: no one sat down and scoped this from the beginning really. Taking different user stories into account 19:16:27 There are also missing commands that people want 19:16:28 and it has more implications than just at upgrade 19:16:36 also do you think this should be a whole kitchensink thing (ie introduce all at once) or could it be split in smaller subtasks, ie relax the requirement of force if a version is specified 19:16:40 specially with deps 19:17:00 mhm, right, which brings me to the next question: are we going to reimplement pip? 19:17:00 Further design proposals and discussion can (and probably should) happen in the ticket. 19:17:11 apollo13: when we scope this work for people to actual work on, it will be broken down into smaller tasks 19:17:31 ok, so in the proposal ticket I can focus on the whole thing. that already helps a lot 19:18:13 Yes, so for now we are in "pre-design" phase, we'll take the proposal and turn it into the actual design and scope before we work on it, and then we'll break it into smaller deliverables 19:18:34 perfect, how much do you think that roles should be part of the design consideration 19:18:48 ie are roles something that is already on the way to beeing deprecated or… 19:18:50 I wouldn't focus too much on roles for now 19:19:08 noted 19:19:24 we can always apply the same ideas to roles later if that is desireable 19:19:35 but would require 'versioning' to work 19:19:50 at least for the situation where you want --upgrade to actually work 19:20:01 which i think is a good compromise and using --force for any 'non versioned/other versioned' role scheme 19:20:06 if you do role install some_role=some_version it wouldn't really matter 19:20:25 The lack of semver for roles is what makes it really tricky. 19:20:34 Personally I think that split would complicate the differences between roles and collections even further 19:20:44 Best to focus on collections for now because roles have other issues that need to be addressed first. 19:21:15 ok, that sounds all good and well and it's great to know that this proposal at least has a chance of surviving :) 19:21:23 thank you for talking the time! 19:21:27 taking* 19:22:03 * cyberpear expects roles to stay around forever 19:22:03 I'm not disagreeing, just saying that role versioning is the wild west because we never set a standard or enforced anything, which is why we never had an update mechanism for roles: no good way to compare arbitrary version strings. 19:22:38 Yes, the general consensu is we want a better update mechanism for collections. 19:22:41 collections are now REQUIRED to use semantic versioniong .. eliminating that problem 19:22:43 We just want to plan and scope it correctly. 19:24:22 sounds good, will it help (or would it be okay) if I write down user stories on the proposal? do you have any examples of how you'd like those to look? 19:25:03 I'm not sure we have examples of how they should look, but you can add them to the proposal in any way you feel is legible. 19:25:23 that would be helpful 19:26:36 ok, that is all I needed to know at this stage I think. Is there anything else you'd need from me? 19:28:17 k, we can add more to the proposal while we wait for a more formal project, so going to stop the discussion on this now and open the floor to other topics 19:28:24 #topic open floor 19:30:31 mhm if open floor means I can throw thoughts around, then I have something else: As a follow up to my proposal I was wondering if there was a way to ensure that depedencies are satisfied when a playbook is run. Image I am using "xxx.yyy" collection in my playbook and the requirements file has it "pinned" to version 10. Now if a user has version 9 installed I want to be able to tell them about that somehow. Do you have any 19:30:31 ideas/thoughts about that (or is that out of scope for here) 19:31:30 currently requirements.yaml is connected to the ansible/ansible-playbook CLI in any way. And I am not really asking for a way to auto download/update dependencies but mostly for a way to prevent dangerous situations where users forgot to upgrade 19:32:17 apollo13: 3rd party or external tooling could be written to do that. We have no plans of adding that functionality to playbook execution 19:34:07 awx/tower already provides a 'install deps and execute' faciliity 19:36:02 runtime would be complicated anyway, especially with includes 19:36:17 static analysis is what I have been playing with lately 19:36:39 sivel: oh I'd be fine with requiring a requirements.yaml in those cases 19:36:52 he, been helping other teams with static analysis .. fun with includes is just the start ` action: module='{{varfromcustomvarsplugin}}'` 19:36:57 Ie just iterate through that and the dependency metadata 19:37:20 * cyberpear happy `meta/requirements.yml` will finally be honored in 2.10 19:37:33 action: module={{ lookup('vars', onevar + lookup('random' ....}} 19:37:40 cyberpear: oh? where does that go, in a role/collection or playbook level? 19:37:46 in roles 19:37:57 to bad :D 19:38:02 but only for `ansible-galaxy install` process 19:38:03 collections have galaxy.yml 19:38:23 ^ looking at making that work as requirements.yml also for cli, so not just on install 19:38:54 I think part of our problem (when we don't use tower) is that we work with multiple ansible "projects" and all having different requirements.txt. The default method of installing collections globally for the user isn't helpful in that regard either. But maybe we are just doing it wrong? 19:39:16 -p and use per project/enviornment directories 19:39:27 the default is useful for most cases, not all 19:39:37 impossible for it to be so 19:39:58 well we currently put ansible.cfg in the project and customize roles and collection paths there to install into a per env directory 19:40:11 Not sure if that is a good solution in the long term though 19:40:39 seems to work at least and is what you suggest with -p without having to actually specify -p 19:40:50 * cyberpear uses an `activate`-like script to set the options via env vars, such as roles_path, etc 19:40:55 (apparently ansible-galaxy installs into the first location in that path) 19:41:07 yes, galaxy installs to the first writable location in the path 19:41:14 first writable 19:41:41 was fun when RHEL shipped with no writable location in their downstream roles_path 19:43:03 bcoca: when you said " ^ looking at making that work as requirements.yml also for cli, so not just on install" what does that mean exactly? 19:43:53 ansible-galaxy install -r galaxy.yml 19:44:19 ah, I got confused by "so not just on install" :D 19:44:20 I see 19:44:45 because that is "on install" :D 19:45:11 it runs when you install the collection, but not when you 'unpack/git clong/etc' 19:45:23 so ansible-galaxy install my.coll1 19:45:35 will run galaxy.yml, but we are also workign so you can 'run directly' 19:45:50 ah neat 19:47:27 also, 2 question on upgrade, a) upgrade current collection? b) upgrade dependencies? 19:47:41 i would only use --force on downgrade 19:50:44 `--force` doesn't affect deps by default 19:51:04 bcoca: depends how the upgrade is triggered imo, ie if you say "galaxy install --upgrade collection" then do as little work as possible to satisfy constraints 19:51:41 uhm that example was probably not the best 19:52:47 I think I kind of prefer b. `--force-with-deps` could just be for downgrading deps then too. 19:53:43 I guess it depends on how dependencies are specified 19:54:13 if dependencies are specified with a version (if possible), then imo galaxy install --upgrade collection should ensure that the dependency is satisfied 19:54:16 even if that means downgrade 19:54:28 ev3en more fun, what do you do when you installed collection B as dep of collection A, but the upgraded B is incompatible? 19:55:05 that can only happen if A specifies a dependency to B without a version right? 19:55:23 in which case you are imo asking for pain 19:55:52 no, can happen if you upgrade B directly 19:55:59 but yes, also in that case 19:56:21 or a > 1.x version, but 2.x didnt exist at that time and now is backwards incompat ... 19:56:25 well even if you update B manually I wouldn't overcomplicate it 19:56:34 many ways to tie noose around neck using package management 19:56:38 ie some failures are to be expected (like with pip) 19:56:46 too many, imho ... 19:57:09 maybe, but then again ansible-galaxy invents yet another package manager XD 19:57:29 yes .. i've had 7yrs to make my peace with that ... will get there .. one day ... 19:58:00 soon *scnr* ;) 19:58:22 * bcoca can point to many rants on 'dont make another package manager' 19:58:46 no rant on "do not make too much rants on not making a package manager" ? 19:59:02 that's a bit meta 19:59:16 misc: no, im pro rant 19:59:36 k, this is going waay off... and its time 19:59:39 #endmeeting