15:03:06 <bcoca> #startmeeting ansible core irc plublic meeting
15:03:06 <zodbot> Meeting started Thu Jun 20 15:03:06 2019 UTC.
15:03:06 <zodbot> This meeting is logged and archived in a public location.
15:03:06 <zodbot> The chair is bcoca. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:03:06 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:03:06 <zodbot> The meeting name has been set to 'ansible_core_irc_plublic_meeting'
15:03:55 <bcoca> #topic include_vars defaults
15:04:06 <bcoca> #info https://github.com/ansible/ansible/pull/23738
15:04:17 <bcoca> #info https://github.com/ansible/proposals/pull/21#issuecomment-503569642
15:05:01 <bcoca> @lhoss ?
15:06:03 <bcoca> #topic open floor
15:06:44 <cyberpear> I think it would be useful; something like include_defaults would suffice
15:07:12 <bcoca> you alreayd have many ways of doing it, w/o having to include more special cases for var importing nor precedence rules
15:07:42 <bcoca> i would recommend restructure vars to be 'distro friendly' e.g packages['centos'] = [ ... ]
15:07:50 <lhoss> Hii
15:08:00 <bcoca> # topic include_vars defaults
15:08:07 <bcoca> hi, had almost moved on
15:09:23 <bcoca> #topic include_vars defaults
15:09:28 <lhoss> So, as I wrote in my latest  comment (last info link -^) it would be a great start  to get the PR from pfink done (1. solution)
15:10:28 <bcoca> im -1 on that, it conflicts with combine_vars/hash_merge
15:11:21 <sivel> I have nothing really to offer here
15:12:06 <cyberpear> I've been tempted to use `include_role: name={{ role_name }} defaults_from` to achieve the outcome
15:12:18 <bcoca> that also works
15:12:35 <bcoca> i would even prefer my !include yaml PR (that was also rejected) over this one
15:12:40 <cyberpear> with `tasks_from=noop.yml`
15:12:46 <lhoss> that include_role hack is not enough
15:13:33 <lhoss> I thought the  hash_merge wasn't an issue finally , after pfink and kustodian's comments, see https://github.com/ansible/ansible/pull/23738#issuecomment-381603457
15:13:33 <bcoca> lhoss: its only not enough if you insist of writing everything in a very specific ways, there are many other ways to write things that 'just work'tm
15:14:12 <bcoca> pfink is wrong on his assumption
15:14:29 <bcoca> and kustodian says as much, though he is willing to live iwth the conflict, i am not
15:14:35 <sivel> honestly, hash_behaviour is a thing we don't tend to recommend using in any case
15:14:43 <bcoca> ^ also
15:14:58 <bcoca> and im VERY wary of adding other cases, specially conflicting ones
15:16:38 <lhoss> if there wasn't  hash_behaviour=merge ,  this wouldn't be an issue at all , right?  (and which is not recommended anyway, as @sivel says)
15:17:09 <bcoca> i would probably be more opent to this since i would not be as aware of the pitfall
15:17:28 <bcoca> i would still think its a bad idea, but not as opposed since i would not be as burnt by it
15:19:38 <sivel> I guess also what I am saying, is that we should shy away from things that work like hash_behaviour. I believe our stance it that it should be done explicitly through things like combine
15:19:50 <sivel> if we don't recommend it in one place, we shouldn't recommend it elsewhere
15:20:19 <lhoss> Yeah well, another topic is if one could remove some more exotic features, like this hash_behavior, in a future ansible (major release .. 3.x?!), that otherwise block progress  .. and also simplify  things due too many choices (just thinking why there's not just one  YML file extension).. anyway
15:20:47 <jtanner> https://www.reddit.com/r/ansible/comments/a7x92q/in_which_case_hash_behaviourmerge_can_become/ec8mf7f/
15:21:12 <lhoss> ( oh will read that later -^)
15:21:38 <bcoca> ', maintenance will haunt you for the rest of the project's lifetime' <= truer words ...
15:23:37 <lhoss> yeah I now that (having 15y+ dev eng. experience),  but assuming I cannot solve  the hash_behavior conflict here  (though I still don't fully get the problem, if one explains the 'override/skip_defaults' flag  as ignoring any hashing)
15:23:53 <lhoss> s/I now/I know/
15:24:12 <bcoca> see the post, it applies also to the feature you are asking for
15:24:40 <lhoss> ok will do ...
15:24:55 <bcoca> that you NEED this, means you are structuring your data/inventory in a specific way, that does not work smoothly with Ansible, adding this feature is 'forcing square peg in round hole', might make you happy for 5 mins, but introduce many issues down the line
15:25:52 <lhoss> in the meantime  *any* hope  to go the proposed strategy °2 (2. info link: https://github.com/ansible/proposals/pull/21#issuecomment-503569642 ) ?
15:26:03 <cyberpear> (I've been annoyed by roles that do `include_vars: with_first_found: distro, os_family, default` because I can't then override them easily)
15:26:18 <cyberpear> (like in host_vars)
15:26:27 <bcoca> everythign overrides defaults
15:26:30 <lhoss> > that you NEED this, means you are structuring your data/inventory in a specific way, that does not work smoothly with Ansible
15:26:37 <bcoca> and with_first_found is a 'false' loop
15:26:38 <lhoss> I do not agree :)
15:27:30 <bcoca> lhoss: as i showed above, structing a packages[<distro>] makes this irelevant, while structuirng <distro>.yml with packages: makes this more attractive vs include_vars
15:28:12 <bcoca> + first_found
15:28:30 <cyberpear> (I've seen `with_first_found` recommended as "best practice", but yes; it's a loop that only has one item)
15:29:11 <lhoss> The use-case to have it is very simple , and I'ld say  *really important* to support following in a good way  (also many other people commented the same) - having role default vars for different distros or even just different version (like centos7+ centos8 ..coming soon!!) - AND being able to override these naturally in group_vars  (as include_vars will break that) -
15:29:44 <cyberpear> structuring as packages[<distro>] also makes it complicated to override; now I have to have packages['fedora'] in my host_vars, and the form will be diff for each distro
15:30:17 <cyberpear> I believe the include_role hack works around most (all?) of those issues
15:30:27 <cyberpear> (ignoring merge_hash)
15:31:11 <cyberpear> but then you have to have empty tasks/noop.yml, handlers/noop.yml, vars/noop.yml, etc
15:32:05 <lhoss> +1  too complicate if one cannot simply override single vars (without using dicts with merge)
15:32:23 <bcoca> not sure how  those are diff
15:32:27 <bcoca> x: overridesx
15:32:38 <bcoca> d['x']: overrides d['x']
15:32:48 <bcoca> and if you want to merge |combine
15:34:12 <lhoss> uugly .. sorry :)    also I haven't see (m)any  roles  doing that workaround .. have you seen geerlingguy's  roles, the workarounds he used?
15:34:40 <bcoca> no, i always foudn his roles overtly complicated
15:34:44 <lhoss> no wonder  he wrote that proposal ca 3y  ago  (which we linked)
15:34:47 <bcoca> so i have avoided them
15:35:11 <bcoca> others do like them, so im not saying they are wrong, i jsut use much simpler approaches
15:36:29 <lhoss> tja  not all roles  can be super simple .. like the one I contributed  to deploy a big-data stack : https://github.com/hortonworks/ansible-hortonworks/tree/master/playbooks/roles/ambari-config/vars
15:36:48 <lhoss> That one is a good example  why we need  a better solution
15:38:03 <bcoca> again, i woudl argue, that is a good example of bad variable organization
15:38:10 <lhoss> I can understand  if you say  for many simple roles, write 1/separate role per distro .. if there's enough role maintainers  etc ..
15:38:15 <bcoca> also, vars is high precedence
15:38:28 <bcoca> no, you can have single role for multiple distros
15:39:00 <jtanner> need to timebox this conversation. maybe extend it to the ansible-devel list?
15:39:16 <bcoca> nothing else on agenda, we can go on for a bit
15:39:58 <lhoss> ..  where there's quite some logic, you *don't want* to copy/paste to different roles just to support various distros , that may just have different var . values ..
15:41:06 <bcoca> there is no need for logic, just how you organize vars
15:43:34 <lhoss> I know that  'vars' is not the right place (higher precedence) , but the maintainer is anyway doing an include_tasks (1.task) where he loads the right vars for the found distro .  That's what I want to change!  (because right now those vars cannot be overriden in group_vars)
15:44:02 <bcoca> again, this is trying to compensate for bad role design
15:44:38 <cyberpear> maybe we need some better docs on proper role design
15:44:58 <lhoss> Right now there's no proper solution  to organize the vars  without any work-arounds (like  using  a duplicate __var  and the sect_fact with  the default filter, that geerlingguy uses)
15:44:59 <bcoca> i understand that the proposed 'fix' allows you to compensate in simpler way for the bad role design, but it does not actually solve the issue
15:45:35 <bcoca> cyberpear: l woudl agree with that statement
15:45:47 <lhoss> If you show me  a better solution with current ansible 2.8.x  ... more than happy :)
15:46:01 <bcoca> lhoss: i can show better with ansilbe 1.5
15:46:28 <bcoca> but we even have disagreement internally on what a 'good role' is
15:46:43 <lhoss> I'm quite sure I understand  all the options .. but feel free to propose me something (you think) I'm missing  (it's enough to send an example link)
15:46:47 <bcoca> most of the docs on the subject have been written with people i personally disagree with
15:47:12 <lhoss> Maybe we should have a majority vote :-o :)
15:47:14 <bcoca> and which jump through a lot of hoops to get ansilbe to work with them
15:47:28 <bcoca> lhoss: my suggestion is very simple and posted above
15:47:58 <bcoca> k, votes for adding overwrite to include_vars?
15:48:00 <bcoca> -1
15:48:26 <sivel> -1
15:48:32 <agaffney> -1
15:49:08 <cyberpear> -1, but I'd be in favor of something like include_defaults to achieve same purpose
15:49:46 <lhoss> +1 (I need pfink, kustodian and ???   I guess difficult battle)
15:50:30 <shertel> -1
15:50:40 <lhoss> "include_defaults"  I'ld be more than happy !  Just need same pointers, it it could get accepted .. what to worry about
15:50:45 <bcoca> lhoss: i can tell you teh more people we add, you'll still lose, we've voted on this many times, unless you prevent more commiters  from voting ...
15:50:46 <agaffney> I had looked into include_defaults, but there were some complications (that I don't remember), and there are existing ways to accomplish the same thing
15:51:07 <bcoca> mostly that  modules wont be able to update role defaults
15:51:14 <bcoca> nor action plugins
15:51:19 <bcoca> requires a rewriting of roles
15:52:32 <bcoca> s/of roles/of role loading and variable scoping/
15:52:34 <cyberpear> Is there a way to `with_first_found` for dict keys?
15:52:38 <lhoss> There must be a way   ..  what about  another way of loading vars, that can be overriden by group_vars ?!
15:52:54 <bcoca> cyberpear:  map + first_found
15:52:58 <bcoca> or select
15:53:12 <bcoca> lhoss: vars plugins
15:53:16 <lhoss> (sry my  sentence  -^   I wrote before saw cyberpear's)
15:53:46 <bcoca> actually, first_found would not need map/select
15:54:24 <lhoss> Aha 'vars plugins' , could be something   (will have a better look .. though nobody yet had that idea)
15:55:38 <bcoca> so im going to close the proposal, this 'feautre' keeps getting voted down
15:55:59 * cyberpear prepares example...
15:57:17 <cyberpear> https://github.com/jamescassell/multi-role/blob/master/defaults/main.yml
15:57:29 <cyberpear> bcoca: is that how you'd do it? ^
15:57:41 <cyberpear> is there a better way to `with_first_found` than the chain of  default()'s there?
15:58:01 <lhoss> I got some other ideas .. but all not nice .. one other 'hack' is  to have default vars  set as value undefined  (using varname: !!null ), and then use group_vars to set per distro ( too bad that playbook group_vars have higher prio  than inventory ..WHY is that btw?)
15:58:10 <bcoca> that is one way
15:58:58 <bcoca> multivar:  multi_var_default|first_found([ansible_distribution, ansible_os_family ...])
15:59:22 <cyberpear> interesting; I didn't know that was possible
15:59:47 <bcoca> not that i would struct that way, but that does give you 'smaller alias' to start on other vars
16:00:23 <bcoca> again, many ways to do it, use what works for you, but some ways are just too hard to use with ansible's basic design
16:00:31 <cyberpear> (I thought first_found only looked for files0
16:00:59 <bcoca> it does, but does not mean you cannot do same for keys
16:01:28 <bcoca> i didnt mean specifically the first_found filter, but a first_match .. or really select|first
16:02:11 <bcoca> multivar:  (multi_var_default|list)|union([ansible_distribution, ansible_os_family ...])|first
16:02:27 <bcoca> ^ i was more talking 'concept' than 'working code'
16:02:37 <bcoca> but last one shoudl be 'close' to working
16:02:51 <bcoca> s/union/intersection/
16:02:57 <lhoss> thx cyberpear for that example .. had seen that also (recently),  but really not so nice  that for each var def  you have that long filter chain..
16:04:18 <bcoca> k, now we exceeded meeting time and we are more talking about var layout design, so closing this for now, we can continue in devel
16:04:22 <bcoca> #endmeeting