19:00:31 <gundalow> #startmeeting Ansible Molecule Working Group
19:00:31 <zodbot> Meeting started Wed May  1 19:00:31 2019 UTC.
19:00:31 <zodbot> This meeting is logged and archived in a public location.
19:00:31 <zodbot> The chair is gundalow. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:00:31 <zodbot> The meeting name has been set to 'ansible_molecule_working_group'
19:00:34 <gundalow> Who's around?
19:00:49 <decentral1se> heyo, I'm lurking
19:01:56 * tima waves
19:02:02 <smyers> ello
19:03:20 <gundalow> #chair decentral1se tima smyers
19:03:20 <zodbot> Current chairs: decentral1se gundalow smyers tima
19:03:22 <gundalow> Hi all
19:03:33 <gundalow> #info agenda https://github.com/ansible/community/issues/427
19:04:24 <gundalow> How is everybody doing?
19:05:45 <gundalow> So we have four things on the agenda
19:05:45 <tima> Doing fine. Keeping busy.
19:05:45 <gundalow> * Me being offline for three months
19:05:45 <gundalow> * Proposal
19:05:46 <gundalow> * Two PRs
19:06:27 <decentral1se> All good, in and out around here but still in touch, I think :)
19:06:46 <decentral1se> I have a tiny update about CI + Hetzner to add
19:07:31 <gundalow> #info As I've mentioned a few times, I'll be on parental leave from 10th May for three months. Will be looking to others to run the IRC meetings, zbr|rover has kindly offered to help. Though first person in can just do `#startmeeting Ansible Molecule Working Group` `#topic foo` #info bar` `#endmeeting
19:07:36 <gundalow> decentral1se: ah, ace
19:07:55 * alongchamps waves
19:07:58 <gundalow> #topic Make Ansible default verifier #2013
19:08:01 <gundalow> #chair alongchamps
19:08:01 <zodbot> Current chairs: alongchamps decentral1se gundalow smyers tima
19:08:03 <gundalow> Hi :)
19:08:08 <gundalow> #info https://github.com/ansible/molecule/issues/2013
19:09:29 <decentral1se> heyo alongchamps
19:09:49 <decentral1se> right, so there was some confusion on this one I think, but that has been explained
19:09:59 <alongchamps> hey all
19:10:30 <decentral1se> one concern which I share: "Also writing good assertions in Ansible is surprisingly hard and brittle."
19:10:49 <decentral1se> and my point was that we would need to have some docs and a wide pre-release time
19:10:58 <decentral1se> discuss :)
19:11:17 <gundalow> Yup, ^ is fair points
19:12:04 <smyers> I love the idea of an ansible verifier, and have done effectively this by using asserts in converge, which is less than ideal but equally effective. So, apples to apples, ansible to testinfra, my opinion is that writing asserts in testinfra is easier than writing them in ansible.
19:12:52 <decentral1se> to be clear, I agree with the idea that we shouldn't have to ask Ansible people to learn Python programming + Pytest
19:13:16 <decentral1se> addressing the "hard+brittle" concern for me would be: what are the best practices and correct modules?
19:13:30 <decentral1se> smyers: you've mentioned assert, is that the best out? I haven't done much verifying with ansible ...
19:13:59 <decentral1se> "best", but well, what are people doing in the wild, I mean
19:14:04 <smyers> Having an ansible verifier means we don't have to ask anyone to learn python, but I think it's reasonable to ask them to do that by default, versus trying to write a test suite in ansible
19:15:06 <smyers> And yeah, I can only speak to my own experience, of course, but I use the ansible assert module to make assertions in ansible for parity with the python assert statement
19:15:14 <decentral1se> ah ok
19:15:18 <smyers> hopefully that makes sense and doesn't sound too silly :)
19:15:31 <decentral1se> I mean, I can imagine it being quite easy to do state:present for files/directories/packages etc.
19:15:49 <smyers> The thing about testing roles is that my team has a philosophy of "you don't ahve to test ansible"
19:16:36 <smyers> So in some cases, native ansible asserts are the onlyy way to ensure that, given a certain set of input vars to the roles, tasks that should be called were called, tasks that shouldn't be called were not called, facts were set appropriately, etc.
19:16:39 <gundalow> Example integration test for the `stat` module using `assert` https://github.com/ansible/ansible/blob/devel/test/integration/targets/stat/tasks/main.yml
19:17:05 <smyers> To test against those things in python would require dumping out a ton of stuff just to parse it in python and run the asserts there, so native ansible asserts win out
19:17:23 <decentral1se> yeah nice examples
19:17:37 <decentral1se> I think we just need a shedload of documentation and examples for this to be an OK transition
19:17:43 <decentral1se> from my POV
19:18:14 <tima> The transition should be part of a 3.0 release.
19:18:35 <decentral1se> Good idea. Are there any other major concerns?
19:18:39 <smyers> I do think it's fine either way, but (and this is perhaps off topic) I think it would be a good idea to cut a release now and consider...what tima just said with fewer words. :)
19:18:45 <gundalow> Yup, also we've not done a release with this in yet, so would be good to "field test it"
19:18:50 <gundalow> smyers: hifive
19:19:33 <tima> I get the arguments for testinfra and using python but Ansible has always prided itself on no programming skills necessary
19:19:50 <tima> And being automation for everybody.
19:21:15 <tima> As the official tool for developing roles, playbooks and eventually collections requiring python to test them...
19:21:18 <smyers> That's a great point, I'm sold on switching it to be the default verifier. My only worry is if changing the default can in any way be a breaking change for existing users. I don't personally think so
19:21:21 <decentral1se> Maybe having this as a default will push the standard on how easy it is to use Ansible as a verifier
19:21:32 <decentral1se> when people are using it more. Maybe the standard is already high though ;)
19:21:32 <tima> Yeah that’s inconsistent.
19:22:03 <decentral1se> oh right, migrating from old versions ...
19:22:12 <tima> I think Ansible could make verifying think easier and more frictionless.
19:22:38 <tima> Verifying things**
19:22:46 <decentral1se> wait, the "upgrade guide" will just be: change your configuration to use testinfra if you already do testinfra
19:22:50 <tima> Docs will help.
19:22:52 <smyers> It would also be great to be able to use multiple verifiers, but that sounds like its own proposal
19:23:07 <decentral1se> we'll have to be careful how we upgrade the molecule definition to not break the configs of existing users
19:23:23 <tima> Better modules for testing even better.
19:23:48 <tima> @decentral1se: +1
19:24:03 <smyers> Yeah, I just double-checked the schema. Assuming I'm reading this correctly, there is no default verifier in the schema, it must be explicitly set. That's how pretty much every section works, so I think the impact to existing molecule users is zero. Existing scenarios will use their configured verifier.
19:24:31 <tima> Hence my 3.0 timing suggestion.
19:24:31 <decentral1se> should we just allow multiple verifiers in this work? That'd be great
19:24:50 <gundalow> decentral1se: interesting idea
19:25:00 <gundalow> what would the switch for that look like?
19:25:32 <smyers> That sounds like a larger effort that should be its own separate thing. I just opened up a tab to start writing up a multiple verifiers proposal :)
19:25:44 <decentral1se> lovely
19:25:54 <decentral1se> yeah, I think it would not be seamless now that I look at the molecule.yml config
19:26:11 <decentral1se> we have verifier: name: testinfra - which is not a list for anything multiple
19:26:30 <decentral1se> but yeah, I think you're right, switching to new default may have zero impact on existing configs
19:26:50 <smyers> I'm thinking we can do some trickery there. If verifier is a dict, there's one verifier. If it's an array/list, there's one or more
19:27:26 <decentral1se> cool, I think we've got a good idea of how to move this on
19:27:28 <smyers> I want to poke at the schema validation a little bit to see if that idea can fly, and maybe come up with some examples of how that won't end up being a nightmare for users
19:27:43 <smyers> Another option is to require either verifier or verifier*s* at the top level
19:28:03 <smyers> But yeah, the wheels are turning :)
19:28:04 <decentral1se> smyers: molecule uses http://docs.python-cerberus.org/en/stable/ for schema validation. It ain't too pretty
19:28:43 <smyers> I'm unfortunately familiar with cerberus, I think it can be coerced to do some of what I'm thinking of, but it will indeed not be pretty
19:29:19 <smyers> Back to the agenda, though, it sounds like there's consensus regarding making the ansible verifier the default verifier
19:29:23 <decentral1se> I've worked with it a bit. Ping me if you want to discuss the hacks :)
19:29:29 <decentral1se> yep
19:29:29 <smyers> rgr
19:29:48 <tima> My only concern is timing
19:29:51 <smyers> I appreciate the heads-up and the offer to help
19:30:03 <tima> And how that transition happens
19:30:55 <gundalow> 1) What exact docs are needed
19:30:55 <gundalow> 2) Do we want to have a release without default to get feedback?
19:31:53 <tima> On 2: +1
19:32:55 <decentral1se> 1) examples on typical use cases? Are services up/packages exists/files,dirs perms check/asserts/which modules are recommended
19:33:34 <decentral1se> 2) sounds fine
19:35:10 <tima> Sorry. Need to drop early today.
19:35:18 * tima waves bye
19:35:52 <smyers> For 2, yes. For 1, probably examples on how to assert and what to assert in an ansible verifier. Having a separate verifier outside of the converge playbook is limiting, since you won't have access to facts registered in converge, so it might be good to document ways to get around that problem? Mostly just thinking out loud here, though
19:35:53 <decentral1se> all good, take it easy
19:36:46 <decentral1se> smyers: I think fabianvf an implementation level question regarding whether we should do that or not
19:36:59 <decentral1se> that sounds like a thing we'd definitely want now reading it ...
19:37:10 <gundalow> tima: Thanks as always for your time
19:37:34 <decentral1se> but anyway, we're missing that in testinfra now too (you have to load vars manually)
19:37:45 <fabianvf> yeah that's what I was going to say
19:38:03 <fabianvf> I don't think it's bad to have your test state separate from the thing you're testing
19:38:09 <decentral1se> %s/an/had an/
19:38:22 <decentral1se> that's a good point too
19:38:27 <smyers> Yeah, having an ansible verifier doesn't mean that I'm going to move my asserts out of converge. It should, but it doesn't.
19:38:34 <decentral1se> depends on the implementors, I suppose
19:39:01 <decentral1se> gundalow: does that shed light on needed docs?
19:40:03 <gundalow> Yup, great thanks
19:40:07 <smyers> I gnerally never test that the tasks in a role worked; that's a job for ansible's test suite. I want to test that my when clauses and includes made the right choices, and there's no way that I know of to test that without having access to facts at the end of converge, either by asserting right there or outputting them somewhere and asserting against them elsewhere.
19:42:11 <decentral1se> ack, yeah, there will be a lot of manual duplication if you can't seamlessly access the facts
19:42:40 <decentral1se> we should raise a ticket against this to think through it? Maybe we can leave it to the implementor too (not us)
19:43:04 <decentral1se> by passing a fact dict through to the verifier ...
19:43:15 <gundalow> I've added documentation requirements to https://github.com/ansible/molecule/issues/2013#issuecomment-488392662
19:43:17 <fabianvf> At least having a way to pass facts through would be nice for sure
19:43:20 <smyers> Yeah, I think it's worth touching on in the docs and maybe coming up with an enhancement that allows you to pass facts through
19:43:25 <gundalow> jopefully I've captured that correctly
19:43:29 <gundalow> #chair fabianvf
19:43:29 <zodbot> Current chairs: alongchamps decentral1se fabianvf gundalow smyers tima
19:43:32 <smyers> For good separation, it's probably not great to pass facts by default
19:43:37 <fabianvf> then it's explicit what goes through
19:43:39 <smyers> So we're forward-compatible already :)
19:44:04 <decentral1se> oh yes, nice ideas
19:44:07 <decentral1se> ticket looks good
19:44:23 <gundalow> decentral1se: of course they are good, I stole what y'all said in here :)
19:45:52 <decentral1se> hehe
19:46:45 <decentral1se> moving on?
19:46:49 <gundalow> Yup
19:47:10 <gundalow> #topic Add a CLI command for collections initialization ansible/molecule#2014
19:47:19 <gundalow> #info https://github.com/ansible/molecule/pull/2014
19:49:29 <decentral1se> so, this is still in draft and I haven't had time to look at it
19:49:40 <decentral1se> very happy to see it coming though
19:49:50 <decentral1se> anyone have any points on this?
19:50:45 <smyers> My only thought for the collections stuff is weighing the balance between having a "collections" branch and bringing collections features to master all at once, and bringing in pieces of collections piece by piece that basically won't be useful until all collections features are done
19:51:58 <decentral1se> I was hoping it would be a case of a typical molecule install, latest or whatever
19:51:59 <fabianvf> This may be slightly off topic, but I did have a question related to initialization - do we ever plan on having code templates for modules/plugins/etc? Presumably this will be a much more common pattern with collections and being able to template out a plugin or module would be really nice (it can be a bit tough to remember exactly what structure the particular thing you are making needs to follow currently
19:52:06 <decentral1se> and then a separate CLI command `init collection` or whatever
19:52:12 <smyers> There's no clear winner for me, at the moment I'd arbitrarily prefer to keep collections features off of master until you can actually aim a `molecule test` sequence at them
19:52:18 <decentral1se> and then just installing ansible from collections branch source
19:53:08 <decentral1se> fabianvf: AFAIK, you can override the entire role template (and not have to override the molecule template)
19:53:38 <smyers> Templates is a great idea. I'm assuming we all just go steal from an existing ansible plugin, gut it, and work from there. If molecule is to be the official authoring tool, I think it makes sense to have the official templates for things like plugins be part of molecule in some way.
19:55:24 <gundalow> decentral1se: thanks for adding the other doc link
19:55:29 <decentral1se> ok, updated the ticket to ask about how this will go into circulation
19:55:32 <gundalow> (to the proposal)
19:55:35 <decentral1se> gundalow: sure thing
19:57:08 <decentral1se> ok, tried to capture concerns in: https://github.com/ansible/molecule/pull/2014#issuecomment-488396666
19:58:16 <fabianvf> smyers: +1, it might also be worth it to include it as part of the collections work, since AFAIU a large part of the value add is packaging/distributing plugins
19:58:24 <smyers> I think that touches on it well. I'm eager to see/help the collections stuff become real, but am nervous about jumping in without have a better sense of the broader plan
19:58:40 <fabianvf> being able to init a plugin/module into the proper place in a collection would be really nice IMO
19:58:55 <smyers> yeah
19:59:05 <decentral1se> wanna open a ticket?
19:59:36 <smyers> things are getting crazy there from an interface perspective, e.g. molecule init collection plugin filter filterpluginfooname, but...that's a potential problem to solve later. :)
19:59:58 <fabianvf> heh, true, but it's even more overwhelming without the list :P
20:00:04 <smyers> yeah
20:00:08 * alongchamps time to head out
20:00:13 <fabianvf> decentral1se: I'd be happy to
20:01:05 <smyers> I'm hacking on the multiple verifiers prop right now, then there's the plugin template thing, and we probably want a ticket about passing facts from converge into an ansible verifier
20:01:15 <smyers> don't think anyone volunteered to write up that last one :)
20:01:15 <decentral1se> thanks!
20:01:33 * smyers backs away slowly, but not *too* slowly
20:01:42 <decentral1se> go go go go go go go :)
20:03:08 <decentral1se> OK, I have one point about the hetzner stuff
20:03:30 <decentral1se> #topic hetzner cloud and CLI
20:03:32 <decentral1se> if I may ...
20:03:48 <decentral1se> basically, I am in touch with hetzner's new ansible hcloud_server module implementor
20:04:10 <decentral1se> and there is some interest in getting support to run the CI properly to validate the new driver implementation
20:04:44 <decentral1se> there was some fair concerns about having another untested driver added to core
20:04:51 <decentral1se> so hopefully this will help address that, that's it!
20:05:03 <gundalow> decentral1se: nice
20:08:42 <smyers> I think the last agenda thing is testing with testinfra + the ansible runner. We've also got an open ticket to make sure testinfra 2 works (and it should...). I can get those tested today or tomorrow if nobody beats me to it.
20:09:11 <gundalow> #topic Test Infra update philpep/testinfra#432
20:09:17 <gundalow> smyers: yup, thanks
20:09:26 <gundalow> #info https://github.com/philpep/testinfra/pull/432
20:09:49 <decentral1se> right, how do we test this? Just upgrade our testinfra installation to this pr branch?
20:10:28 <smyers> That's my initial plan, plus digging into seeing what the branch actually changes and if there's a way to exercise it before and after from molecule
20:10:55 <smyers> My sense of it right now is that if it works, it works, we just need to make sure it works. :)
20:11:59 <decentral1se> cool, I will try to get around to it. Will be great to have this shifting API replaced with the less shifting ansible-runner API :)
20:12:47 <gundalow> :)
20:12:59 <decentral1se> hoping we'll also get a solution for https://github.com/philpep/testinfra/issues/345 from this
20:13:27 <decentral1se> (passing facts through pytest fixtures)
20:14:27 <gundalow> hopefully
20:14:38 <gundalow> bit over time, anything else?
20:14:46 <decentral1se> nothing from me
20:14:56 <gundalow> next week will be my last meeting for a while
20:14:57 <gundalow> cool
20:15:00 <gundalow> Thanks everybody
20:15:02 <gundalow> #endmeeting