19:00:31 <gundalow> #startmeeting Ansible Molecule Working Group 19:00:31 <zodbot> Meeting started Wed May 1 19:00:31 2019 UTC. 19:00:31 <zodbot> This meeting is logged and archived in a public location. 19:00:31 <zodbot> The chair is gundalow. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:31 <zodbot> The meeting name has been set to 'ansible_molecule_working_group' 19:00:34 <gundalow> Who's around? 19:00:49 <decentral1se> heyo, I'm lurking 19:01:56 * tima waves 19:02:02 <smyers> ello 19:03:20 <gundalow> #chair decentral1se tima smyers 19:03:20 <zodbot> Current chairs: decentral1se gundalow smyers tima 19:03:22 <gundalow> Hi all 19:03:33 <gundalow> #info agenda https://github.com/ansible/community/issues/427 19:04:24 <gundalow> How is everybody doing? 19:05:45 <gundalow> So we have four things on the agenda 19:05:45 <tima> Doing fine. Keeping busy. 19:05:45 <gundalow> * Me being offline for three months 19:05:45 <gundalow> * Proposal 19:05:46 <gundalow> * Two PRs 19:06:27 <decentral1se> All good, in and out around here but still in touch, I think :) 19:06:46 <decentral1se> I have a tiny update about CI + Hetzner to add 19:07:31 <gundalow> #info As I've mentioned a few times, I'll be on parental leave from 10th May for three months. Will be looking to others to run the IRC meetings, zbr|rover has kindly offered to help. Though first person in can just do `#startmeeting Ansible Molecule Working Group` `#topic foo` #info bar` `#endmeeting 19:07:36 <gundalow> decentral1se: ah, ace 19:07:55 * alongchamps waves 19:07:58 <gundalow> #topic Make Ansible default verifier #2013 19:08:01 <gundalow> #chair alongchamps 19:08:01 <zodbot> Current chairs: alongchamps decentral1se gundalow smyers tima 19:08:03 <gundalow> Hi :) 19:08:08 <gundalow> #info https://github.com/ansible/molecule/issues/2013 19:09:29 <decentral1se> heyo alongchamps 19:09:49 <decentral1se> right, so there was some confusion on this one I think, but that has been explained 19:09:59 <alongchamps> hey all 19:10:30 <decentral1se> one concern which I share: "Also writing good assertions in Ansible is surprisingly hard and brittle." 19:10:49 <decentral1se> and my point was that we would need to have some docs and a wide pre-release time 19:10:58 <decentral1se> discuss :) 19:11:17 <gundalow> Yup, ^ is fair points 19:12:04 <smyers> I love the idea of an ansible verifier, and have done effectively this by using asserts in converge, which is less than ideal but equally effective. So, apples to apples, ansible to testinfra, my opinion is that writing asserts in testinfra is easier than writing them in ansible. 19:12:52 <decentral1se> to be clear, I agree with the idea that we shouldn't have to ask Ansible people to learn Python programming + Pytest 19:13:16 <decentral1se> addressing the "hard+brittle" concern for me would be: what are the best practices and correct modules? 19:13:30 <decentral1se> smyers: you've mentioned assert, is that the best out? I haven't done much verifying with ansible ... 19:13:59 <decentral1se> "best", but well, what are people doing in the wild, I mean 19:14:04 <smyers> Having an ansible verifier means we don't have to ask anyone to learn python, but I think it's reasonable to ask them to do that by default, versus trying to write a test suite in ansible 19:15:06 <smyers> And yeah, I can only speak to my own experience, of course, but I use the ansible assert module to make assertions in ansible for parity with the python assert statement 19:15:14 <decentral1se> ah ok 19:15:18 <smyers> hopefully that makes sense and doesn't sound too silly :) 19:15:31 <decentral1se> I mean, I can imagine it being quite easy to do state:present for files/directories/packages etc. 19:15:49 <smyers> The thing about testing roles is that my team has a philosophy of "you don't ahve to test ansible" 19:16:36 <smyers> So in some cases, native ansible asserts are the onlyy way to ensure that, given a certain set of input vars to the roles, tasks that should be called were called, tasks that shouldn't be called were not called, facts were set appropriately, etc. 19:16:39 <gundalow> Example integration test for the `stat` module using `assert` https://github.com/ansible/ansible/blob/devel/test/integration/targets/stat/tasks/main.yml 19:17:05 <smyers> To test against those things in python would require dumping out a ton of stuff just to parse it in python and run the asserts there, so native ansible asserts win out 19:17:23 <decentral1se> yeah nice examples 19:17:37 <decentral1se> I think we just need a shedload of documentation and examples for this to be an OK transition 19:17:43 <decentral1se> from my POV 19:18:14 <tima> The transition should be part of a 3.0 release. 19:18:35 <decentral1se> Good idea. Are there any other major concerns? 19:18:39 <smyers> I do think it's fine either way, but (and this is perhaps off topic) I think it would be a good idea to cut a release now and consider...what tima just said with fewer words. :) 19:18:45 <gundalow> Yup, also we've not done a release with this in yet, so would be good to "field test it" 19:18:50 <gundalow> smyers: hifive 19:19:33 <tima> I get the arguments for testinfra and using python but Ansible has always prided itself on no programming skills necessary 19:19:50 <tima> And being automation for everybody. 19:21:15 <tima> As the official tool for developing roles, playbooks and eventually collections requiring python to test them... 19:21:18 <smyers> That's a great point, I'm sold on switching it to be the default verifier. My only worry is if changing the default can in any way be a breaking change for existing users. I don't personally think so 19:21:21 <decentral1se> Maybe having this as a default will push the standard on how easy it is to use Ansible as a verifier 19:21:32 <decentral1se> when people are using it more. Maybe the standard is already high though ;) 19:21:32 <tima> Yeah that’s inconsistent. 19:22:03 <decentral1se> oh right, migrating from old versions ... 19:22:12 <tima> I think Ansible could make verifying think easier and more frictionless. 19:22:38 <tima> Verifying things** 19:22:46 <decentral1se> wait, the "upgrade guide" will just be: change your configuration to use testinfra if you already do testinfra 19:22:50 <tima> Docs will help. 19:22:52 <smyers> It would also be great to be able to use multiple verifiers, but that sounds like its own proposal 19:23:07 <decentral1se> we'll have to be careful how we upgrade the molecule definition to not break the configs of existing users 19:23:23 <tima> Better modules for testing even better. 19:23:48 <tima> @decentral1se: +1 19:24:03 <smyers> Yeah, I just double-checked the schema. Assuming I'm reading this correctly, there is no default verifier in the schema, it must be explicitly set. That's how pretty much every section works, so I think the impact to existing molecule users is zero. Existing scenarios will use their configured verifier. 19:24:31 <tima> Hence my 3.0 timing suggestion. 19:24:31 <decentral1se> should we just allow multiple verifiers in this work? That'd be great 19:24:50 <gundalow> decentral1se: interesting idea 19:25:00 <gundalow> what would the switch for that look like? 19:25:32 <smyers> That sounds like a larger effort that should be its own separate thing. I just opened up a tab to start writing up a multiple verifiers proposal :) 19:25:44 <decentral1se> lovely 19:25:54 <decentral1se> yeah, I think it would not be seamless now that I look at the molecule.yml config 19:26:11 <decentral1se> we have verifier: name: testinfra - which is not a list for anything multiple 19:26:30 <decentral1se> but yeah, I think you're right, switching to new default may have zero impact on existing configs 19:26:50 <smyers> I'm thinking we can do some trickery there. If verifier is a dict, there's one verifier. If it's an array/list, there's one or more 19:27:26 <decentral1se> cool, I think we've got a good idea of how to move this on 19:27:28 <smyers> I want to poke at the schema validation a little bit to see if that idea can fly, and maybe come up with some examples of how that won't end up being a nightmare for users 19:27:43 <smyers> Another option is to require either verifier or verifier*s* at the top level 19:28:03 <smyers> But yeah, the wheels are turning :) 19:28:04 <decentral1se> smyers: molecule uses http://docs.python-cerberus.org/en/stable/ for schema validation. It ain't too pretty 19:28:43 <smyers> I'm unfortunately familiar with cerberus, I think it can be coerced to do some of what I'm thinking of, but it will indeed not be pretty 19:29:19 <smyers> Back to the agenda, though, it sounds like there's consensus regarding making the ansible verifier the default verifier 19:29:23 <decentral1se> I've worked with it a bit. Ping me if you want to discuss the hacks :) 19:29:29 <decentral1se> yep 19:29:29 <smyers> rgr 19:29:48 <tima> My only concern is timing 19:29:51 <smyers> I appreciate the heads-up and the offer to help 19:30:03 <tima> And how that transition happens 19:30:55 <gundalow> 1) What exact docs are needed 19:30:55 <gundalow> 2) Do we want to have a release without default to get feedback? 19:31:53 <tima> On 2: +1 19:32:55 <decentral1se> 1) examples on typical use cases? Are services up/packages exists/files,dirs perms check/asserts/which modules are recommended 19:33:34 <decentral1se> 2) sounds fine 19:35:10 <tima> Sorry. Need to drop early today. 19:35:18 * tima waves bye 19:35:52 <smyers> For 2, yes. For 1, probably examples on how to assert and what to assert in an ansible verifier. Having a separate verifier outside of the converge playbook is limiting, since you won't have access to facts registered in converge, so it might be good to document ways to get around that problem? Mostly just thinking out loud here, though 19:35:53 <decentral1se> all good, take it easy 19:36:46 <decentral1se> smyers: I think fabianvf an implementation level question regarding whether we should do that or not 19:36:59 <decentral1se> that sounds like a thing we'd definitely want now reading it ... 19:37:10 <gundalow> tima: Thanks as always for your time 19:37:34 <decentral1se> but anyway, we're missing that in testinfra now too (you have to load vars manually) 19:37:45 <fabianvf> yeah that's what I was going to say 19:38:03 <fabianvf> I don't think it's bad to have your test state separate from the thing you're testing 19:38:09 <decentral1se> %s/an/had an/ 19:38:22 <decentral1se> that's a good point too 19:38:27 <smyers> Yeah, having an ansible verifier doesn't mean that I'm going to move my asserts out of converge. It should, but it doesn't. 19:38:34 <decentral1se> depends on the implementors, I suppose 19:39:01 <decentral1se> gundalow: does that shed light on needed docs? 19:40:03 <gundalow> Yup, great thanks 19:40:07 <smyers> I gnerally never test that the tasks in a role worked; that's a job for ansible's test suite. I want to test that my when clauses and includes made the right choices, and there's no way that I know of to test that without having access to facts at the end of converge, either by asserting right there or outputting them somewhere and asserting against them elsewhere. 19:42:11 <decentral1se> ack, yeah, there will be a lot of manual duplication if you can't seamlessly access the facts 19:42:40 <decentral1se> we should raise a ticket against this to think through it? Maybe we can leave it to the implementor too (not us) 19:43:04 <decentral1se> by passing a fact dict through to the verifier ... 19:43:15 <gundalow> I've added documentation requirements to https://github.com/ansible/molecule/issues/2013#issuecomment-488392662 19:43:17 <fabianvf> At least having a way to pass facts through would be nice for sure 19:43:20 <smyers> Yeah, I think it's worth touching on in the docs and maybe coming up with an enhancement that allows you to pass facts through 19:43:25 <gundalow> jopefully I've captured that correctly 19:43:29 <gundalow> #chair fabianvf 19:43:29 <zodbot> Current chairs: alongchamps decentral1se fabianvf gundalow smyers tima 19:43:32 <smyers> For good separation, it's probably not great to pass facts by default 19:43:37 <fabianvf> then it's explicit what goes through 19:43:39 <smyers> So we're forward-compatible already :) 19:44:04 <decentral1se> oh yes, nice ideas 19:44:07 <decentral1se> ticket looks good 19:44:23 <gundalow> decentral1se: of course they are good, I stole what y'all said in here :) 19:45:52 <decentral1se> hehe 19:46:45 <decentral1se> moving on? 19:46:49 <gundalow> Yup 19:47:10 <gundalow> #topic Add a CLI command for collections initialization ansible/molecule#2014 19:47:19 <gundalow> #info https://github.com/ansible/molecule/pull/2014 19:49:29 <decentral1se> so, this is still in draft and I haven't had time to look at it 19:49:40 <decentral1se> very happy to see it coming though 19:49:50 <decentral1se> anyone have any points on this? 19:50:45 <smyers> My only thought for the collections stuff is weighing the balance between having a "collections" branch and bringing collections features to master all at once, and bringing in pieces of collections piece by piece that basically won't be useful until all collections features are done 19:51:58 <decentral1se> I was hoping it would be a case of a typical molecule install, latest or whatever 19:51:59 <fabianvf> This may be slightly off topic, but I did have a question related to initialization - do we ever plan on having code templates for modules/plugins/etc? Presumably this will be a much more common pattern with collections and being able to template out a plugin or module would be really nice (it can be a bit tough to remember exactly what structure the particular thing you are making needs to follow currently 19:52:06 <decentral1se> and then a separate CLI command `init collection` or whatever 19:52:12 <smyers> There's no clear winner for me, at the moment I'd arbitrarily prefer to keep collections features off of master until you can actually aim a `molecule test` sequence at them 19:52:18 <decentral1se> and then just installing ansible from collections branch source 19:53:08 <decentral1se> fabianvf: AFAIK, you can override the entire role template (and not have to override the molecule template) 19:53:38 <smyers> Templates is a great idea. I'm assuming we all just go steal from an existing ansible plugin, gut it, and work from there. If molecule is to be the official authoring tool, I think it makes sense to have the official templates for things like plugins be part of molecule in some way. 19:55:24 <gundalow> decentral1se: thanks for adding the other doc link 19:55:29 <decentral1se> ok, updated the ticket to ask about how this will go into circulation 19:55:32 <gundalow> (to the proposal) 19:55:35 <decentral1se> gundalow: sure thing 19:57:08 <decentral1se> ok, tried to capture concerns in: https://github.com/ansible/molecule/pull/2014#issuecomment-488396666 19:58:16 <fabianvf> smyers: +1, it might also be worth it to include it as part of the collections work, since AFAIU a large part of the value add is packaging/distributing plugins 19:58:24 <smyers> I think that touches on it well. I'm eager to see/help the collections stuff become real, but am nervous about jumping in without have a better sense of the broader plan 19:58:40 <fabianvf> being able to init a plugin/module into the proper place in a collection would be really nice IMO 19:58:55 <smyers> yeah 19:59:05 <decentral1se> wanna open a ticket? 19:59:36 <smyers> things are getting crazy there from an interface perspective, e.g. molecule init collection plugin filter filterpluginfooname, but...that's a potential problem to solve later. :) 19:59:58 <fabianvf> heh, true, but it's even more overwhelming without the list :P 20:00:04 <smyers> yeah 20:00:08 * alongchamps time to head out 20:00:13 <fabianvf> decentral1se: I'd be happy to 20:01:05 <smyers> I'm hacking on the multiple verifiers prop right now, then there's the plugin template thing, and we probably want a ticket about passing facts from converge into an ansible verifier 20:01:15 <smyers> don't think anyone volunteered to write up that last one :) 20:01:15 <decentral1se> thanks! 20:01:33 * smyers backs away slowly, but not *too* slowly 20:01:42 <decentral1se> go go go go go go go :) 20:03:08 <decentral1se> OK, I have one point about the hetzner stuff 20:03:30 <decentral1se> #topic hetzner cloud and CLI 20:03:32 <decentral1se> if I may ... 20:03:48 <decentral1se> basically, I am in touch with hetzner's new ansible hcloud_server module implementor 20:04:10 <decentral1se> and there is some interest in getting support to run the CI properly to validate the new driver implementation 20:04:44 <decentral1se> there was some fair concerns about having another untested driver added to core 20:04:51 <decentral1se> so hopefully this will help address that, that's it! 20:05:03 <gundalow> decentral1se: nice 20:08:42 <smyers> I think the last agenda thing is testing with testinfra + the ansible runner. We've also got an open ticket to make sure testinfra 2 works (and it should...). I can get those tested today or tomorrow if nobody beats me to it. 20:09:11 <gundalow> #topic Test Infra update philpep/testinfra#432 20:09:17 <gundalow> smyers: yup, thanks 20:09:26 <gundalow> #info https://github.com/philpep/testinfra/pull/432 20:09:49 <decentral1se> right, how do we test this? Just upgrade our testinfra installation to this pr branch? 20:10:28 <smyers> That's my initial plan, plus digging into seeing what the branch actually changes and if there's a way to exercise it before and after from molecule 20:10:55 <smyers> My sense of it right now is that if it works, it works, we just need to make sure it works. :) 20:11:59 <decentral1se> cool, I will try to get around to it. Will be great to have this shifting API replaced with the less shifting ansible-runner API :) 20:12:47 <gundalow> :) 20:12:59 <decentral1se> hoping we'll also get a solution for https://github.com/philpep/testinfra/issues/345 from this 20:13:27 <decentral1se> (passing facts through pytest fixtures) 20:14:27 <gundalow> hopefully 20:14:38 <gundalow> bit over time, anything else? 20:14:46 <decentral1se> nothing from me 20:14:56 <gundalow> next week will be my last meeting for a while 20:14:57 <gundalow> cool 20:15:00 <gundalow> Thanks everybody 20:15:02 <gundalow> #endmeeting