10:50:33 #startmeeting Ansible Contributor Summit 2020 10:50:33 Meeting started Sun Mar 29 10:50:33 2020 UTC. 10:50:33 This meeting is logged and archived in a public location. 10:50:33 The chair is gundalow. Information about MeetBot at http://wiki.debian.org/MeetBot. 10:50:33 Useful Commands: #action #agreed #halp #info #idea #link #topic. 10:50:33 The meeting name has been set to 'ansible_contributor_summit_2020' 10:50:42 #chair cybette gwmngilfen phips rbergeron gregdek 10:50:42 Current chairs: cybette gregdek gundalow gwmngilfen phips rbergeron 10:52:12 #link Join via Bluejeans https://bluejeans.com/228187643 10:56:58 morning all 10:57:08 morning! 10:57:34 * jhawkesworth lurking/making lunch but trying to lend half an ear to things 10:58:21 o/ 10:58:27 o/ 10:59:39 just a little tip for the ones in the video conf, space bar enables microphone, kind of push to talk. very handy ;) 11:00:17 is it toggling it or is it PTT-Like 11:00:19 I have been using Bluejeans for, like, 5 years now, and I never knew that. 11:00:33 push to talk style 11:02:56 I'm a bit concerned about bluejeans, though 11:03:03 been behaving odd for a couple of days 11:03:06 * dmellado crosses fingers 11:03:30 gregdek: Error: Can't start another meeting, one is in progress. 11:03:36 LOL, ok! 11:04:31 :D 11:05:09 congrats :D 11:06:16 #topic Welcome and intros 11:07:31 * gundalow #info To 11:07:55 #info Welcome to the 8th Ansible Contributors Summit 11:08:11 if you are not on bluejeans or prefer not to speak, feel free to do a short intro here and I can relay it to bluejeans 11:08:44 #info We know the world has become a bit crazy recently, we really appreciate you taking the time to join us today, which is a real testament to the Ansible Community 11:09:58 #info We are now going round the virtual room doing introductions and 1) saying who we are Real & GitHub/IRC names 2) What our interest in Ansible is 3) What we'd like to get from today 11:11:38 when I click on 'speaker view' in bj I see gregdek eating; is that the way it's supposed to work? ;-) 11:11:57 I guess so! 11:12:13 yeah, that's all this is about :D 11:12:19 jpmens: it's a laptop warmer :( 11:12:25 Also hello, glad you could make it :) 11:12:25 lol xD 11:12:34 there are worse things :D 11:12:52 greg 2 :) 11:13:53 it's only confusing for everyone *except* me & gregdek :P 11:18:10 hehe :) 11:27:44 brits who try pronouncing foreign names; LoL ;-D 11:28:06 jpmens: well, I can't tab complete. It's reaaally difficult 11:28:11 Hey, could be worse, could be Americans pronouncing foreign names ;) 11:28:41 you have a point, but I think it's six of one and half a dozen of the other ;) 11:28:57 yup :) 11:29:31 gundalow: that's an idea for #getrichfast 11:31:19 collections, yay! \o/ 11:31:32 what's that? ;) 11:31:44 #link Contributor summit attendance + info for swag: https://forms.gle/Uv9Zdaz7aTAAwtsQ8 11:31:49 something which keeps us working for quite some time :D 11:33:23 does jpmens win for having first kid/family member wander into frame to ask a question? :) 11:33:27 aka "job security" (or "this was such a tire fire you'll be fired at the end", not sure yet) ;) 11:33:47 ah, damn; I asked for a coffee 11:34:05 (or sometimes illogical) 11:34:32 jpmens: heh, I had the kid around (but I disabled the camera) 11:34:50 WFH it's proven challenging on COVID19 times 11:34:56 Yup. 11:34:59 indeed 11:35:10 yep 11:35:22 #topic Collection Overview 11:35:31 I'm going on 7 years of WFH, my kiddos have jumped into many conference calls :) 11:35:42 fun to see them growing up :) 11:36:05 hehe :) 11:36:40 #info Ansible was a single repo, but Ansible module growth has made a single repo less sustainable over time 11:37:17 #info Collections What/why/how overview https://github.com/ansible-collections/overview/blob/master/README.rst#where-we-ve-come-from-and-where-we-are-going 11:37:32 decoupling (aka isolation) as each part can move at its own speed 11:37:56 #info Collections user guide: https://docs.ansible.com/ansible/latest/user_guide/collections_using.html 11:38:09 #info Collections development guide: https://docs.ansible.com/ansible/latest/dev_guide/developing_collections.html 11:38:37 #info collections allow developers to move at their own speed, but users just want a simple way to consume content without worrying about collection release cycles 11:40:10 #info with collections, there is no longer a single global namespace for modules, now each collection has its own namespace 11:41:06 #info Fully Qualified Collection Name (FQCN) = The full definition of a module, plugin, or role hosted within a collection, in the form namespace.collection.content_name. For example `ansible.windows.win_command` 11:41:32 #info See https://github.com/ansible-collections/overview/blob/master/README.rst#terminology for a full list of Collection Terminology https://github.com/ansible-collections/overview/blob/master/README.rst#terminology 11:44:41 #info `ansible/ansible:devel` is now `ansible-base` (ie without most modules) 11:47:06 #info With collections, we will introduce the idea of "execution environments", which are container-based virtual environments using ansible-base + ansible-runner + particular collection requirements, and that's what awx/tower will use to run content in the future 11:47:27 #info Execution environments allow for different environments for different users/use cases 11:47:51 #info Docs on the existing Ansible Tower and collections (not including "execution Environments") https://docs.ansible.com/ansible-tower/latest/html/userguide/projects.html#collections-support 11:49:04 #info q: with having namespaces, module names might not be unique different collection might have the same name. In a play we could define all namespaces, what would haben if you add 2 namespaces with a module named identically? 11:50:05 it was originally intended to allow users to easily override core functionality 11:50:22 ie. you don't like what our file module does you can easily drop one in to do what you want 11:50:30 #info module and plugins operate a first found. You can always use the FQCN (Fully Qualified Collection Name) ie rather than just using `ping` you could use `cisco.ios.ping` or `ansible.windows.ping` 11:50:31 'easily' 11:50:31 but yes it became very confusing 11:50:35 (^ made up example) 11:50:42 jpmens: for some value of easy :) 11:51:26 I don't think many people wanted to override the ec2 module for instance 11:52:50 Adding questions here: https://etherpad.openstack.org/p/ansible-contributor-summit-gothenburg-2020-general 11:53:34 #topic Molecule 11:54:39 #info zbr (The Molecule Maintainer) is doing a short presentation on Molecule 11:54:40 * jtanner wakes up 11:54:47 jtanner: mornin' 11:54:51 morning 11:54:56 #link https://sbarnea.com/slides/molecule/#/ 11:54:57 @jtanner: morning 11:55:22 #info Molecule project is designed to aid in the development and testing of Ansible roles. Molecule provides support for testing with multiple instances, operating systems and distributions, virtualization providers, test frameworks and testing scenarios. 11:55:34 #info Molecule documentation https://molecule.readthedocs.io/en/latest/ 11:56:22 #info Slides which zbr is going through https://sbarnea.com/slides/molecule/#/ 11:56:41 #info If you are interested in Molecule feel free to join #ansible-molecule on IRC 11:58:56 #info Molecule has many drivers they can be found at https://github.com/ansible-community?q=molec&type=&language= 11:59:22 #info We are looking for people to help with Molecule, and especially the drivers (above URL) 12:02:51 Any questions on Molecule? 12:09:34 how can we use ansible vars or env vars in molecule.yml ? 12:09:45 I have been working with Molecule and I found the complete environment a bit fragile. If not all versions matched it broke. Is there any improvement on that? 12:10:46 I was about to mention your question TKersten 12:11:07 for ex molecule version 2 and mol 3 is diff 12:11:14 so lots of breaking changes 12:12:27 zbr: What is the policy of adding new drivers from outside of molecule repo for example? If I wrote some cloud driver "foo". It's worth to have documentation about it <- From Sagi in chat 12:12:37 OK, Thanks. I'll surely look into version 3 in the very near future 12:12:58 Q: What is the policy of adding and using new drivers from outside of molecule repo for example? If I wrote some cloud driver "foo", what are my actions to use it as a driver? It's worth to have documentation about it 12:12:59 We are using molecule for all of our integration tests for about 6 months now and we had almost no problems whatsoever. 12:13:56 are we working on improving the molecule and its driver's documentation with examples ? 12:13:57 question: will molecule the "default" how to test your ansible content (deprecates ansible-test?) cc mattclay ? 12:14:35 Q: Geerlingguy blog post aside, is there any plan to integrate molecule in ansible-test ? 12:14:55 ha nice resmo 12:15:03 resmo: Molecule cannot run unit and sanity tests, so replacing ansible-test is not possible. 12:15:40 ansible-test is mostly a "linter" test, but it can be included in molecule testing 12:16:16 or maybe adjusted/tweaked for being used in molecule scenario as a one of "linter"s 12:16:16 #info Q: Can anyone create drivers A: Yes, only needs the Python entry point. If you think the driver maybe useful for others please ask in #ansible-molecule and we can host it (and provide CI) along side the existing drivers in https://github.com/ansible-community?q=molec&type=&language= 12:16:44 does molecule also do change detection, as ansible-test can do? 12:16:53 tadeboro: ok, right. what about "ansible-test integration" 12:17:09 sshnaidm: Linting is only one part of what ansible-test does (sanity tests). It also runs unit tests and integration tests. 12:17:44 resmo: ansible-test and Molecule cover different use cases. I don't see either one replacing the other. 12:17:54 I see the primary role of ansible-test as the place where we define the rules of what gets accepted into Ansible and what doesn't. 12:18:27 I was wondering if molecule could be called by ansible-test to have a single test command. 12:18:32 mattclay, sorry, missed your reply, but can ansible-test be as one of sub-tests in molecule scenario for example 12:19:34 The Sensu Go Ansible Collection uses ansible-test to run the sanity and unit tests, and we use Molecule for the integration tests. 12:19:35 ........ 12:20:08 I think we have several questions around ansible-test as well :) 12:20:11 thanks for demo @zbr 12:20:14 Thanks for the talk @zbr 12:20:52 gregdek: not just what goes into ansible or not, ansible-test is what I'd use to test the collection's modules in general 12:21:03 Thanks zbr :) 12:21:36 thanks 12:21:50 shaps: yes, speaking broadly, every collection that goes into "Ansible" should be using ansible-test to gate all modules 12:21:55 #topic Community Data 12:22:00 Thanks zbr! 12:22:06 .=.=.=..=.=.=..=.=.=..=.=.=..=.=.=..=.=.=..=.=.=..=.=.=. 12:22:08 .=.=.=..=.=.=..=.=.=..=.=.=..=.=.=..=.=.=. 12:22:09 .=.=.=..=.=.=..=.=.=..=.=.=. 12:22:58 +1000 gwmngilfen for using i3 :D 12:23:07 #info gwmngilfen is now presenting how the Community Team uses data to make decisions 12:24:56 DING DING DING Reminder: If we didn't answer any questions please add them to https://etherpad.openstack.org/p/ansible-contributor-summit-gothenburg-2020-general and we will go though them later on 12:25:12 ^ Particularly important for questions around Collections 12:25:36 is link to the committers map metrics available to general public? 12:25:49 All of this data should be public. 12:25:59 * jimi|ansible -> more coffee, brb 12:26:14 Is this url accessible? https://stats.eng.ansible.com/apps/ 12:26:21 digigrate_: ^^ 12:26:31 https://stats.eng.ansible.com/apps/meetups/ then click on `meetup maps` 12:26:32 gregdek: yeah 12:26:39 #link https://stats.eng.ansible.com/apps/ 12:26:52 gregdek: yes! 12:26:57 All, have a look at the graphs that are being shared 12:27:33 rrey: This (grafana #1 for number of contributors) is your great work :) 12:27:45 \0/ 12:29:28 gundalow: question for Greg Sutcliffe - is he only keeping track of collections under ansible-collections org or also other vendor collections that moved to their own Github organizations? 12:30:09 what counts as a contributor? a raised PR or a merged PR? 12:30:14 ikhan: we can graph anything which we can access via Git (doesn't have to be GitHub). If you have others you'd like to add just ping gwmngilfen a list of URLs 12:30:39 zbr: Merges (in the future we may look at issues and PRs) 12:30:57 I know for sure that lots of people refused to cotribute because they said: "why to fix this when i will be able to make use of it only in 1y+ from now, mainly because of the backporting policy. 12:30:58 gundalow: ++ thanks! 12:31:16 Yeah, that's interesting, I do wonder about the metrics for contributions, would only MERGE then be counted? do you count also open PR ? 12:31:29 zbr: Yup. One of the nice things about Collections is they can be released much more frequently 12:31:53 zbr: in my opinion, for *success* we should be measuring merged PRs -- but we could very well consider "non-merged" PRs as a different critical metric -- who's leaving a lot of PRs on the doorstep? 12:32:17 and possibly why are those PRs being left there 12:32:19 gundalow: yep, this should motivate people to contribute, bad part is that it will take at least one year to really see the benefits, as they first need to upgrade to 2.10 to be able to benefit from patches on collections. 12:32:19 A good thing I can toss on gwmngilfen's todo list :) 12:33:11 zbr: it's always a process - when we did 2.0 there were a lot of people on 1.x for quite a while 12:33:21 it would help identify struggling maintainers too 12:33:22 I'd also look at this -been there for a logn time- as a good sample on stats from an open source project 12:33:25 https://www.stackalytics.com/ 12:33:27 danieldm For the moment we are only looking at merges. Once the repos are fully open (ie we allow easier moving of PRs from ansible/ansible to the individual repos) we will start tracking Issue and PRs backlog. Growing backlog is going to be a concern for (say) https://github.com/ansible-collections/community.general/pulls 12:33:56 o/ 12:33:59 gundalow: gotcha, makes sense! 12:34:04 Hi acozine! 12:34:15 zbr: ah, I think I miss answered your question. People could use ansible-2.9 and the collections should work 12:34:17 good morning, folks! 12:34:23 acozine: Hi :) 12:34:46 Reminder we are using https://etherpad.openstack.org/p/ansible-contributor-summit-gothenburg-2020-general for questions for later talks 12:35:09 2.9.0 should fundamentally work with collections, but that does not mean all collections+plugins are going to work with 2.9 as their import paths are likely to diverge greatly 12:35:13 #action gundalow to add community.network to Collection stats 12:35:34 #action ikhan to privide gwmngilfen with lists of other repos to add to https://stats.eng.ansible.com/apps/collections/contributors/ 12:35:38 gundalow: hmm, i almost forgot about that. this could be game changer. For sure collections are clearly the right change moving forward. 12:36:00 gundalow: sure will get that info 12:37:01 Hey ikhan, good morning :) 12:37:18 gregdek: good morning! 12:37:25 #chair dmellado ganeshrn ikhan jimi|ansible nitzmahone acozine rrey shaps digigrate_ digigrate madonius belfast77 tadeboro jpmens sshnaidm mattclay resmo Pilou TKersten webknjaz 12:37:25 Current chairs: Pilou TKersten acozine belfast77 cybette digigrate digigrate_ dmellado ganeshrn gregdek gundalow gwmngilfen ikhan jimi|ansible jpmens madonius mattclay nitzmahone phips rbergeron resmo rrey shaps sshnaidm tadeboro webknjaz 12:38:02 ah, I just discovered the livestream 12:38:09 * acozine is still waking up 12:38:15 good morning acozine :) 12:38:22 hi acozine :) 12:38:23 (and everyone else who showed up recently) 12:38:58 hallo felixfontein shaps gregdek gundalow 12:40:10 \0/ 12:40:30 #meetingtopic Ansible Contrib Summit 2020 - main stream @ https://bluejeans.com/228187643 12:40:55 nitzmahone: oh, nice I didn't know that feature 12:41:02 cheers 12:41:07 quick q here regarding the stats, would the plan be to also store data regarding authors/commits? if so, things might've been messed a bit up by the migration to collections 12:41:41 AFAIK we didn't maintain git history on migration move 12:41:49 that's my main concern 12:42:13 just noticed it lately when working on some collections and seeign gblame as 12:42:40 yeah it kind of mess with code ownership :/ 12:42:44 it is extreamly hard to maintain history on such moves, when merging is easier but slitting is not practically possible 12:42:46 dmellado, yeah, for a real blame you'll need to look at previous version in github 12:42:56 ah, `git blame` (finding who to ask about something) can be an issues, mean you'd need to do a git blame on the original (now deleted) files in ansible/ansible 12:43:30 CaptTrews yeah xD 12:43:36 I tried to keep it on a keycloak collection, but it also mean that we keep the thousand contributors from ansible/ansible 12:43:42 so you'd have to go back there and so 12:43:49 in openstack I wanted to preserve a history, but it was too much overhead 12:43:52 there's the tag 'pre-ansible-base' in ansible/ansible which is the last commit when everything was still there 12:44:03 will be useful for running `git blame` against :) 12:44:06 so I know it's a huge change, and that that might've been too much complex to handle 12:44:13 as i said in the past, if you endup using `git blame`, is a sign that code was not properly documented. git commit is not the place to document changes. 12:44:14 felixfontein ++ 12:44:15 felixfontein: one has to actually know that, right? 12:44:27 but I wonder if there'd been some way to keep git history over there 12:44:32 madonius: so far, yes, I don't think it is documented yet 12:44:38 #info git tag `pre-ansible-base` in ansible/ansible is the commit before most of the modules got `git rm`'ed 12:44:40 as it might also mess data/stats quite a bit 12:45:12 zbr: you are correct, but reality and theory do not diverge, in theory… :/ 12:45:29 true 12:45:49 Q: to have this metrics relevant you'll need to be aware of collections created right ? 12:46:11 how can one opt in for the metrics ? 12:46:32 rrey: could you expand on that? 12:46:51 is this only for collections under github/ansible-collections? 12:47:07 rrey: or maybe jump on the call and ask 12:47:11 (collection in galaxy might have a private git repo right?) 12:47:15 sure. If I start a collection it won't be in the radar if it is outside ansible-collections 12:47:32 gundalow, I think it's question about including metrics of collection outside of ansible 12:47:45 yep 12:47:45 ah, OK 12:47:50 how do you plan to scale this to the n repos on collections? 12:48:08 gundalow, like openstack modules :) 12:48:13 felixfontein: madonius if either of you would be willing to author a paragraph about "how to git blame on pre-collection changes to a module", I'm happy to merge that into https://docs.ansible.com/ansible/devel/dev_guide/developing_collections.html 12:48:18 At some point, we'll have policies for having new collections included in the Ansible distribution. When we start doing that, pulling in stats will be one of the activities that happens there. 12:50:59 madonius: if you want to write that, feel free :) 12:53:15 I am not quite sure where I would put that 12:53:44 or if "developing collections" would be the right place… :/ 12:54:10 that's the closest we have so far . . . at least it's aimed at developers working on collections 12:54:34 ideally it would be something along the lines of "if you are reading this then you want to go there" 12:54:36 afk 12:54:55 we'll need to expand content about collections on the developer side 12:54:58 Returning in 15 minutes 12:55:12 dmellado: if I can scale to crawling ansible/ansible, i can handle the collections. at least for now :) 12:55:30 #topic 15m break, back @ 14:10 UTC 12:55:39 for comparison, crawling the entire ansible-collections org currently takes ~20mins, crawling ansible/ansible takes ~17 hours :P 12:55:43 Reminder you can add questions to https://etherpad.openstack.org/p/ansible-contributor-summit-gothenburg-2020-general 12:56:14 * nitzmahone suspects `gwmngilfen` is Welsh with that collection of consonants? 12:56:57 nitzmahone: good guess. I'm not, but there is a connection which is much longer and more convoluted than you could guess :P 12:57:17 Sounds like a good story for San Diego :) 12:57:35 stats.eng.ansible.com/apps/ if people want to see some of my other work, please try not to ddos the server :D 13:00:42 gwmngilfen: thanks for the insights :) 13:00:58 my job^w pleasure :D 13:03:49 #info https://stats.eng.ansible.com/apps/collections/contributors/ is the url for the contributor stuff, if anyone missed it above 13:04:41 gwmngilfen: i presume that's cumulative sums? any intent to track active over date ranges? 13:04:58 something like how many were actually active in a given month / week / etc 13:05:29 right, cumulative unique contributors, broken down by week 13:06:42 jtanner: i do have non-cumulative contributors on the prototype dashboard I shoed, so that will probably stay 13:06:47 iirc that is also weekly 13:06:49 k 13:06:56 gundalow: when we get back from break, might be good to let those who joined later to do intros. e.g. acozine , ikhan etc. 13:07:39 what i'm most interested in is showing whether or not a collection has active maintenance or if it's just rotting, despite how many people may have contributed historically 13:09:08 jtanner it could be a nice indicator on galaxy that could help choosing between 2 13:10:14 jtanner: thats fair, for sure. the cumulative stuff only really applies during the migration phase, I think, as a way to see how we're doing regaining momentum. after that, it's back to standard line graphs. 13:10:29 @jtanner it also depends if the code is relatively stable in collection and there is not a whole lot of request/need of new features - you may not see a whole lot of activity on that collection 13:10:38 gundalow: time's up! 13:10:51 yeah, it's nice to see cumulative either way. not trying to diminish it's usefulness 13:11:35 ikhan: true ... but how do we know it's still being used or people have given up because it's always broken 13:12:02 thats a hard problem, for sure 13:12:23 #topic Intros part 2 13:12:27 you cant normalise by issues/PRs, they're correlated for sure 13:12:47 #info Just doing another round of intros for people that have joined the call since we started earlier 13:13:38 acozine: ‘word weaver’ for a living! 13:13:54 jtanner: seems like the "time to response by maintainers" might be a good answer to that question. If every issue/PR gets a quick response from its maintainers, we can probably surmise that it's "well maintained". 13:14:19 thats true. also many issues that dont have attached PRs, perhaps? 13:14:24 especially open ones 13:14:30 and the converse: if they never respond, what can we assume? 13:14:56 I think that's a pretty clear case: no response + no closure = no maintainership 13:15:38 acozine, prior to changelog storage, could we consider choosing changelog tool for collections? Should be the same? For example in case some collection wants to use a different one, like reno 13:15:41 In the collection repos I expect we will be a lot stricter about closing issues that don't have any comments 13:15:46 indeed, i would like to see a "time to respond from cores" measured/reported 13:16:06 jtanner: that's a good point. I hope someone in community is complaining that something stopped working with those stale collections. It is hard to test every collection especially if those platforms are not available for testing. 13:17:42 jtanner: what does "cores" mean here? the maintainers of a given collection? 13:17:49 sshnaidm: so far we've looked at reno and at towncrier . . . open to other suggestions if folks know about other tools 13:17:51 zbr ^ 13:18:00 oops 13:18:15 same colour in my client for some reason :P 13:18:25 on molecule-* we use release-drafter: mainly using PR title + labels to automate release not creation. 13:18:42 acozine, I suppose it's the topic to discuss? https://github.com/ansible-collections/overview/issues/18 13:19:07 it's offical, ansibot is a sham, jtanner just does it all :P 13:19:09 sshnaidm: yes, thanks for posting that 13:19:16 i'm the spammer 13:19:46 #topic ansible-lint 13:20:08 #info zbr is giving an presentation on ansible-lint 13:20:10 zbr: what does "from the cores" mean here? the maintainers of a given collection? 13:20:21 oh, you're speaking, it can wait :) 13:21:08 sshnaidm: acozine so what ever "ansible" choose for changelog, it only should be a recommendation, every project should be free to choose a tool. So basically this reduces to changelog being just a link to the actual changelog file. 13:22:02 resmo ++ 13:22:59 resmo: we will probably end up with a policy of "you may handle changelogs any way you want, however, if you want changes from your collection included in {{ ACD_changelog_file }}, here's how" 13:23:22 acozine: that would be perfectly fine for me. 13:25:06 #info ansible-lint docs: https://docs.ansible.com/ansible-lint/ 13:25:25 #info ansible-lint repo: https://github.com/ansible/ansible-lint 13:26:15 madonius: we need to overhaul https://docs.ansible.com/ansible/devel/community/maintainers.html#maintainers for the NWO, turning it into a Collections Maintainer Guide - that's another potential spot for the docs on finding pre-collections-migration code changes 13:27:47 Would be great to have auto-ansible-lint that will fix lint issues. Like autopep8 that fixes pep8 issues 13:27:58 1. what is the best pratice to lint a whole directory containing ansible-files that are not a playbook. so far I am doing it like this ansible-lint $(find . -type f -name "*.yml" -o -name "*.yaml") is there any better way? ansible-lint is not working recursively afaik? 13:27:58 2. how would I make ansible-lint know about a collection I installed? 13:28:49 sshnaidm: https://github.com/adrienverge/yamllint/issues/62 13:29:58 sshnaidm: pep8 is all code formatting, where not all ansible-lint issues are. you can't easily automatically fix an unnecessary use of command/shell 13:30:09 jtanner, ack, but this is for YAML issues only 13:30:44 agaffney, yeah, like autopep can't fix all issues as well, only easily solvable 13:31:15 for example ansible-lint rule can contain "fixme" code section, which can be applied in autofix 13:31:24 and modifying YAML in-place while preserving formatting and comments is...tricky 13:31:27 @sshnaidm: linters cannot know what the intention of the user was. "autofixing" things would bring more harm 13:31:45 indeed 13:32:07 yes 13:32:09 what is the best pratice to lint a whole directory containing ansible-files that are not a playbook. so far I am doing it like this ansible-lint $(find . -type f -name ".yml" -o -name ".yaml") is there any better way? ansible-lint is not working recursively afaik? 13:32:10 how would I make ansible-lint know about a collection I installed? 13:32:57 iirc, ansible-lint is only scoped to playbooks/task files ... ansible-review attempted to look at other things like vars 13:34:08 cool, thanks 13:34:38 do you have to exclude ansible vault files or vault inline ? 13:34:41 how would I make ansible-lint know about a collection I installed? 13:34:42 jtanner: yeah, I think that was the purpose of ansible-review, but not sure what the state of it is 13:35:07 curious what zbr's thoughts are on ansble-review's future 13:35:30 cool, didn't hear about ansible-review before.. 13:35:31 ok thanks 13:35:43 iirc, awcrosby had been researching/chatting about it with willthames a while back and there was some idea that maybe it could be ported back to ansible-lint 13:35:57 seems like it stopped in 2019 13:39:58 thanks (again) zbr :) 13:40:22 zbr++ 13:41:12 loud enough 13:41:28 #topic Collections: User Experience 13:41:52 #info nitzmahone is now explaining about how we ended up with `ansible-base` 13:42:14 Questions were around how small could we make ansible-base (gh/ansible/ansible) 13:42:59 should I be seeing slides on bj? 13:43:03 We did a prototype of ansible with no modules, there was no way that this could be useful 13:43:07 jpmens: no slides 13:43:11 ta 13:43:17 ah, let me screen share some docs 13:44:15 #info Definition of ansible-base https://github.com/ansible-collections/overview/blob/master/README.rst#q-what-exactly-is-ansible-base-for-and-what-does-it-contain 13:44:21 Is my screen readable? 13:44:28 gundalow: yes 13:44:31 fine here 13:44:32 mostly 13:44:56 maybe bump it one more zoom level to be sure 13:44:56 I'm sharing above URL 13:49:20 first bit o' good news: ansible-doc -l becomes fast again! ;-) 13:49:46 (with 69 modules only) 13:50:00 disclaimer: I invented it :) 13:50:37 At the moment, I think that distributions and other system packagers will package ansible-base into one package. They'll package the same collections as ACD includes into other packages. Then they'll create an ansible package which depends on all of those. Thus when a user installs ansible (example: yum install ansible ) it will pull in the ansible-base and collections packages. 13:51:13 ansible-devel broke molecule CI last week but it was easy to fix it by adding an extra line to install the removed docker modules. 13:51:57 community.general or did those move again? 13:52:07 https://github.com/ansible-community/molecule/pull/2620/files 13:52:15 jtanner: community.general 13:52:27 no plans to move anywhere else AFAIK 13:53:35 tbh, i would think those would be a great candidate for their own collection 13:54:33 https://github.com/ansible-collections/overview/blob/master/README.rst#terminology 13:54:56 my power-tip: setup CI to run on ansible-devel and you will not be hugely surprised on upcoming changes 13:55:01 jtanner: they'd need more maintainers for that 13:55:02 So… batteries included, no more? what will be the new phrase? ‘not so many batteries included’? 13:55:04 Q: Are collections dependencies on user to install? It won't be deps of ACD, right? 13:55:12 felixfontein: that's probably true 13:55:37 jtanner: I'm happy to maintain some of them as long as I use/need them, but there are others which have not so active, or simply no real maintainers 13:56:12 you can have "non-voting" CI on ansible-devel, so you learn about breaking things and have time to *gradually* transition your code 13:56:15 sshnaidm: correct; controller-side collection deps (and related metadata, etc) are part of the execution environment concept 13:56:29 and the docker community is somewhat winding down it seems, like docker-py development has been very inactive over the last 9 months or so 13:56:53 felixfontein: yeah, but that could be another reason to move them out of general ... so they can safely die 13:56:54 nitzmahone, ack, thanks 13:56:55 So just installing the collections doesn't try to address any deps the collection might have 13:56:57 gundalow: why 'ansible-base' vs 'ansible-core' - there have been references to 'core' for a long time afaik 13:57:00 Let's be clear, though: "pip install ansible" will install all of the collections that have been broken out. ACD is a packaging of all collections currently in Ansible 2.9 that should look identical to "Classic Ansible" -- and ACD is really just "Ansible" to most users. 13:57:04 jtanner: true :) 13:57:19 jtanner: though there are quite some better candidates for *that* in community.general ;) 13:57:27 felixfontein: definitely 13:57:42 i think you know as well if not better than most 13:58:00 odyssey4me: good question. Mostly because `core` (ie ansible core) is an overloaded term 13:59:08 gundalow: it is rather overloaded - fair enough 14:00:26 jpmens: and additionally to your comment, many customers want to test in their staging environment, so the testing gets more complex, because they have to test new ansible with existing verison collection and new collection releases with existing ansible. seems work does not go away ;) 14:00:27 offline install needs to be documented in the same page where we document galaxy install collection, thing should be enough. 14:02:13 Warning note...... *Just for testing* You'll want to remove these packages after installing them. 14:02:27 python3 -m pip install -i https://toshio.fedorapeople.org/ansible/acd/ ansible 14:02:27 #actions Docs need updating to make it clearer how Collections can be released offline (without access to galaxy.ansible.com) 14:02:52 (I just put those there.... I'm testing that right now too) 14:03:06 it might be helpful to have a "if i used to x to install ansible and do y with it, now i should do z" doc 14:03:07 question: I assume we get a few RCs before final release? 14:03:16 resmo: oh yes :) 14:03:25 Q: But how do I get the complete set, batteries included? 14:03:25 resmo: nah, do it live ;) 14:03:37 TKersten: what are the "batteries" to you? 14:03:43 Q: can I set a git url in requirements.yml? 14:03:43 TKersten: `pip install ansible` - just like before 14:04:00 Q: do I need to specify also collections: ansible.builtin ? for best pratice usage 14:04:05 ? 14:04:10 nitzmahone: ;) 14:04:23 what does ACD stand for? 14:04:24 gundalow: please also add "installing from local tar" like file:/// ... kind of install, this needs to be covered too. 14:04:31 Ansible Community Distribution 14:04:31 Ansible Community Distribution. 14:04:40 if you needed to always put "ansible.builtin" in the collections keyword, that would be annoying to the point that we would have made it transparent. 14:04:47 nitzmahone: If that's all, then that should be doable. 14:04:47 jtanner: What I mean is just a set like we have now 14:04:49 acd-disorder :D 14:04:57 jtanner: exactly my point 14:05:05 ok 14:05:09 so not :D 14:05:10 thanks 14:05:38 fobhep: the "builtins" are still in their previous locations and still accessible without FQCNs or collection keyword usage 14:05:39 gregdek: it wasn't clear to me; thanks for the clarification 14:06:00 zbr, it's supported, run ansible-galaxy collection install /path/to/tar 14:06:36 sshnaidm: i know the answers, but i mention here because I know that they are missing from the docs, and it took some time to findout. 14:06:51 zbr, it's in ansible-galaxy --help 14:07:14 Eventually the names we call things by should change. So well' start calling current ansible => ansible-base and acd => ansible 14:07:23 question: currently you can use modules via "short name" or fqcn - the short names do not to seem to work for filters 14:08:19 so even if I add a collection in a playbook via "collections" I have to use the fqcn to access a filter form the collection. is that intended behavior? 14:08:20 sshnaidm|afk: in fact you underlines another place which lacks documentation, the CLI, only doc piece is "Role name, URL or tar file" does not seem as a good enough for me. 14:08:42 fobhep: please add that one to the collections questions, I can talk about that one 14:08:48 (in the etherpad) 14:08:51 sure 14:09:11 afk bbiab 14:12:13 But does this mean we get a lot of `ansible-collection......` packages? 14:12:13 Not that that is bad, but just curious 14:12:57 TKersten: depends how OS packagers will package the collections 14:13:21 gregdek: thinking of roles on galaxy, they might use e.g. modules for configure let's say mysql or postgres db, so they depend on collections. what is unclear today is, a galaxy role does not handle that dependency (yet), so they would create a collection with a role in it and depend on other collection (because that is possible today) but I have the feeling this is not how is was meant to work 14:13:55 felixfontein: OK, sounds reasonable. But I can imagine it would not be that bad 14:14:22 resmo: that's a good question. Not sure what the answer is, tbh, but I'm not close enough to the details at this point. 14:14:25 so users create collections which causes more problems, we don't want to deal with 14:16:34 gundalow: exactly! before/after situations would be great to document! 14:16:53 and reassuring 14:16:57 specifically from pov end-user (i.e. admin), later on maybe for developers 14:17:06 rrey: +1 14:17:20 should that be part of the porting guides? 14:17:34 shaps +1 14:18:18 I think redhat will have to to a lot a communication around this (meetups, videos, etc ...) 14:18:43 it must be very easy to end user to find an answer around this change 14:18:47 #action DOCS: "if i used to x to install ansible and do y with it, now i should do z" (install everything; use everything and one new collection; Use a new module in a collection; I don't care about collections I just want to do my day job) 14:19:00 #action DOCS: Need some diagrams 14:19:10 so currently we install ansible via RPM from Red Hat Sat6, are collections going to be available in Red Hat offical repos? 14:19:20 gregdek: indeed much more 'separate things' and they also need, imo, different 'language' :) 14:20:33 jpmens: yep. A lot of this is storytelling, and we haven't been able to focus on that because we were still working out what the story was. 14:20:41 rey: yes, lots of this stuff will need tons of communication. Meetups are interesting right now, obviously :) but having a set of some standard content for folks to use at meetups (virtual, in-person someday, or otherwise) will be a thing. 14:20:42 Now I think we're much clearer on the story. 14:20:49 a tl;dr point of view of the changes in packaging would be hlepful. 14:20:50 another thing from customer pov is also that people are reluctant to install anything other than RPMs. 14:21:03 belfast77: we should talk later- need specifics from you 14:21:09 fobhep: yes :) 14:21:24 there needs to be some work on convincing I guess to use ansible-galaxy as somehting "not evil" 14:21:28 like using pip in production 14:21:30 jpmens: Did you see that I wrote this earlier: python3.8 -m pip install --user --upgrade -i https://toshio.fedorapeople.org/ansible/acd/ ansible For offline install, one way to get that would be to download the two tarballs that are there and pip install them. 14:21:40 I was planning for a Meetup about the Collections. But due to corona-stuff I have to postpone or make it virtual 14:21:52 abadger1999: I didn't; thank you 14:22:04 belfast77: the "official paying customer" packaging story is still WIP, so we'd be curious to understand how you do things today 14:22:18 TKersten: hoping that cybette can help you with some virtual logistics, bluejeans dial-ins, things like that, if you need 14:22:26 TKersten: why not virtual? 14:22:27 jpmens: Note that those are just for testing.... after you're done testing them out, you'll want to pip uninstall ansible ansible-base 14:22:55 TKersten: yes I'll be in touch with organisers to offer infra/logistics support for virtual events 14:23:20 cybette: virtual stickers, please. :-) <3 14:23:39 @nitzmahone sure, I can fill you in (I'm just a little concerned I might lose some good tools if they are moved to different Red Hat repos) 14:23:39 jpmens: :D 14:23:52 repeat question? 14:24:02 (At some point I'll create tarballs with versions lower than 2.10 so that you can install them for testing now but they'll be upgraded once 2.10 final comes out) 14:24:05 gregdek: That would be lovely. I'll keep that in mind 14:24:06 madonius: Probably. Depending on the time corona takes. But I love to see and people. Otherwise we would also have virtual rijsttafel and that is not at the top of my list :-) 14:24:19 cybette: Thanks, we will be in touch 14:26:27 jpmens, TKersten : we're also looking into vendors who offer drop shipments to make it easier to distribute swag to virtual event attendees (need to see how this can be scaled) 14:26:38 I am +1 tarballs, but without a need of galaxy server. 14:27:42 #action gundalow to tidyup Collections content migration & ACD 14:27:43 cybette: That sound good. I can imagine shipping lots of small stuff can cost a lot. 14:29:00 I work with Enterprise PCI/Security Environments which have no internet connections 14:29:29 @gregdek can you add that point as a note in the meeting minutes 14:31:27 #agreed ansible-galaxy cli needs a clear mechanism to install collections via source if the provider makes the source available 14:32:44 main issue around this : We can't use requirement.yml if the collection is not on galaxy 14:33:08 Sample multi-collection repo: https://github.com/ansible-collections/netapp 14:33:14 #action extend user-facing documentation of how to install collections (include options beyond ACD) 14:33:16 rrey: This is something that geerlingguy is reminding us about continually :) 14:33:23 so it means one has to do gruntwork on CI (wget _ ansible-galaxy collection install ) 14:33:54 gregdek I'm not surprised 14:34:04 #action belfast77 / nitzmahone to connect with product management around downstream packaging needs 14:34:27 #action overhaul https://docs.ansible.com/ansible/devel/community/maintainers.html to be a Collection Maintainers page 14:35:22 #actions DOCS: detail the different packaging systems (PyPi, deb, etc, Galaxy) for ansible-base, ACD, individual collections 14:36:16 #agreed collections changelogs can use any solution maintainers prefer, however, we will develop a procedure for amalgamating them based on one solution 14:37:14 #topic Collections: Development and Contributor 14:37:22 2 million? wow 14:37:41 1 million was the ec2 module 14:37:46 (kidding) 14:37:48 heh 14:37:50 #info webknjaz is now explaining how the migration happened from a technical perspective 14:37:51 lol 14:37:53 lmao 14:38:02 ec2 was way more than that :D 14:38:06 thanks to webknjaz as I put him on the spot about 15 minutes ago 14:38:26 webknjaz++ 14:39:02 bye IRC, see you in person next time ! 14:39:08 o/ 14:39:11 rrey: Thanks! 14:39:14 o/ 14:39:15 o/ rrey 14:39:23 o/ 14:45:16 No questions, just glad you were able to take care of it :) 14:45:27 Good overview of some of the issues involved 14:45:45 Thanks webknjaz ! 14:45:47 webknjaz ++ 14:45:51 Thanks webknjaz :) 14:45:54 thanks :) 14:46:06 afk 14:46:08 #topic 15 minute break 14:46:09 * resmo away for dinner, bbl later 14:46:10 thanks! 14:46:14 thanks webknjaz! 14:46:19 * felixfontein needs to fetch some water, I ran out of water some time ago 14:51:46 Btw for those who joined later, please add your attendance + info (for swag) here: https://forms.gle/Uv9Zdaz7aTAAwtsQ8 15:00:49 #chair shertel acozine 15:00:49 Current chairs: Pilou TKersten acozine belfast77 cybette digigrate digigrate_ dmellado ganeshrn gregdek gundalow gwmngilfen ikhan jimi|ansible jpmens madonius mattclay nitzmahone phips rbergeron resmo rrey shaps shertel sshnaidm tadeboro webknjaz 15:01:11 #info related issue regarding collection installation https://github.com/ansible/ansible/issues/61680 15:01:11 DING DING DING Btw for those who joined later, please add your attendance + info (for swag) here: https://forms.gle/Uv9Zdaz7aTAAwtsQ8 15:01:55 gundalow: start recording... 15:02:04 cybette: ^ 15:02:28 #topic Thanks 15:02:36 headdesk 15:02:40 #undo 15:02:40 Removing item from minutes: 15:02:43 rbergeron: omg... 15:02:47 Carol Chen++ 15:02:50 good morning rbergeron! 15:03:08 :) rbergeron long time! 15:03:52 #topic new join intros 15:05:45 #topic Collections development status 15:05:54 resmo: which part :) 15:06:35 SWAG?!?! :p 15:06:36 hey @odyssey4me hello from Hampshire 15:06:41 :) 15:06:49 If I sign up twice, do I get twice the swag? ;) 15:06:52 belfast77: erm, username checks out ;) 15:06:58 lol 15:07:39 rbergeron: UTC-7! 15:07:40 belfast77: ah, you're from the hampshire devops meetup? 15:07:50 yup that's me 15:08:14 #info Collection Status https://github.com/ansible-collections/overview/blob/master/status.rst 15:09:43 #action gundalow status.rst update aws repo names 15:12:53 resmo: yeah. abadger1999 is as well, and nitzmahone too 15:13:17 odyssey4me: YES! i haven't seen you since fosdem last year before you started at red hat :) 15:13:19 and mattclay 15:13:47 #info Most Collections can be found at https://github.com/ansible-collections/ 15:13:49 Example https://github.com/ansible-collections/grafana 15:13:52 rbergeron: man, that is a long time 15:14:12 https://github.com/ansible-collections/grafana/blob/master/galaxy.yml 15:15:30 Python imports https://github.com/ansible-collections/grafana/blob/master/plugins/modules/grafana_folder.py#L165 15:16:21 Note about python imports: we also support relative imports in 2.10 so imports can be changed to relative if people want to (relative is usually shorter) 15:16:27 community collections in galaxy have an A logo which makes me think they're supported, but gundalow just said they're not; confusing. https://galaxy.ansible.com/community 15:16:57 jpmens: so! different logos is part of the issue here. 15:17:25 * jpmens is trying to be constructive :) 15:17:26 The Red A / Red Hat is the logo for Company Supported. The Teal A is the logo for Community Supported. 15:17:36 So `from ansible_collections.community.grafana.plugins.module_utils.base import grafana_argument_spec` could be `from ..module_utils.base import grafana_argument_spec` 15:17:43 #action gwmngilfen to work on tracking which parts of community.general are getting large 15:17:44 gregdek: wow; I didn't know that 15:17:55 No, that's a great point. That's part of the branding discussion, and it's not clear yet. Still in process. 15:18:10 Basically, we're looking at the Fedora / RHEL metaphor here. 15:18:29 Except that in that case, RHEL kept the Red Hat logo, and the community had to create their own logo. 15:18:49 In this case, community will inherit the old logo, and the enterprise version will have the new Red Hat logo. 15:18:52 red A on white background to make it completely different? 15:19:12 Actually, we'll probably phase out the Red A entirely in favor of a Red Hat brand. 15:19:13 gregdek: which is the 'new' RH logo? 15:19:14 "fedoransible" 15:19:25 misc: lol 15:19:25 ansidora 15:19:32 misc: that's better than ansibora 15:19:36 https://www.ansible.com/logos - > Community vs Red Hat logos 15:19:40 o_O 15:19:57 So imagine the logo here: https://www.redhat.com/en/technologies/management/ansible 15:19:58 (for now, I think changes are in progress) 15:20:08 Except instead of the Red A, it'll be the Red Hat. 15:20:48 gregdek: no mention of official/comunity on that page ;) 15:20:49 Again, changes in motion, and as such, always subject to change -- but it's our intention to inherit the classic Ansible logo to refer specifically to the community activities / projects / bits. 15:20:57 jpmens: no, there isn't, lol -- part of the issue. 15:21:20 We've been strengthening our hand to take over key bits of ansible.com, some more work to do yet. 15:21:32 But we're getting there. Everyone understands the need for brand clarity. Now we just have to do it. 15:21:46 #topic routing/redirection in 2.10 15:21:48 #info https://github.com/ansible/ansible/pull/67684 15:22:10 So if an Enterprise customer is installing from RPMs now, they will lose access to those modules without going back to their security teams to get approval for community supported modules ? 15:22:35 THAT WOULD NOT GO OVER VERY WELL 15:22:37 belfast77: That sounds correct to me. 15:22:47 #info nitzmahone is now talking about how 2.9 playbooks will still work out of the box when people install Ansible 2.10 15:23:16 belfast77: And there may be rpms provided by EPEL or something outside of Red Hat. But not by Red hat itself. 15:23:18 belfast77: as far as I understand it, the expectation is that the 'ansible' package will be a virtual package which depends on 'ansible-base' (the core stuff) and another package with the comunity stuff in it. 15:23:45 belfast77: if you plan on installing the ansible RPM then I guess no 15:24:30 My current understanding is that red hat won't be providing an ansible RPM; they will provide an ansible-base RPM. 15:24:51 .oO(collections in library...?) 15:25:07 resmo: ? 15:25:12 Or was it library (module_utils) in library? 15:25:21 just thinking loud but never mind... 15:25:30 abadger1999: ah, so 'ACD' is not going to be packaged by RH? 15:26:12 shaps: ACD (ie the `ansible`) package will contain supported and unsupported content 15:26:39 k, sounds reasonable 15:26:39 shaps: AFAIK, it won't be packaged as an RPM by RH. There's a new thing called execution environments which tower is going to use. There may be one which is based on ACD (and a different execution environment based on pure ansible-base). 15:26:42 collection CNAMEs 15:27:05 I'm unclear whether execution environments will be useful/distributed outside of tower, though. 15:27:06 jpmens: it will be DNS's fault in the end :) 15:27:13 jpmens: collection round robin 15:27:17 shaps: of course 15:27:32 misc: sounds just about right :) 15:27:56 abadger1999: yeah, nitzmahone mentioned exec environments earlier 15:28:03 One of the requirements of Collections has been to make easier to avoid having unsupported content installed (if you are installed the supported RPM) 15:28:15 jpmens: +1 CNAMES is it. 15:28:15 Kris Buytaert will be delighted it :P 15:28:29 *delighted to hear it 15:28:31 but will ACD still be shipped in releases.ansible.com? 15:28:41 heh, leave it to jpmens to use a DNS metaphor :) 15:28:44 shaps: my guess is that RH ships ansible-base as an RPM because they'll have to in order to support other RH Ansible usage... but don't hold me to it, no decisions made yet there afaik 15:29:19 as long as I can grab an RPM from relases.ansible.com I'm happy, not bothered too much about how/what RH ships 15:29:38 ACD RPM that is 15:29:38 Do you care if it's full ACD or ansible-base? 15:29:41 Ah, ok 15:30:12 That's probably a question for abadger1999 15:30:28 personally, I'm really liking that collections can release independently so that we can pick up bug fixes much, much faster 15:30:31 I don't know why we couldn't ship both an ACD tarball and an ansible-base tarball, but I don't know the details 15:30:35 shaps: I've heard that there will no longr be rpms shipped on releases.ansible.com 15:30:40 odyssey4me: that's the hope! 15:30:45 if the repo changes, it means I have to raise a security risk assessment and proxy whitelisting change, etc ... oh joy :) 15:31:10 I could put acd tarballs there since we'll be generating that for pypi as well. 15:31:24 👍 15:31:46 omg, irccloud has emojis like slack, lol 15:31:47 I guess acd tarballs will be fine as well 15:32:15 gregdek: and they render properly on a matrix client. THE FUTURE IS HERE 15:32:22 SHUT UP! wow 15:32:25 lol gundalow my thinking exactly 15:33:02 belfast77: Yeah... and if you're doing things according to github repo..... There will not be a unified repo for all the collections that go into acd. There's scripts to generate ACD but ACD itself is just a conglomerate of several other collections. So the repositories for the code will be those upstream collections. 15:33:22 #topics Collection docs 15:33:37 #topic Collection docs 15:34:06 #info acozine is now talking about how Collection documentation will be generated 15:34:23 abadger1999: I didn't look at the script yet, are you pulling from Galaxy or N collection repos? 15:34:28 * gwmngilfen pays attention, docs == webstats... :P 15:34:51 each time I read "ACD", I think we are 1 letter close to maintain the rock band naming of ansible release (like Ansible Community Distribution of Collection would be perfect) 15:35:05 ACID 15:35:16 #info 2.9 module index https://docs.ansible.com/ansible/latest/modules/modules_by_category.html <- Based on the directory structure under `lib/ansible/modules/` (before modules were deleted) 15:35:29 nitzmahone: The current script pulls from galaxy. I need to talk to legal about GPLv3 obligations. If the ruling from them is, galaxy doesn't constitute source so you need to pull from the git repo, then I'll have to change that. 15:35:52 #info 2.9 Module URL https://docs.ansible.com/ansible/latest/modules/dnsimple_module.html 15:36:08 advertisement ;) 15:39:27 misc: you are not the first person to make that observation ;) 15:39:31 ;) 15:39:33 lol 15:39:58 time to go help with kids. cya tomorrow folks, will be around if anyone has data ideas :) 15:39:59 #info Example Collection doc http://docs.testing.ansible.com/collections/purestorage.flasharray/purefa_user_module.html 15:40:22 Thanks gwmngilfen! 15:42:17 #info Collection level (plugin index) http://docs.testing.ansible.com/collections/purestorage.flasharray/index.html#plugins-in-purestorage-flasharray 15:42:29 #action DOCS: Table showing old and new URLs 15:45:18 Any questions on Collection docs? 15:45:53 I'm wondering what sort of standard interface needs to be in place to make this publication work? 15:46:27 I know that there's been some discussion around changelogs/release notes... and that's free to each collection to handle their own way. 15:46:44 acozine: I have experience, but it's not... good or recent :) 15:46:59 odyssey4me: it would be nice if there's a common machine-readable format used by all collections 15:47:08 odyssey4me: for module & plugin documentation just follow what was done in ansible/ansible. Also use `ansible-test sanity` which will check for most common issues in module documentation 15:47:09 no matter which tool collections end up using 15:47:19 Fortunately, it's a flat list of URLs that needs to be generated once, correct? 15:47:42 (I was just talking about changelogs, porting guides, release notes) 15:47:50 Any plans on providing an "official" tool for exracting the API docs from the modules? 15:48:29 tadeboro: you mean the tool which creates the .html files for docs.ansible.com? 15:49:06 felixfontein: the tool that takes the module source and creates a rst actually. 15:49:25 tadeboro: that's the tool; rst -> html is done by sphinx 15:49:47 tadeboro: at the moment that's in ansible/ansible docs/ 15:50:06 But this tool is really not available to external collections. 15:50:18 .................................... 15:50:31 acozine is now talking about porting_guides and changelogs 15:50:45 #info 2.9 Porting Guides https://docs.ansible.com/ansible/devel/porting_guides/porting_guides.html 15:50:49 tadeboro: ansible-doc opens the module and reads the metadata from the module; it will also support modules in collections 15:50:50 tadeboro: I know that abadger1999 is working on the tool, though not whether it is planned to release it officially (like ansible-test) 15:51:09 gundalow: FYI there's also a sphinx extension in https://opendev.org/openstack/tripleo-ansible/src/branch/master/doc/source/_exts/ansible-autodoc.py which autodocuments roles and modules like this https://docs.openstack.org/tripleo-ansible/latest/ 15:51:20 tadeboro: Is this because you'd like to host your own docs? 15:51:27 odyssey4me: Nice, thanks for sharing 15:51:53 gundalow: I think it would be really great if that tool could be opened, especially for companies who have internal collections they do not publish, and want to generate docs for them 15:52:07 (also for collection developers to see how their docs will look like) 15:52:58 The plugin->rst stuff could probably be done as a special case in ansible-doc, but the thought of trying to package/support html generation is ... :O 15:53:22 Yeah, we needed to document one of our collections and we hacked together a tool that mimics what ansible-doc does and then integrated the output in our Sphinx site. 15:53:29 tadeboro: yeah, what I'm currently working on is tightly integrated into the ansible build process..... I was thinking last week (as I worked on other htings) that it might be nice to extricate that into something else. I hadn't thought about its own tool but I could see the benefit of that. I'd have to talk to acozine about that and also think about how that would impact the time requirements. 15:53:33 A couple of internal "live" HTML generators for Automation Hub etc are in the works 15:53:48 Those are MUCH simpler to deal with than , eg the sphinx pile we use internally 15:54:06 #action Document (best case separate tool chain) that allows people to build module docs outside of Ansible. Or perhaps Collection Explorer. Link to this from dev_guide. https://opendev.org/openstack/tripleo-ansible/src/branch/master/doc/source/_exts/ansible-autodoc.py https://docs.openstack.org/tripleo-ansible/latest/ https://github.com/ansible-community/collection-explorer 15:54:11 This is the tool that we currently use: https://github.com/xlab-si/ansible-doc-extractor 15:54:30 gundalow, maybe I missed, but is ACD going to be released once a half year with ansible-base? Does it mean the new code of collection will be available there only once in half year too? 15:54:32 #info There is a tool that allows you to view locally installed Collection documentation https://github.com/ansible-community/collection-explorer 15:54:42 The collection that we document: https://github.com/sensu/sensu-go-ansible 15:54:51 sshnaidm: I think abadger1999 will address that next 15:55:01 https://sensu.github.io/sensu-go-ansible/ 15:55:06 @sshnaidm I can talk about that in the next section. Please remind me if I forget to talk about that explicitly. 15:55:13 abadger1999, cool! 15:55:21 ^ the generates site 15:55:27 abadger1999: Great! 15:55:27 sshnaidm: I expect collections to be released more freequently 15:55:33 * nitzmahone almost brought that up in the docs discussion, but it makes more sense in the ACD discussion 15:55:49 and collection releases will most likely speed up as we have more confidence and experience 15:56:11 i hope that acd would release as often as possible, as long it passes CI testing. 15:56:13 tadeboro: If you have a mac you can use https://github.com/ansible-community/collection-explorer locally to view collection docs 15:56:28 abadger1999, maybe worth to mention about changelog tool that we mentioned before 15:56:35 Basically collections could be any cadence, ACD will be a regular snapshot of some collections + an ansible-base release pin, and ansible-base will be the slowest cadence 15:56:56 acozine, oops, meant to you (about changelog tool) :) 15:57:06 heh, I was about to say... ;-) 15:57:34 * nitzmahone leaves the other details for abadger1999 to fill in :) 15:57:56 but i hope we will have a “latest” permalink for docs, not one with version on it. 15:58:05 abadger1999, feel free :) 15:58:17 #action gundalow Summit (and hackathon) writeup to Reddit 15:59:00 zbr We are discussing how to do both the latest and devel permalinks. latest is probably very easy to transition. devel will be a little harder but we think we have a plan for that. 15:59:00 zbr++ 15:59:35 gundalow: As an aside, I'd like to say that the snacks and drinks at this conference are top notch. ;) 15:59:42 devel would be "the devel branch of the ansible/ansible git repo plus the latest version of collections in ACD posted on galaxy" 15:59:50 odyssey4me: haha, that made me laugh 15:59:52 odyssey4me: i disagree ;) 16:00:07 * jillr passes nitzmahone some tea and scones 16:00:08 abadger1999: so no devel of collections themselves? 16:00:14 felixfontein: correct. 16:00:15 nitzmahone: you can never please everyone :p 16:00:41 odyssey4me this is the same crap I have at home 16:00:46 felixfontein: at least, at the moment, I haven't figured out a good way to do devel of collections. 16:01:22 #topic 15 minute break 16:01:23 abadger1999: I guess that makes it more important that collections can build their own 'devel' docs to see how they look like 16:01:23 gundalow: how much more leg-streching will we have to do today?!? ;) 16:01:34 * acozine heads to the kitchen to look for snacks 16:01:35 felixfontein: because I wouldn't be sure whether the devel branch of a collection would make it into the next major release of ACD or not. 16:01:35 nitzmahone: you should speak to the manager 16:01:36 it's exasusting 16:01:39 jpmens: how much more would you like? 16:01:43 felixfontein: yeah... good point. 16:01:53 #topic 15min break, back at 16:15UTC 16:01:55 jpmens: oh, I assume you are just walking to the kitchen 16:01:56 jpmens: next time we meet in person, we can do Contributor Summit Yoga 16:02:08 abadger1999: true. collections will probably use all kind of different workflows... 16:02:15 acozine: we will not meet in person, then, unfortunately ;) 16:03:41 I had snacks (finnish/swedish candy) and activities planned but.... oh well, next time :) 16:03:56 @acozine we probably want to write down felixfontein's point somewhere: the fact that we aren't building docs of the developing branch of a collection makes it more important that collections can build their own 'devel' docs to see how they look. 16:05:23 abadger1999: ack 16:06:11 cybette: what kind of activities? leg-stretching? ;-) 16:06:52 weight lifting (at the bar) :P 16:07:40 * nitzmahone NAKs treats from jillr, clearly your TreatsOverIP impl is still WIP 16:08:31 * jillr shoves a second scone into the router, cause what could possibly go wrong there? 16:08:35 🍺 💪 16:08:42 +1 16:08:53 jillr: pics or it didn't happen ;) 16:09:08 cybette: I I'd be happy with a snackathon 16:09:18 that's every day at my house 16:09:30 (and I have the waistline to prove it) 16:10:04 madonius: sounds fun! (and dangerous) 16:10:21 nitzmahone: is a veteran, we are safe 16:10:34 nitzmahone: yeah, and even more so nowadays... 16:11:32 cybette: feel free to include swedish/finnish candy in swag shipments :) 16:12:01 don't want it to go to waste :D 16:12:24 nitzmahone: I'm thinking of how to optimize that because some swag are in raleigh 16:12:48 oh but the finnish ones are already gone (in my tummy). the swedish ones I was going to get when we were in gothenburg 16:12:51 nitzmahone: you'd rather it goes to waist then? :) 16:13:03 lol 16:13:04 heh, and they postponed the replacement of the janky old shipping tool :( 16:13:13 * nitzmahone groans 16:13:20 yeah 16:13:50 * nitzmahone had first experience with that recently to return old laptop to IT 16:13:54 I was really looking forward to scandinavian snacks 16:14:08 now I have to stick to nuts 16:14:31 nitzmahone: lucky you, I had to deal with it on a weekly basis 16:15:41 felixfontein: hmmm I'll see what I can do 16:16:09 cybette: it's ok, they're usually better when they're fresh anyway :) 16:16:26 and hopefully we'll travel there next year 16:16:29 cybette: I'd bet felixfontein is really keen on getting his hands on some Surströmming 16:16:32 (this year probably won't work...) 16:17:01 odyssey4me: definitely not, I don't like seafood ;) 16:17:46 odyssey4me: hahah oh dear 16:17:53 #topic ACD 16:18:06 #topic ACD (the ansible package) bulid process 16:18:18 oh, sorry nitzmahone 16:18:29 no worries, yours was more specific :) 16:18:50 https://toshio.fedorapeople.org/ansible/acd/ansible-2.10-file-layout.png 16:19:30 #info abadger1999 is now talking about how the new `ansible` package will be built 16:19:47 #info https://toshio.fedorapeople.org/ansible/acd/ansible-2.10-file-layout.png shows what `ansible` package will look like on disk 16:21:06 really happy to see that pip install method is supported 16:21:19 hmm, is there a ansible module encapsulating `ansible-galaxy collection install`? 16:21:32 what happens if ansible.posix exists in 2 different directories? 16:21:58 FYI it's already in ansible/ansible@devel 16:22:15 #action DOCS: "How do I X" what happens if ansible.posix exists in 2 different directories? 16:22:47 jpmens: i guess python import tules apply, the first one found in PYTHONPATH, it will be loaded. 16:23:03 exactly 16:23:03 I'm guessing that this means another ansible.ini config option and corresponding bash env variable to specify paths to search? 16:23:25 ANSIBLE_COLLECTIONS_ROOT 16:23:31 https://gist.github.com/abadger/1f14caded92117fbd3036842c875701c 16:24:26 @zbr: https://github.com/ansible/ansible/pull/67093 16:25:11 re 16:25:21 #info possible ACD Build process https://gist.github.com/abadger/1f14caded92117fbd3036842c875701c 16:31:50 https://github.com/abadger/misc-work/tree/master/ansible/build_acd 16:32:27 #info ACD build scripts (work in progress) https://github.com/abadger/misc-work/tree/master/ansible/build_acd 16:33:14 #info ACD parse_args 16:33:23 https://toshio.fedorapeople.org/ansible/acd/acd.in 16:34:04 I'm missing aws.aws.aws 16:34:27 https://toshio.fedorapeople.org/ansible/acd/acd-2.10.build 16:37:37 https://toshio.fedorapeople.org/ansible/acd/acd-2.10.0.deps 16:38:15 interesting to see that this uses yaml format intead of pip constraint format, bit weird 16:38:42 https://toshio.fedorapeople.org/ansible/acd/ 16:39:03 felixfontein: for mod utils? it'll be amazon.aws.aws ;) 16:39:20 i would have prefered a native constraints format because in the future we may need tpo use additional constraints, like python version specific ones 16:39:27 jillr: too bad, I really was looking forward to aws.aws.aws :) 16:39:30 python3.8 -m pip install --user --upgrade -i https://toshio.fedorapeople.org/ansible/acd/ ansible 16:39:31 question: abadger1999 in bugfix releases e.g. 2.10.1, do also bugfix version releases get into acd? 16:40:14 felixfontein: it was an option! I was worried it would cause too much confusion though 16:41:36 jillr: I'm quite sure it would. they way it's now is definitely better. though less funny :) 16:42:08 * jillr parks the aws namespace on galaxy, so felixfontein can't release an April Fools collection next week! 16:42:31 lol 16:42:51 we could try to release a 3.0 on the 1st :D 16:43:10 zbr: lol! 16:43:50 abadger1999: how does a collection get into that list? 16:44:22 resmo: you need to know the right people ;) 16:44:58 in 2.10, basically it's in if it came out of 2.9.Longer term, we need clear requirements for collections to 16:45:13 ...be included as new collections. 16:45:25 We don't have those yet, but we will. 16:47:18 testing would be required because without it, someone would break the acd with their own reqs 16:48:10 #action ACD Testing: testing would be required because without it, someone would break the acd with their own reqs 16:48:27 if we have our own sanity job that is part of their merge-gate, we should be safe. 16:48:54 at least their would not be able to merge a change that break the *building* of the acd. 16:48:57 Will the certified collections be automatically included in the ACD? 16:49:33 either way: I think the initial bar will likely be fairly low, and the community will likely figure out how to level up the appropriate types of testing and whta levels of testing are required over time to be included. 16:49:39 if zuul-ci is going to be used for testing changes, it would be very easy to assure they would not break it 16:50:39 zbr: Yup, I expect that for most community collections we will use Zuul to do the tag & publish into Galaxy. We could add some pre-checks there before the tag can be done 16:50:50 we do this with podman where openstack stack has ci jobs testing that a change in podman would not break openstack. 16:50:58 it will work just fine 16:51:17 I am going to try this "outside" people have been talking about lately… See you! 16:51:30 madonius: Thanks for your time. Did you do the swag survey? 16:51:43 yep :D 16:51:49 I appreciate this is a community discussion I work with syadmins and Ansible Core paying customers, none of then use "python pip install", with the split would people be sitting with old module versions, whilst ACD marches on? 16:51:51 maybe we should rename acd to "the bazar" 16:51:57 (important stuff has to be important, right?) 16:52:08 madonius: o/ 16:52:09 zbr: that is correct. It's entirely possible that we could do something like at least a basic periodic test with zuul to do something like that. over time we could certainly get to a point where we did that more frequently and using gating, etc. rather than just for feedback. 16:52:15 madonius: thanks for joining today, see you ! 16:52:26 thanks for organising 16:52:37 belfast77: sort of, also something we can cover in our talk about product packaging 16:52:38 I did enjoy it and learn a ton 16:53:00 zbr: LOL, +1 for bazar. GRAND BAZAR 16:53:15 zbr: bazaar, or bizarre? 16:53:24 #topic Cloud Collections 16:53:24 zbr: I saw you rofling tho 16:54:02 belfast77: ACD is one of several ways to install content; esp for capital-S-Supported by Red Hat content, there will be some prepackaged options, as well as "official" ways to get supported content 16:54:11 * webknjaz releases ansible-grand-bazar==3.0.0 on Apr 1st 16:54:29 belfast77: The answer is possibly. 16:55:08 The way the resolution works, things installed with pip are the last thing in the lookup. 16:55:16 #info jillr now explains how the different Cloud Collections are setup 16:55:36 #info Ansible Cloud Team: AWS, VMware 16:56:53 some of the certified collections are likely to be included in their upstream form in ACD. they may have different life cycles / policies depending on the vendor, but i suspect that if one is a customer that those collections should be downloaded from automation hub rather than the version in ACD. 16:58:10 for vendors who aren't open sourcing that content or aren't choosing to work upstream, that's their choice -- but we certainly are making them aware of the benefits of doing their work publicly upstream and collaborating with the broader community. 16:58:12 belfast77: So if you only pip install via either acd or pip install some package that a collection owner decides to upload to pypi, and then pip upgrade that, the upgrade will be found. But then if you use ansible-galaxy to install a new version, the ansible-galacy version will be found. 16:58:31 Openstack collection is mirrored to github though 16:58:42 but not developed there, yes 16:59:03 belfast77: And so, even if you upgrade the pip installed version, it will not take precedence over the older ansible-galaxy version. 16:59:08 right, good note, all of the openstack project repos have read-only GH mirrors 16:59:17 https://github.com/openstack/ansible-collections-openstack 16:59:46 #info Release notes & Changelog fragments https://github.com/ansible-collections/overview/issues/18 17:01:11 jillr, also podman modules moved to "containers" github namespace: https://github.com/containers/ansible-podman-collections 17:03:25 there's a "scripts/inventory" directory in community.general for example 17:03:39 question: a vendor can "take over" the cloud modules in community.general? is that correct? I mean, I wrote most of the vultr integration, vultr can come and take it over? 17:03:40 it's possible to see in scenario files 17:04:04 #action jillr check where inventory scripts (not plugins) are 17:04:48 resmo, +1, maintainers of various multiple community modules is an interesting topic 17:05:40 good that modules are GPL :) 17:05:56 #action DOCS: search for `contrib` as that directory no longer exist 17:06:05 but who will have merge rights to community modules repos? 17:06:20 sshnaidm: zuul only! 17:06:30 sshnaidm: depends on the repo 17:06:42 if it's same ansible core, it won't help with burden and bottleneck 17:06:54 usually a set of maintainers. community.general will be special (more similar to ansible/ansible) 17:07:09 #action DOCS: search for `contrib/inventory` in docs to point people to the new individual repos (and new directory structure) 17:07:30 in the case of community.vmware, it's a mix of VMware employees, Ansible core and some comunity conyributors 17:08:00 are there plans to get the Proxmox module in the community.general repo? 17:08:36 @abadger1999 @nitzmahone thanks very much for your answers earlier 17:08:56 leogallego: since it's an existing module in ansible, it should be getting pulled into the community.general collection/repo 17:09:14 rbergeron++ thank you 17:09:16 leogallego: Karma for rbergero changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 17:09:38 #topic Community Process for Collections 17:09:38 LOL zodbot ;) 17:09:39 rbergeron: leogallego: but it's not there right now 17:09:52 ah wait, they are 17:09:56 looked at the wrong place... 17:10:00 I'm happy to give information that I have and say when something still needs discussion :-) 17:10:22 felixfontein: i was about to have a moment of thinking i was absolutely insane :) 17:10:24 https://github.com/ansible-collections/community.general/tree/master/plugins/modules/cloud/misc/ 17:10:25 #info Contributor Collection status https://github.com/ansible-collections/overview/blob/master/status.rst 17:10:49 rbergeron: you're definitely not! (as far as I can judge ;) ) 17:11:22 felixfontein, ohhh look at that, i was looking for the proxmox directory 17:12:14 leogallego: I think it's the same structure as before; was there a proxmox directory? 17:13:01 what's the state of the PR/issue mover? 17:13:12 endless Matts on the core team 17:13:26 :) 17:13:54 hi gundalow's son :D 17:15:39 for issues I'd close and link to the correct collection 17:15:41 is not possible to use transfer option? 17:15:46 shaps: +1 17:16:12 I'd probably like some issues transferred 17:16:17 CI run for a collection https://github.com/ansible-collections/grafana/pull/81 17:16:23 shaps: if transfer is not possible, it make sense to close them with just link, it be a forced-cleanup, not really bad. 17:16:24 but it's probably easier to recreate them one a case-by-case basis 17:17:06 felixfontein: indeed, it someone takes time to recreate them, it means it is still valid. 17:17:26 It would be nice to put a link in every issue that says "if you want to re-file, click this button" which then autofills a new issue in the right place, with proper ownership as well 17:17:37 Make it dead-easy to reopen in the right place 17:17:48 but one hint: make a bot that does close tickets with a limit per day, so we spread the load 17:18:09 gregdek: sounds shiny :) 17:18:13 #link Survey link: https://forms.gle/Uv9Zdaz7aTAAwtsQ8 17:18:21 gregdek: sounds good! there definitely must be a link back to the old issue, thpough 17:18:27 (also in etherpad) 17:18:29 Yes, absolutely. 17:18:32 at least 100% virtual solves the "IF YOU WANT TO BE HEARD YOU MUST USE A MIC" problem :) 17:18:46 I thought there was a fork of ansibot somewhere that did something like what gregdek is describing, but I don't know more than that 17:18:48 nitzmahone: ++ 17:19:16 Thank you everybody for your time 17:19:24 Thanks ansible community team! take care! 17:19:24 stay healthy everybody 17:19:35 thanks everyone as well! it was a nice contributor summit :) 17:19:39 thanks everyone! 17:19:39 Thanks! 17:19:43 * felixfontein waves 17:19:50 Cheers all, sad not to meet in person, but been a good day! See you 'round the interwebs 17:19:51 o/ 17:19:52 Bye all. o/ 17:20:12 #endmeeting