14:31:04 #startmeeting Docs Working Group aka DaWGs 14:31:04 Meeting started Tue Jun 23 14:31:04 2020 UTC. 14:31:04 This meeting is logged and archived in a public location. 14:31:04 The chair is acozine. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:31:04 Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:31:04 The meeting name has been set to 'docs_working_group_aka_dawgs' 14:31:10 #topic intro chatter 14:31:12 hey! :) 14:31:13 who's around? 14:31:19 hey felixfontein! 14:31:24 #chair felixfontein 14:31:24 Current chairs: acozine felixfontein 14:31:31 hi all! 14:31:39 * acozine is tempted to make gundalow a footstool 14:31:45 #chair samccann 14:31:45 Current chairs: acozine felixfontein samccann 14:32:01 #footstool gundalow 14:32:09 :-) 14:32:38 poor gundalow :) 14:32:42 who else wants to chat today? andersson007_ belfast77 briantist cyberpear madonius mrproper Pilou shaps thedoubl3j tributarian zbr 14:33:10 I'm here. 14:33:17 welcome tributarian! 14:33:20 #chair tributarian 14:33:20 Current chairs: acozine felixfontein samccann tributarian 14:33:22 hi tributarian !! 14:33:36 * acozine digs out the agenda 14:33:39 hello 14:34:02 https://github.com/ansible/community/issues/521#issuecomment-644857813 14:34:52 looks like our first two items are: porting guides / changelogs in the collections ecosystem, and collections docs pipeline 14:35:48 I saw updates last night 14:35:54 re changelog: I was working a bit on the ACD changelog generator the last days, here's a first impression of the new code: https://gist.github.com/felixfontein/7d5efcc70c8a25c218d9e8082f5ee92f It doesn't contain the ansible-base changelog yet, and unfortunately there are precisely two collections having a changelog.yaml right now. (I "created" a new version of ansible, 2.10.0a2, so that I would 14:36:00 have more than one entry) 14:36:04 #topic changelogs and porting guides 14:36:30 felixfontein: thanks 14:36:42 right now there's a subsection for every collection; I think it is better to only have them when something changed in the collection, and have a small list of collections which didn't change since the last version 14:36:54 felixfontein: would it help if I generated changelogs for some of the newly released collections in a branch? 14:37:12 samccann: no, since it extracts the changelog.yaml from the published collections on galaxy 14:37:14 apparently most collections don't even have the pre-migration changelog entries that apply to modules there 14:37:14 I did a couple yesterday just to prove it worked. It's just the stuff that comes by default (new modules etc) 14:37:29 ah bummer. 14:37:53 yep, I guess most people didn't went sifting through the deleted fragments yet 14:38:17 What if we created issues in every one of those repos to say please add changelogs in a patch release and that it's required to be in the Ansible changelog/porting guide? 14:38:36 is the changelogs system mature enough now that we can publicize (and write/update) documentation about it? 14:39:04 samccann: I think some of the collections already plan that (or for the next minor release), but an issue might help to get some more to do it :) 14:39:08 I'm in the process of trying to absorb felixfontein's docs into an ansible PR 14:39:13 acozine: on collection level, I think so 14:39:32 samccann: cool! 14:39:40 samccann: I like that idea - we could link to those PRs that removed the changelog entries from ansible/ansible 14:40:20 ok I'd like to get a quick PR ready that brings in Felix's docs first so we can point to published docs on the 'how to' as well. I'll get crackin on that today 14:40:20 I meant the create-an-issue idea; absorbing felix's work into an ansible PR is also great 14:40:30 samccann: excellent! 14:40:31 acozine: I think the latest version (0.4.0 since yesterday) is pretty good, though I guess we'll see when more people start using it 14:41:20 felixfontein: great, as long as you're comfortable with folks using it - I know how uncomfortable it is when people start reporting known issues on WIP stuff 14:41:29 I can start with the ansible-maintained team to get a couple of them done and up on galaxy if you want, before we nag everyone. That said, they will be just the new/removed etc so I don't know how much that helps 14:42:11 I've been using it with my collections, and since jborean and geerlingguy started using it for theirs I think I'm pretty happy. 14:42:32 kewl 14:42:35 I can look through one of those remove-the-changelog-entries PRs to see if I can identify the right destination for a few entries, so we could have a sample PR to a collection 14:42:37 the only thing that I think needs to be better documented is the plugin cache, when it is used in the way geerlingguy seems to be using it in community.kubernetes 14:42:59 hm, he isn't on this channel today 14:43:01 lemme ping him 14:43:38 #action samccann - create PR to bring in `antsibull-changelog` install and how to docs from the repo 14:43:48 acozine: sivel used a script to remove most of them, and that script gave a list of potential collections a changelog fragment belongs to. that (resp. the data it yielded) could help. 14:43:59 a lot of manual postprocessing is needed though 14:44:39 #action samccann create a nagogram issue or 3 pointing to that doc, the removed fragments from ansible/ansible and how/why the collection owner(s) need to generate their changelogs for inclusion in the next ansible release (maybe beta??) 14:44:40 * acozine looks for one of those PRs 14:45:07 https://github.com/ansible/ansible/pull/68565 14:45:41 https://github.com/ansible/ansible/pull/68498#issuecomment-609404786 and then there's another one 14:45:48 yeah, all of mine had a json file that listed the collection the change belonged to. But some of those have changed since then 14:45:49 #info sivel used a script to remove most of the changelog fragments from ansible/ansible and it attempted to list potential collections they belong to. Useful for tracking down collection owners that need to bring in those fragments 14:46:00 I downloaded the JSON, that's awesome 14:46:04 #link https://github.com/ansible/ansible/pull/68565 14:46:15 #link https://github.com/ansible/ansible/pull/68498#issuecomment-609404786 14:46:41 ah, some of them have multiple destination collections 14:47:02 https://www.irccloud.com/pastebin/xKfWzj5E/JSON%20output%20for%20changelog%20removals 14:47:04 sivel: yep. in many cases, simply grepping for all module/plugin names that are in the collection is probably also a good way to find the fragments of interest 14:47:12 #chair sivel 14:47:12 Current chairs: acozine felixfontein samccann sivel tributarian 14:47:25 (if you don't have hundreds or thousands of modules/plugins in your collection) 14:47:43 ah, so those are for changes that affected multiple modules 14:48:00 acozine: yeah, that is correct. Some changes affected multiple modules/plugins that ended up in different collection 14:48:20 * geerlingguy waves 14:48:27 welcome geerlingguy 14:48:29 #chair geerlingguy 14:48:29 Current chairs: acozine felixfontein geerlingguy samccann sivel tributarian 14:48:36 acozine: the fragment you posted belongs partially into ansible-base 14:48:40 hi geerlingguy! 14:48:53 welcome geerlingguy ! 14:49:13 Yesterday I merged in the new changelog/fragments into the k8s collection: https://github.com/ansible-collections/community.kubernetes/pull/131 14:49:25 yeah, we might end up with some fragments reintroduced into ansible-base 14:49:27 woot! 14:49:29 seems to work well at this point 14:50:12 but we'll write more targeted fragments for the ansible-base portion 14:50:20 geerlingguy: excellent! 14:50:45 geerlingguy: btw, you might need the --reload-plugins option for the `antsibull-changelog release` line - because if the version of the collection does not change, antsibull-changelog caches the information on all plugins and won't update its cache until the version in galaxy.yml changes 14:51:00 geerlingguy: so if you add new modules/plugins, they might get missed 14:51:29 * abadger1999 waves 14:51:38 good morning abadger1999! 14:51:42 #chair abadger1999 14:51:42 Current chairs: abadger1999 acozine felixfontein geerlingguy samccann sivel tributarian 14:51:45 I think this is something I haven't really documented yet (this behavior comes from the ansible/ansible changelog generator, where having 1000s of plugins/modules made this necessary) 14:51:50 wow it's a party now! 14:51:51 morning acozine! 14:52:07 felixfontein: For our purposes we would be (currently at least) only generating the new changelog at release time. 14:52:22 So far we aren't requiring contributors to add fragments in their own PRs (just to make contribution easier) 14:52:49 Apparently I don't have this channel set to auto-join. Fixing that. 14:53:03 geerlingguy: in that case you're safe. I think I still have to add this to the docs, or even provide a config setting to disable it by default. 14:53:05 geerlingguy - just curious - how do you track then what has changed? 14:53:27 16:53 <@samccann> geerlingguy - just curious - how do you track then what has changed? 14:53:45 samccann: I guess a maintainer adds the fragment (either to the PR, or in a follow-up) 14:53:49 felixfontein: so far, just going back through history of PRs merged since last tag 14:54:12 geerlingguy: we debated recently (last week? two weeks ago?) whether we should/could require changelogs for all PRs, given that we have a "trivial" category . . . 14:54:14 ah gotcha. thanks 14:54:43 acozine: I'm mixed on that... I think maybe that could be the case for more "professional" collections or definitely for supported collections. 14:54:59 acozine: this is probably something that every collection has to decide for itself... it's probably most important for large collections where we want a bot to merge a lot of things, and we don't have time to manually check every PR whether it had a fragment 14:55:20 But in terms of community-based work, unless we make it blatantly obvious how exactly to format changelog fragments, making that a requirement just adds to the stress of a first-time contributor trying to dot all the i's and cross all the t's 14:56:01 yeah, we had been thinking of adding a template that got auto-added when you opened the PR, but that won't happen in time for 2.10 for sure 14:56:02 geerlingguy: I think the idea floated around to have this checked by a bot, which could even suggest a fragment skeletton based on the PR description/labels 14:56:21 I know I basically run away from some projects when I submit a one line bugfix PR and then get three bot messages back asking me to (1) sign this thingie, (2) resubmit my PR with a signed commit, (3) go here and do that, (4) give them a blood sample, etc. 14:56:56 just say no to blood samples, but I'm all for PRs that require sending us chocolates 14:56:56 felixfontein: that would be okay by me. As long as it's automated, not pushy, and we don't end up slapping 8 follow-up messages on every new PR 14:57:16 samccann++ 14:57:42 so the bot would pull in the description into a fragment, and then the developer could edit it or accept it as is? 14:58:16 something like that, yeah 14:58:25 samccann: no idea, I'm not sure what's possible on github... I mean, someone has to implement this :) 14:58:49 (and forcing people to use random other websites to do this is also not helping) 14:59:06 hmmm.. good and bad to that. Yes we'd have all the fragments we could ever desire, but the vast majority will likely be descriptions, and those aren't always written well. (or even commit messages, which I think `newsreader` uses) 14:59:32 but yeah, problem for a diff. day 14:59:34 Descriptions are often quite poor. 14:59:41 eggzactly 14:59:45 yep 15:00:02 And commit messages are all over the board (different tense, different style, some punctuation some not, often puzzling and unrelated to changes made.) 15:00:11 yeah, any nudge for changelogs would have to be easy and helpful and hide most of the auto-entries as "trivial" so nobody would ever seem them 15:00:15 and changelog entries like `module - removed comment on line 123` are not useful at all :) 15:00:19 maybe it's not worth the work 15:00:34 Unless you're a team of one or two and are absolutely OCD about it, or have pre-commit hooks that validate messages (and even then...) commit text is a terrible form of documentation. 15:00:50 If changelogs are commit histories, then they are worthless :) 15:01:01 indeed! 15:01:48 Because it's super easy to run `git log --pretty=oneline 1.2.4..1.2.3` and see the exact same thing 15:01:53 * acozine once worked on an OCD project that enforced commit-message tense and a lot of other rules, but that was a project run by a bunch of librarians 15:02:06 LOL!!! 15:02:07 haha nice 15:02:29 ansiblers are a different crew 15:02:38 ansiblers++ 15:03:13 ansiblers ++ for being freewheelers and prioritizing making things work over enforcing rules 15:03:36 bazaar > cathedral 15:03:58 #agreed each collection can make its own decision about how much to encourage changelog entries, and we will not require them anywhere 15:04:43 all right, it sounds like the changelogs system has moved into the phase of testing and reporting issues 15:05:01 geerlingguy: commit messages are a decent way to document a changelog, but getting it right is tricky, especially as there is no undo 15:05:44 that is why I currently prefer to rely on github PR names + labels to build the changelog, as they ca be updated even after the merge 15:05:55 do we have anything to cover on porting guide(s)? 15:06:28 samccann: not really from my side, I want to get the changelog into a better shape before trying that out 15:06:38 ok kewl 15:06:48 geerlingguy: example: https://github.com/ansible-community/molecule/releases 15:07:14 one last question - is it worth creating a test collection or two on galaxy that attempts to exercise everything we can thing of wrt fragments etc? 15:08:28 samccann: sure. also feel free to "contribute" to https://github.com/felixfontein/ansible-versioning_test_collection ;) 15:08:58 kewl 15:09:15 let the changelog fragments pour in! 15:09:37 it already got some fancy multi-paragraph entries: https://github.com/felixfontein/ansible-versioning_test_collection/blob/main/CHANGELOG.rst 15:10:49 heh, Bob was there? 15:10:56 yep :) 15:11:28 http://www.galactanet.com/comic/view.php?strip=517 15:12:01 ah, thanks for the reference 15:12:46 the collection actually has a lookup plugin which adds 'Bob' to a list if 'Bob' wasn't there already: https://github.com/felixfontein/ansible-versioning_test_collection/blob/main/plugins/lookup/bob.py 15:13:02 now *that* is commitment 15:13:17 aahahaha!!! 15:13:31 well, I had to test the new plugin detection ;) 15:13:46 ;-) 15:14:00 anything else on changelogs or porting guides? 15:14:50 #topic docs pipeline 15:14:56 don't think so 15:15:20 thanks abadger1999 for https://github.com/ansible-community/antsibull/pull/92 15:15:49 No problem. 15:16:06 I believe https://github.com/ansible-community/antsibull/issues/52 was the last priority issue, is that true? 15:16:09 Lots of porting work to do to the docs themselves to finish that off now ;-) 15:16:11 you mentioned things had to be fixed now in some modules (or something like that) can you elaborate? 15:16:27 for the M() change 15:17:05 smSure. 15:17:33 In ansible/ansible 2.9, documentation uses, for instance: `M(yum)` 15:17:33 ah so it's what used to be M(module) is now M(ansible_collections:community.general.modulename)? 15:17:46 Now you need `M(ansible.builtin.yum)` 15:17:47 Yep. 15:17:56 with or without `ansible_collections:`? 15:18:04 Without `ansible-collection:` 15:18:17 ok so for `ansible-base` it is `ansible.builtin.yum` but for a collection, it is ..what? 15:18:49 For non-ansible base it would be like: `M(acme_account)` => `M(community.crypto.acme_account)` 15:19:14 ah so nobody needs `ansible_collections`. 15:19:20 same thing for `seealso`, right? for example https://github.com/ansible-collections/community.general/blob/f9589d78a8f121d73186da703521ae6f8b3500c4/plugins/modules/database/postgresql/postgresql_ping.py#L39 15:19:21 Correct. 15:20:02 acozine: Correct. That will need to be changed to `community.general.postgresql_info` 15:20:28 So the instructions are : for `ansible_base`, change M(foo) to M(`ansible.builtin.foo`) and for a collection, change it to M(FQCN.foo) ? 15:20:45 I ported community.crypto if you need examples: https://github.com/ansible-collections/community.general/blob/f9589d78a8f121d73186da703521ae6f8b3500c4/plugins/modules/database/postgresql/postgresql_ping.py#L39 15:21:35 wrong URL 15:21:39 abadger1999: that looks like a link to the old thing in community.general 15:22:02 https://github.com/ansible-collections/community.crypto/commit/cc45650e82b769125eb4f6dad5cb5de330be4501 15:22:04 Huh 15:22:10 Copy and paste not working for me today.... 15:22:22 Yep, that's it. 15:22:28 * abadger1999 will restart firefox after the meeting 15:23:02 #info change M() and `seealso:` - for `ansible_base`, change to `ansible.builtin.yum` for example. For a collection, use `.` 15:23:21 samccann: Yes, that's right (`ansible.builtin` is the official but hidden collection name for things in ansible-base) 15:23:28 #info see https://github.com/ansible-collections/community.crypto/commit/cc45650e82b769125eb4f6dad5cb5de330be4501 for an example 15:23:39 kewl 15:23:49 a lot of the M() and seealso links point to modules within the same collection, but some won't, so we should keep an eye out for those 15:24:12 #action document how M() and seealso now work in docstrings and highlight the need to make these changes for collection owners 15:24:55 is this for all M() uses? or only if it's in a different collection? 15:25:07 all of them 15:25:21 acozine: Regarding Priority issues. Yes, I believe that was the last one.... There's other things that should get done before release but I don't think they're showstoppers. In particular I want to port away from ansible-doc as the parsing backend for a variety of reasons.... It currently doesn't handle deprecations and redirects from runtime.yml so I need to implement our own parsing backend to get that information. 15:25:29 #info this is for all occurences of M() and `seealso:` 15:25:39 most if not all of those community.crypto changes were for modules pointing at other community.crypto modules 15:26:52 abadger1999: excellent 15:26:57 abadger1999: that is also bothering me for antsibull-changelog... 15:27:16 samccann: It will need to be all instances: https://github.com/ansible-community/antsibull/issues/52#issuecomment-642721571 15:27:43 k thanks 15:27:52 I can change that behaviour if we want to re-examine the decision. 15:29:08 abadger1999: for ansible-base itself, scanning for modules/plugins is somewhat tricky, because https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/cache/base.py is not a plugin 15:29:15 felixfontein: Cool. I may implement parsing over the course of several PRs if I know you might want to work on some of it ;-) 15:29:15 no I agree with it thanks. just wanna get the docs right 15:29:37 samccann: Cool :-) 15:30:34 #action samccann - add details on M() and `seaalso` work to gundalow's magic list of all things that collection owners need to do https://github.com/ansible-collections/overview/issues/45 15:30:39 abadger1999: I'm definitely interested! 15:31:39 felixfontein: Cool. I'll try to get the old code from ansible/ansible:hacking/[..]/plugin_formatter.py into a PR... it might not work out of the box but it can be a starting point. 15:31:43 we've inched past the 1hr mark 15:32:01 again! 15:32:25 (in particular, the things nitzmahone has implemented for PluginLoader/CollectionLoader should allow for some things to be implemented much better) 15:32:32 can someone remember the good old times when meetings were over after 30 mins or so because there was no topic left? :D 15:32:36 s/somethings/enumeration of the plugins/ 15:33:01 felixfontein: yes . . . but I doubt we'll see that again for a while now 15:33:23 that was fun in one way, this is fun in a different way 15:33:47 abadger1999: yep. it might also be good to have a common plugin/module enumeration API in ansible-base that's used by ansible, ansible-doc, antsibull-docs and antsibull-changelog 15:33:59 acozine: I doubt that too, and yes, it is! 15:34:44 felixfontein: YES! 15:34:50 * acozine quietly sings "Where have all the modules gone, long time passing, where have all the modules go-o-one, long time ago" 15:34:57 Hehe 15:35:09 lol 15:35:19 :D 15:35:40 Does that mean they're all coming back to ansible/ansible in a few years? ;-) 15:35:51 we only have one thing left on the agenda and it should be quick, do we wanna cover it and finally reach the end of the agenda in one meeting?? 15:36:00 samccann: sounds good to me 15:36:01 abadger1999: is someone taking bets on the year? 15:36:14 we could start a pool 15:36:20 (betting pool) 15:36:28 #topic https://github.com/bcoca/collection/issues/3 15:37:01 so the gist of it is, bcoca has a test collection, and wouldn't it be nice if we had it someplace more formal, eh? 15:37:43 I like the idea of having a sample collection 15:37:44 So I love the idea of having a good solid 'example' collection that we build on. 15:38:09 yes. 15:38:14 we should probably merge elements of bcoca's collection, felixfontein's test collection, and maybe others 15:38:23 well not really 15:38:39 we still need the 'dumping ground of all things we need to test against' collection 15:38:59 but we also need a clean 'this is an example of a really good collection' so developers can learn from that 15:39:04 samccann: so you envision two different sample collections, one for "show" and one for testing? 15:39:08 yep 15:39:16 one for learning, one for testing 15:39:43 which doesn't say we can't also consider moving felix's test collection into something more formal is that makes sense as well. 15:39:49 the testing one will contain a lot of ugly things I guess 15:39:57 yeah agreed 15:40:13 samccann: feel free to recycle parts (or most) of it! 15:40:17 should we do a POLL??? a POOLLLLLLLL! 15:40:25 lol 15:40:25 samccann: Yeah, I agree with your thinking. 15:40:42 heh, sure - what's the poll question? 15:40:50 POLL - should we move bcoca's collection into a showcase sample collection under `ansible-community` or some other formal place? 15:40:52 today someone was asking in #ansible-devel for a reference collection showing how to test modules/plugins/... 15:41:01 +1 15:41:02 +1 15:41:03 +1 15:41:05 +1 15:41:14 +1 15:41:16 #chair 15:41:16 Current chairs: abadger1999 acozine felixfontein geerlingguy samccann sivel tributarian 15:41:22 it does need more content though 15:41:46 oh.. did I just ping everyone w/ that? mostly just wanted to see who else was around before we close the poll 15:41:56 felixfontein: what are the first things you see we'd need to add? 15:42:10 samccann: yeah, the chair command pings all chairs 15:42:24 acozine: content :) there are mostly .keep files in bcoca's collection 15:42:25 #agreed - move bcoca's collection into a formal showcase collection 'somewheres'. It likely needs more content added. 15:43:01 the only proper files are the three in the repo's root I think 15:43:27 ah 15:43:29 we also have https://github.com/ansible-collections/collection_template 15:43:56 maybe merge them? 15:44:00 that is repo template, mine is 'full collection structure' 15:44:00 yeah that's the starting point so to speak. clone/copy the template and build up from that (for a new collection creator) 15:44:16 yeah so we may need to add more to the template over time as well. 15:44:19 Make sure to ping gundalow maybe too on that 15:44:23 i.e. there would be the approach "almost no directories - create what you want" (current template), "all directories - delete what you don't need" (bcoca's template) 15:44:32 I think there was also some sort of template he may have set up for community? 15:44:40 #info consider if https://github.com/ansible-collections/collection_template needs more to it as well. 15:44:45 felixfontein mine is not to use, but to document 15:44:59 ansible-galaxy init should be used to initialize collections, not the example 15:45:16 we don't have to merge them, but we should be sure they refer/link to each other in helpful ways 15:45:25 they are not same thing 15:45:26 yeah I don't see bcoca's as a template to copy, more of a full-ish example collection that a new person can learn from. 15:45:43 * geerlingguy afk 15:46:05 and we might want to give them a shared namespace to make them easier to find 15:46:15 ok so a new person would use the github template to create their repo, then the ansible-galaxy init to kickstart the collection structure in that repo... 15:46:53 anyway, we are 15 min over and we don't have to design the whole thing here. at least we agree on the direction 15:47:03 hooray! 15:47:06 https://github.com/ansible-collections/collection_template is what I clone when making new repos, such as community.sops 15:47:22 open the floor and we will have completed our agenda in less than 90 minutes!!! 15:47:27 oooh! 15:47:35 # topic open floor 15:47:58 if you've been lurking and have a question or comment, now's your chance~ 15:48:17 #topic Open floor 15:48:30 oh, no chair 15:48:37 #topic open floor 15:48:40 :) 15:48:41 thanks gundalow 15:49:01 as always, agenda items for next meeting are welcome at https://github.com/ansible/community/issues/521 15:49:06 * samccann floor opens under her feet, she gets sucked into a dark swirling abyss ...aAAAAIIIIIIGGGGHHH 15:49:54 sorry gundalow - we only made you a #footstool at the start 15:50:16 chat is always encouraged in this space, but the DaWGs meetings are the times when we come together and focus on issues and projects 15:50:24 samccann: :) 15:50:55 all are welcome, and docs is an easy way to get involved in Ansible 15:51:36 #info all are welcome, and docs is an easy way to get involved in Ansible. Add agenda items for the next meeting at https://github.com/ansible/community/issues/521 15:51:59 no open floor items today . . . 15:52:30 thanks abadger1999 acozine bcoca felixfontein geerlingguy gundalow samccann sivel tributarian 15:52:38 #endmeeting