14:31:26 <samccann> #startmeeting DaWGs aka Docs Working Group
14:31:26 <zodbot> Meeting started Tue Oct  1 14:31:26 2019 UTC.
14:31:26 <zodbot> This meeting is logged and archived in a public location.
14:31:26 <zodbot> The chair is samccann. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:31:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:31:26 <zodbot> The meeting name has been set to 'dawgs_aka_docs_working_group'
14:31:35 <samccann> who's around to talk the docs?
14:32:08 <samccann> cyberpear bcoca felixfontein ??
14:32:30 <cyberpear> Hi
14:32:43 <samccann> #chair cyberpear
14:32:43 <zodbot> Current chairs: cyberpear samccann
14:32:59 <samccann> hope to discuss the filters page if we have a few more peeps around today!
14:33:53 <samccann> #topic Filter page docs discussion
14:34:09 <samccann> #link https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html
14:34:20 <samccann> do you want to give a summary of your thoughts cyberpear ?
14:36:32 * acozine waves
14:36:46 <samccann> o/
14:36:50 <samccann> #chair acozine
14:36:50 <zodbot> Current chairs: acozine cyberpear samccann
14:36:57 <samccann> welcome to the furniture stage!
14:36:59 <bcoca> sorry, thisis my 3rd conflicting meeting, will try to pay attention ... but split
14:37:47 <samccann> ok thanks bcoca - might try to pick your brains on the filter page docs later as well, to understand what happened back when you wanted to include docstrings with the filter code
14:38:17 <acozine> first thing I see about the filters page is how long it is
14:38:30 <acozine> are there sensible ways to "chunk" it better?
14:38:41 <cyberpear> (definitely needs improvement generally... whenever I need more than the surface-level functionality of any filter, I have to dig out the source to see what's actually happening.)
14:38:46 <samccann> The original comments from cyperpear suggests the page is too long, hard to search, etc
14:39:16 <samccann> and bcoca suggested he'd tried to have docstrings embedded with the code, but ran into difficulties (as in someone didn't like that approach??)
14:39:24 <acozine> Pilou shaps Xaroth andersson007_ who's got ideas about documenting filters?
14:39:27 <cyberpear> yes, also hard to search, and has bad SEO, probably... even if google brought you to the page, you would have a hard time to find the details that are present
14:39:36 <bcoca> i would try to modle pages after the other plugins, 'main paige' that explains general part and link to each with specific docs/examples
14:39:49 <cyberpear> examples++
14:39:52 <samccann> I did a quick check, and the filters page is our 17th most popular page, with 5% of our visitors. So pretty important
14:40:30 <samccann> #info filter docs page is too long, bad SEO experience since cannot search on an individual filter, and lacks examples
14:40:34 <cyberpear> almost needs a structure similar to modules structure, though less involved
14:40:45 <bcoca> filters is 90% of how you manipulate data in ansible .. so expected
14:41:08 <acozine> it seems like we mix in information about specific filters with more general information about how Ansible uses filters
14:41:09 <bcoca> filters are in diff files ... so they can be used as 'categories' if you must
14:41:13 <cyberpear> (side note: can we register dict() as a filter for use w/ map?)
14:41:29 <samccann> #info we could model the filters after the plugins - have a 'main page' that explains in general and then link to each with specific docs/examples
14:41:59 <bcoca> cyberpear: i believe we created dict filter specifically for something like that
14:42:05 <bcoca> or was it 'global' ?
14:42:14 <acozine> bcoca: where do the filters live in the codebase?
14:42:16 <cyberpear> it's only the built-in python dict(), and you have to wrap it around the thing you want to dict-ify
14:42:23 <cyberpear> so can't use it for filtering
14:42:35 <cyberpear> (but o/t, so will take to -devel)
14:43:43 <cyberpear> `lib/ansible/plugins/filter/`
14:43:53 <acozine> ah, thanks
14:44:00 <cyberpear> that's it no subdirs
14:44:24 <cyberpear> 7 files for chunking
14:44:56 <cyberpear> core ipaddr and network are the large ones
14:44:57 <acozine> those files are weirdly lopsided
14:44:58 <cyberpear> the rest small
14:45:00 <cyberpear> yeah
14:45:27 <samccann> ipaddr has its own largish documentation page as well
14:45:40 <felixfontein> hi
14:45:50 <acozine> welcome, Felix!
14:45:50 <samccann> https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters_ipaddr.html
14:45:57 <felixfontein> sorry for being a bit late
14:46:10 <samccann> so'k. still time to be furniture!
14:46:15 <acozine> no problem
14:46:19 <samccann> #chair bcoca felixfontein
14:46:19 <zodbot> Current chairs: acozine bcoca cyberpear felixfontein samccann
14:46:53 <acozine> felixfontein: thoughts on documenting filters?
14:46:59 <acozine> (better than we do now, that is)?
14:48:13 <acozine> the page could use a better title - if you know what a filter is, you can find it with no trouble, but if the word makes you think of water jugs, you might not realize just how useful they are
14:48:35 <acozine> something like `Transforming data with filters`, perhaps
14:49:39 <acozine> a bunch of them seem to handle transforming lists, maybe that could be a sub-heading
14:49:58 <acozine> `Filters for list data`
14:50:24 <bcoca> https://github.com/ansible/ansible/pull/46979
14:50:44 <bcoca> acozine: filters are for 'transforming data'  , most of the time tha tis  intended for looping over it
14:50:49 <bcoca> why lists are very common format
14:51:14 <bcoca> even those in original jinja map/select/reject/selectattr/rejectattr are list handlers
14:51:22 <samccann> #info wip pr adding more examples - #link https://github.com/ansible/ansible/pull/46979
14:51:38 <acozine> bcoca: excellent - the complex examples will make a great addition
14:51:48 <bcoca> ^ that is another issue, do we just document 'ansible ones' or also jinja2 (normally we refer to jinja2 docs .. but in most cases we do better docs)
14:52:01 <samccann> So on the main filters page - do we want to group by sections, or split off sections into suppages to help search results?
14:52:19 <bcoca> we already have sections .. like 'set theory'
14:52:48 <acozine> I think separate pages would help
14:52:55 <acozine> less scrolling
14:53:06 <acozine> but only if we can create sensible groups of filters
14:53:34 <bcoca> i wanted to make them like rest of plugins, but i can also see 'module like' index pages by  category 'list management' , 'dates', networking' etc, which can have same filter in more than one list
14:53:56 <acozine> oh, that's a really interesting idea
14:54:17 <bcoca> been wanting to do that for 4yrs ... but tardis is broken
14:54:21 <acozine> heh
14:54:27 <samccann> yeah the idea behind sensible groupings is a search result on say network filters would bring me to a page that has maybe 1/2 dozen filters instead of this page with a zillion
14:54:31 <bcoca> and clones jsut run off to play and wont work
14:54:44 <samccann> bad clones
14:55:14 <acozine> I have the same trouble with my overnight worker threads - my brain just keeps sending them off to dream instead of getting work done
14:55:35 <acozine> I think we have consensus on 2 things:
14:55:43 <acozine> 1. the current filter docs need work
14:56:07 <acozine> 2. the current page needs to be broken up better
14:56:11 <bcoca> ^ also 'tests' but less priority
14:56:28 <samccann> #info 'tests' have a similar problem to the filters page
14:56:36 <bcoca> the thing is that they both mostly work the same (loading, defintion, etc, since they are jinja2 features we expand on)
14:57:55 <acozine> bcoca: the sanity test pages are already separate
14:58:18 <acozine> https://docs.ansible.com/ansible/latest/dev_guide/testing/sanity/index.html#all-sanity-tests
14:58:28 <bcoca> unrelated
14:58:32 <bcoca> jinja2 tests != sanity tests
14:58:34 <samccann> #agreed - consider breaking up the filters page into high level groupings with more examples for each filter type
14:58:47 <acozine> ah, I see, you mean jinja tests as in conditionals
14:59:05 <bcoca> https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html
14:59:26 <acozine> that page definitely needs a better title
14:59:30 <samccann> #link https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html for jinja2 tests
14:59:33 <bcoca> its confusing
14:59:35 <acozine> because `Tests` could mean so, so many things
14:59:49 <bcoca> plugins/tests would be 'clearer'
14:59:54 <bcoca> playbook-tests is misleading
15:00:01 <bcoca> playbook_jinja2_tests ... might be better
15:00:19 <bcoca> buti would argue same for filters (though we have less overlap)
15:00:44 <acozine> the file name is bad enough, but most people don't read those . . . the title on the page itself is confusing
15:01:13 <samccann> tossing in the analytics here - tests page gets about 1.5% of our docs visitors (unique visitors) vs 5% for the filters page
15:01:30 <acozine> I suspect if we give it a better title, it will get more traffic
15:01:50 <acozine> but filters are a higher priority for sure
15:02:30 <felixfontein> sorry, got side-tracked
15:02:59 <acozine> the meeting is half-over, shall we move on to the next topic? I can have a WIP rewrite of the filters page by next week, maybe sooner
15:03:11 <acozine> unless someone else is eager to take that on?
15:03:36 <felixfontein> one comment on the filters page: I always wonder whether these are just examples, or whether they cover all cases
15:04:06 <acozine> all filters should be represented on that page
15:04:14 <acozine> but we should double-check that they actually are
15:04:28 <acozine> since I've been here, we've required docs with all new filter code
15:04:38 <samccann> i think it's a manual addition but I could be mistaken
15:04:57 <samccann> (as in the coder has to manually remember to add to that page, vs the automagic docstrings we have elsewhere)
15:05:16 <acozine> yes, the docs are directly in rst, not generated from the code
15:05:38 <samccann> yeah so room for error/missing filters
15:05:42 <acozine> yep
15:07:16 <acozine> we have two other items on the agenda for today
15:07:32 <acozine> plus anything folks want to add "live"
15:07:44 <samccann> should we hop on Felixfontein  topic first?
15:07:48 <acozine> sounds good
15:08:29 <samccann> #topic support elements in return values in documentation
15:08:53 <samccann> sample PR - https://github.com/ansible/ansible/pull/62929
15:09:01 <samccann> #link https://github.com/ansible/ansible/pull/62929 for sample PR
15:09:15 <samccann> want to take it from there felixfontein ?
15:10:21 <acozine> felixfontein: you still around?
15:10:26 <acozine> squirrel?
15:11:26 <acozine> I think we lost him . . .
15:12:20 <samccann> I think the general thought was - we now support `elements` for `lists` but not within the return values in documentation.
15:12:40 <samccann> from the 3k don't know what I'm talking about perspective, I like the idea
15:13:09 <samccann> we'd want to test it heavily to make sure funky things don't start showing up in other modules once it's working
15:13:36 <acozine> right - so we could update the return docs to show which complex values are dicts of lists, which are lists of dicts
15:14:20 <acozine> speaking as someone who has struggled into the wee hours trying to get the right data subfield from a registered variable to show up in a later task or template, I'm +1 to the idea
15:15:05 <acozine> it's related to data manipulation
15:15:11 <acozine> which seems to be our theme for this week
15:15:12 <samccann> ok looked more closely at the PR, I don't think it changes code, just adds the `elements` to the doc return values. so definitely +1 from me on this
15:15:26 <samccann> once we generate the output to make sure it looks okay
15:16:17 <acozine> I think we'd need a companion PR to add the ability to generate these in the return docs
15:16:29 <acozine> right now I don't think that branch would generate correctly
15:17:00 <samccann> ah gotcha
15:19:11 <acozine> the code to add subelements to the option docs came from https://github.com/ansible/ansible/pull/59244/files
15:19:40 <felixfontein> sorry, was side-tracked again
15:19:56 <felixfontein> the branch definitely doesn't work, because I didn't modify the docs generation
15:20:11 <felixfontein> I just updated the module return values to use the new features which don't exist yet :)
15:20:31 <acozine> samccann and I both like the idea . . . are you interested in doing the docs generation work felixfontein ?
15:20:41 <cyberpear> (also had a side meeting...)
15:20:55 <acozine> everybody gets a second meeting!
15:21:15 <acozine> you get a second meeting, you get a second meeting . . .
15:21:27 <felixfontein> I'll try to implement that, I guess then we can discuss it again :)
15:21:37 <felixfontein> just wanted to check whether there are reasons why that shouldn't be done or something like that
15:21:41 <acozine> felixfontein: that would be AWESOME
15:22:13 <acozine> the only objection I can think of is that we have a lot of documentation that could be updated with more detail, and it might take a lifetime for that to happen
15:22:24 <acozine> but I'm still in favor of making it possible
15:22:40 <acozine> maintainers who want to do it, should have the ability
15:22:46 <samccann> agreed
15:22:52 * cyberpear has filters comments once open floor
15:23:07 <samccann> #agreed worth adding support for elements in return value docs
15:23:13 <acozine> we don't test the return docs for completeness or correctness, so it shouldn't break anything
15:23:48 <acozine> anything else on the topic of allowing more detailed return value documentation?
15:24:10 <acozine> "hearing" none . . .
15:24:18 <acozine> #topic open floor
15:24:37 <acozine> cyberpear: you're up
15:24:48 <cyberpear> I'd say to document both jinja-native filters and also ansible-specific ones, and clearly delineate which is which.
15:25:15 <acozine> that's a good distinction for our docs to make
15:25:19 <cyberpear> obviously give higher focus to ansible specific ones, though
15:25:39 <cyberpear> but the native ones can be just or more useful
15:25:54 <samccann> hmmm
15:25:57 <acozine> we'll certainly keep docs for any jinja-native filters we already document, and accept docs for additional ones
15:26:12 <acozine> but we can't promise to document them all
15:26:18 <samccann> yeah what she said
15:26:49 <cyberpear> fair enough... perhaps their use in examples could link to upstream docs
15:27:12 <acozine> that might be a good focal point for a "docs day", too - everybody look through their playbooks for filters they use that aren't documented on docs.ansible.com
15:27:36 <acozine> cyberpear: yes, we can definitely link to the jinja docs in a bunch of places
15:28:12 <acozine> yikes, we've only got two minutes left
15:28:24 <samccann> oo I like the 'beef up your favorite ansible section' docs day
15:28:52 <acozine> Doktoberfest?
15:29:01 <samccann> could be!
15:29:01 <cyberpear> (that's all I had)
15:29:08 <acozine> cyberpear: thanks!
15:29:38 <acozine> samccann: do you want to talk about adding PR numbers to changelog fragments?
15:29:44 <acozine> or shall we save that for next week?
15:30:07 <samccann> save for next week. I can rattle a few cages before then to be sure they come to voice their nickel next week
15:30:19 <acozine> heh, sounds good
15:30:26 <acozine> and we're exactly at time
15:30:49 <acozine> thanks cyberpear bcoca felixfontein
15:31:13 <acozine> remember, agenda items welcome any time at https://github.com/ansible/community/issues/389
15:31:23 <acozine> #endmeeting