14:31:10 <acozine> #startmeeting Docs Working Group aka DaWGs
14:31:10 <zodbot> Meeting started Tue May 19 14:31:10 2020 UTC.
14:31:10 <zodbot> This meeting is logged and archived in a public location.
14:31:10 <zodbot> The chair is acozine. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:31:10 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:31:10 <zodbot> The meeting name has been set to 'docs_working_group_aka_dawgs'
14:31:29 <acozine> good morning, good afternoon, good evening, according to your time zone!
14:31:38 <acozine> who's around to talk docs today?
14:32:26 <acozine> just me, huh?
14:32:28 <felixfontein> hey!
14:32:29 <abadger1999> Good morning :-)
14:32:30 <felixfontein> me too :)
14:32:42 <felixfontein> good morning/day/afternoon/evening/night everyone!
14:32:44 * samccann waves after staring in the wrong channel wondering why this hadn't started yet
14:32:48 <acozine> welcome felixfontein abadger1999 samccann
14:32:49 <acozine> heh
14:32:57 <acozine> #chair felixfontein abadger1999 samccann
14:32:57 <zodbot> Current chairs: abadger1999 acozine felixfontein samccann
14:32:59 <abadger1999> Hee hee, that's my usual tactic :-)
14:33:13 <samccann> I promised we'd start with a docs pipeline update
14:33:22 <acozine> #topic docs pipeline update
14:33:36 <samccann> go for it abadger1999
14:33:46 <abadger1999> Cool, I prepared a wall of text for you all.  I'll paste that if you don't mind, andthen you can pepper me with questions.
14:33:55 <abadger1999> Pipeline updates:  I encountered some bugs in ansible-doc last week.  The word from the developer is that the bugs won't be fixed so I'll need to change the pipeline code to use its own parser.  (I'll be able to mostly steal it from the current pipeline code).
14:34:06 <abadger1999> I also started experimenting with writing a Schema for all of the documentation over the weekend.  That went surprisingly well.  Yesterday as I was trying to port all the normalization rules from the former docs pipeline over, I realized that it would be easier to finish the schema rather than trying to work out all the specialcases encoded in the code.
14:34:11 <abadger1999> This is the meat of what I have so far: https://gist.github.com/abadger/5d7ad556052758993b3245f9dc14a1d9
14:34:19 <abadger1999> The schema library I'm using, pydantic, is optimized for conversion: taking your data and converting it into the types and format you specify which lets us be forgiving of various typos and legacy fields that are in the present docs.  It can also spit out a json-schema which I believe is very strict.  So I think we can have ansible-test test things with the json-schema to tell people whether there docs follow "the
14:34:19 <abadger1999> specification" (We don't presently have a specification... it's just, "does it parse?" [which might not even display]).  We'll have to discuss whether to do that later, though, as there are many things that will fail the strict json-schema.
14:35:27 <acozine> abadger1999: this is awesome
14:35:44 <felixfontein> ansible-test currently uses another schema library
14:35:59 <felixfontein> (I always forget its name)
14:36:04 <acozine> felixfontein: for docs sanity testing?
14:36:05 <samccann> #info ansible-doc can't solve what the docs pipeline needs so developing a separate parser with a docs schema. See https://gist.github.com/abadger/5d7ad556052758993b3245f9dc14a1d9 for work thus far
14:36:07 <jtanner> voluptous
14:36:16 <acozine> welcome jtanner
14:36:19 <acozine> #chair jtanner
14:36:19 <zodbot> Current chairs: abadger1999 acozine felixfontein jtanner samccann
14:36:21 <felixfontein> jtanner: yes, that one
14:36:35 <jtanner> i'm sure i spelled it incorrectly
14:36:36 <felixfontein> acozine: for docs and argspec
14:36:46 <abadger1999> Working on the schema also leads me to find tons of questionable choices in the present docs. (For instance, callbacks have a name field on options [like a short_description field).  No other plugins have that and anywhere else, the concept is called short_description
14:36:55 <abadger1999> uous
14:36:56 <jtanner> we use voluptouous to validate botmeta as well in sanity
14:37:00 <felixfontein> jtanner: I only recognize that it is close enough to what I think it should be, I'm bad at remembering its spelling (or its name in general)
14:37:32 <abadger1999> "v8s" ?
14:37:46 <felixfontein> ah, btw, ansible-test currently *only* validates modules, not other plugins
14:37:48 <acozine> abadger1999: +1 for v8s
14:38:02 <felixfontein> v8 reminds me too much of google's(?) javascript engine
14:38:08 <abadger1999> Heh.
14:38:10 <samccann> #info this is using pydantic for the schema. fyi docs sanity test uses volumptuous
14:38:51 <acozine> pydantic is a much better name, easier to remember because it describes what it does (be pedantic with python)
14:39:04 <abadger1999> <nod>  And if we add it to ansible-test, we'd probably be using json-schema (generated by the pydantic schema)
14:40:10 <samccann> so in the future we may try to get them both using the same thing? (aka use pydantic instead of voluptuous?)
14:40:38 <felixfontein> depends on how flexible pydantic is
14:40:55 <felixfontein> v8s allows a lot of nice things, like using callbacks for validating specific fields
14:41:24 <abadger1999> I think it would be nice if the pydantic schema was canonical.  But I'm not sure if ansible-test should use pydantic directly or use jsonschema (which the pydantic schema can generate automatically)
14:42:07 <acozine> abadger1999: so, in layperson's terms, at first we would use pydantic to create the schema and the pipeline would parse existing docs and modify them to match the schema; over time we could work to make the docs themselves match the schema?
14:42:33 <samccann> #info json schema from this may be helpful in the future for developing stricter testing on the module docs.
14:42:41 <abadger1999> felixfontein: Yep, doable in pydantic as well.  I can point out those features in the gist now or after the meeting depending on what everyone else is interested in.
14:42:54 <abadger1999> @acozine Correct.
14:43:05 <acozine> I really like this approach
14:43:11 <felixfontein> abadger1999: I guess not now, there's too much things to talk about :)
14:43:18 <abadger1999> Cool.
14:43:40 <samccann> feel free to discuss at a timeline that works for the both of you. The rest of us can play catchup later as needed
14:43:44 <acozine> it gives us a working pipeline and also a roadmap to better performance through rule-matching
14:43:57 <felixfontein> (also I guess it's too late in the 2.10 dev cycle to replace v8s with pydantic right now)
14:44:08 <abadger1999> (The reasons to prefer the json-schema over the raw pydantic schema: pydantic requires python-3.6+.  The jsonschema is stricter than using pydantic directly... For testing purposes, you generally want stricter)
14:44:36 <abadger1999> Yeah.  People would be upset if ansible-test sanitystarts failing for them now.
14:45:00 <acozine> suddenly failing ansible-test sanity would be a surprise, and not the good kind
14:45:09 <felixfontein> also I would be upset if I have to rewrite all that validation code in my PR(s) :)
14:45:15 <acozine> heh
14:45:19 <abadger1999> hee hee.
14:45:28 <felixfontein> (like, rewrite right now)
14:45:58 <acozine> yeah, now is not the time - we have a lot going on already
14:46:09 <felixfontein> yes
14:46:35 <acozine> abadger1999: what can we do to support your work here?
14:47:31 <acozine> run tests? fix a subset of module/plugin docs? stay out of your way?
14:48:49 <abadger1999> Hmmm.... I'm not sure how much there is for the docs team right this instant... I can show you some of the things I'm thinking the strict-schema should enforce but since we aren't going to make use of that for the 2.10 cycle, it's definitely not necessary.  From felixfontein... if you want to learn how to release new versions of the community collections from gundalow, that could help out.
14:49:36 <abadger1999> If not, gundalow is looking into automating the release process via zuul... so in a few weeks, that process might be simpler.
14:49:49 <samccann> that'd be cool
14:50:07 <felixfontein> abadger1999: I guess right now I would need to get hold of the token somehow?
14:50:15 <abadger1999> The present community collections have some documentation bugs that the schema is finding.
14:50:29 <samccann> abadger1999: when you get a chance, can you check the project board to see if any of the dependencies change based on this approach?  - https://github.com/ansible/ansible/projects/42
14:50:33 <abadger1999> felixfontein: maybe ;-)  gundalow has been handling that 100% so far.
14:50:41 <abadger1999> Ahh...
14:50:42 <acozine> do you have a list of the docs bugs?
14:50:46 <abadger1999> Yes, a lot of them do.
14:50:48 <felixfontein> abadger1999: you mean the present *published* community collections, or the current master branches in the repos?
14:51:10 <felixfontein> abadger1999: I know that he started getting zuul set up (I've seen some PRs for that yesterday)
14:51:25 <cbudz_> :waves
14:51:30 <cbudz_> I'm half here, sorry
14:51:30 <abadger1999> So a question from me would be, are we ready to commit to not using ansible-doc for the 2.10 cycle?  If so we can remove the ansible-doc cards from the project.
14:51:31 <acozine> welcome cbudz_
14:51:34 <acozine> #chair cbudz_
14:51:34 <zodbot> Current chairs: abadger1999 acozine cbudz_ felixfontein jtanner samccann
14:51:53 <abadger1999> (We'll still need the routing PR but the ansible-doc work that was waiting o nthe routing PR won't be.)
14:52:15 <abadger1999> felixfontein: present published.
14:52:22 <felixfontein> what do you mean by not using ansible-doc for the 2.10 cycle?
14:52:29 <felixfontein> I'm a bit confused now
14:52:53 <acozine> not using the command-line `ansible-doc --json` to output docstrings for module/plugin docs
14:52:57 <abadger1999> So right now the code I have uses ansible-doc --list --json and ansible-doc --json $PLUGINNAME to actually parse the plugins for their documentation.
14:53:03 <felixfontein> ah
14:53:36 <abadger1999> Due to bugs and worries about the timeframe for other work being done for that, I think we need to use something based on the current website build's parsing code instead.
14:53:39 <felixfontein> I'm fine with not using it, or using it only to get hold of the filenames or something
14:53:40 <samccann> #info this approach would mean we no longer use ansible-doc --json to output docstrings for module/plugin docs
14:53:47 <acozine> abadger1999: I think we are ready to commit to not using ansible-doc for 2.10 - it won't do what we need right now
14:53:54 <abadger1999> Cool.
14:54:00 <abadger1999> I'll update the project board.
14:54:14 <samccann> I think the drawback is we would be using something that AH and ansible-test does not use (yet)
14:54:26 <abadger1999> samccann: Do you want the ansible-doc cards and issues put into some holding column (re-evaluate for 2.11) or jsut get rid of them?
14:54:28 <felixfontein> oh - I overlooked the link to the project board. that would have made me less confused ;)
14:54:38 <acozine> we can still set that as a long-term goal
14:55:03 <samccann> abadger1999: archive them pls
14:55:12 <abadger1999> Roger dodger.
14:55:19 <samccann> I'm assuming there's some way to find things in the archive in some future date :-)
14:55:42 <felixfontein> or add another column "archive" and move them there ;)
14:55:46 <acozine> heh
14:55:49 <samccann> #action abadger1999 to archive ansible-doc cards in the project board
14:56:46 <samccann> #action samccann to create a 2.11 column for evaluating next steps with this approach (aka integrating the schema json into ansible-test for docs, passing the approach on to AH to consider etc)
14:56:55 <abadger1999> Cool, done.
14:56:59 <samccann> woot!
14:57:26 <acozine> anything else on the pipeline?
14:57:32 <acozine> are we ready to pick up another topic?
14:58:22 <samccann> sounds good to me
14:58:31 <felixfontein> LGTM
14:59:07 <abadger1999> Ready to move on :-)
14:59:29 <acozine> #topic https://github.com/ansible/ansible/pull/69268
14:59:40 <felixfontein> so, this is about version_added for collections
14:59:42 <acozine> hm, that's not actually a good topic for folks looking at the records
14:59:44 <felixfontein> ah wait
15:00:21 <acozine> felixfontein: this is the PR you linked to in the agenda, is it the one you meant to link to?
15:00:34 <felixfontein> lol
15:00:37 <felixfontein> I totally confused PRs
15:00:56 <felixfontein> I wanted to link that one too, but then I thoguth I linked to another one and started adding more text for the other one
15:00:58 <acozine> #topic version_added in collections
15:01:12 <acozine> what's the PR or issue you meant for this topic?
15:01:18 <felixfontein> https://github.com/ansible/ansible/pull/69291
15:01:37 <felixfontein> the changelog PR is also important, but not that important IMO :)
15:01:44 <acozine> thanks
15:01:49 <samccann> #link https://github.com/ansible/ansible/pull/69291
15:02:01 <felixfontein> ok. version_added again.
15:02:13 <acozine> okay, so this is about making it possible to test the `version_added` entries in collections the way we used to test them in ansible/ansible
15:02:17 <felixfontein> I was working on the version_added validation for collections in ansible-test over the weekend, when I found an interesting problem
15:02:39 <felixfontein> namely doc fragments, that could be imported from other collections (and from ansible.builtin = ansible-base)
15:02:59 <felixfontein> for example there's the `files` doc fragment, which has options added f.ex. in Ansible 2.3, so they have `version_added: 2.3`
15:03:24 <acozine> cross-collection doc fragments seem tricky in a lot of ways
15:03:52 <felixfontein> if now a module in community.crypto uses the `files` doc fragment (like openssl_privatekey), it will have `version_added` entries which talk about the community.crypto version, but also the `version_added` entries from `files` which talk about Ansible releases
15:04:09 <felixfontein> so simply comparing `version_added` version numbers to the current collection won't work
15:04:32 <felixfontein> also, the problem exists for removed_in_version in the argspec, which could come from module_utils from another collection (or from ansible-base)
15:04:33 <samccann> ooch
15:04:42 <felixfontein> yeah. fun :)
15:04:53 <felixfontein> mattclay, abadger1999 and me were talking about this on saturday
15:04:54 <acozine> ETOOMANYVERSIONOPTIONS
15:05:19 <felixfontein> one solution for this problem is to not have version numbers, but version numbers with more information, i.e. which collection they refer to
15:05:26 <acozine> what would "the right balance" look like?
15:05:30 <felixfontein> for version_added, which comes from doc fragments, this can be done automatically
15:05:31 <samccann> #info problem occurs when a collection includes a doc_fragment from ansible-base. The collection has version_added for collection version, while doc_fragment reflects ansible-base version
15:05:53 <felixfontein> I've tried this out in https://github.com/ansible/ansible/pull/69291, it adds `collection:` in front of the version number
15:06:33 <felixfontein> this affects also ansible-doc, which will now show where the version belongs to (I added code to strip this tag for the current collection, so version_added which come from the collection the module/plugin is part of won't be shown)
15:06:57 <felixfontein> https://github.com/ansible/ansible/pull/69291#issuecomment-630409820 contains examples of how this looks with ansible-doc
15:07:42 <felixfontein> this approach allows people to simply use version_added as before, and the collection it refers to will be added automatically
15:08:00 <felixfontein> HTML/rST docs could of course show this information in a nicer way :)
15:08:09 <felixfontein> (ansible-doc could too)
15:09:34 <samccann> ok so as a collection owner, I use 'version_added' 1.0.0. , which is my collection version number.
15:09:40 <felixfontein> of course it is best if cross-collection fragments are avoided (I wrote up something about that here: https://github.com/ansible/community/wiki/Version-numbers-for-documentation-deprecation-removal-in-the-collection-world), but in cases like `files` it is hard to avoid
15:09:46 <felixfontein> samccann: yes
15:09:50 <acozine> so this is two things in one PR: testing to make sure new features in collections get accurate `version_added` metadata, and parsing to differentiate `version_added` in collections from `version_added` from ansible-base (mostly older versions)
15:09:54 <samccann> and as an ansible-base coder, I use the same field, for ansible.builtin version
15:10:04 <felixfontein> yes
15:10:14 <samccann> is that correct? and your pr magically adds the builtin vs collection to the output?
15:10:29 <felixfontein> and then there's magic in the doc fragment combiner, and the end result has the collection name inserted before the version number
15:10:36 <felixfontein> yes :)
15:11:26 <felixfontein> (there are some more complicated issues with removed_in_version and deprecated_aliases.version, but these don't affect docs since that information is not shown in the docs)
15:11:37 <acozine> do we ever want to sunset old `version_added` metadata? is there a point at which we say `version_added: 2.3` is irrelevant?
15:11:43 <samccann> #info collections will use version_added to reflect collection version. ansible-base use it to reflect ansible-base version. The PR then parses to add 'built in' vs 'collection' to the version number to clarify for users. (and tests collection version # )
15:12:11 <felixfontein> acozine: I would prefer if version_added is kept and is actively used; it is good to know which version of ansible / collection you need to install to get a feature
15:12:17 <samccann> we have code to hide it in the docs build, for modules/plugins only afaik.  So it will still show up for .rst files that added it
15:12:37 <felixfontein> samccann: I think that code mainly affects too old versions (or is there something new?)
15:12:38 <samccann> (at this point I think we hide up to 2.5? I'll have to check)
15:12:45 <samccann> yes too old versions, thanks
15:13:20 <samccann> each release we bump up the 'too old version' so now I think we are at 2.5.
15:13:35 <abadger1999> samccann: The hiding is going away (at least for non-ansible.builtins plugins.  I'd have to special case ansible.builtins to put it back there)
15:14:14 <acozine> well, if we can't/don't want to simplify the problem, this sounds like a great solution - clear for the user and helpful for testing
15:14:17 <samccann> abadger1999: so will something like version added 2.1 start showing up again? (aka really old)?
15:14:25 <abadger1999> maintainers can change the source code, though.
15:14:28 <abadger1999> samccann: yes.
15:14:41 <abadger1999> Unless I special case ansible.builtins to do something to hide that.
15:14:59 <acozine> I think that's low priority for now
15:14:59 <samccann> meh. we can deal with it after 2.10
15:15:03 <felixfontein> hiding isn't hard to implement, but it isn't high on the priority list I guess :)
15:15:07 <abadger1999> Okay, cool.
15:16:01 <felixfontein> I guess my main question for the docs team is: do we still want to use version_added for collections (I hope the answer is yes ;) ) and show the extra information (where version_added is from) to the user?
15:16:08 <samccann> #action samccann to add card to docs pipeline board for future work - consider whether we need version too old back again
15:16:29 <samccann> felixfontein yes I like that it will show which version we are talking about!
15:16:53 <acozine> felixfontein: yes, we still want to use version_addd
15:16:58 <acozine> added
15:17:07 <felixfontein> good :)
15:17:08 <acozine> I think it will be even more important for collections
15:17:14 <samccann> I'm assuming no extra work for collection owners at this point if they are already using it for their collection version
15:17:24 <acozine> than it was for batteries-included ansible
15:17:27 <samccann> but what about those that have no versions at all? I 'think' it was stripped out during migration?
15:17:48 <felixfontein> I've also put this PR up for the public project meeting later today
15:18:08 <felixfontein> samccann: they have been there since the beginning of the collection
15:18:15 <acozine> samccann: I lost the thread, what would had versions stripped during migration?
15:18:24 <felixfontein> (or not that long, depending on whether collection authors add version_added or not)
15:18:39 <felixfontein> acozine: (almost) all version_added were stripped during migration
15:18:48 <felixfontein> (only return values kept theirs)
15:19:11 <felixfontein> (not intentionally)
15:19:13 <acozine> felixfontein: do the tests enforce "you must add `version_added` for all new features" or only "if you add `version_added` it must match the version of the collection"?
15:19:18 <samccann> yeah so community.general has modules with no version_added.  What has to happen there?
15:20:01 <acozine> if we're enforcing "you must add `version_added`, then we could have a special value for "this existed at migration"
15:20:08 <felixfontein> acozine: the tests I implemented allow both: if you add `--collection-version_added` when calling `ansible-test sanity`, it will enforce that new options get `version_added` set correctly (like in ansible-base)
15:20:10 <acozine> jeez, i can't type today
15:20:39 <felixfontein> acozine: if you don't specify `--collection-version_added`, it will only do sanity checks (like make sure that `version_added` for collections conforms to semver version syntax)
15:21:05 <felixfontein> i.e. for existing options (and modules/plugins) not having version_added is always fine
15:21:41 <felixfontein> the enforcement only happens when `--collection-version_added` is added, and when ansible-test can compare the current branch with a master branch to detect what was changed in the current branch/PR
15:21:57 <mattclay> felixfontein: I'm still not sure that workflow makes sense (having the `--collection-version-added` option). I'm not sure how it would end up being used.
15:22:03 <samccann> #info specify --collection-version_added for ansible-test sanity to enforce new options get version_added set correclty
15:22:27 <samccann> mattclay - couldn't it be added to the CI tests for each collection that wants it?
15:22:48 <mattclay> Where would the version number be coming from?
15:23:08 <felixfontein> either you provide the next major/minor release in `--collection-version_added <verison>`, or you update galaxy.yml beforehand and just specify `--collection-version_added` (and it will use that version)
15:23:50 <felixfontein> mattclay: it depends on how the collection workflow looks like. I'd like for community.crypto and hopefully also for community.general and community.network the following approach: galaxy.yml always contains the *next* minor release version
15:24:08 <felixfontein> i.e. it is updated right after a release
15:24:26 <felixfontein> (essentially what is done in ansible-base's devel branch after a new release)
15:25:02 <acozine> so during a development cycle, in the code repo the `galaxy.yml` file notes the next minor release, and all devs use this to populate `version_added` for new features
15:25:02 <felixfontein> other collections either have to extend their workflow that the version is known  (and somehow pass it to --collection-version_added), or they don't use this test
15:25:31 <felixfontein> that's the main workflow I have in mind
15:25:32 <acozine> and updating that field in `galaxy.yml` is part of the collection release process?
15:25:38 <felixfontein> yes
15:25:46 <mattclay> felixfontein: If the next version number is known, why can't it always be in galaxy.yml?
15:25:47 <samccann> what would happen if community.network needed a patch release then? do you correct the version in galaxy.yml to match that patch release (which could be one less on the minor release version)?
15:26:18 <felixfontein> mattclay: I think it should always be in there. but some collections don't have a galaxy.yml until release time, then it's temporarily created and not checked in
15:26:47 <mattclay> felixfontein: In that case, why can't they generate galaxy.yml before running ansible-test?
15:26:53 <samccann> so 1.0.0 is released and galaxy.yml is set to 1.1.0 for the next minor release. But then 1.0.0 needs a patch.. someone has to remember galaxy.yml needs to be 1.0.1 not 1.1.1
15:27:05 <felixfontein> samccann: patch releases probably won't happen a lot, and if they do, there probably would be a new branch starting with the release commit, and that will be updated
15:27:16 <mattclay> Although I guess that doesn't really solve the problem.
15:27:39 <acozine> basically the collection workflow/release process would mimic the old ansible/ansible workflow/release process
15:27:54 <felixfontein> mattclay: because they release on tag creation, and the tag gives the new version number - so you only know the version number at the point of release
15:28:12 <acozine> with a known next version and stable branches for backports of important bugfixes
15:29:16 <felixfontein> I think the best workflow / release process also depends on the size and aim of the collection
15:29:43 <acozine> mattclay: are you thinking these tests are not necessary, not practical/achievable, something else?
15:29:51 <mattclay> If the version number isn't required to be present (in galaxy.yml or MANIFEST.json) then the tests that depend on it will be optional (requiring `--enable-optional-errors`), so it's going to be easy for people to forget (or not realize) they need an extra option to verify version numbers and the feature may go unused.
15:29:51 <felixfontein> community.general and community.network are closer to the complexity of ansible/ansible used to have, since there are many hundreds of modules with total different maintainer groups
15:30:27 <felixfontein> mattclay: it will definitely get used in community.general, community.network and community.crypto :)
15:31:06 <felixfontein> I'm mainly working on it for the latter two, since they are too large to ensure this without automated tests
15:31:14 <felixfontein> ah. former two
15:31:25 <felixfontein> c.crypto is small enough :)
15:32:17 <mattclay> felixfontein: If the version number is only known at release, then when would the `--collection-version-added` option be used?
15:32:28 <felixfontein> mattclay: I guess these collections would not use it at all
15:33:18 <mattclay> It seems that either a collection has a valid galaxy.yml in it (and thus a version available) and is using this feature, or it doesn't have the file (and version) and is not using the feature.
15:33:24 <felixfontein> to be able to use --collection-version-added you need to know what the next release will be *somehow*
15:33:29 <felixfontein> otherwise you can't fill in version_added anyway
15:33:38 <mattclay> Or is there some other scenario where the collection would not have the file/version but would still know the version?
15:34:08 <felixfontein> they could have it in some CI variable, but not in galaxy.yml itself
15:34:26 <samccann> is that how you plan on using it in those two big community collections ?
15:34:54 <felixfontein> samccann: in the big community collections, I want to make sure that galaxy.yml always contains the version of the next minor release
15:35:29 <felixfontein> it could also be tests/util/shippable/sanity.sh that contains the next version number, but that's easier to forget to update
15:36:08 <samccann> general question - do I have to have galaxy.yml versoin number set before I upload to Galaxy?
15:36:27 <mattclay> felixfontein: The `--collection-version-added` option doesn't feel like the right approach. What happens when we need another value from `galaxy.yml` for a test, add another option? I don't think that scales well, having to add an option to `ansible-test` for every value we need from `galaxy.yml`, just to support collections which don't want to provide one.
15:36:37 <felixfontein> samccann: you need to have it set before building the collection - the built collection no longer contains galaxy.yml, but MANIFEST.json (which contains the version number from galaxy.yml)
15:37:41 <mattclay> I think all the `galaxy.yml` based validation (version, and others in the future) need to be simply on/off based on the file being present. If it's not, then no validation involving that file. If you want the validation, include a valid `galaxy.yml` in the collection.
15:37:50 <felixfontein> mattclay: I'm fine with it being a boolean option, to just enable/disable this (and enable = use galaxy.yml). I mainly allowed to specify the version number there because it might make it easier to use for some other workflows
15:38:25 <mattclay> felixfontein: Yeah, I understand that. I just don't think it's going to result in a good workflow going forward as we add more validation.
15:38:41 <felixfontein> the thing is that all the version_added validation that needs the version number is somewhat special anyway, since it also needs change data and to be able to access the old version from git
15:39:30 <mattclay> felixfontein: I also want to overhaul that a bit. We should have ansible-test collect the change information and send the summary to the container instead of passing `.git` through. The tests internally shouldn't be using git directly.
15:39:35 <felixfontein> mattclay: I'm not sure how it could be done in a better way. at least not if it is supposed to be similar to how the same test works in ansible-base
15:39:51 <mattclay> That's why the `--keep-git` option isn't supported for collections.
15:39:56 <mattclay> I just never had time to finish the second half.
15:40:20 <mattclay> I want to get rid of `--docker-keep-git` entirely.
15:41:07 <felixfontein> I'm not sure how validate-modules does this exactly, but I guess it switches branches back and forth to be able to load the module docs from before the PR
15:42:02 <mattclay> felixfontein: I want `ansible-test` to collect the summary of changes from git (basically on behalf of validate-modules) and then pass down that summary for validate-modules and other tests to use. There's already a bit of that going on in ansible-test, it just needs to work a bit differently to support validate-modules.
15:43:01 <felixfontein> fine for me
15:43:52 <felixfontein> anyway, the version_added test only works when the differences between the current branch/PR and the branch this gets merged in are known
15:44:17 <felixfontein> which makes it different from most other tests in validate-modules
15:45:43 <mattclay> It's special in two ways really. It needs to know information about the current diff and it also needs to know the next release version of the collection.
15:46:02 <felixfontein> true
15:47:08 <felixfontein> so how should I proceed with that test?
15:47:13 <mattclay> If either of those is lacking the test can't run. Since the diff could be unavailable even when galaxy.yml is present, the test would need to be optional.
15:48:00 <felixfontein> the test is already optional in some sense, since these errors can only be obtained while in a PR - after merging, the options are not new anymore
15:48:48 <mattclay> Yeah, that's true, which has been a problem before. Now that we can specify optional codes, it needs to be marked optional to avoid problems caused by that.
15:48:59 <mattclay> People have added ignores for those errors, and they can't be safely ignored.
15:49:20 <mattclay> Marking the codes optional will fix that, since they won't be allowed in the ignore list any more.
15:49:20 <felixfontein> seriously? (ok, TBH I'm not surprised)
15:49:43 <felixfontein> ok, so remove the option, always run the tests (if galaxy.yml contains a version number), but make the errors optional?
15:49:51 <mattclay> Yes
15:49:54 <acozine> so the test for `version_added` wouldn't run on import to Galaxy, only on the PRs in a collection's repository
15:50:04 <felixfontein> ok. in that case, ansible-base CI needs to enable optional errors
15:50:12 <felixfontein> acozine: yes
15:50:42 <felixfontein> acozine: the test needs to be able to compare two different versions, and right now ansible-test can only compare between git branches
15:50:43 <mattclay> Hmm... that may defeat some of the intent behind optional errors.
15:51:05 <mattclay> acozine: Yes, there's no way for Galaxy/AH import to run that test since the information that is needed is missing.
15:51:14 <felixfontein> mattclay: true. being able to enable some optional errors would fix that problem :)
15:51:16 <mattclay> It's the only test we have that must be run at PR time.
15:51:38 <mattclay> Once changes are merged the test is no longer possible.
15:51:49 <acozine> from a docs perspective, knowing which version of a collection or of ansible-base I need to install to get a particular feature is important, and worth some investment
15:51:53 <felixfontein> the generic version_added validation (is it a valid semver version number) will still run
15:52:13 <acozine> so I can't do `version_added: iguana`?
15:52:26 <mattclay> acozine: Agreed, it's just that Galaxy/AH import can't verify that.
15:52:27 <felixfontein> acozine: not without sanity tests yelling at you ;)
15:52:44 <felixfontein> at least in a collection. not sure what happens in ansible-base.
15:53:00 <acozine> heh, might be worth it . . . but I am distracting you both from the main discussion
15:53:09 <felixfontein> the string `historical` is allowed at least
15:53:54 <acozine> aka `lost in the mists of time` or `for pete's sake, upgrade already!`
15:54:56 <mattclay> felixfontein: I need to think a bit more on how we should handle this particular set of tests being optional. I have another meeting coming up, so perhaps we can discuss it later? How much longer will you be around today?
15:55:10 <acozine> so we're stuck on the problem of how optional the optional tests are, and whether or not we can turn the `version_added` test on in ansible-base without messing up the other optional tests?
15:55:20 <felixfontein> mattclay: I will definitely be there for the project meeting, probably somewhat before that
15:55:23 <mattclay> acozine: Basically, yes.
15:55:38 <mattclay> felixfontein: Which meeting is that?
15:56:13 <felixfontein> I think I'll try to split up the PR into two parts: the generic version_added tagging / version number validation (is this a valid semver?), and in another PR having the `version_added has correct value` test for collections
15:56:17 <felixfontein> mattclay: https://github.com/ansible/community/issues/536
15:56:28 <felixfontein> "Public Ansible Project Meeting"
15:56:32 <samccann> #info how optional should the optional test be and can we turn on the version_added test  on ansible-base withought messing up other optional tests?
15:56:51 <mattclay> felixfontein: Ah, OK. Too many meetings.
15:56:59 <felixfontein> mattclay: indeed :)
15:57:44 <mattclay> felixfontein: My next meeting has been cancelled.
15:57:48 <felixfontein> lol
15:57:58 <felixfontein> that are the best meetings :)
15:58:17 <acozine> speaking of meetings, I'm going to close the "official" part of the DaWGs meeting now, since we have gone 30 minutes over already
15:58:26 <samccann> before we cancel
15:58:39 <acozine> the conversation can and should go on, though
15:58:42 <samccann> can we agree on a  supplemental DaWGs meeting later this week?
15:58:42 <acozine> samccann: yes?
15:58:53 <felixfontein> sounds good to me
15:58:59 <samccann> to cover changelog and porting guide?  What day/time works?
15:59:11 <acozine> Thursday half an hour earlier than the usual meeting?
15:59:23 <felixfontein> Thursday is not so good this week for me
15:59:35 <acozine> what would work better?
15:59:48 <felixfontein> any other day :)
16:00:03 <acozine> heh
16:00:26 <acozine> for whatever reason, my mornings are full of meetings this week
16:00:50 <samccann> tomorrow at 11 ET?
16:00:56 <felixfontein> I'd prefer if meetings don't last longer than 21:00 UTC :)
16:01:01 <felixfontein> what's ET in UTC? :)
16:01:10 <felixfontein> or what's now in ET?
16:01:13 <samccann> don't I wish I knew?  I think -5?
16:01:14 <acozine> 11 ET would be 30 minutes later than the usual time
16:01:19 <samccann> it's noon right now ET
16:01:29 <acozine> usual time is 9:30 CT / 10:30 ET
16:01:31 <felixfontein> it's 18:01 CEST
16:01:31 <samccann> ah so 10 ET?
16:01:57 <felixfontein> so tomrrow 11 ET = 17 CEST
16:02:13 <felixfontein> sounds good to me
16:02:25 <acozine> wfm
16:02:48 <acozine> excellent
16:02:51 <samccann> ok #info we will have a DaWGs supplemental meeting 11ET/17CET on Wednesday (aka tomorrow)
16:03:07 <acozine> #topic open floor
16:03:39 <acozine> anybody out there waiting patiently for a chance to introduce an issue or PR? looking to ask a question?
16:04:14 <acozine> hearing none . . .
16:04:43 <acozine> thanks felixfontein abadger1999 cbudz_ samccann jtanner mattclay
16:04:48 <acozine> #endmeeting