19:00:18 #startmeeting Ansible Community Meeting 19:00:18 Meeting started Wed Jan 20 19:00:18 2021 UTC. 19:00:18 This meeting is logged and archived in a public location. 19:00:18 The chair is felixfontein. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:18 Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:18 The meeting name has been set to 'ansible_community_meeting' 19:00:18 #topic Agenda https://github.com/ansible/community/issues/539 19:00:18 abadger1999 acozine andersson007_ baptistemm bcoca briantist cyberpear cybette dericcrago dmsimard felixfontein geerlingguy gundalow gwmngilfen ikhan_ jillr jtanner lmodemal misc nitzmahone resmo samccann tadeboro: ping! 19:00:23 o/ 19:00:23 Thanks felixfontein :) 19:00:27 #chair dmsimard gundalow tadeboro cybette 19:00:27 Current chairs: cybette dmsimard felixfontein gundalow tadeboro 19:00:29 * dericcrago waves 19:00:31 #topic Updates 19:00:34 #chair dericcrago 19:00:34 Current chairs: cybette dericcrago dmsimard felixfontein gundalow tadeboro 19:00:45 hellooo 19:00:48 o\ 19:01:07 * alongchamps waves 19:01:57 #info community.network 1.3.1 will be released this weekend, right now it looks like only two announcement changelog fragments and no other change 19:02:15 (there are a bunch of PRs that need reviewing, so if a bugfix PR is merged in time it could also be included) 19:02:23 #chair briantist alongchamps 19:02:23 Current chairs: alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow tadeboro 19:03:01 a release during the weekend ? :) 19:03:01 * ikhan waves 19:03:15 o/ 19:03:30 #info community.general 1.3.5 should be released beginning of next week 19:03:34 #chair ikhan pabelanger 19:03:34 Current chairs: alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan pabelanger tadeboro 19:03:34 #chair ikhan pabelanger 19:03:34 Current chairs: alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan pabelanger tadeboro 19:03:38 o/ 19:03:40 dmsimard: you're faster :) 19:03:41 #chair samccann 19:03:41 Current chairs: alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan pabelanger samccann tadeboro 19:03:53 I guess Ansible 2.10.6 will be released next week as well? 19:04:10 and will probably be the last (or second-to-last?) 2.10.x release currently planned? 19:04:20 The plan is the 26th 19:04:33 if it's the last, we should announce it so that collection owners can try to get a release out before the Ansible release 19:04:37 o/ 19:04:41 #chair acozine 19:04:41 Current chairs: acozine alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan pabelanger samccann tadeboro 19:04:59 We had a note about the future of 2.10.x in the latest release announcement: https://groups.google.com/g/ansible-devel/c/QXx1SXJk05c 19:05:24 ah, there it is :) 19:05:31 Specifically: "* At the moment, the plan is for Ansible-2.10.6 to be the last 2.10.x release. Further releases will be of Ansible-3.0.x. If there is a desire for maintenance releases of older versions, drop by a Community Working Group Meeting to discuss how you can help. ( https://github.com/ansible/community/tree/main/meetings#wednesdays ) 19:05:48 #info Ansible 2.10.6 will be released on January 26th (current plan), and is probably the last 2.10.x release 19:06:37 ok, any more announcements? 19:06:37 o/ 19:06:41 felixfontein: correct 19:06:44 #chair abadger1999 19:06:44 Current chairs: abadger1999 acozine alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan pabelanger samccann tadeboro 19:06:58 #info The bullhorn edition 18 has been released: https://mailchi.mp/redhat/the-bullhorn-18 19:07:04 \o/ 19:07:22 dmsimard: could you add the cut off for 2.10.x into https://github.com/ansible-collections/overview/issues/45 please 19:07:52 sure 19:07:59 :) 19:08:00 for today, we can continue discussing some policy questions/proposed changes we didn't cover last week. or does someone has something (else/related) that should be discussed first? 19:09:21 None from me 19:10:06 abadger1999: should we discuss the ansible-core / ansible schedule? you brought that up last time, with "Will need discussion at a later meeting" 19:10:26 we are 10 minutes into Updates 19:10:39 felixfontein: nah, it's a post-ansible-3.0.0 decision. 19:10:51 so anything that could affect 3.0 should come first. 19:10:52 ok :) 19:11:03 #topic What should we do with collections that contain non-standard directories? (tadeboro, https://github.com/ansible/community/issues/539#issuecomment-757138811) 19:11:09 long version: What should we do with collections that contain non-standard directories? https://github.com/ansible-collections/ansible.utils contains plugins/{cli_parsers,fact_diff,validate} directories that do not correspond to any of the known plugin types or module utilities. 19:12:25 during the discussion of plugins/plugin_utils, the core team essentially said that all non-standard directories are 'wild west' and that they will not be covered by certain sanity tests 19:12:36 (I haven't checked) do those directories ship when you install the collection ? 19:12:44 I lean towards them being okay at the toplevel but lean towards them not being okay in places that have defined meanings. 19:12:46 dmsimard: yes 19:12:46 what is the concern? 19:12:55 plugins/ would fall under the latter. 19:13:17 pabelanger: Do you know what those directories are for? 19:13:29 * lmodemal sorry I'm late 19:13:34 IIRC, to be used by other network collections 19:13:41 the ansible.utils collection has its own plugin system for certain things, the directories are for plugins for its plugin system AFAIK 19:13:48 #chair lmodemal 19:13:48 Current chairs: abadger1999 acozine alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan lmodemal pabelanger samccann tadeboro 19:13:50 lmodemal: no problem! 19:14:18 don't a bunch of the network collections have `cli_parsers` in them? 19:14:27 pabelanger: it's the first collection we encounter that has a "custom" layout so we're mostly validating whether or not it's a concern 19:14:29 Interesting... I think they belong in module_utils then. 19:14:30 ansible.utils uses them to group the python code just like one would do in regular python library. 19:14:44 there are some collections (community.crypto and community.sops) which use the non-standard plugins/plugin_utils/, to have a place for shared plugin code that is not for modules 19:14:56 ahh, putting them in module_utils makes sense 19:15:01 it comes from ansible.netcommon 19:15:06 abadger1999: cli_parsers are different kind of parsers that we support 19:15:09 abadger1999: the problem with module_utils is that it has very strict module-centric validation. if you import anything from ansible, you have to add a ton of ignore.txt entries 19:15:09 I think we are promoting it up into ansible.utils 19:15:10 (or plugin_utils but that might not be present in all versions of ansible that the network plugins care about) 19:15:22 they are not standard module_utils 19:15:31 so, it already exists in ansible 2.10 19:15:46 (plugin_utils doens't exist in any ansible-base/-core/ansible version, it was a proposal which the core team decided to talk about somewhen later, but not now) 19:16:16 felixfontein: apparently it exists for docker 19:16:30 https://github.com/ansible-collections/ansible.netcommon/tree/main/plugins/cli_parsers is the source of them 19:16:30 https://github.com/ansible-collections/community.docker/tree/main/plugins/plugin_utils 19:16:38 dmsimard: right, I also created plugin_utils there 19:16:56 dmsimard: c.sops and c.crypto have the same content in plugin_utils, c.docker something else ;) 19:17:11 The problem I have with these folders is that there is no clear documentation about whether is it OK to have them in collections or not. 19:17:26 Yeah, I do not think it's a good idea to allow non-standard directories in plugins. 19:17:43 The official docs list a fixed set of folders that have a clear meaning, but say nothing about custom folders. 19:17:52 there is nothing preventing collections from putting any directories and files in them (see that collection that we mistakenly included testing output results) 19:18:08 As just one instance, it can lead to conflicts if a new official plugin type is added later which has a different interface/use. 19:18:12 according to the core team, custom folders work now. I guess they can randomly cause problems in the future when core starts using them for something else 19:18:19 abadger1999: exactly :) 19:18:50 where else would these files go ? under module_utils ? 19:19:18 ansible.netcommon 1.x.y doesn't seem to have these folders btw 19:19:31 ikhan: They are module_utils in the senset hat module_utils is where we put shared code. You are correct that in this instance,t he shared code is not useful to general third party modules. 19:19:32 abadger1999: if we do not want to allow custom folders then it should be documented and enforced via tooling. We did not set that bar for collection owners 19:19:42 dmsimard: yes. but that causes a lot of annoyance with respect to sanity tests screaming about a lot of things 19:19:57 ikhan: But I see it similar to how plugins of python programs get placed into /usr/lib/pythonX.Y/site-packages/FOO 19:20:28 felixfontein: where are you looking? they should be in 1.2.0+ 19:20:29 Collections are missing at least one folder for "shared code that will not be used by modules". 19:20:31 ikhan: Right... I think that this section of the meeting is for proposing and adding such things to documentation. 19:21:00 pabelanger: the latest commit I have locally is from November 10th, no idea which version (since galaxy.yml has no version in it) 19:21:02 10 minutes into Collections with non-standard directores, 21 min in the meeting 19:21:20 tadeboro: exactly. https://github.com/ansible/proposals/issues/186 :) 19:21:21 tadeboro: that's my understanding if module_utils is not the right place 19:21:38 felixfontein: git tags, will give the version info 19:22:19 Do we want to make a proposal here? 19:22:57 pabelanger: I updated to main, now I see plugins/cli_parsers, but not the other two 19:23:24 abadger1999: I think this belongs more in core land 19:23:34 abadger1999: at least for a generic thing like 'plugin_utils' 19:23:59 though we could of course say that we accept it, and hope that core sooner or later follows suit and doesn't decide to do something else :) 19:24:00 abadger1999: I guess felixfontein already has a proposal so I guess the question is whether or not this represents a blocker 19:24:39 felixfontein: How about we have a specific list of directories and we use the plugin types currently supported by core as the basis. 19:24:41 I wouldn't want it to be a blocker, given there is no accepted proposal 19:24:44 also my proposal is just for plugin_utils; in ansible.utils there are other folders for new 'plugin-plugin' types 19:24:51 Then we can decide whether we should add plugin_utils to that list 19:24:59 and would basically block the collection, until decided 19:25:07 pabelanger: Since it came here, it is currently as blockler 19:25:18 So it has to be resolved here in some way to unblock the review 19:25:33 plugin_utils and cli_parsers is already used by collections in Ansible 2.10 19:25:43 right 19:25:45 (the former in c.docker and c.crypto, the latter in a.netcommon) 19:25:55 this is already in side 2.10 today, so I don't see why it is a blocker 19:26:17 cli_parsers is also in the docs https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst#developing-cli-parser-plugins-in-a-collection 19:26:47 maybe we should have a rule/guideline "collections shouldn't use other subdirectories in plugins/ without a good explanation of why they need them" :) 19:27:03 Things that made it in previously can still be considered wrong in the future and old code can either be grandfathered until fixed or proposals can be made to fix it now if important enough. 19:27:27 I would rather have an explicit list and then we add to the list when a good reason is made. 19:28:01 That way we don't have conflicts cropping up. 19:28:18 that would be ok for me 19:29:04 does someone wants to compile a list for next week, with current official ones, and potential new ones (i.e. plugin_utils, cli_parsers, and the other two from ansible.utils)? 19:29:14 sivel: Would the core team be okay with us maintaining a list of "extra plugin types" in the colection guidelines and anytime the core team wanted to designate a new plugin typethey would make sure it didn't conflict with anything on the current list? 19:29:56 does `ansible-core` need to explicitly support the various directories under `plugins/` in collections? 19:30:22 I January stepped away for lunch. cc @nitzmahone 19:30:27 just* 19:30:34 gundalow: I suppose it works today so I would tend to say no ? 19:30:36 I'm wondering how the `plugins/facts_diff,validate` etc get loaded 19:31:03 we're 20 min into Collections with non-standard directores, 31min in the meeting 19:31:04 pabelanger: ikhan: do you know details about ^? 19:31:07 gundalow: List like any other python code. They are not plugins like other ansible plugins. 19:31:09 nitzmahone: Would the core team be okay with us maintaining a list of "extra plugin types" in the colection guidelines and anytime the core team wanted to designate a new plugin typethey would make sure it didn't conflict with anything on the current list? 19:31:45 In general "it's just Python", so collections are more or less free to do what they want, but there's no mechanism for things like documentation, discovery, or sanity testing of things we don't know about. 19:31:46 felixfontein: that is a question for @ganeshrn. We can find that out 19:31:50 gundalow: ansible.utils uses standard "import bla" (or its dynemic cousin) to load python modules. There is no plugin loader involved. 19:31:57 felixfontein, pabelanger : Problem:: next week is freeze, so if the collection doesn't qualify for inclusion before that, it won't make it until ansible-4.0 19:32:00 I guess we should give the core team a veto right, so they know that we won't just create a huuuuuuuuuuuuuuuuuuuuuuuuuge list ;) 19:32:06 tadeboro: ah, thanks for the info 19:32:51 tadeboro: are you sure about that ? fact_diff appears to me written as a module https://github.com/ansible-collections/ansible.utils/blob/main/plugins/fact_diff/native.py 19:33:01 abadger1999: true. we could vote on accepting these folders in ansible.utils (and other collections) for now, despite not having proper guidelines 19:33:13 abadger1999: okay, but if we have no solution to fix the issue, because of policy. We as collection owners, cannot really fix the issue 19:33:18 nitzmahone: So what I'm asking is if we start saying here "collections can place X type of content into plugins/cli_parsers" , then is the core engine willing to never make use of plugins/cli_parsers unless it is for exactly the same content used in the same way? 19:33:23 tadeboro: ah, nevermind, there is an actual fact_diff module https://github.com/ansible-collections/ansible.utils/blob/main/plugins/modules/fact_diff.py 19:33:36 dmsimard: it's more like a plugin than a module, since it's derived from a base class 19:33:42 given we've been using it for 4 months, and in 2.10. It's a little difficult to hear, may not get included because of it 19:33:57 dmsimard: the 'module' is an action plugin, the file you linked to is docs only 19:34:12 * dmsimard sighs 19:34:23 felixfontein: https://github.com/ansible-collections/ansible.netcommon/pull/109 is original commit from Qalthos. 19:34:24 https://github.com/ansible-collections/ansible.netcommon/pull/109 | closed, created 2020-08-04T17:52:36Z by Qalthos: New module: cli_parse [gate] 19:34:31 pabelanger: you could put them into module_utils. 19:34:31 As far as I can tell, all the things from those custom directories could be grouped under a single /plugins/plugin_utils (or whaterev) dir. 19:34:32 he would know more how it is loaded 19:34:43 https://github.com/ansible-collections/ansible.utils/blob/main/plugins/action/fact_diff.py#L53-L90 <-- that's the 'plugin loader' 19:34:46 (I think) 19:34:48 pabelanger: but lets see if nitzmahone will approve us keeping a list. 19:34:51 pabelanger: I don't think we are suggesting removing it. More trying to think if we need to check incase another collection owner starts adding random directory under `plugins/` that may cause them issues now or in the future 19:34:52 abadger1999: in general I think that's a reasonable approach- the name conflict is the easiest of the issues to solve though 19:34:55 Things in ansible.utils uses importlib to load modules. 19:35:04 nitzmahone: oh, what are we not considering? 19:35:48 gundalow: sure, that is completely fair. I am just saying, if we need to land a new policy, and freeze is in 1 week. We'd want to make the change, just that the policy would exist in time for 3.0 freeze 19:36:05 abadger1999: Oh, just that the rest of the tooling (docs, sanity checks, etc) is built around a known list, so unless/until those things are augmented to support arbitrary additions, they'll be invisible to those things 19:36:19 so, feels awkward to be delayed to 4.0, for a new policy that doesn't yet exist 19:36:39 tadeboro: since Ansible modifies Python's module loader, importlib will also find things from other collections 19:37:25 nitzmahone: does that mean things like "must pass sanity tests" would be a no-op for things that were in plugins/foo ? 19:37:32 exactly 19:37:39 felixfontein: I would say this is not problematic since things are nested under ansible_modules.namespace.collname 19:37:50 abadger1999: only some sanity tests (compile) run for arbitrary plugin code 19:38:19 Okay.... that feels like arbitrary pluguin dirs are not something we're ready for then. 19:38:28 abadger1999: pylint and pep8 should run for arbitrary dirs 19:38:51 so there is some *basic* sanity testing, just nothing fancy 19:39:17 abadger1999: there's nothing stopping anyone from doing it today, and I don't anticipate that breaking from a runtime perspective, but yeah, the rest of the ecosystem isn't really "ready" for it IMO 19:39:19 we also have unit / integration test coverage in ansible.utils / ansible.netcommon 19:39:35 pabelanger: test all the things \o/ 19:40:18 I *think* the 'import test for plugins' PR (https://github.com/ansible/ansible/pull/72497) should also apply to random new directories in plugins/ 19:40:19 https://github.com/ansible/ansible/pull/72497) | open, created 2020-11-05T13:34:41Z by felixfontein: Import sanity test for plugins [affects_2.11,core_review,docs,docsite,feature,has_issue,stale_ci,support:core,test] 19:41:03 30 min in topic - Collections with non-standard dirs, 41min into the meeting 19:41:04 Quality of the ansible.utils code is not problematic (it is actually really good), but doing things that are not officially declared OK is problematic. 19:41:22 felixfontein: that might or might not make sense though, depending on what the plugin actually does... To support something like that properly, a given dir/type needs to be able to declare what it is and how it should be treated/ignored 19:41:33 This is like using private API - you can do it, but you are on your own. 19:42:05 (by various test/sanity/whatever tooling) 19:42:22 tadeboro: it is not officially declared? is it? 19:42:22 nitzmahone: the import test for plugins mainly makes sure that everything in plugins/ can be imported in Python when only a very small set of dependencies is around. I think that all code in plugins/ should satisfy that anyway 19:42:42 PROPOSAL: collections cannot only put plugins recognized by ansible-core into the plugins/ directories at this time [include list of directories]. Once ansible-test is updated to run tests on those directories, we will update these guidelines to allow other explicit directories. For now, code should be moved into a subdirectory of plugins/module_utils 19:42:59 fixing typo.... 19:43:09 ikhan: All of the documentation I saw always specified what folder are available under plugins. Nothing talked about arbitrary folders. 19:43:49 if something else than the official standard dirs is not included in that list, this is making collection developer's life unnecessarily hard. there's really need for something like plugin_utils/. 19:44:06 PROPOSAL: At this time collections MUST only put plugins into subdirectories of plugins/ which are recognized by ansible-core [include list of directories]. Once ansible-test is updated to run tests on those directories, we will update these guidelines to allow other explicit directories. For now, code should be moved into a subdirectory of plugins/module_utils 19:45:01 abadger1999: that proposal is not really concrete, since some sanity tests already cover arbitrary subdirectories 19:45:18 while some others do not (and most of them make no sense for arbitrary directories anyway, only very few - like import) 19:45:18 Okay.... what is our bar? 19:45:25 Or do we want to make it more vague? 19:46:31 also cli_parsers is mentioned in (some) core docs, but has no extra sanity tests either 19:46:43 Hmm... for the specific directories that we're talking about, are there many or only a few tests that are missing? 19:46:55 abadger1999: felixfontein mentioned earlier that module_utils was problematic for other reasons (like ansible-test complaining about it ?) 19:47:00 the tests that apply to any given thing will depend on the thing 19:47:17 Am would vote +1 on what abadger1999 suggested, but at the same time, would allow some exceptions for 3.0.0 until core devs pitch in on what is acceptable and what is not. 19:47:24 dmsimard: false positives can be silenced via ignors if needed, though 19:47:26 Is it Python? Will it run in the controller or on a remote, or both? 19:48:16 abadger1999: fair 19:48:38 I would definitely support an exception for the ansible.netcommon collection 19:49:06 tadeboro: My guess is that ansible-test changes wouldn't make it until ansible-2.12 at the earliest... (sivel asked a few days ago if we have anything we'd need the core team to work on for 2.12) 19:49:33 that's the set of ignore.txt entries for one (!) file in module_utils: https://github.com/felixfontein/community.crypto/blob/71a8796a899870af41478997a79a03d6b5c7b834/tests/sanity/ignore-2.11.txt#L1-L8 19:50:02 abadger1999: But we can "force" included collections to update the layout early once we know what core things about custom dirs. 19:50:26 Blocking a few days before the freeze seems too harsh. 19:50:44 ... so long as nothing external is using them where they're at now (import redirection can handle some of that, but probably shouldn't be relied on heavily) 19:51:14 40 min on topic - Collections with non-standard dirs; 51min into the meeting 19:51:26 (eg, get some caution tape on them to discourage external usage) 19:51:29 Okay... choices.... 19:51:50 a) allow all custom subdirectories, b) limit to standard ones + small set, c) limit to standard ones 19:51:58 Any change to ansible-test will have to wait until after split controller is done, which is slated to land shortly after feature freeze for 2.12. So nothing new for 2.11 19:52:17 does that mean that my PRs have to sit around for some more months? 19:52:19 Vote to block extra subdirs. Vote to allow extra subdirs. Vote to allow specific collections to use them pending an actual guideline which they'll have to comply with. 19:53:05 Could I have a strawpoll of which of those paths we want to go down? 19:53:11 I would vote for the third one. 19:53:17 I think we should make exception for now to get it. We have clear owners of the collections who are testing all the bits. I do like to see clear documented policy around this for future, so we can inform partners. I bet a lot of partners who have their collections are also may run into this as well. 19:53:20 third one 19:53:22 I vote for third 19:53:23 I would vote for the third of abadger1999's list 19:53:32 * abadger1999 writes quick proposal 19:53:45 given there is no guidelines, I don't see how we can reject them. 3rd 19:53:46 +1 on granting an exception to ansible.utils pending guideline considering felixfontein's existing proposal https://github.com/ansible/proposals/issues/186 19:54:03 And I am willing to make myself responsible making sure a few collections that I approved with exceptions will do the right thing once things are clear. 19:54:31 we should have guardrails of some kind, because if we leave it wide open, people will find crazy things to do, but meanwhile the netcommon collection is useful and tested and should be in 3.0 19:55:14 PROPOSAL: collections cannot only put plugins recognized by ansible-core into the plugins/ directories at this time. The following collections have a temporary exception to use the plugin_utils and cli_parsers directories for the 2.10 and 3.0 release cycles until we figurfe out a final policy: ansible.utils, ansible.netcommon, community.crypto 19:55:19 cybette: thanks for keeping time, this was an important topic :) 19:55:25 s/was/is/ 19:55:27 felixfontein: ^ Did I get the list of dirs and list of collections right? 19:55:45 community.docker (in 2.10) and community.sops (applying for 3.0.0) also use plugin_utils/ 19:55:46 dmsimard: sure, not rushing anyone, just stating the time :) 19:56:10 abadger1999: there are two other directories specifically for ansible.utils: "fact_diff" and "validate" 19:56:13 also ansible.utils has two more subdirectories 19:56:14 ansible.utils contain a few additional dirs. 19:56:16 what dmsimard said :) 19:56:35 PROPOSAL: collections cannot only put plugins recognized by ansible-core into the plugins/ directories at this time. The following collections have a temporary exception to use the plugin_utils, cli_parsers, fact_diff, and validate directories for the 2.10 and 3.0 release cycles until we figurfe out a final policy: ansible.utils, ansible.netcommon, community.crypto, community.docker, and community.sops 19:56:39 cybette: I'm afraid this is a topic that needs to be finished today, no matter how long it takes :) 19:56:51 and community.crypto stays on that list? 19:56:59 cannot or MUST abadger1999 ? 19:56:59 abadger1999: yes 19:57:06 samccann: right.... 19:57:08 abadger1999: it was the first user of plugin_utils (at least AFAIK) 19:57:38 PROPOSAL: collections MUST only put plugin types recognized by ansible-core into the plugins/ directories at this time. The following collections have a temporary exception to use the plugin_utils, cli_parsers, fact_diff, and validate directories for the 2.10 and 3.0 release cycles until we figurfe out a final policy: ansible.utils, ansible.netcommon, community.crypto, community.docker, and community.sops 19:57:53 +1 19:57:54 +1 19:57:55 @abadger1999: +1 from me 19:57:56 +1 19:58:05 +1 19:58:06 s/figurfe/figure :) 19:58:06 +1 19:58:07 +1 19:58:09 +1 19:58:12 +1 19:58:19 +1 19:58:34 #chair 19:58:34 Current chairs: abadger1999 acozine alongchamps briantist cybette dericcrago dmsimard felixfontein gundalow ikhan lmodemal pabelanger samccann tadeboro 19:58:51 with the caveat it goes immediately into the next bullhorn and docs so anyone from this week onward has a place to know what they can or cannot do 19:58:53 +1 19:59:33 samccann: something like 'If you use some of these or other directories in your collections, please talk to us'? 19:59:34 samccann: Hmm... feels like we should have a "New decisions made since last bullhorn" section :-) 20:00:05 is cli_parsers only mentioned in the devel docs, or also in the 2.10 docs? 20:00:09 There were some decisions made last week too. 20:01:02 #agreed collections MUST only put plugin types recognized by ansible-core into the plugins/ directories at this time. The following collections have a temporary exception to use the plugin_utils, cli_parsers, fact_diff, and validate directories for the 2.10 and 3.0 release cycles until we figure out a final policy: ansible.utils, ansible.netcommon, community.crypto, community.docker, and 20:01:05 felixfontein: it's in 2.10 too: https://github.com/ansible/ansible/blob/stable-2.10/docs/docsite/rst/network/dev_guide/developing_plugins_network.rst 20:01:05 abadger1999: yup, that's a good call 20:01:08 community.sops 20:01:10 felixfontein: more like in the collection structure section of the docs, we add a big ol note to say 'do not stray from this set!' or something to say no extra dirs allowed etc 20:01:39 #action abadger1999 will add a PR to the overview requirements and checklist for the custom subdirectories decision. 20:01:44 we don't have to name the exceptions in the docs. only that 'this policy is in effect until blablabla' 20:01:59 dmsimard: I guess the docs should mention when it was added, since it's not supported by every ansible-base 2.10 / ansible.netcommon version 20:02:05 #action samccann will steal abadger's wording to put into the docs themselves 20:02:09 ;-) 20:02:14 :+1: 20:02:29 * abadger1999 has a thing... back in five minutes 20:02:34 samccann: it's worth noting that this is in the context of accepting collections in the ansible package and the criteria/standards of quality we expect 20:02:57 thanks for the reminder. we can word it that way for sure. 20:03:22 I would say "do not create new folders under plugins" is something all devs should follow. 20:03:25 dmsimard: you brought up the topic "https://github.com/Andersson007/ansible_reviewer vs integration with ansible-test sanity (or how to get that feedback in the hands of collection maintainers/ensure compliance over time)" - do you want to discuss it now, or should we discuss it later (i.e. in 1-2 weeks)? 20:03:26 collections without interest in being included are free to do whatever they want, it's not a technical problem 20:03:43 felixfontein: no need to go further over time for that one, let's table it until next week 20:04:10 is there usually an open discussion topic? 20:04:15 #topic community.general: increase minor release frequency 20:04:17 pabelanger: yeah open floor at the end 20:04:20 pabelanger: eventually yes, at the open floor 20:04:22 ack 20:04:34 #info https://github.com/ansible/community/issues/539#issuecomment-733933170 20:04:51 I think it makes sense to release community.general more often than every two months 20:05:01 (I'm talking about minor releases :) ) 20:05:14 originally we said "minor releases every two months, major every six months" 20:05:27 Has anyone asked for more frequent releases? 20:05:38 I would propose to have minor releases in sync with ansible 3.x.0 releases 20:05:45 felixfontein: what kind of frequency are you thinking of ? monthly ? 20:05:54 gundalow: users often want to use new features, and right now it can happen that you have to wait for two months until they show up 20:06:10 dmsimard: more like ~every three weeks 20:06:19 (assuming there is anything to release) 20:06:38 I agree, that pace sounds much more palatable to me 20:06:50 right now, we have a couple of new PRs (bugfixes and features) around every three weeks, so there's always something to release 20:06:56 I don't have an objection 20:06:59 felixfontein: yup, wondering if anyone had actually asked for it 20:07:15 we already decided to release bugfixes more often after the last minor 1.x.0 release was out 20:07:37 If you or whoever is making the releases is up for it, I'm +1 20:07:51 gundalow: I think nobody explicitly said "why not release more often?", it's more like "when is this feature available in a release? oh, I have to wait two more months?" 20:08:22 I'm happy doing the releases. if I stop doing them, we can re-adjust the schedule if the next person doesn't want to do it that often :) 20:09:11 VOTE: should community.general minor releases aim to be in sync with Ansible releases (i.e. a few days before)? 20:09:37 +0 (let the maintainers decide) 20:09:58 felixfontein: I thought you wanted to hand over c.g and c.n to someone else 20:10:03 I'm also +0, no objections but I'm also not the one doing the work 20:10:06 gundalow: just c.n, I'm fine continuing with c.g 20:10:11 +1 20:10:14 ah, OK 20:10:14 +1 20:10:16 +1 from my side 20:10:54 +1 as it sounds like a good idea, but as others have said, I'm not doing the work 20:11:04 if other collections are interested in adopting the same release cadence so that the collection is up to date before an ansible dot release that's fine too 20:11:09 +1/+0 (up to maintainer) 20:11:31 also the amount of work is not that much. it's mainly waiting for CI to run through (which takes some time :) ), and then the publishing job to run through (even more time) 20:12:14 felixfontein: out of curiosity, was the 2 month release cycle formalized/documented somewhere already ? 20:12:31 dmsimard: https://github.com/ansible-collections/community.general/issues/582 20:12:51 neat 20:13:18 we might also want to adjust the major release cycle a bit to accomodate the new ansible-core / Ansible release cycle, but these needs to be finalized first before it makes sense to talk about that ;) 20:13:39 #agreed community.general minor releases should aim to be in sync with Ansible releases (i.e. a few days before) 20:13:44 thanks! 20:14:26 ok, anyone has something important before we continue with open floor? (I assume you don't want to continue the 'new content for community.general?' discussion today :D ) 20:15:10 not me 20:15:29 #topic open floor 20:15:30 ok then :) 20:15:36 pabelanger: do you have something? 20:15:39 The Bullhorn is published every other Wednesday, and I'm thinking it might be better to have it on Thursdays instead so we can include decisions from Wednesday's community meeting. 20:15:51 cybette: +1 20:15:52 cybette: good idea! 20:15:58 cybette: good call 20:16:00 yah 20:16:06 cybette: good idea 20:16:15 I wanted to discuss https://github.com/ansible-collections/overview/blob/main/collection_requirements.rst#ci-testing and nightly jobs 20:16:27 I would like to propose downgrading it from MUST to SHOULD 20:16:37 which part exactly? 20:16:44 >All CI tests MUST run regularly (nightly, or at least once per week) to ensure that repos without regular commits are tested against the latest version of ansible-test from each ansible-base/ansible-core version tested. 20:16:51 also note that we do not require nightly CI, we require regularly run CI, at least weekly 20:16:58 in our case, nightly or periodic jobs are basically wasted resources. We gate every commit into our collections, and all testing MUST pass before merging 20:17:10 so, in that case, there is really no need for a periodic job 20:17:20 pabelanger: that's only correct if you have at least one commit per week 20:17:28 pabelanger: If you do not pin everything, then nightly runs are not a waste of time. 20:17:29 right, which we do 20:17:43 The above requirement was added so that changes to `ansible-core` (or any dependent external collection) are tested. 20:17:47 if you have at least one commit per week, then you already satisfy the "CI must run at least once per week" criterium 20:17:50 (IMO) 20:17:53 in our case, they are. We have done nightly testing for years, with minimal focus on it 20:18:18 If a collection doesn't have a PR go though for a week or more then it's possible a change to (say) ansible-base:devel could break the collection and no one would know 20:18:40 until issues start being created by annoyed users :) 20:18:45 I'm a little leery about the whole concept of that section. But if we have it, I don't think SHOULD will encapsulate what we want. Probably need to lay out what the rationale is better and make satisfying the rationale a MUST. 20:18:59 abadger1999: +1 20:19:07 gundalow: beause we don't say our collections work against devel 20:19:11 we cap it at latest stable 20:19:17 and also 20:19:23 testing against devel is SHOULD, not MUST 20:19:41 pabelanger: that's at your own risk, you might end up with the need to fix A LOT of things once there's a new stable branch :) 20:19:56 yup, a lot of work 20:20:02 testing with devel is suggested, but not required 20:20:13 right, but again it isn't recommended 20:20:16 we do it however 20:20:22 it's recommended, but not required. 20:20:36 pabelanger: "You MUST run ansible-test sanity from the latest stable ansible-base/ansible-core branch" 20:20:52 "You SHOULD suggest to additionally run ansible-test sanity from the ansible/ansible devel branch so that you find out about new linting requirements earlier." 20:20:55 and the stable branches can change at anytime, right? 20:21:06 my point in this case, we have periodic jobs today, that are mostly useless for us. Given we don't really see value in them, being force to run them, for inclusion, isn't the best 20:21:22 (I think the 'suggest' in that sentence can be removed, or it should say "We suggest you SHOULD ...") 20:21:58 if you run tests once per week by adding commits, you satisfy the criteria 20:22:03 abadger1999: yes, everything we test is from git, not stable releases 20:22:19 if you don't add commits at least once a week, you should manually trigger CI once a week if you don't have an automatism 20:22:53 pabelanger: yes.... I'm saying... if you change what gundalow said from ansible-base:devel to ansible-base:stable-2.10, the same argument applies, does it now? 20:23:17 *does it not 20:23:48 I agree 20:24:30 so, is the issue that a released collection will fair against a newer version of ansible-core? 20:24:33 I think felixfontein is saying we could clarify the rule to show that this doesn't have to be automated. Does that work for everyone? 20:25:08 abadger1999: I wouldn't advertise that too much, it might incentivise collections which don't test much to use it as an excuse to test even less :) 20:25:35 10 min into Open floor; 85min in the meeting 20:25:36 Heh. So allowed but on the down low? 20:25:56 What about: All CI tests MUST run at least once per week to ensure that repos without regular commits are tested against the latest version of ansible-test from each ansible-base/ansible-core version tested. A time-based job is the best way to achieve this. 20:25:56 optional, that is really what I am asking 20:26:04 abadger1999: kind of 20:26:10 I would *not* make it optional though. 20:26:17 why? 20:26:17 that really defeats the purpose 20:26:23 in our case, we have gated commits 20:26:32 so, all testing has to pass for code to merge 20:26:38 pabelanger: I don't think people want to make the requirement to run CI at least once per week optional. (but seem willing to say that how you achieve that is up to you) 20:26:39 there isno manually option to merge it 20:26:42 because of what we said above: we want that collections keep still working (at least with the tests they have), even if they don't get new commits for months 20:27:16 felixfontein: okay, but how do you enforce that 20:27:27