19:00:17 #startmeeting Ansible Community Meeting 19:00:17 Meeting started Wed Nov 11 19:00:17 2020 UTC. 19:00:17 This meeting is logged and archived in a public location. 19:00:17 The chair is felixfontein. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:17 Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:00:17 The meeting name has been set to 'ansible_community_meeting' 19:00:21 so, who's around? 19:00:43 \o 19:01:12 * gundalow waves 19:01:24 abadger1999: acozine: dericcrago: dmsimard: gundalow: jtanner: lmodemal: samccann: agaffney: geerlingguy: tadeboro: 19:01:28 I'm sure I forgot some 19:01:34 #chair dericcrago gundalow 19:01:34 Current chairs: dericcrago felixfontein gundalow 19:01:36 \o 19:01:40 #chair samccann 19:01:40 Current chairs: dericcrago felixfontein gundalow samccann 19:01:55 Good day 19:01:56 every time I plan to pre-compile a list of nicks, and then I forget again... :) 19:01:59 #chair abadger1999 19:01:59 Current chairs: abadger1999 dericcrago felixfontein gundalow samccann 19:02:09 #topic agenda: https://github.com/ansible/community/issues/539#issuecomment-725234986 19:02:11 * agaffney looks up 19:02:36 I have no idea what's going on, but it's a welcome break from writing a helm chart 19:02:38 andersson007_ isn't around for today's meeting. he told me how he wants to vote for the first topic though :) 19:02:41 o/ 19:03:01 agaffney: we have the weekly community meeting right now, I just thought I can also ping you if you're interested :) 19:03:05 #chair agaffney tadeboro 19:03:05 Current chairs: abadger1999 agaffney dericcrago felixfontein gundalow samccann tadeboro 19:03:26 #topic announcements 19:03:37 #info ansible-base will be renamed to ansible-core, I think for 2.11 19:03:53 does anyone else has something to announce? 19:04:04 so the ansible-xxx that will be in Ansible 2.11 will be ansible-base, right? 19:04:40 samccann: correct 19:04:49 yes, I think so. we'll have ansible-base 2.10 in ansible 2.11, and ansible-core 2.11 will get included in ansible 2.12 19:04:57 (assuming ansible itself won't be renamed as well) 19:04:59 ansible-2.12 is where ansible-core will show up. 19:05:05 is the rename to ansible-core going to cause another pip-related issue on upgrade? 19:05:08 #info docs team is discussing how to split up the docsite to support independent releases of ansible-core and Ansible. See https://etherpad.opendev.org/p/ansible-documentation-split for scratchpad of current ideas. We meet on Thursdays 10:30am ET on #ansible-docs 19:05:19 agaffney: webknjaz thinks so. 19:05:31 agaffney: I think it won't be as bad for users of the ansible package. 19:05:33 agaffney: probably depends on how it is done, I hope not, but could well be 19:05:50 But there will be cornercases. 19:06:16 I think pip install ansible==2.9.0 ; pip install --upgrade ansible==2.12.0 will work out of the box. 19:06:32 But a user could do pip uninstall ansible-base afterwards and break everything. 19:06:40 I'll have to test that, though. 19:07:07 Err..... actually..... 19:07:37 going from 2.9.0 to anything will probably break i nthe same way as upgrading 2.9 => 2.11..... 19:07:56 i should have written pip install ansible==2.10.0 ; pip install --upgrade ansible==2.12.0 19:08:28 if someone does `pip install --upgrade` after upgrading from ansible 2.10 to ansible 2.11, and there's an update for ansible-base but not for ansible-core, I guess the result will not be good. 19:08:45 felixfontein: yeah, that would break things. 19:08:56 actually..... that might be worse breakage. 19:09:10 yep 19:09:31 it wouldn't break immediately on upgrade from ansible-2.10 to ansible-2.12.... it will break at some indefinite time in the future. 19:09:43 and if ansible-base and ansible-core update at the same time, fun things can also happen 19:10:04 * bcoca waits for next rename, probably by friday 19:10:20 relrod, webknjaz: felixfontein pointed out a way that the ansible-core rename will be worse than the ansible => ansible-base rename. 19:10:56 felixfontein: I suppose refusing to install if ansible-base is installed is the best way to handle that. 19:11:14 abadger1999: I think so too, which would lead to the same fun as for 2.9 -> 2.10 19:11:20 Yep. 19:11:28 without reopening pandoras box - is there a one-sentence summary on why the rename is needed? 19:11:43 The core team would have to tell us. 19:11:58 sivel: ^ Can you summarize why the rename to ansible-core is necessary? 19:11:59 "because someone said so"? 19:13:13 I personally like ansible-core better than ansible-base, but I also don't have to personally deal with the fallout 19:14:21 ok, I'm not sure whether sivel is around, so let's maybe start with the main topics of this meeting :) 19:14:39 +1 19:14:44 yeah, can do.... I think relrod is catching up but maybe we can revisit in open floor. 19:14:46 there are two topics which we should talk about today - one is the continuation of the last two meetings, the other one the continuation of several more meetings ;) 19:14:52 I just sat down for lunch 19:14:54 abadger1999: sounds good 19:15:07 sivel: enjoy your lunch then, we'll talk about it in open floor 19:15:09 in short, Marketing doesn't like the name ansible-base/base/Base 19:15:22 So they approached produt management, who approached us 19:15:25 how about ansible-foundation? :D 19:15:40 we were in favor, PM and Marketing finalized and approved the rename 19:15:47 ansible-bedrock 19:15:51 ok thanks for the summary sivel 19:16:01 sivel: thanks. and for how long this time? :) 19:16:16 felixfontein: hopefully for a while. internally we have always called it "core" 19:16:34 new meme: every time I upgrade ansible, ansible explodes until I format my hard drive and start over 19:16:36 let's hope so :) 19:16:55 #topic Should we accept new content in community.general (and community.network)? 19:17:16 yes* 19:17:20 felixfontein: I have to run to lunch, but my opinion is still a hard no on that 19:17:31 this is a discussion we started two weeks ago. last week we boiled it down to a list of options (https://github.com/ansible/community/issues/539#issuecomment-721928283) 19:18:02 after a bit of discussion with andersson007_ this morning, I added another item, and ended up with 5 options: https://github.com/ansible/community/issues/539 19:18:09 sorry :) 19:18:09 * geerlingguy campaigns to re-merge all collections back into community.general otherwise, since it will become the de-facto 'blessed' ansible content repository 19:18:11 https://github.com/ansible/community/issues/539#issuecomment-725234986 19:18:27 #info 5 possible options listed in https://github.com/ansible/community/issues/539#issuecomment-725234986 19:18:46 The options range from (1) absolutely no new content to (5) allow all new content that passes review 19:19:02 (n+1) allows more than (n) (or at least that was the intention) 19:19:17 "(Right now, we are somewhere between 3 and 5.)" — right now I'm somewhere between #1 and could be paid off handsomely to maybe consider #2 19:19:18 I hope you all thought a bit about this since last week :) 19:19:45 Anything from iii, iv, and v == community.general becomes ansible-modules-extra v2.0 19:19:48 geerlingguy: I mean with 'right now' what's currently happening, not what the current opinion is :) 19:19:58 ah yes 19:20:22 anyways, I gotta eat, so have to run, but my vote is strongly i, and maybe someone could convince me after a long discussion that ii might have some merit. 19:20:44 does someone wants to discuss the five points, or suggest some more? or should we go right at voting? 19:20:56 i - Moved the problem else where. Means there is a massive overhead of having to host and maintainer a new collection. If we go down that route we'd need a lot better toolding, and likely literally hundreds of repos under gh/ansible-collections/ 19:21:19 there are a lot of pros and cons for this one, but it's somewhat side-stepping the issue that the barrier to entry is somewhat higher without allowing new content in community.general, because creating/maintaining/publishing your own collection is "hard" 19:21:22 gundalow: I thought the idea is that people would create their own collections 19:21:31 the answer there is making it not so onerous to maintain collections 19:21:36 ii - Similar to `i`, Do we really want collections containing a single collection, will people maintain them? 19:21:41 kind of like roles 19:21:42 agaffney: I agree 19:22:23 otherwise if we are going to give up on trying to make collections easier to build, maintain, and update, then I might understand level iii-v 19:22:24 One thing I thought about this week: it doesn't seem that bad to move content later. 19:22:25 we talked about a `community.beta` or some better name, that would be a proving ground for new modules. 19:22:34 gundalow: I think for some content, one plugin/module per collection is an acceptable start (when it is clear there will be more in the future). for others, not really 19:22:57 abadger1999: already we are annoyed by the difficulty of *cleanly* (e.g. no user disruption and well-tested process) moving content from c.kubernetes to somewhere else 19:22:59 so `community.beta` would become the techpreview/unstable/user beware this stuff will move if it's popular 19:23:03 yup, I think `nagios` is my go to example here, there's only been a single module for years 19:23:04 it doesn't seem frictionless 19:23:05 abadger1999: depends on how long 2.9 will stay and how many people will use 2.9 instead of 2.10+ 19:23:08 I think it's pretty transparent for end users as long as runtime.yml entries in community.general gets updated appropriately. 19:23:27 Ugh. yeah. 19:23:48 And I suppose... how long we care about 2.9. 19:23:50 also moving later makes it potentially impossible to update ansible-base's ansible_builtin_runtime 19:24:12 which means that people need to install community.general even if they just want to use community.xyz because they need it for the redirects 19:24:19 samccann: that's good, but what about this: how and when modules will move from community.beta to c.g or other collections or its own collection? 19:24:31 geerlingguy: what are the points of friction because I couldn't think of any (not thinking about 2.9, though) 19:24:34 all these things point to "let's not make one massive repository to dump everything into" IMO 19:24:35 I think the available options cover the whole spectrum of options. That being said, my preferred option would be ii with an added clause that the things covering one "area" should move out of c.g as soon as possible. 19:24:39 but maybe let's keep that subject for later, and continue with the current question: no more new content for c.g/c.n? 19:25:05 aminvakil: yeah we'd need a process around that...but it helps c.g from becoming yet another dumping ground of seldom/not maintained modules over time. 19:25:13 abadger1999: testing the redirects, writing and maintaining docs that help users make the transition, adding the routing info from the old collection, planning release of old collection where redirects will occur... 19:25:22 I'm confused on the difference between `iii` and `iv` 19:26:09 geerlingguy: ah.... you're talking about the maintainer work. Not the user experience. 19:26:22 I know we want to be happy and kind to end users in all this—but the whole reason for this collections migration in the first place was to try to make ansible maintainable again. My opinion is that if we open the floodgates—even a tiny smidge—it turns c.g into ansible-modules-extra-next 19:26:25 gundalow: iv) is iii) with an exemption: if someone (like this meeting) can be convinced that something new should be in c.g, it can still be added 19:26:32 maintainer burnout is real :P 19:26:32 gundalow: IV sounds like a lower barrier of entry, since you don't have to explain why it shouldn't be in its own collection 19:26:34 felixfontein: ah, thanks 19:26:43 ansible_builtin_runtime is already be frozen, although I suppose on a 1 off basis we might consider changes, but I cannot gaurantee we will 19:26:50 s/be// 19:26:51 ah, the other way around 19:26:57 geerlingguy: yes it is.... otoh, that is true both at the front end and hte backend. 19:27:14 sivel: I think we agreed with nitzmahone that we can have one more change beginning of next year 19:27:23 sivel: which is why I dislike later moves 19:27:51 I'll make sure I get clarification as well :) 19:27:55 You're talking about maintainers getting overloaded at the point when they decide to move the content. Disallowing new content into c.g means overloading them when they first generate the content. 19:28:39 slightly higher initial barrier of entry for module author vs. more work for c.g maintainers in the long term 19:28:54 agaffney: I think it's more than 'slightly' 19:29:00 at least right now 19:29:20 I'm happy to accept action items to make it easier to own/manage/maintain a collection 19:29:30 I haven't actually created a collection, but my impression is that it's not much harder than creating a role and publishing it to galaxy 19:29:52 agaffney: I'd say the most intensive process is setting up CI 19:29:54 agaffney: if you want to do it properly with changelog, CI, ..., it is definitely some work 19:29:56 one extra step, there is a build/package 19:30:04 Creating collection is not that hard. Setting up a CI/CD is a bit of an PITA, but still. 19:30:12 sivel: that is not required to create and publish a collection (advised, yes) 19:30:24 tadeboro: creating it is still not that easy, if you've never done it before 19:30:45 but same with roles, not hard to create/publish, but if you want CI/Cd .. that is eveyr project 19:30:48 the other side of that is that content discovery will be harder with one-off modules in one-off collections 19:31:01 it probably would be nice to have a 'one click solution', maybe some github app where you can create a new collection repo with everything already set up? 19:31:07 but that's already kind of an issue with the collection split 19:31:21 content discovery is the key feature of galaxy 19:31:31 how hard is it to get hold of a namespace on galaxy? I think 'back then' every user got their own, but is that still the case now? 19:31:37 was already an issue with roles having plugins .. but actually collection makes it easer to find em 19:31:49 iirc, open a ticket 19:32:00 felixfontein: So majbe we should work on improving the create-a-new-collection experience instead of stashing everything in one place that has just happens to have everything set up already? 19:32:19 tadeboro: I think making it easier has to be done either way 19:32:23 bcoca: that's an additional barrier for random users who just want to create a small module 19:32:51 tadeboro: yes, but there's also the time until the point it is easy enough that we have to consider :) 19:33:15 felixfontein: less than they had before, also 'random module' is not a good thing to distribute 19:33:34 having it 'orphan in anyonymous colleciton' is slightly better than orphan in well known and included collection 19:34:06 So to recap where we are 19:34:27 Yeah, making a new collection that goes into the ansible tarball could be a significant undertaking and likely that barrier gets higher as time goes on. 19:34:50 at this point, I think I'd vote for option #3, if only because this problem has multiple facets and isn't easy to solve, and it's a good middle-of-the-road option 19:34:59 I think we should probably make some temporary rule that expires when tadeboro's suggestion is implemented..... 19:35:21 agaffney: I agree, and that is also andersson007_'s favorite option 19:35:40 A) Sounds like everybody is OK blocking groups of modules from been added to community,general or community.network. Therefore we should enforce that now 19:35:40 B) We need to identify and document the painpoints on creating/maintaining a collection 19:36:09 I'm good with ii as well. 19:36:16 abadger1999: I don't think we're making permanent rules anyway, it's all valid for a time until it needs to be adjusted again :) 19:36:19 Sorry. 19:36:21 iii 19:36:47 heh, that's why I used Arabic numerals :) 19:36:52 Yeah :-) 19:37:05 agaffney: I did use them, but github changed them to roman... 19:37:17 hah 19:37:37 (probably because it's a enumeration inside a list item) 19:37:42 felixfontein: that's true :-) It does help if we know what our parameters are. 19:37:46 hey guys, I've been reading quietly, but my $0.02 - I agree we should streamline the creation of new collections (heck, we should write an ansible playbook for that, shouldnt we?). On top of that, I think we might benefit from having a couple of objective criteria for new modules in c.g, other than just the number of files 19:38:12 felixfontein: with option iii are we committing to creating community.incubator now? 19:38:16 #chair russoz 19:38:16 Current chairs: abadger1999 agaffney dericcrago felixfontein gundalow russoz samccann tadeboro 19:38:19 like: must not add any lines to ignore files (I'm trying to clean those, so yep, a bit of a grunt) 19:38:28 depends on scope of 'create', ansible-galaxy init 'creates a collection' 19:38:37 russoz: Morning :) 19:38:44 morning! :) 19:38:44 #chair bcoca sivel jtanner geerlingguy 19:38:44 Current chairs: abadger1999 agaffney bcoca dericcrago felixfontein geerlingguy gundalow jtanner russoz samccann sivel tadeboro 19:38:49 ============================ 19:39:03 looks like I forgot to add some people, if anyone more wants to get chaired, ping me 19:39:04 setting up ci/cd is very diff isusue and has to do more with repo management and a requirement to be part of the distribution vs being publishe din galaxy 19:39:06 How about we change topic to "How can we make it easier to setup a collection"? 19:39:23 like: if this modules releases changes more than X times/week => it should be on its own 19:39:24 what's the delta between creating a role and creating a collection? i don't remember a lof of grief over the process for creating a role 19:39:26 bcoca: we're talking about the former here, not the latter. 19:39:30 gundalow: I think if we start discussing that now, we'll never get to topic #2 for today 19:39:34 i would divide taht in 'for publishing to galaxy/including in ansible community distribution' 19:39:47 yes, but that is not clear in the wording 19:40:17 felixfontein: fair point, maybe we should just move to that anyway 19:40:26 jtanner: I think it's just the build step, but the reason there's grief here is because we've also taken away the option of creating a PR in ansible/ansible to add your module 19:40:40 I would prefer ii, but iii is also fine with me if we revisit the decision later when other things are in a better shape. 19:40:49 yeah, well the later has gone away for good reason 19:40:49 in any case, if we decide to cut all new stuff for c.g, I think we should maybe make an exception for existing PRs, or at least still give them a chance 19:40:51 I recall reading somewhere about how much time the postgresql modules were taking in the CI => we should definitely have a criteria for that (like it cannot be more than 5% the total time - or 10% or whatever makes sense) 19:41:21 gundalow: do you think we can answer the 'more content for c.g/c.n' question without discussing further how to make collection creation easier? 19:41:24 jtanner: of course, but just thinking about it from an outside contributor's perspective 19:41:49 i heard grief over the module submission process for years and it lead to people thinking we should "lower the bar" 19:41:52 russoz: one reason they take so much time in CI is because they actually have good tests. most modules take almost no time in CI because they have no tests at all. 19:42:19 almost tempted to say, that is good barrier to entry 19:42:50 *some* barrier to entry is a good thing here, and some people are always going to complain that they can't do a drive-by code dump 19:42:51 As the owner of community.general the bar will not be lowered 19:43:26 i mean adding a collection to ansible package 19:43:45 right now we only allow new stuff into c.g/c.n which actually has tests 19:44:07 felixfontein: I think it's `ii` or `iii`. Maybe we say `ii` and label all `iii` content to see what else could be done with it 19:44:15 i would argue ux gets overlooked just cause things pass tests 19:44:39 my suggestion: gather list of "why I can't just make my own collection" and focus on those issues, instead of further bloating c.g. and c.n. 19:44:46 felixfontein: noted. not sure how to move forward without creating too many barriers 19:44:47 bcoca: that is definitely possible 19:45:02 agaffney: I think you've indirectly hit on soemthing.....I think initial code quality isn't really that important compared to "I'm going to maintain this" vs "I want to dump this code on someone else" 19:45:10 jtanner: "why not both?" 19:45:15 gundalow: I currently prefer iii, at least until the process of creating new collections is simplified enough 19:45:27 bcoca: This is why we require integration tests: they are playbooks that demonstrate how modules will be used. 19:45:35 * gundalow ====================== 19:45:35 ====================== 19:45:36 ====================== 19:45:37 ====================== 19:45:51 =. 19:45:56 usage test existing is not same as good ux 19:46:01 * felixfontein *pling* 19:46:02 POLL: Are people OK with (for at least the moment) `ii New content only in areas that already have content, i.e. new GitLab modules would be accepted, new RandomNewCodeHoster modules would not be accepted.` 19:46:09 -1 19:46:09 +1 19:46:12 -1 19:46:14 -1 19:46:18 -1 19:46:19 +1 19:46:21 +1 19:46:21 -1 19:46:25 andersson007_ -1's this one as well 19:46:40 geerlingguy is +1 (or -1 because he tends to (i)) 19:46:47 +1 19:47:07 +1 on ii, with the addendum that there be an effort to break that existing content area into a new collection 19:47:22 All those saying -1, please state `i`, `ii`, ... 19:47:30 ie state your prefered option 19:47:38 agaffney: everyone who wants to help with creating and maintaing collections for the areas, is welcome to step up! 19:47:46 (i) 19:47:49 +1 with same ^ and also i would add, there can be exceptions made if sufficient support from maintainers .. in case the 'golden module' is submitted 19:47:56 iii 19:47:59 iii 19:48:00 (iii) 19:48:06 iii 19:48:09 -1 (i) 19:48:12 andersson007_ says iii 19:48:13 i 19:48:15 geerlingguy says i 19:48:19 felixfontein: I say that with no intention to help with maintaining collections, in full honesty 19:48:25 ah he just wrote it himself :) 19:48:30 agaffney: honesty is OK :) 19:48:53 i or ii is my preference 19:49:01 agaffney: I'm already set up and am maintaining some breakout collections, and I don't want to have a whole group of them :) 19:49:03 i have the intentino, but honestly, not the time 19:49:03 felixfontein: the problem with the idea of 'until collections are easier' is—if we don't _FORCE_ collection maintenance to be easier by making us have to make it easier now, it ain't ever gonna happen 19:49:07 it will never be anyone's priority 19:49:24 right now collection maintenance is not easy, but there are tons of quick wins that can make it way easier 19:49:26 bcoca: yeah, I guess that's more where I am. although, it's more a matter of motivation than time 19:49:41 felixfontein: the breakout should assuem people with the time and dedication, its no good to breakout if its going to overwhelm you 19:49:48 I would change to i or ii but only after there's an alternative way to easily get new content into ansible 19:49:53 like galaxy incorporating docs, making publishing easy by having galaxy do it (like docker hub, vagrant boxes, et all does it) 19:50:03 bcoca: I agree, but if we want to break out all areas, someone needs to do it :) 19:50:10 for instance, i or ii with community.incubator would be okay with me. 19:50:12 felixfontein: does not mean you 19:50:39 abadger1999: isnt that just creating an alternate c.general? 19:50:39 bcoca: definitely not me. but people who want to do that don't grow on trees 19:50:40 If it were as easy to publish a collection to galaxy as it is a role, I don't think we'd even have a discussion 19:50:46 what is the end goal for c.g in a year (or 2) from now: bigger, smaller, the same, non-existent? 19:50:55 +1 to community.incubator 19:50:58 felixfontein: agreed, why it has to be clear what code will rot, imho 19:51:08 abadger1999: +1 and it matters how modules from community.incubator will get out to other collections 19:51:30 dericcrago: seems like there is no unified goal there, i think all of the above 19:51:45 -1 to community.incubator, it's another 'blessed' collection IMO. Any kind of central collection that is a conglomerate goes against what I think is the very reason collections exist! 19:51:56 bcoca: I today looked at a PR of russoz fixing some sanity errors in modules/cloud/google/, and seeing how the modules there look... *shudder* I think quite of them are already rotten 19:52:15 will community.incubator get included in ansible? 19:52:16 without procedure for process of moving things out of community.incubator, i'm not sure about that 19:52:25 what about 'incubator' w/o being community. so you can push/test ci/cd against your plugin/module w/o having to set it up? 19:52:25 and will it have CI (and if yes, who takes care of it)? 19:52:43 yeah it has to have a process - it either graduates within a year to a collection, or gets deprecated and deleted 19:52:48 -1 to blessed collections of any kind. +1 to making it easier for people to maintain collections so the 'average joe' could create and publish one and push updates to it as easily as someone could do roles on galaxy today 19:52:52 community.incubator that is 19:53:13 samccann: if we could automate that deprecation/deletion cycle I'd be less -1 to it 19:53:14 and my nickel is no, community.incubator is never part of Ansible 19:53:22 geerlingguy: i agree, that woudl be best case 19:53:28 e.g. the day a module is added, timer is started. One year later it is deleted, full stop 19:53:39 samccann: +1 19:53:40 if it didn't graduate then it's gone 19:53:53 felixfontein: yes, if community.incubator doesn't get included in ansible it defeats the purpose of having it. 19:53:54 sounds reasonable 19:53:56 then we need a graduation process, and people implementing it 19:54:07 geerlingguy - what if incubator wasn't included in the distribution so there was incentive to get it properly collection'd 19:54:12 the reason I like `community.incubator` is it 'lowers' the barrier to entry, but it's not a full entry. It's just a foot in the door. and if the developer doesn't keep maintaining or finding an apropriate home over time, it gets deleted 19:54:22 felixfontein: it seems to me a number of modules have spun-off ansible project a while ago, for the obvious reasons, and we keep carrying them legacy versions around, while the bright and shiny new versions are being maintained elsewhere by their owners 19:54:31 that would be even better, but the incentive would be way less 19:55:10 samccann: I would hinge deletion on whether it has a maintainer rather than time based. But I do think that ability to delete from an incubator is an important thing for it. 19:55:12 russoz: true... it's still hard to decide how to proceed without really knowing the modules 19:55:14 Do any other projects/programming languages have the concept of an incubator? 19:55:14 Or anything else that self destruct unless it finds a sponsor to graduate it? 19:55:27 apache as a foundation has an incubator 19:55:27 abadger1999: if incubator is included in ansible, what's the difference between it and general? 19:55:30 (I hate being Mr. Negativity, but I just feel like if we burn the community three times, we're going to completely lose a lot of it) 19:55:37 gundalow: Apache Foundation 19:55:42 but apache doesn't try to lump everthhing together in a blob 19:55:56 gundalow: Helm just went through getting rid of a central repository 19:56:00 aminvakil: things are automatically deleted, so people should never ever use it for production, so content in there has not much chance of actually graduating :) 19:56:03 they made it easier for people to submit and maintain things on Helm Hub 19:56:35 ^ whichi s the 'galaxy approach' ... and i think that is better than relying on the kitchen sync package 19:56:43 geerlingguy: Thanks for saying that (burning community) 19:57:00 It's important to remember 19:57:00 unless community burns back (torches!) 19:57:30 aminvakil: Right now, community.general wears many hats. (1) Colletcion for all miscellaneous content that used to live in ansible/ansible (2) Place where people who want to contribute content but don't want to learn and maintain all the extra baggage that goes into making a collection that can make it into the ansible tarball. 19:57:35 pitchforks! 19:57:37 gundalow: that's my biggest concern, it's hard enough with people going 'cloud native' and thinking configuration doesn't matter anymore, plus terraform eating some market—if we told people last year that "we will have some growing pains with collections, but look at all this data showing they'll make maintenance better"! 19:57:52 And then 6 months later we have community.general (and/or incubator) showing the same problems 19:58:27 then that jpmens post will be just as relevant in the collections-based world, except now we have two problems: Ansible is more complex with collections, plus the big centrally-manage collections are just as unmaintainable as ansible/ansible used to be 19:58:32 yup, totally agree with that 19:58:49 aminvakil: The idea of community.incubator would be to separate out those two. community.general can slowly shrink over time as content is moved out and no new content is moved in. community.incubator can contain new content that needs more contributors to reach the critical mass of having their own collection. 19:58:49 So on that point, we will be doing fortnightly PRs days 19:58:58 i think if we collectively keep trying to persist the model of <=2.9, we'll keep repeating the same problems it had 19:59:05 that's not gonna solve the problem 19:59:13 we could have PR days every day and it won't solve the problem 19:59:25 yeah, cannot rename the repo and expect the same issues to go away 19:59:29 in the long term, definitely 19:59:30 again, I hate to be so negative, I just think we're barking up completely the wrong tree 19:59:51 which tree should we be barking up? making collection maintenance easier? 20:00:11 the right thing IMO: make it easy to maintain collections (right now it is far from it). Make it easy and straightforward to get collections into ansible ('community edition') 20:00:13 samccann looking at inclusion into ansible package 20:00:23 samccann: yes, that's the #1 issue IMO 20:00:33 as a role maintainer who is trying to maintain collections, here's my experience: 20:00:38 +1 geerlingguy 20:00:52 1. Roles — I maintain over 100, and it is very easy, and I can focus mostly on code concerns and features. 20:00:55 geerlingguy: yep, I think that's the best end goal. 20:01:21 perhaps we should look at the Terraform model (at least for inspiration). they did a similar split with the providers. they don't ship everything with ansible, but they make it automatic to fetch them when referenced 20:01:26 2. Collections - I maintain 2, and I've spent way more time working on dumb issues getting them into galaxy, trying to have documentation show up for end users, figuring out test integration, trying to get them working with ansible versions, etc. 20:01:50 the ratio of time spent fighting the tooling vs. time spent actually working on code is radically bad on the collection side of things 20:01:52 geerlingguy: well, part of that is roles were soo limited you were not expected to do those things 20:02:04 but galaxy took care of everything, which is part of my point. 20:02:05 and if it broke with ansible x.y .. well .. that is life 20:02:12 no, it really didnt 20:02:21 it auto-imports roles 20:02:26 it ignores most issues 20:02:27 I know it doesn't do packaging 20:02:38 geerlingguy: the problem being that we're not going to get there in time for 2.11 (We'll be hard pressed to design workable policy for 2.11 and I highly doubt we'll have time to then work to automate and ease the learning curve sufficiently for those policies until 2.13.) 20:03:02 ah, well yeah that is a fundamental model change in galaxy under the covers. roles were only indexed, whereas collections have to a published artifact that lands in pulp 20:03:32 My point is *AS AN END USER* (taking off my 'has done a lot of internal work lately'), collections are like 1000x more daunting than roles 20:03:36 roles mostly had very little checking overhead, collections carry much more responsibility, specially with documentation .. but its also bigger value to user 20:03:55 geerlingguy: end user of collections or author of collections 20:03:58 and part of the reason I hammer that drum is there are a lot more community members who might contribute roles and playbooks than Python modules and plugins 20:04:09 and one problem is that a lot of things for collections aren't specified yet. especially with regard to documentation. 20:04:09 geerlingguy: yeah, i agree ... i think things can be done to make that more automated (if the org had the desire and the time) 20:04:14 And what jtanner said right now is what enterprise users actually want. I talked to some of our customers and their experiences with roles are ... not the best. 20:04:16 bcoca: both—and speaking as someone who writes way more not-module-code than module code 20:04:35 geerlingguy: for that i would still continue using roles, but i know that im in minority there 20:04:57 collections are more complex by nature, they deliver a LOT more than roles 20:05:18 isnt it possible to 1) implement an easy way to automatically install collections, as soon as they are named in a task/role and b) split community.general in smaller, more focussed collections? firewalld for example is in the posix collection, but iptables is in builtin and ufw is in community. 20:05:31 not saying we should not create tooling to make coll author life's easier, but there is an expected overhead compared to roles 20:05:50 true, true, but right now the gap is a chasm. 20:05:56 daniel-wtd: part of the reason things are already spread out is that some stuff is Supported by RH, while other isn't 20:06:07 It's like going from an iPad to CLI-only Linux. Most people can't make that leap. 20:06:09 geerlingguy: i dont disagree, but also look at the diff in scope and features, big chasm also 20:06:29 daniel-wtd: and for the rest... splitting c.g completely up needs a lot more work on maintaining a long list of collections 20:06:34 roles have some yearss, collections are new, its expected teh tooling is not all there .. but we do need to make it better 20:06:41 daniel-wtd: but in general that would be great :) 20:07:25 hmhm, I thought it would be easier to maintain some focussed collections like community.firewall or community.storage instead of a huge general 20:07:39 bcoca: all true. But my main overarching point is still "if the problem was that centralized content is unmaintainable at certain volumes... how is having one/few central collections with the same volume of content going to solve that problem?" 20:07:44 We can't just clone felixfontein 8 times 20:07:44 geerlingguy: even with the tooling, always expect more overhead with collections, since there is more to maintain, docs alone is a good example 20:08:17 daniel-wtd: depends what you mean by maintaining :) if you mean making sure that CI is not broken, and sanity checks don't fail, a big collection is easier since there are less places where you have to work on 20:08:31 geerlingguy: totally agree, why my thinking is to eventually move away from that approach, but that is not the general thinking 20:08:43 felixfontein: understood :) 20:08:50 daniel-wtd: that is one option, we did a load of research into that and we found that people didn't contribute to more that one area under (say) storage 20:09:09 daniel-wtd: if you mean proper maintaining with implementing bugfixes, features etc., having smaller collections is definitely better 20:09:14 geerlingguy: actually... that is one option.... Not cloning felixfontein himself. But making sure that the number of empowered community maintainers goes up in proportion to the amount of content being added. 20:09:17 For end users, having everything in ansible/ansible was desireable for the 'one place to look' angle 20:09:47 minus roles 20:09:54 abadger1999: we still run into the same problems at scale that we hit with ansible/ansible though, just differently 20:10:02 ^ plenty of peopel asked us to ship roles in ansible/ansible 20:10:04 overall, whether things are all in one collection ofr in many, our success at keeping that ratio healthy is what will determine whether end users get a good experience or not. 20:10:22 yup 20:10:32 bcoca: We've avoided shipping roles in k8s collection, too specific for a generic group of plugins 20:10:33 daniel-wtd: Look at the tabs under https://stats.eng.ansible.com/apps/collections_structure/ to see about people contributing to more than on subdirectory under `lib/ansible/modules/` 20:10:53 I honestly wish roles could remain standalone and weren't being forced (someday?) into collections 20:10:56 geerlingguy: taht is the beuaty of collections, its up to maintainer to decide scope 20:11:02 geerlingguy: no. 20:11:17 geerlingguy: if ihad my way, it would never be forced 20:11:47 If we don't maintain a healthy ratio then whether we have a single collection or a lot of collections, then end users are still screwed. 20:11:57 abadger1999: +1 20:12:25 i would say the issue is that 'we' dont need to maintain all this .. nor ship it 20:12:31 ok. I think we really need to switch topics now. 20:12:35 but this also intends, that someone really cares about the modules/plugins 20:12:38 It does make it easier for us to communicate to end users that they're screwed (Collection A B and C are going to be removed because we don't have maintainers for them) but that doesn't help the users of A B and C any more than if that content was in a single collection. 20:12:41 I guess we have to continue next week with this topic 20:13:02 then there will be diff collections, well maintained, badly maintained , the big issue is making it clear to users in a good way, which is which 20:13:13