15:12:14 <bcoca> #startmeeting core public irc meeting
15:12:14 <zodbot> Meeting started Thu May 21 15:12:14 2020 UTC.
15:12:14 <zodbot> This meeting is logged and archived in a public location.
15:12:14 <zodbot> The chair is bcoca. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:12:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:12:14 <zodbot> The meeting name has been set to 'core_public_irc_meeting'
15:13:07 <cyberpear> o/
15:13:09 <bcoca> james ?
15:13:21 <cyberpear> good morning
15:13:41 <bcoca> #topic https://github.com/ansible/proposals/issues?q=sort%3Aupdated-desc+collection
15:13:56 <bcoca> @thaumos ^
15:14:02 * thaumos waves
15:14:27 <cyberpear> https://github.com/ansible/community/issues/536#issuecomment-630999040
15:15:12 <cyberpear> Collections - what are its goals and features?
15:15:33 <cyberpear> I kind of know "through the grapevine" but I can't find any design documentation, nor a proposal for it
15:15:46 <cyberpear> is the Proposal still the way big changes are made to ansible?
15:16:00 <cyberpear> s/made/agreed upon and-or discussed/
15:16:03 <sivel> not necessarily
15:16:42 <bcoca> it is ONE way
15:17:08 <sivel> it's a facility for external contributors to lay out plans for big changes they want to make, and get feedback before working on code
15:17:57 <bcoca> also for internal contributors when looking for feedback from the community
15:18:08 <cyberpear> so for plans made by "internal contributors", is there meant to be any transparency or feedback from the external contributors?
15:18:51 <cyberpear> or is it meant to be more of a "throw it over the wall" type of Open Source, with occasionally accepted community contributions?
15:19:23 <sivel> We do not necessarily ask for community feedback on all features or funcitonality we plan
15:19:32 <bcoca> neither and both
15:19:40 <cyberpear> so, I guess it's a specific, "what's the documented design purpose and goals of Collections", but brings about a more general "how is ansible developed in relation to its community contributorrs"
15:19:57 <sivel> Also, this blog post is largely the genesis of collections from an announcement perspective: https://www.ansible.com/blog/the-future-of-ansible-content-delivery
15:20:06 <thaumos> thx for pasting that sivel
15:20:14 <thaumos> I was just about to  as well.
15:20:25 <sivel> took me a little longer than anticipated to find it :)
15:20:47 <thaumos> @cyberpear, collections aren't a new thing we're just implementing now.  So the discussions for all of this date back to more than a year ago.
15:21:00 <cyberpear> yes, I'm aware it's been in progress for some time
15:21:11 <thaumos> What pre-dated even the Collection itself was discussions around the "ansible installer" as it was called before that.
15:21:21 <cyberpear> but for the 2.8 timeframe, there was no public mention of it at all, and I only became aware of it because it was blocking the 2.8 release
15:21:54 <cyberpear> and back during the 2.8 cycle, I was reading all the chat in #ansible-devel channel
15:22:06 <cyberpear> (I don't have enough bandwidth for that in the past few months, though)
15:22:14 <bcoca> cyberpear: its been something that has been brewing for 6 years, 2.8 was just hte moment we decided, we cannot go on like this w/o it
15:23:03 <cyberpear> why all the secrecy?
15:23:15 <thaumos> I am sorry, but there has been no secrecy around this.
15:23:24 <thaumos> We have and continue to communicate our goals with this.
15:23:28 <sivel> https://github.com/ansible/ansible/projects/30#card-13708579
15:23:40 <sivel> That is from the project board linked in the 2.8 roadmap
15:24:03 <sivel> Maybe, let's step back for a second.  What is the *real* goal of this discussion?
15:24:17 <sivel> What do we hope to accomplish here?
15:24:49 <sivel> I'm not sure convincing you that collections are beneficial is a worth while goal
15:25:49 <gregdek> @cyberpear We've communicated this aggressively in several channels and at every Ansible Contributor Summit for the past year.
15:26:07 <cyberpear> I guess I'd like to see better documentation of "the ideal" trying to be achieved with the collections concept
15:26:09 <thaumos> more than that gregdek, and thanks for chiming in.
15:26:17 <cyberpear> are there internal design documents that could be published?
15:26:35 <sivel> cyberpear: what would that help us achieve at this point?
15:26:35 <cyberpear> I assume the driving force is "too big  backlog" on ansible/ansible
15:26:55 <gregdek> Did you read the blog posts that thaumos pointed you to?
15:27:07 <thaumos> A lot is in flux at the moment with regards to documentation, the ansible repo, collection structures for the community, etc.  So of course, the eventual goal is to document this all approrpriately.
15:27:10 <bcoca> cyberpear: no, that is a side effect
15:27:27 <bcoca> cyberpear: read blog, it states the goals
15:27:51 <gregdek> https://www.ansible.com/blog/thoughts-on-restructuring-the-ansible-project
15:27:55 <thaumos> @cyberpear, the main driving force for this originally was to enable our content creators a means to provide content at a much more rapid pace than what the core framework could provide on it's own.
15:27:56 <cyberpear> I'll re-read it.  I didn't read it today.
15:28:27 <cyberpear> but I definitely read each when it was published
15:28:35 <sivel> cyberpear: I'm still interested to know what we hope to accomplish with this discussion.  What is the real goal/puprose?
15:28:52 <sivel> What outcome would you like to see as a result?
15:30:07 <cyberpear> sivel: 2 parts.  1. better goals documentation, both conceptual and on-the-ground technical of the Collections world, though at this point, it'd likely be "as-implemented" rather than "as-designed" docs
15:30:27 <cyberpear> 2. better transparency for changes going forward, with more opportunity for community input
15:30:41 <thaumos> So yes, like I said earier. We have a lot in flux.  The goal is to have documentation around this.
15:30:46 <bcoca> cyberpear:  i can show you initial docs, they resemble little of what we have, cause it is a continious process with community and user feedback
15:31:07 <thaumos> Secondly, I don't know how much more transparent we could have been about this subject.  As gregdek and I and others have said, we have communicated a lot about this.
15:31:42 <gregdek> From my perspective, the post I linked is clear about the goals, and pretty much laid out the conceptual roadmap that we've followed since.
15:32:06 <cyberpear> okay, I'll re-read those.
15:32:08 <bcoca> also, its not only core team that wrote this, many parts are being written by community members
15:32:25 <bcoca> others are just influenced by them
15:32:26 <cyberpear> I'm also concerned that collections are by-and-large... too large
15:33:00 <bcoca> well, depends on each collection and there is much debate on that subject,  ... and that will probably change in the future as feedback is incorporated
15:33:01 <cyberpear> in the roles world, I could download use a single role, or a single module distributed as a role
15:33:18 <bcoca> you can still do that will collections
15:33:19 <cyberpear> now in the collections world, I'm forced to download the entire collection instead of just the part I need
15:33:35 <gregdek> Roles will continue to be supported.
15:33:43 <bcoca> also, not really true what you stated
15:33:51 <cyberpear> roles are getting migrated from repo-per-role to monorepo-per-collection-of-100-roles
15:33:52 <bcoca> you needed to download the plugin and 'everything it depends on'
15:34:02 <cyberpear> sure, there's dependencies
15:34:20 <cyberpear> and finally w/ 2.10, the meta/requirements.yml is a thing that's honored by the tools
15:34:32 <bcoca> that still works the same, the only big diff, is from role you could throw away structure and use library/ with collection you still need the structure (just not all the other content)
15:35:38 <jtanner> i think we (the core team) has as much control over the size and shape of all collections as much as the rpm package inventors have over all the rpm packages  out in the world. we build a framework and it's on the framework users to define what they want to put in their widget
15:35:51 <cyberpear> but it's a lot harder to convince a corporate security departement, (either for software I'm shipping, or just for internal use) that my dependencies have been audited -- it's easy to audit a single role... much harder to audit a collection of 100 roles
15:36:00 <cyberpear> sure, but you do push users toward one way vs another way
15:36:11 <cyberpear> the galaxy.ansible.com is pushing people away from publishing roles
15:36:15 <bcoca> cyberpear dont use the collection with 100 roles ...use one with 1 role
15:36:16 <cyberpear> and toward collections
15:36:25 <bcoca> not that roles are all same size and tiny
15:36:31 <thaumos> @cyberpear, but to jctanner's point; it is up to the content creators/curators whomeever that is creating that collection to make it that way.
15:36:33 <bcoca> i've seen roles with 100s of task files and plugins
15:36:50 <thaumos> Not every collection will have 100s of roles or whatever else could be in it.
15:37:09 <thaumos> The same structure could exist as you are suggesting.  There could be a collection with a single role and a plugin.
15:37:13 <bcoca> collections does not alter the scope/size of content, just like roles , its up to the author
15:37:16 <thaumos> The directory structure is just changed.
15:37:30 <cyberpear> specific to the collections that have been split off of core, there's now copied libraries and/or module_utils, which makes maintenance harder for those components -- which copy is canonical?
15:37:40 <bcoca> collections just make it a LOT easier to distribute and consume that content (no need to ikmport/execute role tasks for access to a plugin)
15:38:02 <bcoca> cyberpear: good question and a problem the maintainers will have to solve
15:38:12 <cyberpear> the directory structure is quite annoying, and I haven't checked recently if the stupid `collections/ansible_collections` structure is still required
15:38:19 <bcoca> but that is not part of what the colleciton design is meant for, just what content authors chose to do
15:38:28 <bcoca> cyberpear: it is
15:38:37 <thaumos> The way that the community.general collection has been structured is a result of mustering people together as resources to curate together.  It is not meant to be the standard by which everyone operates.
15:39:18 <geerlingguy> (what I do is set collections_paths=./, and that puts collections in ./ansible_collections/namespace/collection). But I'm still planning on using roles for my playbooks outside of collections, so they can be in ./roles/[rolename] without namespacing)
15:40:00 <geerlingguy> (otherwise for simple local roles that I don't share on Galaxy, they'd be in ./ansible_collections/example/local/roles/[rolename] which is just not fun to navigate when working on playbooks ;)
15:40:31 <cyberpear> the one big benefit I see for ansible content creators is distribution of playbooks in a standard way, but `ansible-playbook namespace.collection.playbook` apparently isn't planned for 2.10
15:41:01 <thaumos> There is a lot still to be implemented.
15:42:07 <thaumos> This isn't done.  However, we do have to pick and choose what we can do with the resources that we have. Right now, it felt like all focus should be on getting content into collections.  That effort required focus from all the team.
15:42:24 <thaumos> After this release, we will have more to do with collections.
15:42:25 <bcoca> cyberpear: i have a PR that might make it, its not not planned, we just are not going to hold the release for it
15:42:42 <cyberpear> I think that in a collections, world, it'd be much more feasible to implement something like Fedora's Change Process for all changes, given that the core is much smaller.
15:43:01 <cyberpear> https://docs.fedoraproject.org/en-US/program_management/changes_policy/
15:43:03 <bcoca> core wont handle content outside ansible/ansible
15:43:13 <cyberpear> right
15:43:16 <cyberpear> but for core itself
15:43:42 <cyberpear> and maybe for ACD
15:43:48 <cyberpear> ^ is that still what we're calling it?
15:44:02 <gregdek> Bear in mind that it took Fedora a decade to get to that change management process.
15:44:06 <bcoca> also note that like roles, collections will mainly be 3rd parties, we will have no say on what/how they are operated or distributed .. nor do we want to have a say
15:44:27 <gregdek> And no, ACD is now just Ansible.
15:44:38 <cyberpear> yes, I well understand that third parties do whatever they want
15:45:00 <cyberpear> so ACD->"Ansible" and the ansible/ansible -> "Ansible Base"?
15:45:01 <bcoca> and ACD is not under core, it has it's own team
15:45:19 * cyberpear confused by changing terminology
15:45:24 <cyberpear> right
15:45:34 <bcoca> u are not alone, i just found out about the change as you did
15:45:42 <gregdek> That is correct.
15:45:50 <cyberpear> I brought up separately that ACD should have its own meetings, and I believe they're now scheduled for Wednesdays
15:46:28 <cyberpear> "Ansible Base" or "Ansible Core", or "Ansible  Base maintained by the Core Team"?
15:46:36 <sivel> ansible-base is ansible/ansible
15:47:06 <sivel> effectively ansible-base is still "ansible", but from a packaging perspectice, "ansible" is what we used to call ACD
15:47:46 <sivel> project vs package
15:48:36 <cyberpear> can we make a glossary?
15:49:05 <cyberpear> ansible/ansible is the ansible-base package maintained by Core Team
15:49:24 <cyberpear> ACD (does it have a repo to track what goes in?) is the ansible package maintained by the Community Team?
15:49:31 * bcoca tempted to drop ansible=base for ansible-core to minimize num of names
15:49:58 <gregdek> I think a lot of these conversations are actually better for the Wednesday meeting, where we can expect your help :)
15:50:00 <sivel> cyberpear: ACD doesn't really have code. It has some build scripts, and the community team is responsible for packaging "ansible"
15:50:29 <sivel> the 2.10+ ansible package, only contains bundled collections, and has a dependency on ansible-base
15:50:38 <cyberpear> it has some code or machinery that determines which parts get pulled in, though
15:50:49 <gregdek> That is correct. The build code.
15:51:11 <gregdek> The goal of which is to provide continuity to users who want to continue to consume batteries-included Ansible.
15:51:33 <sivel> https://github.com/ansible-community/ansibulled/
15:52:39 <cyberpear> is batteries-included version expected ever to go away?
15:53:00 <cyberpear> (will there be stats on who installs the "ansible" package vs just "ansible-base"?)
15:53:47 <sivel> too early to answer that first question
15:54:12 <sivel> that will depend on stats, which I assume we'll have via some method
15:54:16 <cyberpear> I realize some of these might be better questions for the newly established ACD meeting... I hope it gets some turnout...
15:54:33 <thaumos> I don't think there is a plan to get rid of the batteries included model.
15:54:35 <gregdek> "ever" is a long time.
15:54:53 <gregdek> There are no immediate plans to get rid of the batteries-included distro, certainly.
15:54:54 <cyberpear> right, ansible might be replaced by opsmop, some day :P
15:55:02 <sivel> lmao
15:55:30 * bcoca suspects cfengine will be only one left 300 yrs from now
15:56:12 <bcoca> cyberpear: consider this .. with collections you can create your own ansible package customized the way you want
15:56:14 <bcoca> ansible-aws
15:56:17 <bcoca> ansible-azure
15:56:23 <bcoca> asnible-nomysqlbuteverythingelse
15:57:58 * cyberpear typing
15:58:36 <cyberpear> so I'd propose: 1. glossary of terms in the New World Order (and simplification/deconflict as appropriate) 2. do more to involve the community in /designing/ future changes (not just implementing) (ansible-devel mailing list?) 3. better docs for /design/ of collections, even after the fact
15:59:02 <gregdek> Sounds like an excellent agenda for our Wednesday meeting.
15:59:15 <bcoca> cyberpear: define 'do more' .. since the process has alwasy invovled the community
15:59:41 <bcoca> unsure what else you want there, communty feedback was incorporated, community devs are contributing
15:59:41 <cyberpear> send proposed changes to the ansible-devel list before they're ultimately decided upon
16:00:00 <bcoca> that was done
16:00:07 <bcoca> also blogs
16:00:29 <bcoca> and literally 6-7 yrs of conversations, this startted before IBM and even RH aquisitions
16:00:43 <bcoca> i remember greg and i discussing this at the 2nd ansible co all hands
16:01:00 <bcoca> and the same in the irc channel and MLs
16:01:18 <cyberpear> bcoca: can you point me to the ML archive about collections?
16:01:28 <bcoca> not the 'name' but the concept, yes
16:01:49 <bcoca> read scrollback, at one point this was called ansible-installer (i believe thaumos mentioned)
16:01:53 <bcoca> it has gone by MANY names
16:01:56 <cyberpear> (it's possible it was before my time, but I've been on there for probably 2 years or more)
16:02:37 <bcoca> apb ansible playbook bundle .. orb .. many many bad names ...
16:02:37 <sivel> When I was hired in Dec 2017, we were discussing it then, and had been discussed far longer
16:02:48 <cyberpear> indeed, changing the names makes it very hard to track a concept... especiallly if the introduction doesn't say "previously known as XYZ"
16:02:54 <bcoca> ^ and sivel (like myself) was hired from the community
16:04:05 <sivel> but yes, at around the time I was hired, bcoca was using the name "ansible-installer"
16:04:12 <bcoca> well, we are over time and at this point i thnk we can close the meeting, to be continued in ACD weekly
16:04:38 <cyberpear> seeing "ansible-installer", I would have glossed over it thinking, "that's what RPM is for"
16:04:44 <cyberpear> "or pip"
16:05:01 <cyberpear> but now that I know, I can trawl my archives
16:05:02 <bcoca> 'content installer' .. yes, one of many reasons that name was dropped, also it was very confusing (does it install ansible?)
16:05:35 <bcoca> cyberpear:  but name is the least of it, i probably have 100s of hours of discussion about the concept w/o naming it in this and #ansible-devel channels
16:06:00 <bcoca> a few posts in the ML, but i dont think we ever had a pure discussion thread on it until around 2.7/2.8
16:06:10 <bcoca> also .. lack of name .. makes it hard to pinpoint
16:07:07 <cyberpear> I'd like to see more discussion on the mailing list since you can have threads related to a topic more easily, and better searching
16:07:27 <cyberpear> (also, let's kick the "please help me use ansible" folks over to ansible-project list)
16:07:54 <bcoca> why? that is one of the reasons that list exists
16:08:09 <bcoca> to help people use ansilbe, ansible-devel is for those developing ansible or content for asnible
16:08:11 <cyberpear> I assumed there was ansible-devel list for folks developing ansible
16:08:20 <cyberpear> and ansible-project for announcements and end-user support
16:08:32 <bcoca> also  ansible-announcements
16:08:37 <cyberpear> also that
16:08:42 <bcoca> plase help me use ansible seems 'user support to me'
16:08:48 <cyberpear> maybe we need a new list for development of ansible itself, not ansible content?
16:09:10 <bcoca> k, add that to next agenda, 10mins over time and need to close this one
16:09:17 <bcoca> considering original point discussed
16:09:19 <bcoca> #endmeeting