15:03:32 #startmeeting ansible core irc public meeting 15:03:32 Meeting started Thu Jun 27 15:03:32 2019 UTC. 15:03:32 This meeting is logged and archived in a public location. 15:03:32 The chair is bcoca. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:03:32 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:03:32 The meeting name has been set to 'ansible_core_irc_public_meeting' 15:04:01 #topic ansible collections and shipping of community modules 15:04:10 @cyberpear i hear you had some concerns last night? 15:05:17 .hello2 15:05:18 maxamillion: maxamillion 'Adam Miller' 15:05:21 yeah, I heard "through the grapevine" that community modules were going away from a default install of ansible 15:05:49 the grapevine is not necessarily a reliable source of information 15:06:12 cyberpear: grapes are rancid in this case 15:06:12 agreed, which is why I come here for reliable info 15:06:17 we have some Ansible community folks here, but they may be slow to respond 15:06:40 So. :) 15:06:44 ^ 15:06:49 cyberpear: there is very old proposal of having 'ansible-minimal' package (mine from 5-6 yrs ago if you must know) but that was as ALTERNATE and not substityte of current kitchen-sync 15:06:56 from discussion in -devel last night, it sounds like there may be an effort to remove all/most modules from core, and package them separately as collections 15:07:19 cyberpear: yes and no 15:07:27 I think gregdek is responding, so let's wait for that 15:07:34 then the question would be "which collections get included by default?" 15:07:39 That's correct from a development perspective. From a deployment perspective, the answer is not clear yet. 15:08:17 Likeliest scenario, from my perspective, is essentially the Fedora/RHEL equivalent. 15:08:32 cyberpear: also note that packaged as collecitons does not mean that they wont be present in rpm also 15:08:32 huh? 15:09:21 how might that work with a pip install? 15:09:22 This is a much longer discussion that's past due, to be honest, so what I say here is still in the category of "figuring out details". 15:09:28 But here are the issues: 15:09:36 1. Core is way too big to be supportable long term. 15:09:41 2. Modules need to move out. 15:10:06 * agaffney wonders how many times that modules are going to be moved into and out of the main repo :) 15:10:07 s/Modules/plugins/ but moduels are by far the largest 15:10:23 * bcoca smacks agaffney with ansible-extras 15:10:24 3. We need to identify the difference between "modules that are supported by someone officially (whether RH or partner)" and "modules that are supported by the community". 15:10:33 agaffney: I think the answer to that will perpetually be "at least once more" 15:10:40 At least once more. :) 15:10:46 boom 15:10:55 o/\o 15:11:08 But the data is clear: we're just too big to be able to sustain current processes indefinitely. 15:11:24 truth 15:11:36 It's clear that collections is a key piece. 15:11:43 another problem is that people get the impression that anything in ansnible/ansible is supported by the core team .. which most of people here, KNOW is not the case 15:11:46 How those collections will be organized is still a point of discussion. 15:12:09 ^ collections also allow for other teams to have quicker release schedules 15:12:10 bcoca: right. We desperately need clarity on that point for the sake of the business long-term. 15:12:20 aside from "supported" or "not supported", i think there's also a misconception that modules delivered with ansible are all functional 15:12:29 or tested 15:12:41 We should still have those expectations, btw. 15:12:48 or have a certain quality 15:12:48 collections doesn't necessarily fix that, but the whole ecosystem we're trying to build might help a lot 15:13:05 we do test and pass minimal quality for modules, but it is far from what core modules are expected to be 15:13:11 collections at least makes it more clear that it's not core's responsibility 15:13:22 maybe 15:13:32 also will show which modules are 'orphan' 15:13:51 and it does prevent teh 'drive by' syndrome 15:13:54 we also haven't decided where the code goes, if we even pull them out of the main repo, at least in the short term 15:14:15 So regardless of the development process, which we still have to iron out, we'll have a lot of deployment options. The option I prefer at present is "a big bundle of all the officially supported modules" and "a big bundle of all the community supported modules". 15:14:29 the only part we are 'mostly sure of' is 'partner collections' , external teams that are already maintaining their own modules/plugins 15:14:51 We can dig in more, but that's the gist of it. 15:14:55 regardless of where they live, I think there should be a process to have community modules included in a default install of ansible 15:15:20 cyberpear: sure, could be done via an ansible-kitchen-sink RPM :) 15:15:22 cyberpear: code maintenance and 'shipping' are not the same, even 'core modules' might end up in a collection 15:15:23 alll of them? 15:15:28 kind of like fedora packages: community can maintain a package, but if it's orphaned, it's dropped from the next release 15:15:29 I think the terraform model makes a lot of sense here, where each "provider" lives in its own separate git repo (not necessarily maintained by the terraform devs) and terraform is only shipped as the main binary, and all providers are downloaded on the fly as plugins 15:15:52 agaffney: our history of "batteries included" makes that a jarring change. 15:16:13 agaffney: that was my 'ansible-minimal' idea, but 'kitchensync' is way too popular for a large number of users 15:16:22 so its always an 'alternate', not a 'substitute' 15:16:27 cyberpear: it may well be that this is the moment of the Fedora/RHEL split equivalent of Ansible. And it depends on what we call "Ansible". 15:16:36 Is "Ansible" Fedora or RHEL? 15:16:41 gregdek: yes, that's the main concern; since everything was "batteries included" and a big selling point 15:16:43 there are technical solutions, such as a kitchen.sink collection with is just a list of all versioned collections as dependencies 15:16:53 which* is 15:17:06 We can easily bundle an "everything install". But that won't be the thing that the business supports. 15:17:09 the "batteries included" problem can be solved with packaging without affecting the way that development is done 15:17:24 in any case, we are far from any decisions, and as such, we don't have much in the way of describing what things will look like. 15:17:37 I think we know enough to describe the likely options, though. 15:17:42 i very much doubt we'll ever stop shipping 'kitchen sync' ... too many people like that option 15:17:46 so I wouldn't necessarily worry yet, and I am sure there will be a lot of time for community involvement 15:17:56 +1 bcoca - that's a really good point that seems to be lost, we could even move to a point where even "core modules" are in a "core collection" and everything is basically either "the ansible runtime engine" (or whatever you want to call core) and then *everything* is upstreamed as a collection, then a set of collections come together via a release/distribution mechanism 15:18:04 And simply not discussing them is just making people nervous. I mean, we've built up some trust, but that's not indefinite. 15:18:13 agaffney: +1 15:19:10 once jordan's galaxy cli PR is merged, i think conversations are going to have a lot more substance and tangible validation of assumptions 15:19:28 well, part of it is we dont have anything set in stone and the needs/wants have been the same for 6yrs 15:19:43 ^ once we have 'tangible', yes 15:20:52 I'm just hopeful that a good default set of collecitons will be pulled in by an RPM install... As a sysadmin, I abhor things like 'pip install' for system packages; same for core ansible functionality 15:21:11 i think that is a reasonable request 15:21:17 cyberpear: +1 15:21:34 it would also be nice for that to happen with a pip install for parity 15:21:41 i'd like a minimal ansible that still remains useful for most things ... where to draw the line is the trick 15:21:57 cyberpear: my initial proposal was for rpm packages .. which we could do from collections easily ... 15:22:17 or epel could do it 15:22:20 so many options 15:22:28 Do we have a "wish list" and or list of "user stories" somewhere for people to view? 15:22:28 jtanner: you and i have <200 modules ... many others would disagree ... 15:22:42 ie: cyberpear literally just gave the "as a sysadmin" example 15:22:52 rbergeron: feature request github issues 15:23:03 or do you mean specific to this topic? 15:23:07 ( a lot of places frown on epel, unfortunately, as useful as it is) 15:23:10 i meant specific to this topic :) 15:23:46 cyberpear: true, but it does make a nice transition step between inclusion in fedora and inclusion in rhel 15:23:53 rbergeron: 'as a sysadmin' i like fine grained control on what is installed to keep 'misuse surface' at a min 15:24:05 any packaging of collections should not be specific to RH, imo. it should extend to pip and the ansible-built RPM/DEB packages 15:24:11 so there are many stories .. just as much as corps/envs/users 15:24:19 agaffney++ 15:24:30 agaffney: yes, when i say rpm, 'deb' is implied 15:24:37 and probably tgz 15:24:41 well, EPEL was mentioned 15:24:57 cause wer are RH ... so do expect home bias 15:25:13 agaffney: i'm not sure pip can set the install root somewhere other than site-packages easily 15:25:22 but we do distribute .debs and that will not change in forseeable future 15:25:22 Right, I think EPEL is probably an obvious choice for our version of that -- but we're going to need packaging rules for other distros. 15:25:34 jtanner: pep370 15:25:35 That will likely be parallel. 15:25:38 jtanner: I'm not either, which is why I asked earlier (or maybe in the -devel conversation last night) if it was feasible with pip 15:25:58 gregdek: we already host a ppa, which is 'epelish for debian' 15:26:03 I guess, I'd like to see most currently-available modules available right next to ansible, even if it's an additonal `yum install` or equivalent 15:26:06 rbergeron: +1 - it's probably a good idea to add something to a github project space as user story cards and such 15:26:14 (sorry, multi tasking and trying to catch up) 15:26:35 then have a separate process of making new collections into "officially recognized high quality" but community collections 15:26:37 some data for thought http://tannerjc.net/ansible/galaxy.html 15:26:51 (similar to submitting a PR for a new ansible module today) 15:27:13 jtanner: how far down in that list before we hit a non-core-supported module? 15:27:17 @maxamillion mostly, i'm just looking at a lot of good ideas / suggestions / thoughts coming from this convo and wondering what the action items are from this meeting or what to do with that feedback 15:27:28 rbergeron: + 15:27:31 *sigh* 15:27:32 rbergeron: +! 15:27:34 omg 15:27:39 even if the answer is "we're just talking it out right now because we need to talk about it more" is fine 15:27:39 nvm, you get it 15:27:42 jtanner++ love the data! 15:27:44 agaffney: docker_container i believe 15:27:46 cyberpear: one thing we are doing for galax/collections is enabling the same testing infra (actually galaxy will have more, molecule, lint, etc) 15:27:49 #30 15:28:40 agaffney: and even then, there's only a few scattered in there that are not core supported until like ... 50? 15:29:05 which might imply that some of those modules *should* be core-supported 15:29:20 Too much of the discussions here have been internal at this point. 15:29:21 agaffney: opinions on that vary A LOT 15:29:34 I'd be surprised if they didn't 15:29:46 That's the crux of the issue: we need to move key discussions here from internal to external. 15:29:47 many currently core modules i would kick out 15:29:58 * bcoca stares at lineinfile 15:30:11 * agaffney hugs his precious lineinfile 15:30:12 People love lineinfile. You're stuck with it. :) 15:30:16 it's good for the sumple things 15:30:16 * thaumos stares at synchronize 15:30:21 does not make me stop wishing 15:30:29 thaumos: that is probably unanimous in core 15:30:31 maxamillion: but i hate not having a way to take the comments and put them ... somewhere, or enable folks who aren't in this meeting or can't ever make this meeting time to easily weigh in 15:30:52 rbergeron: was thinking we start a thread in ML 15:30:59 rbergeron++ 15:31:00 cyberpear: Karma for rbergero changed to 1 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:31:02 i have assumed for a while now that this was a big topic for fest contrib summit this year 15:31:18 rbergeron++ 15:31:19 maxamillion: Karma for rbergero changed to 2 (for the current release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:31:49 cyberpear: we've also been waiting to actually have tooling for collections and a 'tangible thing' before bringing up this discussion again 15:31:57 oh man, i do not need to accumulate more fedora karma :) 15:32:00 (unfortunately, it's hard to make it to the summit) 15:32:02 we HAVE discussed this at previous fests and online in public meetings many times 15:32:27 we have not discussed in a while cause we always 'went nowhere' .. now its different, we have someething tangible 'collections' 15:32:47 at a certain point, it's not really feasible to make accomidations for all the people that may want to take part in these discussions while still keeping the discussion followable by everybody 15:32:52 and soon (no pressure jborean93) we will have the tooling in devel to use them properly 15:33:02 cyberpear: we do try to webex/bluejeans the contrib summit 15:33:26 cyberpear: also irc channel is used if webex/bj is not possible 15:33:45 agaffney: maybe it's at least providing a good summary of recent discussions (be it in regular meeetings, contributor summit, otherwise) 15:34:03 rbergeron: sure 15:34:07 rbergeron: a periodic 'state of the union' ? 15:34:18 jordan is almost done, it'll be very soon 15:34:28 in more obvious places -- mailing list, blog post, whatever. we do have a ton of contributors who are able to follow along here, but we have a ton of users and more potential contributors out there they don't even know that they might want to actually follow along or participate in this discussion. 15:34:59 * bcoca votes for gundalow to start a weekly/montly newsletter/forum about ansible development 15:35:01 bcoca: maybe. or more like "here's the big topic and uh, here's the last 2 weeks of discussion, and the places in the next two weeks where you can participate, and here's the highlights" 15:35:15 +1 15:35:31 right now we do this mostly in irc .. but hard to read across the noise 15:35:39 proposals were a way to start this, but stagnated 15:36:04 ^ we could try to revitalize those 15:37:02 not everyone is fortunate enough to be able to follow ansible-devel / mailing list / etc. as their full time job. 15:37:24 and i think proposals without some regular mechanism to advertise what is actually goign on would suffer the same fate as everything else 15:38:02 lots of module maintainers and role developers do that as a side part of their job, not their full time job, and they have fires of their own -- we may need to be sticking things more in their faces. 15:38:08 anyway. 15:38:23 It may be that current mechanisms are ok-ish, but we need better focus mechanisms. 15:38:25 Fedora Magazine? 15:38:31 (not sure how much exposure that gets) 15:38:43 Fedora has a magazine? 15:38:47 that's what i'm throwing out there -- aside from the question of, where do we go with the feedback from this meeting? 15:38:53 Funny you should mention, we've got a new team member focusing on a regular Ansible column on opensource.com 15:38:55 agaffney: they do, it's reasonably popular https://fedoramagazine.org/ 15:38:56 agaffney: it's like a virtual magazine 15:39:02 agaffney: lots of good stuff there 15:39:20 i personally found that keeping up with what was happening under the hood was better served by reading planet.fp.o 15:39:36 regular 'official' reddit post? 15:39:37 but i think people don't blog as much these days 15:39:53 we're all youtube stars now 15:39:54 alloftheabove? 15:40:04 jtanner: as long as pants are still optional ... 15:40:20 * rbergeron wonders if someone is operating meetbot 15:40:32 #delete previous comment 15:40:38 nope, not working 15:41:14 Man, I miss planet. 15:41:22 cyberpear: can i assume this answers most of your questions? rest is ongoing process, we will be looking at making it better/more inclusive 15:43:00 who started the meeting? :) 15:43:03 for the sake of the next item, im goign to consider silence a 'yes' and move on 15:43:04 yes, it's good to have some clarification... and would be appreciated to have more discussions in the public, kind of like fedora-devel list 15:43:09 #topic make ini_file core 15:43:18 (meetbot seems to mostly just record the conversation and the topics; not much more features used here in ansible-meeting) 15:43:29 rbergeron: me 15:43:30 so ini_file is #46 on th elist posted earlier 15:43:39 yeah, i was just thinking it might have been useful to at least notate some of the ideas. oh well :) 15:44:05 cyberpear: sevearl on top that are not core either 15:44:27 more a general question of "what belongs in core?" as a start 15:44:38 rbergeron: i post link with logs on agenda ticket, but we dont ussually create action items 15:45:00 I am inherently against modules that modify small bits of files. I'd love for things like lineinfile to become not core 15:45:07 just my personal opinion 15:45:15 cyberpear: mostly 2 things a) stuff that is basic for managemnet b) stuff that we decided was business critical 15:45:33 cyberpear: having template and replace i would consider all other 'piecemeal format modules' to not be core 15:45:45 specially with something that has as many non standard standards as ini 15:45:55 sivel: unfortunately, they are very useful in certain situations, and managing an entire file with 'template' is not always a viable alternative 15:45:55 bcoca: replace isn't part of core, I believe 15:46:09 Also, we tend to not be the ones that make the final decision. We can make recommendations, but promotion to core is a business decision as well 15:46:46 any it's only happend twice: once to mark all of them, and then again with reboot 15:46:50 cyberpear: i would prefer repleace be core and lineinfile is not, some are 'historically grandfathered' 15:47:14 cyberpear: 'core support' also implies 'redhat paid support' 15:47:52 So if you feel strongly, I suppose a proposal is the best way forward, but keep in mind, that as mentioned before, it's a business decision, and we've only ever reclassified 1 module to make it core 15:48:05 which one was that? 15:48:09 reboot 15:48:22 and any decision is likely not to be made in this setting 15:48:24 yeah, 'core support' implies "if you pay for Ansible and call to yell at Red Hat, someone's going to look at the issue" 15:48:34 with an SLA 15:48:42 sweet sweet SLA 15:49:04 not that it stops for people for asking same for non core modules, but at least we get to point at something to say no, it isn't" 15:49:25 SO a proposal would be referred to the business powers that be, to make such a decision 15:49:26 on the other side of that, calling certain really popular modules "community supported" is a bit of a cop out 15:50:33 agaffney: could be, but its our commitment to make 15:50:44 popularity wasn't really something we had tried to figure out how to measure when the metadata was initially created 15:50:46 and 'our' means not just 'core devs' but RH as co 15:50:58 I just thought of a random question about collections...what happens if two different collections ship a module of the same name? 15:51:02 On top of SLA, it means that someone who is paid by Red Hat will ultimately be responsible for maintaining it 15:51:07 agaffney: namespaces 15:51:08 and there are finite resources 15:51:08 agaffney: the business has to draw a line in the support matrix somewhere, and while the core team does as much as we can, we do have priorities that we have to focus on ... it's just a reality we face, it's not some disingenuous, "we won't touch it because it's not core" 15:51:17 i'm still not completely sure how to classify "popularity" of a module, and my hope is that galaxy will get us closer to an answre 15:51:32 jtanner: who uses caps most in irc? 15:51:40 k, going on tangent 15:51:41 Re: "calling certain really popular modules "community supported" is a bit of a cop out" -- this is the exact same problem that EPEL/RHEL has. The business makes a decision. 15:51:51 agaffney: also note that in that statement I mean "core team" as "the core team inside red hat" ... I do recognize and agree that the "core team" in community space is a superset of those at Red Hat 15:51:56 quick vote for those that want to start the process of making ini_file core? 15:51:58 -1 15:52:08 gregdek: +1 15:52:10 +1 15:52:12 each module is a "product" 15:52:14 bcoca: -1 15:52:32 -1 15:52:36 maxamillion: ? 15:53:04 what exactly does it mean at this point for ini_file to become a core-supported module? the question doesn't really make much sense (imo) until there's the possibility that it won't be shipped with ansible 15:53:28 (I wouldn't care so much but for the uncertainty of the previous conversation, which was originally based on FUD, but business decisions are sometimes made on FUD) 15:53:31 means that core will put it on maintained list and will be part of triage/work list for the core team 15:53:36 agaffney: that Red Hat will put an SLA behind it for paying customers, and that the core team is ultimately responsible for maintaining it 15:54:09 and this vote will not decide if it makes it on the list, just if there is enough 'core team' support to even try 15:54:27 -2/1 (idk w maximillian is voting on) 15:54:32 I personally don't care about the RH support aspect of it, as I don't pay for support, and I've never filed an issue against ini_file, so... *shrug* 15:54:33 From the perspective of us ultimately being reponsible for maintaining it, I am -1 15:54:33 ^ need more votes 15:54:39 I cannot talk about SLA 15:54:45 -1 15:55:30 agaffney: core tends to ignore 'community' modules we do go over 'core labeld' ones, so even if not paying for support it might get you more attention 15:56:11 so -3/+1 .. no one else want to vote? 15:56:17 bcoca: I was voting on what you said with a -1 and I was previously throwing a +1 at gregdek in agreement with what he said (which I realize from the backscroll is a bit confusing) 15:56:31 so -4 15:56:42 maxamillion: confused, since its the same thing, its also a RH call between rhel/fedora/epel 15:57:08 bcoca: wait, what? 15:57:34 bcoca: I was saying +1 to " Re: "calling certain really popular modules "community supported" is a bit of a cop out" -- this is the exact same problem that EPEL/RHEL has. The business makes a decision." 15:57:47 bcoca: and voting -1 against ini_file being core supported 15:58:01 so you are +1 to ini_file ? 15:58:09 No, he is -1 to ini_file. 15:58:18 i thought you were -1 to me saying 'it might be copout but its RH decision' 15:58:21 And +1 to my glorious wisdom. 15:58:36 gregdek: its always +1 to your glorious wisdom 15:58:52 Gerrymandering. 15:59:01 gregdek: too soon man, too soon 15:59:15 maxamillion: you said -1 to what i said, and i said -1 .. so double negative, please just '-1 the thing directly' 15:59:16 bcoca: yes, -1 to ini_file 15:59:28 bcoca: +1 to gregdek glorious wisdom 15:59:29 or you confuse math geeks like me 15:59:53 -1 * -1 = 1 16:00:45 cyberpear: sorry, seems -4/1 against, still this does not mean we will stop shipping it as per previous discussion 16:00:56 so sounds like core itself doesn't want to support more modules, ack 16:01:13 to be honest i dont think we CAN support our current load 16:01:19 much less take on more 16:01:22 too many core modules, too few people 16:01:36 too many modules ... 16:01:46 which is a good problem for a project, but sucks for those maintaining it 16:01:48 bcoca: sorry, my irc != reasonable math 16:02:27 maxamillion: i'll use it to tease you in the future 16:02:36 not to mention, core people being sent off to work on other non-core things which reduces the throughput potential as it is ... >.> 16:02:39 bcoca: that's fair 16:03:08 cyberpear: even if we were not over the limit, i would still consider this a 'non core module' for tthe reasons i gave above 16:03:35 i've also said the same to json/yaml edidting modules (actually outright rejected them when core was the gatekeeper) 16:03:37 RH could pay a few of the more active community folks as part-time core maintainers ;) 16:03:49 we've slowly hired all of them 16:03:51 agaffney: but you are doing it for free now! 16:04:06 <= how i was hired 16:04:10 with minimal motivation 16:04:21 agaffney: like money would change that for you! 16:04:29 and I generally don't touch modules 16:04:34 it would help :) 16:04:38 ;-p 16:04:50 ok, so going to close this before we get even more off topic 16:04:54 #topic open floor 16:04:57 tons of stuff in /etc is ini-like format, so it looks like "core management" to me, but understood about the resoruces 16:05:03 actually, no its over time 16:05:05 #endmeeting