18:09:14 <abadger1999> #startmeeting Ansiblefest: Galaxy
18:09:14 <zodbot> Meeting started Mon Oct  1 18:09:14 2018 UTC.
18:09:14 <zodbot> This meeting is logged and archived in a public location.
18:09:14 <zodbot> The chair is abadger1999. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:09:14 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:09:14 <zodbot> The meeting name has been set to 'ansiblefest:_galaxy'
18:09:47 <mattclay> gregdek: Start Blue Jeans recording.
18:09:51 <abadger1999> #chair alikins gregdek tima
18:09:51 <zodbot> Current chairs: abadger1999 alikins gregdek tima
18:10:02 <abadger1999> #chair mattclay willthames jhawkesworth_
18:10:02 <zodbot> Current chairs: abadger1999 alikins gregdek jhawkesworth_ mattclay tima willthames
18:10:34 <jhawkesworth_> etherpad for this session: https://etherpad.openstack.org/p/ansible-summit-october-2018-galaxy
18:10:36 <abadger1999> #link https://etherpad.openstack.org/p/ansible-summit-october-2018-galaxy
18:18:22 <felixfontein> what's the link to the mentioned statistics?
18:19:00 <abadger1999> jtanner: ^
18:19:08 <abadger1999> #chair felixfontein jtanner
18:19:08 <zodbot> Current chairs: abadger1999 alikins felixfontein gregdek jhawkesworth_ jtanner mattclay tima willthames
18:19:28 <abadger1999> tanner is looking for the link
18:19:58 <felixfontein> thanks!
18:20:13 <abadger1999> #action jtanner to publish a link to the gathered statistics
18:22:43 <jtanner> http://tannerjc.net/ansible/galaxy.html
18:25:24 <jtanner> example https://github.com/jctanner/mazer-example-1
18:26:32 <alikins> https://galaxy-qa.ansible.com/testing/ansible_testing_content
18:26:43 <alikins> https://github.com/atestuseraccount/ansible-testing-content
18:27:48 <alikins> https://github.com/ansible/mazer/tree/devel/tests/ansible_galaxy/collection_examples/hello  is a 'hello world' example
18:31:52 <alikins> https://gist.github.com/alikins/d09e34e68af2b7573b60f917b37846b4  example of the 'built' artifact
18:36:40 <abadger1999> Talking with nitzmahone, this import looks like it needs to be changed: https://github.com/atestuseraccount/ansible-testing-content/blob/master/modules/newmodule.py#L13 but I'm not sure if the final location has been determined yet.
18:40:46 <tima> #info we are not abandoning the traditional role format
18:42:03 <defionscode> #info versioning is staying at the repo level; subcomponents of a collection will not be independently versioned
18:42:05 <tima> #info we will version at the collection level. each piece of content will *not* have their own individual version number. (that would be insane)
18:42:14 <defionscode> ha
18:44:46 <abadger1999> #info packaging roles/content via mazer will be able to have versioned content
18:45:18 <bearrito> upgrade++
18:45:30 <defionscode> force push tags...ew
18:46:26 <tima> #info migration guide and tool for going from traditional roles to content collections.
18:46:50 <tima> right @deficonscode
18:48:03 <defionscode> geerlingguy: i imagine you'd have quite a bit of work to do, probably?
18:48:28 <geerlingguy> yeah...
18:48:43 <geerlingguy> maintaining 100% roles in two different repos?
18:48:47 <geerlingguy> 100+ I mean
18:49:27 <defionscode> no sweat, right? (╯°□°）╯︵ ┻━┻
18:49:48 <bcoca> just create a role that manages that
18:50:15 <geerlingguy> I already have enough shell scripts to manage all the deprecation changes I have to deal with across all those roles :P
18:50:50 <tima> #info collection == repo
18:50:53 <bcoca> no collection of collections?
18:51:01 <bcoca> submodule/subfolder/subrepo
18:51:28 <geerlingguy> git submodules--
18:51:40 <geerlingguy> also, I still don't quite get the use case for collections—they sound not-composable
18:52:02 <bcoca> ^ that is why collection of collections question, the workaround we all hate (submodules)
18:52:06 <geerlingguy> e.g. I have 35 different ways I build a webserver that uses Apache... each one uses 5-10 roles, but all have different set of roles
18:52:24 <geerlingguy> I would love to have a collection that says "here are 3 roles specific to this collection, plus grab 7 other roles from Galaxy"
18:52:38 <bcoca> dependencies
18:52:39 <geerlingguy> but not have 35 collections that all have tons of git submodules to manage
18:53:04 <geerlingguy> so collections _can_ have deps?
18:53:52 <defionscode> geerlingguy: I'll add to that, _can_ they have deps...and will deps behavior be staying the same? I mean you could presumably have deps on collection, no?
18:55:09 <bcoca> or you would need to interject a 'fake role' that just lists dependencies for that collection
18:55:25 <geerlingguy> that sounds hacky and annoying :/
18:55:28 <bcoca> but that is a hack, imho
18:55:36 <geerlingguy> also an icky .json file... :O
18:55:38 <defionscode> yeah
18:55:48 <geerlingguy> you got JSON in my YAML!
18:55:59 <bcoca> he, not meant to be human read/edited
18:56:04 <geerlingguy> true true
18:56:16 <bcoca> and for 'web' its easier being json
18:58:24 <tima> #info need to scan/check/warn of variables coming from multiple roles.
18:58:37 <geerlingguy> is it possible we could have a way for one repo (e.g. a role) to drive both mazer and galaxy? I'm a little confused by what happens if e.g. I want to push geerlingguy.apache and support Ansible 2.6 and 2.7-mazer (or whatever version mazer comes in)
18:59:01 <geerlingguy> I like the idea of unchangeable release archives a lot, wish that were how git tags were handled on existing galaxy
18:59:32 <tima> @alikins ^^^ @chouseknecht
19:01:20 <bcoca> that could be done via git archive on import
19:02:09 <bcoca> -10
19:02:22 <bcoca> i expect to erase at least 10 dep solvers
19:03:11 <geerlingguy> also our poor CPUs. ansible-galaxy is so lightweight. I imagine we'll be like npm or composer, using gigabytes of ram building the graph
19:09:00 <geerlingguy> Maybe 3.0 would be a clean time to switch things to mazer-driven?
19:09:15 <abadger1999> geerlingguy: I think we want it sooner.
19:09:21 <geerlingguy> Fine to intro sooner
19:09:36 <abadger1999> Or put in the other way, That would be too soon for 3.0 ;-)
19:09:39 <geerlingguy> but I mean like switch to "this is the only really supported way—others are deprecated"
19:10:00 <defionscode> as in deprecate 'old galaxy' style
19:10:05 <geerlingguy> exactly ^^
19:10:07 <abadger1999> Also, I'm not sure that this deprecates anyhting that is actually in core ansible.
19:10:27 <geerlingguy> not in core ansible, but in 'the way things work' in terms of roles, dependency management
19:10:30 <abadger1999> it deprecates how to build content for galaxy.. but I don't think anything goes away in ansible itself.
19:10:31 <bcoca> the plan is to deprecate current ansible-galaxy
19:11:23 <abadger1999> <nod> that's probably not a 3.0 worthy change.
19:12:01 <abadger1999> (But I would say actual removal should probably be one of those long deprecation cycles I talked about earlier)
19:13:15 <geerlingguy> I'm just thinking that since galaxy and ansible are so closely linked, trying to radically change the way deps are handled without a major ansible version change (even if ansible core itself doesn't require major changes) feels... weird
19:13:52 <bcoca> current role dependencies are really both 'install'  and 'runtime' things, collections would be separating these
19:14:08 <bcoca> you'll have 'install' deps .. runtime will depend on runtime refs, instead of mixing both
19:14:30 <bcoca> though we might want the tool to generate the former from scanning the latter
19:15:29 <geerlingguy> Is mazer going to be 'ansible-galaxy' in the future? Or separate completely?
19:15:46 <defionscode> i understand it as its successor...
19:15:48 <geerlingguy> like will you have to install it after a `pip install ansible` (eventually)
19:16:21 <abadger1999> Sounds like undecided but both alikins and nitzmahone want it to be. I'm not sure though, it does feel like a separate thing.
19:16:32 <abadger1999> Although if pip install ansible is the target....
19:16:42 <abadger1999> Perhaps ansible's requirements specifying mazer?
19:16:48 <geerlingguy> I feel like even if separate, it should be a requirement
19:17:06 <abadger1999> (this is just my own thought.... nitzmahone and alikins are much closer to this)
19:17:16 <defionscode> yeah as a req would make sense
19:17:31 <willthames> not necessarily - ansible-galaxy could just say "please install mazer" if it's not installed
19:18:03 <bcoca> i suspect, short term its diff project, once it gains traction it will be merged into core and eventually replace ansible-galaxy
19:18:08 <abadger1999> req feels right to me as installing ansible could then get the tool that users would need but it could develop on its own faster or slower schedule.
19:18:08 <geerlingguy> that introduces a little friction (imo) but wouldn't be the worst thing
19:18:29 <geerlingguy> If it were up to me, I vote for adding it as a requirement :)
19:18:34 <abadger1999> :-)
19:18:41 <defionscode> abadger1999: but would the req be pinned? seems like you'd keep it as latest?
19:19:14 <geerlingguy> also someone needs to do a blog post talking about the name 'Mazer'
19:19:21 <geerlingguy> another chapter in the history of Ansible lore :P
19:19:51 <abadger1999> That's a good question.  I'd kinda want latest but as long as mazer is developing faster than ansible, it probably needs some sort of maximum mazer version.
19:20:07 <abadger1999> (as mazer may need support from within ansible for some htings)
19:20:21 <defionscode> in ender's game I suppose 'The ansible' did predate Mazer since he led teh first super battle against the aliens
19:20:41 <abadger1999> #chair retr0h
19:20:41 <zodbot> Current chairs: abadger1999 alikins felixfontein gregdek jhawkesworth_ jtanner mattclay retr0h tima willthames
19:20:48 <abadger1999> #topic Molecule!
19:20:50 <defionscode> abadger1999: at the same time im thinking for folks that may want/need to upgrade mazer independent of Ansible
19:20:56 <abadger1999> Yeah.
19:21:03 <abadger1999> That was my thought at first
19:22:50 <abadger1999> We could abuse mazer's versioning I suppose.
19:23:34 <bcoca> eventually i expect it to merge back into core, it makes little sense to include ability to install pip/rpm/npm but not ansible content in basic ansible
19:23:51 <bcoca> but as a project that is rapidly evolving, its fine as an outside thing for now
19:23:56 <abadger1999> mazer version X works with ansible-2.8+ so ansible 2.8's requirement on mazer is mazer<2.9
19:24:17 <bcoca> abadger1999: but mazer requries ansible ...
19:25:15 * abadger1999 wonders if pip can do circular deps (and then wonders if mazer will be able to do circular deps ;-)
19:25:24 <abadger1999> Oh the fun things I remember from yum code...
19:25:36 <bcoca> hehe, this is why i hate creating new package managers
19:25:36 <defionscode> oh man...
19:26:08 <bcoca> a) version handling b) dep solving c) dep solving d) conflict resolution e) dep solving
19:26:39 <defionscode> throw in caching for good measure
19:26:55 <bcoca> version pinning/masking
19:27:16 <bcoca> the only easy part is 'package verification' using checksums
19:27:38 <bcoca> but there is also 'source verification' which now requires signed keys/certificates ...
19:27:51 <bcoca> pub key management
19:28:01 <bcoca> stop me when you've seen enough 'fun'
19:28:11 * defionscode grabs popcorn
19:29:32 <abadger1999> Welp, pip used to handle circular deps at least: https://github.com/pypa/pip/blob/1.5.6/pip/req.py#L1089
19:29:57 <alikins> geerlingguy: (catching up scrollback) At the moment, mazer can install traditional style roles to some degree. For example, `mazer install geerlingguy.apache` will install that role to ~/.ansible/content/geerlingguy/roles/apache. Though eventually, ~/.ansible/content roles will have diff behaviour (plugin paths etc) so still a problem
19:30:53 <bcoca> handle or create them?
19:31:10 <bmbouter> pulp uses a lot of libdebsolve to do all this work
19:31:28 <bmbouter> turns out dependency resolution graphs are complicated enough for their own project!
19:31:38 <geerlingguy> alikins: will everything (including just one single role in a repo) need to be a 'collection'?
19:31:46 <geerlingguy> That wasn't clear from earlier convo
19:31:53 <bcoca> he, only people that believe dependency resolution is easy have never tried to write a dep solver
19:31:57 <defionscode> abadger1999: then there is this 'problem' that npm handles okay-ish https://npm.github.io/how-npm-works-docs/npm3/how-npm3-works.html
19:32:14 <defionscode> so deps that have the same dep but at diff vers
19:32:23 <geerlingguy> dependency resolution is insane, and I will not touch it with a 10 ft pole
19:32:27 <bcoca> geerlingguy: you can have 'current roles' become collections easily by just adding the manifest
19:32:47 <bcoca> geerlingguy: after 300 feet my pole was not usable, but i still feel its too short
19:32:54 <geerlingguy> lol
19:33:23 <abadger1999> defionscode: yeah, that is something that setuptools didn't handle and I think pip inherited that broken behaviour.
19:33:34 <defionscode> yep
19:34:11 <abadger1999> But in this case, it's just a question of whether ansible can put mazer in requirements.txt and it looks like it can.
19:36:03 <alikins> geerlingguy: " will everything ... need to be a 'collection"  Yes and no. Collections will require it, but traditional style roles unlikely to go away anytime soon (ever?)
19:36:05 <defionscode> but should it? i dunno
19:36:11 <abadger1999> sivel: It's a step up from button presser!
19:37:13 <gundalow> DING DING. If we finish up at 14:45, if people want to grab their badges and a drink we can start the next sessions at 3pm
19:37:26 <bearrito> this may have been asked already. will mazer allow for private / on-premise hosting?
19:37:44 <abadger1999> galaxy is open source.
19:37:51 <abadger1999> alikins: ^ Is that all that's needed?
19:38:17 <geerlingguy> `Lines should be no longer than 100 chars` :(
19:38:37 <geerlingguy> is there a way to tell molecule to use this rule set?
19:38:39 <gundalow> geerlingguy: I'm not sure about that one
19:38:39 <abadger1999> #link https://github.com/ansible/galaxy
19:38:42 <bcoca> bearrito: currently ansible-galaxy can install directly from a repo, are you asking about that?
19:39:09 <defionscode> geerlingguy: i believe you can spec lint rules in molecule.yml
19:39:24 <defionscode> we're starting to integrate it into ansible-lockdown content
19:39:33 <abadger1999> alikins: (I was referencing bearrito's question)
19:40:12 <geerlingguy> chouseknecht-- terrible code quality
19:40:14 <bearrito> yep. been on projects where there's a want to be able to install from public github and their private github / bitbucket using ansible-galaxy. also been asked for purely self hosted, but that's less
19:40:32 <defionscode> bearrito: it sounds like the path to a 'private mirror' of galaxy stuff is in the works via pulp
19:40:32 <bmbouter> it's not galaxy, but this is what pulp_ansible is able to do
19:40:35 <bearrito> as for both, it's just pointing the requirements.yml to the private repos i believe. is that right?
19:40:44 <bearrito> ah ok
19:40:59 <bmbouter> https://github.com/pulp/pulp_ansible/
19:41:03 <alikins> bearrito: Yes, albeit with a standalong (open source) galaxy server. And/or soon, a pulp setup. I'd like to support a static-metadata-over-http style hosting solution as well but doesn't exist atm.
19:42:14 <abadger1999> #info on-prem/self-hosted galaxy server will work with mazer.  pulp and static-metadata-over-http may work/be implemented as well
19:42:29 <bmbouter> we added a commit to ansible-client CLI to be able to install from an on-premise pulp_ansible. it's released in the GA ansible-client now actually
19:42:33 <abadger1999> #topic galaxy web ui update
19:42:42 <bmbouter> it's at the bottom of the readme at the link above
19:43:25 <defionscode> bmbouter: i feel like that should be promoted more, bummed i just found out about pulp_ansible
19:43:55 <bmbouter> we're just getting started. pulp has been around for a long time, but this is on pulp3 which is new
19:44:39 <defionscode> ah ok, going to bring it up a couple of clients, it seems quite slick
19:46:44 <abadger1999> #endmeeting