16:00:29 <akasurde> #startmeeting Ansible VMware Working Group Meeting
16:00:29 <zodbot> Meeting started Mon Oct 22 16:00:29 2018 UTC.
16:00:29 <zodbot> This meeting is logged and archived in a public location.
16:00:29 <zodbot> The chair is akasurde. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:29 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:29 <zodbot> The meeting name has been set to 'ansible_vmware_working_group_meeting'
16:01:43 <moshloop_> hi
16:01:58 <akasurde> #chair moshloop_
16:01:58 <zodbot> Current chairs: akasurde moshloop_
16:05:00 <moshloop_> Is there an agenda for today?
16:06:05 <akasurde> moshloop_, no agenda as such
16:06:15 <akasurde> do you want to discuss any PR or issues ?
16:06:15 <ckotte> aksurde, can we create a page on github (a gist?) with vmware module standardization?
16:06:23 <akasurde> #chair ckotte
16:06:23 <zodbot> Current chairs: akasurde ckotte moshloop_
16:06:43 <akasurde> ckotte, what kind of standardization
16:06:44 <ckotte> like standard options (esxi_hostname and cluster_name)
16:06:59 <ckotte> documentation standards
16:07:30 <akasurde> OK
16:07:31 <ckotte> '{{ vcenter }}' or vcenter_server (discussion with Dag)
16:08:12 <dag> ckotte: probably Wiki is best while it is being discussed
16:08:13 <ckotte> I also asked him about check mode verbose message. he said everybody can do it like he wants to
16:08:43 <akasurde> ckotte, yes you can create a page here - https://github.com/ansible/community/wiki/VMware
16:08:43 <dag> ckotte: I didn't say that :) I said it's not standardized, so effectively people do it differently :)
16:08:48 <ckotte> ok wiki. at least a central page to track (and we could point every PR owner too automatically later)
16:08:48 <akasurde> chair #dag
16:09:17 <ckotte> ok. I will prepare something
16:09:38 <dag> ckotte: once it's set in stone, we can move it into a VMware development page in the official documentation
16:09:49 <ckotte> ok
16:09:49 <dag> ckotte: Thanks !
16:10:53 <akasurde> #action ckotte To create a wiki page to record standarization on VMware module development
16:14:37 <ckotte> aksurde, any plans to work on other modules or existing PRs?
16:15:18 <akasurde> I have create 2 pages related to VMware documentation - https://github.com/ansible/ansible/pull/47117 and https://github.com/ansible/ansible/pull/43992
16:15:30 <akasurde> yes, been busy with VMware CI work
16:17:00 <ckotte> what is this about?
16:17:19 <akasurde> PR or CI work ?
16:17:34 <ckotte> the PRs
16:17:55 <ckotte> where I have to install my CA certificates on my VM to use validate_certs: true?
16:18:08 <ckotte> on my server, Docker image, etc.
16:18:08 <akasurde> PRs are related to general concepts related to VMware
16:18:49 <ckotte> ok. any plans to implement this "connection cache module"?
16:19:09 <akasurde> typically on your delegate_to host
16:19:10 <ckotte> playbook runs take really long. Not sure if the connection cache can speed this up
16:19:59 <akasurde> I have no experience with connection cache module
16:20:23 <akasurde> so can not comment on this
16:20:42 <ckotte> dag mentioned it in one of the PRs
16:21:31 <dag> connection cache ?
16:21:37 <dag> I don't know anything about that
16:21:56 <dag> but reusing sessions by using a vcenter connection plugin, that may help
16:22:14 <ckotte> I meant that
16:22:19 <akasurde> maybe he is trying to say about connection reusage
16:22:25 <ckotte> I forgot what you said exactly..
16:22:40 <dag> and it makes playbooks a lot more concise as you wouldn't have to repeat connection credentials/properties (like validate_certs)
16:22:55 <ckotte> a new session is made for every tasks
16:23:02 <akasurde> dag, do you have any existing module
16:23:07 <ckotte> I have lots of tasks per ESXi host
16:23:07 <akasurde> example
16:23:30 <dag> no, but it will resemble what ckotte already did for vmware_tools
16:23:48 <ckotte> I didn't do that. it was someone else
16:23:53 <dag> a copy could even copy files to vcenter datastore, and fetch could get from a datastore
16:23:58 <dag> ah sorry
16:24:04 <akasurde> dericcrago, did it
16:24:06 <dag> yes, I am mixing things up, dericcrago
16:24:28 <dag> it shouldn't be too hard to do
16:24:32 <akasurde> vmware_tools works for vms not esxi
16:25:02 <dag> correct
16:25:08 <akasurde> dag, yes that will be something similar to that vmware_tools module kind ...
16:25:42 <dericcrago> hi
16:26:06 <akasurde> dag, ckotte did you check this - https://github.com/Akasurde/ansible-vmware-http ?
16:26:11 <dag> hey dericcrago !
16:26:15 <akasurde> #chair dericcrago
16:26:15 <zodbot> Current chairs: akasurde ckotte dericcrago moshloop_
16:26:19 <akasurde> hi dericcrago
16:27:17 <dag> akasurde: a REST api is easier, but I don't see how we can drop VMware 6.0 and earlier just yet
16:27:45 <dag> each module would need to implement both :-/
16:28:10 <akasurde> REST api is easy and fast I guess
16:28:15 <dag> or we need to work on a new set of modules targeting the REST api (with a different namespace)
16:28:49 <akasurde> I think we should continue with Pyvmomi module only and REST api can be Role or Playbook
16:28:55 <dag> not sure if it is faster, if we don't reuse sessions
16:29:23 <dag> ah, you're reusing cookies
16:29:37 <dag> akasurde: I don't think roles/playbooks is something we want to promote though
16:29:38 <ckotte> do you guys recomment to use REST API directly instead of Pyvmoni?
16:29:59 <dag> ckotte: I don't think we are offering a better product if we tell people to use the uri-module
16:30:08 <dag> we are basically telling people to program in playbooks
16:30:08 <ckotte> no. definetly not
16:30:20 <dag> which goes against Ansible's philosophy
16:30:36 <ckotte> the REST API handling should be done in a module
16:30:47 <ckotte> I did that already for VCSA API
16:31:06 <ckotte> however, why should this be better than existing PyVmoni?
16:31:25 <akasurde> ckotte, nothing is inferior or superior right now
16:31:36 <akasurde> REST API is limited now as compared to PyVmomi
16:31:47 <akasurde> I would say very limited
16:31:51 <ckotte> isn't it basically the same?
16:32:00 <dag> so there are 2 options:
16:32:00 <dag> 1. we have a vcenter and a httpapi connection plugin, and every module is told to understand both connection methods
16:32:00 <dag> 2. we have different modules for pyvmomi and rest API's and people can decide which ones to use (over time)
16:32:37 <pdellaert> right now, the REST API's expose a tiny fraction of the SOAP API
16:32:40 <ckotte> we have to rewrite every module
16:32:45 <akasurde> dag, like I said REST API is very limited
16:32:47 <dag> option 1. is more complex for us, but easier for end-user
16:32:47 <dag> option 2. is easier for us, but more complex to enduser
16:32:52 <akasurde> so can not throw people at it
16:32:56 <dag> akasurde: for now, yes
16:32:56 <ckotte> PyVmoni uses SOAP API?
16:33:00 <pdellaert> yes
16:33:05 <akasurde> #chari pdellaert
16:33:07 <ckotte> ok
16:33:14 <akasurde> ckotte, XMLRPC + SOAP
16:33:19 <akasurde> #chair pdellaert
16:33:19 <zodbot> Current chairs: akasurde ckotte dericcrago moshloop_ pdellaert
16:33:21 <ckotte> then it doesn't make sense at the moment
16:33:21 <akasurde> pdellaert, hi
16:33:22 <dag> akasurde: I am not thinking about now, I am thinking about the future
16:33:25 <pdellaert> so until the REST API is feature parity with SOAP, there is absolutely no use to move to REST
16:33:48 <dag> akasurde: unless you are saying VMware is leaving the REST api already, which makes it not future-proof anyway
16:33:51 <pdellaert> btw, it's python/pyvmomi that is slow
16:33:57 <moshloop_> I think maybe for the most common use cases - e.g. listing, it might make sense
16:33:59 <akasurde> dag yes for future
16:34:30 <dag> pdellaert: we're opening a new session for every task, I don't think that helps our case
16:34:30 <pdellaert> govmomi improves performance 10x compared to pyvmomi
16:34:37 <pdellaert> dag: true
16:34:52 <dag> pdellaert: and that's where a vcenter connection plugin is going to be helpful
16:34:57 <pdellaert> it definitely doesn't help :)
16:35:03 <pdellaert> (the session per task)
16:35:19 <dag> dericcrago: since you have the experience, what does it take to make a basic vcenter connection plugin ?
16:35:34 <pdellaert> but pyvmomi is just slow in general... It's better than pysphere ;)
16:35:46 <akasurde> dericcrago, dag is talking about connecting to ESXi or Vcenter and reusing connection
16:36:00 <akasurde> pdellaert, pysphere is hacking stuff :)
16:36:10 <pdellaert> i know :)
16:36:15 <dag> pysphere used to be the only thing we had
16:36:24 <pdellaert> i know that as well
16:36:30 <pdellaert> the 'good' old days
16:36:31 <dag> because VMware only cared about perl :p
16:36:36 <dag> and then powershell
16:36:52 <dag> maybe we ought to do powershell modules on Linux instead :p
16:36:59 <akasurde> Swiss army knife == perl
16:37:07 <pdellaert> ugh, dag
16:37:10 <pdellaert> just...
16:37:12 <pdellaert> don't :p
16:37:12 <ckotte> it's MS technology
16:37:16 <ckotte> it's evil
16:37:30 <dag> ckotte: not for long, Github will be MS technology
16:37:42 <dag> be prepare to selfdestruct in 3, 2, 1... :)
16:37:42 <pdellaert> the whole 'MS is evil' thing, i'm over that
16:38:02 <ckotte> I'm migrating our VMware PowerShell scripts to VMware ...
16:38:03 <bcoca> MS can still be evil, but its much less relevant today
16:38:05 <dericcrago> I'm not sure I understand the question as I don't understand how ansible uses a connection longer than a task
16:38:06 <pdellaert> just not a fan of powershell on linux ;)
16:38:08 <akasurde> people let us get back to VMware
16:38:16 <akasurde> #chair bcoca
16:38:16 <zodbot> Current chairs: akasurde bcoca ckotte dericcrago moshloop_ pdellaert
16:38:21 <pdellaert> yes, thanks, akasurde
16:38:30 <pdellaert> back to the meeting :)
16:38:42 <pdellaert> REST API: don't waste time on it for now, forr sure
16:38:54 <ckotte> I meant: I'm migrating our VMware PowerShell scripts to Ansible ...
16:38:55 <pdellaert> Connection cache: something to look at
16:39:18 <akasurde> bcoca, can you shade some light on = how can we use connection cache plugin to speedup VMware tasks ?
16:39:29 <ckotte> this should support multiple vCenter servers in the same run as well
16:39:31 <pdellaert> but ckotte brings up a more important aspect (ws going through the backlog): standardization... We've been doing some work, but we're not there yet
16:39:48 <dag> dericcrago: you wrote a vmware_tools connection plugin, on the connection level you can reuse e.g. a cookie, or a pipe
16:39:55 <pdellaert> so having a page that describes this, with a set of rules, would be helpful
16:40:18 <pdellaert> akasurde: btw, did you create a vmware project to track everything, like Windows has?
16:40:40 <dag> akasurde: don't call it a connection cache plugin, it's just a connection plugin (caching to me is something different)
16:41:40 <dag> the aim for connection plugins is to handle the connection throughtout a play or playbook run
16:42:05 <akasurde> pdellaert, I think right now Wiki page is place we can track because no one from community can move tickets
16:42:24 <akasurde> bcoca, connection plugin
16:45:33 <dericcrago> dag - I don't think I'm answering your question, but the connection stays open until it is either closed or times out.
16:46:16 <dericcrago> the details of which are mostly handled by pyvmomi
16:46:17 <dag> dericcrago: it doesn't have to :) maybe we can improve this for the vmware_tools connection plugin
16:46:17 <ckotte> the question is. can a module use an existing connection instead of opening a new one every time
16:46:33 <dag> ckotte: a module cannot, but the connection plugin can
16:46:36 <ckotte> not sure if this speed up the run much tbh
16:46:56 <dag> ckotte: you're not authentication every time, so yes, it should
16:47:08 <ckotte> so the module requires the connection plugin to run before?
16:47:27 <ckotte> no hostname, password, etc. anymore required?
16:47:28 <dag> ckotte: the connection plugin runs before the tasks
16:47:57 <dag> what's more, the connection plugin can also gather facts and make them available to the playbook, which gives some additional possibilities
16:48:09 <dag> ckotte: correct
16:48:38 <dericcrago> something calls `.close()` on the connection plugin
16:49:32 <dag> there's an ansible-connection process handling the session's persistence
16:51:11 <dag> the connection plugin also handles a session that expires, or HTTP/network related issues
16:51:15 <dag> so it offers som resilience
16:51:18 <dag> some
16:52:18 <dag> we still may want one session per target, rather than multiplexing different targets over the same session
16:52:45 <dag> I guess we want to test this, especially on a larger number of parallel targets
16:58:07 <ckotte> I need to connect to different vCenter servers when I execute the playbook for all ESXi hosts in one site
16:58:18 <ckotte> maybe you need to take such scenarios into account as well
16:58:40 <ckotte> I use --limit <site>-esxi
16:59:08 <ckotte> I also have clusters defined in my inventory, but I never use them with --limit
16:59:22 <dericcrago> if you had the vars setup at a group level, I don't think it should matter
17:00:14 <ckotte> I have the vcenter vars at the cluster vars
17:00:26 <ckotte> but I need different connections in one run
17:01:17 <ckotte> if the plugin handles this automatically
17:01:29 <dericcrago> we can take that offline if you want ckotte
17:01:57 <ckotte> not important at the moment. We can talk about something else as well
17:02:33 <ckotte> I just don't expect much help from VMware :)
17:02:48 <moshloop_> could we discuss a couple open PR's i  have?
17:03:10 <dericcrago> I know we're running over, but what next steps do I need to do to move the vmware_tools connection plugin forward? https://github.com/ansible/ansible/pull/47072
17:08:40 <dag> ckotte: different vCenters are always different sessions
17:09:28 <akasurde> We are little over time for meeting. Let us continue discussing things on IRC
17:09:31 <akasurde> #endmeeting