16:00:29 #startmeeting Ansible VMware Working Group Meeting 16:00:29 Meeting started Mon Oct 22 16:00:29 2018 UTC. 16:00:29 This meeting is logged and archived in a public location. 16:00:29 The chair is akasurde. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:29 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:29 The meeting name has been set to 'ansible_vmware_working_group_meeting' 16:01:43 hi 16:01:58 #chair moshloop_ 16:01:58 Current chairs: akasurde moshloop_ 16:05:00 Is there an agenda for today? 16:06:05 moshloop_, no agenda as such 16:06:15 do you want to discuss any PR or issues ? 16:06:15 aksurde, can we create a page on github (a gist?) with vmware module standardization? 16:06:23 #chair ckotte 16:06:23 Current chairs: akasurde ckotte moshloop_ 16:06:43 ckotte, what kind of standardization 16:06:44 like standard options (esxi_hostname and cluster_name) 16:06:59 documentation standards 16:07:30 OK 16:07:31 '{{ vcenter }}' or vcenter_server (discussion with Dag) 16:08:12 ckotte: probably Wiki is best while it is being discussed 16:08:13 I also asked him about check mode verbose message. he said everybody can do it like he wants to 16:08:43 ckotte, yes you can create a page here - https://github.com/ansible/community/wiki/VMware 16:08:43 ckotte: I didn't say that :) I said it's not standardized, so effectively people do it differently :) 16:08:48 ok wiki. at least a central page to track (and we could point every PR owner too automatically later) 16:08:48 chair #dag 16:09:17 ok. I will prepare something 16:09:38 ckotte: once it's set in stone, we can move it into a VMware development page in the official documentation 16:09:49 ok 16:09:49 ckotte: Thanks ! 16:10:53 #action ckotte To create a wiki page to record standarization on VMware module development 16:14:37 aksurde, any plans to work on other modules or existing PRs? 16:15:18 I have create 2 pages related to VMware documentation - https://github.com/ansible/ansible/pull/47117 and https://github.com/ansible/ansible/pull/43992 16:15:30 yes, been busy with VMware CI work 16:17:00 what is this about? 16:17:19 PR or CI work ? 16:17:34 the PRs 16:17:55 where I have to install my CA certificates on my VM to use validate_certs: true? 16:18:08 on my server, Docker image, etc. 16:18:08 PRs are related to general concepts related to VMware 16:18:49 ok. any plans to implement this "connection cache module"? 16:19:09 typically on your delegate_to host 16:19:10 playbook runs take really long. Not sure if the connection cache can speed this up 16:19:59 I have no experience with connection cache module 16:20:23 so can not comment on this 16:20:42 dag mentioned it in one of the PRs 16:21:31 connection cache ? 16:21:37 I don't know anything about that 16:21:56 but reusing sessions by using a vcenter connection plugin, that may help 16:22:14 I meant that 16:22:19 maybe he is trying to say about connection reusage 16:22:25 I forgot what you said exactly.. 16:22:40 and it makes playbooks a lot more concise as you wouldn't have to repeat connection credentials/properties (like validate_certs) 16:22:55 a new session is made for every tasks 16:23:02 dag, do you have any existing module 16:23:07 I have lots of tasks per ESXi host 16:23:07 example 16:23:30 no, but it will resemble what ckotte already did for vmware_tools 16:23:48 I didn't do that. it was someone else 16:23:53 a copy could even copy files to vcenter datastore, and fetch could get from a datastore 16:23:58 ah sorry 16:24:04 dericcrago, did it 16:24:06 yes, I am mixing things up, dericcrago 16:24:28 it shouldn't be too hard to do 16:24:32 vmware_tools works for vms not esxi 16:25:02 correct 16:25:08 dag, yes that will be something similar to that vmware_tools module kind ... 16:25:42 hi 16:26:06 dag, ckotte did you check this - https://github.com/Akasurde/ansible-vmware-http ? 16:26:11 hey dericcrago ! 16:26:15 #chair dericcrago 16:26:15 Current chairs: akasurde ckotte dericcrago moshloop_ 16:26:19 hi dericcrago 16:27:17 akasurde: a REST api is easier, but I don't see how we can drop VMware 6.0 and earlier just yet 16:27:45 each module would need to implement both :-/ 16:28:10 REST api is easy and fast I guess 16:28:15 or we need to work on a new set of modules targeting the REST api (with a different namespace) 16:28:49 I think we should continue with Pyvmomi module only and REST api can be Role or Playbook 16:28:55 not sure if it is faster, if we don't reuse sessions 16:29:23 ah, you're reusing cookies 16:29:37 akasurde: I don't think roles/playbooks is something we want to promote though 16:29:38 do you guys recomment to use REST API directly instead of Pyvmoni? 16:29:59 ckotte: I don't think we are offering a better product if we tell people to use the uri-module 16:30:08 we are basically telling people to program in playbooks 16:30:08 no. definetly not 16:30:20 which goes against Ansible's philosophy 16:30:36 the REST API handling should be done in a module 16:30:47 I did that already for VCSA API 16:31:06 however, why should this be better than existing PyVmoni? 16:31:25 ckotte, nothing is inferior or superior right now 16:31:36 REST API is limited now as compared to PyVmomi 16:31:47 I would say very limited 16:31:51 isn't it basically the same? 16:32:00 so there are 2 options: 16:32:00 1. we have a vcenter and a httpapi connection plugin, and every module is told to understand both connection methods 16:32:00 2. we have different modules for pyvmomi and rest API's and people can decide which ones to use (over time) 16:32:37 right now, the REST API's expose a tiny fraction of the SOAP API 16:32:40 we have to rewrite every module 16:32:45 dag, like I said REST API is very limited 16:32:47 option 1. is more complex for us, but easier for end-user 16:32:47 option 2. is easier for us, but more complex to enduser 16:32:52 so can not throw people at it 16:32:56 akasurde: for now, yes 16:32:56 PyVmoni uses SOAP API? 16:33:00 yes 16:33:05 #chari pdellaert 16:33:07 ok 16:33:14 ckotte, XMLRPC + SOAP 16:33:19 #chair pdellaert 16:33:19 Current chairs: akasurde ckotte dericcrago moshloop_ pdellaert 16:33:21 then it doesn't make sense at the moment 16:33:21 pdellaert, hi 16:33:22 akasurde: I am not thinking about now, I am thinking about the future 16:33:25 so until the REST API is feature parity with SOAP, there is absolutely no use to move to REST 16:33:48 akasurde: unless you are saying VMware is leaving the REST api already, which makes it not future-proof anyway 16:33:51 btw, it's python/pyvmomi that is slow 16:33:57 I think maybe for the most common use cases - e.g. listing, it might make sense 16:33:59 dag yes for future 16:34:30 pdellaert: we're opening a new session for every task, I don't think that helps our case 16:34:30 govmomi improves performance 10x compared to pyvmomi 16:34:37 dag: true 16:34:52 pdellaert: and that's where a vcenter connection plugin is going to be helpful 16:34:57 it definitely doesn't help :) 16:35:03 (the session per task) 16:35:19 dericcrago: since you have the experience, what does it take to make a basic vcenter connection plugin ? 16:35:34 but pyvmomi is just slow in general... It's better than pysphere ;) 16:35:46 dericcrago, dag is talking about connecting to ESXi or Vcenter and reusing connection 16:36:00 pdellaert, pysphere is hacking stuff :) 16:36:10 i know :) 16:36:15 pysphere used to be the only thing we had 16:36:24 i know that as well 16:36:30 the 'good' old days 16:36:31 because VMware only cared about perl :p 16:36:36 and then powershell 16:36:52 maybe we ought to do powershell modules on Linux instead :p 16:36:59 Swiss army knife == perl 16:37:07 ugh, dag 16:37:10 just... 16:37:12 don't :p 16:37:12 it's MS technology 16:37:16 it's evil 16:37:30 ckotte: not for long, Github will be MS technology 16:37:42 be prepare to selfdestruct in 3, 2, 1... :) 16:37:42 the whole 'MS is evil' thing, i'm over that 16:38:02 I'm migrating our VMware PowerShell scripts to VMware ... 16:38:03 MS can still be evil, but its much less relevant today 16:38:05 I'm not sure I understand the question as I don't understand how ansible uses a connection longer than a task 16:38:06 just not a fan of powershell on linux ;) 16:38:08 people let us get back to VMware 16:38:16 #chair bcoca 16:38:16 Current chairs: akasurde bcoca ckotte dericcrago moshloop_ pdellaert 16:38:21 yes, thanks, akasurde 16:38:30 back to the meeting :) 16:38:42 REST API: don't waste time on it for now, forr sure 16:38:54 I meant: I'm migrating our VMware PowerShell scripts to Ansible ... 16:38:55 Connection cache: something to look at 16:39:18 bcoca, can you shade some light on = how can we use connection cache plugin to speedup VMware tasks ? 16:39:29 this should support multiple vCenter servers in the same run as well 16:39:31 but ckotte brings up a more important aspect (ws going through the backlog): standardization... We've been doing some work, but we're not there yet 16:39:48 dericcrago: you wrote a vmware_tools connection plugin, on the connection level you can reuse e.g. a cookie, or a pipe 16:39:55 so having a page that describes this, with a set of rules, would be helpful 16:40:18 akasurde: btw, did you create a vmware project to track everything, like Windows has? 16:40:40 akasurde: don't call it a connection cache plugin, it's just a connection plugin (caching to me is something different) 16:41:40 the aim for connection plugins is to handle the connection throughtout a play or playbook run 16:42:05 pdellaert, I think right now Wiki page is place we can track because no one from community can move tickets 16:42:24 bcoca, connection plugin 16:45:33 dag - I don't think I'm answering your question, but the connection stays open until it is either closed or times out. 16:46:16 the details of which are mostly handled by pyvmomi 16:46:17 dericcrago: it doesn't have to :) maybe we can improve this for the vmware_tools connection plugin 16:46:17 the question is. can a module use an existing connection instead of opening a new one every time 16:46:33 ckotte: a module cannot, but the connection plugin can 16:46:36 not sure if this speed up the run much tbh 16:46:56 ckotte: you're not authentication every time, so yes, it should 16:47:08 so the module requires the connection plugin to run before? 16:47:27 no hostname, password, etc. anymore required? 16:47:28 ckotte: the connection plugin runs before the tasks 16:47:57 what's more, the connection plugin can also gather facts and make them available to the playbook, which gives some additional possibilities 16:48:09 ckotte: correct 16:48:38 something calls `.close()` on the connection plugin 16:49:32 there's an ansible-connection process handling the session's persistence 16:51:11 the connection plugin also handles a session that expires, or HTTP/network related issues 16:51:15 so it offers som resilience 16:51:18 some 16:52:18 we still may want one session per target, rather than multiplexing different targets over the same session 16:52:45 I guess we want to test this, especially on a larger number of parallel targets 16:58:07 I need to connect to different vCenter servers when I execute the playbook for all ESXi hosts in one site 16:58:18 maybe you need to take such scenarios into account as well 16:58:40 I use --limit -esxi 16:59:08 I also have clusters defined in my inventory, but I never use them with --limit 16:59:22 if you had the vars setup at a group level, I don't think it should matter 17:00:14 I have the vcenter vars at the cluster vars 17:00:26 but I need different connections in one run 17:01:17 if the plugin handles this automatically 17:01:29 we can take that offline if you want ckotte 17:01:57 not important at the moment. We can talk about something else as well 17:02:33 I just don't expect much help from VMware :) 17:02:48 could we discuss a couple open PR's i have? 17:03:10 I know we're running over, but what next steps do I need to do to move the vmware_tools connection plugin forward? https://github.com/ansible/ansible/pull/47072 17:08:40 ckotte: different vCenters are always different sessions 17:09:28 We are little over time for meeting. Let us continue discussing things on IRC 17:09:31 #endmeeting