20:01:14 #startmeeting Windows Working Group 20:01:14 Meeting started Tue Jul 11 20:01:14 2017 UTC. The chair is nitzmahone. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:14 Useful Commands: #action #agreed #halp #info #idea #link #topic. 20:01:14 The meeting name has been set to 'windows_working_group' 20:01:25 hello 20:01:51 #info agenda: https://github.com/ansible/community/issues/153 20:02:35 hello 20:02:49 #chair jhawkesworth dag jborean93 20:02:49 Current chairs: dag jborean93 jhawkesworth nitzmahone 20:03:14 hello everyone 20:03:21 hey, welcome 20:03:24 hey 20:04:16 hi!, I'm just sitting here today to watch and learn how you do these meetings 20:04:50 #topic Windows setup facts 20:04:51 https://github.com/ansible/community/issues/153#issuecomment-312738930 20:04:59 o/ 20:05:05 * nitzmahone digs through agenda ticket- probably time to declare bankruptcy and start a new one soon 20:05:16 haha 20:05:26 I would say it is time, just lucky I had it prepared :) 20:05:40 Let me know and I'll do the necessary admin work 20:05:48 basically there are a few PR's and Issues floating around with either adding facts or fixing existing ones 20:06:07 I'm hoping to get a consensus about when we add facts and/or when we push people to use their own fact scripts 20:06:26 Does this include adding support for gather_subset? That's a feature parity thing we need to do too... 20:06:49 I think performance is one good reason 20:07:10 If we have gather_subset support, it becomes *slightly* less an issue since folks can opt out, but at some point the size of the setup module is a concern, too. 20:07:18 I have a feeling that the Windows setup module is slower than on Linux (but it could be winrm playing a bigger part) 20:07:52 windows setup definitely feels slower for me. 20:08:10 Yeah, I think there are a couple of compatibility things that can slow it down, too 20:08:19 there are a lot of things it gathers, I need to look at it but definitely not the quickest 20:08:41 (eg, Get-WmiObject vs Get-CimInstance) 20:08:45 but winrm boat load faster than it used to be, so I'm much less fussed about runtime now. 20:08:54 I'm wary of adding more in the case we do move to gather_subset 20:09:45 Well, the gather_subset allows you to shut parts off 20:09:59 But it might be good to do that all at once 20:10:19 gather_subset is not a free ticket for adding whatever IMO 20:10:23 agreed 20:10:34 it should be 1. generally useful 2. efficient 20:10:53 ok, so sounds like we hold off adding stuff to Windows facts until we start supporting gather_subset? 20:11:20 Related question: do we want to do that for 2.4? It's not on the roadmap, but that's mainly because I keep forgetting about it 20:11:34 sounds like a reasonable position. 20:11:34 unless it is fast/free and very useful to have in general 20:11:38 2.4 engine freeze is August 15 and module freeze is August 28 20:11:39 I'm happy to take a stab at it 20:11:52 WFM 20:11:54 #action jborean93 to implement gather_subset for windows facts 20:12:12 #topic https://github.com/ansible/community/issues/153#issuecomment-312966236 backup support 20:12:12 would gather_subset be something implemented using win equivalent of shared module code? 20:12:14 Then we can look at the individual enhancement requests and see what's worth doing 20:12:21 Do we have a list of facts that we lack (compared to python) or a list of facts that we really should have ? 20:12:38 Nah, just basically a standard arg to setup to allow it to shut off parts (eg, disk/network) 20:12:43 Might be useful to start collecting this too, so we can draw a clearer line ;-) 20:13:37 sounds like comparison work we should document changes for and work to reduce the differences 20:13:39 Hard part there is that python side is very OS/distro/installed tool-dependent. I can't even get buyoff to include it all in RETURN docs, so it's basically "you have to go look at the code or run it on your platform of choice" to see what it might return. 20:14:23 jborean93: unfortunately we do have some inconsistencies too, but also some bad python setup stuff we should overhaul and do correct from the start 20:14:46 dag filled in some holes there for 2.3 20:15:10 that sounds fine, as long as they are close, we can never get 100% due to differences in how the OS works 20:15:20 jborean93: sure 20:15:45 We also need to be careful about admin requirements in setup 20:15:47 I think what we have is quite good though, except for disk-related stuff, I didn't need anything else 20:15:50 i toyed with returning installed windows packages in facts but... they are so inconsistent anyway. 20:16:05 ... and can be slow to collect 20:16:18 The Windows Tower scan job collects that IIRC 20:16:23 jhawkesworth_: seems to be a perfect example to add as a seperate _facts module 20:16:40 makes sense. 20:16:40 I think we should try and encourage the fact_path and document more on how that works 20:16:54 comment about the facts received from setup: I had to keep my own output because never found an documented output of facts 20:17:00 for some reason I do very little reasoning in playbooks with windows facts. 20:17:02 Or just encourage people to do custom fact modules 20:17:17 I disable fact gathering like 95% of the time 20:17:26 * dag is still hoping to get a better fact-system implemented, so people can add their own facts plugins into a directory to be added to the normal setup phase 20:17:27 If I need something, I'll call setup explicitly. 20:17:40 nitzmahone: change gathering: explicit in config 20:17:50 some $CUSTOMER have their own specific needs to their setup/applications 20:17:53 * nitzmahone rarely runs with a config file either though 20:18:12 nitzmahone: but you are special ! ;-) 20:18:13 yea we used things like ansible_domain where I was for environment stuff 20:18:14 nitzmahone: also env var 20:18:55 #action We make a list of possible missing/new facts in the Wiki 20:19:00 +1 20:19:06 + 20:19:14 * jborean93 forgot to turn numlock on 20:19:18 lol 20:19:25 +[end] 20:19:36 cool next topic around the backup parameter 20:20:03 dag you have the floor 20:20:04 I haven't looked to see which *nix modules support that explicitly- IIRC just template and lineinX? 20:20:13 yeah, so this came up as part of the xml module 20:20:43 But I think it's a fine idea... 20:20:43 and we should have basic backup-infrastructure that all the other modules can use (win_copy, win_template, win_lineinfile, win_get_url) 20:21:09 sounds like a great thing to put in shared module code 20:21:14 richardcs proposed to do this 20:21:28 and this will bring us closer to feature parity with python modules 20:21:29 same, I'm just wary of adding more to powershell.ps1 when the modular side is coming in 20:21:30 I'll commit to shipping the module_utils plugin loader 20:21:51 I'll shoot to have it in before this meeting next week 20:21:59 yeah, I think it's more important to have this written and discussed, and as soon ad we have the modular powershell.ps1 we can retrofit that in 20:22:06 #action nitzmahone to work on module_utils plugin loader before next week 20:22:09 that would be awesome. 20:22:11 nitzmahone: even better 20:22:34 dag everything we add in there we need to keep going forward for some time 20:22:42 The assembly/usage side is the hard part, and that's been in since 2.3 20:22:42 trying to avoid that as much as possible 20:22:44 as long as one module has implementation we like can move it 20:22:44 I wasn't expecting any objections, but maybe you have insights for the implementation 20:23:07 So long as we remember to move it before 2.4 ships, I have no problem if it's built on powershell.ps1 and moved 20:23:10 jborean93: we can discuss the PR and amend without merging, and then retrofit into the new stuff 20:23:37 I think if we want the module in we should just have the backup code in that module and then we refactor out 20:23:49 I don't see a good reason to hold things off (especially since this was announced for some time now ;-)) 20:24:09 jborean93: it's like that now, but I'd prefer a common approach from the start myself 20:24:29 and we may want to discuss the format of the backup file, not sure if the python default makes sense in a windows world 20:24:43 (wrt. extension and filename-parts) 20:25:04 maybe Windows has a very good convention and functionality for this that other applications are using 20:25:21 I'd say build the PR on powershell.ps1 and don't merge it 20:25:48 i would suggest creating a file.ps1 for 'file related functions' ... this is something we did wrong by putting everything in basic.py 20:26:06 so 'backup' and 'creates/removes' features would belong there 20:26:10 Then we'll rebase the PR once the PS module_utils loader is in and move the backup stuff to its own file. 20:26:10 bcoca: yes 20:26:21 Yep- what bcoca said 20:26:34 We also are lacking in the file-related departement 20:26:48 also, backup returns 'backup_file' key in results to allow 'playbook' to fetch/copy the file for 'centralized storage' 20:26:58 the python modules allow to set ownership/permissions etc for almost all file-relates modules 20:26:59 +1000 20:27:10 a bit harder with windows .. acl 20:27:11 bcoca: yes ! 20:27:34 Not really, but that's again stuff that will be much easier to do with shared module code 20:27:35 but we do have acl module in unixland ... should make that and selinux/xattr default also in filemanagment (if enabled/existing/) 20:27:43 We have a decent YAML definition of Windows ACLs. 20:28:02 So if we share the code that consumes that, we can plug it in wherever it's needed. 20:29:05 ok so we are happy with how we want to approach the backup parameter for win_xml? 20:29:28 I think so: want me to action that? 20:29:42 I've got one already on your for module_util plugin 20:29:56 yeah, I meant the rest of it 20:30:00 sure 20:30:41 cool, next topic 20:30:50 #topic https://github.com/ansible/community/issues/153#issuecomment-312974050 windows sprint 20:31:01 #action @richardcs to propose shared backup code in win_xml PR via powershell.ps1 - we'll hold off merge until shared module_loader is available then rebase and move to separate file handling module_util file 20:31:40 so, what do people think about the sprint we did ? 20:31:46 How did you guys feel the sprint went? I think I'd like to leave that a "community-driven" activity 20:31:55 I thought the sprint was fine, my only feedback would be to reduce the amount of work we wanted to do and really focus on it 20:32:24 * jborean93 needs to be on a more powerful computer before doing it again 20:32:35 jborean93: the feature ideas were an oversight of my part, but on the upside, most of those have been implemented now too ;-) 20:32:46 good point. I know we had some hitches but it took a while just to go through the PRs. 20:32:48 My general opinion on that is that if there's someone (dag, jon, whoever) that wants to organize and run those regularly, great, and we (Ansible, Inc folks) might or might not participate given time constraints. 20:32:57 https://github.com/ansible/community/wiki/Windows:-sprints 20:33:27 I feel like intention is that it should be community taking load off core team. 20:33:46 but thanks jborean93 for big cleanup of win_regedit 20:33:51 well, my number 1 reason is to get more people involved 20:34:17 and I do think we need core people involved to make it (more) successful 20:34:27 Yeah- I think it can be a great way to get more attention on modules that we won't have b/w to maintain but that are still important to the community 20:34:32 to give it some more weight for newcomers to join 20:35:26 I don't think I could do them if they were weekly but as an adhoc or more sporatic task I'm happy to help 20:35:41 but as nitzmahone say would depend on my current workload 20:35:47 I'm not going to say we *won't* participate, but I'll also say we'll probably get pushback if we're committing to them regularly. 20:36:15 we do need to close more issues if we want to get the queue emptied 20:36:26 (emptied is the optimistic lookout ;-)) 20:36:33 I'm trying to get through those 20:36:44 the PR queue is increasing now faster too ;-) 20:36:46 a few are duplicated/people not responding/waiting on some stuff 20:37:05 We're working on better ways to deal with that on Ansible (paid) team as a whole too, but mostly for things that are tagged support:core 20:37:36 Doubling the size of the Windows team should help ;) 20:37:48 * dag still waiting for the badge-support to be merged, so we can make those numbers more upfront ;-) 20:38:16 for the Spinrts, we do need a better working BlueJeans 20:38:38 it's really important we can speak up while looking at stuff 20:38:49 IRC or BlueJeans chat is a bit distractor 20:38:55 Might do better with like Google Hangouts or something 20:39:12 For the size we're currently talking, it'd probably be easier 20:39:26 possibly, we'll experiment 20:39:43 yep worth a try 20:39:46 We don't have a way to easily hand off a Bluejeans session to someone else on a regular basis without breaking it for ourselves 20:39:47 we also had issues at the start with infrastructure, both me and Jon struggled with that 20:40:07 yeah. dag needs windows vms! 20:40:19 and that's for me a general theme with Windows development, I am always waiting for CI 20:40:32 jhawkesworth_: and Windows media 20:40:43 dag just download the evaluation ISOs 20:40:50 That shouldn't be an issue- have you played with packer-windows? 20:40:57 nitzmahone: no 20:41:01 get $current_employer to purchase msdn 20:41:03 Only way to fly 20:41:24 jhawkesworth_: problem is that I am completely disconnected from $CUSTOMER environment (Z laptops) 20:41:27 2 laptops 20:41:33 jhawkesworth: I have a blank check to buy MSDN whenever I need it, but I haven't needed it in 2 years since I started usign packer-windows 20:42:23 ooh. will investigate 20:43:14 About once a year I'll redo my vagrant boxes just to refresh on updates- Matt Wrock also has a technique for auto-sysprepping after first boot that can be used to refresh the licensing on the trial ISOs so it's fresh 180d right out of the box. 20:43:34 I just hate that he uses Chef to build his instances ;) 20:43:43 maybe we should put this info up on the development guide as well ? 20:43:54 IIRC you can only use sysprep to reset that trial so many times? 20:44:25 You don't even need sysprep for that, but you can get over a year out of a trial instance (and since they're vagrant boxes, I usually redo them every couple months anyway) 20:44:29 Ok, so if people could look at https://github.com/ansible/community/blob/master/SPRINTS.md and amend where needed 20:44:32 I'm a happy camper 20:44:32 (slmgr /rearm) 20:45:12 cool is there anything else we wish to discuss around the sprints? 20:45:24 BTW we should have started the sprint with asking who opened one of the issues, because we didn't do some in the end owned by someone in the room 20:45:47 I agree could be good to have this packer-windows in development guide, I'm just starting and it sound very useful 20:46:25 and maybe do the Ansible integration tests with packer ? 20:46:53 that integration would be real nice 20:47:05 You can run them locally that way (using vagrant/vbox/vmware boxes built via packer) 20:47:09 I already do 20:48:09 Matt Wrock's got some nice (more modern) patterns too at https://github.com/mwrock/packer-templates 20:48:24 But he's using boxstarter/chef for a lot of stuff (since, well, he works for chef) 20:48:34 His 2016 stuff is all chef 20:48:55 ok, we're almost nearing 23PM here, let's continue the meeting 20:49:13 ok 20:49:22 #topic https://github.com/ansible/community/issues/153#issuecomment-313378484 20:49:22 #topic paths and windows 20:49:35 * jborean93 was too slow 20:49:50 heh 20:50:10 So that's one of the first actions out of the Sprint, finally writing that documentation related to Windows paths 20:50:37 I think what dag and jhawkesworth_ is good so far. I'm don't know how much we want to focus on the legacy format though considering all the issues that arise when we use it 20:50:38 indirectly this lead to: https://github.com/ansible/ansible/issues/26623 and https://github.com/ansible/ansible/pull/26490 20:51:03 jborean93: the legacy format is the mostconvenient for the ansible cli 20:51:09 misleading to call it 'legacy' format key=value not going away as needed for ad-hoc 20:51:21 +1 to "apply caution tape liberally to k=v format" 20:51:30 yeah, we now call it the key=value format ;-) 20:51:57 I'll remove the "legacy" there 20:52:24 there's one more complication, which is Jinja2 20:52:26 I'm happy with what is there, probably best to get it in and we can change accordingly as we find out more or need to tweak something 20:52:32 Jinja2 works differntly 20:52:49 I had a go at writing much the same in a more prescriptive style here: https://github.com/ansible/community/wiki/Windows-Paths---prescriptive-style 20:53:03 In Jinja2 you have to escape backslashes, even when using single quotes 20:53:17 jhawkesworth_: +1 20:53:42 so the idea is that we move the general information somewhere in the platform-independent docs 20:53:59 and use the prescriptive style docs in the Windows docs (with a reference to those details) 20:54:01 * nitzmahone is booking travel concurrently with running this meeting, so will be distracted for a few minutes 20:54:21 I doubt we can do anything about the way jinja2 behaves, which would be my preference, just have to add 'when using jinja2 templating' section I guess 20:54:39 yep the best we can do is document it 20:55:09 #action Dag will make a PR out of this if he finds the correct location to put the pieces 20:55:38 wary of time will move on unless there is anything pressing 20:55:46 #topic https://github.com/ansible/community/issues/153#issuecomment-313718853 -LiteralPath 20:56:07 Should we modify (most) modules to use -LiteralPath instead ? 20:56:14 what are the benefits 20:56:23 That globs are not supported 20:56:24 I suggest DetailedGuides -> Windows 20:57:05 If you swap to -LiteralPath you might get unexpected outcomes 20:57:14 I prefer the path type does the check and fails if it doesn't conform 20:57:17 Now we do Test-Path -Path C:\foo\bar, but that allows C:\foo\bar\* 20:57:46 ah, pablodav is joining us through the Wiki link ;-) 20:58:04 Globs aren't supported either way (not sure if dag saw my comments on that PR) 20:58:14 nitzmahone: ah, I misread that 20:58:29 The only wildcards it appears that Test-Path understands with -IsValid are the [a-z] kind 20:58:49 I think it is a lot of work to redo all the modules when we have something that works right now 20:58:50 nitzmahone: I am not talking about -IsValid (not sure if you read my reply to your comment ;-)) 20:59:02 unless there is an issue we are coming across right now 21:00:10 jborean93: the issue is that most of the modules will fail now if you add a glob-path 21:00:56 wouldn't that be handled with Get-AnsibleParam 21:01:05 instead of changing all our modules 21:03:03 * jhawkesworth_ trying to think where glob paths would have utility. 21:03:16 win_robocopy comes to mind 21:03:17 jborean93: possibly, but then globs are not allowed anywhere 21:03:19 win_find perhaps. 21:03:36 win_find already has functionality outside of globs to find files 21:03:39 jborean93: so if we want to go that way, we have to allow for a second type to allow globs 21:05:05 nitzmahone: Did you see my win_get_url PR ? It fixes the issue we had yesterday 21:05:19 * nitzmahone hasn't looked at it yet, but yeah 21:07:13 ok, so any decision, or shall we defer ? 21:07:29 maybe we should get a better impact/scope for the issue 21:07:31 +1 on defer 21:07:36 I'm tempted to leave as is. could add a globpath later if needed 21:08:02 unless already solved in python world. 21:08:02 jhawkesworth_: I wouldn't change Add-AnsibleParam, but rather fix it in the modules to ensure that's taken care of correctly 21:08:32 jhawkesworth_: In python globs are an acceptable path by the argspec 21:08:55 jhawkesworth_: it's up to the module to handle the input (globs are not automatically expanded either) 21:09:12 the problem is that in Windows, Test-Path does expand if you use -Path instead of -LiteralPath 21:09:21 so I think it's a Windows issue we have to fix in the modules 21:09:30 but let's look at the impact/scope 21:09:44 we can add integration tests, and fix those failures 21:09:59 makes sense to me 21:10:12 yep let's look at the impact more 21:10:40 And then we can document the modules that do accept globs 21:10:46 if that's unclear 21:11:14 #action Look at impact/scope on Windows modules of globs in paths (not using -LiteralPath) 21:11:43 next topic ? 21:11:54 we are overtime, wary of keeping you guys back 21:12:23 I prefer to finish it :p 21:12:45 ok next time then. like the idea of diagnostic script could go in examples 21:13:35 I would prefer to see the diagnostic script as part of the new ConfigureRemotingForAnsible.ps1 21:13:46 so we have a single tool to configure/status 21:14:11 with all the necessary options, and something we could distribute like trondhindenes was hinting on 21:15:09 I'm not sure about bundling it together, the purpose I had for this was a read only script that gave supporters and the user warnings/errors about what we know can go wrong 21:16:02 I suppose if you put it in the same script, you can at least rule out the 'have you run ConfigureRemotingForAnsible' question :-) 21:17:41 haha 21:17:47 ok ok ;-) 21:18:03 but people running that script, and having issues would already have the script 21:18:34 true, well I'll write a sample script but if we are going to be changing the ConfigureRemoting... then I won't merge it in until we know more about that 21:18:46 maybe just making ConfigureRemotingForAnsible.ps1 spit out the info you want. 21:18:47 especially if you're templating images in VMware, it's not easy to get something provisioned before you can access it, so for troubleshooting that is a win 21:18:57 perhaps with a -validate option that doesn't change, just reports 21:19:15 no, I don't want to modify ConfigureRemotingForAnsible.ps1 21:19:37 trondhindenes was looking at making a more modular/versatile equivalent 21:19:41 sure the better version distributed via ps gallery 21:19:46 yes 21:20:02 so if we have that diagnosis script now, and later we merge it in, that's fine 21:20:15 I don't want one thing be held off because another is vaporware 21:20:21 agreed 21:20:27 ok 21:20:35 #action jborean93 to work on diagnostic script for WinRM 21:20:42 but I also prefer too look at the end-goal (I am an utopist afterall) 21:21:16 hmm, reading up on what that means, I am using that wrong ;-) 21:21:41 idealist, perhaps? 21:21:57 nah, that's not the same ;-) 21:22:02 final topic 21:22:09 #topic https://github.com/ansible/community/issues/153#issuecomment-314325343 SID utils 21:22:34 basically it balls down to whether we want to support service accounts like SYSTEM without specifying NT AUTHORITY\ in the beginning 21:22:49 I'd like be able to use SYSTEM and NETWORK SERVICE without the NT AUTHORITY guff 21:23:06 feels more user friendly 21:23:20 I feel we have modules that support that already as well 21:23:44 then they could use common code eventually without interface change 21:23:48 And anyone that's managed to name domain accounts with those names deserves what they get 21:24:10 I just tried it now and SYSTEM is a reserved name 21:24:13 not sure on the others 21:25:04 I think most things are pretty good about not letting you choose those as names for domain objects 21:25:22 (so like I said, if you managed to do that, you're on your own ;) ) 21:25:34 either way if they have issues they can still just specify DOMAIN\SYSTEM to get it working 21:25:41 SYSTEM would just favour the local NT AUTHORITY object 21:25:53 but god save them if they come to that 21:26:10 sounds good 21:26:28 +1 21:26:30 looks like we want to be able to specify NT AUTHORITY\* objects without the prefix in SID lookups 21:26:40 +1 21:26:48 Yep- it's a well-known list 21:26:55 I assume you guys know what you're doing ;-) 21:27:06 ok unless someone has anything else pressing I'll draw this meeting to a close 21:27:22 well, lot's of open PR's ;-) 21:27:46 and a lot of new modules too 21:27:59 yep some are waiting on people to make changes, others for review 21:28:00 have working mobile data again as of today, will start tackling again tomorrow 21:28:03 We should do a press on some of that stuff before we get too close to freeze 21:28:15 nitzmahone: +1 21:28:27 some are looking quite good 21:28:31 I hate trying to clear a bunch of that stuff hours before freeze. ;) 21:28:50 and we also have a bunch of integration tests to write :-/ 21:28:54 More eyeballs looking is always good- thankful for all y'all! 21:29:07 Those can definitely happen after freeze though 21:29:09 do we have a view on when the Windows infra is going to be faster ? 21:29:52 I wouldn't expect any changes during 2.4 timeframe. We're looking at internal RH datacenter resources that would allow us to spin up more Windows stuff faster and cheaper, but it's a slog 21:30:06 nitzmahone: tests always improved the quality of the modules, so I'd prefer not to postpone... 21:30:23 but resources... 21:30:40 jhawkesworth_: when do we do the next sprint ? 21:30:47 Sure, I'm just saying I'd prefer to get modules merged before freeze even if they're missing tests (we can be filling in tests after freeze) 21:31:04 as opposed to front-loading test writing and potentially missing out on modules for 2.4 21:31:36 right 21:31:38 how about next week for another sprint? 21:31:41 agreed 21:32:39 jhawkesworth_: maybe next week is going to be hard, need to check 21:32:51 week after we are on vacation for 2 weeks 21:34:42 I'll let you guys organise that, got to head off for some breakfast 21:34:45 we are going to be getting into a popular time for vacation 21:34:50 #endmeeting