20:00:00 <nitzmahone> #startmeeting Windows Working Group
20:00:00 <zodbot> Meeting started Tue Nov 21 20:00:00 2017 UTC.  The chair is nitzmahone. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:00:00 <zodbot> The meeting name has been set to 'windows_working_group'
20:00:19 <jborean93> hey all
20:00:54 <nitzmahone> Noice, I timed it so it started *exactly* at 20:00:00 :D
20:01:19 <nitzmahone> #chair jborean93
20:01:19 <zodbot> Current chairs: jborean93 nitzmahone
20:01:22 <jborean93> must be fate :)
20:01:22 <jhawkesworth_> hey
20:01:31 <nitzmahone> #chair jhawkesworth_
20:01:31 <zodbot> Current chairs: jborean93 jhawkesworth_ nitzmahone
20:02:18 <nitzmahone> #info agenda @ https://github.com/ansible/community/issues/195
20:02:33 <dag> o/
20:02:51 <nitzmahone> #chair dag
20:02:51 <zodbot> Current chairs: dag jborean93 jhawkesworth_ nitzmahone
20:03:48 <nitzmahone> #topic https://github.com/ansible/community/issues/195#issuecomment-345934601
20:05:37 * nitzmahone shakes fist at `get_md5`
20:05:41 <jborean93> hopefully self-explanatory, just asking for some feedback around new parameters and how we want to handle files that have multiple hard link relationships
20:06:58 <nitzmahone> Well, so the options are a polymorphic value (scalar or list), always a list, or "first found", basically, right?
20:07:38 <nitzmahone> Much as I dislike it for usability purposes, I think "always list" is more usable than "sometimes list"
20:07:43 <jborean93> `hlnk_targets` is a list of paths that is only returned if there are 2 or more
20:08:18 <jborean93> I was more asking whether that was a good name to use/whether we always want to return it even if there is only 1 (normal file)
20:08:38 <nitzmahone> Yeah, I'd def vote for "always return"
20:08:47 <jborean93> Plus the question around do we consider a file that is a hard link to be considered a link, `stat` doesn't
20:09:12 <jborean93> I think it is useful, as while it is a file, it is technically a link to some data on the drive somewhere
20:09:21 <nitzmahone> (ie `islnk=true` in that case?)
20:10:38 <jborean93> yep, I think `isreg=True` is also set in that case for `win_stat`, but `stat` only set's `isreg=True` for hard links
20:10:54 <nitzmahone> I think the names are fine in keeping similarity with `stat` (would vote otherwise if this were a truly greenfield module)
20:12:14 <jhawkesworth_> I think the names are fine too.
20:12:19 <jborean93> What about the behaviour for hard links?
20:12:28 * nitzmahone doesn't have a strong opinion
20:12:51 * jhawkesworth_ re-reads PR
20:12:54 <nitzmahone> I guess which way gives you more useful info? I'd kinda argue maybe stat has it right
20:13:13 <nitzmahone> But not strongly
20:13:25 <jborean93> does it, it says the file is a regular file when in reality it is a file hard linked to some data
20:13:36 * jborean93 does not know the correct terms
20:13:56 <nitzmahone> But that's technically any file
20:14:28 <nitzmahone> (if I'm understanding the distinction you're trying to make)
20:14:49 <nitzmahone> So it would only be a hard-link if there were more than one reference, right?
20:15:05 <jborean93> Yea, that's why I only returns `hlnk_targets` if it is more than 1 and set `islnk=True` and `lnktype=hard` in those cases
20:15:42 <nitzmahone> Yeah, that makes sense, but for any use case I can think of, I'd expect `islnk=False` on an ordinary file.
20:16:09 <nitzmahone> Unless you're doing something I'm not making the connection on
20:16:25 <dag> a hard link is a regular file in Unix
20:16:55 <dag> a hard link is exactly the same as the original, there's no distinction between both, they are identical
20:16:59 <nitzmahone> Right
20:16:59 <jborean93> `islnk=False` is set when there is only 1 target
20:17:02 <dag> there's only a reference count
20:17:09 <jborean93> But set to True is there are multiple
20:17:39 <jborean93> dag that's fair enough but then `stat` doesn't really have a way of telling you if the file is also referenced (hard linked) to another path
20:17:43 <jborean93> this way it does
20:18:08 <nitzmahone> Maybe we should just do `islnk=False` always on hard links and just return the other pointers in `hlnk_targets`
20:18:30 <jborean93> Ok, so also remove `lnk_type=hard` as well?
20:18:44 <dag> jborean93: I wonder how stat does it, on Unix you cannot easily list all files linking to an inode (unless you scan the whole filesystem)
20:18:44 <nitzmahone> Preserve islnk just for symlinks and junctions
20:18:45 <jhawkesworth_> islnk seems to mean is symlink then.
20:18:58 <nitzmahone> (or junction)
20:18:59 * dag checks python
20:19:09 <dag> python stat()
20:19:22 <dag> s_ISLNK is only for symbolic links
20:19:31 <jborean93> in Python yep only for symbolic links
20:19:42 <jborean93> Windows I had for junctions/sym and hard links but I can remove hard links
20:19:58 <dag> Is a hardlink the same on Windows ?
20:20:02 <jhawkesworth_> actually stat module doc is clear islnk is just about symlinks, now I look
20:21:05 <nitzmahone> @dag: yes and no- it's still an ordinary file, but the link count is tracked by the filesystem
20:21:08 <jborean93> I don't think junction points are a valid concept in unix?
20:21:21 <nitzmahone> well, closest thing is a mount point
20:21:58 <nitzmahone> graft a different filesystem onto a node of an existing one
20:22:41 <jborean93> I can remove `lnktype` and add another return value `isjunction` to keep it more in line with the python `stat`
20:23:08 <nitzmahone> That might not be a horrible plan, esp if we're not going to treat hardlinks specially (other than just returning hlnk_targets)
20:23:33 <dag> Windows documentation refers junctions as soft links
20:23:47 <dag> but they only work on directories ?
20:23:52 <nitzmahone> So then islnk would be false for junctions as well (technically breaking change but I'd be surprised if anyone noticed)
20:24:21 <jhawkesworth_> I have never spotted a junction in the wild on a windows box
20:24:23 <jborean93> Yea I don't think backwards compatibility is a big issue on the link stuff
20:24:50 <nitzmahone> https://en.wikipedia.org/wiki/NTFS_reparse_point
20:25:10 <jborean93> ok cool, so I can make that change, we also wanted to return `hlnk_targets` for everything, even if it is only 1 (not a hard link)?
20:25:23 <nitzmahone> @jhawkesworth_: I've used them for some things myself; IIRC that's also what things like "Documents and Settings" are now
20:25:54 <nitzmahone> I'd be fine with hlnk_targets always there even if either empty or includes self...
20:26:01 <nitzmahone> Maybe empty would be better though
20:26:14 <nitzmahone> (do we currently include self?)
20:26:27 <jborean93> Yep
20:26:32 <jborean93> I can change that though
20:26:43 <jborean93> So only show other hard linked files
20:26:48 <nitzmahone> +1
20:26:52 <jhawkesworth_> thanks nitzmahone good to know
20:27:35 <nitzmahone> Yeah, for the use cases I can think of, you'd probably need to remove your own link from the list if you were poking at the others...
20:27:46 <nitzmahone> (so just don't put it in there)
20:28:15 <jborean93> Ok, sounds good
20:28:41 <jborean93> #action jborean93 to remove `lnk_type`, add `is_junction` and fix of `hlnk_targets` according to what was dicussed
20:28:52 <nitzmahone> #topic: https://github.com/ansible/community/issues/195#issuecomment-346039030 (win_get_url timeout bug)
20:28:55 <jhawkesworth_> sounds great
20:29:26 <nitzmahone> Haven't tried to repro myself yet, but this just sounds like a bug where we never tried it with a big remote file that took too long.
20:29:53 <jhawkesworth_> I use win_get_url a lot so surprised I haven't hit this before.  Is this latest devel dag?
20:30:09 * jhawkesworth_ checks I'm not still using a custom version
20:30:26 <nitzmahone> https://msdn.microsoft.com/en-us/library/system.net.webrequest.timeout(v=vs.110).aspx
20:30:53 <nitzmahone> Looks like this just changed in August
20:31:16 <jborean93> Yep I believe 2.4 added the feature
20:31:32 <nitzmahone> Doesn't look like WebRequest offers a granular connect-vs-operation timeout
20:32:03 <jhawkesworth_> still running 2.3 in prod at the moment, likely why I haven't hit it
20:32:21 <nitzmahone> Seems like we might need a backgrounded timer or something to kill it instead of relying on that property
20:32:35 <dag> Yes, it was added in v2.4
20:32:50 <nitzmahone> So something like:
20:33:14 <dag> If we keep it as-is, it works differently between Unix/Windows, but the default 10 seconds will be a problem
20:33:26 <nitzmahone> if timeout is set, set timer before we make the request, and when the timer expires, if we haven't gotten any content yet, nuke the request and bomb the module result
20:33:38 <nitzmahone> (do an async fetch)
20:34:01 <nitzmahone> So a poor-man's connection timeout
20:34:06 <dag> personally I don't see a good use for timing out the whole operation
20:34:12 <nitzmahone> Nor do I
20:34:17 <nitzmahone> (hence the proposed change)
20:34:45 <jborean93> Yea, I think we should revert the timeout stuff for the 2.4.2 release and look at re implementing it at a later time
20:34:55 <dag> could you guys test this first, to see if my assessment is correct ?
20:35:10 * nitzmahone tries it
20:35:34 <dag> I only noticed this on a slow network, because in the lab everything used to finished within 10 secs (40Gbit links)
20:36:42 <jborean93> I think you are correct, I believe I saw this when creating `win_wait_for` but will see what nitzmahone says
20:37:48 <dag> it is weird nobody reported it though
20:37:58 <nitzmahone> Hmm, on devel, with `timeout=1` downloading a 500M file, it appears to be working fine
20:38:20 <nitzmahone> So I just specified url, dest, and timeout
20:38:46 <nitzmahone> lemme try a smaller one
20:40:05 <nitzmahone> Hrm, I'm not seeing the failure
20:40:22 <dag> Specifies how long the request can be pending before it times out. Enter a value in seconds. The default value, 0, specifies an indefinite time-out.
20:40:51 <dag> so this seems to imply a "pending" request, so related to DNS lookups etc.
20:41:14 <dag> so maybe that's what I am seeing in the lab
20:41:29 <dag> I'll open an issue where we can discuss
20:42:12 <nitzmahone> Yeah- the docs are definitely clear as mud on what exactly that property does
20:42:37 <nitzmahone> But doesn't look like it's related to the actual download time, so (whew)
20:42:49 <dag> hehe
20:43:27 <nitzmahone> #action dag to open issue for win_get_url timeouts (not related to download length apparently)
20:43:28 <dag> so next topic ?
20:43:31 <nitzmahone> #topic open floor
20:44:10 <dag> jhawkesworth_ proposed to clean up the Wiki docs now that everything is in devel branch
20:44:24 <jhawkesworth_> that's in progress.
20:44:41 <dag> I have been looking at open ideas/tasks and stroke some that we have finished in the meantime
20:44:45 <jhawkesworth_> been though the windows guides now.  Small pr with tiny typo fixes in it coming
20:46:35 <jhawkesworth_> cool.  in the middle of moving a lot of servers to s2012r2 at the moment.
20:46:44 <dag> where are we with Windows coding style and coverage ?
20:46:49 <nitzmahone> agenda updated w/ today's resolutions
20:46:58 <nitzmahone> Was just talking about that with mattclay today
20:47:16 <nitzmahone> Now that we have the unified container, I'm going to redo the stuff I had to use it instead of requiring the MS container
20:47:37 <dag> unified container ?
20:47:41 <nitzmahone> (just need to add the requisite bits to the unified container Dockerfile and rebuild)
20:47:49 <nitzmahone> The ansible-test container where the sanity tests run
20:48:29 <dag> aha
20:48:34 <nitzmahone> I haven't come up with a sane way to do it locally yet, because PSScriptAnalyzer can't accept a list of files, only a single dir tree, so we filter the stuff on the host and copy only the relevant bits to the container
20:49:08 <nitzmahone> Might do a PR to PSScriptAnalyzer to add support for that, I *think* it'd be pretty easy (famous last words)
20:49:33 <mattclay> It's the container we use with `ansible-test` when passing `--docker default` (soon to actually be the default so you can use just `--docker`).
20:50:15 <nitzmahone> If we could just pass it a list of files/dirs. then running locally would be trivial
20:50:21 <dag> once we have it, I guess we prefer to clean up the existing modules in a big bang ?
20:50:46 <dag> or do we make a legacy whitelist file
20:50:50 <nitzmahone> It supports an ignore list  like other things
20:50:50 <jborean93> I think it's going to be an exclusion list kind of thing, similar to the pep8 stuff
20:50:51 <mattclay> nitzmahone: Ideally we'd be able to support both local (passing file names) and in our `default` docker container (for CI).
20:50:55 <dag> (which is a blacklist in fact)
20:51:06 <nitzmahone> @mattclay yep, that'd be the idea
20:51:45 <mattclay> I'd go for the simple `skip.txt` approach and then clean up the files incrementally until that list is empty.
20:52:26 <dag> ok
20:52:33 <nitzmahone> So yeah- I don't want to make everything passing gate the inclusion of the sanity check; IIRC the way I had it it would also error if something was on the ignore list that didn't need to be
20:52:53 <mattclay> Yeah, that part is good to have to avoid regressions later.
20:53:11 <jhawkesworth_> nice
20:53:22 <dag> I'd probably be working on that, since I find it important to have all our modules be perfect examples for future contributors
20:53:24 <mattclay> nitzmahone: So you went the same route as the existing pep8 check then?
20:53:34 <nitzmahone> Yeah, I based it off pep8
20:53:45 <mattclay> That should work good.
20:53:46 <nitzmahone> can't remember if I got all that stuff working though
20:53:54 <nitzmahone> The basics were definitely working fine
20:54:02 <nitzmahone> Just don't remember where I left the ignore stuff
20:54:16 * mattclay goes back to fixing invalid escape sequences in modules
20:54:44 <nitzmahone> OK, anything else for today?
20:55:00 <jhawkesworth_> previously mentioned doc fixes PR now raised: https://github.com/ansible/ansible/pull/33161
20:55:17 <dag> I'd be good to discuss the different proposed style changes here
20:55:24 <jborean93> thanks jhawkesworth_ will have a look today
20:55:41 <jhawkesworth_> may contain British English :-)
20:56:48 <jborean93> perfect for me, the Queen's English is the only English in my mind
20:57:14 <jhawkesworth_> :-)
20:57:23 <dag> :-)
20:57:40 <nitzmahone> We've got 3 minutes left- do we want to open the bikeshed?
20:57:47 <dag> one more quick question
20:58:04 <nitzmahone> fire away
20:58:06 <dag> is there any progress on the new argspec handling (more in line with python)
20:58:07 <jhawkesworth_> If I recall the plan was to start with a few style rules and see what got flagged up and then iterate?
20:58:20 <dag> that was the other major item on the list iirc
20:58:30 <nitzmahone> It's like 3-4 down on the stack right now :(
20:59:03 <nitzmahone> yeah, I think I was using the ruleset that Jordan proposed in the original issue
20:59:41 <nitzmahone> I still have a few more issues to work out around how to get clean exits from both PS and .NET callers
20:59:55 <nitzmahone> (WRT the declarative argspec stuff)
21:00:14 <dag> how about the threaded implementation ? :-)
21:00:32 <dag> is that going to be v2.5 material ?
21:00:48 <nitzmahone> Heh, I think jimi's actually been doing some work on that- it'll probably be experimental for 2.5, but I think it's planned to be "in"
21:01:10 <dag> nice
21:01:18 <dag> and jinja2 native types
21:01:20 <nitzmahone> He's got it so it's feature-flagged with an envvar now, so you can flip back and forth
21:01:25 <dag> v2.5 will be great :)
21:01:38 <jhawkesworth_> yeah, sounds like great progress
21:01:39 <dag> configuraby great
21:01:40 <nitzmahone> Yeah, I think Tanner hit a snag or two with the native types, but don't remember where it got left
21:02:01 <nitzmahone> It's in upstream Jinja 2.10, but may have a bug
21:02:46 <nitzmahone> I've been doing `platform` prototyping the past couple days; hit some snags there too :(
21:03:23 <dag> I haven't seen an official roadmap for v2.5
21:03:45 <dag> only some bits and pieces from network people
21:03:49 <jhawkesworth_> is it just https://github.com/ansible/ansible/projects/9 ?
21:03:56 <nitzmahone> #info 2.5 roadmap https://github.com/ansible/ansible/projects/9
21:03:57 <nitzmahone> yep
21:04:13 <dag> oh !
21:04:14 <nitzmahone> That's the project board for all the RH-generated stuff
21:04:22 <jborean93> There is a roadmap file now as well
21:04:34 <nitzmahone> But the project board is the best place for up-to-date status
21:04:39 <jborean93> https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/roadmap/ROADMAP_2_5.rst
21:05:07 <dag> I didn't know we're using the projects stuff now for development (the network team was doing it before)
21:05:13 <dag> jborean93: thanks !
21:05:27 <dag> it didn't exist last time I looked
21:05:40 <dag> need to updates some Wiki pages now ;-)
21:05:49 <jborean93> I think thaumos recently merged it but the github board is definitely the source of truth for this release
21:06:01 <nitzmahone> Yeah, it's kinda tough since non-committers can't participate, so that's basically just what we're using for core team project mgmt
21:06:26 <dag> aha, and deadlines have been set
21:06:46 <dag> good to know
21:06:58 <dag> and already pretty close :-(
21:07:06 <nitzmahone> Indeed :(
21:07:39 <nitzmahone> Gonna be a busy 8 weeks (esp with holiday/shutdown eating a big chunk out of that)
21:08:09 <jborean93> Always close, we've gotten a few things out for the Windows side when it comes to the engine, become is a lot more handy than before
21:08:38 <nitzmahone> woot for that
21:09:05 <dag> \o/
21:12:23 <nitzmahone> OK, calling it. Thanks all!
21:12:28 <nitzmahone> #endmeeting