00:02:22 #startmeeting azure_working_group 00:02:22 Meeting started Thu Feb 1 00:02:22 2018 UTC. The chair is Kylie_. Information about MeetBot at http://wiki.debian.org/MeetBot. 00:02:22 Useful Commands: #action #agreed #halp #info #idea #link #topic. 00:02:22 The meeting name has been set to 'azure_working_group' 00:02:37 #chair nitzmahone 00:02:37 Current chairs: Kylie_ nitzmahone 00:02:56 Hey 00:03:58 I have three topics today. One is leave time to Yuwei and Matt for API versioning to see any open. The other one is to consult how to do bug triage effectively (seems we cannot add some labels and assign bugs). The last one is to discuss PR and Issue. 00:04:17 Any other topic? Please raise up. Then we could ensure it will be captured in today meeting. 00:04:40 #topic API versioning 00:05:04 @nitzmahone, @Yuwei: where are we now? 00:06:01 I've had a little discussion going with Laurent about fixing some of the exception cases so we could get rid of the special case for containerinstance, but overall I'm reasonably happy with the pattern (after numerous trials with it) 00:06:20 Yuwei had one question I wasn't clear on, but otherwise I think we're OK at the moment. 00:07:06 #chair jborean93 zikalino yuwei 00:07:06 Current chairs: Kylie_ jborean93 nitzmahone yuwei zikalino 00:07:12 good morning / afternoon :-) 00:07:20 o/ 00:07:50 so already started, or could we start with keyvault? 00:08:15 I'm going to make those couple changes and finish review as soon as we're done here 00:08:57 Zim, yes. Now the topic is API versioning. @Yuwei, any comment here? After it, we could discuss Key vault. 00:09:51 I looked matt's pr it's all good 00:10:10 ok, so i am fine with api versioning in general, it would be great improvement for all existing modules. 00:10:10 Ok. Then once Matt finalized it, we are ok to move forward. Thanks. 00:10:26 Nice:) 00:10:45 Then Zim, for Key Vault, what is your question? 00:10:49 @nitzmahone i am just wondering what's your opinion on moving into using pure dict in the future (as we already do with all the new modules) 00:11:07 @yuwei if you could follow up on https://github.com/ansible/ansible/pull/35538/files#r165218905 - just want to make sure I'm understanding what you're asking about 00:11:11 @zikalino in an email requested to discuss facts module RETURN value changes/deprecation 00:13:20 @zikalino I'd say let's see how it goes during 2.5-2.6 on some of those modules, and with the older ones still on the SDK objects. I've been playing with pycharm type-hinting to get basic completion stuff working properly at least on the "latest" model versions/operations- moving to pure-dict would make that impossible... 00:13:59 If SDK stability remains poor through the next couple of versions, pure-dict may be a better solution, but I don't think we should make that call yet. 00:14:51 #topic return value deprecation 00:15:34 Also, one topic for tomorrow I think, but as I am writing this e-mail is worth mentioning beforehand. 00:15:35 So, as we agreed on format of data returned by _facts modules, we have some old modules which don’t follow these new standards. I was just wondering how we should handle them. So here’s my suggestion: 00:15:35 - As we upgrade _facts modules we add new format. 00:15:36 - We add “old_format” option with default set to True and mark it as obsolete 00:15:36 - Documentation would include only new data format 00:15:37 - At some point in the future we remove “old_format” option 00:15:47 ok, copied & pasted my suggestion 00:16:34 Deprecating return values with warnings is something I have a few ideas for in core engine (ie, a way to get Ansible to issue a warning if you're using deprecated return values), but wouldn't ship until 2.7 at the earliest since 2.6 will likely not allow many (if any) new core engine features. 00:16:53 In the past, I've just added the new format to the facts module and set a manual warning in the docs 00:17:05 But meantime, old modules with the "state" value we can just leave that as-is and put new sane keys in the root result dict. 00:17:37 yep, we can add the warning in a future version but returning both options is probably the easiest 00:17:50 @jborean93 yeah, that's about the best we can do without the core wrapper support necessary to issue runtime warnings on usage of deprecated return values 00:18:15 ok, that sounds good, so from now on if i update any facts i will just add a new format + note 00:18:50 Ok. Then move to next topic? 00:19:03 also while discussing _facts i am also thinking of a bit of improving "network" / "application gateway" area 00:19:05 Yeah- I'm not generally a fan of polymorphic return types on Ansible modules- it's hard to document clearly (in the current doc structure anyway), so I'd be -1 for the "change the format" module arg. 00:19:41 There are always exceptions, but I don't think we want to do it as a general rule 00:19:45 well, but old format can be just undocumented, and deprecated so nothing breaks 00:20:35 Yeah- we have a lot more leeway since most of these modules are marked preview... 00:22:00 IIRC the return value documentation was often sparse or incomprehensible on a lot of them anyway, so I'd be OK with "undocumented" (or just document that "state" has been deprecated and get rid of the docs for what's underneath) 00:22:34 Generally for deprecation we do a 4 release cycle before we remove (and that includes docs), but for preview stuff we can be a little more "rude" ;) 00:23:21 ok, i think now it's really clear how to apporach it :-) 00:23:29 so about "application_gateway" 00:23:30 we have a few modules that belong to "application gateway" API and need major improvements, API changes, like lists instead of single item, etc. 00:23:30 for example: 00:23:31 azure_rm_loadbalancer 00:23:31 azure_rm_subnet 00:23:32 so i am going to suggest creating new modules, for example: 00:23:32 azure_rm_appgwloadbalancer 00:23:33 azure_rm_appgwsubnet 00:23:33 and making old ones deprecated. 00:24:49 Hrm, azure_rm_subnet is just for virtual network subnets- that'd be a big change 00:25:38 is this a similar change to what was recently done with azure_rm_networkinterface? 00:26:16 Tossing the whole module to make some feature additions seems a bit severe, unless the resource "shape" itself is just wrong 00:27:06 ok, maybe subnet is not the best choice to change first... 00:28:49 But yeah, agree that current load_balancer module is not very useful as-is with the little that I played with it. Whether we should change it in place or rebuild from scratch is up for discussion. 00:29:12 I'm guessing those wouldn't make it in for 2.5 though 00:29:34 so yes, this one is a good example as we need to make major changes anyway. 00:30:21 no, not 2.5, i just wanted to learn your opinion on that, so this is not very urgent. 00:30:23 I'm hoping to avoid last-minute new modules (as that's how the current load_balancer made it in in the first place- it actually came into 2.4 after freeze) 00:30:32 Zim, I guess I misunderstood something. I think the name should align with Azure Rest API. Right? 00:30:49 https://docs.microsoft.com/en-us/rest/api/load-balancer/ 00:31:06 I thought load-balancer is a sub item of application gateway. 00:31:26 But it seems from above link, it is level 1 category. 00:31:44 IIUC they're different things (L3 vs L4 load balancers) 00:32:10 Application Gateway is a L4 load balancer, "load balancer" is L3 00:32:19 Ok. 00:32:35 So current azure_rm_loadbalancer is L4. 00:32:45 other way around 00:32:55 Then it should be the sub-item of Application Gateway if it is L4. 00:33:00 new appgwloadbalancer would be L4 00:33:14 current load_balancer is L3 00:33:23 completely different APIs/purposes 00:33:36 O~ Ok. clear for me now:) 00:33:43 There shouldn't be any module overlaps on those 00:33:52 (eg, separate modules for each) 00:34:11 Thank you for clarification. 00:34:12 hmm... i believe current load balancer uses api from appgateway, but i might be wrong. 00:34:50 nope, network.load_balancers 00:35:58 "/subscriptions/{}/resourceGroups/{}/providers/Microsoft.Network/loadBalancers" 00:36:30 Anyway, appwg module coverage for 2.6 would be a great thing- make it so ;) 00:36:36 So Zim, we should not change that file. But create a new module for L4 LB of Application gateway. 00:37:08 well, as far as i am concerned from REST API perspective network == application gateway 00:37:28 Then need more time from you Matt and Jordan to review them. We have more for Key Vault and Application Gateway in the pool. 00:38:24 i will dig some more details on that... but let's switch to KeyVault as it's 1st priority i think 00:39:14 Ok some confusion here. 00:39:18 Azure Load Balancer works at the transport layer (Layer 4 in the OSI network reference stack). It provides network-level distribution of traffic across instances of an application running in the same Azure data center. 00:39:18 Application Gateway works at the application layer (Layer 7 in the OSI network reference stack). It acts as a reverse-proxy service, terminating the client connection and forwarding requests to back-end endpoints. 00:39:36 Correction: LB is layer 4; Application LB is layer 7. 00:39:57 Yes, let us offline discuss the diff of LB. 00:39:59 * nitzmahone is rusty on OSI defs off the top of my head, but yeah, sounds right 00:40:06 go ahead for key vault. 00:40:15 AppGW is HTTP, "Load Balancer" is TCP 00:40:40 Yes, Matt: TCP/IP: 4 layers/7 layers definition. 00:41:17 @zikalino, key vault, what do you want o ask here? 00:42:10 nitzmahone so what's the status? 00:42:32 Will do review and idempotence tweak right after this meeting 00:42:53 (slept in today, yesterday was a very long day ;) ) 00:43:06 ok, that would be great :-) 00:43:27 when we merge it i will update tests for keys and secrets modules (existing prs) 00:43:34 BTW, since we have new PR, should we close this one? https://github.com/ansible/ansible/pull/32756 00:43:51 what's idempotence tweak? i think i have added idempotence support there 00:44:27 Kylie_: yep let's close that PR 00:44:52 Thank you Jordan. 00:45:01 @zikalino: access rule order is not important to the module, but it's treated as an ordered list. I'm going to change it to a set and rely on hashability for fast/simple change detection 00:45:15 s/to the module/to the keyvault resource/ 00:45:32 ok, sounds good 00:46:05 Shouldn't be any end-user visible change to the UX, but equivalence/"no change" state will not rely on order anymore. 00:47:32 So there are also a bunch of container-related modules and the Azure DB modules outstanding for 2.5 00:48:21 Yes. https://docs.microsoft.com/en-us/azure/ansible/ansible-matrix 00:48:39 yes, i have still a few prs opened, but it all depends if you guys have time to look at them and merge. these are P2 modules, so i wasn't pushing them. 00:48:57 All in azure_module are ones merged for 2.5 - container instance, container registry, database... 00:49:23 Highest priority for us now are key vault and a bug fix for LB (accept multiple rules) which are needed by openshift. 00:49:29 Then Application gateway. 00:50:01 All are in azure_preview_module are ready to send out pull requests to you:) 00:50:14 i haven't created PR for app gateway yet, i wasn't planning it for 2.5 :-) 00:50:33 Do it. 00:50:51 ok, i will do it today 00:50:58 I definitely have a few things I wanted to wrap up for 2.5 myself as well (biggest was getting a preview of the Azure inventory plugin shipped), so if you guys are cool with us stopping at the keyvault stuff and LB changes, that'd be ideal... 00:51:36 If we need to get everything that has a PR open today reviewed/tested/merged, none of the other stuff is probably going to happen. 00:51:57 Ok. 00:52:00 As that will probably consume several days of my time (depending on how much churn is required) 00:52:04 Then let us focus on Key Vault. 00:52:15 Cool 00:52:16 Current one is a basic one. Key and Secret are still needed. 00:52:21 azure_rm_keyvault - - Yes 00:52:21 azure_rm_keyvault_facts - - Yes 00:52:21 azure_rm_keyvaultkey - - Yes 00:52:21 azure_rm_keyvaultsecret - - Yes 00:52:44 We want all 4 of those in for 2.5 though, right? 00:53:07 Yes, not sure how important of facts but key and secret are needed. 00:53:27 OK, I'll commit to getting those in 00:53:49 Then that means 5 PRs + API versioning. Are we on the same page? (If any breaking issue, hot fix will be needed). 00:53:50 #agreed nitzmahone to review/merge all keyvault-related modules for 2.5 00:53:57 yep 00:54:06 Thank you. 00:54:07 ok, let's focus on keyvault today, i will make sure all the tests are running for keys and secrets after nitzmahone merges main keyvault module. 00:54:44 😃 yeah 00:54:48 @yuwei - are you good to apply the pattern from my API versioning PR across the rest of the modules? 00:55:01 (once I merge my PR and necessary changes to shared code) 00:55:50 sure 00:55:52 (so dynamic models from operations, kwargs on all SDK objects) 00:56:44 OK- I'll plan to merge that within the next day or so (want to make sure Laurent doesn't have any further recommendations or changes, but I think we're good) 00:57:04 Anything else for today then? 00:57:31 We are overrun. Let me just ask two questions for last topic. 00:57:44 #topic how to run bug triage effectively? 00:57:54 1. How could we assign bugs to ourselves? 00:58:11 #1 can't happen- GitHub limitation, unfortunately :( 00:58:26 You have to have commit rights on our repos to have issues assigned IIRC 00:58:41 But you can ask the bot to add/remove label 00:59:02 Opps. Then we have to retrieve message from the comment to understand who is working on it or maintain an internal list. 00:59:10 hmm... can we add custom labels? just wodnering 00:59:11 ask bot assign bug to Yuwei or Zim? 00:59:30 GitHub won't allow things to be assigned to people outside the Ansible org 01:00:01 We might be able to get a few people added to the org as read-only to allow the bot to assign issues 01:00:11 You could have a board outside of the ansible repo and just put the link to each PR/issue for a person 01:00:29 (eg a project in ansible/community) 01:00:51 Could it be automated? 01:01:14 unlikely (at least in the short term) 01:01:29 Our bot maintainer also has a large backlog of other things 01:02:13 :( 01:02:17 If you have a project in ansible/community, I believe you can have full control over that 01:02:34 What kind of automation are you looking for? 01:02:44 link the project of ansible/community to ansible/ansible? 01:03:47 Again, Github restrictions probably make that less than useful. :( 01:04:04 Let me see what our options are for external collaborators 01:04:09 2. When ready_for_review is added in comment, any lable will be added? 01:04:11 It's been awhile since I tried 01:05:47 We should disucss how to make branch maintainer have right to assign bug within that branch. Otherwise the role of maintainer seems only the reviewer. We have long-term plan to respond customers (issues or PRs) timely. Effective bug triage is the baseline. 01:06:45 We overrun today. I will leave time to you guys to review PRs. bug triage topic will be the one to discuss after 2.5 code freeze. Thank you guys. 01:06:47 Long-term, the solution will likely be to have the development of these modules happening completely outside the ansible repo (and maybe not even ship them in the box), but that's 2.7 or beyond probably 01:07:08 2.7 means one year later. Right? 01:07:17 Fall of this yer 01:07:20 *year 01:07:31 ~November probably 01:07:33 Our plan targets this June. 01:07:59 Anyway, not something we're going to solve today, so we can discuss next time :D 01:08:25 Do you mean since 2.7, core will be on ansible branch and others will be in seperate branch? Then for Azure, core + Azure libaries? 01:08:47 That's one way we're looking at 01:08:50 that's how i understand it 01:09:06 em. 01:09:22 Matt, one thing I will need your help is about windows support. 01:09:34 https://www.irccloud.com/pastebin/KOknkv2a/ 01:09:35 how so? 01:09:45 ah 01:09:52 One question about Windows. Do you have insight? 01:10:02 Yeah, it's not as easy on Azure 01:10:02 I can give you a sample playbook that does this 01:10:28 Potentially we should document it either on the azure_rm_virtualmachine or in the Azure docs in general 01:10:41 Great. I will send the question to you Jordan and Matt to follow up with you. 01:11:05 Good idea. 01:11:20 Thank you. Then that's all for today meeting. 01:11:29 #endmeeting azure_working_group