09:18:03 <nishanth> #startmeeting check-in_20170118
09:18:03 <zodbot> Meeting started Wed Jan 18 09:18:03 2017 UTC.  The chair is nishanth. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:18:03 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
09:18:03 <zodbot> The meeting name has been set to 'check-in_20170118'
09:18:17 <nishanth> #topic ankush's updates
09:18:51 <ankush> Working on https://github.com/Tendrl/dashboard/issues/102
09:19:34 <ankush> Wrote features for file-share in Behavior Driven development
09:20:42 <ankush> regarding the creation of repo I need to to mrugesh. https://github.com/Tendrl/dashboard/issues/100
09:21:33 <nishanth> ok
09:21:37 <nishanth> #chair brainfunked
09:21:37 <zodbot> Current chairs: brainfunked nishanth
09:21:44 <brainfunked> thanks nishanth
09:21:53 <brainfunked> sorry, Internet connection died
09:22:27 <brainfunked> #info I want to drive the check-ins this week to be focused on the build we're delivering, so the format will be a bit different
09:22:49 <brainfunked> #topic framework updates
09:23:19 <brainfunked> k4n0, please link to the pull requests that make commons changes to be tracked by all the components
09:24:40 <k4n0> https://github.com/Tendrl/commons/issues?q=is%3Apr+author%3Ar0h4n+is%3Aclosed
09:24:53 <k4n0> https://github.com/Tendrl/node-agent/issues?q=is%3Apr+author%3Ar0h4n+is%3Aclosed
09:28:15 <brainfunked> Shu6h3ndu, nishanth, given that you guys are in training, will you be able to work today?
09:28:43 <Shu6h3ndu> brainfunked, we are in training tomorrow and day after
09:28:59 <brainfunked> oh ok, I thought it was today and tomorrow
09:29:00 <brainfunked> in which case
09:30:10 <brainfunked> #action anmol, gowtham, ndarshan, Shu6h3ndu and nishanth to track k4n0's commits to commons (https://github.com/Tendrl/commons/issues?q=is%3Apr+author%3Ar0h4n+is%3Aclosed) and node agent (https://github.com/Tendrl/node-agent/issues?q=is%3Apr+author%3Ar0h4n+is%3Aclosed) repositories and ensure that the component repositories are kept in sync with them
09:30:30 <gowtham> brainfunked, ok
09:31:17 <brainfunked> #info k4n0's changes are targeted towards enabling and fulfilling all the framework changes required from https://github.com/Tendrl/specifications/pull/100 and https://github.com/Tendrl/specifications/issues/54#issuecomment-269947580
09:31:26 <Shu6h3ndu> brainfunked, I feel there are no more changes to commons at least for this release so that we can concentrate making changes for other modules
09:32:03 <brainfunked> Shu6h3ndu, k4n0 is working on changes currently, so the tracking needs to be done continuously throughout this week
09:32:17 <brainfunked> Shu6h3ndu, if you've caught up with the changes, then no problem
09:32:34 <brainfunked> #topic import cluster workflow
09:32:45 <anmol> brainfunked, I have verfied my prs with latest node-agent and commons PRs can they be reviewed and closed
09:32:51 <brainfunked> gowtham, what are you working on presently?
09:33:08 <kaverma> brainfunked, import clusyer workflow is done with sample response format
09:33:25 <gowtham> brainfunked, i have push new testcase PR for platform identification
09:33:29 <brainfunked> anmol, yes, as I've said, you need to keep tracking the PRs and keep ensuring that the changes are caught up with
09:33:42 <brainfunked> kaverma, hold on, I'll get to UI later
09:33:51 <kaverma> brainfunked, ok
09:34:06 <gowtham> brainfunked, i am updated logging implementation spec, as per anmol's comment
09:34:26 <brainfunked> gowtham, we need updates from all the flows we're shipping on priority, rather than the entire framework
09:34:28 <gowtham> brainfunked, please give your suggestion for logging and networking spec
09:34:37 <gowtham> brainfunked, oh ok
09:34:44 <brainfunked> gowtham, network spec is not a priority this week, I'll be making some updates on the logging spec
09:34:56 <gowtham> brainfunked, ok
09:35:08 <brainfunked> gowtham, the current priority is to finalise the message metadata format, so that the UI can be coded to parse it
09:35:10 <gowtham> brainfunked, just tell me what i have to do, i will do it
09:35:31 <gowtham> brainfunked, ok
09:35:38 <brainfunked> #action gowtham needs to put in update messages for each of the flows that we're shipping
09:36:31 <brainfunked> #action brainfunked will be making some comments against the logging specification, which will provide additiona implementation details to gowtham for job updates and correspondingly provide information to kaverma, neha and ankush for job status update display on the UI
09:37:26 <brainfunked> gowtham, what I'll be commenting will essentially be stating that we need to specify the origin of the message and the priority
09:37:42 <gowtham> brainfunked, sure
09:37:47 <brainfunked> the original needs to clearly signify that the update is a result from the flow
09:38:46 <brainfunked> so for instance, if the node agent picks up a job and decides that it needs to spawn multiple child jobs, the update needs to indicate that the origin is the flow, with the flow name (namespaced) and also the list of children created and a textual description of exactly why those children were created
09:39:06 <brainfunked> gowtham, keep in mind that the messages need to be human readable without tendrl specific implementation details
09:39:21 <gowtham> brainfunked, ok
09:39:23 <brainfunked> gowtham, this needs to be carefully done flow-by-flow and  atom-by-atom
09:39:31 <gowtham> brainfunked, ok
09:39:54 <brainfunked> gowtham, in addition, you'll need to setup the job update structure so that the job has a directory, inside which the updates are sequentially created objects
09:40:15 <gowtham> brainfunked, ok
09:40:45 <brainfunked> #action gowtham to enable the job updates directory structure with sequential updates: https://github.com/Tendrl/specifications/issues/55#issuecomment-269940764
09:41:02 <brainfunked> ndarshan, what's your status?
09:41:59 <ndarshan> I have quiet a few patches pending for review and merge. Please refer : http://paste.fedoraproject.org/529332/18690148/
09:43:21 <ndarshan> Once these are In will start working on fixing node_agent with the latest common changes
09:43:21 <brainfunked> ndarshan, great, I'll review these
09:43:54 <brainfunked> ndarshan, from your side, currently, have you sent in all the changes applicable to the tag based job routing feature?
09:44:34 <ndarshan> yes, when all these patches are merged. This feature will be completed in backend
09:44:52 <ndarshan> anoop will need some API changes
09:45:18 <brainfunked> ack
09:45:36 <brainfunked> ndarshan, in that case, as you've already mentioned, the priority is to ensure the stability of the node agent codebase
09:46:54 <brainfunked> #accept ndarshan has sent pull requests for the tag based job routing: https://github.com/Tendrl/node-agent/pull/144, https://github.com/Tendrl/commons/pull/117, https://github.com/Tendrl/ceph-integration/pull/82. https://github.com/Tendrl/gluster-integration/pull/114, https://github.com/Tendrl/node-agent/pull/146
09:47:18 <ndarshan> sure, brainfunked. But we can start once the framework changes are freezed. Else any change that we do will again need to be fixed.
09:47:48 <brainfunked> ndarshan, you'll need to go commit by commit
09:47:55 <Shu6h3ndu> brainfunked, agree with ndarshan here
09:48:07 <brainfunked> this is a continuous process
09:48:26 <brainfunked> and yes, it may involve some re-work, but at any point, any change to commons must be reflected across all the other components
09:48:51 <brainfunked> changes to commons will never be completely done, it'll always be building on the existing code
09:48:59 <brainfunked> hence, these changes must be tracked commit by commit
09:49:16 <brainfunked> plus, it's easier to do that rather than to read through 2k lines of code in one go and then make the changes
09:49:48 <brainfunked> which also could lead to more re-work, because it's possible to keep making changes to the components that are in conflict with some commits to commons
09:49:56 <Shu6h3ndu> brainfunked, by the time we finish changes something changes drastically so lot of rework :( also newer issues injected so we most of the cases spend time debugging the issues
09:50:15 <anmol> brainfunked, can there be atleast an interim deadline by which the changes will be freezed momentarily and then communicated well in the team at which point the rest of the team can implement those changes? so that the work can be prioritized accordingly
09:50:25 <brainfunked> k4n0, your comments?
09:50:54 <brainfunked> Shu6h3ndu, anmol and ndarshan, while a valid concern, you need to be extremely specific, based on the actual commits
09:51:03 <k4n0> brainfunked, whats the topic?
09:51:08 <k4n0> I must have disconnected
09:51:37 <brainfunked> k4n0, ndarshan, Shu6h3ndu and anmol feel that it is expensive to keep track of commons changes commit by commit and are asking for some kind of a freeze
09:51:44 <brainfunked> I disagree with that
09:52:12 <brainfunked> the freeze based approach doesn't work in a distributed development environment
09:52:37 <brainfunked> unless the commits drastically break stuff from the previous commits, I don't think this concern is valid
09:52:53 <k4n0> brainfunked, I agree, It is bit expensive, but if you are relying so heavily on the framework to get the code right, it is like tracking any other third party lib
09:53:37 <brainfunked> except that if you don't track, you have non-working code in master
09:54:13 <brainfunked> in addition, everyone must be reading each and every commit made to commons
09:54:19 <k4n0> brainfunked, Lets get on with it, without commons, the individual projects get too much code which is completely different from each other, commons is the only to ensure code re-use
09:54:31 <brainfunked> agreed
09:54:31 <k4n0> brainfunked, ack
09:55:27 <k4n0> If there are other ways to achieve this, glad to discuss
09:55:39 <brainfunked> Shu6h3ndu, anmol, ndarshan, comments?
09:56:22 <anmol> brainfunked, I am in favour of checkpointing way of doing this so that we can easily ensure everyone is on track
09:56:57 <brainfunked> anmol, every commit is a check point
09:57:14 <brainfunked> it's k4n0's responsibility to ensure that the commits don't break the previous ones
09:57:35 <brainfunked> it's completely possible that it happens once in a while and that's OK too, just needs to be communicated
09:58:21 <anmol> brainfunked, I feel it would be nice to consider a checkpoint achieved once all modules concerned by that checkpoint are in sync
09:58:48 <brainfunked> anmol, this discussion is for how to get in sync
09:58:55 <Shu6h3ndu> brainfunked, here the case is like newer commits do break from the previous and that's why I raised this point
09:59:00 <brainfunked> anmol, checkpoint achieved after the sync is valid
09:59:39 <brainfunked> k4n0, ^
10:00:48 <k4n0> Folks, I am sure no one wants to actually break backward compat, lets ensure we track commons and node-agent per commit
10:02:31 <mkudlej> brainfunked: hi, I would like to go eat, so my status is that I'm working on https://github.com/Tendrl/usmqe-tests/issues/9 and some Tendrl unrelated tasks
10:02:57 <brainfunked> #info Discussion about having freezes on commons for other components to catch up with. However, k4n0 and brainfunked are stressing that everyone needs to read each and every commit to commons and keep their components in sync for each.
10:03:41 <brainfunked> #action pending reviews to the tag based routing feature, ndarshan to focus on keeping node agent in sync with commons changes and to work on the stability of the node agent codebase, including test coverage, if applicable.
10:04:17 * mbukatov has nothing to report today
10:05:18 <ltrilety> brainfunked: I will work on https://github.com/Tendrl/usmqe-tests/issues/12,  for now I am blocked as packages are missing
10:05:20 <brainfunked> #action k4n0, brainfunked to review https://github.com/Tendrl/node-agent/pull/145, invoking integrations as a service from the node agent
10:05:35 <brainfunked> mkudlej, mbukatov, ltrilety, ack, I'll log your updates in due course
10:05:40 <fbalak> #action fbalak No update related to tendrl today.
10:06:18 <brainfunked> #action k4n0, brainfunked to review https://github.com/Tendrl/node-agent/pull/144 https://github.com/Tendrl/commons/pull/117 https://github.com/Tendrl/ceph-integration/pull/82 https://github.com/Tendrl/gluster-integration/pull/114 https://github.com/Tendrl/node-agent/pull/146
10:06:25 <brainfunked> Shu6h3ndu, your updates please?
10:07:15 <Shu6h3ndu> sure
10:07:22 <Shu6h3ndu> https://github.com/Tendrl/commons/pull/109 (fixes)
10:07:23 <Shu6h3ndu> https://github.com/Tendrl/node-agent/pull/149 (startup fixes)
10:07:23 <Shu6h3ndu> https://github.com/Tendrl/gluster-integration/pull/111 (volume atoms fixes)
10:07:38 <Shu6h3ndu> these are few patches which are mostly fixes due to commons changes
10:07:44 <Shu6h3ndu> need to be reviewed
10:08:42 <Shu6h3ndu> first twwo are must for stabilizing the node agent
10:08:48 <Shu6h3ndu> two*
10:08:50 <brainfunked> Shu6h3ndu, was the specification added for platform and sds detection?
10:09:26 <Shu6h3ndu> sds detection there is a PR https://github.com/Tendrl/specifications/pull/117
10:09:38 <Shu6h3ndu> platform nishanth need to add one I feel
10:10:02 <Shu6h3ndu> i have to add job based reload details to https://github.com/Tendrl/specifications/pull/117 though
10:10:27 <brainfunked> checking
10:11:46 <brainfunked> ack, I'll review
10:12:15 <Shu6h3ndu> brainfunked, old PR pending are https://github.com/Tendrl/specifications/pull/98 (pool utilization monitoring) and https://github.com/Tendrl/specifications/pull/93 (pool utilization)
10:12:23 <brainfunked> nishanth, has anyone picked up the detected cluster object related changes from the import flow comment?
10:12:25 <Shu6h3ndu> brainfunked, needs review and closure
10:12:29 <brainfunked> ack
10:12:52 <nishanth> brainfunked, I am working on that chnage
10:13:21 <brainfunked> k4n0, nishanth is working on the detected cluster object related change, from the bluejeans discussion earlier in the week
10:14:44 <nishanth> brainfunked, https://github.com/Tendrl/specifications/issues/54#issuecomment-269947580
10:15:07 <brainfunked> Shu6h3ndu, nishanth will you guys have any time to work tomorrow and day after?
10:15:28 <Shu6h3ndu> brainfunked, doesnt looks like as trng is for full day
10:15:35 <brainfunked> ack
10:15:52 <brainfunked> so nishanth, you'll need to send in a PR for that change today
10:16:03 <brainfunked> Shu6h3ndu, what would you be completing today?
10:16:09 <nishanth> brainfunked, Probably I will be able to spend few hours after training
10:16:36 <nishanth> brainfunked, mostly yes
10:16:51 <Shu6h3ndu> brainfunked, I would try to make sure gluster integration is up to date for volume operations
10:17:21 <ndarshan> brainfunked, k4n0, Please review and merge https://github.com/Tendrl/commons/pull/109 and https://github.com/Tendrl/node-agent/pull/149 from Shu6h3ndu on priority . So that I can continue work on stabilizing node_agent on top of this.
10:17:40 <brainfunked> ndarshan, ack
10:18:07 <anmol> brainfunked, I haven't had my turn of update :(
10:18:28 <brainfunked> #action brainfunked, k4n0 to review and merge https://github.com/Tendrl/commons/pull/109 and https://github.com/Tendrl/node-agent/pull/149 from Shu6h3ndu on priority, ndarshan is blocked on it
10:18:48 <ndarshan> brainfunked, thanks
10:18:50 <brainfunked> #action Shu6h3ndu to focus on ensuring gluster integration stability
10:19:23 <brainfunked> #acction brainfunked to review the updates to the sds detection specification https://github.com/Tendrl/specifications/pull/117
10:19:36 <brainfunked> #action brainfunked to review the updates to the sds detection specification https://github.com/Tendrl/specifications/pull/117
10:19:46 <brainfunked> anmol, we're still on import cluster :)
10:19:52 <anmol> ok
10:20:11 <brainfunked> nishanth, apart from the cluster detection stuff, anything else you'll be able to look at today? it's fine if you can't
10:21:47 <nishanth> brainfunked, I think I will be occupied with that, if I have bandwidth, I will work on testing the import cluster flow with latest codebase
10:22:00 <brainfunked> awesome, thanks
10:22:36 <brainfunked> #action nishanth working on removing the detected cluster details from the context and moving them to a separate object, based on https://github.com/Tendrl/specifications/issues/54#issuecomment-269947580
10:22:51 <brainfunked> #topic alerting and performance monitoring
10:23:00 <brainfunked> anmol, alright, your turn
10:23:11 <anmol> [a] Updated https://github.com/Tendrl/node-agent/pull/67 -- node_agent: Atom to check service status + Alerts socket
10:23:11 <anmol> -- Verified monitoring portions with rebased code(with latest node-agent master with Fix commons base flow changes)
10:23:11 <anmol> with fixes in node-agent alerting socket related pieces(corresponding to commons changes)
10:23:11 <anmol> [b] Updated https://github.com/Tendrl/performance-monitoring/pull/28 -- Fix pep8 issues, api response type and make api threaded
10:23:12 <anmol> -- Verified with latest commons(with Update tendrl-node-agent.spec, remove logging and config directives latest commons master merged patch)
10:23:17 <anmol> [c] Updated https://github.com/Tendrl/alerting/pull/26 -- Fix pep8 issues, api response type and make api server threaded
10:23:20 <anmol> -- Verified with latest commons(with Update tendrl-node-agent.spec, remove logging and config directives latest commons master merged patch)
10:23:23 <anmol> [d] Gave requirements from alerting perspective for rpm spec to tim reviewing hos patch - https://github.com/Tendrl/alerting/pull/28/files.
10:23:26 <anmol> [e] Discussing details about performance-monitoring and node-monitoring with tim from perspective of rpm spec requirements.
10:23:29 <anmol> He has asked for a few details which I'll be providing
10:23:31 <anmol> [f] Working on extract out alerting apis related portions of https://github.com/Tendrl/specifications/blob/master/specs/pluggable_alert_delivery.adoc
10:23:34 <anmol> into a new spec which Anup can pick up for implementation in tendrl-api rather than the currently implemented piece in alerting module.
10:23:51 <anmol> I also need Shubhendu's commons patch #109 but currently I have verified my patch by pulling it locally
10:28:01 <brainfunked> anmol, btw, the alerting APIs should be part of the core API, so you can either implement it in Ruby in the core API, or provide details for Anup to implement them
10:28:46 <brainfunked> anmol, and to confirm, are all the data points needed for list views and possibly, import cluster enabled from your side?
10:28:47 <anmol> brainfunked, alerting api is already merged. Now I am writing a spec capturing the api details as you had told me in the meeting
10:28:54 <anmol> so that anup can work on it
10:29:06 <brainfunked> anmol, ok, I thought you were working on extending the merged API
10:29:09 <anmol> brainfunked, ^ and this was also part of a spec that was merged
10:29:15 <brainfunked> ok
10:29:26 <anmol> brainfunked, alerting api as part of alerting module
10:29:44 <anmol> brainfunked, anyway I'll borrow details ffrom that spec and put it new spec
10:29:56 <anmol> so that anup can use it
10:30:08 <brainfunked> great
10:30:36 <brainfunked> anmol, if all the features are enabled, please concentrate on code quality and stability this week
10:30:50 <anmol> brainfunked, yes sure
10:31:19 <anmol> I need to add delivery guratees still
10:31:25 <anmol> and also snmp handler
10:31:58 <brainfunked> anmol, concentrate on smtp for now, snmp need not be prioritised right now
10:32:04 <anmol> ok
10:32:59 <brainfunked> #action anmol working on specification to implement the alerting APIs in the core APIs
10:33:27 <brainfunked> #action anmol to ensure code quality and stability for this week's build, work on delivery guarantees for alerts
10:34:07 <brainfunked> anmol, btw, I may make a change to the alerting in etcd. essentially, have an events directory instead of alerts, and you'll need to filter out only the alerts to be delivered
10:34:28 <brainfunked> anmol, I'll make updates to the logging spec, as I'd mentioned to gowtham earlier, we can discuss the changes there
10:34:41 <anmol> brainfunked, ok
10:34:50 <brainfunked> anmol, if approved, it shouldn't be too big a change for you to implement this week
10:35:02 <anmol> brainfunked, ok
10:35:12 <brainfunked> #topic UI
10:35:20 <brainfunked> neha, ankush and kaverma, your updates please
10:35:45 <kaverma> import cluster workflow UI is done with mocked json. code is reviewed and merged.
10:35:59 <kaverma> incorporating comments on https://github.com/Tendrl/dashboard/pull/99 - related to "rename from tendrl-frontend to dashboard". got merged now.
10:35:59 <kaverma> and spend some time with verifying and testing rpm . and now going to write spec for task progress page.
10:36:36 <brainfunked> ok
10:36:43 <neha> I am working on task progress page for import cluster - https://github.com/Tendrl/dashboard/issues/101.
10:37:02 <brainfunked> #accept import cluster UI code has been merged, needs to be tested against a running API instance
10:37:46 <brainfunked> #action brainfunked will be making updates to the logging specification, which will provide information about the message metadata for task status updates on the UI
10:38:13 <brainfunked> #action neha and kaverma working on task progress page for import cluster https://github.com/Tendrl/dashboard/issues/101
10:38:21 <brainfunked> #info ankush is away
10:38:34 <brainfunked> #topic updates from the testing team
10:39:15 <brainfunked> #undo
10:39:15 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x20bae7d0>
10:39:18 <brainfunked> #undo
10:39:18 <zodbot> Removing item from minutes: INFO by brainfunked at 10:38:21 : ankush is away
10:41:01 <brainfunked> #topic packaging
10:41:05 <brainfunked> t1m1, your status?
10:41:17 <t1m1> https://github.com/Tendrl/gluster-integration/pull/110 → updated the pach which fixes
10:41:18 <t1m1> tendrl-bug-id: Tendrl/gluster-integration#108
10:41:18 <t1m1> tendrl-bug-id: Tendrl/gluster-integration#109
10:41:18 <t1m1> Sent a patch for issue: https://github.com/Tendrl/alerting/issues/27
10:41:18 <t1m1> for review: https://github.com/Tendrl/alerting/pull/28
10:41:18 <t1m1> Working on https://github.com/Tendrl/performance-monitoring/issues/29
10:41:43 <t1m1> facing issue in performance_monitoring that it needs to build 2 rpm
10:41:55 <t1m1> one is tendrl-performnace-monitoring and
10:42:14 <gowtham> brainfunked, ping
10:42:14 <zodbot> gowtham: Ping with data, please: https://fedoraproject.org/wiki/No_naked_pings
10:42:15 <t1m1> another one is performance-monitoring-node
10:42:33 <t1m1> both has Setup.py file
10:42:42 <brainfunked> is anmol aware of the issues?
10:42:50 <t1m1> node-performance-monitoring is not a sub package
10:43:17 <t1m1> today only i have started working on this pkg build, i have informed him
10:43:19 <t1m1> in the morning
10:43:28 <anmol> brainfunked, tim didn't mean its an issue he jsut meant that 1 is not subpackage of the other
10:44:08 <anmol> and he is exploring a way of achieving 2 rpms from single source base where 1 is not sub-package of the other
10:44:17 <t1m1> shall we have two different spec file ?
10:44:31 <brainfunked> two spec files
10:44:44 <brainfunked> it's not possible to have two packages in the same spec, unless they're sub packages
10:45:53 <brainfunked> #action t1m1 is working packaging the alerting and performance monitoring components
10:46:04 <brainfunked> #topic testing team's updates
10:46:05 <t1m1> i am exploring the possibilities (both has similar files for version)
10:46:26 <brainfunked> #info mkudlej is working on https://github.com/Tendrl/usmqe-tests/issues/9
10:47:09 <brainfunked> #info ltrilety is working on https://github.com/Tendrl/usmqe-tests/issues/12 and needs packages to proceed further
10:47:16 <brainfunked> #info mbukatov and fbalak have no updates
10:47:22 <brainfunked> dahorak, do you have an update?
10:48:39 <ankush> brainfunked, I am back
10:49:03 <brainfunked> ankush, could you please repeat your updates provided to nishanth?
10:49:51 <ankush> Working on https://github.com/Tendrl/dashboard/issues/102
10:50:10 <ankush> Wrote features for file-share in Behavior Driven development.
10:51:14 <brainfunked> ok
10:51:19 <brainfunked> #endmeeting