15:00:37 <hagarth> #startmeeting
15:00:37 <zodbot> Meeting started Wed Mar 12 15:00:37 2014 UTC.  The chair is hagarth. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:37 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:42 <sasundar> hagarth, hi :)
15:00:48 <ndevos> hello _o/
15:01:05 <hagarth> #agenda rollcall
15:01:10 * jdarcy is here.
15:01:18 <hagarth> who all do we here apart from the acks that we've got so far?
15:01:22 * kkeithley_ is still here
15:01:36 <jdarcy> purpleidea was here early.
15:01:41 <itisravi> i'm here
15:01:44 <shyam> here
15:01:46 <pk> so am I
15:01:49 * lalatenduM is here
15:01:54 * aravindavk here
15:02:05 * vkoppad here
15:02:14 <hagarth> cool, we seem to have quorum to get started
15:02:21 <thotz> jiffin here
15:02:28 <hagarth> #topic Features for 3.6
15:02:38 * xavih is here
15:02:55 <hagarth> I setup an etherpad for this meeting : #link - http://titanpad.com/glusterfs-3-6-planning
15:03:12 * msvbhat is here
15:03:19 * purpleidea is here, hi
15:03:35 <hagarth> we can probably get started with the list of features here
15:03:49 <hagarth> I think we have got majority of the feature owners on now
15:04:29 <hagarth> The agenda for today is to identify what features are core for 3.6 and what can be nice to have.
15:04:36 <hagarth> so, let us get rolling
15:04:43 <hagarth> 1.  Features/better-ssl
15:05:03 <hagarth> I like this feature proposal.
15:05:07 <jdarcy> Core.  We've already been burned on this.
15:05:28 <hagarth> jdarcy: +1, would love to see this in 3.6
15:05:28 <jclift> Oops, late.
15:05:29 * jclift is here
15:05:47 <hagarth> jclift: thanks and can you help us with the minutes on etherpad?
15:06:13 <lalatenduM> +1 for better ssl, this came in gluster mailing lists couple of time
15:06:27 <hagarth> ok, core for better-ssl!
15:06:27 <jclift> hagarth: I'll assist where I can
15:06:33 <hagarth> jclift: thank you!
15:06:41 <johnmark> doh - here
15:06:47 <hagarth> moving on, 2.  Features/data-classification
15:06:52 <purpleidea> hagarth: jdarcy, etc: fwiw, i actually really like the interface for using the current ssl.
15:07:06 <hagarth> purpleidea: cool
15:07:10 <purpleidea> (i can make it especially elegant to manage it with that style of interface with puppet)
15:07:18 <jdarcy> Yay!
15:07:26 <johnmark> heh...
15:07:44 <hagarth> I think data-classification is an important feature gap that we need to address
15:08:00 <purpleidea> dlambrig: o hai
15:08:04 <sas> hagarth, agree +1 for tiering
15:08:09 <johnmark> hagarth: is there any summary text somewhere?
15:08:11 <jdarcy> Especially now that Ceph has tiering, plus HDFS, Swift, etc. will within this release cycle.
15:08:15 <johnmark> jdarcy: +1
15:08:22 <hagarth> johnmark: http://titanpad.com/glusterfs-3-6-planning
15:08:27 <johnmark> hagarth: thanks
15:08:38 <johnmark> it's going to be a huge isue - because Randy Bias says so ;)
15:08:58 <hagarth> jdarcy: I am inclined to have it at core - but do we have any idea on how much effort the implementation would take?
15:09:02 <jdarcy> johnmark: Yeah, gotta fight the marketing juggernaut.
15:09:36 <jdarcy> hagarth: Medium effort.  It's far from trivial, but the I/O path is pretty simple.  Most of the difficulty is on the management side.
15:09:52 <lalatenduM> hagarth, who will be the dev for data classification?
15:09:56 <hagarth> jdarcy: right, would we need assistance to pull that off?
15:09:57 <jclift> Gahh
15:10:03 <jdarcy> hagarth: If we get the I/O infrastructure in place, we can add management use cases incrementally.
15:10:04 <jclift> TitanPad just went offline for me.
15:10:19 <kkeithley_> jclift: me too
15:10:25 * purpleidea management side.... :) (jdarcy)
15:10:25 <hagarth> jdarcy: right
15:10:36 <lalatenduM> jclift, same for me
15:10:46 <dlambrig> titatpad looks about as good as "bluejeans" video conferencing..
15:10:53 <jdarcy> hagarth: Somebody needs to work on it full time or nearly so for most if not all of the release cycle.
15:11:10 <hagarth> did we /. titanpad? :)
15:11:16 <jclift> dlambrig: Damn.  I was hoping bluejeans would be good. :/
15:11:22 <jdarcy> I can't promise more than half of that myself.
15:11:25 <jclift> hagarth: k, lets try a different pad
15:11:26 * kshlm reporting in.
15:11:28 <hagarth> jdarcy: agree
15:11:58 <purpleidea> hagarth: down for me too
15:12:10 <hagarth> jdarcy: I think we should propose it as core and try to find collaborators in the community?
15:12:13 <jdarcy> I think it's a high enough priority that we can pressure RH management to assign someone.
15:12:28 <johnmark> jdarcy: sure, but what will we have to give up as a result?
15:12:41 <purpleidea> johnmark: glusterforce ;P
15:12:42 <jdarcy> Community collaborators would be even better, but might be scarce for the lower-level parts.
15:12:45 <purpleidea> forge*
15:12:45 <tdasilva> titanpad is back
15:12:50 <johnmark> In order to prioritize that, I can already anticipate the dreaded question of what to de-prioritize
15:12:51 <hagarth> btw, titanpad is back
15:13:00 <johnmark> purpleidea: :P
15:13:13 <jdarcy> johnmark: AFR2
15:13:21 <johnmark> jdarcy: ugh
15:13:36 <dlambrig> deprioritize anything with the word "cinder" in it
15:13:41 <johnmark> LOL
15:13:44 <hagarth> jdarcy: right about community collaboration
15:14:02 <hagarth> jdarcy: AFRv2 is nearly done, so that will not have an effect on data classification
15:14:06 <jdarcy> For now, I think we need to evaluate based on importance, and then make the priority case later.
15:14:13 <johnmark> dlambrig: actually, I can easily make that argument, and I believe I will
15:14:40 <dlambrig> Over 20 people on the scrum meeting invite list .. hmm
15:15:05 <hagarth> jdarcy: I am still inclined to find help in the community and willing to be the mentor for this feature.
15:15:07 <dlambrig> (for cinder)
15:15:20 <hagarth> dlambrig: let us focus on data classification for now :)
15:15:32 <jdarcy> hagarth: If you can drive it (at least until I get free) I think that would be excellent.
15:15:56 <hagarth> jdarcy: sounds like a plan, let us shoot for core. Else I'm afraid we might drop the ball.
15:16:10 <johnmark> hagarth: +1
15:16:45 <hagarth> ok, core for data-classification and feature to be jointly owned by jdarcy and me.
15:16:50 <hagarth> moving on.
15:16:53 <hagarth> 3. Features/heterogeneous-bricks
15:16:58 <shyam> How does data-classification compare to current 4.0 thoughts, would it have to be revisited or reused or extended?
15:17:51 <hagarth> shyam: good question. my thoughts seem to be indicate that 4.0 is still far away and I don't think we have to be encumbered by that for 3.6.
15:17:52 <jdarcy> hagarth: Updated the feature page to reflect your involvement.
15:18:07 <hagarth> jdarcy: cool, thanks.
15:18:08 <dlambrig> I think data classification may well play into the SSD stuff we are experimenting with in the performance group
15:18:23 <dlambrig> Soo.. may want to suggest a dependency between those two efforts.
15:18:26 <hagarth> dlambrig: right
15:18:29 <jdarcy> shyam: The tiering case is too immediately important to wait until 4.0
15:18:56 <purpleidea> +1
15:19:02 <shyam> jdarcy: hagarth:ok
15:19:04 <jdarcy> Hetereogeneous bricks are also core, but this one's easy.  Just some tweaks to the layout calculation.
15:19:11 <hagarth> jdarcy: +1
15:19:30 <shyam> +1 for heterogeneous-bricks
15:19:36 <purpleidea> jdarcy: is there a document or some code about this layout calculation somewhere you can point me to?
15:19:51 <hagarth> ok, core for Features/heterogeneous-bricks.
15:20:02 <jdarcy> purpleidea: Yeah, I pretty much did all of the "think" pieces minus the integration once already.
15:20:05 <shyam> hagarth: +1
15:20:08 * social_ thinks data classification is for underlying FS and not for gluster
15:20:41 <hagarth> social_: would it be fine if we discuss that later?
15:21:13 <hagarth> moving on
15:21:16 <hagarth> 4. Features/new-style-replication
15:21:48 <hagarth> jdarcy: would you want to provide an update on NSR?
15:21:57 * lpabon is here (although a little late)
15:22:22 <tdasilva> is AFR2 the same as NSR?
15:22:26 <jdarcy> NSR is chugging along.  Still a lot of work on reconciliation (equiv. to self-heal) and integration.
15:22:42 <hagarth> tdasilva: no, they are different
15:23:03 <hagarth> tdasilva: AFR2 is still client based replication where as NSR is server side replication
15:23:17 <jdarcy> We could make it, but clearly only one (at most) of NSR and AFRv2 could be considered core for 3.6
15:23:24 <tdasilva> hagarth: ok, thanks
15:23:37 <sks> I guess NSR is important feature
15:23:42 <sks> =1
15:23:44 <sks> +1
15:23:49 <hagarth> jdarcy: AFRv2 is in a much advanced state than NSR in terms of implementation
15:23:55 <shyam> I would state AFR, NSR seems too radical a change for 3.x line
15:24:00 <shyam> rather AFR2
15:24:09 <jclift> NSR == 4.0 is mentally a bit more simple for end users
15:24:16 <sas> jclift, +1
15:24:17 <lalatenduM> I will vote AFR2 for 3.6
15:24:32 <lalatenduM> jclift, +1
15:24:38 <hagarth> can we consider NSR as nice to have for 3.6 or would it be too ambitious?
15:25:09 <jclift> Meh.  Keep it 4. ;)
15:25:21 <shyam> Nice to have is good, ambitious is something jdarcy needs to answer :)
15:25:25 <johnmark> hagarth: I htink having it as a beta feature for 3.x would bre nice
15:25:29 <jdarcy> I'll keep most of my opinions about AFRv2 to myself, but *no* new replication system should be in 3.6 without quorum on by default and a reasonable way to fix split-brain.
15:25:33 <johnmark> and then think about it as core feature for 4.x
15:25:38 <lalatenduM> hagarth, data classifications and nsr both involves jdarcy , we should go for data cla* for 3.6 :)
15:25:40 <overclk> jdarcy: should we have it as a tech preview?
15:26:00 <jdarcy> We've screwed our users with that for far too long already.
15:26:05 <hagarth> jdarcy: agree, that we need better protection for split-brain
15:26:29 <johnmark> overclk: no such thing as tech preview - that's product lingo
15:26:42 <johnmark> we have beta features and "needs more testing" features
15:26:57 <hagarth> jdarcy: there are other feature proposals which seem to  reduce the chances of split-brain. Let us talk about that there.
15:27:20 <jclift> k.  What's the verdict so I can minute it?
15:27:32 <hagarth> nice to have +1 from me
15:27:45 <shyam> nice to have +1 from me as well
15:27:50 <jdarcy> hagarth: Same here.
15:28:01 <jclift> Done :)
15:28:24 <hagarth> ok, Moving on
15:28:34 <hagarth> 5. Features/thousand-node-glusterd
15:28:50 <hagarth> I think we ought to move this beyond 3.6
15:29:11 <purpleidea> +1
15:29:17 <hagarth> don't think we will get to this in the 3.6 time frame, though I would love to see this happen.
15:29:22 <dlambrig> how  many customers are using large clusters... my understanding was it was not many.
15:29:42 <dlambrig> not to belittle the feature. it seems important.
15:29:48 <jdarcy> Not even NTH.  Just defer to 4.0, with regrets.
15:29:52 <sks> dlambrig, yeah right
15:29:56 <lpabon> from a gluster-swift point of view -- we need to reach exabyte amount of storage
15:30:01 <lpabon> or more
15:30:06 <jdarcy> dlambrig: How many *would* use it, if they could?
15:30:18 <johnmark> jdarcy: +1
15:30:21 <lpabon> i would :-)
15:30:32 <dlambrig> jdarcy: I do not know. I would be interested in discussing that with big data folks.
15:30:32 <hagarth> right, let us defer this feature
15:30:40 <johnmark> lpabon: careful, you might be taskd with it ;)
15:30:56 <hagarth> moving on
15:30:59 <hagarth> 6. Features/Trash
15:31:05 <johnmark> ugh
15:31:15 <hagarth> thotz: would you want to provide an update on trash translator?
15:31:22 <lpabon> yeah, but fyi, there are swift deployments in production with exabytes of storeage....
15:31:32 <lpabon> storage*
15:31:34 <hagarth> johnmark: the intent of this feature is not to trash your data :)
15:31:46 <anoopcs> hagarth: It's working
15:31:51 <johnmark> lpabon: who woudl implement it?
15:31:53 <johnmark> hagarth: heh
15:31:54 <purpleidea> lpabon: exabyte range is ~10,000 servers range, right? (based on my quick calculations)
15:31:54 <hagarth> folks, let us avoid having side conversations please
15:32:08 <johnmark> hagarth: ok
15:32:10 <lpabon> depends on the vertical storage
15:32:16 <lpabon> but we can take offline
15:32:22 <thotz> trash translator is working and more useful for internal ops  such as rebalance
15:32:31 <sas> hagarth, trash xlator +1 for 3.6
15:32:32 <anoopcs> Patch already sent upstream for review
15:32:47 <jdarcy> Do we know of users pushing for Trash?
15:32:54 <johnmark> jdarcy: my question, too
15:32:56 <hagarth> I am inclined to call this core as we seem to have made progress on this.
15:33:30 <hagarth> jdarcy, johnmark: I have seen a few users requesting this for protection against fat fingers.
15:33:52 <lpabon> doesn't samba have this already?
15:33:55 <jdarcy> My criterion for "core" is that not having it will either harm existing users or significantly impede adoption by new ones.
15:34:01 <anoopcs> jdarcy: In case of accidental deletion of files
15:34:20 <hagarth> lpabon: yes, samba has a trashcan module
15:34:29 <anoopcs> trash may be helpful
15:34:37 * sas nods
15:34:55 <jdarcy> hagarth: Does the Samba functionality depend on the underlying system, or is it internally implemented in Samba itself?
15:35:16 <lpabon> afaik it is in samba itself
15:35:27 <hagarth> I view trash as a helpful addon and also being useful to catch accidental unlinks that our internal maintenance operations like self-heal/rebalance might do.
15:35:35 <lalatenduM> lpabon, I am not aware of this samba feature, will like to talk to u abt it
15:35:36 <lpabon> but we could base the algorithm on that implementation
15:35:40 <hagarth> jdarcy: this is in Samba itself.
15:36:11 * jdarcy doesn't really have an opinion on this one, just trying to establish context.
15:36:13 <anoopcs> As thotz said, trash is working in case  of self-heal and rebalance
15:36:14 <jclift> Sounds like Nice To Have, as people won't be dying if it's not in 3.6.
15:36:22 <lpabon> lalatenduM: i'm not very familiar with it (i investigated it 4 years ago).. may want to talk to Ira about it
15:36:24 <jclift> That said, sounds pretty likely to make it anyway
15:36:39 <hagarth> jclift: so, no harm in calling it core?
15:36:45 <jclift> Sure
15:36:46 <johnmark> jclift: sicne it's mostly done...
15:37:08 <jclift> k, it's marked down as Core
15:37:13 <lalatenduM> hagarth, +1 for trash xlator in 3.6 not sure abt core or "go to have"
15:37:22 <hagarth> I am basing core/nice to have also on implementation status - so let us call it core.
15:37:26 <lalatenduM> lpabon, sure will do
15:37:28 <hagarth> Moving on
15:37:32 <anoopcs> jclift :Thanks
15:37:39 <hagarth> 7. Features/disperse
15:37:53 <hagarth> xavih: would you want to provide an update on disperse?
15:38:16 <xavih> disperse is corrently working (much more tests needed)
15:38:17 <jdarcy> Another feature our competitors all have or will have within this release period, BTW>
15:38:42 <xavih> the self-healing part is not enabled yet, but is basically implemented
15:38:49 <hagarth> jdarcy: right, would love to see this being core
15:38:53 <jdarcy> +0.9 for core
15:39:01 <pk> +1 for core
15:39:03 <johnmark> heh
15:39:09 <sas> hagarth, +1
15:39:12 <kkeithley_> +1 core, tech preview?
15:39:20 <johnmark> kkeithley_: beta
15:39:22 <dlambrig> xaviih: I hope to help you with some testing.. we have a large gluster in our lab which may be of assitence, I'll ping you offline
15:39:38 <hagarth> xavih: beta or GA for 3.6?
15:39:39 <johnmark> and since xavih will be in SF in April...
15:39:42 <xavih> dlambrig: ok
15:39:55 <johnmark> we can have a meeting of the minds :)
15:40:12 <xavih> hagarth: we will try to have it fully functional by the release time of 3.6
15:40:31 <jdarcy> xavih: Yay, I can buy you a refreshing beverage!
15:40:34 <xavih> we are making good progress recently and solving many problems
15:40:36 <hagarth> xavih: great!
15:40:43 <johnmark> xavih: that would be awesome.
15:40:52 <hagarth> so disperse is core
15:40:59 <johnmark> jdarcy: you coming to SF?
15:41:02 <dlambrig> xavih: Im very curious about the performance for the v1 version... we shall see soon :)
15:41:12 <jdarcy> johnmark: Never really had a choice, did I?
15:41:15 <hagarth> OK, Moving on
15:41:20 <johnmark> jdarcy: lol... gues not
15:41:21 * sas votes for disperse to be core
15:41:26 <hagarth> 8. Features/IPv6
15:41:36 <xavih> dlambrig: it's really poor for small files... maybe caching could alleviate that...
15:41:42 <hagarth> ndevos: the floor is yours now :)
15:41:45 <dlambrig> xsavih: caching +1
15:41:48 <jdarcy> NTH unless we have users (e.g. federal) who require it.
15:42:14 <ndevos> well, it seems that IPv6 is not working easily for all users in #gluster
15:42:21 <hagarth> i see occasional reuqests on users ML
15:42:49 <ndevos> some manage to get it working, others not - we'll need to do some testing and make it easier to work with IPv6
15:42:53 <johnmark> hagarth: agreed
15:42:54 <hagarth> ndevos: would you be working on this one?
15:43:08 <johnmark> and some snarky hipster types who rag on anything that doesn't support it
15:43:12 <ndevos> hagarth: I'll try to, but can not guarantee that
15:43:25 <hagarth> ndevos: shall we aim for Nice to have?
15:43:37 <ndevos> hagarth: yes, sounds good to me
15:43:59 <hagarth> right, IPv6 is NTH for 3.6.
15:44:07 <hagarth> Moving on to 8. Features/glusterd-volume-locks
15:44:12 <hagarth> s/8/9/
15:44:26 * hagarth looks at glusterbot to auto-correct
15:44:34 * kkeithley_ has to drop off, biab
15:44:36 <hagarth> the feature page looks rather incomplete
15:44:44 <hagarth> but the code is mostly in master
15:44:45 <jdarcy> NTH
15:44:59 <hagarth> do we have Avra here?
15:45:11 <hagarth> guess not, so NTH +1
15:45:31 <hagarth> volume locking for glusterd is NTH.
15:45:31 <johnmark> hagarth: if feature page is incomplete...
15:45:50 <kshlm> hagarth: its a requirement for snapshot, so if it is core the volume locks is core as well.
15:45:58 <hagarth> johnmark: providing it the benefit of doubt since the implementation is ready
15:46:12 <hagarth> kshlm: for a core feature, we need better feature pages.
15:46:16 <johnmark> hagarth: ok
15:46:18 <kshlm> btw, is snapshot in the 3.6 list?
15:46:26 <lalatenduM> kshlm, +1
15:46:29 <hagarth> kshlm: down the list
15:46:31 <johnmark> kshlm: then they better get on teh ball
15:46:52 <hagarth> OK, I will reach out to feature owners for this one
15:47:04 <kshlm> hagarth: i don't see it.
15:47:11 <hagarth> NTH till we hear otherwise for this feature
15:47:29 <hagarth> kshlm: hang on, we'll get there :)
15:47:34 * kshlm sees it now.
15:47:36 <hagarth> Moving on
15:47:39 <hagarth> 10. Features/persistent-AFR-changelog-xattributes
15:47:50 <hagarth> itisravi: would you want to provide an update?
15:48:01 <itisravi> The implementation for this is done and the patches have had intial reviews
15:48:23 <itisravi> I'd say core because it is neede for snapshots
15:48:28 <hagarth> itisravi: +1
15:48:34 <pk> itisravi: +1
15:48:48 <itisravi> and for allowing remove-brick when self heals are pending
15:48:49 <shyam> +1 although not a UUID based approach for solving the vol naming across xlators, something needed for snapshots
15:48:52 <jdarcy> What's the interaction between this and AFRv2?
15:49:12 <itisravi> jdarcy: The changes are independent of AFR v1/v2
15:49:32 <kshlm> jdarcy: this is mostly a glusterd change
15:49:52 <hagarth> itisravi: right, this provides better manageability for remove-brick operation too.
15:50:02 <itisravi> hagarth: yup
15:50:40 <hagarth> ok, let us call this core since the improvement seems to be quite useful.
15:50:54 <hagarth> Moving on
15:51:05 <hagarth> 11. Features/better-logging
15:51:09 <shyam> Feature is more a system administrator friendly message guide for messages that gluster spews out, which requires the stated additions to the messages in gluster code
15:51:18 <shyam> This can help build our trouble shooting knowledge base and also help sysadmins figure and fix some of the more common issues
15:51:24 <shyam> But, this is NTH and not core from a feature standpoint according to me, if implementation status based then I would state "Core"
15:51:27 <shyam> :)
15:51:29 <hagarth> shyam: +10 for this feature :)
15:52:21 <hagarth> shyam: shall we call it NTH?
15:52:37 <jclift> shyam: Patches for this are ready?
15:52:40 <shyam> Are we considering implementation status?
15:52:55 <shyam> yes, for the infrastructure, but messages still need to change
15:52:56 <jdarcy> Aren't changes for this going to conflict with, well, everything?
15:53:01 <kshlm> +1 NTH
15:53:02 <hagarth> shyam: yes, 50 - 50 split between the two factors
15:53:14 <shyam> hagarth: yup, agreed
15:53:20 <hagarth> i.e importance and implementation status
15:53:25 <shyam> NTH for me, but will get in
15:53:38 <shyam> or make the cut offs for code completion etc.
15:53:48 <jclift> If it makes things a lot easier for SysAdmins, I'm all +1 core ;D
15:54:06 <hagarth> shyam: do we plan to have everything mentioned in the feature page for 3.6?
15:54:15 <itisravi> jclift: For Devs too :)
15:54:16 <jclift> Manageability is one of our key differentiators, etc.
15:54:17 <dlambrig> jclift: +1 agree on user experience r.e. sysadmins
15:54:20 <jdarcy> Maybe we should define a staging plan for which components will be updated in which order.
15:54:26 <shyam> maybe not SF5 but everything else, yes
15:54:50 <jclift> jdarcy: For git rebasing purposes?
15:54:53 <shyam> jdarcy: I have the component list, but no staging plan, ATM
15:55:22 <shyam> jclift: I would assume we do this directly on master than a feature branch and them merge, the headache would be a bit too much
15:55:23 <hagarth> shyam: we can possibly co-ordinate that activity in the gluster.org wiki
15:55:35 <shyam> hagarth: yes
15:55:54 <hagarth> ok, NTH but will definitely be in :)
15:55:57 <hagarth> let us move on
15:56:03 <johnmark> hagarth: wait
15:56:09 <jdarcy> This is going to be a long slog, it would be good to get the most important kinds of debugging (e.g. related to components with known problems like split-brain or GFID mistmatch) scheduled first.
15:56:12 <hagarth> johnmark: waiting
15:56:14 <johnmark> still curious about jdarcy's comment "doesn't this change... everything"
15:56:28 <hagarth> johnmark: I don't think we need to change everything in one shot
15:56:36 <johnmark> hagaok
15:56:39 <jdarcy> johnmark: Those log statements are everywhere.  Updating them will create a lot of path conflicts.
15:56:39 <johnmark> gah
15:56:41 <johnmark> hagarth: ok
15:56:47 <johnmark> hrm
15:56:48 <johnmark> ok
15:56:52 <hagarth> we can do that incrementally and have some co-ordination plan going in the wiki
15:56:54 <jclift> jdarcy: That's a good idea.  But discussion for it on mailing list, etc.
15:57:19 <johnmark> jclift: yes
15:57:38 <jclift> k, it's marked down as NTH
15:57:44 <hagarth> right, let us discuss in gluster-devel on how we want to do this
15:57:45 <shyam> johnmark: jdarcy: The changes can be done incrementally, we would deprecate the older logging APIs post a point
15:57:58 <shyam> hagarth: ok, I will initiate the thread on devel
15:58:07 <hagarth> shyam: thanks
15:58:09 <hagarth> 12. Features/Object_Count
15:58:19 <hagarth> Object_counter is a better name, I will change that
15:58:35 <shyam> hagarth: Missed, Features/Better peer identification?
15:58:35 <jdarcy> (What happened to better peer identification?)
15:58:42 <hagarth> This was core for 3.5 but slipped since the quota merge happened late in the cycle
15:58:45 <semiosis> :O
15:58:50 <hagarth> oops, I will take that up as #13
15:58:57 <jclift> #action shyam will initiate discussion about incremental logging changes on gluster-devel mailing list
15:59:02 <jclift> Heh
15:59:14 <lpabon> is object count count the number of files (not dirs) under a volume?
15:59:23 <hagarth> Implementation status is partial but I plan to wrap it up for 3.6
15:59:32 * johnmark will have to drop off for a while
15:59:32 <lpabon> is there an email/doc that describes it?...
15:59:32 <hagarth> lpabon: yes
15:59:45 <jdarcy> Seems NTH generally, though other components might need it badly enough to make it core.
15:59:48 <hagarth> lpabon: feature page has minimal details - will add more later
15:59:58 <shyam> jdarcy: +1
16:00:18 <lpabon> hagarth: we may need to talk about this offline, i'm not sure how this will integrate with object store
16:00:19 <shyam> Do we know of other components that need this?
16:00:40 <hagarth> lpabon: sure, I see clear use cases with ceilometer etc.
16:00:51 <lpabon> hagarth: ok
16:00:58 <hagarth> this will also be the foundation for inode quotas
16:01:28 <hagarth> I am okay with this being NTH (but I'll mostly get this in from an implementation standpoint)
16:01:43 <hagarth> NTH it is then.
16:01:52 <hagarth> #13 Better Peer Identification
16:01:59 <jclift> BPI +1 core
16:02:19 <jdarcy> Similar benefit to better logging, but not nearly as invasive.
16:02:32 <jdarcy> So +1 from me as well
16:02:38 <hagarth> +1 from me too
16:02:46 <kshlm> +1 core. I'll get it in this time.
16:02:47 <msvbhat> +1 from me too
16:03:02 <hagarth> kshlm: thanks, we'll owe you one if it makes into 3.6 :)
16:03:15 <jclift> kshlm: :)
16:03:25 <hagarth> so, BPI is core.
16:03:34 <hagarth> Moving on
16:03:37 <hagarth> 14. Features/Easy addition of custom translators
16:03:42 * jdarcy needs to add multi-network support to Quattro
16:03:46 <jclift> Meh, no-one to work on it. :(
16:03:53 <jclift> Easy custom translators
16:03:59 <kshlm> I'd say this is a feature for 4.0
16:04:00 <jclift> So, unlikely to see daylight
16:04:12 <kshlm> when overhaul our management framework.
16:04:19 <dlambrig> we already claim to be better than ceph in this deparment
16:04:21 <kshlm> s/when/when we/
16:04:31 <hagarth> jclift: OK, I think we can do a minor hack to make it possible but let us discuss this offline
16:04:35 <shyam> +1 on custom translators, but would be interested on how we keep version compatibility for custome translators and core (if the problem exists)
16:04:47 <jclift> hagarth: Sure, np
16:05:03 <hagarth> jclift: NTH so that it does not fall off the radar?
16:05:16 <jclift> hagarth: Nah.  Just leave it.
16:05:22 <jdarcy> NTH, with an AI for a more complete description for what these JSON fragments would actually look like and how they'd translate into different volfiles
16:05:25 <jclift> We'll bring it up again in due course. :)
16:05:28 <lpabon> i would be interested in proposing a method for easy translators
16:05:44 <hagarth> jclift: OK, let us discuss more on gluster-devel
16:05:48 <hagarth> Moving on
16:05:48 <jclift> Sure
16:05:57 <hagarth> 15. Features/Exports_Netgroups_Authentication
16:06:00 <itisravi> hagarth: think we missed SE Linux integration
16:06:13 <hagarth> itisravi: thanks for reminding, will pick that as #16
16:06:37 <hagarth> anoopcs: any update on this feature? would you be able to complete it by 3.6?
16:06:47 <hagarth> the patches are already available for review
16:06:52 <shyam> Netgroups and NFS Ganesha, what is the way forward for NFS and Gluster?
16:07:02 * itisravi likes to make Dan Walsh weep :/
16:07:04 <anoopcs> hagarth: Only some tests are pending.
16:07:25 <anoopcs> Not some. Only one.
16:07:34 * jdarcy tries to find a feature page for Ganesha/NFSv4/etc.
16:07:37 <hagarth> shyam: NFS Ganesha is an add on feature for now, we will provide an easy way from glusterd to spawn Ganesha and then start deprecating the native implementation.
16:07:49 <hagarth> anoopcs: cool, let us tag this as NTH for 3.6
16:07:57 <hagarth> though I feel we will get this in
16:08:20 <hagarth> NTH it is then.
16:08:30 <hagarth> itisravi: let us not make him weep, so moving on to
16:08:34 <hagarth> 16. Features/SELinux_Integration
16:08:49 <hagarth> bfoster, avati: any updates on this one?
16:09:07 <bfoster> the policy stuff for server side enablement is mostly set
16:09:18 <bfoster> i have a script for usability that needs to get in
16:09:31 <bfoster> the client side label bits are further out, depends on selinux and fuse changes
16:09:40 <hagarth> bfoster: ok
16:09:46 <jdarcy> NTH?
16:10:00 <jclift> "further out"... like weeks, or months?
16:10:06 <jclift> bfoster: ^^
16:10:10 <hagarth> yeah, NTH sounds fine to me.
16:10:24 <bfoster> jclift: no idea, particularly because of the selinux bits
16:10:31 <jclift> NTH then. :)
16:10:40 <hagarth> ok, cool :)
16:10:44 <hagarth> Moving on
16:10:48 <hagarth> 17. Features/Gluster_Volume_Snapshot
16:11:05 <jdarcy> NTH  >:-}
16:11:10 <hagarth> jdarcy: :)
16:11:34 <hagarth> core from me given that there has been quite a few requests.
16:11:35 <msvbhat> +1 volume snapshot to core
16:11:40 * lpabon is curious on what NTH means
16:11:48 <msvbhat> lalatenduM: Nice to have
16:11:50 <jdarcy> The amount of resources being poured into this mean it'll be there regardless of how we tag it.
16:11:50 <hagarth> lpabon: NTH = Nice To Have
16:12:13 <lalatenduM> msvbhat, you meant lpabon :)
16:12:16 <kshlm> jdarcy: +1 :)
16:12:22 <jdarcy> Also, competitors yadda yadda.
16:12:23 <msvbhat> lalatenduM: Yes :)
16:12:39 <hagarth> and since the feature has already appeared on r.g.o, core +1
16:12:39 <dlambrig> my understanding was snapshot is a given at this point.
16:12:47 <lpabon> msvbhat: thanks :-)
16:12:52 <lalatenduM> +1 snapshot should be in core
16:13:02 <dlambrig> so, +1 FWIW
16:13:13 <shyam> +1 core
16:13:23 <hagarth> done, core for snapshot
16:13:28 <hagarth> Moving on
16:13:36 <hagarth> 18. Features/Gluster_User_Serviceable_Snapshots
16:13:42 <jdarcy> NTH (seriously this time)
16:13:46 <jclift> :)
16:13:48 <hagarth> jdarcy: +1
16:14:15 <msvbhat> +1 user serviceable snapshots to NTH
16:14:16 <hagarth> I don't see Anand Subramanian here too .. so NTH
16:14:22 <shyam> hagarth: Do we have any status update on this to make an implementation based call?
16:14:40 <raghug> +1
16:14:40 <shyam> Like code ready or design ready etc.
16:14:54 <raghug> shyam: Design is ready
16:15:00 <hagarth> shyam: I am not aware of the implementation having gone beyond prototyping
16:15:07 <raghug> shyam: Anand is working on a prototype
16:15:22 <hagarth> shyam: I think a few elements of the design are still being sorted out.
16:15:33 <shyam> raghug: hagarth: ok
16:15:40 <hagarth> ok, let us move on
16:15:43 <hagarth> 19. Features/afrv2
16:15:53 <jdarcy> It's an improvement where one is sorely needed, so +1
16:16:10 <pk> hagarth: +1
16:16:12 <itisravi> +1 from me too
16:16:16 <hagarth> pk: any update on the implementation status?
16:16:17 <shyam> +1 improves the current situation and implementation is quite advanced, core
16:16:40 <semiosis> where can I read more about afrv2?
16:16:40 <hagarth> +1 core from me too
16:16:40 <avati> its an improvement and almost done, +1
16:16:41 <pk> hagarth: there are some bd regression failures because of CALLOC/GF_FREE
16:16:49 <hagarth> semiosis: #link http://www.gluster.org/community/documentation/index.php/Features/afrv2
16:16:51 <semiosis> thx!
16:16:54 <pk> hagarth: I am gonna send the patch out after this meeting
16:17:08 <hagarth> avati, pk: cool, thanks for the update!
16:17:14 <hagarth> core it is then
16:17:27 <hagarth> Moving on to 20. Features/outcast
16:17:36 <jclift> semiosis: This is the etherpad for this meeting if it helps.  Everything has links in there: http://titanpad.com/glusterfs-3-6-planning
16:17:51 <semiosis> jclift: nice
16:17:56 <lpabon> pk: do you mind putting me as a reviewer in the patch?
16:18:17 <hagarth> lpabon: feel free to chip in
16:18:31 <pk> lpabon: sure
16:18:36 <jdarcy> Outcast +1.  Keeps us from losing user data in some cases.
16:18:41 <hagarth> I think outcast is important for better consistency
16:19:00 <avati> what's the state of implementation of outcast?
16:19:01 <semiosis> +1, fwiw
16:19:12 <avati> pk?
16:19:14 <hagarth> pk: any thoughts on whether we will be able to do this for 3.6?
16:19:42 <pk> avati, hagarth: I am not sure. Design also is not ready... :-|
16:19:53 <avati> NTH then?
16:19:59 <pk> avati: yep
16:20:24 <hagarth> ok, NTH but a strong bias towards core for this
16:20:32 <hagarth> Moving on
16:20:41 <hagarth> 21. Features/pbspbr
16:20:49 <hagarth> well, that is some acronym :)
16:21:01 <pk> hagarth: Sorry about that :-)
16:21:04 <hagarth> policy based split brain resolution for the uninitiated
16:21:06 <jdarcy> *Some* form of better split-brain resolution has to be core for 3.6.  Might as well be this.
16:21:15 <hagarth> jdarcy: +1
16:21:30 <avati> +1 for pbspbr
16:21:33 <pk> this is easier to implement IMO
16:21:36 <hagarth> given the frequency of split-brains, I would say that this is core.
16:21:41 <shyam> This looks like a configuration overhead for the admins
16:21:43 <avati> makes you dlysexic
16:21:44 <jdarcy> pk: Easier than what?
16:21:45 <dlambrig> I hear a lot about split brain problems from field people.
16:21:53 <shyam> Should this be the one?
16:21:59 <pk> Easier than outcast, because I know what needs to be done
16:22:29 <hagarth> shyam: yes, let us evolve this and make it easier to resolve split-brains
16:22:30 <avati> outcast == preventive, pbspbr == curative
16:22:32 <jdarcy> TBH, I'd favor manual (but not back-end) resolution, with policy stuff layered on top.  Maybe we can move toward that in the implementation.
16:22:32 <jclift> With is likely to have the better result?
16:22:42 <jclift> s/With/Which/
16:22:44 <dlambrig> I'
16:22:49 <hagarth> jdarcy: +1
16:22:50 <dlambrig> I'd got with curative over preventative
16:23:08 <hagarth> are we all in agreement that this is core?
16:23:08 <dlambrig> if I had to choose.
16:23:15 <jdarcy> hagarth: OK by me
16:23:15 <avati> i vote for core
16:23:22 <hagarth> ok, core it is then
16:23:30 <hagarth> let us move on to 22. Features/rest-api
16:23:44 <hagarth> this would be very nice to have
16:23:47 <aravindavk> Based on the current POC I think I can complete in time.
16:23:50 * jclift hopes it's a daemon
16:23:54 <jdarcy> NTH, and needs to support the same level of security as the CLI/glusterd path.
16:23:58 <aravindavk> NTH
16:24:11 <avati> NTH
16:24:12 <hagarth> aravindavk: great, NTH then.
16:24:15 <jclift> NTH.  But _really_ NTH. :)
16:24:25 <hagarth> jclift: +1 :)
16:24:28 <avati> heh :)
16:24:33 <hagarth> let us move on to 23. Features/RDMA_Improvements
16:25:00 <raghug> hagarth: +1
16:25:01 <hagarth> to overcome user reported problems, I would like to call this core
16:25:04 <shyam> Are there users lined up for this one? Seems to get postponed too often
16:25:10 <dlambrig> I think RDMA will become increasing important for big data scenarios, but we may not being hearing the desirability on this as much today.
16:25:18 <hagarth> shyam: we have a lot of user requests on this one.
16:25:26 <jdarcy> We've had people trying to use RDMA for years.  Many have given up.
16:25:40 <avati> if we have hardware resources ready available i would vote for core
16:25:41 <jdarcy> OTOH, I think Argonne and/or Harvard might be succeeding with it.
16:25:55 <jclift> We have hw available both in house and in the Community too
16:25:56 <dlambrig> We do have infiniband equipment in the lab here.
16:25:59 <hagarth> avati: we can also find hardware from FSL or other community deployments
16:26:01 <shyam> Well then based on that and the fact that this is moving across releases, +1 core
16:26:11 <raghug> +1 core
16:26:17 <jclift> Yeah.  +1 core
16:26:25 <shyam> hagarth: Do we have the people?
16:26:25 <hagarth> great, rdma is core for 3.6.
16:26:37 <hagarth> shyam: I volunteer myself with raghug :)
16:26:44 <shyam> hagarth: cool :)
16:26:49 <avati> raghug == du?
16:26:52 <dlambrig> I believe Kalab mentioned something about this, you may want to check with him as well.
16:26:56 <raghug> avati: yup
16:26:56 <hagarth> avati: right
16:27:08 <hagarth> dlambrig: sure, more hands will be useful.
16:27:13 <hagarth> Ok, Moving on
16:27:16 <hagarth> 24. Features/Archipelago_Integration
16:27:23 <jdarcy> NTH
16:27:30 <hagarth> jdarcy: +1
16:27:31 * jdarcy needs to look into Archipelago some more
16:27:34 <jclift> Pretty clearly not core
16:27:40 <avati> yep
16:27:52 * jclift marks it as NTH
16:27:53 <hagarth> ok, NTH then for 3.6
16:28:08 <hagarth> and the last feature for today
16:28:10 <hagarth> 25. Features/Server-side_Barrier_feature
16:28:24 <kshlm> another snapshot requirement
16:28:27 <avati> isn't that a sub-feature/dependency of snapshot anyways? should ib e listed separate?
16:28:27 <sas> hagarth, Barrier for core
16:28:34 <jclift> How "supplemantary" is it?
16:28:37 <shyam> kshlm: +1 core
16:28:39 <hagarth> core since it is snapshot dependency
16:28:42 <jclift> Sounds kind of "required_ for snapshots
16:28:44 <jclift> Heh
16:28:47 <hagarth> jclift: right
16:28:48 <jclift> +1 core then
16:28:53 <jdarcy> Priority inheritance FTW.
16:28:57 <jclift> ;)
16:29:01 <hagarth> ok, core it is then.
16:29:03 <kshlm> it's ready btw.
16:29:04 <dlambrig> if it is a snapshot requirement, why call it out independently
16:29:10 <hagarth> kshlm: right
16:29:31 <hagarth> ok, let us quickly talk about acceptance criterion
16:29:38 <shyam> dlambrig: can be used independent of snaps as well for other purposes, hence...
16:29:42 <hagarth> I have lined four items at the bottom of the etherpad
16:30:13 <avati> (ehterpad link again please? joined late :)
16:30:15 <hagarth> we were not very rigorous in 3.5, but I would like to follow them rigorously for 3.6
16:30:18 <jclift> http://titanpad.com/glusterfs-3-6-planning
16:30:22 <jclift> avati: ^
16:30:27 <avati> thanks!
16:30:31 <jclift> :)
16:30:53 <hagarth> what do we think about the criterion?
16:30:54 <shyam> hagarth: Clarify "Documentation for changes in user experience"
16:31:08 <shyam> hagarth: rather, could you please...
16:31:17 <hagarth> shyam: any user perceivable change should be documented in admin-guide
16:31:28 <jclift> People that use the feature must be able to understand and learn it, without having to ask on IRC or the mailing list(s)
16:31:37 <hagarth> which is yet another patch to doc/admin-guide
16:31:40 <dlambrig> would that include logging changes?
16:31:44 <shyam> hagarth: jclift: ok
16:31:57 <shyam> dlambrig: I would assume so, yes
16:32:04 <hagarth> dlambrig: yes
16:32:39 <hagarth> I think we can also have checkpoints twice in the release cycle to see how we are doing wrt feature classification
16:33:01 <hagarth> is the criterion good enough? or do we need more / less?
16:33:07 <jdarcy> I would say *changes to existing regression tests* should be an option for #2
16:33:08 <tdasilva> unit tests?
16:33:11 <lpabon> what percentage of unit test coverage should be the entry/exit criteria?
16:33:12 <shyam> hagarth: +1 on checkpoints
16:33:13 <jdarcy> e.g. for AFRv2
16:33:34 <dlambrig> No dataloss regressions? :)
16:33:35 <hagarth> lpabon: since unit tests are relatively new, I would not add that dependency in 3.6
16:33:44 <hagarth> jdarcy: right
16:33:56 <jdarcy> Also, I'd rather have a requirement for performance methodology and results than "no performance regressions" per se.
16:34:03 <lpabon> hagarth: gotcha
16:34:30 <jdarcy> "No performance regressions" without proof is worthless.  OTOH, a slight but quantifiable performance regression might be OK.
16:34:31 <hagarth> dlambrig: what is a dataloss? my code never causes that ;)
16:34:36 <jclift> jdarcy: Well, a perf methodology would be good to have regardless, so we can test reliably
16:34:59 <jclift> jdarcy: (maybe a 3.7 feature? :>)
16:35:09 <hagarth> jdarcy: right, should owners update the feature page with some quantitative performance analysis?
16:35:11 <dlambrig> I think BenE has some methods for performance testing developed the past few years which may fit the bill.
16:35:41 <jdarcy> hagarth: That's what I thinking.  We don't have to settle on a single methodology for now (as dlambrig suggests that could be 3.7) but the info should be there for evaluation.
16:35:46 <jclift> hagarth: I have some data loss examples here for you... I just can't find them right atm...
16:35:53 <avati> we would need to think of performance beyond datapath, including glusterd/cli changes
16:36:08 <jclift> (j/k)
16:36:09 <hagarth> it does look like a rich topic to discuss over a ML thread.
16:36:17 <hagarth> jclift: glad that you don't see it ;)
16:36:32 <jdarcy> At least AFRv2, NSR, disperse and snapshot are affected by this.
16:36:57 <hagarth> shall we defer the performance discussion to gluster-devel? I have some thoughts and we can chime in with all our ideas there.
16:37:01 <jclift> Yeah
16:37:07 <jdarcy> Sounds good.
16:37:12 <dlambrig> it seems large enough to warent a sep conversation
16:37:39 <hagarth> alright, that's all I seem to have for now. I will schedule checkpoints twice in the 3.6 cycle.
16:37:42 <jclift> Maybe start with people's generally used IObench settings, but different accepted sets of settings for different situations
16:37:43 <dlambrig> happily, we seem to be faster than ceph
16:37:56 <hagarth> #action hagarth to schedule checkpoints for 3.6
16:38:00 <jclift> (eg ssd backed, 10GbE networked, etc)
16:38:36 <hagarth> since it has gone well beyond 60 minutes, I am ending the meeting now. Feel free to have more discussions/debates on #gluster-dev or gluster-devel ML.
16:38:38 <jclift> Can everyone please check on the etherpad that they're in the "attendee's" list before they go?
16:38:50 <hagarth> thanks everybody for making it today and having a productive discussion on 3.6
16:38:56 <hagarth> onward to 3.6!
16:39:07 <hagarth> #endmeeting