14:01:24 #startmeeting 14:01:25 Meeting started Wed Nov 27 14:01:24 2013 UTC. The chair is hagarth. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:25 Useful Commands: #action #agreed #halp #info #idea #link #topic. 14:01:25 Meeting started Wed Nov 27 14:08:03 2013 UTC. The chair is hagarth. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:01:25 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:01:29 hi all 14:01:46 hello 14:01:47 meeting agenda is at http://titanpad.com/gluster-community-meetings 14:02:14 #topic 3.5 release updates 14:02:40 so, here are the updates on 3.5 14:02:50 1. release-3.5 has been branched 14:03:11 you can checkout release-3.5 in git now 14:03:22 2. we will start with a regular stream of 3.5 qa releases 14:03:33 OH HAI 14:03:40 hagarth: thank you 14:03:46 3. we had planned for a test day on nov 29th 14:04:03 hagarth: sweet re: 3.5 14:04:06 considering that it is the holiday week in US, I am inclined to push it out by a week 14:04:16 any thoughts on pushing the test day by a week? 14:04:24 #agree 14:04:25 hagarth: yes, definitely next week 14:04:44 hagarth: I assume 3.5 builds? and we can actually try stuff out? 14:04:56 johnmark: that will start from tomorrow 14:05:00 will rpm's be built as usual? 14:05:12 purpleidea: yes, as with all qa releases rpms will be available 14:05:27 we also need to note that all features in core did not make it to 3.5 14:05:36 such features will get pushed out to 3.6 14:06:00 hagarth: cool 14:06:13 hagarth: we just need to list those out 14:06:20 * johnmark is tracking etherpad, adding notes 14:06:26 for features that made it to 3.5, we need to have test cases defined in the appropriate feature pages so that everybody can give it a spin 14:06:36 johnmark: right, I will take that AI on me 14:06:47 hagarth: so if we push the test day out by a week, then that will make it December 6 14:06:48 #action hagarth to update planning35 page 14:06:50 hagarth: thank you 14:06:55 johnmark: right 14:07:18 #action feature owners to update test plans in feature pages 14:07:39 we also need to evolve documentation for 3.5 14:07:56 i have this plan in mind, let me know if you have other thoughts on how we can reach there 14:08:10 hagarth: ok 14:08:21 we have standardized on markdown for writing our docs 14:08:24 hagarth: so feature owners need ot get on the ball if they want their features tested 14:08:33 johnmark: right 14:09:02 I think we need the entire admin-guide to be updated and made browseable 14:09:14 hagarth: +1 14:09:18 hagarth, #agree 14:09:21 december 6 test day in what time zone? 14:09:26 hagarth: as you may or may not know, i wrote docs (in markdown) for puppet-gluster. feel free to integrate them into gluster docs (if you want) and/or link to my DOCUMENTATION.md file. 14:09:36 #action jmw to add admin guide to gluster-docs-project repo 14:09:45 purpleidea: awesome :) 14:09:49 can we have one or more documentation marathons scheduled before 3.5? would there be enough interest? 14:10:06 hagarth: I think we need to try it, but we need the docs team to particiapte 14:10:13 where are the features documented? is there a page that shows the markdown documentation in a rendered form? 14:10:26 gkleiman: december 6 for 24 hours, hopefully the community can support all day 14:10:31 ndevos: those are two separate questions 14:10:40 ndevos: you can browse the markdown documentation in github 14:10:58 ah, github! 14:10:59 ndevos: this is the feature planning page: http://www.gluster.org/community/documentation/index.php/Planning35 14:11:03 ndevos: the features have minimal feature pages on gluster.org 14:11:07 ndevos: pandoc can render to pdf, see https://github.com/purpleidea/puppet-gluster/blob/master/Makefile 14:11:19 purpleidea: ...and lots of other formats 14:11:20 johnmark: yes, but that is not markdown, or is it? 14:11:39 ndevos: github renders markdown 14:11:41 fyi, we use markdown doc in github also for gluster-swift 14:11:51 ndevos: nope. and that gest into the territory of our migration of ht eweb site to git, markdown, and flat files 14:11:58 ndevos: but that is a topic for another day 14:11:58 ndevos johnmark : github markdown is similar to markdown but slightly different 14:12:19 you can write markdown that is compatible with both. 14:12:22 ndevos: Example: https://github.com/gluster/gluster-swift/blob/master/README.md 14:12:24 ndevos: but yes, we'll need to move the planning pages (and everythign else) to markdown shortly 14:12:29 i plan to add a documentation day as part of the 3.5 schedule, any objections to that? 14:12:37 hagarth: none. go for it 14:12:52 #action hagarth to add documentation day in 3.5 schedule 14:13:03 yes, looks like we need that :) 14:13:21 another thing around 3.5 is the other ecosystem projects around glusterfs 14:13:33 should we have a co-ordinated release of such projects? 14:13:35 hagarth: yes 14:13:44 oh wait... hang on 14:13:49 hagarth: yes, we need to discuss that 14:13:53 i am referring to gluster-swift, gluster-deploy, puppet-gluster etc. 14:13:57 hagarth: I think purpleidea will have opinions :) 14:14:12 johnmark: uh? i do? 14:14:14 we have lpabon also :) 14:14:31 hagarth: i'm not sure if that is possible. Gluster-swift follows openstack-swift releases 14:14:35 hagarth: yes - we have a board meeting next week to decide on which projects currently meet the standard for graduation 14:14:42 lpabon: agree 14:14:50 lpabon: hrm, ok 14:15:02 lpabon: so releasing it with glusterfs 3.5 is not possible? 14:15:12 lpabon: or simply not recommended? 14:15:15 i have no problem coordinating a release with gluster. my only issue is i lack vm's or hardware to appropriately test a qa build, or anything that isn't gluster stable (home cluster) 14:15:19 lpabon we could track a separate document for swift and mention it somewhere in the main 3.5 docs 14:15:25 purpleidea: ok 14:15:31 if there are projects which are dependent on 3.5, we can consider having a co-ordinated release. does that make sense? 14:15:41 johnmark: I mean that releasing with 3.5 can just grab whatever has been stabilized.. which may mean a Havana, or Grizzly, or Icehouse release 14:15:50 hagarth: sure. so release 3.5, and then sometime later have a "distribution release" 14:15:53 or whatever we call it 14:15:56 lpabon: ah, ok 14:15:59 johnmark: right 14:16:23 hagarth: we should get the contributors to the candidate projects together and decide on a schedule 14:16:40 johnmark: sounds like a good idea 14:16:41 hagarth: which is something I've been meaning to do 14:16:54 johnmark: ok 14:16:58 #action jmw to convene contributors' meeting 14:17:15 Maybe we can coordinate what versions are available from other projects at the time of a GlusterFS release? 14:17:36 right now, the main candidates are gluster-swift, gluster-hadoop, pmux, and maybe one of gluster-puppet or gluster-deploy 14:17:46 lpabon: yeah, that would be nice to have. We often get questions like how to use project xyz with GlusterFS 3.x 14:17:59 +1 for lpabons idea too 14:18:08 lpabon: hmm... that's a good idea 14:18:17 since we are moving test day by a week, I think we need to move all dates by a week at least. 14:18:22 lpabon: I think we should get some people together next week to hash out a plan 14:18:28 hagarth: +1 14:18:31 hagarth: is glupy something someone is still going to hack on. looks 7 months old, but very interesting. 14:18:31 Maybe on the release page for a version of glusterfs we can provide a list of versions from other projects? 14:18:35 at least the beta and release dates 14:18:42 lpabon: +1 to that 14:18:42 johnmark: ok, we can do that 14:18:55 purpleidea: yes, glupy is very much hackable 14:18:57 lpabon: that works 14:19:13 purpleidea: jclift was workign on that, and he'll be back from vacation next week 14:19:14 hagarth: would love to see glupy releases along side gluster main releases. :) 14:19:27 purpleidea: but I don't know where his latest code lives 14:19:32 #action johnmark and lpabon and others to discuss multiple project release coordination 14:19:37 or if he committed it before he left for the month 14:19:40 purpleidea: yeah, that is a good idea 14:19:49 ok, anything else on 3.5 that we want to discuss atm? 14:19:51 purpleidea: yup 14:20:14 looks like we are done with 3.5 14:20:20 hagarth: yay :) 14:20:23 #topic community meeting schedule 14:20:29 so the big question 14:20:51 a) does this time work for a weekly meeting? b) should we have a poll to decide the meeting schedule? 14:21:01 rather big questions :) 14:21:05 hagarth: it works great for me :) 14:21:06 * avati is barely awake at this hour :p 14:21:07 this time is lousy for me 14:21:10 lol 14:21:17 its a little early for me as well 14:21:31 hagarth: maybe we can have the next one 12 hours later? and alternate between those two? 14:21:31 but i don't think i'm very important for this sort of thing. i can read logs 14:21:32 I agree that it is difficult for the PST folks 14:21:33 but i can do it if this works better for most people 14:21:35 or is that too confusing? 14:21:58 but when we spring forward, would 7 am be doable for PST folks? 14:22:07 johnmark, might get confusing :) 14:22:12 lalatenduM: true :) 14:22:28 johnmark: even I feel that it would be confusing 14:22:34 hagarth: and it will be 10am for me :) 14:22:39 hagarth: fair enough :) 14:22:57 avati, Technicool, gkleiman: are you folks fine with this schedule for the next 3 months or so? 14:22:58 what are all the various TZs we have people frome today? 14:23:12 Eastern 14:23:17 I'm in CEST 14:23:17 fine with me 14:23:21 IST 14:23:31 I figure India is the eastern most in the meeting right now 14:23:33 Eastern 14:23:36 yeah, that's quite a span 14:23:37 avati: EST, not an early bird 14:23:37 hagarth, yes this works ok short term, and long term if need be 14:23:50 hrm... and Paul's not here 14:23:54 gkleiman: ^^^ 14:24:08 johnmark: it would be well past midnight in New Zealand :) 14:24:14 oh! heh 14:24:19 * johnmark didn't know that 14:24:24 true, later would be better to have PaulC on 14:24:25 the farthest we can get east is Japan at this hour 14:24:31 hagarth: ok 14:24:33 it's 3.30AM for Paul 14:24:38 ouch. never mind 14:24:46 gkleiman: that might affect participation from India and the vicinity 14:24:53 let's face it, no matter what time we choose, it will always suck for somebody 14:25:03 johnmark: yeah 14:25:14 almost seems that we should have two meetings.. maybe centered in India and US time 14:25:16 lets make sure it affects someone we all dont like then...thats the only fair solution 14:25:24 shall we stick with this schedule and re-visit this after a month or so? 14:25:25 figure out who is critical and plan around them 14:25:30 Technicool: i vote for this idea ;) 14:25:34 LOL 14:26:00 gkleiman: problem is, critical people are stretched across pacific, eastern US and IST 14:26:18 or is there merit in moving the meeting by an hour to UTC 3 pm ? 14:26:24 Pacific isn't even the horror... Australia is. :/ 14:26:30 One meeting could be first, and the second could review the actions and notes from the previous and continue the discussions. Maybe PLs could then combine the minutes? 14:26:39 lpabon: that's a splendid idea 14:26:52 and perhaps the only workable solution 14:27:08 lpabon: yeah, that might be a possibility. 14:27:18 shall we reach there once we have more participation? 14:27:18 Would AM India, afternoon Pac, and evening Eastern time work better 14:27:42 this would allow you to include NZ 14:27:46 gkleiman: we will miss out ndevos and folks in EU then 14:27:54 I really cannot do evenings thats Wife time :-) 14:28:00 okay, here's my proposal: 14:28:10 1. we stick to this time for the next month 14:28:18 2. tune if necessary after that 14:28:38 +1 14:28:43 s/1.*/1. stick to this time+1hr for the next month/ ? 14:29:13 agree with avati 14:29:25 ok. time + 1 for the next month (3 PM UTC) 14:29:32 hagarth: got it 14:29:38 * msv agrees 14:29:40 I will send out invites for that 14:29:45 hagarth: thank you :) 14:29:46 works for me :) 14:29:50 ps - what about 3.4.2? 14:29:52 #action hagarth to send out revised invites. 14:29:55 hagarth: ^^^ 14:29:57 #topic 3.4.2 14:30:06 ok 14:30:10 patches are trickling in for 3.4.2 14:30:20 we need some more help with dht backports 14:30:31 there seem to be a few issues with rebalance in 3.4.1 14:30:34 i vote johnmark helps with the dht backport ;) 14:30:43 and all of them seem to have been addressed with mainline/3.5qa1 14:30:56 purpleidea: lol :P 14:31:04 purpleidea: +1 ;) 14:31:06 hagarth: so who can help backport that? 14:31:11 hagarth: ie. not me :) 14:31:20 hagarth: trust me, nobody wants that 14:31:32 johnmark: I will talk to the dht developers to see if we can have them soon 14:31:39 hagarth: ok, thank you 14:31:45 but I worry that we might have to do a lot of backports to 3.4.2 14:31:55 hagarth, do think I can help with the backports ? I mean , am I qualified enough? 14:32:12 lalatenduM: you certainly are 14:32:27 hagarth, awesome :), I will try to do some 14:32:29 lalatenduM: me and avati can offer assistance there 14:32:37 hagarth: is there a list of patches that need backporting? 14:33:10 ndevos: the crux is in identifying the list. I know that few members in the community have been actively bisecting to determine what all are necessary. 14:33:20 we already have enough backports to spin out a 3.4.2qa1 while the other patches come in 14:33:30 ok 14:33:37 ndevos: for example - https://bugzilla.redhat.com/show_bug.cgi?id=1033576 14:33:39 Bug 1033576: unspecified, high, ---, sgowda, NEW , rm: cannot remove Directory not empty on path that should be clean already 14:33:42 avati: agree 14:33:57 let us get 3.4.2qa1 out and continue working on backports 14:33:59 avati: then we should do that 14:34:09 but we need to set a deadline for more patch merges 14:34:16 * avati fires up a build in parallel 14:34:20 hagarth: can we create some processes around fixes committed to master? 14:34:42 hagarth: so that we're not continually trying to run down patches to be backported? 14:34:44 johnmark: what kind of process are you suggesting for fixes committed to master? 14:34:46 johnmark: yes, we need dedicated release maintainers who keep pollign master regularly 14:34:53 ah 14:35:03 it would seem that anyone who commits patches has a responsbility ot apply backports if it's an open bug 14:35:04 i see what you mean 14:35:15 if there's any interest for release-3.5, please mail me indicating your interest to do so 14:35:21 avati: otherwise, we always have this problem of trying ot contain the patch management beast 14:35:24 I was thinking of an rfc.sh enhancement asking a Y/N for a possible backport 14:35:37 avati: oh - that might help 14:35:39 avati: not all patches would be straightfoward backports 14:35:43 johnmark: Not all bug fixes back port... I'm more along with avati here. 14:35:44 hagarth: true 14:35:50 ira: ok 14:36:11 hagarth: it's to just tag the commit id for someone to scrub git log 14:36:27 in any case, if you need more patches to be included in 3.4.2, please add it here - http://www.gluster.org/community/documentation/index.php/Backport_Wishlist 14:36:34 avati: right. we need some way to identify whihc ones are candidates for backporting 14:36:40 Usually, the people committing the bug, have an interest in the backport, and open up the BZ for the backport. 14:36:45 hagarth: thanks. we need to circulate that 14:36:55 avati: sounds like a good idea 14:36:55 ira: right 14:37:11 the Linux kernel uses a stable@ email address to CC important bugfixes, I guess ./rfc.sh can do something similar indeed 14:37:28 ndevos: yep, that was the inspiration 14:37:30 shall we set a tentative date of Dec 10th for 3.4.2? 14:37:54 any volunteers for the rfc.sh enhancement? 14:38:08 hagarth: I'll do it 14:38:15 avati: thanks 14:38:38 #action avati to enhance rfc.sh to aggregate patches that need to be backported 14:38:55 any opinions on the 3.4.2 date? too early, late or seems right? 14:39:07 hagarth: sounds about right, with the usual caveats 14:39:20 * ndevos has no opinion 14:39:25 johnmark: thanks, let us target that 14:39:42 now to start spinning up the testing day things... 14:39:50 any other 3.4.2 related discussions? 14:40:26 i guess not 14:40:26 * lalatenduM has nothing discuss more 14:40:36 #open discussion 14:40:42 #topic open discussion 14:40:43 rather 14:40:56 hagarth: thank you for taking this on 14:41:12 i guess i have one question/comment or two 14:41:14 hagarth: also, dec. 10 is the date for GA of 3.4.2, right? 14:41:17 purpleidea: roll it 14:41:22 * purpleidea rolls 14:41:22 johnmark: right 14:41:59 * johnmark looks at his watch 14:42:06 purpleidea: go ahead 14:42:39 ok, some quick updates on happenings around the gluster world 14:42:46 obviously my interest is puppet-gluster; how important is this to gluster community. if so, i've asked before, but are there vm's i can get access to to test, or budget for some cheap iron i can buy? 14:43:04 * hagarth holds back 14:43:05 purpleidea: there are two possibilities 14:43:22 purpleidea: 1. we have a rackspace account that you could use to spin up some VMs, but that's on a smaller scale 14:43:50 purpleidea: 2. there's the OSUOSL, which might have the capability you're looking for. I can intro you to Lance there 14:44:16 i would need 2-5 vm's max for testing, occasionally a few more to try some fancy things. they can be cheap and small, i don't care about performance. 14:44:25 purpleidea: actually, there are others, but they're not yet ready, including a large data center here in Massachusetts that's looking to be a testing ground for things like that 14:44:34 joe tried to set something up with lance, but we haven't heard back since linuxcon 14:44:40 purpleidea: Please coordinate with PaulC (Cuzner) since he is pursuing similar work 14:44:40 purpleidea: oh, we can accomodate that 14:45:06 gkleiman: +1, purpleidea I can loop you in with Paul Cuzner 14:45:08 gkleiman: can you do an intro about this topic by email for me? 14:45:19 #action jmw to coordinate with purpleidea and Paul Cuzner re: testing facilities 14:45:20 hagarth: that would be great 14:45:48 ok, quick updates from gluster world 14:45:59 ganeti folks are looking to integrate with glusterfs - http://docs.ganeti.org/ganeti/master/html/design-glusterfs-ganeti-support.html 14:46:45 hagarth, yup 14:46:51 excuse my ignorance, but what is genati, in one sentence please? 14:47:22 ndevos: ganeti is a virtualization management stack 14:47:27 ndevos: think virt/cloud management, a la ovirt, cloudstack, et al 14:47:34 ok! 14:47:37 ndevos: but google 14:47:49 google created it for their own test labs a while back... 14:47:50 ndevos: and the OSUOSL has a rather large deployment with GlusterFS 14:48:11 archipelago project also has started integrating with glusterfs - https://code.grnet.gr/projects/archipelago 14:48:15 ndevos: hence OSL's interest in this by sponsoring a GSOC engineer to work on it 14:48:42 ndevos: would you want to provide a quick update on the cloudstack work that you were involved with last week? 14:49:13 ah, yes, last week there was a CloudStack hackathon in Amsterdam at the CloudStack Conference 14:49:36 I've discussed a little with some CloudStack devs, who explained how to integrate Gluster 14:50:10 is this for using gluster as a vm store? 14:50:14 at the moment, I can create a Primary Pool (for VM storage) in CloudStack - but I broke other bits in the process 14:50:28 yes, avati, but also as secondary pool 14:50:48 ok.. 14:50:58 guys - have to check out for a bit. Will check in later and read notes, IRC buffer 14:50:59 currently NFS and S3 are available as secondary pool, but the CloudStack people would like to see Gluster there too 14:51:30 source and start of the docs at: https://forge.gluster.org/cloudstack-gluster/pages/Home 14:51:42 there is also some interest to integrate with Xen (along the lines of qemu-kvm - libgfapi), if anybody is interested please let us know 14:52:12 ndevos: cool, seems like an interesting project. 14:52:14 does xen have a block layer plugin architecture? 14:53:08 avati: the same qemu - libgfapi integration should be consumabe by Xen but it requires some plumbing in libxl 14:53:15 and, of course anyone who likes to contru=ibute to the CloudStack integration (Java + JavaScript) or testing is welcome! 14:53:24 libxl is the equivalent/fork of libvirt that xen use 14:53:29 *xen uses 14:53:54 any other updates for today? 14:54:04 one more q 14:54:08 purpleidea: sure 14:54:31 is this the right venue to propose a technical feature (that i have some code for?) 14:54:31 was that a recent enhancement in qemu, to plumb the block drivers into xen? 14:54:52 purpleidea: sure 14:55:03 hagarth: okay, i actually have two. (very quick) 14:55:32 avati: i believe there is a mechanism, but we need to double check the exact details. 14:55:36 purpleidea: go ahead 14:55:52 1) i've been looking into how an admin automatically adds/removes nodes for large pools without having to compute the add/remove command ordering. would also be awesome if it worked for chained commands. as a result, i've got testcases and logic to try and do this. 14:56:15 it would also be awesome if there is a notion of a "standardized" brick naming scheme, and ordering. see: http://paste.fedoraproject.org/57200/56401813 14:56:23 for a very rough draft i'm hacking on now. 14:57:08 purpleidea: sounds like a good idea, i think this would make a nice discussion on gluster-devel 14:57:24 hagarth: okay 14:57:36 purpleidea: these are some topics which touch upon in the glusterfs 4.0 plan as well - FYI (i'm due to sending out the draft) 14:57:50 yay for 4.0! 14:57:51 hagarth: 2nd comment. i've been looking at gluster+reflinks. has anyone looked at this? i have a lot of ideas. 14:58:14 (would need a compatible brick fs like btrfs for example) 14:58:18 reflinks? 14:58:31 purpleidea: i've been following reflink dev.. i don't think the syscall interface is finalized yet? 14:58:37 avati: it is 14:58:40 purpleidea: not that i am aware of. The btrfs integration with 3.5 will make it more visible. 14:58:46 oh cool 14:58:48 avati: In fact samba is using it. :) 14:58:57 in fact, i've written a draft write up about why this is THE killer feature. comments welcome: http://ttboj.wordpress.com/?p=482&shareadraft=5296088f6f9cc 14:59:03 (actually, comments would be appreciated) 14:59:06 ira: is it using the btrfs specific ioctl() or is there a generic reflink syscall now? 14:59:08 avati: i could be wrong though 14:59:27 avati: I can check. I know they've been plumbing like mad. 15:00:00 purpleidea: nice, will have some comments 15:00:02 ira: avati: i remember reading through all the mailing list patches. it looked finalized. cp --reflink exists 15:00:23 ok, that brings us to the top of the hour. 15:00:25 hagarth: would very much appreciate knowing how feasible this is. if it is, i guarantee this is the #1 feature any sysadmin would want. 15:00:33 purpleidea: sure 15:00:40 hagarth: appreciate it, thanks 15:00:44 purpleidea: i was thinking of introducing a reflink-like FOP which would be the interface for making clones. that will by deafult call into the backend's reflink mechnaism, or get intercepted by BD for its cloning or qemu-block for its external snapshots 15:00:44 shall we end the meeting and convene again next week? 15:00:59 ira: cp --reflink makes btrfs specific ioctl(), not a generic syscall 15:01:00 yup 15:01:03 thanks everybody for joining in 15:01:06 #endmeeting