12:01:09 #startmeeting 12:01:09 Meeting started Wed May 6 12:01:09 2015 UTC. The chair is hagarth. Information about MeetBot at http://wiki.debian.org/MeetBot. 12:01:09 Useful Commands: #action #agreed #halp #info #idea #link #topic. 12:01:18 Let's get rolling. :) 12:01:22 #topic Roll Call 12:01:26 * jimjag is here 12:01:28 who do we have here today? 12:01:31 * spot is here 12:02:00 is everybody else too busy with 3.7.0? :) 12:02:03 * ndevos is here _o/ 12:02:08 here 12:02:31 looks like that is the case .. let us get started nevertheless 12:02:34 * JustinCl1ft waves 12:02:43 #link agenda - https://public.pad.fsfe.org/p/gluster-community-meetings 12:02:45 * overclk is here 12:02:45 * soumya is here 12:02:46 * krishnan_p _o/ 12:02:59 #topic Action items from last meeting 12:03:01 hep 12:03:01 * raghu is here 12:03:02 * kkeithley1 is here 12:03:14 raghu to create new 3.6.4 tracker bug today 12:03:17 raghu: is this done? 12:03:22 yeah. Donew 12:03:38 https://bugzilla.redhat.com/show_bug.cgi?id=1216965 12:03:47 raghu: cool, you can also set an alias for that. 12:03:59 sure 12:04:05 next AI - raghu to release 3.6.4beta1 by the end of next week 12:04:20 raghu: on track? 12:04:33 Yeah. Its on track. 12:04:52 raghu: great, thanks. 12:04:59 next AI - ndevos to do release announcement for 3.5.4beta1 when the packages are ready 12:05:08 I would have done that already. But there was a crash for which a patch has been backported by jdarcy. I am waiting for that to get merged 12:05:08 ndevos: ready with this? 12:05:13 raghu: ok 12:05:34 oh, I think the packages have been tested, but hchiramm or lala was going to get back to me about the 12:05:36 m 12:05:47 hchiramm: any updates here? 12:05:55 maybe I've missed their email, or I'm waiting for one 12:06:06 hagarth, yes 12:06:24 ndevos, hagarth the packages are available at download.g.org 12:06:36 hchiramm: cool! 12:06:48 ndevos: maybe we should just send out an announce note then? 12:06:58 hagarth: yes, I'll get that done 12:07:09 ndevos: great, thanks. 12:07:22 next AI - overclk will review the Overlay feature page by next meeting or Gluster Summit 12:07:34 overclk: sounds like next week - is that right? 12:07:53 yeh 12:08:00 overclk: ok 12:08:03 next AI - JustinClift to have basic forge v2 operational by Gluster Summit 12:08:06 during summit :) 12:08:11 overclk: ok :) 12:08:18 hagarth, not only review but will talk about it :) 12:08:26 overclk: that would be fantastic! 12:08:58 JustinCl1ft: any updates here? 12:09:01 hagarth: Working on it ;) 12:09:17 Some repos have been mirrored to GitHub, and stats collector updated 12:09:18 JustinCl1ft: cool, is there a drop dead date for us to move off forge? 12:09:20 But, that's about it. 12:09:27 No, there isn't. :) 12:09:48 JustinCl1ft: maybe we should have a deadline for this :) 12:09:55 Meh 12:10:07 * tigert is in 12:10:17 one of the issues that kshlm noticed was gerrit's async mirroring to both github & forge was causing load spikes 12:10:23 but more on that in the infra part of the meeting 12:10:31 next AI - JustinClift to prod the alternate weekly meeting times discussion back to life 12:10:31 Interesting 12:10:36 Haven't done it 12:10:43 Anyone else want to pick it up? 12:10:46 overclk ? 12:11:01 JustinCl1ft: ok, carrying forward this AI 12:11:19 JustinCl1ft: maybe have a discussion on this next week at BCN? 12:11:28 BCN ? 12:11:28 * kshlm is late 12:11:33 Barcelona 12:11:55 * JustinCl1ft nods 12:12:14 #action JustinCl1ft to hold a discussion on alternate weekly meeting times in Barcelona 12:12:18 JustinCl1ft, I'll take a pass on that. anyway we'll discuss in BCN.. 12:12:22 next AI - JustinClift to reach out to likely testers when 3.7.0beta1 RPMs are available 12:12:36 JustinCl1ft: any luck here? 12:12:43 Haven't done that. The main tester I was goingt o reach out to had issued with 3.6.3 upgrade 12:12:49 (yesterday) 12:12:55 So, I didn't want to push it ;) 12:13:03 JustinCl1ft: yes, looks like we will not get much testing pre 3.7.0 GA 12:13:15 ??? 12:13:30 Why the hard cut-off-date for GA, instead of doing it after testing? 12:13:33 JustinCl1ft: much (non-developer) testing before GA 12:13:41 What happened to "when it's ready" ? 12:13:56 JustinCl1ft: that has not been our strategy 12:14:01 perf is testing it, Ben Turner is testing it 12:14:14 hagarth: I know. See the state of the regression tests? That's the result 12:14:15 we decided to move to a time based schedule, rather than a content based one a few releases back. 12:14:32 Can we change this? Please. 12:14:44 JustinCl1ft: the regression tests are a lot better atm. 12:14:53 JustinCl1ft: another sound topic for Barcelona/post 3.7.0. 12:15:01 * JustinCl1ft grumbles 12:15:14 timebased is good, features that are not ready do not delay other features, and will have to wait for the next release :) 12:15:17 JustinCl1ft: we need to figure out how we can get more community testing before a release 12:15:25 Btw "we decided to move to a time based schedule, rather than a content based one a few releases back." 12:15:41 ^^^ We didn't communicate that to our Community did we? 12:15:43 We should move to merge window based strategy in the future (4.0?), hopefully improving our quality ahead of release. 12:15:48 JustinCl1ft: we did that 12:15:53 do regression tests do anything that would stress Gluster in the slightest? 12:15:59 JustinCl1ft: I think we discussed that mainly in this meeting? 12:16:09 ndevos: Hmmmm. 12:16:17 JustinCl1ft: I can look up the archives but maybe for later. 12:16:20 k 12:16:42 bene2: regression tests are not very intensive as far as I can see. 12:17:20 bene2: however regression tests open up interesting latent races which we have been fixing in the recent past. 12:17:26 so it aide the cause of stability. 12:17:33 bene2: They're not supposed to be *stress* tests. They exercise a wide range of *functionality* but not all at once. 12:17:41 next AI - jdarcy will post a GlusterFS 4.0 update to the list(s) 12:17:52 I guess I should do that, huh? 12:18:01 jdarcy: +1 :) 12:18:23 next AI - tigert to announce new Gluster website layout demo to the mailing lists 12:19:04 did that, not much response though 12:19:09 tigert: right 12:19:10 though we lack content most 12:19:25 tigert: we can discuss more in the website section of the meeting 12:19:35 tigert: what about annoucing planet.gluster.org too? 12:19:46 let me take that as an action item 12:19:52 and include the rss urls for people that like to add it to their readers? 12:20:10 yeah 12:20:16 jdarcy, I'm not talking wide range of functionality. Just more than 1 thread, 1 client or 1 server at a time, simple tests. 12:20:33 #action tigert will announce planet.gluster.org and include the rss urls too 12:20:37 I need to dig it up, I don't do much rss myself (though won't the readers be smart and figure those out themselves?) 12:20:39 bene2: Patches welcome. 12:21:10 http://glusternew-tigert.rhcloud.com/ has our documentation linked now btw 12:21:12 bene2: a few tests run on multiple servers .. in addition we are looking to overhaul our upstream testing to have more categories of tests. 12:21:24 bene2: Some of the tests start up several nodes on the same host for testing interaction stuff. It's for bug testing more than anything though. 12:21:29 tigert: I dont know, maybe checking if there is an rss feed it enough 12:21:38 next AI - spot to bring up the Branding? question for it in response 12:21:40 ndevos: yeah I think it has, I'll check 12:21:50 spot: I believe this is done? 12:22:05 hagarth: should be, i'll double check to make sure. 12:22:12 * spot has been juggling a lot lately 12:22:18 spot: sure 12:22:33 btw I like software {re}defined storage too :) 12:22:43 bene2: There's also a question of how much we want to slow down our *pre commit* check. Obviously we should do real stress/longevity tests before a *release*, but before every commit? 12:22:57 agreed, don't want to do this every commit 12:23:38 jdarcy: our nightlies do cover multiple servers and clients .. but we don't have coverage for all possibilities with gluster today. 12:23:38 but better to do it upstream in an automated fashion than downstream when it's too late 12:24:07 that ends our list of AIs from the previous meeting 12:24:15 moving on 12:24:17 bene2, jdarcy: it would be amazing if we have an easy to run automated test, we can schedule that in Jenkins and include qemu/samba/ganesha things there too 12:24:18 bene2: Absolutely. We should define that pre-release testing process, preferably (IMO) in the form of a script. 12:24:28 #topic Gluster 3.7 12:24:30 jdarcy: +1 12:24:48 Sounds like we need a "testing" topic for the open floor 12:24:50 3.7.0 is all set to be released tomorrow 12:24:56 JustinCl1ft: please add that 12:25:08 hchiramm, raghu et al have been helping with release notes 12:25:09 yes, +1 for testing during open floor 12:25:13 JustinCl1ft: Sorry for helping to sidetrack. 12:25:14 Done 12:25:18 hagarth: no beta2 for added patches? 12:25:32 hagarth: tiering got couple of patches to backport .. we are doing it ... will send a list of them later for merging ;) 12:25:32 don't derail the juggernaut. ;-) 12:25:33 ndevos: no, since we are not getting any feedback 12:25:48 Joe_f: sure 12:25:54 hagarth: well, we got some for packaging issues... 12:26:12 ndevos: yes, let us discuss that in the openfloor? 12:26:17 hagarth: Have we killed all of the regression failures? 12:26:23 Like, we have 0 now? 12:26:37 Not even close. 12:26:47 We agreed to do that. 12:26:48 JustinCl1ft: the regular offenders have been killed .. lot of regression tests are passing now 12:27:07 We agreed to fix themm all. Not just "some of them". 12:27:12 JustinCl1ft: yes, but infra issues are not helping 12:27:14 question on 3.7.0. Is the GA planned on 7th or 8th? 12:27:24 That's not an excuse 12:27:25 is there some sort of report that summarizes? 12:27:29 Some of the worst offenders are still being skipped in run-tests.sh, and there's a whole rogues' gallery of others that fail less often individually but quite a bit as a group. 12:27:30 hagarth: well, I think there are still some patches that need to get in before 3.7.0 GA... 12:27:43 JustinCl1ft: no, but we can't keep moving the release forever. that's the point I am trying to make. 12:27:57 ndevos: certainly 12:28:00 No-one is suggesting we move it forever 12:28:12 I'm saying we stick with what we said we'd do 12:28:32 hagarth: should we not cleanup the tracker before deciding when to GA? 12:28:45 https://bugzilla.redhat.com/showdependencytree.cgi?maxdepth=2&hide_resolved=1&id=glusterfs-3.7.0 lists *many* bugs still in POST :-/ 12:28:45 JustinCl1ft: sure, I don't have a problem with that. 12:28:53 k. 12:29:00 ndevos: yes, unfortunately it is beyond my limited bz skills :-/ 12:29:12 and I really need help from bz owners to clean them up. 12:29:27 hagarth: yeah, I understand its not easy to do... but at the moment we have no idea whats pending :-/ 12:29:52 Where's the etherpad with what's pending? 12:29:57 ndevos: I will send one more reminder on the list about that .. there are quite a few assigned to bugs@gluster.org 12:30:19 Or is the etherpad really a fair representation? 12:30:21 kkeithley: the URL that I sent in a note to gluster-devel 2 days back has it 12:30:24 hagarth: yes, an other reminder would help 12:30:41 kkeithley: https://goo.gl/j8PzVk (new & assigned list) 12:30:41 kkeithley: we have bugzilla for tracking the status of bugs, dont etherpads get oiut of sync? 12:30:42 * kkeithley was hoping someone had the link handy 12:30:49 we can similarly evolve a POST list 12:31:12 kkeithley, https://public.pad.fsfe.org/p/pending_gluster_3.7_patches 12:31:15 yeah, nothing is perfect 12:31:21 so here's my take on regression tests: 12:31:42 1. we clean up whatever is affecting us before 3.7.0 GA 12:32:09 2. if a test is listed in the spurious failures etherpad but doesn't happen anymore easily, we don't hold GA for such tests. 12:32:16 does this seem reasonable? 12:32:51 what happens to 3.7.0 tracker bugs? 12:32:57 soumya: at this point I am also inclined to push out patches to 3.7.1 if they are not absolutely essential for 3.7.0 12:33:11 hagarth: once a test failed, other tests are not run, we should not 'overrule' those failed regressions 12:33:18 krishnan_p: move all non-URGENT bugs to 3.7.1 after one more round of cleanup by individual developers 12:33:30 ndevos: noted, haven't been doing that this week. 12:33:39 ok :) 12:33:40 hagarth, sounds reasonable. In fact, bugs that are left unattended should be moved out in bulk 12:33:53 hagarth: is there a 3.7.1 tracker already? 12:33:56 krishnan_p: right 12:34:01 ndevos: will need to create one 12:34:02 Why does the 3.7 tracker depend *directly* on bugs on master? 12:34:19 all bugs left unattended? That seems like the wrong criterion 12:34:29 jdarcy: probably from before the branching of 3.7 from master 12:34:40 s/probably/HOPEFULLY/ 12:34:40 jdarcy: we also do not have a 3.7pre tag 12:34:41 hagarth, where do we specify the patches which are must for 3.7.0 (example rpc-protocol changes)? 12:35:04 soumya: please add them in the etherpad :) 12:35:17 soumya: they should be blockers of the 3.7.0 tracker 12:35:28 bene2, hmm. What would be the alternative? 12:35:39 hagarth, ndevos ..thanks :) already done..i thought they may get pushed out even then to 3.7.1 12:35:40 bene2: we are trying to get better at bug triaging .. not something that we can fix in a short order. 12:36:12 soumya: add a MUST tag to those patches which you absolutely want to see in 3.7.0 12:36:20 hagarth, sure 12:36:42 want to touch a bit upon release maintenance of 3.7.x 12:36:57 krishnan_p, a bug can be fatal but unattended, that's my point. The solution here is to prioritize and attend to the bugs that matter most, such as data integrity. 12:37:00 we've had a single release maintainer for our previous release trains 12:37:33 since 3.7.x is a feature packed release train, I propose that all maintainers manage 3.7.x too. 12:37:35 bene2, perfect. Its the maintainer/developer responsibilty. Not release maintainer's 12:37:55 sounds good to me 12:38:22 hagarth, hmm. Are we dividing all the responsibilities of a release maintainer to all component maintainers? 12:38:26 krishnan_p: in my opinion the release maintainer should keep an eye on the open bugs for the release he/she maintains 12:38:32 krishnan_p: yes 12:39:00 hagarth, It make sense to divide the patch maintainence. But GA coordination and decision making? Too many cooks? 12:39:03 krishnan_p: that always has been the case - if you look up the responsibilites for maintainers 12:39:10 release maintainer can alway revert changes he or she doesn't approve of 12:39:22 hagarth, I am more than happy to chip in. I don't want a dining philosophers problem :) 12:39:29 if the component maintainer oversteps his or her bounds 12:39:57 krishnan_p: we will announce a schedule and get an ack from all maintainers as per the schedule (i.e. before we release). 12:39:59 oh, but currently component maintainers are not supposed to merge the changes into release-* branches, only review+approve them 12:40:57 for 3.7.x, component maintainers will be merging changes into release-3.7. 12:40:59 * kkeithley thinks, hence this discssion 12:41:06 hagarth, by that the 'we' is responsible for coordinating. Isn't that what current release maintainers do, over and above patch maintenance 12:41:10 *discussion* 12:41:13 is it really much work for a release maintainer to click the "submit" button on completely reviewed/tested patches? 12:41:46 ndevos: I foresee the volume of patches to be a lot more for release-3.7 12:42:17 * krishnan_p wonders if we should try merge windows approach for 3.7.x. 12:42:23 hagarth: I do not see a bottleneck in the release maintainer if component maintainers review and +2 patches in a timely matter 12:42:54 ndevos: I have felt that in the run upto 3.7.0 itself .. hence this change. 12:43:11 krishnan_p: yes, it almost boils down to announcing a schedule for all 3.7.x releases 12:43:27 and getting all patches into release-3.7 during the merge window 12:43:39 anything beyond that gets moved to the next 3.7.x update. 12:43:41 hagarth: yes, I understand, but I think many component maintainers could be more responsove, and review with +2 instead of +1 12:43:50 *responsive even 12:44:01 ndevos: agree there. 12:44:24 ndevos: why don't we try this for release-3.7 and see how this fares? we can always do a course correction if necessary. 12:44:26 yeah, I don't understand this propensity to give multiple +1s 12:44:28 hagarth, I will seek more clarification in the mail that you are plannng to send 12:44:48 hagarth: sure, lets try it out 12:44:50 hagarth, my biggest concern is coordination. Probably a detailed plan for your proposal would shed more light 12:45:06 kkeithley: well, multiple +1's would be for a patch that hits different components 12:45:13 krishnan_p: I may not be able to put everything in black & white, but I'll try :). 12:45:29 or, someone that is not a component maintainer should not +2 it, maybe 12:45:32 hagarth, It could evolve in the ML 12:45:45 krishnan_p: I will help with larger co-ordination needs. 12:45:49 ndevos: that would be reasonable, but I see it a lot on patches that only touch a single component. 12:45:50 krishnan_p: sure, let us do that. 12:46:00 kkeithley, ndevos all the +1/+2 behaviour is mostly cultural 12:46:09 Nagaprasad: release date contingent on cleaning up our regression test sanity :) 12:46:18 anything more on 3.7.x ? 12:46:41 thanks 12:46:41 krishnan_p: understood. But it's something we should work on. ;-) 12:46:56 kkeithley, I think so too. Old habits die hard :) 12:47:01 figure not, moving on 12:47:06 #topic Gluster 3.6 12:47:06 krishnan_p: sure, and there is a difference between +1 and +2, but component maintainers should be happy to +2 some changes :) 12:47:43 ndevos, agree. 12:47:44 raghu: updates here? 12:47:48 Not much on release-3.6. I have merged some patches. Found that there was a crash. jdarcy has backported the patch. Its going through regressions. Once it passes, I will make a beta1 of 3.6.4 12:48:36 raghu: I will also send the CLI backport (in response to the issue reported by corvidtech yesterday). 12:49:03 bye 12:49:04 hagarth: sure. Will wait for it. 12:49:11 #action raghu to release 3.6.4beta1 this week 12:49:18 moving on to 3.5 12:49:22 #topic Gluster 3.5 12:49:40 I'll announce 3.5.4beta1 later today 12:49:49 ndevos: cool 12:50:05 there are some changes that I would like to include as well, so I'll do a beta2 later this week 12:50:16 ndevos: right, ok 12:50:18 or early next week, depending on how busy I am with other tasks 12:50:48 * ndevos does not have anyting else to sat 12:50:50 *say 12:50:51 ndevos: sounds good to me, we can probably do beta2 in Barcelona :) 12:51:01 #topic Gluster 4.0 12:51:08 Jeff - any updates here? 12:52:09 There was a very productive meeting in BLR last week. 12:52:33 Losts of ideas about 4.0 features, which we'll surely be discussing in Barcelona next week. 12:52:46 People seem pretty excited to get started. 12:53:08 jdarcy: yay! 12:53:27 jdarcy, could you share the notes you took down? 12:53:49 kshlm: Sure. I'll send to the ML. 12:53:57 I missed the meeting last week, and would like to read up before BCN. 12:54:03 Thanks. 12:54:14 anything more on 4.0? 12:54:21 Nope. 12:54:35 ok, thanks. Moving on 12:54:42 #topic Open Floor 12:54:54 first one - Infra issues (gerrit etc.) 12:55:08 gerrit has been a laggard .. partly it could be related to iWeb 12:55:20 JustinCl1ft: can we raise a ticket with iWeb about packet losses? 12:55:56 kshlm: any thoughts that you would like to share from your debugging in r.g.o? 12:56:26 There is a problem with the network in their datacenter itself. 12:56:37 Lots of packet loss within their network. 12:56:38 I've seen one regression test that passed, but the ssh command to set +1Verified had a connection timeout. Related? Am I the only one that's seen this? 12:56:51 kkeithley: nope, I have seen those too. 12:56:51 hagarth: I'll try. 12:57:19 hagarth: Last time we had problems they couldn't authenticate me. But they could see the firewall was having issues, so they rebooted it anyway 12:57:19 JustinCl1ft: thanks, kshlm sent a note on gluster-infra about packet loss yesterday. 12:57:34 BTW are we planning test day/week this time? Before GA? 12:57:37 JustinCl1ft: I see .. maybe Johnmark can help too. 12:57:39 hagarth: ^^ 12:58:03 msvbhat: no, since we did not have a lot of traction previously, we decided to not have one this time around. 12:58:34 hagarth: Ah 12:58:41 kshlm: Just to ask.. are you measuring that from multiple source locations? 12:58:57 kshlm: Just in case it's something to do with an interaction from your source location 12:59:29 I did. 12:59:34 Hmmm.... although we have backups for Gerrit happening nightly, I don't think we ever for the Jenkins nightly backup happening 12:59:48 Might be a good idea to get that happening pretty pronto. Just in case. 12:59:50 Tested from newgerrit, RH-bengaluru, and my home network 12:59:54 kshlm: Cool 13:00:03 I also pinged build.g.o and r.g.o 13:00:23 from each other, lots of packet loss even though they're in the same DC. 13:00:33 hagarth: We can setup Gerrit in Rackspace fairly easily 13:00:35 JustinCl1ft: since the number of patches being sent and users are also more at this time, it seems to cause problems too. 13:00:47 JustinCl1ft: we probably need to move r.g.o to a more beefy server. 13:01:06 We have new server gear... it has no public ips though 13:01:16 So, it's pretty much dead in the water until misc gets back 13:01:30 JustinCl1ft: let us discuss this over in BCN too .. far too many things already for there :) 13:01:32 The newer gerrit site seems to keep a lot more connections open to the server. The gerrit server isn't able to repond to all of the requests quickly. 13:01:42 we probably need more days for the summit :D 13:01:43 The network issues on top don't help. 13:01:54 k 13:01:57 Yeah 13:02:07 next topic - Doc update 13:02:17 hchiramm: want to provide a quick update here? 13:02:34 hagarth, yep 13:02:57 the documentation is getting rendered in readthedocs 13:03:01 as u can see here 13:03:20 http://gluster.readthedocs.org/en/latest/ 13:03:28 the plan is to keep only developer docs inside 13:03:33 gluster source 13:03:43 and move everything to a seperate github repository 13:03:49 for easy contribution 13:03:57 hchiramm: Markdown format or RST? 13:03:59 yeah, sounds like a good approach 13:04:01 I will send out an email soon about the proposed changes 13:04:07 aravindavk, markdown 13:04:18 hchiramm: thanks, sounds good to me! 13:04:33 next topic - Summit agenda 13:04:41 spot: anything about $TOPIC? 13:04:45 Summit agenda is up on the wiki 13:04:57 http://www.gluster.org/community/documentation/index.php/GlusterSummit2015 13:05:27 spot: cool, thanks! 13:05:44 All of the speakers have been notified as to their sessions and should be prepared to present! 13:05:50 spot: will there be any room for smaller BoF like discussions? 13:05:59 spot: I don't have any slides ready yet ;) 13:06:02 We have a second room for that 13:06:07 spot: ok, cool. 13:06:43 moving on.. 13:06:57 next topic - https://glusternew-tigert.rhcloud.com 13:07:15 tigert: would you like to provide an update here? 13:07:59 tigert seems to be MIA .. moving on 13:08:14 next topic - Testing 13:08:34 JustinCl1ft, kkeithley: thoughts here? 13:08:38 do we really need to test? 13:08:46 how about switching the focus from regression to unit tests? 13:08:57 kkeithley, +1 13:09:05 oh, YES, cmocka tests should be required 13:09:06 kkeithley: define unit tests ? 13:09:14 cmocka-based tests 13:09:28 I wouldn't want to miss regression tests even if we add unit tests. 13:09:44 no, just a focus shift, not a complete sea change 13:09:50 indeed, regression tests are a must, and unittests should be a must too 13:10:05 ndevos, kkeithley: I am fine with that 13:10:05 Could we atleast reduce the set of regression tests run for every change submitted to gerrit? 13:10:21 add a new function, add a unittest 13:10:32 kshlm: if we can determine what are a good subset of tests to be run, we can do that. 13:10:39 Stop 13:10:45 * hagarth stops 13:10:51 kshlm: we could run more in ... 13:10:56 kshlm: I think you're meaning "can we make regression tests run faster?" 13:11:03 Running the full test suite and waiting for a minimum of 2 hours to get the results for every change is just a waste of time IMO. 13:11:11 That's really what you're after yeah? 13:11:28 kshlm: add a non-durable mode to glusterd and we probably should get that back in 60 minutes ;) 13:11:31 kshlm: you do not need to watch the console log for the test, you may switch to an other task ;-) 13:11:45 Guys, hold on 13:11:59 kshlm is right, in that the wait time affects our iteration speed 13:12:08 If we can get that sped up, we iterate faster 13:12:12 However... 13:12:18 JustinCl1ft, thanks for getting the iterate word. 13:12:28 That's what I was looking for. 13:12:43 We might be able to do the full regressions tests "differently" and still get the same coverage... faster 13:12:44 sure, we already started to move tests into seperate directories, that enables use to run tests in parallel - only needs the Jenkins support 13:12:53 Yep 13:12:57 That's the idea 13:13:20 In the same kind of way we have the "multiple things voting", we could do "multiple parts voting" 13:13:37 In theory, we might be able to get the whole thing done in under 20 mins 13:13:43 indeed, I would love to see a "passed the NFS tests" somewhere :) 13:13:52 ndevos: indeed! 13:14:04 So, I don't think cutting out test is the right approach 13:14:15 JustinCl1ft: I think we also need more scale-out on demand 13:14:16 As that reduces our coverage of finding edge case bugs 13:14:23 hagarth: And yeah. We need that too. 13:14:23 Well, we do not need to run every test for every change. 13:14:37 What use is running io-tests for glusterd only changes? 13:14:54 I would also like to discuss a broader testing strategy soon. 13:14:54 kshlm: Finding unexpected bugs 13:15:03 kshlm: but how would the test framework know when not to run certain tests? 13:15:26 As long as neither the number of tests nor the number of machines changes, the number of complete tests per hour won't either. We'll still be backlogged during "rush hour" every day. 13:15:40