15:01:48 #startmeeting Community Meeting 2017-02-15 15:01:48 Meeting started Wed Feb 15 15:01:48 2017 UTC. The chair is kshlm. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:48 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:48 The meeting name has been set to 'community_meeting_2017-02-15' 15:02:10 I'll wait for 3 more minutes for people to show up. 15:03:51 oh, is this the roll call topic already> 15:03:53 ? 15:04:03 * ndevos <- is present 15:04:15 In the meantime, add anything you want to discuss to https://bit.ly/gluster-community-meetings 15:04:48 ndevos, I'd not called for it yet :) 15:05:36 * shyam is here 15:05:42 * amye is here 15:06:04 I'm vaguely here. 15:06:06 Okay, roll call time 15:06:09 #topic Roll call 15:06:29 hi, I'm here! 15:06:31 o~ 15:06:44 Welcome to todays meeting everyone, please chime in if you're participating today. 15:07:02 Hi ndevos and nigelb! 15:07:10 hello kshlm _o/ 15:07:13 And shyam and amye! 15:07:28 howdy 15:07:29 * shyam is here (ok just did that again ;) ) 15:08:09 Okay let's start 15:08:19 We have 2 topics for discussion. 15:08:45 #topic GlusterFS-3.10 reminder - last week before release 15:08:50 sweet! 15:09:02 shyam, You're up. 15:09:10 Ah ok... 15:09:20 So some general reminders (mails going out as well) 15:09:41 Please test and report testing feedback on the created github issues 15:10:10 Please mark any bugs that should block 3.10 against its tracker (https://bugzilla.redhat.com/show_bug.cgi?id=1416031 ) 15:10:25 ndevos has been good at reminding people about this, thanks! 15:10:59 Please review the release notes and make any chahnges to them (whoever has features delivered this release) 15:11:12 shyam, How has the testing feedback been till now? 15:11:20 shyam: do you know of a github-url that shows all the issues where I am assigned? 15:11:28 I have some from bitrot and glusterd 15:11:40 ndevos: give me a few minutes and I will get that for you 15:12:01 I am doing our some perf tests and upgrade tests, results out this weekend 15:12:13 Other than that, nothing more ATM 15:12:22 ndevos, https://github.com/gluster/glusterfs/issues/assigned/nixpanic 15:12:34 shyam: thanks, I'd like to check and probably re-assign some of them to others :) 15:13:02 shyam, Cool. Do you require any more help? 15:13:07 RC1 will be tagged when we have the current blockers fixed (hopefully this weekend again), reminder mail going out on pending issues today! 15:13:53 Other things look good, fstat is quiet about 3.10, git cherry reports for backports in 3.9/8 to 3.10 looks healthy (finishing up last minute weekly checks on that) 15:14:15 Nothing more for 3.10 kshlm TEST! TEST! TEST! is all that is needed :) 15:14:41 shyam, want me to start firing things out on the social medias about this too? 15:14:47 shyam, I'll do my bit for GlusterD. 15:14:55 ndevos: Link to testing issues: http://bit.ly/2kDCR8M (need to add the assigned filter to that) 15:15:14 3.10 has to be our most methodical and documented releases ever. 15:15:28 amye: yes we need to polish the messaging based on the release notes and start the media stuff as well 15:15:37 shyam++ for doing all this 15:15:38 kshlm: Karma for shyam changed to 1 (for the f25 release cycle): https://badges.fedoraproject.org/tags/cookie/any 15:15:57 shyam, yep. Let us connect offline on that, but if we're at TEST TEST TEST all the things, then it's time. 15:16:10 * kshlm wishes we could track karma per gluster release like zodbot just did for fedora 15:16:11 yes, agree 15:16:29 Oops, yes agree amye 15:16:32 np 15:16:36 shyam: btw, how do we give feedback? Should we add a label like 'VERIFIED' or something? 15:16:59 Add comments to the issue, on what was tested etc. and then close the issue if you are done with testing the same 15:17:20 ok, just closing works too 15:17:31 And about blockers? 15:17:43 Here is a good example: https://github.com/gluster/glusterfs/issues/90 15:17:48 blockers are bugs right, not github issues 15:18:03 yes, blockers are bugs, tracked against the BZ tracker 15:18:17 (for this release, and hopefully for the last time ;) ) 15:18:19 Cool! 15:18:55 (yes, we'll need to have the tools and scripts updated to work with github too) 15:18:56 Any more github issues queries, would be good to address those, so that there is little confusion there... 15:20:22 ok I am done with 3.10 related call to actions... thanks 15:20:22 yes, add those to the bug triage guidelines/process doc - uhm, and probably revisit the whole process for github? 15:20:43 shyam, Thanks. 15:20:45 Moving on. 15:20:55 #topic GlusterFS-3.8.9 packaging problems 15:21:05 ndevos, I guess this was you. 15:21:09 yeah :) 15:21:30 kkeithle is on holidays and has instructed multiple volunteers for packaging on several occasions 15:21:47 unfortunately, nobody picks up the packaging work when he is out 15:22:21 3.8.9 was tagged on monday, and I've done the CentOS Storage SIG builds, others are still not there yet 15:22:37 Is the process documented somewhere? If so someone could give it a try. 15:22:51 (if they can get the right access to the builders) 15:22:58 I think so (no idea where), but you also need the right permissions 15:23:30 nigelb, You've spoken to kkeithle about this haven't you. 15:23:32 I can do the Fedora builds part of the distribution, and will probably do them tomorrow 15:23:41 Do you any suggestions? 15:23:43 teh others for download.gluster.org need to be done by others 15:24:12 I have. I don't have any suggestions at the moment. 15:24:28 Just saying: we need this automated, I faced similar when RC0 for 3.10 was released 15:24:36 I have ideas, but they're not in a stage where it will be useful for the current problem. 15:24:42 nigelb, Do you know of any documentation for this? 15:24:49 That will be helpful. 15:25:17 unfortunately not everything can be automated, most distributions require a person to do the actual commits to their builders 15:25:23 Also, you could probably provide the permissions required for the volunteer. 15:25:45 Nope, sorry. 15:26:05 The permissions are not at our level. 15:26:08 They're at the distro level. 15:26:12 all volunteers should be subscribed to the packaging@gluster.org list, but there are also others subscribed... 15:26:50 And the gpg signing key for gluster, is rightfully private (even I don't actually have access per se). 15:26:52 For download.gluster.org? 15:27:12 Ah, download.gluster.org, I'm happy to coordinate access. The problem is signing key. 15:27:18 The distro level stuff that kkeithle did was for fedora, ubuntu ppas and opensuse build service. 15:27:42 You know what, I'll happily take an action item to document the process. 15:27:51 I want to automate this, so it's useful for me to document this anyway. 15:28:06 nigelb, as you wish :) 15:28:37 #action nigelb will document kkeithleys build process for glusterfs packages 15:28:48 that sounds like a very useful step :) 15:28:51 That works. It doesn't sound like that's a transparent process these days. 15:29:07 Part of it needs to stay private. Part of it will be public. 15:29:18 but, I wont wait for it before announcing glusterfs-3.8.9, and users of non CentOS distributions will have to wait a little longer 15:29:21 Now that we've established we're basically useless without kkeithle around to build packages, what do we do with 3.8.9? 15:29:36 * amye facepalms 15:30:04 When's kkeithle back? 15:30:21 Coming Monday (if I remember right) 15:30:51 ^ 15:30:59 I suggest waiting. 15:31:00 What I'm hearing is that we're blocked until Monday on this, we should document that this is a blocker and we should work to assign responsibility for 'backup packager person'. 15:31:01 and by then there shouls also a 3.9 update :) 15:31:13 it's only a few days. 15:31:27 But also +1 to backup packaging person. 15:31:27 ...hmm, I think my keyboard is dropping letters and words 15:31:30 Asking people clearly isn't working, so we should work to fix this more directly by March's release cycle. 15:32:21 +1 to get a better fix for this problem 15:32:34 we should probably figure out who signed up for Kalebs packaging sessions, and get those engineers to commit to working on it 15:32:39 So we all agree to wait till kkeithle returns? 15:32:44 that was me and misc, ndevos 15:32:47 I'll work with nigelb to get the documentation together and we'll put out a call on Gluster-infra to make this more transparent. 15:32:58 ndevos, I signed up as well, but never attended any. 15:32:59 nigelb: not only, kaleb gave multiple sessions 15:33:11 My long term solution: Having d.g.o process fully automated. 15:33:15 kshlm: well, s/signed up for/attended/ 15:33:39 I should have some emails on the process, may be I can dig it up. 15:33:41 Short term solution - let's have 3-4 people who are available for packaging that can rotate through. 15:33:45 Part of this work is the nightly pipeline that I was working on. My time has been taken over by glusto lately. 15:34:00 of course, the automation should be done at one point, and we can even do it for the CentOS Storage SIG packages with a special bot account if we want to 15:34:04 Eventually, I want d.g.o process to be one click. 15:34:49 That's less manual work for Kaleb. 15:35:24 nigelb: yes, every single distribution build would already reduce work, its just difficult to get bot accounts everywhere 15:36:06 unless we want to host our own builders, but I guess that will be waaaay to much work to maintain in our infra 15:36:18 Other ridiculous idea: the release managers also get trained on packaging? 15:36:32 If you are a release manager, you can package for other versions and maybe your own? 15:36:39 if the end goal is to have the package in the distribution, our own builder do not help 15:37:19 misc: that is the end goal, but we provide different supported glusterfs versions, most distributions allow a single version only 15:37:34 amye, ndevos is already partially there. 15:37:49 kshlm, that's what made me think of it. 15:38:14 But that would be a lot of work for a single release-maintainer to do all the packaging stuff. 15:38:17 amye: doing it for all distributions needs quite some knowledge about all of their packaging systems :-( 15:38:17 ndevos: yup, because having too much supported version is too complicated and increase workload 15:38:32 ndevos, and it means more documentation work as well :) 15:38:48 Don't we just support 3 distros? Centos/Fedora/Ubuntu? 15:38:52 misc: we have a list of which versions are in which distributions - http://gluster.readthedocs.io/en/latest/Install-Guide/Community_Packages/ 15:38:58 *support with packages 15:39:08 nigelb: and suse 15:39:19 nigelb: and maybe the real debian too 15:39:23 Ah, SuSE. 15:39:23 nigelb: and debian 15:39:31 Before I forget 15:39:37 Noted. 15:39:42 #agreed delay 3.8.9 till kkeithle returns on monday 15:39:59 With our apologies to kkeithle. 15:40:00 ndevos: my eyes bleed at that ascii table 15:40:14 (hopefully, I have a red tshirt, so that's not gonna be too weird) 15:40:24 kshlm: do you mean delaying the release announcement of 3.8.9, or only the packaging for 'kalebs' distributions? 15:40:57 misc: well, yes, it took me some time to make sense of the table too... 15:41:09 ndevos, The announcement. I assume that's what everyone agreed to. 15:41:44 kshlm: oh, I would have done the announcement later today... the CentOS packages have been community tested with emails on gluster-users 15:42:07 ndevos: So for 3.10 as well the CentOS packages will get community tested mails? 15:42:10 so, users are aware that the release is available, somewhat 15:42:12 That's fine by me as well. 15:42:55 shyam: I've requested the build environment for 3.10 at the centos rel-eng team, I think I Need to poke them now they are back from travelling 15:42:57 So we're gonna announce 3.8.9 today, and announce the rest of the packages when the get built next week? 15:43:27 ndevos: in the end, I think that's not exactly what the target demographic want to know (if I can speak myself as part of the said demographic) 15:43:28 shyam: it is up to the community members to start testing them, many are only interested in the version they actively use 15:43:28 ndevos: thanks! 15:43:42 there is no explanation on what is ltm/stm on that page 15:44:02 that list where to find the package, and sometime the distribution 15:44:27 misc: we're *bad* at writing text, LTM/STM is documented on https://www.gluster.org/community/release-schedule/ and that should have been linked 15:44:40 (ie, if centos storage sig is the source of the package, Fedora is the target distro) 15:45:05 and there is no link to instruction on how to use them :/ 15:45:30 ndevos: I still think we should make a hack day on the doc or something 15:45:44 I am sure if we can do that in Malta, people would be more motivated 15:46:14 oh, I've never been to Malta, happy to go an write docs there 15:46:53 Malta++ 15:47:05 kshlm: I think we're done with the 3.8.9 topic ;-) 15:47:17 if we did want to do a hack day, when is the best time? 15:47:30 summer time :) ? 15:47:33 you know that time when we have outage schduled? 15:47:39 that's a good time for a docs hackday ;) 15:48:15 nigelb, I think you just stole my thunder. That was what I was thinking. 15:48:35 I think the best time differs for many of us, we need a list of items that need addressing 15:49:11 and outage time is not suitable for misc and nigelb, they will be busy elsewhere 15:49:12 I see one more topic. 15:49:25 Let's table this as we don't have a ton of time left and I'm sure.. yes that. Another item. 15:49:26 Let's give some time for it. 15:49:36 * ndevos also needs to catch his train in 10 minutes 15:49:45 ndevos: depend, I am gonna just wait on people moving stuff around 15:49:45 #topic Infra outage in March 15:50:05 The community cage is undergoing an outage in March. 15:50:14 are we still on March 14th? 15:50:14 (unless we push it) 15:50:23 amye: I suspect, yes 15:50:27 at least for the network 15:50:39 The entire rack of servers are being moved from one DC to another. 15:51:08 The outage window is for 72h, but we're hoping that it gets fixed in less than 24h. 15:51:43 well, there is 10 racks of equipement to move and to plug, so 12h should be enough, depend how many people are working on it 15:51:56 Is there anything planned for the week of March 14th from our side that we haven't already discussed? 15:52:06 nigelb, misc, what all would we be losing in that time? 15:52:15 We'll lose gerrit and jenkins. 15:52:22 3.10 *should* be out by then ... 15:52:40 It may come in between our bugzilla to github move... but that should be ok 15:53:03 amye, there would be a 3.8 release happening around the time 15:53:06 (Centos CI will also be down) 15:53:13 nigelb, What about d.g.o? 15:53:13 I think we should announce this on both -users and -devel so that everybody can plan around the outage 15:53:19 d.g.o will be up. 15:53:23 kshlm: d.g.o is ok 15:53:26 kshlm, that's what I was afraid of 15:53:32 That's hosted on rackspace (no plans to move it at the current time) 15:53:42 We will once we have final confirmation. 15:53:53 nigelb: well, one da, I will have it in HA in rackspace and the cage 15:54:10 but the list of servers/VM in the cage: https://github.com/gluster/gluster.org_ansible_configuration/blob/master/hosts#L27 15:54:45 I should be able to get the 3.8.10 release out before the outage... (but will be travelling, and that may be causing difficulties) 15:55:32 That's pretty much it about the outage. 15:55:50 How long do you think it'll before we have a confirmed time? 15:55:59 We've got a cage meeting tomorrow 15:56:08 well, the outage was supposed to be done in 2 part 15:56:21 then during planning, it was proposed to do all at once 15:56:25 we should know tomorrow at the meeting. 15:56:41 now, i sent a email to say "pls stick to the plan kthxbye" :) 15:57:14 * ndevos drops off to catch a train, will be back online later 15:57:18 Cool. In any case, the earliest will be 14 March. We should have enough time to plan for things. 15:57:23 Correct. 15:57:32 Alright. 15:57:36 in a ideal world, we would had a redundant gerrit setup 15:57:45 and redundant jenkins too 15:57:53 but that's not doable in 1 month 15:58:47 That's it for today. 15:58:54 I'm ending the meeting now. 15:59:12 Thank you! 15:59:12 Thanks for attending everyone. I'll be sending out the meeting notes tomorrow. 15:59:22 Good night! 15:59:28 #endmeeting