18:00:01 #startmeeting proventesters (2011-10-05) 18:00:01 Meeting started Wed Oct 5 18:00:01 2011 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:01 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:01 #meetingname proventesters 18:00:01 The meeting name has been set to 'proventesters' 18:00:15 #topic Intro/Gather more agenda 18:00:23 any folks around for a proventesters meetup? 18:00:31 * satellit_ listening 18:00:38 decided to show up to figure out how to become one 18:01:25 jwb: cool. ;) it's easy. 18:01:35 * tflink is around 18:01:53 http://fedoraproject.org/wiki/Proven_tester#Joining_the_proven_testers 18:02:16 basically apply, read the docs as to what you are supposed to do, get approved 18:02:28 * maxamillion is here-ish 18:02:57 nirik, thx 18:03:30 I had 2 items on my plan... recruitment ideas, and one stop shopping page for resources... did any other folks have items? 18:03:44 nothing here 18:04:12 none here 18:04:15 * nirik will wait a min more for folks. 18:04:19 rgr 18:05:26 ok, lets go ahead then... 18:05:35 #topic Recruitment 18:05:57 so, any ideas on how we can get more proventesters involved? and/or more testers for updates in general? 18:06:37 I had a few random ideas: 18:06:44 I forget, do we have any step-by-step instructions for installing updates-testing and giving karma? 18:06:53 could we mine old updates for people who have given lots of karma but are not proventesters? 18:06:55 I've actually thought about this in the past ... I think maybe getting a little funding to offer swag of some sort would be good ... like a "Fedora QA Community Member" t-shirt or something (likely a less expensive option to start out with) 18:06:58 * tflink will wait for nirik's ideas before giving mine 18:07:31 well, all we have currently is http://fedoraproject.org/wiki/Proven_tester 18:07:40 which lists some things, but not sure in how much detail. 18:07:44 I remember an interesting conversation about metrics and their human effect 18:08:00 I really think most of it is that people don't know how easy it is to become a proventester 18:08:01 https://picasaweb.google.com/103451550643082159750/B#5576235128261512338 <--- I doodled up a t-shirt idea a while back 18:08:09 as soon as there is some sort of reward for doing X based on measurement Y, people will generally start behaving differently 18:08:20 maxamillion: that would be cool. dunno if we have funds for such a thing tho 18:08:22 tflink: well ... true 18:08:25 nirik: yeah 18:08:25 nirik: yeah, and how much they're needed 18:09:03 ideally, everyone who provides karma on any kind of regular basis should take the few minutes to become a proventester, IMHO. 18:09:54 I wonder if stuff like step by step instructions would help 18:10:00 maybe screencasts, too? 18:10:14 the other two things that I can think of are: 18:10:25 people worried about borking their system and losing data/time 18:10:46 make f-e-k a little more user friendly 18:11:12 I've thought about trying to make a gui for f-e-k before 18:11:38 I'm not sure how active the developer is these days... but he was around not long back 18:11:40 to make it easier to find test cases, related packages and skip over stuff that you don't feel right karma-izing 18:12:05 I'd be interested in doing it but the issue is available time 18:12:13 yeah. 18:12:57 is it something that we really think would help? 18:13:04 also, folks providing feedback on list could be pointed at becoming proventesters... 18:13:38 Would a GUI avoid the "skip" confusion? Or can that be solved at the command line? 18:13:49 not sure. I think a bigger win might be to setup something so PK would say "hey, there's an update of foo you have installed in updates-testing, would you like to test it?" 18:14:01 just leaving karma would be a start, not even becoming a proven tester 18:14:21 nirik: that's a cool idea 18:14:34 nirik: yeah, that would be good. Give a list of testing updates available and let the user select which ones 18:14:40 it's been floated many times before, but has noone working on it. ;) 18:14:41 to install 18:14:55 a yum plugin might be easier. ;) 18:15:55 I don't think there's a magic bullet... 18:15:58 More precisely needed is a way to identify which packages a user actually used and for libraries, how they got there 18:16:07 A lot of packages are installed by default I would never use 18:16:27 Cerlyn: yeah, sometimes its a bit of a puzzle... 18:16:37 and updated libraries that come down and I have no idea what they do 18:16:39 libfoo is an update. What the heck do I use libfoo for? 18:16:49 * tflink was wondering if a dependency tree + description might help 18:17:14 appbar -> libbar -> libfoo (appbar does magical stuff) 18:17:19 yeah, it does give summary now, but tree would be nice. 18:17:39 not sure how hard that would be to implement. 18:17:52 It also makes no sense to tell me about a new ATI Xorg driver is available, but I have nvidia/Intel/unknown-to-me instead. Saying video still works could be useless. 18:17:56 me neither, I've never even looked @ f-e-k code 18:18:29 Cerlyn: yeah, all the x drivers are installed by default, since they are small and you could move your install to another box and want it to work. 18:18:44 but it means you have a number that aren't actually in use. 18:20:15 so, I think short term here: mine existing karma for non proventesters and ask them to join, more press (blog posts, point people on test list), and look at f-e-k improvements? 18:21:10 yeah 18:21:42 longer term: gui for f-e-k and having pk or something like it offer updates-testing 18:21:50 another idea was tie it into abrt. ;) 18:22:09 foo has just crashed and you reported a bug on it... but foo has a updates-testing update, would you like to test? 18:22:28 that's a cool idea 18:24:17 #idea short term here: mine existing karma for non proventesters and ask them to join, more press (blog posts, point people on test list), and look at f-e-k improvements 18:24:26 #idea longer term: gui for f-e-k and having pk or something like it offer updates-testing 18:24:33 #idea longer term: tie into abrt 18:24:40 anything more on this? or shall we move on? 18:25:26 #topic One stop page for updates testing resources 18:25:41 Another idea I had was to try and create a short page with all the resources. 18:25:47 we have stuff all over the place now. 18:25:50 that's probably a good one 18:25:58 I thought we had that at one point ... or atleast a mockup of one 18:26:00 yeah, especially for the short term 18:26:01 on the wiki somewhere 18:26:04 bodhi, fedora-easy-karma, rss feeds, reports to mailing lists, etc 18:26:51 would anyone be able to put in some time to make one? ;) 18:27:01 I could help out, but not sure how much time I have to devote to it. 18:27:48 I can try and make something tho.. or ask on the list. ;) 18:28:10 oh ... like all in wonder page for QA efforts, that's different than what I had in mind but I like it 18:28:46 one of the complaints has been that it's hard to see all the stuff you need... 18:28:57 if we can get all the links in one place it might help 18:29:18 yeah 18:29:19 yeah 18:29:40 and for long term, see if we can't get some of that stuff merged into bits of the revamp to fedora community? 18:29:44 #info will try and create a package updates tester page with links to resources. 18:30:03 maxamillion: possibly. Not sure if thats a focus for it, but it could be nice for that. 18:30:56 lets figure out what's really being used before trying to integrate it :) 18:31:08 or what would be the most useful 18:31:10 fair enough 18:31:23 rgr 18:31:39 #topic Pending updates 18:32:00 so, did we want to try and look at the updates pending in 15/14 currently and see if we could move along the ancient ones? 18:32:06 or is that better done out of the meeting? 18:32:53 11 critpath ones in f14 18:33:12 19 in f15 18:33:53 some of them are ancient due to almost no one having the hardware. 18:34:00 some are just unclear how to test. 18:35:30 I'm hoping to have an old laptop setup to help test 14/15 soon... 18:35:41 #info 11 critpath updates pending in f14 18:35:49 #info 19 critpath updates pending in f15 18:35:56 #topic Open Floor 18:36:01 ok, anything for open floor? 18:36:15 should we continue meeting? or does it seem to be doing any good? 18:36:37 nirik: I came in late - so trying to get the gist of the discussion... 18:36:44 would there be a limited proventester category for testers who mainly do installs to Hardware? 18:36:59 mike-c: basically we are trying to increase updates-testing package testing. ;) 18:37:07 satellit_: what would the benefit of that be? 18:37:37 mike-c: get more proventesters, or general testers involved so packages don't get stuck in updates-testing. 18:37:37 I am not sure I have the skills to do some of the other kinds of testing.... 18:38:00 Any value in asking testers what main kinds of testing and which versions of Fedora they are mainly doing? 18:38:30 satellit_: then just don't add karma where you don't feel you have tested enough... ;) 18:38:34 satellit_, if you are running a vm to test a package that is good as well 18:38:37 ok 18:38:49 mike-c: well, not sure... we want more people particularly for the older releases... 18:38:57 i am running f14 so i do test f14 native and f15 in a vm 18:39:49 For example I am very keen to test kernels, nouveau and similar, as well as mail server and similar but also test other things where I can - mainly f16 currently with a little f15 18:40:16 thats why i think a proven-testers mailing list that on monday and thursday spam with a list that needs tested would be great 18:40:27 in general I think we have more testers on newer releases... fedora is so fast moving, most of the folks involved tend to want to move fast with it. 18:40:58 * nirik has a f14 and f15 vms, and a bunch of f15 machines and a few f16 machines where I do most of my testing. 18:41:31 Makes sense if testers do run older versions as servers and where stability is essential but newer versions where they want to test upcoming performance? 18:42:23 Southern_Gentlem: would a web page with links to the current stuff meet your needs? 18:42:34 mike-c: yeah, I think some of that is happening. 18:43:13 Southern_Gentlem: I am happy with being spammed on the test list with the list of packages that need testing with one spam shot per fedora version.... 18:44:22 Southern_Gentlem: I floated the idea of another list last week, and the reaction was pretty negative on the test list. Do chime in there if you like... 18:45:43 I guess also that for most people availability to run tests varies in time depending on other commitments... for example I do try to join the graphics test days but can't always be free then - similarly during the normal cycle of listed packages for test from the announcements on the list 18:46:48 yeah, absoutely. 18:47:13 I tend to run fedora-easy-karma once a week or so to collect all the feedback from using the machine... 18:47:22 or more often if there's a specific update that needs testing 18:47:26 Would there be any value in asking people who join proventester what their main interest is i.e. testing packages for released N, N and N-1, N and N+1 etc so you could have a list of what the pool of testers are likely to be doing? 18:47:31 mike thats why i think another list would be good thing 18:48:12 mike-c: well, I'm not sure how that helps us... I guess we would know more who we have out there, but it doesn't help us grow or provide better coverage... 18:49:38 I guess - it would be nice to have a balanced division of effort - though I know that is why this discussion is happening! At the moment it is pot luck what gets tested in any given period... 18:50:03 It probably wouldn't be too hard to push a survey to everyone already in the group as well 18:50:13 sure, but we can't easily say: 'hey, you, go test f14' (well, we could, but they could say 'no, I don't care about f14') 18:51:30 Sure - testing is dependent on voluntary effort - and there have been several attempts to encourage testers to do things - Adam Will occasionally makes a specific request to the test list 18:51:56 yeah, although those are almost always for fN-branched. 18:52:08 occasionally a maintainer will ask for testing... 18:52:13 I guess that it only needs one or two people to test any one package so provided the pool of testers is reasonable it "should" work! 18:52:35 I guess this is related to another idea I read somewhere (can't recall where): 18:52:58 revive the 'qa-contact' field for packages... so people could sign up for specific packages. 18:53:29 pkgdb has a 'Initial qa contact' field... which I don't think we currently use for anything. 18:54:00 Interesting! That may well work for some testers who have specific interests and do test the ones they have a specific love for on a regular basis - perhaps across several fN 18:54:07 nirik: should qa-contact tag everyone in proventesters for all critpath packages? 18:54:28 well, I am not sure what actual effect it has. It might be cc'ed on all bugs only. 18:54:29 or just divvy it up accordingly? 18:54:34 rgr 18:54:35 abadger1999: you happen to be around? 18:54:42 nirik: yep 18:54:52 qa-contact is for bugzilla 18:54:57 per-package 18:55:09 takes a single fas username 18:55:26 ok, so thats not any different than inital cc really? 18:55:32 yeah 18:55:42 Just a different field in bugzilla 18:55:52 (qa contact field instead of CC field) 18:56:11 I think it has some purpose if Bugzilla is setup to use a test-in-build style of operation 18:56:37 Perhaps it could be offered for testers to sign up on the bz to "take" the "qa-contact" field? 18:56:40 we could perhaps co-opt it some in bodhi to send those people info about updates to that package I guess. 18:56:46 I had another idea - could Bodhi notify people who tested the last N versions of a package and gave non-zero karma points that a new one has been released? 18:56:54 but I think it's going to require more thought/design. 18:58:11 (The QA contact role has its own set of email preferences for users in bugzilla) 18:58:35 Cerlyn: we could, but I bet people would complain. 18:59:19 ok, we are running low on time. 18:59:36 any last minute items? do folks want to try and meet next week? 19:00:31 lets continue on list... 19:00:38 Thanks for coming everyone! 19:00:41 #endmeeting