16:00:07 #startmeeting 16:00:17 #meetingtopic Fedora QA meeting 16:00:25 #topic gathering 16:00:59 Hey gang, show of hands folks attending the Fedora Quality Assurance meeting 16:01:09 I know adamw and f13 are unable to attend today 16:01:28 hellú.. 16:01:32 * wwoods waves 16:01:42 Viking-Ice: wwoods: Hey guys! 16:01:44 * mclasen pays a visit 16:01:53 mclasen: welcome :) 16:02:51 hello all, i am new here, i have joined fedora qa in Brno today 16:03:14 dpravec: glad to have you, thanks for joining in from home! 16:04:10 okay, I think that captures most folks ... let's dive into the brief follow-up items from last week 16:04:15 * skvidal is lurking 16:04:15 #topic Previous meeting follow-up 16:04:26 skvidal: hey there 16:05:01 Okay, I've got the still outstanding item of updating our goals page to be more current 16:05:04 * [jlaska] - update QA/Goals wiki document 16:05:38 I don't have big updates here, other than I finally dusted off my fingers and am getting started. I should have draft content soon. 16:05:58 One of the pre-reqs for me was pulling together the test day feedback ... so having that done helps me now 16:06:22 as always, input welcome on the test day survey or recommendations ... https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html 16:06:27 s/welcome/encouraged/ 16:06:39 # [viking_ice] - check-in with abrt 16:06:55 Viking-Ice: I've got you down next for a update on some discovery efforts around abrt? 16:09:00 uh oh, we may have lost Viking-Ice 16:10:05 okay, we can come back to this. 16:10:30 I spoke to Viking-Ice earlier today on the abrt subject, so I believe he's been poking around in that space and offering some RFE's for the abrt team. 16:10:50 * [wwoods] - add post-install network test to basic functionality in rawhide acceptance plan 16:11:24 I believe you had this done before last weeks meeting concluded? 16:13:14 wwoods: did you go where Viking-Ice went? 16:13:26 yeah, I added two cases to the plan 16:13:32 post-install network and post-install input, I think 16:14:26 Nice, what's the linky? just https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan#Basic_Functionality 16:15:19 okay ... going with that for now 16:15:40 #topic F12 - Raising the bar 16:15:59 we're missing a few folks today, so much of the agenda is just touching in on several fronts 16:16:28 The first is an effort mclasen announced to the list ... see https://www.redhat.com/archives/fedora-test-list/2009-June/msg00741.html 16:16:38 not sure if I need to say something about it ? 16:16:58 we want to do a series of user-task focussed test days 16:17:03 rather than feature-focussed 16:17:20 I hope some of the fine qa folks come by... 16:17:34 We want to do the first one next Tuesday 16:18:02 there's been some interesting feedback on the f-d-list for this, but overall this is a really cool initiative. I'm looking forward to contributing! 16:18:10 ? 16:18:15 mclasen: is the schedule of events - http://fedoraproject.org/wiki/Fit_and_Finish ? 16:18:41 yes, as far as we have it yet 16:19:02 I expect this effort to start a bit slow, and then pick up speed 16:19:11 mclasen: are there any additional preparations you guys are looking to make for the event? 16:19:16 as more of my team finish their f12 feature work and can join in 16:19:31 jlaska: we are trying to get live cds building and working 16:19:38 ajax was looking into that earlier today 16:19:53 so far we have the first (ie have them building) 16:20:05 but not the second (boots into runlevel 3, but no X) 16:20:16 also, they're a bit oversize 16:20:25 mclasen: you guys are closer to the issues affecting building them on rawhide right now, but if we can help ... just shout 16:20:41 * jlaska hasn't tried building one this month ... kind of a nice break from that :) 16:20:49 I expect ajax will maybe post an update on that sometime soon 16:20:52 I'll ask him 16:20:59 even better, he's here... 16:21:37 other than that, for preparations, I will spend some more time filling up the test day page with information 16:21:49 and I have to figure out how we are going to handle the issues that are found 16:22:00 mclasen: you've got some nice focus areas listed http://fedoraproject.org/wiki/Fit_and_Finish/Display_Configuration 16:22:05 bugs with a tracker bug, I guess... 16:22:12 that's right 16:22:45 mclasen, so you are using rawhide or f11 for your test days? 16:23:22 so for the QA folks, I encourage folks to join in. Our teams are working together so that we can provide a consistent test day experience ... while also helping to identify use case gaps + bugs 16:24:09 Southern_Gentleman: the plan is to use rawhide, but I don't think it is essential for user-task testing that everyone runs the same 16:24:24 you may well find issues in F11 that have a high chance to still be relevant in rawhide 16:24:55 as we move further into F12 development cycle, I suspect the gap will become wider? 16:25:59 hopefully we'll fix some of the issues that are found, so yes... 16:26:23 cool 16:27:02 * Viking-Ice back 16:27:09 Thanks for kicking things off on the Test Day front. Having you and your crew involved should really help fast track issues that come out of this 16:27:45 I've updated the upcoming events section to point to #fedora-fit-and-finish next Tuesday 16:27:46 btw mclasen reporters are missing what to report on you Fit and finish 16:28:01 which files to include etc .. 16:28:37 I will try to clarify some of that when I get to responding in that thread 16:28:52 mclasen: Viking-Ice raises a good point. Something that surfaced from last weeks meeting was the need for improved details on writing good bug reports 16:29:18 mclasen: if it makes sense at all to link to upstream docs or add some content into https://fedoraproject.org/wiki/Category:Debugging ... that's an area we can assist in 16:29:29 mclasen: all X components are covered 16:29:53 https://fedoraproject.org/wiki/Xorg/Debugging#All_bug_reports 16:30:29 okay ... going to move on to next topic (there is a meeting room conflict in 30 minutes) 16:31:34 #topic AutoQA - update from wwoods 16:31:44 wwoods: what's the word on the autoqa front? 16:31:59 So the plan (as mentioned earlier) is complete: https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan 16:32:07 7/12 cases have been written 16:32:42 looking over the roadmap: https://fedorahosted.org/autoqa/milestone/israwhidebroken.com 16:33:07 the stuff there isn't really in order but all the steps are there. 16:33:24 so the next step after finishing writing test cases is to attempt to automate one 16:33:46 which requires an autotest instance, which requires packaging autotest 16:33:53 that's https://fedorahosted.org/autoqa/ticket/9 16:34:15 jlaska and f13 are handling that and it seems to be going well 16:34:21 yeah ... good progress on that front 16:34:36 I'm happy with the autotest packaging, and have started futzing with the autoqa package to tie into it 16:34:48 hopefully by next week we'll have the test cases all finished and at least one automated 16:34:53 once we know how to write tests for autotest 16:35:00 I have to discuss with f13 how we'll manage some of the python dependencies to get this into shape for fedora-infrastructure 16:35:05 we can start trying to figure out how to organize and package those tests in the autoqa repo 16:35:44 wwoods: was there a test you had in mind that seemed the most ready for automating? 16:36:20 something where we can use the off-the-shelf approach? 16:36:26 the repodata/comps validity stuff is really easy to automate 16:36:37 as is installer image existence 16:36:53 there are existing scripts / tools / example code to mess with that stuff 16:37:10 cool, that should help! 16:37:30 I've got some bits of that .treeinfo validation if it's useful 16:37:43 the other thing to figure out is.. how to allow manual test reporting 16:37:58 since there's always going to be things that we can't automate (or haven't figured out how to automate yet) 16:38:34 wwoods: is this a general test result reporting solution? 16:38:45 that's a whole big can of worms, because then you need identification/authentication to know who is allowed to report the results 16:38:58 I really don't want to rewrite something like testopia 16:39:32 and we've been promised some magical Test Case Management System for months and months and it never seems to happen 16:39:39 because it's a huuuge problem to solve 16:39:46 so no - that would get way, way out of hand 16:39:52 agreed! 16:39:53 I'm trying to keep this as simple as possible 16:40:37 I don't know how robust it is, but I think autotest does support remote communication 16:40:45 so that might help solve the immediate need, who knows 16:42:46 wwoods: thanks for the update, we'll stay tuned to fedora-test-list for any updates or if test review is needed 16:43:29 wwoods: any other updates on the autoqa front? 16:44:08 nothing I can think of at this time 16:44:17 okay, changing topic ... 16:44:22 #topic Install Testing SOP draft - update from lili 16:45:00 hopefully Liam is sleeping right now (being 12:44am for him) ... but wanted to give a quick update on an SOP he posted for review 16:45:31 jlaska: this is fedora, we don't allow sleeping :-P 16:45:42 Liam is trying to document the tasks involved when planning+preparing and executing a collaborative installation test event (see mail at - https://www.redhat.com/archives/fedora-test-list/2009-June/msg00734.html) 16:45:50 jeremy: shhh! aren't you supposed to be sleeping! 16:45:52 :) 16:46:36 A few folks have chymed in already and helped clean up the wiki, so thanks to arxs and beland 16:47:33 while this largely affects the folks who have been testing RC's, feedback from a wider audience is encouraged ... as that's the audience we're hoping benefits from added transparency 16:47:38 https://fedoraproject.org/wiki/User:Liam/Draft_Install_Test_SOP 16:48:06 unless any questions/comments ... changing topics ... 16:48:28 #topic Improve Bug Reporting proposal - update from viking_ice 16:48:51 Viking-Ice: wanted to give you an opportunity for any updates on the proposal you drafted 16:49:01 looks like some good stuff come from that discussion last week 16:49:13 Yup things are emerging.. 16:49:24 Took a look at abrt 16:50:06 Filed an rfe for gui improvements https://fedorahosted.org/abrt/attachment/ticket/53/Automatic%20Bug%20Reporting%20Tool%20Mockup1.png and https://fedorahosted.org/abrt/attachment/ticket/53/ABRT_simple_bug_filing_form.png 16:51:13 Viking-Ice: I know the abrt guys follow their trac instance, but you may want to check in with them directly. I know they are keenly interested in improving the look'n'feel (and ease of use) of abrt 16:51:35 * mclasen recommends iterating over that design a bit :-) 16:52:31 mclasen: you're much closer than me to what is helpful when it comes to designing GUI apps 16:52:32 Bringing the gui closer to the simpler bug reporting from that's coming with the upcoming release of bugzilla The Idea here is to reduce the learning curve for reporters so if users have gotten used to work in the upcoming bugzilla simple reporting form they have no problem getting used to use abrt and also if we need to get reporters for some reason to bugzilla they would be familiar with the reporting form there... 16:53:44 mclasen: over my proposed designed or the current? 16:53:51 I guess it's a question for the abrt team whether abrt is intended as a graphical desktop client for filing bugs, or a generic problem reporting tool ... where one of the reporting "plugins" is bugzilla? 16:54:13 Viking-Ice: both 16:54:34 the abrt guys need to get together with some of our designers, really 16:54:46 definitely! 16:55:17 Look at what huge improvements they've suggested for setroubleshoot ... https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements 16:55:20 Sorry to say but I think they should try to have this as close to the bugzilla form as possible.. 16:55:45 * mclasen doesn't even think bugzilla should figure at all in this... 16:55:55 anyway, probably offtopic here 16:56:06 true 16:56:27 Viking-Ice: the Debugging pages were a nice offshoot from your topic last week, I know Adam is raising these during BugZapper meetings 16:56:36 did you have any news or recommendations on that front? 16:57:15 debugging visualization seems complete https://fedoraproject.org/wiki/Reporting_virtualization_bugs 16:57:46 mean virtualization 16:58:06 yeah, markmc and the virt team have done a great job outlining that page 16:58:17 it came in handy during the virt test day ... and hopefully will for the F12 virt testday 16:58:35 oh yeah I should mentioned selinux 16:58:49 https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements 16:59:24 Notice the report Bug <-- 17:00:11 are there things here you think the QA team should be considering? 17:00:44 yeah the "report bug" dialog there is closer to what you'd want to see from a bug-reporting client 17:00:46 The only thing flawed in this design is calling the button Report Bug instead of Report Alert 17:01:36 This is a security application and thus user might think all alerts are just bugs and start to ignore it 17:02:43 Just thought to mention what's happening on selinux front for users and the great job they are doing... 17:03:23 Anyway i'm done 17:03:39 thanks for the update ... anything you want to check-in on next week on this subject 17:03:53 or things you want the larger f-t-list to review/consider before next meeting? 17:04:09 ping ? 17:04:36 #topic Open Discussion 17:05:00 We've run over, but I just wanted to toss a note out there for any feedback/input on the Fedora 11 test day survey results 17:05:03 https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html 17:05:06 nope low on time these day's.. we have to finish something next week before everybody start bailing for vacations.. 17:05:16 here @ work.. 17:05:27 I've gathered the feedback to the wiki and starting to toss recommendations on the list for how we can improve F12 based on the feedback 17:05:34 Viking-Ice: no worries, understood. 17:05:45 That's all I had for today ... any other thoughts / concerns? 17:06:13 Can we use the room ? 17:06:20 fhornain: sure can, we are wrapping up 17:06:33 Thx 17:06:51 Thanks for your time today, let's keep things going on f-t-list and #fedora-qa 17:06:57 #endmeeting