16:00:06 #startmeeting Fedora QA Meeting 16:00:06 Meeting started Mon Jan 17 16:00:06 2011 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:06 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:12 #meetingname fedora-qa 16:00:12 The meeting name has been set to 'fedora-qa' 16:00:22 #topic Gathering in the lobby 16:00:25 * jsmith lurks 16:00:28 * fenrus02 waves 16:00:32 show of electronic hands ... 16:00:39 fenrus02: jsmith: Hi folks 16:01:13 robatino: welcome! 16:01:27 * Viking-Ice here 16:01:35 * mkrizek lurks from behind the books 16:01:37 jlaska: This meeting is at the same time as an internal staff meeting, but I'll multi-task :-) 16:01:48 mkrizek: woah, welcome :) 16:01:54 Viking-Ice: hiya 16:02:34 * kparal welcomes mkrizek 16:02:41 * rbergeron peeks in 16:02:48 hi kparal + rbergeron 16:03:01 mkrizek: we're like a bad cold, you just can't shake us :P 16:03:14 jlaska: :D 16:04:05 alright ... another minute or two for any stragglers, then we'll get started 16:05:17 yo 16:05:24 hey brunowolff + adamw 16:05:28 okay, let's get started ... 16:05:33 I am here. 16:05:36 #topic Previous meeting follow-up 16:05:48 #info Bodhi feedback patch from fcami (see ticket#701 (infrastructure) awaiting review 16:06:12 There was a bodhi update last week, I don't know if this was included? 16:06:14 * jlaska checks changelog 16:06:30 * jlaska looking at http://lists.fedoraproject.org/pipermail/devel-announce/2011-January/000742.html 16:06:53 I don't think it was included 16:07:06 any updates from fcami or lmacken ? 16:07:54 they both seem to be afk at the moment 16:08:05 yeah, okay ... guess we'll leave this open then 16:08:15 not sure what else to do with it 16:08:34 that's all I had from last week, on to the main event 16:08:38 #topic Update on critpath test definition (ticket#154) 16:08:42 as long as we've all agreed on what should be changed and there's a ticket open against bodhi, i don't see what else we can do 16:09:17 adamw: agreed, I was worried since we don't have any ACK/NACK on the idea, it's on our plate until it's on the roadmap 16:09:41 adamw: you've been pretty busy this past week on ticket#154, what's the latest? 16:09:43 i guess maybe someone should poke luke outside of the meeting 16:09:51 * jlaska will poke 16:09:55 I pushed the policies into production and announced on the lists 16:10:10 #action jlaska - check-in with lmack on http://lists.fedoraproject.org/pipermail/devel-announce/2011-January/000742.html 16:10:11 I also started converting test cases to the new category system, I have about half of our existing test cases done now 16:10:21 if anyone wants to help it'd be welcome =) 16:10:23 #link http://lists.fedoraproject.org/pipermail/test-announce/2011-January/000171.html 16:10:42 i also mailed infrastructure list to discuss the possibility of adding a mediawiki extension which would allow multi-category search 16:10:47 adamw: I'll be trying to adjust some wiki pages for the upcoming test day, I'll shout if I could use some guidance 16:10:49 which we kinda need to make the system really work 16:11:07 #info started converting test cases to the new category system, I have about half of our existing test cases done now 16:11:17 #info mailed infrastructure list to discuss the possibility of adding a mediawiki extension which would allow multi-category search 16:11:24 http://lists.fedoraproject.org/pipermail/infrastructure/2011-January/009829.html 16:11:56 looks like Ian points out what we found as well ... with regards to manual category searching using the API 16:12:18 yeah 16:12:30 i actually only just read his reply, i'd forgotten about the infrastructure thread until now... 16:12:42 oooh, new mediawiki update coming soon ... that'll be a nice improvement 16:12:43 the only problem with what ian suggests is that every client would have to re-do the searching code 16:12:47 right 16:12:52 unless we implemented some kind of intermediary 16:13:17 we could add support to that fedora-mw library ian pointed me too 16:13:28 what was that? 16:13:43 ergh, I worried you'd ask :) one sec ... 16:14:04 #link http://koji.fedoraproject.org/koji/packageinfo?packageID=11263 (python-simplemediawiki)_ 16:14:19 ah 16:14:38 I've done some coding against this library for some metrics gathering scripts, I could work something up to meet your needs? 16:14:43 that would be awesome 16:14:55 the other tricky thing we need to deal with under my current plan is recursion 16:15:01 okay, I'll find you after for some details 16:15:09 i.e. we need to know 'all pages that are members of categories which are members of this category' 16:15:20 since i came up with that clever-clever system for splitting up groups of test cases for a given package 16:15:30 #info discussed the possibility of adding multi-category searching to the python-simplemediawiki API 16:15:33 if that's too complicated we can always change the policy to not allow recursion or tweak it some other way 16:15:58 or cap it at some 3 levels deep or something arbitrary? 16:15:59 so i want to get this sorted before talking to luke and till about adding support to koji and f-e-k 16:16:09 as i don't want to request something they can't really practically do yet 16:16:17 that makes sense 16:16:17 jlaska: oh yeah, with the current policy there'd only be at most one level to deal with 16:16:38 #info need to determine if/how to handle Category recursion on the wiki 16:16:52 there hasn't been a lot of response yet in terms of people creating test cases, but i think once we get the mechanism somewhat exposed it should help, and as people get used to using it through test days and so on 16:17:07 exposure througrh test days will help too 16:17:26 * jlaska just referenced your new test case SOP today for the 3rd time 16:17:28 i was thinking of getting up on my hind legs at fudcon and talking about it too, though i haven't proposed that as a talk yet and it may be too late 16:17:29 that can't hurt! 16:17:37 never too late! 16:17:38 so it could be an unconference topic or something 16:18:07 #idea Discuss new test case SOP and wiki organization at FUDCon Tempe 16:18:37 adamw: okay, I'll get back to you tomorrow (wed latest) on the simplemediawiki stuff ... I need to get some feedback to kparal for another item first 16:18:45 ok 16:18:56 alright, great progress on this activity. 16:18:58 adamw: it's not too late. 16:19:05 anything else you wanted to highlight to the minutes? 16:19:07 barcamp = everything gets picked saturday morning :) 16:19:14 rbergeron: yeah, i meant for the talks track 16:19:32 rbergeron: it'd probably be best for barcamp anyway, though, because i can't talk about it for long 16:19:40 just a few minutes, then have time for people to actually try it out, would be best 16:19:55 and if no-one shows up we can just turn it into time for me and jlaska to convert some existing test cases 16:19:55 adamw: talks are barcamp too. 16:19:57 it's a win-win =) 16:19:59 rbergeron: ahhh, cool 16:20:18 was it always like that? have I been at jsmith's crack pipe again? 16:20:49 adamw: no idea. I've not been to a fudcon yet. I plan on coming though this time. :D 16:20:58 i think it probably was and i just forgot 16:21:01 anyway! moving on 16:21:06 nothing else to highlight for the meeting 16:21:09 okay, thanks for the updates :) 16:21:12 did anyone have any comments on the SOPs or anything? 16:21:49 ah, i bored everyone to death, good! plan to take over the world, step 1 complete 16:21:51 #link https://fedoraproject.org/wiki/QA:SOP_test_case_creation 16:21:58 #link https://fedoraproject.org/wiki/QA:SOP_package_test_plan_creation 16:22:01 heh 16:22:15 alright, next topic ... 16:22:21 #topic Latest and greatest on autoqa-0.4.4 16:22:41 kparal: want to spend a few minutes on recent autoqa updates? 16:22:44 let's dub it as "busy busy week" :-) 16:22:46 sure 16:22:59 yeah, lots of activity ... that's great to see 16:23:29 as usually I cheated a little and prepared some bullets in advance. here we goL 16:23:40 * We have simplified the test object template a bit by using 'self.__class__' where possible. Now even a three-year old can write an AutoQA test object (as long as he/she know Python basics) :-) 16:23:40 https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001494.html 16:24:06 haha, I like that 16:24:18 * clumens' branch is now merged into master. 16:24:18 This means we have one new hook 'git-post-receive' that triggers everytime a new commit is made into a git repository (anaconda, presently) and three new tests 'anaconda_checkbot', 'anaconda_storage' and 'compose_tree'. The first performs checks on anaconda, the second one perform many installation tests from our installation testing matrix (yay!) and the third one attempts to create Fedora boot.iso images. This is going to be run after 16:24:18 https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001473.html 16:24:36 #info "So easy, a three year old can do it" - simplified the test object template a bit by using 'self.__class__' where possible 16:25:59 * mkrizek implemented logging support and unittest execution support. This stuff is now in separate branches and will be merged once we have a little time to review it. 16:25:59 https://fedorahosted.org/autoqa/ticket/196 16:25:59 https://fedorahosted.org/autoqa/ticket/258 16:26:05 * kparal sending kudos to mkrizek to Denmark 16:26:09 #info Clumenas branch merged - New git-post-receive hook and several installer tests (see https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001473.html) 16:26:18 #info mkrizek implemented logging support and unittest execution support. This stuff is now in separate branches and will be merged once we have a little time to review it 16:26:32 kparal: feel free to use #info #help #idea #action etc... 16:26:56 s/Clumenas/Clumens/ 16:27:02 (sorry Chris) 16:27:04 #info jskladan adjusted his 'new_koji_watcher' patch and the debate ensued 16:27:11 It certainly caused some flutter :-) After all the concerns are settled, this should provide us with a better implementation of the bodhi event detection (by using koji -pending tags) and also test batching support (provides better performance). 16:27:18 #link https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001485.html 16:27:39 seems like more thorough detection too ... based on your latest mail 16:27:56 it seems that currently both implementation have some gaps :-) 16:28:20 #info wwoods posted 'depcheck' patch 16:28:28 Another big chunk of code dropped on the head of the poor reviewers :-) I still haven't managed to look at it, but I'll do that soon. jlaska 'the superman' already did. depcheck is the most anticipated test of the upcoming autoqa release. 16:28:34 #link https://fedorahosted.org/pipermail/autoqa-devel/2011-January/001512.html 16:29:11 ok, who did I forget? raise your hands 16:29:42 #info hongqing researching anaconda virtio support and integrating that into rats_install/install.py 16:30:04 oh, great 16:30:31 we've been talking about this a lot privately, but I expect things will move to autoqa-devel@ soon. Basically, this would allow us to mimic more real-world install scenarios by using virtio monitoring, rather than our network-based minimon script 16:31:06 jlaska: does that mean that you could monitor files on disk inside VM, while VM is running? 16:31:40 kparal: yeah exactly, you could monitor installation progress (by way of installer log files) from the virt host 16:31:48 perfect 16:31:52 using a dedicated virt serial port between the host and guest 16:32:11 #link https://fedoraproject.org/wiki/Anaconda/Logging 16:32:29 based on what hongqing uncovers, we may use this for all subsequent installation tests 16:33:12 kparal: that's it from me 16:33:35 in summary: big patches last week, big reviews this week :-) 16:33:54 yay patches! 16:33:59 big fun next week? 16:34:14 (or big crash) ^^ 16:34:24 heh, fun of a different sort :) 16:34:51 okay kparal, thanks for the updates. Kudos to jskladan and mkrizek as well 16:35:20 Hurry asked me to raise the next topic ... 16:35:27 #topic Update on TCMS nitrate analysis (ticket#152) 16:36:10 #link https://fedoraproject.org/wiki/Rhe/tcms_use_cases 16:36:18 #info Rhe regrouped the use cases by general and main test events(runs) use cases. The general cases cover the basic uses of wiki, and the events cases covers the detailed certain steps for organizing the events. 16:37:16 I'm just now absorbing her latest edits, but this seems to do a good job of capturing our current use cases involving the wiki and test events/cases/plans etc.. 16:37:42 Hurry asked for comments/concerns/ideas on the wiki discussion page 16:37:52 #link https://fedoraproject.org/wiki/Rhe/tcms_Comparison 16:37:58 #info The comparison is grouped according to the use cases. It has overlaps between each other which I need to further update. But the basic idea is shown, please review them and welcome to provide any comments about the grouping idea/design format/contents. 16:38:59 cool, thanks hurry! 16:39:05 i'll take a look at the new layout soon... 16:39:13 this looks like when organized by the use cases. Same thing, any thoughts/comments ... please drop Hurry a line on the Discussion tab 16:39:34 #help Feedback requested ... last chance to review use cases and comparison 16:40:41 the eventual goal here is to identify what features are MUSTHAVE and SHOULDHAVE when it comes to our test case management needs 16:41:06 okay, unless any questions, moving on ... 16:41:18 #topic Open discussion - 16:41:24 I got one 16:41:28 I have 2 quick topics for open discussion as well 16:41:31 looks really good to me 16:41:34 Viking-Ice, why don't you go first 16:41:38 Ok 16:42:40 #link http://lists.fedoraproject.org/pipermail/advisory-board/2011-January/010271.html 16:43:06 Rex mentions here "There was a decision, a list of criteria to be met before being blessed:" 16:43:24 #topic Open discussion - Media_Handout_Requirements 16:43:29 #link https://fedoraproject.org/wiki/Media_Handout_Requirements 16:43:31 #link http://lists.fedoraproject.org/pipermail/advisory-board/2011-January/010271.html 16:43:56 https://fedoraproject.org/wiki/Media_Handout_Requirements#Quality_Assurance <--- (since you are linking also it skip it ) 16:44:17 Who and When was this decided this within the QA community? 16:44:32 got me ... adamw is this on your radar? 16:44:38 sorry, was just looking at something else 16:44:42 yeah, this is on my agenda 16:44:46 okay 16:44:50 Viking-Ice: afaict it wasn't; board never approached us 16:44:55 they just came up with the requirements off their own bat 16:45:07 So some kind of communication break down 16:45:19 i think board just wanted to Get It Done 16:45:23 A quick inspection shows the release criteria cover most of these points 16:45:23 but yeah, not great process 16:45:34 mostly what they've written is actually fine, we can just tweak it a bit 16:46:01 anything else that the board has decided but not shared with us that ye might be aware of? 16:46:03 there are some requirements on rel-eng as well regarding SOP's 16:46:05 the things that worry me are the use of the phrase 'test plan', which can be a bit more loaded than probably the board expects, and the reference to one person carrying out the test plan, with 'oversight' from QA, which doesn't really match the scheme we've developed for qa'ing spins 16:46:13 Viking-Ice: not that i'm aware of.. 16:46:29 Viking-Ice: i only caught this by following the flamewar soap opera that is board-advisory-list, though 16:46:47 this certainly does not reflect our current workflow 16:47:02 yeah, i think we can tweak it to reflect the current workflow reasonably easily though, and the board would probably be okay with it 16:47:14 anyone want to volunteer? 16:47:22 my plan was to come up with a proposed modification of the board's text and post that to -test for people to look at, then forward it to the board 16:47:30 To be honest the board should ask us to provide them with whatever they were after 16:47:45 Viking-Ice: yeah, but that's water under the bridge now, i'd rather just look at what we should do from here 16:48:10 adamw: cool, please include Hurry in case any adjustments are needed on the install event side 16:48:26 if anyone else wants to draft the modifications that'd be great, go ahead and volunteer =) 16:48:52 #help volunteers welcome - need to adjust current draft to match QA workflow 16:49:04 also worth noting there's a general plan to discuss the 'future of spins' at tempe 16:49:07 we have to make sure this kind of communication breakdowns wont happen 16:49:09 which we should get in on, i will be sure to get included in that 16:49:39 the other thing that worries me about this is it's expected to apply to all spins, so the board is kind of expecting us to have test plans for everything, down to stuff like broffice and the electronics suite that currently we just don't bother with 16:49:55 eeeergh :( 16:49:56 not really an onus on *us* as the idea is the spin sig develops the test plan, but it's just a point to note 16:50:06 right, right 16:50:13 agreed 16:50:33 Viking-Ice: i did try and let the board know that we'd really prefer they come to us about this kind of thing in future 16:50:40 alright, anything else here to track? 16:50:45 Viking-Ice: if you have any other ideas to get the point across to the board...:) 16:51:10 thanks for raising the issue btw 16:51:31 agreed, certainly a benefit of transparency 16:51:39 okay ... two quick ones from me ... 16:51:43 broffice is being dropped as the trademark issue is moot now. 16:51:52 brunowolff: oh right, that's one less to worry about. 16:51:56 yay, -1 16:52:13 #topic Volunteer interested in data mining smolt? 16:52:21 #help Anyone interested in understanding and gathering smolt statistics for use with the upcoming graphics test days? 16:52:39 ooh, nice one 16:52:41 This was something rbergeron raised while we were discussion the upcoming gnome-shell changes 16:52:51 bskeggs and i have discussed it too 16:52:57 you kinda need direct DB access to do that efficiently 16:53:07 there is a gold mine of data in smolt, but we just need someone with a few spare cycles to dig through and present some queries 16:53:10 yeah, dealing with graphics adapters is tricky as there's so many of the damn things 16:53:13 Viking-Ice: maybe, maybe not 16:53:37 jlaska: it is quite tricky from the web front end as what we need to do here is reduce several thousand separate adapters into a few broad categories 16:53:39 I'm sure ro db access could be granted if our needs were clearly articulated 16:53:46 so say for nvidia 16:53:47 adamw: definitely! 16:53:48 first question what exactly is the information trying to be gather for 16:53:58 what ben's interested in is 'how many people are using each generation of nvidia card' 16:53:58 Viking-Ice: great question :) 16:54:17 so then we can use the test days to evaluate how well gnome-shell works on each generation 16:54:19 and once you have that data what are you planning to do with it 16:54:35 yeah, I'd love to see a high-level list of all the variations of cards in those big 3 (nvidia, intel, ati) 16:54:39 and then that in turn will let us say 'okay, looks like we'll be able to get gnome shell working for 70% of nvidia users' or whatever it turns out to be 16:55:00 ben can say quite confidently 'okay, it's basically working on generations X, Y and Z' 16:55:09 Viking-Ice: as adamw said, it's to assist qa and devel with release decisions and bug decisions 16:55:13 but we need smolt to let us know how many of the cards in use are in those generations 16:55:20 yup 16:55:33 so ... skills required here? 16:55:40 as i said, the icky bit is that there are dozens of separate PCI IDs for each generation, probably hundreds for some 16:55:51 it's mostly data manip 16:55:54 so let's say we fetch all graphics cards info then what I mean do the graphics driver developers have those cards handy ? 16:56:16 if they dont we cant reliably say this works on x y z 16:56:18 figure out a way to get a raw dump of all graphics cards out of smolt, then categorize them into generations, then set up queries that abstract out the generations, so we can just say 'find all cards in NVIDIA generation foo' 16:56:29 * jlaska info's 16:56:32 #info figure out a way to get a raw dump of all graphics cards out of smolt, then categorize them into generations, then set up queries that abstract out the generations, so we can just say 'find all cards in NVIDIA generation foo' 16:56:32 Viking-Ice: at the driver level that's more or less how it works 16:56:54 Viking-Ice: there may be isolated bugs with particular cards but that's more commonly a 'it doesn't work at all' thing than an OpenGL-level issue 16:56:57 okay, that's all I had on this topic ... just humble plea for volunteer(s) on something that's pretty important for this next release 16:57:23 Viking-Ice: as far as mesa goes, any nvidia 8xxx card works pretty much the same as any other nvidia 8xxx card 16:57:38 I see 16:57:44 next quick topic is more of a reminder ... 16:57:45 so if we test and find that gnome shell is working well on five or six 8xxx cards, we can be pretty sure it's working fine across that generation 16:57:49 * jlaska waits 16:58:08 and the same thing applies for intel and radeon 16:58:17 some other 8xxx cards may hit some bug at the kernel / DDX level which means EDID retrieval is broken or they don't activate the right video output or whatever, so X doesn't work...but there's always been those isolated bugs. 16:58:17 adamw: thanks, you articulated my concerns better than I could have :) 16:58:17 Viking-Ice: yup 16:59:04 ye might wanna post a bit more detail on how you want this data to be delivered 16:59:10 alright, in the interest of time ... I'm moving on. But we can continue this on the list or in #fedora-qa 16:59:12 Viking-Ice: yeah, might be a good topic for a blog post 16:59:18 good idea 16:59:31 #topic Reminder - 2010-01-27 - Network Device Naming test day 16:59:40 #link https://fedoraproject.org/wiki/Test_Day:2011-01-27_Network_Device_Naming_With_Biosdevname 16:59:48 just a reminder our first F15 test day is coming up next week 17:00:06 and this applies to what Dell and HP hw ? 17:00:12 more or less 17:00:23 i think specific dell/hp hw 17:00:32 i think the kinda corporate lines more than the home stuff...but imbw 17:00:46 I've been coordinating with Narendra on the wiki, and I've got several updates to the wiki I'd like to do 17:00:47 yeah pizza boxes blades and bricks 17:00:55 * adamw was about to suggest to the test organizer that they include a bit more 'user friendly' stuff on how to know if you have relevant hw :) 17:00:58 namely related to adamw's new wiki format 17:01:06 adamw: can you add that to discussion tab? 17:01:13 jlaska: i'm going to put it in the trac ticke 17:01:14 t 17:01:19 even better, thanks! 17:01:35 yeah and dont forget to mention that when advertising the test day so we dont end up wasting reporters time 17:01:41 i was just going through the test day page when the graphics card thing came up =) 17:01:43 Viking-Ice: yup 17:01:47 certainly! 17:01:56 okay, that's it from me 17:02:04 #topic Open discussion - 17:02:10 anything else not already discussed? 17:02:15 I got one more 17:02:28 Viking-Ice: can you keep it short? 17:02:50 I haven't put much time into the bugzilla enhancement lately, but I am still planning on getting it to work. 17:02:51 if so ... bring it, otherwise we can follow-up on list 17:02:51 We might wanna advertise our services to maintainers as we did for a short while ( more of a reminder ) 17:03:06 Viking-Ice: what do you have in mind? 17:03:23 just announcement mail to devel etc 17:03:30 brunowolff: no worries. I admit I'm not fully up to speed on the changes, and how we'd get them enabled in our bugzilla.redhat.com instance :( 17:03:40 Just what we did before 17:04:32 Yeah, it's more of a long term project, which is why I bumped it for some short term Fedora projects I have been working on recently. 17:04:46 Viking-Ice: sorry, I'm drawing a blank here ... got an example link? 17:05:11 not handy I need to dig through the archives 17:05:19 to find it 17:05:22 okay 17:05:33 somewhere around the time we setup the QA trac instances 17:05:46 #topic Open discussion - Post QA team services to devel-announce@ 17:06:06 #info Viking-Ice suggested someone post to devel-announce@ to remind about services offered by QA team 17:06:25 Viking-Ice looking through archives to find sample post 17:06:34 will follow up on -test list 17:06:39 okay 17:06:55 alright folks ... let's close the meeting in 2 minutes 17:08:13 closing meeting in < 1 minute 17:08:41 T-15 seconds ... 17:08:52 Thanks everyone for your time, I'll send minutes to the list shortly 17:08:55 #endmeeting