15:00:05 #startmeeting Fedora Quality Assurance 15:00:05 Meeting started Mon Apr 19 15:00:05 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:07 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:09 #meetingname fedora-qa 15:00:10 The meeting name has been set to 'fedora-qa' 15:00:24 #topic Waiting for participants ... 15:00:32 * maxamillion is here 15:00:51 fenris02: maxamillion greetings! 15:00:52 * kparal waves 15:01:18 g'morning * 15:01:20 * jlaska salutes kparal and jskladan 15:01:27 * jskladan ready to serve his master 15:01:33 lol 15:02:21 haha, whatever ... I'm here to serve you! 15:03:25 a few more minutes ... waiting for wwoods and adamw 15:03:43 * wwoods poof 15:03:53 always with the grand entrance! :D 15:04:07 its all about presentation :) 15:04:25 indeed 15:05:08 it might be early still for adamw ... or he's still recovering from the blocker meeting on Friday 15:05:31 let's get started, and any straglers can join as we go 15:05:39 #topic Previous meeting follow-up 15:05:52 Good news ... I've got nothing in the previous meeting follow list 15:06:01 +1 15:06:04 Bad news ... I've moved it all to the agenda 15:06:06 :) 15:06:22 awwww 15:06:23 -1 15:06:27 net zero! 15:06:28 :) 15:06:33 ? 15:06:56 so, let's get started ... we've got a few check-ins to cover today 15:07:21 I'll try to keep this under an hour so we can all get back to being productive Fedora contributors 15:07:31 #topic Fedora 13 RC test status 15:08:04 first up ... just a big thank you to those who helped get the Beta in shape for posting to mirrors 15:08:37 the desktop and install matrices (and release criteria) seem to be helping keep focus on the issues that will impact getting the bits out 15:08:42 +1 ... the beta has been awesome! 15:09:03 of course, now for our most important QA milestone yet :) 15:09:07 the release candidate 15:09:23 15:09:34 * kparal inserts thunder 15:09:35 dun dun dunnnn! 15:09:48 I'll list a few highlights from the QA schedule (http://poelstra.fedorapeople.org/schedules/f-13/f-13-quality-tasks.html) 15:10:12 #info Blocker review - Last friday we had our first F-13-RC blocker meeting, and another is scheduled for this week 15:10:29 #link http://poelstra.fedorapeople.org/schedules/f-13/f-13-quality-tasks.html 15:10:46 #info Acceptance test - This thursday a test run is scheduled for a new post-beta anaconda build 15:11:12 other significant milestones ... 15:11:19 #info 2010-04-29 - Test 'Final' Test Compose 15:11:26 #info 2010-05-06 - Test 'Final' RC 15:11:40 am I missing anything there? 15:11:45 * jlaska notes ... test days ... 15:12:00 2 installation related test events coming up 15:12:29 Ideally this will help shake out any remaining issues with the storage UI rewrite ... and also put the install test matrix in good shape for the RC 15:12:44 #info 2010-04-22 - https://fedoraproject.org/wiki/Test_Day:2010-04-22_StorageFiltering 15:12:53 oh, random side note on test days, i was wanting to have a test day based around xfce spin and around gnome-shell (separate obviously) .... I'd like to move forward and schedule a test day for the xfce spin 15:13:00 #info 2010-04-29 - Preupgrade test day (https://fedoraproject.org/wiki/Test_Day:2010-04-29_Preupgrade) 15:13:10 if we find some blocker bugs on April 29th in Preupgrade test day, what then? 15:13:35 but the gnome-shell test day I think I'm going to not move forward on as there seem to be some driver specific issues that i think would skew the results of the test day 15:14:15 kparal: yeah, the timing corresponds with the 'test compose', so we'll have to view any bugs in light of the release criteria ... and raise exceptions as needed 15:14:27 maxamillion: awesome, would these be for F-13 (or after)? 15:14:33 * jlaska points to https://fedoraproject.org/wiki/QA/Test_Days/Create 15:14:56 jlaska: for F-13 ... I'd like to move forward on the gnome-shell stuff targeted towards F-14 since that will be the default then 15:15:13 maxamillion: okay 15:15:38 cool, given the graphics bling ... I imagine that'll be a well attended event 15:15:56 Only other F-13 testing point I wanted to make ... 15:16:21 #info F-13 QA retrospective - now accepting ideas for improvement https://fedoraproject.org/wiki/Fedora_13_QA_Retrospective 15:16:33 gnrrk? 15:16:35 why is the meeting now? 15:16:47 I know most of you have already been recording what worked, what didn't etc... so thank you for that 15:17:12 adamw: last discussed, we agreed to have meeting follow local time, not UTC 15:17:29 oh, didn't know you changed it already :) 15:17:37 yeah, I could have publicized that better 15:17:49 * jlaska just did it 15:17:53 adamw: I didn't either, I was just around ;) 15:18:12 okay, any other F-13 test thoughts before moving on? 15:18:15 meetings need an ical link imho 15:18:41 fenris02: when the fedora calendar solution is online, we'd certainly look to integrate with that 15:18:55 f13b is quite distant from an 'update'd system now. 15:19:12 jlaska: fedora calendar solution - what's that? 15:19:25 a lovely lovely pony 15:19:30 #link https://fedoraproject.org/wiki/XFCE_Test_Day:Tentative 15:19:43 * maxamillion needs to schedule that but figured I'd link about it 15:20:03 kparal: there are older threads on infrastructure list ... but adamw sums it up well 15:20:08 moving on ... 15:20:09 #topic Bodhi QA workflow status check-in 15:20:14 can we get a BFO image for f13b? 15:20:44 fenris02: yes we can ... we'll need ot file a request with infrastructure team to update their boot.fp.org config 15:21:06 fenris02 are you able to follow up on that? 15:21:24 -enoclue ... but i'll take it offline 15:21:26 I thought there was BFO for F13Beta: http://fedoraproject.org/get-prerelease 15:21:34 #info fenris02 asked how to update boot.fedoraproject.org with F-13-Beta 15:22:07 kparal, it is listed, but last i looked the image did not give you the f13 option 15:22:36 ok 15:22:54 http://git.fedorahosted.org/git/?p=fedora-infrastructure.git;a=blob;f=bfo/pxelinux.cfg/fedora_install.conf;h=26acb46afa285f999356c912305395294240b0dd;hb=HEAD 15:23:10 looks like a config exists in the pxelinux config for boot.fp.org 15:23:14 not sure if that change is live yet 15:23:22 let's follow-up after 15:23:45 #action jlaska + fenris02 to review whether F-13-Beta is an option for boot.fedoraproject.org 15:23:51 thanks 15:24:01 fenris02: thanks for raising the question 15:24:04 okay ... back to /topic 15:24:25 adamw, do you have any updates on this front? 15:24:53 nothing springs to mind. 15:25:15 (though my mind feels rather like goo so it's entirely possible i'm forgetting something) 15:25:31 ah, monday mornings :) 15:25:56 The item I was tracking here was identifying the bodhi2.0 roadmap so we can plan around it 15:26:20 s/around/for/ 15:26:57 lmacken: around? 15:27:34 oh, yeah. actually i see emails from luke and mathieu about that in my inbox, from yesterday. haven't read 'em yet. 15:28:12 okay ... then let's move on for now and we can circle back outside the meeting 15:28:20 objections/concerns? 15:28:27 looks like they're happily making things work but have not in fact answered my questions yet :) 15:28:58 heh, okay ... we like forward progress! 15:29:08 so, recap: I asked them when they're actually going to be able to implement the improved feedback system in a version of bodhi the rest of us are using 15:29:30 possible tangent ... are there requirements for this next version somewhere? 15:29:39 right now mathieu is working on it in the 'tg2 branch' (turbogears 2), which seems to be the sort of tinkering playground 15:30:18 #info adamw asked lmacken and mathieu for a heads up as to when the improved feedback system will be available 15:30:24 jlaska: they've been working off mine and doug's emails as a template, i believe 15:30:40 okay 15:31:00 alright, thanks for the update adamw 15:31:07 shall we move on? 15:31:09 sure 15:31:18 maxamillion: I've got you up next .. 15:31:22 not sure if you're still around 15:31:23 #topic Proventesters check-in 15:32:00 #info As a nice April fools joke, maxamillion sent out the proventesters draft (http://lists.fedoraproject.org/pipermail/test/2010-April/089847.html) 15:32:34 we have have lost maxamillion ... he's double booked at the moment 15:33:22 anyone else know what's next for the current provent testers Draft? 15:33:39 * jlaska wonders if this is dependent on Bodhi workflow changes too 15:34:34 shouldn't be 15:34:51 we can go ahead with proventesters without the new feedback stuff. in fact we should, we're still relying on the qa+releng bodge right now 15:35:18 right 15:35:34 okay ... we'll come back to this later in the meeting, or queue it up for next week 15:36:13 This seems like a critical piece if we want to solicit test assistance against critpath outside just this small group 15:36:20 #topic Package Acceptance Test Plan check-in 15:36:32 We've not talked about the package acceptance test plan in a while 15:36:55 but I wanted to bring it back up since the package update acceptance criteria have been finalized (https://fedoraproject.org/wiki/Package_update_acceptance_criteria) 15:36:56 #link https://fedoraproject.org/wiki/User:Kparal/Proposal:Package_update_acceptance_test_plan 15:37:16 kparal has a killer draft test plan (see link above) out for review 15:37:48 My first question ... is this something we want to track completion prior to F-13 ... or is this a post F-13 task? 15:38:39 well I don't think we can implement this before F13 15:38:59 kparal: when you say implement ... do you mean the test plan and test automation? 15:39:39 jlaska: yes. sure we can have a plan, but it will take long time before we do it with AutoQA automatically 15:40:17 oh definitely, while the steps in your draft could be done manually ... we'll never scale for that 15:41:15 should we move this out of draft ... and track automation separately? 15:41:23 or ... keep it in draft until all automation is complete? 15:42:01 jlaska: we can move it out of draft when we feel it's complete 15:42:12 we don't have to wait for AutoQA implementation 15:42:19 okay 15:42:22 sorry, got called afk 15:42:47 kparal: what do you think is remaining, or ... what needs to be addressed to move it out of draft? 15:43:03 maxamillion: no worries, I can switch back to that topic shortly if you like 15:43:33 I was in a $dayjob meeting as well as this one and got attention called away from the netbook 15:43:38 jlaska: well I would like someone credible to say it's ok :) 15:43:55 jlaska: its up to you ... I skimmed the backlog and it looks like we're ready to move forward with the proposal 15:44:07 but I don't know if we need some approval process or something 15:44:30 I think it's more QA concern 15:44:39 kparal: yeah, I agree 15:45:00 it seems good to me 15:45:05 kparal: I wouldn't call myself credible ... but I'll be happy to reread and send thoughts to the list 15:45:29 so, let's have a final conversation over it and we are satisfied, we can mark it as final 15:45:36 this problem space is still a moving target in some respects ... so it wouldn't surprise me if the test plan were to adjust as we learn 15:45:47 kparal: sounds like a plan 15:45:58 sounds like an action item for me :) 15:46:14 #info kparal asked for additional input on the package update acceptance test plan 15:46:25 kparal: are you able to send that out to the list this week? 15:46:34 action item - kparal will bother other QA members if they have some comments to the test plan :) 15:46:46 #action kparal will bother other QA members if they have some comments to the test plan :) 15:46:53 hehe 15:47:08 kparal: awesome, thank you :) 15:47:21 alright 15:47:32 kparal: anything else you want to track or note? 15:47:38 like i said, the plan looks fine to me 15:47:56 jlaska: no, that's fine 15:48:09 okay ... jumping back ... 15:48:10 #topic Proventesters check-in 15:48:33 #info maxamillion notes, "it looks like we're ready to move forward with the proposal" 15:48:52 should we start up a vote? or more review needed? 15:49:08 i don't think it really needs voting on? that's not how we've usually implemented policies 15:49:20 oh ok 15:49:24 i'd say just send a note to the list that it's the final draft and it'll go into effect unless anyone has serious objections 15:49:25 maxamillion: how do you feel about the QA mentor aspect ... is that established or well known? 15:49:30 I'm new to this :) .... its my first policy draft 15:49:38 maxamillion: same here :P 15:49:52 did you remove the CLA requirement as per jesse's feedback? 15:50:06 "It is compulsory to sign the CLA (contributor licence agreement) for participating in this group. " 15:50:18 jlaska: well if it were something that required FESCo approval or something obviously we'd have to send it to them, but AFAICT it isn't. this is entirely within QA's domain... 15:50:43 adamw: okay 15:51:05 so once this is finalized ... what's next? 15:51:09 adamw: no, that I had not 15:51:15 * link to kparal's package update acceptance test plan? 15:51:25 * link to [[Category:Test Cases]] for test guidance? 15:51:28 adamw: I left that just because it was pulled from the ambassadors 15:51:40 * request proventester FAS group ... and start managing this sucker? 15:52:20 sounds about right 15:52:34 maybe call out on the list for people to apply 15:53:16 +1 15:53:48 jlaska: if you'd list, just make some action items for me and I'll take care of that stuff :) 15:54:01 maxamillion: oh boy, carte blanche for action items! 15:54:19 maxamillion: so you're sending the draft to the list for final comments? 15:54:23 maxamillion: my god. what have you done? 15:54:30 unleash the tasks! 15:54:33 lol 15:54:52 #action maxamillion - send Proventester draft to test@l.fp.org for final review 15:55:04 #action Request new proventester FAS group 15:55:05 jlaska: yeah, send for final comments, then put in request to infrastructure/rel-eng to have the appropriate changes made on their end for the group 15:55:13 #undo 15:55:14 Removing item from minutes: 15:55:25 #action maxamillion - Request new proventester FAS group 15:55:49 that should do it for now ... thanks maxamillion! 15:56:04 okay ... last topic for today ... 15:56:07 #topic AutoQA check-in 15:56:34 wwoods: kparal and jskladan have a lot of updates on the AutoQA front from last week 15:56:50 I didn't want to spend a lot of time reviewing the details here ... but hopefully each of you can point out the highlightzx 15:56:53 hightlights 15:57:01 sure 15:57:04 and links for additional info in case others want to participate or contribute? 15:57:12 who wants to go first? 15:57:24 jskladan :) 15:57:29 * jskladan me. me me me me :) 15:57:34 go man go! 15:57:40 hehe 15:57:44 jskladan: go for it my good man 15:58:03 * jlaska will follow with the annoying meetbot tags 15:58:09 first of all, we finally managed to push Autotest test tag (id in database and some other stuff which identifies testrun) to the autotest-client 15:58:13 although, feel free to use #link and #info 15:58:24 i.e. to the AutoQA tests 15:58:52 there is some discussion going on about the patch in mailing list 15:59:10 but hopefully it will be upstream until the end of the week 15:59:31 #info Test results sent to autoqa-results will include a link to the autotest results for additional details (patches under review) 15:59:41 #info Getting URLs of test results from AutoQA testruns is nearly finished 15:59:51 heh 15:59:53 #undo 15:59:55 jskladan: cool, nice work! and thanks to _lmr_ for the tip! 16:00:08 <_lmr_> :) 16:00:20 <_lmr_> I am so glad it worked out OK 16:00:41 other that that - beakerlib based tests (initscrip testing) finally got to the production machine, and are producing results 16:00:52 * jlaska grabs link ... 16:01:18 #link https://fedorahosted.org/pipermail/autoqa-results/2010-April/016011.html 16:01:28 upstart, sysvinit or both types of initscripts? 16:01:46 #info the beakerlib-based initscript test is now in production and sending results 16:02:00 sysV i guess - it's testing LSB compliance of return codes 16:02:10 * Oxf13 peeks in 16:02:25 Oxf13: welcome :) 16:03:10 and finally, we made a progress on the ResultsDB front. We have finished the db schema and accepted a input API 16:03:26 #info made a progress on the ResultsDB front. We have finished the db schema and accepted a input API 16:03:45 and we'll start prototyping the application this week 16:04:04 i'll set up a XMLRPC based API and hack few tests to put results into the ResultDB 16:04:19 exciting stuff! 16:04:48 next important step on the ResultsDB road will be creating a frontend which will publish the data 16:05:27 which will help us in proposing the correct API for digging data from ResultsDB 16:05:45 it's important to note that we're designing the resultsDB backend to be a unified storage backend for *multiple* frontends for various types of tests 16:06:12 #info Next steps ... prototyping the resultsdb and using an XMLRPC based API to populate with results 16:06:31 * jskladan passing the talk-stick to kparal & wwoods 16:06:36 #info Next steps ... Creating a front-end which will be used for review/publish the data 16:06:42 jskladan: thanks for the updates :) 16:06:45 there may not ever be a generic resultsdb interface/frontend - and that's intentional 16:07:06 wow, jskladan said almost everything 16:07:13 I think just PST remains for me 16:07:19 jskladan: stole the thunder! 16:07:40 well then, Package Sanity Tests 16:07:41 wwoods: good point ... I don't see a General Test Plan on the horizon any time soon :D 16:07:53 * jlaska once again encourages excessive use of #info 16:08:00 here are some links 16:08:04 #link https://fedoraproject.org/wiki/Test_Package_Sanity 16:08:09 #link https://fedorahosted.org/autoqa/milestone/Package%20Sanity%20Tests 16:08:29 Package Sanity ensures clean install, removal, upgrade, etc of the packages 16:09:08 there is already a 'pst' script in the repository which you can try to perform sanity tests on a downloaded package or package in a repo 16:09:37 (altough there is now a bug reported that downgrade test fails) 16:09:41 * jlaska sees a autoqa pst branch ... cool! 16:09:48 I work on this with Petr Splichal 16:10:15 #info code for sanity tests is in AutoQA pst branch 16:10:25 #link http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/pst 16:10:48 the autoqa wrapper is not in yet, but I'm working on some basic proof of concept 16:11:05 this will depend on the new bodhi features and the whole depcheck stuff wwoods is working on 16:11:24 we need to 1) detect that new updates is pushed to bodhi 16:11:26 * jlaska likes the sample test output @ http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/package_sanity/examples/cheese-rpm-summary.txt;h=830a0a9311eb4e8442fd3858c0ad67ce694ad6c9;hb=refs/heads/pst 16:11:34 and 2) download the package and all relevant packages 16:12:24 so this is pretty related to stuff wwoods is working on with lmacken 16:12:42 alright, that would be it for today, passing microphone along... 16:12:44 kparal: bonus points for a good segway :) 16:13:01 kparal: thanks for the updates ... another cool use of beakerlib :) 16:13:41 wwoods: any highlights from your AutoQA realm? 16:15:08 uh oh ... did we lose wwoods? 16:16:19 quick ... this means wwoods gets the remaining action items that we were saving for maxamillion 16:16:28 :D 16:17:01 * jskladan would like to pity wwoods, but is afraid, that the action items woul fall to his head then :-D 16:18:08 alright, looks like we lost wwoods ... hopefully he can follow-up on the list with any highlights 16:18:20 that's it for the agenda today ... open discussion time 16:18:23 #topic Open discussion - 16:19:11 If no comments or concerns, I'll #endmeeting in 1 minute 16:19:24 ah! 16:20:10 I don't have any real developments to report WRT autoqa; most of last week was discussions about bodhi updates for depcheck etc. and resultsdb discussion (which jskladan covered) 16:20:51 I think we have enough info about current bodhi to write a update watcher, but it may have some shortcomings and will soon be obsoleted 16:20:56 * jlaska amazed at how complex depcheck has become (wrt rejecting bodhi updates that fail acceptance criteria) 16:21:21 it's a pretty complicated problem overall 16:21:33 but solving it allows us to do a lot of similar testing 16:21:45 so it's worthwhile 16:21:46 yeah, your discussion w/ lmacken + Oxf13 last week definitely opened my eyes to that 16:22:16 * jlaska info's 16:22:33 what's funny is all the people who have been saying "It's so easy, why aren't you doing it already?" 16:22:42 #info AutoQA depcheck - we have enough info about current bodhi to write a update watcher, but it may have some shortcomings and will soon be obsoleted 16:22:42 funny, with a capital FU 16:23:02 Oxf13: :D 16:23:10 wwoods: alrighty, thanks for the updates 16:23:17 if it's so easy, why hasn't anyone done it yet? or why haven't YOU done it, Mr. Hypothetical Question Guy 16:23:33 I haven't heard any additional thoughts discussion topics ... and we're well over hour time slice 16:23:40 oh also - did I mention this last week? - we have a proposed storage encoding for putting test plan metadata in the wiki 16:23:51 and some code to pull that data back out as nice python dicts (or whatever) 16:24:13 #info AutoQA - we have a proposed storage encoding for putting test plan metadata in the wiki, and some code to pull that data back out as nice python dicts 16:24:29 see http://fedoraproject.org/wiki/QA:Test_Plan_Metadata_Test_Page 16:25:13 https://fedorahosted.org/autoqa/ticket/137 has the link to the parsing code 16:25:35 that's all I can think of, and yeah, we're way in overtime 16:25:47 wwoods: thanks for the links 16:26:00 okay gang, thanks for your time ... go in peace! 16:26:06 #endmeeting