16:00:35 <adamw> #startmeeting 2009-09-28 QA meeting
16:00:35 <zodbot> Meeting started Mon Sep 28 16:00:35 2009 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:42 <adamw> #topic gathering bodies
16:00:44 <adamw> who's here?
16:00:46 * jlaska 
16:00:59 * Viking-Ice reappears after being away for what a month?
16:01:09 <adamw> welcome back!
16:01:29 <Viking-Ice> Thank you..
16:01:35 <adamw> Oxf13: wwoods: dpravec: ping
16:01:53 <Oxf13> I'm in jury duty so not quite here
16:02:05 <Viking-Ice> Guilty.. .
16:02:28 <adamw> yeah, hang 'em high :)
16:02:40 <Oxf13> Haha
16:03:03 <adamw> any other bodies?
16:03:17 <jlaska> adamw: I think kparal and dpravec might be out today due to holiday
16:03:24 <adamw> ok
16:03:29 <adamw> well, short meeting coming then :)
16:03:38 <adamw> jlaska: any idea what happened to wwoods?
16:04:23 <jlaska> adamw: he's around ... might be away from irc
16:04:37 <adamw> if you could grab him for an autoqa update that'd be good
16:04:51 <jlaska> grabbing inprogress
16:05:11 <adamw> ok
16:05:17 <adamw> #topic previous meeting follow-up
16:05:27 * wwoods appears
16:05:50 <jlaska> queue the music and smoke machines!
16:06:01 <adamw> so here I was hoping to follow up on the discussion from last week around development process problems
16:06:14 <adamw> but quite a bit of that was on oxf13 so maybe next week...
16:06:50 <adamw> denise: anything on that from your side?
16:07:45 <denise> not really
16:07:53 <adamw> OK...thanks
16:08:09 <adamw> then let's dim the lights, hit the smoke machine and turn the floor over to wwoods for the...
16:08:11 <adamw> #topic autoqa update
16:08:20 <adamw> wwoods: from the last two weeks if you please :)
16:08:43 <wwoods> well, I'll sum up in a screenshot
16:08:49 <wwoods> http://wwoods.fedorapeople.org/screenshots/irb.png
16:09:06 <adamw> legendary!
16:09:13 <wwoods> the rats_* tests submit those results when they complete
16:09:14 * jlaska goes to locate socks again ... stop showing that screenshot! :)
16:09:38 <wwoods> (i've actually added two new tests - anaconda package install and anaconda bootloader setup - since that screenshot)
16:09:50 <denise> that is beautiful!
16:09:53 * adamw hands jlaska some sock suspenders
16:10:01 <wwoods> it's a TG1 webapp - source is here: http://fedorapeople.org/gitweb?p=wwoods/public_git/israwhidebroken.git;a=summary
16:10:03 <jlaska> wwoods: nice, is that for additional granularity for the progress of anaconda?
16:10:29 <wwoods> jlaska: kind of - the tests (as defined previously) did not cover the installation process after anaconda probed the disks
16:10:40 <jlaska> oh that's right, thx
16:10:46 <wwoods> so if anaconda started up and found the disks, then crashed.. that was still considered all-pass
16:11:01 <wwoods> so i've extended coverage to the end of the installation process
16:11:47 <jlaska> awesome, very nice indeed
16:11:53 <wwoods> the webapp authenticates off FAS so anyone in the qa group can modify the results for the manual tests
16:12:06 <wwoods> this week I'm working on adding links to the pass/fail results that let you view the test logs for automated tests
16:12:35 <wwoods> we'll try to have this running in production on infrastructure hardware very soon
16:13:00 <wwoods> there's a couple stupid bugs to work out - notably when you run tests late (like all the PPC tests that ran today) they get reported as being for the *current day's* rawhide tree
16:13:04 <jlaska> wwoods: a quick glance shows that the hardware PO is moving along ... I'll see if we have an ETA later today
16:13:06 <wwoods> rather than the day the tree was actually built
16:13:08 <adamw> now you just need to make the thing file automatic bug reports and we can all spend all week in our underwear drinking beer
16:13:12 <wwoods> so suddenly today I have a bunch of ppc results
16:13:24 <wwoods> from a ppc tree built last week. oops.
16:13:41 <wwoods> anyway we'll file down a couple of those rough edges and get it into production ASAP
16:13:51 <wwoods> except the links to logs won't be that useful until autotest itself is also in production
16:14:16 <adamw> i'm thinking this point may be blog post-worthy?
16:14:24 <wwoods> so we might end up doing a stupid stopgap where we dump the logs somewhere public so people can peruse them
16:15:03 <wwoods> oh I also need to fix it up so you can add a bug ID for failed tests
16:15:11 <wwoods> so we can track the cause of failures
16:15:44 <wwoods> hm. yeah, might be worthy of a blarg post
16:15:54 <wwoods> oh, I need to finish up the test cases for the two new tests
16:15:59 <jlaska> wwoods: is this service something that could run on the same host as the autotest scheduler/autoqa tests?
16:16:31 <wwoods> jlaska: sure, just need to work out how you set up TG stuff in production like that
16:16:37 <jlaska> okay
16:16:54 <wwoods> oh, two new test cases:
16:16:58 <wwoods> #link https://fedoraproject.org/wiki/QA:Anaconda_package_install_test_case
16:17:03 <wwoods> #link https://fedoraproject.org/wiki/QA:Anaconda_bootloader_setup_test_case
16:17:05 <denise> so the link to logs is where the anaconda team can go to see what died?
16:17:10 <wwoods> denise: yep
16:17:29 <wwoods> currently that all lives on an internal machine, but since most of the anaconda folks are inside the RH firewall
16:17:33 <wwoods> they can look at that stuff now
16:17:39 * dpravec is here
16:17:41 <wwoods> it just takes a little bit of digging
16:18:16 <denise> i see a new morning routine ;-)
16:18:52 <wwoods> yeah it's pretty nice to come in and pop up the dashboard and see what's working
16:19:11 <jlaska> here here
16:19:26 <wwoods> so that's autoqa/israwhidebroken status
16:19:40 <adamw> excellent
16:19:42 <adamw> thanks wwoods
16:20:01 <adamw> anyone have any other comments / questions on autoqa?
16:20:19 * denise claps wildly
16:20:32 <adamw> =)
16:20:38 <wwoods> hah
16:20:45 * jlaska still trying to locate socks
16:20:54 <wwoods> once this bit is in production we'll start looking at things like
16:21:07 <wwoods> - keeping a constant "last-known-good" symlink to the last tree that passed all the tests
16:21:16 <adamw> wwoods: jlaska: like i said, just make sure no-one tells jay about this thing and we can go on permanent vacation ;)
16:21:37 <dpravec> wwoods: awesome job
16:21:44 <adamw> hi dpravec
16:21:55 <wwoods> - possibly using the results of the testing to reject builds that cause the tests to fail
16:22:21 <wwoods> and then we want to extend this whole idea to doing per-package  package-level functional testing
16:22:45 <jlaska> that might work nicely with the efforts kparal and psss are investigating (wrt to automating rpmdiff runs)
16:22:54 <wwoods> errr. per-package testing - package-level dependency/conflicts stuff and functional testing - and using that to decide whether to tag new builds
16:23:00 <wwoods> but that's all a long way down the pipeline
16:23:14 <wwoods> but I mean, this project is really only ~90 days old, and we're already pretty far along
16:23:18 <adamw> yeah indeed
16:23:36 <wwoods> so give it a few months but I think we can probably get there
16:24:22 <dpravec> wwoods: i am investigating how to test anaconda by simulating user input with dogtail..  this should be included also
16:25:17 <wwoods> write the tests first and then we'll figure out how to get it integrated
16:26:40 <dpravec> yes will do
16:29:25 <wwoods> that goes for anyone wanting to do autoqa stuff: write the test first. everything else will get figured out eventually.
16:29:37 <adamw> ok.
16:29:39 <adamw> moving on!
16:29:43 <adamw> #topic upcoming events
16:30:07 <adamw> so we're into beta phase for sure now
16:30:32 <adamw> this week we have the beta freeze and candidate build process, and the fourth blocker bug review day on friday
16:30:42 * jlaska points to http://poelstra.fedorapeople.org/schedules/f-12/f-12-quality-tasks.html
16:30:46 <adamw> yup
16:30:47 <jlaska> err ...
16:30:49 <jlaska> #link http://poelstra.fedorapeople.org/schedules/f-12/f-12-quality-tasks.html
16:31:11 <adamw> i don't think there's any real action items for us, except to test the beta candidates when they show - i think we're looking in decent shape now last week's excitement has calmed down
16:31:36 <jlaska> Liam has a ticket open requesting a Beta build from rel-eng iirc ...
16:32:04 <jlaska> I believe Oxf13 is already in the loop on that
16:32:06 <jlaska> #link https://fedorahosted.org/rel-eng/ticket/2191
16:32:09 <denise> there are a few blockers in mod that would appreciate retest if anyone has time
16:32:51 <adamw> yep
16:33:05 <jlaska> I haven't circled fedora-test-list fully yet, was there a summary of the 'Test Compose' results sent?
16:33:12 <adamw> https://bugzilla.redhat.com/showdependencytree.cgi?id=507678&hide_resolved=1
16:33:23 <adamw> no...it got a bit messy with the lvm stuff
16:33:52 <jlaska> adamw: gotcha
16:33:59 <adamw> the only bug showing in modified atm is https://bugzilla.redhat.com/show_bug.cgi?id=516042 (anyone with access to an NFS rawhide repo should be able to test that)
16:34:00 <buggbot> Bug 516042: medium, low, ---, rvykydal, MODIFIED, Unable to add NFS yum repo during installation
16:35:41 <jlaska> I should be able to test that once situated here
16:35:46 <adamw> great
16:36:20 <denise> there was a related one blocking testing, IIRC - the fix was just reviewed and will go in tomorrow when Brno returns
16:38:10 <adamw> right.
16:38:19 <adamw> it was blocking for the initial reporter, but may not block for anyone else testing iirc.
16:38:26 <adamw> (was somewhat specific to his situation)
16:38:41 <adamw> so, that's upcoming events...jlaska, anything to add there?
16:39:01 <jlaska> we've got a install-related test day planned for this thursday as well
16:39:13 <jlaska> #link https://fedoraproject.org/wiki/Test_Day:2009-10-01_Anaconda/Features/StorageFiltering
16:39:36 <adamw> oh yes, missed that somehow :)
16:39:42 <denise> hmmm - i think we asked to cancel that one
16:39:54 <denise> because the filtering is UI and UI changes not in til F13
16:40:14 <jlaska> oh ... my impression of this test day was all the F11 storage bugs
16:40:18 <denise> clumens was suppost to talk with someone who had sent test cases
16:40:32 <denise> if storage bugs, thats fine -
16:40:42 <denise> the filtering part not there
16:40:59 <adamw> so far i've been writing it up as if it were a generic storage test day
16:41:04 <adamw> and the test cases seem to be geared to that
16:41:12 <denise> ok, thats fine then
16:41:12 <adamw> so we could tweak the page name a bit and go ahead i think
16:41:23 <denise> go for it ;-)
16:41:24 <jlaska> with some focus on the storage cleanup bugs listed https://fedoraproject.org/wiki/Anaconda/Features/StorageFiltering#Detailed_Description
16:41:29 <denise> more storage testing is better
16:41:40 <denise> will be sure developers are there to help
16:41:49 <denise> lets hope we have a rawhide
16:41:58 <jlaska> denise: storage test days always need some lead up time to ensure things have settled for the test day
16:42:17 <jlaska> denise: are there any things we can line up ahead of time to ensure anaconda related packages are in a testable state?
16:42:47 <denise> jlaska, i'll ask dlehman towork with you on this ok?
16:43:16 <jlaska> sure, let's include rhe on this, she is organizing the test day
16:44:44 <denise> rhe - in beijing?
16:44:51 <jlaska> denise: yeah
16:45:04 <denise> clumens may have asked her to cancel - we should doublecheck
16:45:14 <jlaska> I'll circle the wagons
16:45:58 <denise> thanks
16:46:33 <adamw> ok
16:46:40 <adamw> i think that's everything on the list...
16:46:52 <adamw> so, time for
16:46:55 <adamw> #topic open floo
16:46:59 <adamw> #topic open floor
16:47:06 <adamw> oh dear, my floo is open - how terribly embarrassing
16:47:10 <adamw> anyone have open floor topics?
16:48:22 <jlaska> just a big thanks and nice job for the group for keeping things moving on test days, blocker meetings and ISO testing for the past few weeks
16:49:03 <Viking-Ice> did you go awol as well ?
16:49:29 * Viking-Ice pulled in ca 300 workhours last month
16:49:30 <jlaska> Viking-Ice: just a little vacation
16:50:03 <dpravec> Viking-Ice: he is not a president to go awol without consequences :)
16:50:20 <denise> jlaska, can you ping dlehman about test day? he will work with you
16:50:26 <jlaska> denise: inprogress
16:51:55 <Viking-Ice> vacation and sleep is overrated :)
16:52:08 <adamw> =)
16:53:25 <adamw> well in that case i guess we're done
16:53:27 <adamw> thanks for coming everyone!
16:53:33 <Viking-Ice> Thanks for hosting..
16:53:44 <adamw> and thanks wwoods for being awesome
16:53:48 <adamw> #endmeeting