16:00:23 <jlaska> #startmeeting Fedora QA Meeting
16:00:24 <zodbot> Meeting started Mon Feb 15 16:00:23 2010 UTC.  The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:28 <jlaska> #meetingname fedora-qa
16:00:29 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:40 <jlaska> #topic Gathering
16:00:48 * kparal raises up his hand
16:00:49 <adamw> morning
16:00:57 <jskladan> hello
16:01:25 <jlaska> greetings folks
16:01:37 <adamw> Software Engineer Barbie also says hi
16:01:37 <adamw> http://www.engadget.com/2010/02/14/barbies-slides-into-the-cubical-becomes-a-computer-software-en/
16:01:43 <jlaska> I believe maxamillion has a conflict today
16:02:15 <jlaska> finally, some mainstream recognition! :)
16:02:29 <adamw> looks like me on a good day!
16:02:49 <jlaska> adamw: does it come with more accessories (external monitors, bluetooth keyboard(s) etc...)
16:03:03 <adamw> this being barbie, all signs point to 'yes'
16:04:19 <jlaska> alright ... let's get started.  hopefully wwoods, Viking-Ice and tk009 can join in as we go
16:04:35 <jlaska> #topic Previous Meeting Review
16:04:51 <jlaska> I've only captured 2 items from last week ...
16:04:55 <jlaska> #info jlaska will follow-up w/ nirik after the meeting to setup zodbot feed watchers
16:05:02 <jlaska> thanks nirik, we can check this off
16:05:20 <jlaska> we have zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs
16:05:34 <jlaska> we might want to consider fedora-qa TRAC ticket additions as well
16:05:49 <jlaska> #info completed ... IRC zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs (thanks nirik)
16:05:57 <jlaska> next up ...
16:06:01 <jlaska> #info jlaska to work w/ poelstra to give 'release validation' spin to quality milestones
16:06:20 <jlaska> This is completed as well ... I suggested a minor wording change (nothing crazy)
16:06:44 <jlaska> the test milestones are no longer specific to install
16:06:51 <jlaska> (e.g. Test Alpha 'Test Compose' and Test Alpha Candidate)
16:06:59 * Oxf13 is here, sorry
16:07:03 <jlaska> Oxf13: welcome!
16:07:19 <jlaska> #info completed, the wording around scheduled test milestones are no longer specific to install
16:07:28 <jlaska> that's all I had to track from last week ... anything I missed?
16:08:20 <jlaska> alright ... diving into the proposed agenda
16:08:33 <jlaska> #topic Rawhide Acceptance Test #3
16:08:57 <jlaska> Just a quick update on the last of 3 rawhide acceptance test (RAT) drops
16:09:09 <jlaska> you probably saw the announcement ... so I'm just summarizing here
16:09:24 <jlaska> #info RATS#3 test recap at http://lists.fedoraproject.org/pipermail/test/2010-February/088407.html
16:09:51 <jlaska> This milestone helped flesh out a few patches against rats_install
16:10:07 <jlaska> thanks wwoods for review feedback, I'll follow-up on those shortly
16:10:40 <jlaska> I need to update the RATS watcher script to watch the new URL (see ticket#115) ... I don't have immediate plans to look into that
16:10:48 <jlaska> so anyone with interest and time is welcome to pitch in
16:11:07 <jlaska> as always, Oxf13 thanks for coordinating the drops with dlehman
16:11:25 <jlaska> that's all I had on that ... let's dive into the Alpha ...
16:11:35 <jlaska> #topic Alpha TC test status
16:11:58 <jlaska> just a quick summary ... then onto some ideas raised by kparal and adamw
16:12:27 <jlaska> #info F-13-Alpha-TC2 is available for testing
16:12:41 <jlaska> #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Install_Test_Results
16:12:45 <jlaska> #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Desktop_Test_Results
16:12:57 <adamw> yay desktop testing
16:13:02 <adamw> wait, now I actually have to do some
16:13:03 <jlaska> woot!
16:13:06 <adamw> whose stupid idea was this?
16:13:08 <jlaska> adamw: deatils deatils
16:13:16 <jlaska> details too :)
16:13:49 <jlaska> so first note I have here was from adamw ...
16:14:15 <jlaska> #topic IDEA - publish live images along with 'test composes'
16:14:31 <jlaska> adamw: you raised this last week, I put this on the agenda so we wouldn't forget
16:15:33 * Viking-Ice sneaks in..
16:15:41 <jlaska> Oxf13 I believe live images are provided for the release candidate drops (alpha, beta and final) ... with the new focus around desktop validation, are we able to provide them for 'test compose' as well?
16:15:50 <jlaska> Viking-Ice: welcome!
16:16:13 <Oxf13> jlaska: well, we already make live images nightly
16:16:19 <jlaska> we've historically relied on the nightly images ... but depending on the state of the repo ... we can go days without any live images
16:16:21 <Oxf13> duplicating that work would be, well, duplicated work
16:16:34 <Oxf13> if the repo is busted for making the nightly images, how am I supposed to make them?
16:16:36 * jeff_hann here, sorry I'm late
16:16:39 * wwoods appears
16:16:43 <adamw> as I said in IRC, there's no need for a separate image generation process
16:16:44 <jlaska> wwoods: jeff_hann hey gang
16:16:53 <adamw> just take the nightlies from the appropriate day and stick them in the TC directory
16:17:06 <adamw> just want to have a single 'approved' live image that everyone's working from
16:17:14 <adamw> again, if it's too much work, it's not vital.
16:17:15 <Oxf13> ...
16:17:31 <jlaska> yeah, doesn't matter to me which images we use ... just that we have images to test with
16:17:33 <Oxf13> pointing to the live isos at the same time you point to the TC doesn't work?
16:17:44 <adamw> Oxf13: no, because anyone who reads it the next day can't use the same image
16:17:53 <adamw> but they can still get the right TC images
16:17:55 <Oxf13> I suppose I could just make hardlinks
16:18:11 <adamw> Oxf13: the nightlies don't stick around, they get wiped and regenerated every day
16:18:21 <jlaska> would kparal's suggestion of always keeping around the previous successful live image build solve this?
16:18:31 <Oxf13> hardlinks would solve this
16:18:38 <adamw> no, because you still have the problem if the next build is successful
16:18:52 <adamw> it's not that i'm worried a later build won't be 'successful' and no one will be able to test
16:19:02 <adamw> just worried that people would be testing with different stuff, leading to possibly confusing results...
16:19:33 <adamw> (tester A tests with 2010-02-15 and feature B is broken but feature C works, tester D tests with 2010-02-16 and gets the opposite result, world goes 'huh'?)
16:20:01 <jlaska> seems like a reasonable request
16:20:05 <kparal> nicely said :)
16:20:33 <adamw> Oxf13: yeah, a hardlink should do it I guess.
16:20:41 <jlaska> so the issue is making it clearer what people should be testing with ... instead of providing a URL to a directory that may or may not contain images?
16:20:57 <Oxf13> well
16:21:03 <adamw> not just that. as I said, at present, it's not just an information issue
16:21:05 <Oxf13> there is the problem of testing the static image instead of the next nightly
16:21:12 <Oxf13> because the next nightly is what's going to be in the release
16:21:25 <Oxf13> so testing what's older isn't all that useful, if the newer is broken
16:21:28 <adamw> Oxf13: but that applies equally to the non-live bits
16:21:38 <Oxf13> adamw: except we don't generate the non-live bits every day
16:21:40 <adamw> the test compose isn't what's going to be in the release either
16:21:44 <Oxf13> so we don't have the opportunity to test the newer bits
16:22:08 <adamw> still, it's difficult to do consistent testing and be sure about the results if different testers are testing different bits.
16:22:23 <Oxf13> adamw: but we're still in "things change daily"
16:22:26 <Oxf13> mode
16:22:34 <Oxf13> we haven't stopped yet
16:22:46 <Oxf13> these TC images aren't us stopping
16:23:03 <Oxf13> they're just a snapshot of something we haven't yet produced this cycle to see what might be broken
16:23:15 <Oxf13> in those specific things we don't always produce
16:23:20 <jlaska> so adamw, want to summarize what QA is requesting?
16:23:31 <adamw> i think we've wasted enough time already and should just move on
16:23:40 <jlaska> yup ... summarize, next steps and we'll move on
16:23:54 * jlaska suggests excessive use of #info
16:23:56 <adamw> if it takes this much debate to get it done it's probably not worth doing
16:24:34 <adamw> #info adamw would like a single set of nightlies to be associated with each TC build and kept available in that TC directory during TC testing
16:24:41 <adamw> #info oxf13 doesn't believe it's necessary
16:24:43 <adamw> next topic.
16:25:17 <jlaska> alrighty ... next up, I noted a topic from kparal.  But kparal and Oxf13 may have discussed this already
16:25:21 <Oxf13> more like counterproductive to testing the bits that will be in the release.
16:25:27 <kparal> I had almost the same question, this time about Alpha TC naming
16:25:31 <jlaska> Oxf13: it's something QA is asking for
16:25:36 <jlaska> I don't think it's counterproductive
16:25:49 <jlaska> we'll follow this up in a ticket after the meeting
16:25:55 <jlaska> #topic IDEA - Unique ISO image names for each drop
16:26:03 <jlaska> kparal: what's the latest
16:26:19 <kparal> well currently all TC iso's are named the same as the final Alpha iso
16:26:34 <kparal> Oxf13 replied: by not naming the isos etc the same for TC and the alpha release, we risk missing some bug in the behavior of Anaconda and the release name/number
16:27:16 <jlaska> you may have already discussed, but I wasn't clear on how the ISO file names are related to testing itself, was that cleared up?
16:27:36 <kparal> I had similar concerns as adamw, people would be testing something other than we expect. or they might suppose they already have the F13 Alpha final iso (it's called like that, isn't it?), but they won't
16:27:52 <pjones> kparal: Is there some reason to believe we're likely to have that kind of bug?
16:28:19 <pjones> considering that we change the names for every release and haven't been having issues with it...
16:28:33 <jlaska> we have people downloading the old ISO's a lot
16:28:38 <kparal> well, I wasn't sure why the names are not simply unique
16:28:44 <jlaska> normally, we have them validate the CHECKSUMS
16:29:04 <Oxf13> not only that, but the TC stuff is on a completely different mirror system
16:29:10 <kparal> so the Oxf13's explanation is above. I didn't know about anaconda issues, then I suppose it makes more sense
16:29:13 <Oxf13> we can't really protect against stupid people.
16:29:36 <Oxf13> buildinstall does do things with version and product which does change anaconda behaviour
16:30:23 <wwoods> is there a TC# in the metadata anywhere? It'd be simplest to say "current test candidate is TC7" and then if .treeinfo says test-candidate = 6..
16:30:28 <Oxf13> of course, this is one of the dangers of opening pre-release test isos up to more people
16:30:36 <wwoods> but I don't really want to add new data if we don't have to. timestamps work, but people always get confused
16:30:38 <Oxf13> you risk more people getting them who won't use them responsibly
16:30:49 <Oxf13> wwoods: no
16:30:56 <jlaska> Oxf13: I don' think there is bad intent for the images
16:31:01 <Oxf13> wwoods: it's at the top of the URL, but from that point down, by design, it looks exactly like the Alpha would
16:31:18 <Oxf13> jlaska: I don't mean malintent, I mean ignorance
16:31:18 <jlaska> just looking for ways to make sure we are doing what's reasonable to make sure people know they are using the latest test images
16:31:22 <jlaska> okay
16:31:44 <kparal> I think we can move on
16:31:48 <jlaska> kparal: Oxf13: so in summary the filename used for the ISO image files is tightly coupled to the whole build process?
16:31:56 <Oxf13> jlaska: yes
16:32:07 <jlaska> ah okay, not something I knew about
16:32:17 <jlaska> probably one of the new SOPs has this listed
16:32:22 <jlaska> https://fedoraproject.org/wiki/Composing_test_images ?
16:32:27 <Oxf13> the input to the compose process drives the name of the iso, I don't manually make the isos and pick a new name each time
16:33:00 <Oxf13> jlaska: probably Composing_Test_Composes once we make that
16:33:15 <jlaska> Oxf13: ah sweet, thanks
16:33:33 <jlaska> alrighty, kparal anything else to discuss here, otherwise we'll dive into AutoQA
16:33:43 <Oxf13> test_images doesn't have any isos produced (beyond boot.iso)
16:34:11 <kparal> we can go on
16:34:20 <jlaska> alrighty
16:34:23 <jlaska> #topic AutoQA - depcheck update
16:34:37 <jlaska> wwoods: you have the mic ... any updates or milestones to share?
16:36:29 <jlaska> wwoods: we can come back to this later if you prefer?
16:37:19 <jlaska> alright ... moving on to resultsdb ...
16:37:30 <jlaska> #topic AutoQA - resultsdb discussion
16:38:00 <jlaska> kparal, wwoods or jskladan, anything to add here?
16:38:26 <kparal> well I would just mention that we had a small QA team discussion over phone
16:38:35 <kparal> and the results are available in autoqa-devel ML
16:38:43 * wwoods was afk a sec, deferring to kparal for the moment
16:38:46 <jlaska> #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000201.html
16:38:51 <jlaska> wwoods: okay
16:39:08 <kparal> we are currently studying other projects, how they define their results database, etc
16:39:24 <kparal> so any input can be sent to autoqa-devel
16:40:09 <kparal> that's all I suppose
16:40:30 <jlaska> kparal: okay, thanks for the update
16:40:54 <jlaska> #topic AutoQA - beakerlib
16:40:54 <wwoods> the current notion - if memory serves - is that we want to provide one unified resultsdb that al the autoqa tests can use to report (somewhat simplified) results
16:41:21 <wwoods> which will give us one place to view all test results - and ideally a few nice views of that data.
16:41:36 <jlaska> #topic AutoQA - resultsdb discussion
16:42:07 <jlaska> #info we want to provide one unified resultsdb that all the autoqa tests can use to report (somewhat simplified) results
16:42:11 <jlaska> one sec ... meeting tags
16:42:34 <jlaska> #info had a small QA team discussion over phone last week (results on autoqa-devel ML)
16:42:34 <wwoods> I think we had some example views we wanted to see - test results for a given package, test results for a given installable tree, all test results for package builds / updates for a given day, etc.
16:42:57 <wwoods> if you can think of a good use case you'd like - a specific view of the autoqa results data - please let us know
16:43:29 <jlaska> nice, thanks wwoods and kparal
16:43:29 <wwoods> further discussion will be on autoqa-devel.
16:43:49 <wwoods> oh, and as for depcheck: no progress, owing to the fact that I was out of the office and then sick all last week
16:44:01 <jlaska> status - plague :(
16:44:14 <jlaska> #topic AutoQA - beakerlib
16:44:29 <jskladan> ok, time for me to speak, i suppose :)
16:44:31 <jlaska> Adding a new topic here for the work jskladan and kparal have been looking into around beakerlib
16:44:45 <jlaska> any updates or problems you are experiencing?
16:45:15 <jskladan> well, idk if all af you are familiar with beakerlib
16:45:34 <jlaska> #link https://fedorahosted.org/beakerlib/
16:46:02 <jskladan> its a set of libraries which RHEL guys use for writing automated tests
16:46:24 <jskladan> so we'd like to reuse those in out testing system (autoqa)
16:47:06 <jskladan> today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests
16:47:17 <jskladan> #link http://jskladan.fedorapeople.org/beakerlib_helloworld.tar
16:47:46 <jskladan> it's nothing fancy - in fact, it only makes sure, that you have beakerlib installed, and then runs the test itself
16:47:50 <jlaska> nice, using the provided beakerlib vim syntax highlight! :)
16:48:06 <jlaska> rlAssertRpm "setup"
16:48:06 <jlaska> rlAssertExists "/etc/passwd"
16:48:06 <jlaska> rlAssertGrep "root" "/etc/passwd"
16:48:08 <jlaska> cool
16:48:46 <jskladan> and of course stores the report produced by beakerlib in the directory for autotest-lib result files
16:49:56 <jskladan> we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses
16:50:17 <jlaska> this is great news, it's moving along faster than I had expected
16:50:22 <jlaska> nice work :)
16:50:30 <jskladan> so that's about it for now - if kamil does not have anything to add
16:50:45 <jlaska> #info today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests
16:50:55 <jlaska> #info we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses
16:51:20 <kparal> jskladan described it all
16:51:40 <jlaska> alright thanks for the update gang
16:51:50 <jlaska> I'll do a quick run through on the 2 remaining AutoQA topics
16:51:56 <jlaska> #topic AutoQA - install automation
16:52:14 <jlaska> #info Last week, Liam summarized current DVD automation efforts in an email to autoqa-devel
16:52:21 <jlaska> #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000199.html
16:52:51 <jlaska> Automating media (CD/DVD) installs using virt is funky b/c you have to do some strange stuff to provide your own boot arguments
16:53:01 <jlaska> so Liam outlined several techniques he's tried
16:53:12 <jlaska> including using dogtail to add custom boot args using virt-viewer
16:53:26 <jlaska> any feedback folks have on those proposed solutions would be good to have
16:53:52 <jlaska> once back from Chinese new year, he'll be working to integrate the best method into the current test
16:53:55 * Oxf13 has to bail, need to prepare for next appointment
16:54:07 <jlaska> and last up ...
16:54:12 <jlaska> #topic AutoQA - packaging
16:54:26 <jlaska> more small progress last week on identifying new build dependencies
16:54:48 <jlaska> it exploded to 50+ missing deps at one point, but akurtakov helped identify how to address that
16:55:08 <jlaska> This seems like the list of deps isn't every finalized :(
16:55:21 <jlaska> I'd really like to squash this so I can start making packaging progress
16:55:27 <jlaska> any ideas would be appreciated
16:55:49 <jlaska> My current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all
16:56:03 <wwoods> where's the work-in-progress stuff for people who want to help out?
16:56:09 <jlaska> #link https://fedoraproject.org/wiki/User:Jlaska/gwt
16:56:39 <adamw> i would suggest you make progress by starting packaging
16:56:57 <adamw> the only way you can ever be sure about the deps list is when you actually start packaging stuff and find out if it works
16:57:02 <jlaska> adamw: I'd like to as well, but I feel like I still don't know how long the packaging road is
16:57:18 <jlaska> I'm trying to find the leaf nodes to package ... and every time I look, I find another deps branch
16:58:38 <jlaska> so I'll revisit this week, and work towards adamw's suggestion ... just start packaging :)
16:58:52 <jlaska> #info current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all
16:58:58 <jlaska> #info I'll revisit this week, and work towards adamw's suggestion ... just start packaging
16:59:03 <jlaska> alright ... open discussion time
16:59:14 <jlaska> #topic Open discussion - <Your topic here>
16:59:26 <jlaska> anything not previously mentioned that we need to discuss today?
17:00:32 <jlaska> If no suggestions, I'll close out the meeting in 1 minute
17:01:23 <adamw> nup
17:01:36 <jlaska> alrighty ... thanks for the updates everyone
17:01:46 <jlaska> as usual, I'll send minutes to the list for review
17:01:53 <jlaska> #endmeeting