16:00:23 #startmeeting Fedora QA Meeting 16:00:24 Meeting started Mon Feb 15 16:00:23 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:26 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:28 #meetingname fedora-qa 16:00:29 The meeting name has been set to 'fedora-qa' 16:00:40 #topic Gathering 16:00:48 * kparal raises up his hand 16:00:49 morning 16:00:57 hello 16:01:25 greetings folks 16:01:37 Software Engineer Barbie also says hi 16:01:37 http://www.engadget.com/2010/02/14/barbies-slides-into-the-cubical-becomes-a-computer-software-en/ 16:01:43 I believe maxamillion has a conflict today 16:02:15 finally, some mainstream recognition! :) 16:02:29 looks like me on a good day! 16:02:49 adamw: does it come with more accessories (external monitors, bluetooth keyboard(s) etc...) 16:03:03 this being barbie, all signs point to 'yes' 16:04:19 alright ... let's get started. hopefully wwoods, Viking-Ice and tk009 can join in as we go 16:04:35 #topic Previous Meeting Review 16:04:51 I've only captured 2 items from last week ... 16:04:55 #info jlaska will follow-up w/ nirik after the meeting to setup zodbot feed watchers 16:05:02 thanks nirik, we can check this off 16:05:20 we have zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs 16:05:34 we might want to consider fedora-qa TRAC ticket additions as well 16:05:49 #info completed ... IRC zodbot notification for any additions to the F13Alpha, F13Beta and F13Blocker bugs (thanks nirik) 16:05:57 next up ... 16:06:01 #info jlaska to work w/ poelstra to give 'release validation' spin to quality milestones 16:06:20 This is completed as well ... I suggested a minor wording change (nothing crazy) 16:06:44 the test milestones are no longer specific to install 16:06:51 (e.g. Test Alpha 'Test Compose' and Test Alpha Candidate) 16:06:59 * Oxf13 is here, sorry 16:07:03 Oxf13: welcome! 16:07:19 #info completed, the wording around scheduled test milestones are no longer specific to install 16:07:28 that's all I had to track from last week ... anything I missed? 16:08:20 alright ... diving into the proposed agenda 16:08:33 #topic Rawhide Acceptance Test #3 16:08:57 Just a quick update on the last of 3 rawhide acceptance test (RAT) drops 16:09:09 you probably saw the announcement ... so I'm just summarizing here 16:09:24 #info RATS#3 test recap at http://lists.fedoraproject.org/pipermail/test/2010-February/088407.html 16:09:51 This milestone helped flesh out a few patches against rats_install 16:10:07 thanks wwoods for review feedback, I'll follow-up on those shortly 16:10:40 I need to update the RATS watcher script to watch the new URL (see ticket#115) ... I don't have immediate plans to look into that 16:10:48 so anyone with interest and time is welcome to pitch in 16:11:07 as always, Oxf13 thanks for coordinating the drops with dlehman 16:11:25 that's all I had on that ... let's dive into the Alpha ... 16:11:35 #topic Alpha TC test status 16:11:58 just a quick summary ... then onto some ideas raised by kparal and adamw 16:12:27 #info F-13-Alpha-TC2 is available for testing 16:12:41 #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Install_Test_Results 16:12:45 #link https://fedoraproject.org/wiki/QA:Fedora_13_Alpha_TC_Desktop_Test_Results 16:12:57 yay desktop testing 16:13:02 wait, now I actually have to do some 16:13:03 woot! 16:13:06 whose stupid idea was this? 16:13:08 adamw: deatils deatils 16:13:16 details too :) 16:13:49 so first note I have here was from adamw ... 16:14:15 #topic IDEA - publish live images along with 'test composes' 16:14:31 adamw: you raised this last week, I put this on the agenda so we wouldn't forget 16:15:33 * Viking-Ice sneaks in.. 16:15:41 Oxf13 I believe live images are provided for the release candidate drops (alpha, beta and final) ... with the new focus around desktop validation, are we able to provide them for 'test compose' as well? 16:15:50 Viking-Ice: welcome! 16:16:13 jlaska: well, we already make live images nightly 16:16:19 we've historically relied on the nightly images ... but depending on the state of the repo ... we can go days without any live images 16:16:21 duplicating that work would be, well, duplicated work 16:16:34 if the repo is busted for making the nightly images, how am I supposed to make them? 16:16:36 * jeff_hann here, sorry I'm late 16:16:39 * wwoods appears 16:16:43 as I said in IRC, there's no need for a separate image generation process 16:16:44 wwoods: jeff_hann hey gang 16:16:53 just take the nightlies from the appropriate day and stick them in the TC directory 16:17:06 just want to have a single 'approved' live image that everyone's working from 16:17:14 again, if it's too much work, it's not vital. 16:17:15 ... 16:17:31 yeah, doesn't matter to me which images we use ... just that we have images to test with 16:17:33 pointing to the live isos at the same time you point to the TC doesn't work? 16:17:44 Oxf13: no, because anyone who reads it the next day can't use the same image 16:17:53 but they can still get the right TC images 16:17:55 I suppose I could just make hardlinks 16:18:11 Oxf13: the nightlies don't stick around, they get wiped and regenerated every day 16:18:21 would kparal's suggestion of always keeping around the previous successful live image build solve this? 16:18:31 hardlinks would solve this 16:18:38 no, because you still have the problem if the next build is successful 16:18:52 it's not that i'm worried a later build won't be 'successful' and no one will be able to test 16:19:02 just worried that people would be testing with different stuff, leading to possibly confusing results... 16:19:33 (tester A tests with 2010-02-15 and feature B is broken but feature C works, tester D tests with 2010-02-16 and gets the opposite result, world goes 'huh'?) 16:20:01 seems like a reasonable request 16:20:05 nicely said :) 16:20:33 Oxf13: yeah, a hardlink should do it I guess. 16:20:41 so the issue is making it clearer what people should be testing with ... instead of providing a URL to a directory that may or may not contain images? 16:20:57 well 16:21:03 not just that. as I said, at present, it's not just an information issue 16:21:05 there is the problem of testing the static image instead of the next nightly 16:21:12 because the next nightly is what's going to be in the release 16:21:25 so testing what's older isn't all that useful, if the newer is broken 16:21:28 Oxf13: but that applies equally to the non-live bits 16:21:38 adamw: except we don't generate the non-live bits every day 16:21:40 the test compose isn't what's going to be in the release either 16:21:44 so we don't have the opportunity to test the newer bits 16:22:08 still, it's difficult to do consistent testing and be sure about the results if different testers are testing different bits. 16:22:23 adamw: but we're still in "things change daily" 16:22:26 mode 16:22:34 we haven't stopped yet 16:22:46 these TC images aren't us stopping 16:23:03 they're just a snapshot of something we haven't yet produced this cycle to see what might be broken 16:23:15 in those specific things we don't always produce 16:23:20 so adamw, want to summarize what QA is requesting? 16:23:31 i think we've wasted enough time already and should just move on 16:23:40 yup ... summarize, next steps and we'll move on 16:23:54 * jlaska suggests excessive use of #info 16:23:56 if it takes this much debate to get it done it's probably not worth doing 16:24:34 #info adamw would like a single set of nightlies to be associated with each TC build and kept available in that TC directory during TC testing 16:24:41 #info oxf13 doesn't believe it's necessary 16:24:43 next topic. 16:25:17 alrighty ... next up, I noted a topic from kparal. But kparal and Oxf13 may have discussed this already 16:25:21 more like counterproductive to testing the bits that will be in the release. 16:25:27 I had almost the same question, this time about Alpha TC naming 16:25:31 Oxf13: it's something QA is asking for 16:25:36 I don't think it's counterproductive 16:25:49 we'll follow this up in a ticket after the meeting 16:25:55 #topic IDEA - Unique ISO image names for each drop 16:26:03 kparal: what's the latest 16:26:19 well currently all TC iso's are named the same as the final Alpha iso 16:26:34 Oxf13 replied: by not naming the isos etc the same for TC and the alpha release, we risk missing some bug in the behavior of Anaconda and the release name/number 16:27:16 you may have already discussed, but I wasn't clear on how the ISO file names are related to testing itself, was that cleared up? 16:27:36 I had similar concerns as adamw, people would be testing something other than we expect. or they might suppose they already have the F13 Alpha final iso (it's called like that, isn't it?), but they won't 16:27:52 kparal: Is there some reason to believe we're likely to have that kind of bug? 16:28:19 considering that we change the names for every release and haven't been having issues with it... 16:28:33 we have people downloading the old ISO's a lot 16:28:38 well, I wasn't sure why the names are not simply unique 16:28:44 normally, we have them validate the CHECKSUMS 16:29:04 not only that, but the TC stuff is on a completely different mirror system 16:29:10 so the Oxf13's explanation is above. I didn't know about anaconda issues, then I suppose it makes more sense 16:29:13 we can't really protect against stupid people. 16:29:36 buildinstall does do things with version and product which does change anaconda behaviour 16:30:23 is there a TC# in the metadata anywhere? It'd be simplest to say "current test candidate is TC7" and then if .treeinfo says test-candidate = 6.. 16:30:28 of course, this is one of the dangers of opening pre-release test isos up to more people 16:30:36 but I don't really want to add new data if we don't have to. timestamps work, but people always get confused 16:30:38 you risk more people getting them who won't use them responsibly 16:30:49 wwoods: no 16:30:56 Oxf13: I don' think there is bad intent for the images 16:31:01 wwoods: it's at the top of the URL, but from that point down, by design, it looks exactly like the Alpha would 16:31:18 jlaska: I don't mean malintent, I mean ignorance 16:31:18 just looking for ways to make sure we are doing what's reasonable to make sure people know they are using the latest test images 16:31:22 okay 16:31:44 I think we can move on 16:31:48 kparal: Oxf13: so in summary the filename used for the ISO image files is tightly coupled to the whole build process? 16:31:56 jlaska: yes 16:32:07 ah okay, not something I knew about 16:32:17 probably one of the new SOPs has this listed 16:32:22 https://fedoraproject.org/wiki/Composing_test_images ? 16:32:27 the input to the compose process drives the name of the iso, I don't manually make the isos and pick a new name each time 16:33:00 jlaska: probably Composing_Test_Composes once we make that 16:33:15 Oxf13: ah sweet, thanks 16:33:33 alrighty, kparal anything else to discuss here, otherwise we'll dive into AutoQA 16:33:43 test_images doesn't have any isos produced (beyond boot.iso) 16:34:11 we can go on 16:34:20 alrighty 16:34:23 #topic AutoQA - depcheck update 16:34:37 wwoods: you have the mic ... any updates or milestones to share? 16:36:29 wwoods: we can come back to this later if you prefer? 16:37:19 alright ... moving on to resultsdb ... 16:37:30 #topic AutoQA - resultsdb discussion 16:38:00 kparal, wwoods or jskladan, anything to add here? 16:38:26 well I would just mention that we had a small QA team discussion over phone 16:38:35 and the results are available in autoqa-devel ML 16:38:43 * wwoods was afk a sec, deferring to kparal for the moment 16:38:46 #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000201.html 16:38:51 wwoods: okay 16:39:08 we are currently studying other projects, how they define their results database, etc 16:39:24 so any input can be sent to autoqa-devel 16:40:09 that's all I suppose 16:40:30 kparal: okay, thanks for the update 16:40:54 #topic AutoQA - beakerlib 16:40:54 the current notion - if memory serves - is that we want to provide one unified resultsdb that al the autoqa tests can use to report (somewhat simplified) results 16:41:21 which will give us one place to view all test results - and ideally a few nice views of that data. 16:41:36 #topic AutoQA - resultsdb discussion 16:42:07 #info we want to provide one unified resultsdb that all the autoqa tests can use to report (somewhat simplified) results 16:42:11 one sec ... meeting tags 16:42:34 #info had a small QA team discussion over phone last week (results on autoqa-devel ML) 16:42:34 I think we had some example views we wanted to see - test results for a given package, test results for a given installable tree, all test results for package builds / updates for a given day, etc. 16:42:57 if you can think of a good use case you'd like - a specific view of the autoqa results data - please let us know 16:43:29 nice, thanks wwoods and kparal 16:43:29 further discussion will be on autoqa-devel. 16:43:49 oh, and as for depcheck: no progress, owing to the fact that I was out of the office and then sick all last week 16:44:01 status - plague :( 16:44:14 #topic AutoQA - beakerlib 16:44:29 ok, time for me to speak, i suppose :) 16:44:31 Adding a new topic here for the work jskladan and kparal have been looking into around beakerlib 16:44:45 any updates or problems you are experiencing? 16:45:15 well, idk if all af you are familiar with beakerlib 16:45:34 #link https://fedorahosted.org/beakerlib/ 16:46:02 its a set of libraries which RHEL guys use for writing automated tests 16:46:24 so we'd like to reuse those in out testing system (autoqa) 16:47:06 today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests 16:47:17 #link http://jskladan.fedorapeople.org/beakerlib_helloworld.tar 16:47:46 it's nothing fancy - in fact, it only makes sure, that you have beakerlib installed, and then runs the test itself 16:47:50 nice, using the provided beakerlib vim syntax highlight! :) 16:48:06 rlAssertRpm "setup" 16:48:06 rlAssertExists "/etc/passwd" 16:48:06 rlAssertGrep "root" "/etc/passwd" 16:48:08 cool 16:48:46 and of course stores the report produced by beakerlib in the directory for autotest-lib result files 16:49:56 we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses 16:50:17 this is great news, it's moving along faster than I had expected 16:50:22 nice work :) 16:50:30 so that's about it for now - if kamil does not have anything to add 16:50:45 #info today kamil and me created a simple test, which demonstrates usage of beakerlib-based tests 16:50:55 #info we'll be trying to assimilate other tests this week - probably init script testing stuff - and look into some more complex uses 16:51:20 jskladan described it all 16:51:40 alright thanks for the update gang 16:51:50 I'll do a quick run through on the 2 remaining AutoQA topics 16:51:56 #topic AutoQA - install automation 16:52:14 #info Last week, Liam summarized current DVD automation efforts in an email to autoqa-devel 16:52:21 #link https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000199.html 16:52:51 Automating media (CD/DVD) installs using virt is funky b/c you have to do some strange stuff to provide your own boot arguments 16:53:01 so Liam outlined several techniques he's tried 16:53:12 including using dogtail to add custom boot args using virt-viewer 16:53:26 any feedback folks have on those proposed solutions would be good to have 16:53:52 once back from Chinese new year, he'll be working to integrate the best method into the current test 16:53:55 * Oxf13 has to bail, need to prepare for next appointment 16:54:07 and last up ... 16:54:12 #topic AutoQA - packaging 16:54:26 more small progress last week on identifying new build dependencies 16:54:48 it exploded to 50+ missing deps at one point, but akurtakov helped identify how to address that 16:55:08 This seems like the list of deps isn't every finalized :( 16:55:21 I'd really like to squash this so I can start making packaging progress 16:55:27 any ideas would be appreciated 16:55:49 My current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all 16:56:03 where's the work-in-progress stuff for people who want to help out? 16:56:09 #link https://fedoraproject.org/wiki/User:Jlaska/gwt 16:56:39 i would suggest you make progress by starting packaging 16:56:57 the only way you can ever be sure about the deps list is when you actually start packaging stuff and find out if it works 16:57:02 adamw: I'd like to as well, but I feel like I still don't know how long the packaging road is 16:57:18 I'm trying to find the leaf nodes to package ... and every time I look, I find another deps branch 16:58:38 so I'll revisit this week, and work towards adamw's suggestion ... just start packaging :) 16:58:52 #info current plan, is to see if akurtakov (and others) can dedicate some time on IRC To help walk the list of missing deps once and for all 16:58:58 #info I'll revisit this week, and work towards adamw's suggestion ... just start packaging 16:59:03 alright ... open discussion time 16:59:14 #topic Open discussion - 16:59:26 anything not previously mentioned that we need to discuss today? 17:00:32 If no suggestions, I'll close out the meeting in 1 minute 17:01:23 nup 17:01:36 alrighty ... thanks for the updates everyone 17:01:46 as usual, I'll send minutes to the list for review 17:01:53 #endmeeting