15:00:24 <jlaska> #startmeeting Fedora QA Meeting
15:00:24 <zodbot> Meeting started Mon Aug 23 15:00:24 2010 UTC.  The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:24 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:07 <jlaska> #topic Gathering ....
15:01:12 * kparal available for the next 45 minutes
15:01:13 <jlaska> #meetingname fedora-qa
15:01:13 <zodbot> The meeting name has been set to 'fedora-qa'
15:01:17 * vaschenb 
15:01:42 * jsmith is here
15:02:05 * wwoods here
15:02:28 <adamw> yo
15:02:56 * Viking-Ice here
15:04:41 * jlaska figures joza is lurking
15:04:59 <jlaska> alright ... let's get started so we can have kparal here for autoqa time
15:05:06 <jlaska> #topic Previous meeting follow-up
15:05:47 * jskladan lurks
15:05:54 <jlaska> jskladan hey there lurker!
15:06:03 <jlaska> #info adamw to update http://fedoraproject.org/wiki/QA:Testcase_desktop_updates to better describe the test experience on live images
15:06:27 <jlaska> looks like this got some recent edits
15:06:39 <adamw> yeah, it should be right now.
15:07:00 <jlaska> thanks adamw,  I see the live qualifications ... looks good
15:07:18 <jlaska> #info jlaska to update https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver so that it better captures the post-install xdriver=vesa expectations
15:07:27 <jlaska> I believe this is complete ... and not because of me
15:07:30 * adamw did that too
15:07:33 <jlaska> adamw was a wiki editing machine last week
15:08:17 <jlaska> #info maxamillion update rel-eng TRAC#3860 to request RC#5
15:08:38 <jlaska> maxa completed this ... but turned out we had a course correction and RC#5 wasn't required
15:08:51 <jlaska> #info adamw and jlaska to propose artwork final release criteria
15:09:07 <adamw> man, we suck
15:09:19 <jlaska> a classic ... adamw, my name has been on that one, but I haven't touched it.  Do you want me to reach out to design-team for thoughts?
15:09:30 <adamw> if you could that'd be great
15:09:36 <adamw> or if you're busy i can do it
15:09:40 <adamw> either way
15:09:52 <jlaska> alright, I'll fire that off this week
15:10:07 <jlaska> that's all I had from last week ... let's dive into the agenda
15:10:24 <jlaska> #topic F14 Alpha QA Status
15:10:41 <jlaska> well, first off ... congrats everyone
15:10:51 <jlaska> yet another Fedora release milestone behind us
15:10:55 <adamw> *party poppers*
15:10:56 <jlaska> they just keep on coming, don't they
15:11:40 <kparal> *enjoys*
15:11:48 <jlaska> adamw: I know you wanted to raise awareness on the radeon issue ... was there anything you wanted to discuss there?
15:12:28 <adamw> jlaska: yeah, i'm a bit uneasy on that, i was assuming we'd have the readiness meeting to do it, but apparently we didn't schedule another readiness meeting even though the release was delayed (seems odd to me)
15:12:41 <adamw> so i'll have to contact docs and websites teams directly, i haven't done  that yet, i'll do it today
15:13:06 <jlaska> jsmith: anything you can think of to help elevate the visibility of http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda
15:15:00 <jlaska> #info adamw wants to raise awareness of known radeon display issue (http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda)
15:15:26 <jlaska> have folks been testing with F-14-Alpha + updates-testing?
15:15:36 <wwoods> that's what I'm running
15:16:03 <jlaska> it was only an improvement for me after the update ... that was good news
15:16:54 <jlaska> adamw: CommonBugs looks in good shape ... anything else on your radar to close out F-14-Alpha for QA?
15:16:58 <adamw> me too
15:16:58 <adamw> it's slightly rough with all the gnome changes
15:17:02 <jsmith> jlaska: I've asked the Docs folks to add an admonition to the one-page release notes for F14 Alpha
15:17:37 <jlaska> jsmith: ah nice, thank you
15:17:40 <adamw> jsmith: awesome, thanks
15:18:01 <adamw> jsmith: i'm planning on asking the web group if they can put a one-liner with a link to release notes or common bugs on the download page
15:18:02 <jsmith> I think we'll also highlight it in the announcement email
15:19:22 <jlaska> both good ideas
15:19:47 <adamw> no need to self-flagellate, just a quick mention with a link is fine
15:20:04 <jlaska> adamw: I created an F14 QA retrospective wiki stub last week ... but I'll clean that up a bit and send that out so we can start collecting the good/bad/ugly
15:20:31 <jlaska> #action jlaska to publish F-14-Alpha QA retrospective page
15:22:11 <jlaska> in case it wasn't clear ... *huge* thanks again to all who contributed testing, especially the fast turn around on RC#4.  While we did end up slipping for another issue, it was very helpful to have a sense for how the installer and desktop test matrices held up to the alpha release criteria
15:22:35 <jlaska> adamw: any other Alpha topics/thoughts (or Jerry Springer final thoughts)?
15:23:10 <adamw> of course, what this has all showed is that the really important thing is family...
15:23:20 <jlaska> LOL!
15:23:24 <adamw> er, nope. :)
15:23:48 <jlaska> alrighty ... well done
15:24:19 <jlaska> alright ... do you hear the music?
15:24:26 <jlaska> that's not Duff man ... it's AutoQA time
15:24:36 <jlaska> #topic AutoQA Package Acceptance
15:25:38 <jskladan> well, no funky updates from me - i've been testing F14 a bit, and focused mostly on the "how to write autoqa tests" wiki
15:25:41 <jlaska> I know F-14 pulled away some time+energy from some autoqa work, but it appears there was some progress last week
15:26:45 <kparal> my patch that standardized some hook args was accepted, so it created better options for multihook tests
15:27:07 <jlaska> jskladan: awesome, I can't wait to see the updated wiki page ... so much has changed with some of the recent patch sets
15:27:23 <jlaska> standardized hook args being one of the big ones!
15:27:38 <jlaska> how's the multi-hook milestone looking?
15:27:39 * jlaska checks
15:27:48 <wwoods> and control.autoqa, and new stuff in control files and test wrappers
15:28:18 <jlaska> down to only 1 ticket in the multi-host milestone ... that's great
15:28:19 <jlaska> https://fedorahosted.org/autoqa/ticket/203
15:28:23 <kparal> multi-hook milestone is almost finished, now only to schedule tests for multiple hooks, which should be piece of cake. the framework is ready
15:30:01 <jlaska> vaschenb: have you had a chance to look at those multi-hook tests kparal mentioned?  Does that seem do-able with the time remaining?
15:31:00 <jlaska> vaschenb: I'm sure multi-hook guru kparal can help answer any questions too :)
15:31:12 <jlaska> wwoods: how's the depcheck front progressing?
15:31:15 <vaschenb> jlaska: not yet, today I worked on upgradepath output...
15:31:35 <kparal> I requested some upgradepath output format improvement at vaschenb :)
15:31:37 <wwoods> jlaska: we've got some new unittests in depcheck that actually, like, prove that depchecking works as expected
15:31:58 <wwoods> at least partially
15:32:04 <vaschenb> jlaska: I'll take a look at it, but today I'm functionless
15:32:18 <jlaska> wwoods: Bonus points for using built-in python unittest module!  That's really cool looking stuff
15:32:22 <wwoods> I split it up into a module and a CLI so we can keep the tool and the tests in different files.. little cleaner that way
15:32:43 <wwoods> so the next thing to do is to start using mash instead of createrepo
15:32:55 <wwoods> so we can actually handle multilib stuff properly
15:32:58 <jlaska> #info jskladan improving 'how to write autoqa tests' wiki page
15:33:00 * jlaska info's
15:33:18 <jlaska> #info kparal standardized hook argument patch set accepted ... multi-hook tests now possible
15:33:36 <jlaska> #info vaschenb improving upgrade-path test output
15:34:01 <jlaska> #info wwoods posted info about depcheck unit test framework
15:34:50 <wwoods> mostly last week was about cleaning things up - f14a kind of took priority
15:35:24 <jlaska> yeah ... these darn milestones!
15:35:43 <jlaska> #info wwoods posted thoughts on depcheck next steps -- https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001010.html
15:36:35 <jlaska> wwoods: you get any lucky volunteers for the mash work yet?
15:36:47 <wwoods> jlaska: unfortunately I think I volunteered myself
15:36:59 <jlaska> did everyone step back (but you)?
15:37:12 <jsmith> jlaska: I think that's closer to the truth
15:37:20 <jlaska> doh!
15:37:35 <jskladan> wwoods: i said i'll dig into that
15:37:44 <jskladan> maybe not too loud :)
15:37:56 <jlaska> hehe
15:38:07 <wwoods> jskladan: heh! maybe so. That'd be great
15:38:28 <jlaska> yeah, +1 on the thanks jskladan
15:38:38 <jskladan> wwoods: yeah, no problem at all. I'll have it on my radar, once i finish the wiki edit
15:38:41 <wwoods> I'll try to help round up some info from the mash 'experts'
15:38:52 <skvidal> s/mash experts/notting/
15:39:09 <jlaska> there were 2 other non-depcheck items that I was aware of ... kparal do you want to talk about your latest patch to the list?
15:39:10 <wwoods> mash seems to be one of those things where nobody fully understands it, so anyone who touches it immediately becomes a new 'expert'
15:39:46 <kparal> ah, yes. well
15:40:05 <kparal> this patch: https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html
15:40:26 * jskladan looking forward to be the so called expert on yet another fancy field :)
15:40:38 <jlaska> #info some rpmguard tests failing, see kparal's proposed patch https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html
15:41:21 <kparal> I think the code was wrong all the time. we requested the last release of a certain package, but we didn't check the provided repo, just its parents
15:41:38 <kparal> which I didn't really understood and I believe it was wrong, until someone proves me otherwise :)
15:42:12 <wwoods> very possible
15:42:14 <kparal> so I have rewritten it to a behaviour I suppose it's correct, and by the way it also fixed the traceback we received before
15:42:26 <jlaska> should I just apply this patch to our current instance?
15:42:38 <kparal> jlaska: I believe it should work (better than before)
15:43:10 <jlaska> kparal: okay, I'll apply and respond with results
15:43:25 <kparal> jlaska: yes please
15:43:29 * kparal has to run, sorry
15:43:30 <jlaska> #action jlaska apply and provide results against autoqa patch (https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html)
15:43:34 <jlaska> kparal: cya
15:43:47 <jlaska> the other non-depcheck issue was vaschenb upgrade-path test
15:44:16 <jlaska> I commented that I thought we'd probably want a method of running this test outside of the test harness (for folks who play the autoqa home-game)
15:44:39 <jlaska> but then kparal and vaschenb pointed out that you can run the tests locally using a method I wasn't aware of
15:44:49 <jlaska> just using 'autoqa post-bodhi-update --local ...'
15:44:50 * jskladan likes (and heavily uses) the --local)
15:45:08 <jlaska> wwoods: did you have any thoughts on that approach vs a stand-alone script?
15:46:20 <wwoods> either way seems fine to me
15:46:30 <jlaska> I really like the stand-alone version, but then that means we have to write multiple option parsers etc...
15:47:07 <jlaska> I don't often install 'autoqa' when testing out of git ... so I'm not familiar with this method
15:47:33 <wwoods> I guess I just want to make sure we recognize that writing a test this way is not the most obvious (or simplest) path for general audiences
15:47:52 <jlaska> as long as we document how to run the test somewhere, right?
15:48:08 <wwoods> if you're skilled at hacking around in autoqa already then yeah, this totally works and saves you having to write option parsers etc.
15:48:46 <wwoods> but for people who are like "hey I want to write a test, how does autoqa work" I'm still going to suggest "Write the test first, you don't need to know the full details of how autoqa works until you have a working test"
15:48:57 <jskladan> well, my opinion is also a bit torn between these two options - having the test as "standalone" is absolutely superb for non-autoqa geeks
15:49:15 <jskladan> but some tests can benefit a big time from the autoqa libs
15:49:27 <jlaska> jskladan: you can still use autoqa libs for the stand-alone test
15:49:31 <jlaska> just not autotest libs
15:49:59 <jskladan> well, if you "make install" the autoqa, that is
15:50:05 <wwoods> in short: I'm happy with both methods, but that means we need to support two ways of writing tests
15:50:18 <jlaska> at least, that was my understanding of 'stand-alone'  ... but this topic pointed out that the definition of 'stand-alone' was a bit vague
15:50:23 <wwoods> one for autoqa hackers only, and one that's for everyone
15:50:49 <jskladan> i do like the --local, because i do not have to provide a arg parser for my script
15:50:56 <jskladan> (because autoqa parses it for me)
15:51:02 <jlaska> that's nice
15:51:24 <jskladan> but i sure do understand the stand-alone benefits
15:51:49 <jskladan> and i totally agree with will on "write the test first" for non-autoqa-hackers
15:52:02 <jlaska> so it doesn't seem there is a _wrong_ way to do this ... and this can be left as an exercise for the author?
15:52:34 <wwoods> so yeah I think we're agreed here: non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers
15:53:07 <jlaska> #info in response to Vojta's upgrade-path patch, the group agreed that non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers
15:53:12 <jskladan> i belive, that we can all agree, that we should document (and highlight) the --local option
15:53:29 <jskladan> so the non-hackers are able to run current tests
15:53:39 <jlaska> more docs!
15:53:45 <jskladan> (maybe a note on the "how to write tests"?)
15:53:55 <wwoods> yeah, definitely!
15:53:58 <jskladan> jlaska: docs rock the world :)
15:54:12 <jlaska> jskladan: :D
15:54:33 <jlaska> alrighty ... anything other autoqa topics?
15:54:40 <jskladan> so, i'll add a "how to run the test" note to the wiki page
15:54:57 <jlaska> jskladan: feel free to queue up a ticket for future consumption
15:55:16 <jlaska> this is one that even I can probably help with ... esp since I'd like to learn this method
15:55:17 <jskladan> #action jskladan to add "how to run autoqa test" note to the "how to write autoqa tests" wiki
15:55:33 <jlaska> vaschenb: wwoods: anything else from you guys?
15:55:43 <wwoods> nope, I'm good
15:55:48 <vaschenb> jlaska: nothing
15:56:15 <jlaska> alright, thanks folks ... love seeing the patch review and test progress on autoqa-devel@lists.fedorahosted.org
15:56:25 <jlaska> great to see things moving forward
15:56:38 <jlaska> Okay ... it's that time again ...
15:56:48 <jlaska> #topic Open Discussion - <your topic here>
15:57:06 <jlaska> anything to discuss/review/debate that we haven't already touched on?
15:57:52 <jlaska> if nothing raised, I'll close out the meeting in 2 minutes
15:58:28 * adamw has nothing
15:58:38 <topy> I search technical documentation about architectural of fedora cluster (redhat), but I don't find nothing of interesting. Can someone drive me to some useful links ?
15:59:04 <jsmith> topy: Please don't interrupt the meeting to ask a non-related question
15:59:30 <jlaska> Closing out in 30 seconds ...
15:59:46 <Viking-Ice> jlaska: I wanted to ask if it has been considered to the nightly composes actually would use updates-testing or special koji repo for their compose..
16:00:12 <Viking-Ice> so testers would get the latest bits instead of some outdated ones
16:00:14 <jlaska> Viking-Ice: I believe we talked about that at a previous meeting ... kparal raised that topic in your absence
16:00:20 * jlaska finds
16:00:23 <Viking-Ice> Oh I see
16:00:54 <jlaska> https://fedoraproject.org/wiki/QA/Meetings/20100816#Nightly_live_composes
16:01:29 <jlaska> Viking-Ice: you might want to add your point to the QA retrospective once I post the link ...
16:01:55 <adamw> note, not captured in the summary there, nirik's explanation:
16:01:57 <adamw> "	 the idea was that we should be testing the thing that we would ship."
16:02:02 <jlaska> it would be a nice addition to the current live image process, but I don't think I'd want it to *replace* the live images used now
16:02:11 <jlaska> adamw: thanks
16:02:15 <Viking-Ice> we got alpha beta etc images for that
16:02:55 <satellit_> Are the nightly composes working? last one for soas seems to be the 18th
16:03:50 <Viking-Ice> satellit_: offtopic ask in QA
16:03:55 <satellit_> ok
16:04:01 <jlaska> satellit_: they appear to have failed ... we can follow-up with nirik in #fedora-qa
16:04:14 <jlaska> alright folks ... thanks for your time
16:04:25 <jlaska> I'll send minutes to the list
16:04:29 <jlaska> #endmeeting