15:00:24 <jlaska> #startmeeting Fedora QA Meeting 15:00:24 <zodbot> Meeting started Mon Aug 23 15:00:24 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:24 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:07 <jlaska> #topic Gathering .... 15:01:12 * kparal available for the next 45 minutes 15:01:13 <jlaska> #meetingname fedora-qa 15:01:13 <zodbot> The meeting name has been set to 'fedora-qa' 15:01:17 * vaschenb 15:01:42 * jsmith is here 15:02:05 * wwoods here 15:02:28 <adamw> yo 15:02:56 * Viking-Ice here 15:04:41 * jlaska figures joza is lurking 15:04:59 <jlaska> alright ... let's get started so we can have kparal here for autoqa time 15:05:06 <jlaska> #topic Previous meeting follow-up 15:05:47 * jskladan lurks 15:05:54 <jlaska> jskladan hey there lurker! 15:06:03 <jlaska> #info adamw to update http://fedoraproject.org/wiki/QA:Testcase_desktop_updates to better describe the test experience on live images 15:06:27 <jlaska> looks like this got some recent edits 15:06:39 <adamw> yeah, it should be right now. 15:07:00 <jlaska> thanks adamw, I see the live qualifications ... looks good 15:07:18 <jlaska> #info jlaska to update https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver so that it better captures the post-install xdriver=vesa expectations 15:07:27 <jlaska> I believe this is complete ... and not because of me 15:07:30 * adamw did that too 15:07:33 <jlaska> adamw was a wiki editing machine last week 15:08:17 <jlaska> #info maxamillion update rel-eng TRAC#3860 to request RC#5 15:08:38 <jlaska> maxa completed this ... but turned out we had a course correction and RC#5 wasn't required 15:08:51 <jlaska> #info adamw and jlaska to propose artwork final release criteria 15:09:07 <adamw> man, we suck 15:09:19 <jlaska> a classic ... adamw, my name has been on that one, but I haven't touched it. Do you want me to reach out to design-team for thoughts? 15:09:30 <adamw> if you could that'd be great 15:09:36 <adamw> or if you're busy i can do it 15:09:40 <adamw> either way 15:09:52 <jlaska> alright, I'll fire that off this week 15:10:07 <jlaska> that's all I had from last week ... let's dive into the agenda 15:10:24 <jlaska> #topic F14 Alpha QA Status 15:10:41 <jlaska> well, first off ... congrats everyone 15:10:51 <jlaska> yet another Fedora release milestone behind us 15:10:55 <adamw> *party poppers* 15:10:56 <jlaska> they just keep on coming, don't they 15:11:40 <kparal> *enjoys* 15:11:48 <jlaska> adamw: I know you wanted to raise awareness on the radeon issue ... was there anything you wanted to discuss there? 15:12:28 <adamw> jlaska: yeah, i'm a bit uneasy on that, i was assuming we'd have the readiness meeting to do it, but apparently we didn't schedule another readiness meeting even though the release was delayed (seems odd to me) 15:12:41 <adamw> so i'll have to contact docs and websites teams directly, i haven't done that yet, i'll do it today 15:13:06 <jlaska> jsmith: anything you can think of to help elevate the visibility of http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda 15:15:00 <jlaska> #info adamw wants to raise awareness of known radeon display issue (http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda) 15:15:26 <jlaska> have folks been testing with F-14-Alpha + updates-testing? 15:15:36 <wwoods> that's what I'm running 15:16:03 <jlaska> it was only an improvement for me after the update ... that was good news 15:16:54 <jlaska> adamw: CommonBugs looks in good shape ... anything else on your radar to close out F-14-Alpha for QA? 15:16:58 <adamw> me too 15:16:58 <adamw> it's slightly rough with all the gnome changes 15:17:02 <jsmith> jlaska: I've asked the Docs folks to add an admonition to the one-page release notes for F14 Alpha 15:17:37 <jlaska> jsmith: ah nice, thank you 15:17:40 <adamw> jsmith: awesome, thanks 15:18:01 <adamw> jsmith: i'm planning on asking the web group if they can put a one-liner with a link to release notes or common bugs on the download page 15:18:02 <jsmith> I think we'll also highlight it in the announcement email 15:19:22 <jlaska> both good ideas 15:19:47 <adamw> no need to self-flagellate, just a quick mention with a link is fine 15:20:04 <jlaska> adamw: I created an F14 QA retrospective wiki stub last week ... but I'll clean that up a bit and send that out so we can start collecting the good/bad/ugly 15:20:31 <jlaska> #action jlaska to publish F-14-Alpha QA retrospective page 15:22:11 <jlaska> in case it wasn't clear ... *huge* thanks again to all who contributed testing, especially the fast turn around on RC#4. While we did end up slipping for another issue, it was very helpful to have a sense for how the installer and desktop test matrices held up to the alpha release criteria 15:22:35 <jlaska> adamw: any other Alpha topics/thoughts (or Jerry Springer final thoughts)? 15:23:10 <adamw> of course, what this has all showed is that the really important thing is family... 15:23:20 <jlaska> LOL! 15:23:24 <adamw> er, nope. :) 15:23:48 <jlaska> alrighty ... well done 15:24:19 <jlaska> alright ... do you hear the music? 15:24:26 <jlaska> that's not Duff man ... it's AutoQA time 15:24:36 <jlaska> #topic AutoQA Package Acceptance 15:25:38 <jskladan> well, no funky updates from me - i've been testing F14 a bit, and focused mostly on the "how to write autoqa tests" wiki 15:25:41 <jlaska> I know F-14 pulled away some time+energy from some autoqa work, but it appears there was some progress last week 15:26:45 <kparal> my patch that standardized some hook args was accepted, so it created better options for multihook tests 15:27:07 <jlaska> jskladan: awesome, I can't wait to see the updated wiki page ... so much has changed with some of the recent patch sets 15:27:23 <jlaska> standardized hook args being one of the big ones! 15:27:38 <jlaska> how's the multi-hook milestone looking? 15:27:39 * jlaska checks 15:27:48 <wwoods> and control.autoqa, and new stuff in control files and test wrappers 15:28:18 <jlaska> down to only 1 ticket in the multi-host milestone ... that's great 15:28:19 <jlaska> https://fedorahosted.org/autoqa/ticket/203 15:28:23 <kparal> multi-hook milestone is almost finished, now only to schedule tests for multiple hooks, which should be piece of cake. the framework is ready 15:30:01 <jlaska> vaschenb: have you had a chance to look at those multi-hook tests kparal mentioned? Does that seem do-able with the time remaining? 15:31:00 <jlaska> vaschenb: I'm sure multi-hook guru kparal can help answer any questions too :) 15:31:12 <jlaska> wwoods: how's the depcheck front progressing? 15:31:15 <vaschenb> jlaska: not yet, today I worked on upgradepath output... 15:31:35 <kparal> I requested some upgradepath output format improvement at vaschenb :) 15:31:37 <wwoods> jlaska: we've got some new unittests in depcheck that actually, like, prove that depchecking works as expected 15:31:58 <wwoods> at least partially 15:32:04 <vaschenb> jlaska: I'll take a look at it, but today I'm functionless 15:32:18 <jlaska> wwoods: Bonus points for using built-in python unittest module! That's really cool looking stuff 15:32:22 <wwoods> I split it up into a module and a CLI so we can keep the tool and the tests in different files.. little cleaner that way 15:32:43 <wwoods> so the next thing to do is to start using mash instead of createrepo 15:32:55 <wwoods> so we can actually handle multilib stuff properly 15:32:58 <jlaska> #info jskladan improving 'how to write autoqa tests' wiki page 15:33:00 * jlaska info's 15:33:18 <jlaska> #info kparal standardized hook argument patch set accepted ... multi-hook tests now possible 15:33:36 <jlaska> #info vaschenb improving upgrade-path test output 15:34:01 <jlaska> #info wwoods posted info about depcheck unit test framework 15:34:50 <wwoods> mostly last week was about cleaning things up - f14a kind of took priority 15:35:24 <jlaska> yeah ... these darn milestones! 15:35:43 <jlaska> #info wwoods posted thoughts on depcheck next steps -- https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001010.html 15:36:35 <jlaska> wwoods: you get any lucky volunteers for the mash work yet? 15:36:47 <wwoods> jlaska: unfortunately I think I volunteered myself 15:36:59 <jlaska> did everyone step back (but you)? 15:37:12 <jsmith> jlaska: I think that's closer to the truth 15:37:20 <jlaska> doh! 15:37:35 <jskladan> wwoods: i said i'll dig into that 15:37:44 <jskladan> maybe not too loud :) 15:37:56 <jlaska> hehe 15:38:07 <wwoods> jskladan: heh! maybe so. That'd be great 15:38:28 <jlaska> yeah, +1 on the thanks jskladan 15:38:38 <jskladan> wwoods: yeah, no problem at all. I'll have it on my radar, once i finish the wiki edit 15:38:41 <wwoods> I'll try to help round up some info from the mash 'experts' 15:38:52 <skvidal> s/mash experts/notting/ 15:39:09 <jlaska> there were 2 other non-depcheck items that I was aware of ... kparal do you want to talk about your latest patch to the list? 15:39:10 <wwoods> mash seems to be one of those things where nobody fully understands it, so anyone who touches it immediately becomes a new 'expert' 15:39:46 <kparal> ah, yes. well 15:40:05 <kparal> this patch: https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html 15:40:26 * jskladan looking forward to be the so called expert on yet another fancy field :) 15:40:38 <jlaska> #info some rpmguard tests failing, see kparal's proposed patch https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html 15:41:21 <kparal> I think the code was wrong all the time. we requested the last release of a certain package, but we didn't check the provided repo, just its parents 15:41:38 <kparal> which I didn't really understood and I believe it was wrong, until someone proves me otherwise :) 15:42:12 <wwoods> very possible 15:42:14 <kparal> so I have rewritten it to a behaviour I suppose it's correct, and by the way it also fixed the traceback we received before 15:42:26 <jlaska> should I just apply this patch to our current instance? 15:42:38 <kparal> jlaska: I believe it should work (better than before) 15:43:10 <jlaska> kparal: okay, I'll apply and respond with results 15:43:25 <kparal> jlaska: yes please 15:43:29 * kparal has to run, sorry 15:43:30 <jlaska> #action jlaska apply and provide results against autoqa patch (https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html) 15:43:34 <jlaska> kparal: cya 15:43:47 <jlaska> the other non-depcheck issue was vaschenb upgrade-path test 15:44:16 <jlaska> I commented that I thought we'd probably want a method of running this test outside of the test harness (for folks who play the autoqa home-game) 15:44:39 <jlaska> but then kparal and vaschenb pointed out that you can run the tests locally using a method I wasn't aware of 15:44:49 <jlaska> just using 'autoqa post-bodhi-update --local ...' 15:44:50 * jskladan likes (and heavily uses) the --local) 15:45:08 <jlaska> wwoods: did you have any thoughts on that approach vs a stand-alone script? 15:46:20 <wwoods> either way seems fine to me 15:46:30 <jlaska> I really like the stand-alone version, but then that means we have to write multiple option parsers etc... 15:47:07 <jlaska> I don't often install 'autoqa' when testing out of git ... so I'm not familiar with this method 15:47:33 <wwoods> I guess I just want to make sure we recognize that writing a test this way is not the most obvious (or simplest) path for general audiences 15:47:52 <jlaska> as long as we document how to run the test somewhere, right? 15:48:08 <wwoods> if you're skilled at hacking around in autoqa already then yeah, this totally works and saves you having to write option parsers etc. 15:48:46 <wwoods> but for people who are like "hey I want to write a test, how does autoqa work" I'm still going to suggest "Write the test first, you don't need to know the full details of how autoqa works until you have a working test" 15:48:57 <jskladan> well, my opinion is also a bit torn between these two options - having the test as "standalone" is absolutely superb for non-autoqa geeks 15:49:15 <jskladan> but some tests can benefit a big time from the autoqa libs 15:49:27 <jlaska> jskladan: you can still use autoqa libs for the stand-alone test 15:49:31 <jlaska> just not autotest libs 15:49:59 <jskladan> well, if you "make install" the autoqa, that is 15:50:05 <wwoods> in short: I'm happy with both methods, but that means we need to support two ways of writing tests 15:50:18 <jlaska> at least, that was my understanding of 'stand-alone' ... but this topic pointed out that the definition of 'stand-alone' was a bit vague 15:50:23 <wwoods> one for autoqa hackers only, and one that's for everyone 15:50:49 <jskladan> i do like the --local, because i do not have to provide a arg parser for my script 15:50:56 <jskladan> (because autoqa parses it for me) 15:51:02 <jlaska> that's nice 15:51:24 <jskladan> but i sure do understand the stand-alone benefits 15:51:49 <jskladan> and i totally agree with will on "write the test first" for non-autoqa-hackers 15:52:02 <jlaska> so it doesn't seem there is a _wrong_ way to do this ... and this can be left as an exercise for the author? 15:52:34 <wwoods> so yeah I think we're agreed here: non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers 15:53:07 <jlaska> #info in response to Vojta's upgrade-path patch, the group agreed that non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers 15:53:12 <jskladan> i belive, that we can all agree, that we should document (and highlight) the --local option 15:53:29 <jskladan> so the non-hackers are able to run current tests 15:53:39 <jlaska> more docs! 15:53:45 <jskladan> (maybe a note on the "how to write tests"?) 15:53:55 <wwoods> yeah, definitely! 15:53:58 <jskladan> jlaska: docs rock the world :) 15:54:12 <jlaska> jskladan: :D 15:54:33 <jlaska> alrighty ... anything other autoqa topics? 15:54:40 <jskladan> so, i'll add a "how to run the test" note to the wiki page 15:54:57 <jlaska> jskladan: feel free to queue up a ticket for future consumption 15:55:16 <jlaska> this is one that even I can probably help with ... esp since I'd like to learn this method 15:55:17 <jskladan> #action jskladan to add "how to run autoqa test" note to the "how to write autoqa tests" wiki 15:55:33 <jlaska> vaschenb: wwoods: anything else from you guys? 15:55:43 <wwoods> nope, I'm good 15:55:48 <vaschenb> jlaska: nothing 15:56:15 <jlaska> alright, thanks folks ... love seeing the patch review and test progress on autoqa-devel@lists.fedorahosted.org 15:56:25 <jlaska> great to see things moving forward 15:56:38 <jlaska> Okay ... it's that time again ... 15:56:48 <jlaska> #topic Open Discussion - <your topic here> 15:57:06 <jlaska> anything to discuss/review/debate that we haven't already touched on? 15:57:52 <jlaska> if nothing raised, I'll close out the meeting in 2 minutes 15:58:28 * adamw has nothing 15:58:38 <topy> I search technical documentation about architectural of fedora cluster (redhat), but I don't find nothing of interesting. Can someone drive me to some useful links ? 15:59:04 <jsmith> topy: Please don't interrupt the meeting to ask a non-related question 15:59:30 <jlaska> Closing out in 30 seconds ... 15:59:46 <Viking-Ice> jlaska: I wanted to ask if it has been considered to the nightly composes actually would use updates-testing or special koji repo for their compose.. 16:00:12 <Viking-Ice> so testers would get the latest bits instead of some outdated ones 16:00:14 <jlaska> Viking-Ice: I believe we talked about that at a previous meeting ... kparal raised that topic in your absence 16:00:20 * jlaska finds 16:00:23 <Viking-Ice> Oh I see 16:00:54 <jlaska> https://fedoraproject.org/wiki/QA/Meetings/20100816#Nightly_live_composes 16:01:29 <jlaska> Viking-Ice: you might want to add your point to the QA retrospective once I post the link ... 16:01:55 <adamw> note, not captured in the summary there, nirik's explanation: 16:01:57 <adamw> " the idea was that we should be testing the thing that we would ship." 16:02:02 <jlaska> it would be a nice addition to the current live image process, but I don't think I'd want it to *replace* the live images used now 16:02:11 <jlaska> adamw: thanks 16:02:15 <Viking-Ice> we got alpha beta etc images for that 16:02:55 <satellit_> Are the nightly composes working? last one for soas seems to be the 18th 16:03:50 <Viking-Ice> satellit_: offtopic ask in QA 16:03:55 <satellit_> ok 16:04:01 <jlaska> satellit_: they appear to have failed ... we can follow-up with nirik in #fedora-qa 16:04:14 <jlaska> alright folks ... thanks for your time 16:04:25 <jlaska> I'll send minutes to the list 16:04:29 <jlaska> #endmeeting