15:00:24 #startmeeting Fedora QA Meeting 15:00:24 Meeting started Mon Aug 23 15:00:24 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:24 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:07 #topic Gathering .... 15:01:12 * kparal available for the next 45 minutes 15:01:13 #meetingname fedora-qa 15:01:13 The meeting name has been set to 'fedora-qa' 15:01:17 * vaschenb 15:01:42 * jsmith is here 15:02:05 * wwoods here 15:02:28 yo 15:02:56 * Viking-Ice here 15:04:41 * jlaska figures joza is lurking 15:04:59 alright ... let's get started so we can have kparal here for autoqa time 15:05:06 #topic Previous meeting follow-up 15:05:47 * jskladan lurks 15:05:54 jskladan hey there lurker! 15:06:03 #info adamw to update http://fedoraproject.org/wiki/QA:Testcase_desktop_updates to better describe the test experience on live images 15:06:27 looks like this got some recent edits 15:06:39 yeah, it should be right now. 15:07:00 thanks adamw, I see the live qualifications ... looks good 15:07:18 #info jlaska to update https://fedoraproject.org/wiki/QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver so that it better captures the post-install xdriver=vesa expectations 15:07:27 I believe this is complete ... and not because of me 15:07:30 * adamw did that too 15:07:33 adamw was a wiki editing machine last week 15:08:17 #info maxamillion update rel-eng TRAC#3860 to request RC#5 15:08:38 maxa completed this ... but turned out we had a course correction and RC#5 wasn't required 15:08:51 #info adamw and jlaska to propose artwork final release criteria 15:09:07 man, we suck 15:09:19 a classic ... adamw, my name has been on that one, but I haven't touched it. Do you want me to reach out to design-team for thoughts? 15:09:30 if you could that'd be great 15:09:36 or if you're busy i can do it 15:09:40 either way 15:09:52 alright, I'll fire that off this week 15:10:07 that's all I had from last week ... let's dive into the agenda 15:10:24 #topic F14 Alpha QA Status 15:10:41 well, first off ... congrats everyone 15:10:51 yet another Fedora release milestone behind us 15:10:55 *party poppers* 15:10:56 they just keep on coming, don't they 15:11:40 *enjoys* 15:11:48 adamw: I know you wanted to raise awareness on the radeon issue ... was there anything you wanted to discuss there? 15:12:28 jlaska: yeah, i'm a bit uneasy on that, i was assuming we'd have the readiness meeting to do it, but apparently we didn't schedule another readiness meeting even though the release was delayed (seems odd to me) 15:12:41 so i'll have to contact docs and websites teams directly, i haven't done that yet, i'll do it today 15:13:06 jsmith: anything you can think of to help elevate the visibility of http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda 15:15:00 #info adamw wants to raise awareness of known radeon display issue (http://fedoraproject.org/wiki/Common_F14_bugs#radeon-anaconda) 15:15:26 have folks been testing with F-14-Alpha + updates-testing? 15:15:36 that's what I'm running 15:16:03 it was only an improvement for me after the update ... that was good news 15:16:54 adamw: CommonBugs looks in good shape ... anything else on your radar to close out F-14-Alpha for QA? 15:16:58 me too 15:16:58 it's slightly rough with all the gnome changes 15:17:02 jlaska: I've asked the Docs folks to add an admonition to the one-page release notes for F14 Alpha 15:17:37 jsmith: ah nice, thank you 15:17:40 jsmith: awesome, thanks 15:18:01 jsmith: i'm planning on asking the web group if they can put a one-liner with a link to release notes or common bugs on the download page 15:18:02 I think we'll also highlight it in the announcement email 15:19:22 both good ideas 15:19:47 no need to self-flagellate, just a quick mention with a link is fine 15:20:04 adamw: I created an F14 QA retrospective wiki stub last week ... but I'll clean that up a bit and send that out so we can start collecting the good/bad/ugly 15:20:31 #action jlaska to publish F-14-Alpha QA retrospective page 15:22:11 in case it wasn't clear ... *huge* thanks again to all who contributed testing, especially the fast turn around on RC#4. While we did end up slipping for another issue, it was very helpful to have a sense for how the installer and desktop test matrices held up to the alpha release criteria 15:22:35 adamw: any other Alpha topics/thoughts (or Jerry Springer final thoughts)? 15:23:10 of course, what this has all showed is that the really important thing is family... 15:23:20 LOL! 15:23:24 er, nope. :) 15:23:48 alrighty ... well done 15:24:19 alright ... do you hear the music? 15:24:26 that's not Duff man ... it's AutoQA time 15:24:36 #topic AutoQA Package Acceptance 15:25:38 well, no funky updates from me - i've been testing F14 a bit, and focused mostly on the "how to write autoqa tests" wiki 15:25:41 I know F-14 pulled away some time+energy from some autoqa work, but it appears there was some progress last week 15:26:45 my patch that standardized some hook args was accepted, so it created better options for multihook tests 15:27:07 jskladan: awesome, I can't wait to see the updated wiki page ... so much has changed with some of the recent patch sets 15:27:23 standardized hook args being one of the big ones! 15:27:38 how's the multi-hook milestone looking? 15:27:39 * jlaska checks 15:27:48 and control.autoqa, and new stuff in control files and test wrappers 15:28:18 down to only 1 ticket in the multi-host milestone ... that's great 15:28:19 https://fedorahosted.org/autoqa/ticket/203 15:28:23 multi-hook milestone is almost finished, now only to schedule tests for multiple hooks, which should be piece of cake. the framework is ready 15:30:01 vaschenb: have you had a chance to look at those multi-hook tests kparal mentioned? Does that seem do-able with the time remaining? 15:31:00 vaschenb: I'm sure multi-hook guru kparal can help answer any questions too :) 15:31:12 wwoods: how's the depcheck front progressing? 15:31:15 jlaska: not yet, today I worked on upgradepath output... 15:31:35 I requested some upgradepath output format improvement at vaschenb :) 15:31:37 jlaska: we've got some new unittests in depcheck that actually, like, prove that depchecking works as expected 15:31:58 at least partially 15:32:04 jlaska: I'll take a look at it, but today I'm functionless 15:32:18 wwoods: Bonus points for using built-in python unittest module! That's really cool looking stuff 15:32:22 I split it up into a module and a CLI so we can keep the tool and the tests in different files.. little cleaner that way 15:32:43 so the next thing to do is to start using mash instead of createrepo 15:32:55 so we can actually handle multilib stuff properly 15:32:58 #info jskladan improving 'how to write autoqa tests' wiki page 15:33:00 * jlaska info's 15:33:18 #info kparal standardized hook argument patch set accepted ... multi-hook tests now possible 15:33:36 #info vaschenb improving upgrade-path test output 15:34:01 #info wwoods posted info about depcheck unit test framework 15:34:50 mostly last week was about cleaning things up - f14a kind of took priority 15:35:24 yeah ... these darn milestones! 15:35:43 #info wwoods posted thoughts on depcheck next steps -- https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001010.html 15:36:35 wwoods: you get any lucky volunteers for the mash work yet? 15:36:47 jlaska: unfortunately I think I volunteered myself 15:36:59 did everyone step back (but you)? 15:37:12 jlaska: I think that's closer to the truth 15:37:20 doh! 15:37:35 wwoods: i said i'll dig into that 15:37:44 maybe not too loud :) 15:37:56 hehe 15:38:07 jskladan: heh! maybe so. That'd be great 15:38:28 yeah, +1 on the thanks jskladan 15:38:38 wwoods: yeah, no problem at all. I'll have it on my radar, once i finish the wiki edit 15:38:41 I'll try to help round up some info from the mash 'experts' 15:38:52 s/mash experts/notting/ 15:39:09 there were 2 other non-depcheck items that I was aware of ... kparal do you want to talk about your latest patch to the list? 15:39:10 mash seems to be one of those things where nobody fully understands it, so anyone who touches it immediately becomes a new 'expert' 15:39:46 ah, yes. well 15:40:05 this patch: https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html 15:40:26 * jskladan looking forward to be the so called expert on yet another fancy field :) 15:40:38 #info some rpmguard tests failing, see kparal's proposed patch https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html 15:41:21 I think the code was wrong all the time. we requested the last release of a certain package, but we didn't check the provided repo, just its parents 15:41:38 which I didn't really understood and I believe it was wrong, until someone proves me otherwise :) 15:42:12 very possible 15:42:14 so I have rewritten it to a behaviour I suppose it's correct, and by the way it also fixed the traceback we received before 15:42:26 should I just apply this patch to our current instance? 15:42:38 jlaska: I believe it should work (better than before) 15:43:10 kparal: okay, I'll apply and respond with results 15:43:25 jlaska: yes please 15:43:29 * kparal has to run, sorry 15:43:30 #action jlaska apply and provide results against autoqa patch (https://fedorahosted.org/pipermail/autoqa-devel/2010-August/001028.html) 15:43:34 kparal: cya 15:43:47 the other non-depcheck issue was vaschenb upgrade-path test 15:44:16 I commented that I thought we'd probably want a method of running this test outside of the test harness (for folks who play the autoqa home-game) 15:44:39 but then kparal and vaschenb pointed out that you can run the tests locally using a method I wasn't aware of 15:44:49 just using 'autoqa post-bodhi-update --local ...' 15:44:50 * jskladan likes (and heavily uses) the --local) 15:45:08 wwoods: did you have any thoughts on that approach vs a stand-alone script? 15:46:20 either way seems fine to me 15:46:30 I really like the stand-alone version, but then that means we have to write multiple option parsers etc... 15:47:07 I don't often install 'autoqa' when testing out of git ... so I'm not familiar with this method 15:47:33 I guess I just want to make sure we recognize that writing a test this way is not the most obvious (or simplest) path for general audiences 15:47:52 as long as we document how to run the test somewhere, right? 15:48:08 if you're skilled at hacking around in autoqa already then yeah, this totally works and saves you having to write option parsers etc. 15:48:46 but for people who are like "hey I want to write a test, how does autoqa work" I'm still going to suggest "Write the test first, you don't need to know the full details of how autoqa works until you have a working test" 15:48:57 well, my opinion is also a bit torn between these two options - having the test as "standalone" is absolutely superb for non-autoqa geeks 15:49:15 but some tests can benefit a big time from the autoqa libs 15:49:27 jskladan: you can still use autoqa libs for the stand-alone test 15:49:31 just not autotest libs 15:49:59 well, if you "make install" the autoqa, that is 15:50:05 in short: I'm happy with both methods, but that means we need to support two ways of writing tests 15:50:18 at least, that was my understanding of 'stand-alone' ... but this topic pointed out that the definition of 'stand-alone' was a bit vague 15:50:23 one for autoqa hackers only, and one that's for everyone 15:50:49 i do like the --local, because i do not have to provide a arg parser for my script 15:50:56 (because autoqa parses it for me) 15:51:02 that's nice 15:51:24 but i sure do understand the stand-alone benefits 15:51:49 and i totally agree with will on "write the test first" for non-autoqa-hackers 15:52:02 so it doesn't seem there is a _wrong_ way to do this ... and this can be left as an exercise for the author? 15:52:34 so yeah I think we're agreed here: non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers 15:53:07 #info in response to Vojta's upgrade-path patch, the group agreed that non-standalone tests are totally acceptable - and really handy for proficient autoqa hackers 15:53:12 i belive, that we can all agree, that we should document (and highlight) the --local option 15:53:29 so the non-hackers are able to run current tests 15:53:39 more docs! 15:53:45 (maybe a note on the "how to write tests"?) 15:53:55 yeah, definitely! 15:53:58 jlaska: docs rock the world :) 15:54:12 jskladan: :D 15:54:33 alrighty ... anything other autoqa topics? 15:54:40 so, i'll add a "how to run the test" note to the wiki page 15:54:57 jskladan: feel free to queue up a ticket for future consumption 15:55:16 this is one that even I can probably help with ... esp since I'd like to learn this method 15:55:17 #action jskladan to add "how to run autoqa test" note to the "how to write autoqa tests" wiki 15:55:33 vaschenb: wwoods: anything else from you guys? 15:55:43 nope, I'm good 15:55:48 jlaska: nothing 15:56:15 alright, thanks folks ... love seeing the patch review and test progress on autoqa-devel@lists.fedorahosted.org 15:56:25 great to see things moving forward 15:56:38 Okay ... it's that time again ... 15:56:48 #topic Open Discussion - 15:57:06 anything to discuss/review/debate that we haven't already touched on? 15:57:52 if nothing raised, I'll close out the meeting in 2 minutes 15:58:28 * adamw has nothing 15:58:38 I search technical documentation about architectural of fedora cluster (redhat), but I don't find nothing of interesting. Can someone drive me to some useful links ? 15:59:04 topy: Please don't interrupt the meeting to ask a non-related question 15:59:30 Closing out in 30 seconds ... 15:59:46 jlaska: I wanted to ask if it has been considered to the nightly composes actually would use updates-testing or special koji repo for their compose.. 16:00:12 so testers would get the latest bits instead of some outdated ones 16:00:14 Viking-Ice: I believe we talked about that at a previous meeting ... kparal raised that topic in your absence 16:00:20 * jlaska finds 16:00:23 Oh I see 16:00:54 https://fedoraproject.org/wiki/QA/Meetings/20100816#Nightly_live_composes 16:01:29 Viking-Ice: you might want to add your point to the QA retrospective once I post the link ... 16:01:55 note, not captured in the summary there, nirik's explanation: 16:01:57 " the idea was that we should be testing the thing that we would ship." 16:02:02 it would be a nice addition to the current live image process, but I don't think I'd want it to *replace* the live images used now 16:02:11 adamw: thanks 16:02:15 we got alpha beta etc images for that 16:02:55 Are the nightly composes working? last one for soas seems to be the 18th 16:03:50 satellit_: offtopic ask in QA 16:03:55 ok 16:04:01 satellit_: they appear to have failed ... we can follow-up with nirik in #fedora-qa 16:04:14 alright folks ... thanks for your time 16:04:25 I'll send minutes to the list 16:04:29 #endmeeting