16:00:05 #startmeeting Fedora QA Meeting 16:00:05 Meeting started Mon Oct 12 16:00:05 2009 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:05 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:18 #topic Gathering in the lobby 16:00:49 * kparal gathered 16:00:57 Howdy kparal 16:01:14 We will be without a few folks today, including lili, rhe, dpravec and adamw 16:01:36 who knows, that could make for a short meeting :) 16:01:43 wwoods: Viking-Ice: around? 16:01:48 * Oxf13 16:01:54 I think Oxf13 went on a breakfa ... nope 16:01:57 Oxf13: howdy! 16:02:02 * Viking-Ice here.. 16:02:11 jlaska: eating it now 16:02:13 * iarlyy Hi all 16:02:20 jlaska here ... 16:02:56 * wwoods around..ish 16:03:20 Viking-Ice: iarlyy: denise: wwoods: greetings folks, welcome 16:03:49 * pknirsch is a lurker and hides 16:03:58 pknirsch: we welcome lurkers 16:04:02 :) 16:04:08 okay, let's get started 16:04:17 the script is available at https://www.redhat.com/archives/fedora-test-list/2009-October/msg00194.html 16:04:24 #topic Previous meeting follow-up 16:04:33 Just a few quick action items to walk through 16:04:37 # jlaska to catch up with mmcgrath on delivery of autoqa hardware to PHX 16:05:29 Had a short chat with mmcgrath about the game plan once the hardware arrives. That helped outline a few action items needed on my end in terms of preparing the AutoQA 'israwhidebroken' software environment 16:06:02 there are tickets already in the autoqa instance that are assigned to me ... I'd like to get some traction on the packaging and autotest-0.11 upgrade this week 16:06:07 but beta is the priority 16:06:38 #info hardware shipment in transit to PHX2 ... jlaska and mmcgrath will work the next steps as hardware comes online 16:07:32 next up ... 16:07:38 # jlaska - sync-up with clumens to identify any blocking issues from installer test days 16:08:05 I had a brief chat with a few folks from the anaconda-devel group for help in isolating blockers from the 2 previous installer test days 16:08:13 thanks to denise dlehman and clumens for help there 16:08:24 3 bugs landed on the blocker list from the first anaconda storage test day 16:08:41 and I believe 2 of the bugs filed by greenlion landed last week 16:08:51 on this topic, I respectfully request we don't schedule these things during a freeze when there is minimal (one might say no) time to fix any of the issues found 16:09:01 it's a good way to guarantee a slip 16:09:14 so don't test during a freeze? 16:09:31 by all means test, but please schedule it /before/ the freeze 16:09:37 our freezes are extremely short 16:09:41 we did both this milestones 16:09:57 there really isn't time built in to freeze, test, spend 2 weeks fixing everything found, get a release out, unfreeze 16:09:58 we had 2 pre-freeze test events that Liam organized 16:10:14 #link https://fedoraproject.org/wiki/Test_Results:Fedora_12_PreBeta_Install 16:10:18 #link https://fedoraproject.org/wiki/Test_Results:Fedora_12_Beta_TC_Install 16:10:23 a test day like anaconda storage test day done after the freeze is about 100% guaranteed to cause a slip 16:10:40 and then we followed up with 2 installer test events to handle the post-freeze change 16:10:51 Oxf13: we have test coverage both before and after ... I'm not sure what else we can offer? 16:11:01 maybe I'm just grumpy 16:11:12 hah, it is early monday for ya! 16:11:19 but it really seemed like we waited until after the freeze happened to find some installer bugs that suddenly are beta blockers, when it was too late to fix them 16:11:42 unless these really are regressions from changes since we entered the freeze... 16:12:02 there were a good amount of changes since the Beta Test Compose and the pre-beta testing 16:12:08 all around the distro 16:12:22 yes, and that's expected given how fast Fedora moves 16:12:38 I'll look back at those 5 bugs that landed on the blocker list ... but I feel like this time we had the freeze book-ended with testing 16:12:38 but I'm still not convinced that the current or late added anaconda blockers were regressions since the TC 16:12:52 but it would be good to identify opportunities to locate those bugs sooner 16:13:16 another challenge all around is just time 16:13:23 it takes time to find and file good bug reports 16:13:30 it takes time to digest and act on good bug reports 16:13:53 we try our best to find the low hanging fruit first 16:14:34 #info Oxf13 expressed concerns that recent blocker bugs could have been found prior to the freeze 16:15:14 it just seemed like we were in good shape until the storage test day happened, and then it seemed that a number of beta worthy bugs popped up that basically caused instant slip 16:15:44 and I'm really wondering if those bugs were regressions or just stuff that didn't get tested until the test day 16:15:55 (when it was too late to fix them without slipping) 16:16:06 well, we had the best coverage we've ever had in the QA group with this test compose ... https://fedoraproject.org/wiki/Test_Results:Fedora_12_Beta_TC_Install 16:16:45 I'll take an action item to look at the 5 bugs that were escalated to F12Beta related to anaconda and see if I can get more detail 16:17:20 #action jlaska will investigate the 5 anaconda F12Beta issues to determine when the failures were introduced 16:17:57 I'll hold off before taking any action there 16:18:18 okay ... next up ... I had poelcat checking in on a bz 16:18:25 # poelcat to check in on beta blocker status of 510249 16:18:45 and I think we already had some traction there (thanks mclasen) 16:19:12 that's all I had for last week 16:19:23 anything I missed someone else would like to discuss? 16:19:27 (from last week) 16:20:04 alright ... first up ... 16:20:06 #topic F-12-Beta RC1 QA strategy 16:20:33 We are waiting on clarification for the last remaining unresolved F12Beta blocking issue 16:20:36 which is ... 16:20:41 bug#526899 16:20:42 Bug https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=526899 medium, low, ---, anaconda-maint-list, NEW, Encrypted partitions has "Unknown" type in table and ext4 in editor 16:21:07 and most recent status .. denise: yes. I'm trying to figure out out how to debug on a live install. 16:21:18 that should let us know if we can proceed with the ISO's Oxf13 has composed already, or whether we can expect a new drop later today 16:21:25 denise: ah good, thanks for the update 16:22:03 denise: if dlehman needs systems/environment testing of a patch ... I'm more than willing to help out 16:22:18 jlaska, thanks .. 16:22:32 #info dlehman determining how to debug 526899 on a live install 16:23:08 so we'll stay tuned for an update on that issue 16:23:22 Liam has already constructed a Beta RC1 test matrix in preparation for the ISO drop 16:24:01 Liam, Rui and I have begun testing non-storage related use cases and recording results into the wiki 16:24:05 #link https://fedoraproject.org/wiki/Test_Results:Fedora_12_Beta_RC1_Install 16:24:36 but with regards to the ISO drop, we'll wait for more news on this last bug 16:24:52 I believe today is the go/no_go ... Oxf13 ? 16:24:58 in theory 16:25:50 heh, in theory and in the schedule 16:25:55 #link http://poelstra.fedorapeople.org/schedules/f-12/f-12-releng-tasks.html 16:26:27 alright, that's really all I have on the Beta QA update ... Liam will have something out to the list once we have confirmation on the release candidate 16:26:53 Liam will outline a plan of attack for the test matrix ... I encourage folks to pitch in with a few tests on their systems 16:27:03 using live media, boot.iso, kvm ... whatever 16:27:21 any questions/concerns on the Beta? 16:28:03 alllrighty ... stay tuned to fedora-test-list for updates then 16:28:10 next up ... wwoods and kparal ... 16:28:15 #topic AutoQA Updates 16:28:21 wwoods you want to kick it off? 16:29:31 * wwoods checking notes 16:29:47 I've spent the past couple days working on that preupgrade blocker so I gotta remember what I was doing before that 16:30:04 heh, and yeah by the way ... way to pinpoint that issue! 16:30:23 yeah but it turns out to have some other side effects, *plus* I didn't test my patch (booooo) 16:30:34 so there will probably be a 1.1.1-4 build of preupgrade soon 16:30:55 #info wwoods isolated and patched the preupgrade issue (526208), another update coming in a 1.1.1-4 build of preupgrade 16:31:12 anyway, autoqa: still haven't found a satisfactory way to get a reference from a test back to the results of that test in the autotest frontend 16:31:13 wwoods: good catch finding that before beta 16:31:24 adding that info to the codepaths in autotest turns out to be a huge pain 16:31:50 but that's really the last code change needed for the first major milestone: 16:32:03 https://fedorahosted.org/autoqa/ticket/65 16:32:29 * jlaska notes ... we need a tracbot 16:32:35 looking through the code has given me an idea for a simpler way to accomplish this 16:32:42 (I think zodbot knows how to deal with trac stuff but I don't know how it works) 16:32:58 when I get back to autoqa that's what I'll be looking at 16:33:09 I also set up a second instance on a separate machine just to make sure it's possible 16:33:19 very good 16:33:41 (it worked OK so far - gotta export the data off my workstation so I can reinstall my system with F12Beta) 16:33:43 wwoods: will this link back to the results on the autotest server also be posted into the autoqa-results output ... e.g. https://fedorahosted.org/pipermail/autoqa-results/2009-October/001438.html 16:33:47 * nirik notes zodbot just has basically aliases for trac urls so you can look up specific tickets in a trac instance currently. 16:34:24 jlaska: that's *kind of* the goal - links from israwhidebroken (and the emails) back to the full test results/logs 16:34:38 the emails themselves are really just a stopgap until we have Something Better 16:35:24 right on 16:36:08 dcantrell has been helping by looking into the rats_install/i386 failure ... this might be a good 'test case' to determine what info is needed by devel to assess problems 16:36:28 I'll follow-up with you if any ideas come up ... but it sounds like you've got the cases planned already 16:36:45 right on 16:37:31 wwoods: anything else new and exciting on the horizon in the world of AutoQA 16:38:02 well hopefully we'll have the production stuff coming online soon 16:38:10 so, yay, full test results 16:38:22 um ... yes ... awesome! :) 16:38:36 oh the other thing I was looking at was getting the critpath results put into a wiki page 16:38:53 python-fedora even has a nice Wiki client class 16:39:08 unfortunately it uses JSON and AFAICT you can't edit/create pages with the JSON format/methods 16:39:13 * nirik also notes zodbot can do rss feeds -> irc if that would be useful. ;) 16:39:53 wwoods: here's that link I mentioned from mmcgrath - http://git.fedorahosted.org/git/?p=fedora-data.git 16:40:19 #info wwoods looking into posting critical path package list to the wiki - currently blocked since JSON doesn't allow edit/create pages 16:40:31 * Oxf13 has to step out for a house inspection. 16:40:36 Oxf13: good luck! 16:41:00 nice, I'll check that out 16:41:11 wwoods: I might need your skills later this week for some thoughts around packaging your israwhidebroken git repo for "production" 16:41:24 we're not packaging a repo 16:41:30 as you've seen before, Ican probably get it packaged ... but it might not be the "ideal" way of doing it 16:41:37 wwoods: no, I meant your TG code 16:41:41 as I understand it TG apps can be nicely packaged as with most python things 16:41:59 so just using setuptools? 16:42:02 it's got a setup.py and all 16:42:06 gotcha gotcha 16:42:15 heh, there we go ... that's something I can do :) 16:42:15 so yeah, AFAIK it's just a standard python-template RPM spec 16:42:21 nice 16:42:43 #action jlaska to investigate packaging of israwhidebroken.com for production instance 16:43:06 wwoods: I don't know if that'll need an official non fedorapeople git repo ... or if it matters etc... 16:43:19 right, we'll see what the admin dudes think 16:43:34 roger 16:43:44 but yeah, this link describes what you need to do to set up a TG app in production: 16:43:55 #link http://fedoraproject.org/wiki/TurboGears_Infrastructure_SOP 16:44:04 that's the one, thx 16:44:30 alright, shall we turn it over to kparal for an update on the project currently known as packagediff? 16:44:51 ok 16:44:58 sure! 16:45:02 Well, I have more or less implemented majority of the important checks for differences between rpm packages. You can see the progress here: https://fedorahosted.org/autoqa/ticket/54#comment:2 16:45:10 cool, thanks for the updates wwoods! 16:45:14 #topic AutoQA Updates (packagediff) 16:45:24 There are surely many bugs in the moment, but I am just working on some fake rpm packages to test the script. So hopefully it will be flawless soon [joke]. 16:45:39 When this is done, I plan to put the script into the public, so everyone can comment on that and tell me what I have done wrong :) And we can at the same time start to integrate it with AutoQA. 16:45:50 #info kparal has implemented a majority of the important rpm checks (see https://fedorahosted.org/autoqa/ticket/54#comment:2) 16:46:08 I would also like to rename 'packagediff' to something more suitable and maybe even funnier. If you have some idea other than my current 'rpmguard', 'rpmjudge' or 'rpmblame', let me know. 16:46:53 The last thing is that my internship in Fedora QA will end soon and don't know yet if I will continue. The question is where to store the resulting script. I can leave it in my fedorapeople account, or I can publish it elsewhere. 16:47:29 #info kparal taking project naming suggestions (candidates include: 'rpmguard', 'rpmjudge' or 'rpmblame') 16:47:42 :) 16:47:51 kparal: I don't know if this would make sense inside the autoqa project ... wwoods would be best for guidance there 16:48:17 why not rpmdiff ? 16:48:23 already exists 16:48:36 and actually i am using rpmdiff's output 16:49:18 kparal: I would think long term ... having all your changes merged into rpmdiff would be ideal 16:49:22 that's my guess at least 16:49:45 i believe so too, merged or available as another tool in rpmlint package 16:49:45 but I suspect we might need a location for the test script while/until those changes are integrated upstream 16:50:19 right. it can be part of autoqa, or just somewhere privately hidden 16:50:28 but it won't be so visible in that case 16:51:16 I think you've got the best 2 approaches on the table so far ... I guess that depends on what wwoods (autoqa) or skvidal (rpmlint) think? 16:51:52 or maybe ville skytta 16:51:59 yeah if you just want to check it in somewhere, we can treat it like an autoqa test 16:52:31 alright, that can be a good temporary solution 16:52:34 wwoods: does it fit the autoqa module to create a tests/rpmdiff (that has a wrapper that calls rpmlint's rpmdiff) 16:53:25 I think it does since that's one of those tests that is independent of a package 16:53:36 kparal: that might be a good segway into your next topic then ... integrating with autoqa 16:54:06 you mean task. yes. 16:54:14 yeah, you got it 16:54:38 okay, I'll get off my butt and request autoqa-devel list now 16:54:42 jlaska: yeah, that would work. unfortunately we don't yet have a hook/watcher for post-build tests 16:54:49 but there's no harm in having a place for the test(s) to live 16:55:00 #action jlaska to request autoqa-devel@ for autoqa development discussion and patch review 16:55:05 wwoods: okay cool 16:55:12 there is also one question 16:55:28 do we want this script to be also available as a separate shell tool, or just integrated as autoqa test? 16:55:53 ideally, it will just be the 'rpmdiff' tool provided by rpmlint right? 16:55:56 or maybe it can be done both at the same time 16:56:22 but the wrapper for the script (and short-term adjustments) might live in autoqa/tests/rpmdiff ? 16:56:31 at least that's my guess 16:56:35 ok 16:56:54 i got a little confused, forget it :) 16:56:58 wwoods is the expert there, but that seems sane-ish 16:57:03 no, the tool should definitely work as a normal shell tool 16:57:21 and then we write a nice autoqa wrapper for it 16:57:30 right, i see now 16:57:33 ah okay ... so there's an extra layer 16:57:43 kparal: doh! sorry :( 16:57:50 just like the install test.. you can just run install.py independent of autoqa/autotest 16:57:53 same with sanity.py 16:58:42 ok. that's all from me i believe. 16:59:06 kparal: okay, thanks for the update ... and nice job working down the different comparative tests 16:59:35 alright, next up I asked Viking-Ice for his thoughts on the next-steps on the "How_to_Debug_" thread 16:59:42 #topic How_to_Debug_ wiki pages 17:00:08 Viking-Ice: got a few minutes to discuss what the next steps needed are on that front? 17:00:51 # info Viking_Ice started a great discussion around how best to improve the current Category:Debugging content - https://www.redhat.com/archives/fedora-test-list/2009-October/msg00112.html 17:00:55 #info Viking_Ice started a great discussion around how best to improve the current Category:Debugging content - https://www.redhat.com/archives/fedora-test-list/2009-October/msg00112.html 17:01:58 uh oh ... did we loose Viking-Ice? :( 17:02:24 * Viking-Ice takes a sip of an hour (c)old coffee which only makes you stronger.. 17:02:31 eeew 17:02:32 :) 17:02:36 Viking-Ice: hey there! 17:02:42 Greetings 17:03:15 Viking-Ice: didn't mean to put you on the spot, but wanted to give you a chance to talk about where things are on that thread ... and what's needed to move forward 17:03:44 hang on a sec.. 17:04:12 so https://www.redhat.com/archives/fedora-test-list/2009-October/msg00112.html 17:04:49 The things are just stuck on minor details.. I think the general look and feel is ok 17:05:16 might I add a big kudos on the template 17:05:28 #link https://fedoraproject.org/wiki/Template:How_to_debug 17:05:46 more or less based on the Dracut page.. 17:05:54 again, which you started 17:05:58 the naming 17:06:01 I'm sensing a trend! 17:06:30 How to debug. or component X problems or bug info component is a bit holding it back.. 17:06:51 I can respond, but I liked the preference list you posted 17:07:43 #info Viking-Ice would like feedback on a naming convention for future pages. Candidates include: "How to debug ". or "component problems" or "bug info " 17:08:05 * gtirloni votes for how to debug 17:08:18 * jlaska seconds that vote 17:08:31 I share the same view as campbecg "As a reporter, looking at the wiki, I would see a page called "Bug Info ComponentX" and expect a page relating to known bugs on componentx, a page called "ComponentX Problems" to be full of known issues related to using componentx, and "How to Debug ComponentX" at a howto about how to actually debug it" 17:09:03 as he mentioned in thread https://www.redhat.com/archives/fedora-test-list/2009-October/msg00152.html 17:09:03 yeah, that's a good breakdown 17:09:36 and it seems like the content of these pages falls under "a howto about how to actually debug it"? 17:10:26 an components single report/debug page for a reporter 17:11:04 I don't follow, can you explain that last point? 17:11:37 this is not just a components debug guide 17:11:47 as you can see here https://fedoraproject.org/wiki/How_to_Debug_Thunderbird 17:12:32 so there sometimes might be content that would be better to share with other debug guides? 17:12:39 As I see it this is the how_to_debug_ pages should be a single pages refereed to reporter by maintain/triager to gather the needed info to file a proper report 17:13:42 Viking-Ice: we're missing some of the folks involved in that thread, including beland and adamw 17:14:06 but we can include what the next obstacle is for moving forward in the minutes and working that out of meeting 17:14:54 well the naming and the debugging are more less what's holding it back 17:15:31 can we have the template dictate the big sections and let each writer to work out the details for each package? 17:15:52 Feels like the naming of the pages is answered by campbecg's definitions 17:16:00 You need to more or less adjust it to each component 17:16:46 to gtirloni's point, I think the template you started offers a really good starting point that we can let subject matter experts expand on 17:16:50 hence having a template is no go other than being an skeleton to copy and paste 17:17:07 it's hard to restrict what the content of each page will look like without content for each page 17:17:17 the debugging output stops 17:17:23 ups 17:17:29 so maybe we choose a approach now ... and set a point to readjust/reassess? 17:17:32 the debugging can more or less be a template 17:18:39 okay, if no objections ... let's start wrapping things up for today 17:18:40 I would say we go with how_to_debug to move on with that objections ? 17:19:08 Viking-Ice: yeah I think that's a sensible plan based on the feedback to the list so far 17:19:17 and nothing speaks better than proven examples 17:19:22 indeed 17:19:36 I would want to have a bit more feed back from reporters on how they want to have it since they are the once that actually are going to be using those pages 17:19:47 and it's no big deal really ... if a few months from now it turns out we didn't make the _perfect_ choice ... we can always correct 17:20:08 Viking-Ice: perhaps picking the most frequently reported components from bugzappers? 17:20:13 it's probably going to require many iterations to get it right.. that's normal. I think we've a very good template in place 17:20:27 #info Viking-Ice would like more feedback from reporters on what Debug pages to see next 17:20:29 gtirloni: agreed 17:20:53 Viking-Ice: okay anything else you'd like to include in the minutes? 17:21:02 I was more less speaking about the how_to_debug pages not which component go tackle next 17:21:24 ah, so more feedbcak on the naming convention then? 17:21:26 I was going to start work on critical path components but I probably will start on virtualzation as markmc ask for 17:21:35 good point 17:22:09 #info Viking-Ice may focus attentiong on improving the existing Virtualization bug reporting guide 17:22:15 jlaska: not the naming more on the template itself as basis but as you pointed out we can redesign it later if necessary.. 17:22:39 Viking-Ice: I'd suggest going with what you've go there now (inherited from the dracut debug page) 17:22:59 and should the Virtualization work poke some holes in the template ... we can raise issues as needed there 17:23:54 okay ... let's open up the floor 17:24:01 #topic Open Discussion - 17:24:14 Viking-Ice: thanks for the status update on the debug thread! 17:24:26 alright, before we wrap up ... any other topics not yet discussed? 17:25:19 going once ... 17:25:38 :) 17:25:49 ... twice ... ;) 17:26:15 okay ... let's close it out for today 17:26:20 thanks folks for your time and updates 17:26:41 follow-up to #fedora-qa or fedora-test-list@redhat.com for any additional issues 17:26:44 #endmeeting