16:01:41 #startmeeting Fedora QA Meeting 16:01:42 Meeting started Mon Feb 1 16:01:41 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:44 #meetingname qa 16:01:44 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:01:46 The meeting name has been set to 'qa' 16:01:53 #topic Gathering 16:02:42 yo 16:02:57 adamw: happy monday :) 16:03:18 who else can we convince to join us for a meeting? 16:03:28 * kparal convinced 16:03:31 yeah, yeah, keep it short, whatever, i have important cellphone hacking to do :P 16:03:38 I can see that! :P 16:03:52 adamw: you must join the meeting using one of those phones 16:03:53 adamw: you are a blogging superman :) 16:04:04 * maxamillion is here 16:04:11 jlaska: gimme 2 minutes and i will 16:04:13 hi kparal, maxamillion 16:04:22 adamw: uh oh ... that wasn't meant as a dare :) 16:05:11 who else we have? wwoods Viking-Ice tk009 ? 16:06:13 alright, let's get started and hopefully they can join in 16:06:17 #topic previous meeting follow-up 16:06:34 #info jlaska to add bugzilla shared query links to Common_Bugs wiki page 16:07:02 #info beland added the links to the F-13 page already ... https://fedoraproject.org/wiki/Common_F13_bugs 16:08:15 The only other thing that occured to me was perhaps updating our IRC RSS feeds that zodbot tracks 16:08:48 * nirik can do that later today if you let me know what to change it to/etc. 16:08:51 I'll take an action item to grab the blocker bug RSS feeds and see if nirik can update zodbot to watch those links 16:09:09 nirik: thanks! 16:09:18 no problem. 16:09:30 #action request zodbot tracking for updated F-13 QA RSS feeds (blocker bugs, common_bugs? etc...) 16:09:36 * adamw now meetingggggg on his phone :) 16:09:40 I noticed the other day it spewed something about the f12 blockers... meant to look at it. 16:09:56 nirik: we can probably replace the F-13 ones with the new feeds? 16:10:06 adamw: you're nuts :) 16:10:12 okay, next up I have ... 16:10:13 Pic available if proof desired! 16:10:19 #info wwoods and kparal to talk about design ideas 16:10:20 f12 you mean? yeah. I think currently it's announcing any change to the f12 blocker 16:10:29 I think this came out of the AutoQA discussion last week 16:10:49 nirik: yeah, sorry ... replace the F-12 ones with the newer F-13 feeds 16:11:17 kparal: wwoods: do you have insight as to what was being tracked by this action item? 16:11:56 jlaska: I believe it meant we should have consulted the future results database design? 16:12:12 jlaska: which we didn't :/ 16:12:22 http://www.happyassassin.net/extras/irc_meeting.jpg 16:13:00 adamw: nice! 16:13:05 kparal: ah right ... hmm, what are your thoughts there. what is needed to do a good design review on that? 16:13:17 adamw: that's cute :) 16:13:54 adamw: what irc client is that? 16:13:57 adamw: normally, I'd say this would impact your words/minute. But I don't think it will :) 16:14:00 maxamillion: andchat 16:14:13 adamw: interesting ... I'll have to check it out 16:14:15 :) 16:14:17 jlaska: i'm cheating, i'm typing on my pc now :P 16:14:19 jlaska: yeah we were going to talk about design ideas for the results database 16:14:23 jlaska: I believe several should get involved, so we don't make any oversights right in the design. maybe some people that already designed some koji parts could help us 16:14:40 maxamillion: it vibrates for nick alerts - stop buzzing my knee :D 16:14:41 adamw: when did you get an android device? 16:14:45 LOL 16:14:48 *several people 16:14:51 maxamillion: > #fedora-qa let's not derail 16:14:57 rgr, sorry 16:16:21 kparal: re: koji ... do you mean like lmacken? 16:16:39 perhaps a micro-FAD would be helpful to get the right folks involved 16:17:26 jlaska: I don't know about lmacken, for e.g. dmach hinted me to look at his kobo library several times. https://fedorahosted.org/kobo/ 16:17:38 kparal: aah right 16:17:48 ugh 16:17:56 xmlrpc with cookies? krb5? 16:17:57 jlaska: it might be some inspiration for us, I think there is a hub part that store results 16:17:58 kparal: probably a good idea for us to outline what our needs are first? 16:18:02 kobo looked waaaay to overengineered for the stuff I needed 16:18:17 kobo is a great idea for building RHEL-internal apps 16:18:33 I haven't studied it yet 16:18:35 but very poorly suited to fedora needs 16:18:56 just looking for places where we can gather some ideas about the results DB 16:19:21 ugh 16:19:23 yeah it's still worth looking at other things that are around 16:19:26 jlaska: yes, we should brainstorm about the needs first 16:19:29 b/c we need ANOTHER implemention of rpm header parsing 16:19:47 but there's no reason the results db needs to be able to parse RPM headers 16:20:12 don't dump it everyone, we can have a look at some specific parts, not use everything at once :) 16:20:13 honestly there's no reason it needs to be anything more than, say, a TG app, as far as I can see 16:20:29 but that's still a very very broad canvas for design 16:20:40 in the interest of time ... is there a next step to keep track of here? 16:20:59 wwoods already has some expertiese with irb frontend, so I suppose he will be the leader in this area :) 16:21:17 I'd suggest we should outline some requirements and goals before we start looking at tools/implementations 16:21:27 that seems sensible 16:21:31 the next step should be the brainstorming meeting for those of us concerned I think 16:21:36 kparal: agreed 16:21:54 we just must not forget about it :) 16:22:12 would you like me to organize a gobby meeting this week to review? 16:22:34 sounds good for me 16:22:52 sure 16:22:57 #action jlaska to organize a gobby meeting to begin requirements/goal definition for a results db 16:23:07 alright ... last follow-up item ... 16:23:21 #info adamw update on security policy 16:24:02 The info I have is Adam moved the discussion from test@l.fp.org to devel@l.fp.org (http://lists.fedoraproject.org/pipermail/devel/2010-January/129978.html) 16:24:12 adamw: any other updates? 16:24:49 sorry! 16:25:00 nope, that's it - i did that before the weekend so haven't read the responses yet 16:25:02 adamw: no TXT'ing while MTG :) 16:25:28 i intend to follow the same process i did for test list 16:25:37 okay, should we keep this on the radar for next week? 16:25:44 do a new draft from the current responses, then send that, keep cycling until no-one has anything to complain about :) 16:25:45 sure 16:26:03 #info adamw will continue monitoring feedback (http://lists.fedoraproject.org/pipermail/devel/2010-January/129978.html) and propose updated drafts as needed 16:26:16 alright, let's dive in ... 16:26:21 #topic Release validation wiki updates 16:26:29 adamw: I've got you on the spot for this as well, can you cover for rhe? 16:26:49 Just wanted to give a quick update on the wiki changes you and rhe have been working on 16:27:05 yup 16:27:17 so far we've added the installation validation testing wiki page: 16:27:25 #link https://fedoraproject.org/wiki/QA:Installation_validation_testing 16:27:39 a desktop validation testing wiki page, which is still full of duct tape: 16:27:46 https://fedoraproject.org/wiki/QA:Desktop_validation_testing 16:27:54 and modified the QA/Join page to mention this testing: 16:28:09 heh, the many uses of duct tape :) 16:28:26 https://fedoraproject.org/wiki/QA/Join#Release_validation 16:29:06 adamw: I wanted to ask, would you like to have the QA schedule changed to reference 'release validation' and not specifically 'install validation' so there is room for other efforts? 16:31:24 okay, we can come back to that if needed, lemme keep on going 16:31:32 nice wiki updates rhe and adamw :) 16:31:34 jlaska: i think that would be a good change 16:31:52 adamw: okay, I'll ping get in touch with poelcat later this week then 16:32:15 #action jlaska to discuss updating QA schedule with poelstra to reflect new 'release validation' focus 16:32:18 there will be other types of validation we could do in future; there's already a couple of criteria we can test that don't really come under installation or desktop 16:32:40 I'd love to get off my butt and do the QA release checklist 16:32:45 to include in that mix 16:33:19 adamw: anything else on the wiki front? 16:33:44 um, nothing springs to mind 16:33:47 was that a leading question? :) 16:33:56 i didn't get around to updating commonbugs yet :( 16:34:06 no, not a leading question 16:34:32 alright, moving on to another quick update ... 16:34:37 #topic Rawhide Acceptance Testing 16:34:50 we had another RATS drop last Thursday 16:35:11 jkeating composed a tree for rats_install (see rell-eng ticket#3292). 16:35:40 I had to correct a few typos with the rats_install patches designed to support providing stage2= and repo= ... but all-in-all it worked well 16:35:51 #link https://fedoraproject.org/wiki/Test_Results:Fedora_13_Rawhide_Acceptance_Test_2 16:35:52 did it pass? 16:36:05 Oxf13: well it passed, but there's a _but_ 16:36:20 bug#559597 prevents initramfs creation ... so it won't boot without that fixed 16:36:47 oh right 16:36:50 so one dracut-004-5 lands, we should be good 16:36:56 but you can use the images with a later rawhide tree 16:37:00 at least for the automated portion of rats_install 16:37:08 Oxf13: right, the packages from a newer rawhide 16:37:17 I don't think we need to update the install images for this, right? 16:38:01 Oxf13: so how would you like to proceed ... want a ticket to update the 'last known good' ? 16:38:25 jlaska: we probably shouldn't do that until the package tree has the right dracut level and we verify that 16:38:33 Oxf13: oh agreed 16:38:37 this does bring up a good question though 16:38:54 does "last known good" mean a set of images, a set of packages, or the combination of the two? 16:39:13 #info Oxf13 asked ... does "last known good" mean a set of images, a set of packages, or the combination of the two? 16:39:28 from a testing perspective ... it means images + packages 16:39:47 as soon as one changes, I think that adds a caveat to the 'last known good' 16:39:56 I suppose for users it's useful to have images which we expect to work, which can be tested against package sets which we're not sure of 16:40:18 all releng would be tracking and providing symlinks to is the set of images 16:40:50 Oxf13: true, so this is intended to increase install reliability ... but is still open to rawhide package churn 16:40:56 i think it's as simple as putting a rawhide date on the 'last known good' listing 16:41:04 and a note that says it was known to work with that day's packages 16:41:12 may work with other days, but we can't say for sure 16:41:31 agreed, I think we just need some wording around this so it's clear 16:42:09 * poelcat thought LKG was a fully installable source, including images so people did not have to go all the way back to F12 for a reliable starting place 16:42:24 think it's time to get some documentation down for this 16:42:30 any volunteers :) 16:43:03 i can do it if you let me know roughly what's needed 16:43:31 adamw: given the questions here, I think defining what 'last known good' means 16:44:27 it's a term we use a lot, but sounds like there is still some confusion around what's included 16:45:09 part of that was just working through the mechanics of actually doing it 16:45:15 true true 16:45:25 it's far easier to store a small set of images, and then reference an existing tree of packages somewhere 16:45:36 or let the images autopick the latest rawhide for more fun 16:45:39 i think this is a case where we should do the mechanics and then i will write an explanation which sounds like we knew what we were doing all along =) 16:46:08 okay, in that case I think we're probably done then 16:46:19 Oxf13: are there other changes coming on the rel-eng side of building these? 16:46:35 um... 16:46:41 how do you mean? 16:47:12 how do we know when we're done with providing 'last known good'? 16:47:18 how can we [X] check it off 16:47:21 * adamw is the king of retrospective-justification-through-wikis 16:47:35 adamw: it's still 20/20 right? :) 16:47:56 jlaska: in my head there is a simple webpage that is maintained by QA 16:48:13 this webpage describes Last Known Good and the fact that rawhide no longer has images of it's own 16:48:30 onthis page would be links to the install image files, and to the tree of packages it was found to be good with 16:48:38 with instructions on how to combine the two into an install 16:48:58 if you give me canonical locations for lng installer images and package tree i can totally make a wiki page for that. 16:49:08 also, those instructions :D 16:49:09 Oxf13: ah that's helpful 16:49:09 and finally notes that without picking a specific repo of packages, the images will attempt to install the latest repo of packages on the mirror system 16:49:19 which has not been validated, and may not work 16:49:47 alright, adamw I'll be happy to work the details with you 16:50:06 okay 16:50:07 action it? 16:50:07 #info jlaska and adamw to work on drafting a 'last known good' wiki page 16:50:13 err ... 16:50:17 #action jlaska and adamw to work on drafting a 'last known good' wiki page 16:50:43 Oxf13: thanks ... will probably pick your brain again during the week 16:50:56 no problem 16:50:59 okay ... time for AutoQA update! 16:51:02 * jlaska hits the gong 16:51:05 #topic AutoQA project update 16:51:13 #topic AutoQA project update - deps/conflicts prevention 16:51:30 wwoods: you want to kick things off for the AutoQA update? 16:51:54 sure 16:52:08 Friday afternoon I got a working yum-based prototype of the depcheck code 16:52:30 http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/depcheck/depcheck 16:52:35 saweet! 16:52:48 a mere 147 lines, even 16:53:15 I need to start creating test cases to make sure it works as expected 16:53:24 typical runtime is ~20s 16:53:32 wow, you really brought that down from the 60s 16:53:51 ~15s for leaf-node packages (e.g. something like hamster-applet) 16:54:07 ~19-20s for packages with lots of prov/req data (e.g. gcc) 16:54:16 that's pretty darned good 16:54:18 but "20s" is a safe estimate 16:54:35 probably it can be sped up further 16:54:43 but premature optimization is the root of all evil, and all that, so.. later 16:54:50 heh 16:55:00 * jlaska stuffs a few #info tags into the mix 16:55:08 #info Friday afternoon I got a working yum-based prototype of the depcheck code 16:55:13 #link http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/depcheck/depcheck 16:55:56 not that this is blocking your efforts, but I'm waiting for some feedback from jhutar, but rpmfluff should be an official fedora package soon 16:56:03 excellent 16:56:20 yeah I think I'd like to use rpmfluff etc. to generate test packages/repos 16:56:23 for the test cases 16:56:36 that'll be great to see 16:56:37 yum's test cases involve creating objects which *simulate* test packages/repos/etc. 16:57:17 not sure if that makes a significant difference 16:57:37 just having a testsuite at all for this I think will pay off in the long run 16:57:42 definitely. 16:57:43 given the complexity 16:57:50 nice work! 16:58:31 wwoods: anything else on the depcheck front? 16:58:40 no, I think that covers it 16:58:48 sweet, alright ... kparal's up next 16:59:01 ok 16:59:02 #topic AutoQA project update - rpmguard and resultsdb 16:59:18 so I have updated a few tickets regarding rpmguard 16:59:22 first is https://fedorahosted.org/autoqa/ticket/113 16:59:24 take it away kparal 16:59:54 when comparing the exactly same packages the rpmguard will now just skip the comparison and print a notice that they are the same 17:00:18 that could save a little computing power 17:00:57 aha, nice ... that test run confused the heck out of me :) 17:00:58 it was connected to our problem of comparing the same package unintentionally 17:01:34 and second ticket connected to that is https://fedorahosted.org/autoqa/ticket/114 17:01:59 before we compared the current package with the latest package from stable, updates or updates-testing 17:02:18 now we compare only to stable and updates, as per discussion on autoqa-devel 17:02:36 nice to see all this stuff getting hashed out 17:02:44 and also we compare to the latest *previous* package (so it should not happen to compare two equal packages) 17:03:35 this also allows us to run the test any time in the future, even against older packages (in that time already in stable or updates, not in updates-candidate) and receive the same results 17:03:51 oh so if I ran that test again, it should be grabbing ... yup, you just answered it 17:03:58 this can be beneficial when we find some problem in autoqa/rpmguard and need to re-run the test suite again 17:04:04 #info I have updated a few tickets regarding rpmguard 17:04:11 #link ttps://fedorahosted.org/autoqa/ticket/113 17:04:12 #link ttps://fedorahosted.org/autoqa/ticket/114 17:04:19 #link https://fedorahosted.org/autoqa/ticket/113 17:04:19 #link https://fedorahosted.org/autoqa/ticket/114 17:04:21 side note on rpmguard: we should talk about the privesc stuff at some point, since that looks like it may turn into a Real Thing 17:04:36 * jlaska gets flagged for incorrect use of meetbot 17:04:54 adamw: privesc? 17:05:39 #info adamw discussed possible rpmguard and privilege escalation coordination 17:05:45 ah 17:05:48 kparal: that's the security policy Adam'w been working 17:05:58 just didn't know the abbreviation :) 17:06:03 jlaska: fyi, meetbot picks up links automatically, you don't have to #link them 17:06:10 Oxf13: ah, thx 17:06:35 ok, that's all from me 17:07:02 kparal: also the results db discussion earlier, hopefully I can help get some of you and wwoods ideas on "paper" this week 17:07:26 kparal: thanks for the updates! 17:07:31 e-paper = wiki :) 17:07:36 you got it 17:07:49 #topic AutoQA project update - install automation 17:08:23 #info Liam added a dvd_install.py script to the autoqa git repo last week 17:08:27 https://fedorahosted.org/pipermail/autoqa-devel/2010-January/000178.html 17:08:47 it raises some interesting questions for me on how we'll approach automating the install matrix 17:09:02 wwoods: if you have some time this week, I could use your input on that thread as well 17:09:22 jlaska: yeah I've been meaning to give that a closer look 17:09:43 ah thank you 17:10:08 you know the usual ... trying to figure out what path is the most sustainable and scalable for building new install tests :) 17:11:15 Liam has some more thoughts on the mailing list, I'll try to reply later today 17:11:25 so that's great news, hopefully more install tests to come soon 17:11:34 alright ... last up ... 17:11:39 #topic AutoQA projet update - packaging 17:11:55 #info Process akurtakov's feedback and remove all uncertain JAR files from the list 17:12:17 I've got the list down to as small as I can get it ... adamw suggested reaching out to GWT upstream for some additional feedback 17:12:31 so the plan this week is to Seek guidance from upstream GWT for ... 17:12:42 * Cases where multiple versions of a JAR are bundled 17:12:59 * Need for remaining bundled JARs? (see https://fedoraproject.org/wiki/User:Jlaska/gwt#Status_uncertain) 17:13:20 and ideally, I'd like to have one of the listed JPackage dependencies already started through the Fedora packaging process 17:13:31 https://fedoraproject.org/wiki/User:Jlaska/gwt#JPackage_Dependencies 17:14:04 aside from that ... I've been going through the gwt.spec and removing JAR files and linking them to their equivalents (and adding BuildRequires) 17:14:13 alright ... open discussion time 17:14:20 #topic Open discussion - 17:14:38 I'll close out the meeting in 2 minutes unless any topics come up 17:14:47 ok, one topic from me 17:14:54 kparal: sure, what's up? 17:14:56 package update acceptance test plan 17:15:08 I just have a quick call for links :) 17:15:13 oh oh, right on 17:15:26 #topic Open discussion - Package Update Acceptance test plan 17:15:31 #info kparal put out a call for links 17:15:40 kparal: what sort of links are you looking for? 17:15:45 I'm currently working on package update acceptance test plan, that means what should be our approach to decide if some package update is good or not 17:16:06 I am currently looking into other distributions how they do it and trying to gather some inspiration 17:16:38 if somebody is aware of some documented policies and have links for them, I will be very glad if you send them to me 17:17:04 it is quite possible that I don't find everything that is available 17:17:12 kparal: I wonder if the MUST and SHOULD items could provide some insight - http://fedoraproject.org/wiki/Packaging:ReviewGuidelines 17:17:44 yes, currently I have found mainly requirements for the initial review process 17:18:00 but I haven't found many documents concerning package updates requirements 17:18:20 i'm sure debian must have a policy 17:18:22 perhaps what wwoods is working on might be included in the mix? the depscheck stuff 17:18:27 it is possible that other distributions also don't have strict policies about that 17:18:46 but - if you know about such a policy and have a link, please send it to me. thanks :) 17:19:04 that was all from me, just a call for links 17:19:04 kparal: http://wiki.mandriva.com/en/Policies/SoftwareMedia 17:19:12 kparal: i wrote that for MDV back when I was there 17:19:14 Oxf13: you have any handy links? 17:19:23 kparal: see the definitions for the 'updates' repositories 17:19:34 adamw: thanks, will look at that 17:19:52 kparal: also see http://wiki.mandriva.com/en/Policies/Support , the full policy for maintainers about updates 17:20:12 kparal: that was written by vincent danen when he was at mandriva, he also now works at red hat so you could always poke him for thoughts :) look for vdanen 17:20:28 adamw: great, thx 17:20:31 jlaska: I don't 17:21:40 #topic Open discussion - 17:21:52 anything else to cover, if not let's close it out 17:22:15 nothing frm me 17:22:46 alright gang, thanks for your time! 17:22:54 I'll follow-up to the list with minutes etc... 17:22:56 #endmeeting