16:02:44 <jlaska> #startmeeting
16:03:11 <jlaska> #topic Gathering bodies
16:03:17 <jlaska> #topic Fedora QA - Gathering bodies
16:04:28 <jlaska> #meetingtopic Fedora QA Meeting
16:04:34 <jlaska> #topic Gathering bodies
16:04:51 <adamw> bueller?
16:04:52 <jlaska> sorry for the noise ... just getting a little too familiar with fedbot
16:05:05 <jlaska> who else is around for a QA gathering?
16:05:19 <jlaska> adamw: I heard he passed out at 31 flavors last night?
16:05:29 <adamw> that sounds improbable
16:05:53 * wwoods liiiiives
16:06:02 <jlaska> hey there
16:06:39 <jlaska> anyone else from the usual suspects? (Viking-Ice or f13)
16:06:53 <jlaska> I believe poelcat is at a conference today
16:07:36 <jlaska> well, should be quick then with the 3 of us :)
16:07:59 <jlaska> #topic Preview meeting follow-up - https://fedoraproject.org/wiki/QA/Meetings/20090610#Action_items
16:08:17 <jlaska> adamw: sorry for the late notice last week, I took down notes and the irc log from the meeting
16:08:30 <jlaska> we only have a few small follow-up items on the list around release notes
16:08:43 <jlaska> you guys may have already followed up on this ...
16:08:45 <jlaska> #  [stickster] - who will be handling release_notes bugs to help with RHBZ #499585
16:08:48 <jlaska> #  [adamw] - propose draft wording of minimal requirements for the release_notes team to digest
16:09:04 <adamw> i did the minimal requirements stuff on the bug
16:09:11 <adamw> https://bugzilla.redhat.com/show_bug.cgi?id=499585
16:09:12 <buggbot> Bug 499585: medium, low, ---, relnotes, ASSIGNED, clarify minimum hardware requirements
16:09:38 <f13> hi I'm here
16:09:41 <f13> was a little late finishing my bike ride this morning
16:09:44 <jlaska> f13: welcome
16:10:20 <jlaska> adamw: so that one is now in the fedora-release-notes court then?
16:10:44 <adamw> yeppers
16:10:55 <adamw> of course, now i'm on the docs team i could play both ends of the rally =)
16:11:05 <adamw> i might bring it up through those channels too somehow.
16:11:36 <jlaska> you're just a glutton for pain huh?
16:11:52 <adamw> heh
16:12:09 <jlaska> that's all I had for follow-up ... unless anything else I missed, we can move along
16:13:07 <jlaska> #topic F-11 Retrospective feedback
16:13:33 <jlaska> so, poelcat hosted a F-11 Retrospective (aka post-mortem) yesterday
16:13:58 <jlaska> which was an cool mix of communication channels with gobby and phone
16:14:22 <jlaska> I just wanted to briefly chat about experiences from that meeting, or the preparation we did leading up to it
16:14:26 <adamw> i feel like i came from the future
16:14:45 <f13> the meeting was good and good
16:14:52 <jlaska> I tried to gather the QA talking points for that meeting ahead of time ... https://fedoraproject.org/wiki/QA/Meetings/20090603#F-11_QA_Post-mortem_discussion
16:15:05 <f13> good in that many people recognized the efforts QA and bugzappers were putting into the release
16:15:26 <f13> good also in that many people recognized that more of that efforts and better tooling around those efforts will again have positive effect
16:15:28 <jlaska> did folks think that worked well, needed more/less prep from QA.  Were your concerns appropriately raised
16:15:47 <f13> I was a slacker and "prepared" while other people were talking
16:16:15 <jlaska> I wasn't 100% sure of the format of that meeting, so I wanted to tmake sure we had QA discussion points ahead of time
16:16:21 <adamw> i think it worked well
16:16:29 <adamw> except that i had nothing to say and had to ramble like an idiot :)
16:16:36 <jlaska> adamw: that was my job
16:16:37 <f13> heh
16:16:44 <adamw> but you sounded all smart and on top of things
16:16:47 <jlaska> anything we'd want to change for next time ... in terms of recording pain points so we don't forget them ?
16:16:54 <f13> just like with the FAD, the proof will be in the pudding.  If after a months time, nothing came of the talk, then I don't think it worked well
16:17:17 <f13> however if after a month's time, we have some progress on what people thing went poorly (or more effort in what people thing went well) then I think it'll have been a success
16:17:18 <jlaska> f13: great point!
16:17:21 <adamw> true
16:17:34 <jlaska> so ... lemme take an action item I've been delaying for a long while
16:18:09 <jlaska> I'd like to propose updates to our QA/Goals wiki ... which I hope will show how we are incorporating FAD and F11 lessons learned into our planning
16:18:26 <jlaska> objections/alternatives/favorite_jokes ?
16:18:37 <f13> jlaska: I look forward to seeing the updates?
16:18:55 <adamw> I also!
16:19:12 <adamw> work being done by other people is always a good thing
16:19:17 <jlaska> heh, okay I'll take it for next week ... /me fiddles
16:19:32 <jlaska> #action update QA/Goals wiki
16:20:05 <jlaska> okaly dokaly ... anything else on the retrospective?
16:20:06 * wwoods plays banjo
16:20:16 <jlaska> do folks think we should be using gobby or phone for QA meetings?
16:20:23 * adamw chews grass stalk
16:20:35 <adamw> nah, I think IRC is fine.
16:20:53 <wwoods> the output isn't heavy enough to require collaborative editing of full-fledged docs
16:21:00 <jlaska> true true
16:21:08 <adamw> and if we decided to go all 21st century we should use fedora talk. and if it has these muting issues or whatever - fix 'em! eat our own dog food and all that.
16:21:25 <jlaska> would you guys want that for this meeting?
16:21:29 <jlaska> re: fedora talk?
16:21:32 <wwoods> IRC seems more accessible than Fedora Talk
16:21:33 <adamw> honestly, i don't think so
16:21:36 <jlaska> okay
16:21:37 <adamw> it's a bit more of a speedbump
16:21:54 <jlaska> a'ight
16:21:58 <adamw> don your headset (or get a headset), start ekiga (most people aren't running it 24/7), dial in...meh
16:21:59 * jlaska flashes gang sign
16:22:06 <adamw> and, of course, it would be much harder to log
16:22:10 <wwoods> yup
16:22:14 <jlaska> good points
16:22:35 <jlaska> cool, just want to make sure we're hosting these in the most effecient/productive means
16:22:41 <adamw> we could use Opera Unite ;)
16:22:44 <wwoods> I don't see any benefit other than "it's coooool"
16:22:47 <wwoods> I don't even have a headset
16:22:58 <jlaska> wwoods: I could make a "confidential" headset :)
16:23:06 <jlaska> adamw: I don't even want to google on that subject
16:23:19 <jlaska> ... if it's anything like her favorite movie pick from 2 years ago
16:23:32 <adamw> jlaska: you didn't see it? it's Opera's new system to own all your data. hey, makes a change from Google!
16:23:33 <wwoods> Opera, not Oprah
16:23:41 <f13> adamw: phone calls are prohibitively difficult for folks who can't hear (or speak)
16:23:43 <jlaska> wwoods: doh! :D
16:23:56 <adamw> f13: indeed, accessibility issues too
16:24:06 <jlaska> #topic QA Review of related FAD topics
16:24:07 <f13> I think voice meetings should be reserved for a specific task where we're sure we're not excluding anybody
16:24:22 <jlaska> f13: true, like FAD's
16:24:27 <adamw> although i suppose there may be those for whom voice is easier than a typed meeting...blind / er...missing hands?
16:24:38 * nirik thinks zonkers points are pretty valid: http://www.dissociatedpress.net/2009/06/03/pros-and-cons-of-irc-meetings/
16:24:40 * ricky agrees with sticking to IRC meetings when possible, for whatever little or nothing that's worth.
16:24:56 <f13> adamw: yeah, there are tradeoffs.
16:24:57 <jlaska> nirik: thanks for the link
16:25:22 <f13> adamw: pure email would probably be the least exclusive as people can use technology to compose emails at their own pace, but that draws things out way too long
16:25:26 <f13> (in my opinion of course)
16:25:51 <adamw> so, anyway, i think we're pretty unanimous for sticking with irc.
16:25:56 <jlaska> f13: it does, but we also follow up to the list with minutes of hte meetings for that crowd I guess
16:26:03 <jlaska> yeah ... okay next topic
16:26:04 <f13> nod
16:26:40 <jlaska> Chatting with wwoods yesterday who wanted to add a reminder to the agenda that we are nearing the end of open-feedback for the FAD proposals that have impact on QA
16:27:06 <jlaska> meaning, let's get input/concerns in on these proposals since the large wheels of progress are starting to rotate
16:27:13 * jlaska finds proposal linkies
16:27:52 <jlaska> the proposals are outlined in Jesse's update to fedora-test-list ... https://www.redhat.com/archives/fedora-test-list/2009-June/msg00385.html
16:28:10 <f13> something failed in the wiki watch and there is a lot of feedback I missed, so I need to review the wiki pages today
16:28:56 <jlaska> https://fedoraproject.org/wiki/Milestone_Adjustment_Proposal - Bill
16:28:56 <jlaska> Nottingham
16:28:56 <jlaska> https://fedoraproject.org/wiki/Israwhidebroken.com_Proposal - Will Woods
16:28:56 <jlaska> and James Laska
16:28:56 <jlaska> https://fedoraproject.org/wiki/Koji_Build_Autosign_Proposal - Jesse
16:28:58 <jlaska> Keating
16:29:00 <jlaska> https://fedoraproject.org/wiki/Critical_Path_Packages_Proposal - Seth
16:29:03 <jlaska> Vidal
16:29:05 <jlaska> https://fedoraproject.org/wiki/No_Frozen_Rawhide_Proposal - Jesse
16:29:08 <jlaska> Keating
16:29:10 <jlaska> 
16:29:13 <jlaska> doh!
16:29:19 * jlaska made happy fun ball angry
16:29:22 <f13> hi
16:29:46 <jlaska> f13: how long to folks have to get feedback in on these proposals?
16:29:48 * nirik sighs. Thought I fixed that. Sorry about it.
16:30:05 <jlaska> nirik: no worries, poor irc form on my part
16:30:06 <f13> jlaska: I figured at least until after I get back from Germany
16:30:17 <jlaska> that's about 2 weeks right?
16:30:20 <f13> I think I'll give a recap of the FAD at FUDCon Berlin
16:30:33 <f13> which may prompt some feedback
16:30:38 <jlaska> oh good, FUDCon Berlin should pull in some additional focus on those topics
16:30:44 <f13> jlaska: i get back in a little over a week
16:31:00 <f13> I think we told some other people that it was open at least for 2 weeks as of yesterday
16:31:13 <f13> of course, not all proposals are ready for voting, many need a lot more work
16:31:14 <jlaska> okay ... so until the end of the month (June 30)
16:31:29 <f13> sounds right
16:31:46 <f13> realistically anything we do that we want to target F12 on has to be done before the Feature Freeze of Alpha
16:31:55 <f13> which is late July I do believe
16:32:17 <jlaska> okay, cool ... wwoods were there any other issues you wanted to raise on this topic?
16:33:00 <wwoods> nope - I'd hope by now that all interested parties have had time to read the proposals
16:33:15 <wwoods> and any comments or concerns have been made or addressed
16:33:52 <jlaska> are there any specific groups that haven't responded that we'd really like feedback from?
16:34:06 <f13> what might spurn more discussion is a first blush look at the critical path, and send that out to fedora-devel-list as well as all the maintainers of packages within the critical path
16:34:15 <f13> that might poke a few people into realizing that "we mean them"
16:34:30 <jlaska> heh, yeah I suspect that will be a healthy thread :
16:34:30 <wwoods> yeah, if we want to feel like everyone's had their chance to vent
16:34:32 <jlaska> )
16:34:36 <wwoods> and suggest their favorite color for the bikeshed
16:34:48 <wwoods> then we should probably send very simple summaries of the proposed changes to f-d-l and/or f-t-l
16:34:58 <wwoods> just to make sure everyone knows exactly what we're really going to change
16:35:13 <wwoods> and that we're seriously going to move along with this stuff ASAP
16:35:57 <wwoods> but, honestly, no matter how hard we try there's still going to be people who are suprised and angry when we make the changes
16:36:05 <jlaska> yeah true true
16:36:22 <wwoods> we may already be past the point where more effort buys us more support
16:36:37 <wwoods> or at least more constructive discussion
16:36:38 <jlaska> this bleeds into the next topic a bit, but I think we'll have a barrage of discussion around israwhidebroken
16:37:06 <f13> my suggestion on the critical path issue is that you pitch it more toward "QA is signing up to help with these packages, isn't that great?" as opposed to "suckers, you own special packages and now you're our BITCH!"
16:37:23 <wwoods> I feel that we should just take it as read that everyone's had their chance to comment
16:37:29 <wwoods> and start implementing the proposed changes
16:37:34 <jlaska> yeah
16:37:56 <f13> irb.com is pretty self contained, and doesn't really require anything from anybody else.
16:38:01 <f13> so yeah, take the comments there, and run with it
16:38:29 * jlaska still needs to come up to speed with how QA will engage with critical path
16:38:43 <wwoods> it requires some stuff from infrastructure, but nothing crazy
16:39:16 <jlaska> okay, if nothing else ... changing gears to a related topic
16:39:32 <jlaska> #topic AutoQA update
16:39:44 <jlaska> wwoods: you've got the floor
16:40:40 <wwoods> well, basically
16:41:21 <wwoods> israwhidebroken.com (hereinafter irb.com) can be thought of as a simple proof-of-concept of an AutoQA system
16:41:55 <wwoods> the goals for irb.com are basically a simple set of tests running automatically in response to a trigger (new rawhide being built)
16:42:00 <wwoods> with public results reported
16:42:03 <wwoods> that's autoqa!
16:42:27 <wwoods> so I've established irb.com as the first milestone for the autoqa project
16:42:34 <wwoods> and filed a buncha tickets that outline the plan
16:42:47 <wwoods> see https://fedorahosted.org/autoqa/milestone/israwhidebroken.com
16:43:17 <jlaska> sweet, I like the component %'s trac displays on that link ... didn't know it did that
16:43:32 <wwoods> the first step is to outline the test plan for how we check to see if rawhide is broken
16:43:35 <wwoods> and enumerate the test cases in that plan
16:44:17 <wwoods> we've settled on trying to use autotest as the test harness / API for writing automated tests
16:44:30 <wwoods> that's http://autotest.kernel.org/
16:44:54 <wwoods> it'll need to be packaged to be acceptable for use in Fedora, which is ticket #9, which f13 is working on
16:44:54 <jlaska> wwoods: I forgot to tell you, I made an official mediawiki template for test plans a while back ... that you can use if you like (https://fedoraproject.org/wiki/Template:QA/Test_Plan).  I'm going to tweak it so that many sections are optional, but shout if you want format changes too
16:45:13 <jlaska> it's a bit heavy I know ... but it's a start
16:45:21 <f13> wwoods: that particular item is going to take some effort
16:45:25 <wwoods> jlaska: that's cool, and yes, I used that template (using the {{subst:TEMPLATE_NAME}} trick stickster showed us) for the first draft
16:45:30 <adamw> sorry if i'm quiet. i am working out MIME types. MIME types are fun! wait, that is not true.
16:45:37 <f13> wwoods: the big thing is that it requires google web toolkit at build time to turn some .java files in to java script
16:45:39 <wwoods> https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
16:45:52 <f13> and gwt isn't in Fedora yet, it's a big pile of java stuff, so loads of fun waiting for me there.
16:46:01 <wwoods> f13: right, I figured there'd be some thorny bits in there.
16:46:19 <wwoods> as with testopia, we're going to assume that it's probably all OK
16:46:22 * mchua reads backlogs, listens with interest
16:46:29 <wwoods> and move forward with a test instance
16:46:29 <jlaska> mchua: welcome!
16:46:38 <wwoods> and work on packaging in parallel
16:46:50 <wwoods> and if it turns out there's giant evil lurking legal monsters
16:47:00 <wwoods> we'll have to drop it on the floor and find something else
16:47:00 <wwoods> again.
16:47:15 * jlaska loads bfg9000 in preparation for any monsters
16:47:28 <wwoods> I think we feel much more confident that this codebase is going to turn out to be OK in the end, though
16:47:41 <f13> yeah
16:47:49 <f13> I've audited autotest itself and it's pretty clean
16:47:58 <f13> the only gotchas are bundled deps which we can patch around
16:48:11 <jlaska> f13: I know you and lmr have been going gang busters on working through packaging issues ... are you comfortable continuing down that route?
16:48:12 <f13> it's just GPLv2 and one chunk of BSD code
16:48:20 <f13> jlaska: yep, it's fine by me
16:48:32 <f13> it's enough "release engineer" for me and my manager to be happy (:
16:48:33 <wwoods> so autotest brings us a test harness and a results reporting web frontend
16:48:42 <jlaska> sweet, because when I looked at the code I knew it was going to need someone with a packagers swiss army knife
16:48:47 <wwoods> which is two things that beaker wasn't planning to provide anyway
16:48:54 <wwoods> so there's no duplication of effort AFAIK
16:49:21 <wwoods> they are complementary efforts
16:49:47 <wwoods> and we can figure out how to bind them together later on
16:49:51 <f13> jlaska: $ wc -l autotest.spec
16:49:51 <f13> 190 autotest.spec
16:49:54 <f13> it's a bit beastly right now
16:50:02 <wwoods> but anyway, the basic outline of The Roadmap:
16:50:15 <f13> $ grep __install autotest.spec |wc -l
16:50:15 <f13> 67
16:50:22 <wwoods> write test plan for "how to check to see if rawhide is broken." write individual test cases for each requirement in the plan.
16:50:29 <f13> 67 calls to __install so far.  (there is no "make install" let alone "make" in the upstream)
16:50:30 <wwoods> automate test cases.
16:50:42 <wwoods> get automated cases running automatically when rawhide finishes building.
16:50:55 <wwoods> generate simple HTML summary of test results.
16:51:15 <wwoods> put that summary on israwhidebroken.com.
16:51:39 <wwoods> toast with champagne and retire to the bahamas.
16:51:49 * jlaska looking forward to final step
16:52:14 <adamw> yeah, i'll definitely join in for that bit
16:52:21 <wwoods> the tickets include some of the other details involved.
16:52:22 <adamw> i may be inexplicably busy during the earlier parts though
16:52:49 <wwoods> I may want help writing/automating test cases. I expect there to be 10 or so.
16:53:04 <jlaska> I'd like to help with that
16:53:13 <wwoods> we'll obviously be documenting how to set up the test system, how to write tests, &c
16:53:15 <wwoods> along the way
16:53:21 <f13> wwoods: how far out do you see this effort?  Would it be worth scheduling a FAD to get a few of us a day or two code sprint?
16:53:41 <f13> http://en.wikipedia.org/wiki/Sprint_%28software_development%29
16:54:12 <wwoods> the timeline we're kicking around would have the steps I outlined above finished by F12 Alpha
16:54:28 <wwoods> or thereabouts
16:54:42 <wwoods> which is convenient since that's where people will start jumping on rawhide more
16:55:01 <wwoods> and will be wanting this info
16:55:29 <jlaska> I feel like you've got some flex time in the plan in case there are speed bumps
16:55:32 <wwoods> I'd like to have the test plan and cases all finished by the next meeting
16:55:37 <wwoods> (June 24)
16:56:25 <wwoods> my notes indicate that we also wanted to have all the cases *automated* by then
16:56:30 <f13> woof
16:56:31 <wwoods> but that sounds like a typo
16:56:52 <jlaska> wwoods: I think that was by 7/24?
16:56:57 <wwoods> oh 7/24!
16:57:03 <wwoods> yeah 6/24 is next week
16:57:11 <jlaska> that could have been my typo
16:57:18 <wwoods> Hi! I'm Will, and I can't keep June and July straight
16:57:24 <jlaska> LOL
16:57:55 <wwoods> So yes. Tentative timeline:
16:57:57 <wwoods> test plan/cases written by next meeting (June 24)
16:58:10 <wwoods> test instance of autotest in Fedora Infrastructure by July 1
16:58:41 <wwoods> also July 1: autoqa (i.e. our tests and watcher scripts) packaged for deployment on infrastructure-run systems
16:58:55 <wwoods> all test cases automated by July 24
16:59:22 <wwoods> initial report generation by Aug 1
16:59:36 <wwoods> er, Aug 4
16:59:41 <f13> FYI, Kyle (owner of that domain) wasn't aware of the proposal.  I've just pointed him to it
17:00:05 <wwoods> (IIRC that's Alpha freeze)
17:00:08 <f13> wwoods: july1 is going to be a bit tough to meet, given my trip to Germany
17:00:40 <wwoods> well, it's a tentative schedule
17:00:55 <f13> ok
17:00:59 <f13> I'll do my best
17:01:04 <wwoods> the only target I'd really be sad to miss is having reports happening daily by Aug 18 (F12a release)
17:01:05 <jlaska> I don't know if we've got owners for each of these pieces ... so that's a good point
17:01:10 <f13> but I basically have 3 days left to get it done
17:01:18 <jlaska> if there are things that only you can do we may need to adjust as needed
17:01:23 <f13> well 4 if you count the 30th
17:01:44 <biertie> can we (the fedora-be guy's) have the channel now? =)
17:01:55 <adamw> yikes, someone took our overflow slot?
17:01:56 <adamw> we're doomed =)
17:02:23 <wwoods> f13: thanks for notifying kyle about that - it's not *required* that we use that domain, but hey, if we have it..
17:02:35 <jlaska> biertie: sorry for overflow ... we'll wrap up here
17:02:50 <biertie> jlaska: np ^^
17:02:54 <wwoods> sorry for running long on this topic
17:03:00 <wwoods> but I think I've covered the plan pretty well
17:03:12 <biertie> we have an overflow slot, so if we start 10minutes later, we just end 10minutes later too :D
17:03:20 <wwoods> bits of scheduling may need revisiting
17:04:09 <wwoods> but other than that I think we're good to go on the plan.
17:04:12 <f13> We can retire over to #fedora-qa
17:04:23 <jlaska> wwoods: nice job on the plan
17:04:48 <jlaska> #action update on irb.com acceptance test plan
17:05:04 <jlaska> #topic Open Discussion
17:05:25 <jlaska> I had one topic I wanted to mention, but we are short on time
17:06:07 <jlaska> had a good discussion with mchua yesterday about mediawiki+semantic.  Don't have anything now, but hopefully we have some more news next week
17:06:16 <jlaska> any other notes before we wrap things up?
17:07:28 <jlaska> okay, closing things out for today
17:07:32 <jlaska> thanks everyone!
17:07:43 <jlaska> I'll follow-up to the list with minutes
17:07:54 <wwoods> jlaska: woo! thanks!
17:08:09 <jlaska> #endmeeting