16:00:18 <jlaska> #startmeeting Fedora QA Meeting
16:00:19 <zodbot> Meeting started Mon Feb 22 16:00:18 2010 UTC.  The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:20 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:21 <jlaska> adamw: you betcha
16:00:27 <jlaska> #meetingname fedora-qa
16:00:27 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:34 <jlaska> #topic Gathering in the lobby
16:00:36 * Oxf13 
16:00:40 * jskladan is here
16:00:49 <jlaska> Oxf13: jskladan: welcome
16:00:54 * kparal joins
16:01:00 <adamw> ello
16:01:03 * jlaska tips hat to adamw & kparal
16:01:11 * jlaska needs a new formal greeting method
16:01:33 <adamw> wave of the cane?
16:01:46 <adamw> the natural, jauntier counterpart to the hat tip
16:01:53 <jlaska> adamw: ooh, good choice
16:02:02 * wwoods appears
16:02:19 <jlaska> I believe we also get 100% of maxamillion today
16:02:26 <jlaska> wwoods: heyo!
16:02:58 <jlaska> I have a hard stop after 30 minutes today, so if we run longer I could use someone to continue chairing
16:03:16 <jlaska> I kept the agenda light today ... but suggestions are always encouraged
16:03:22 <jlaska> #topic Previous meeting follow-up
16:03:26 <jlaska> This should be easy ...
16:03:36 <jlaska> we had nothing listed from last weeks meeting
16:03:54 <jlaska> anything not listed that people would like to update the team on?
16:04:07 <adamw> i'm sure i did lots of important things last week
16:04:10 <adamw> so, yeah, I'm awesome
16:04:18 <jlaska> adamw: I think we have a t-shirt for that
16:04:18 <adamw> oh, we had the test day. it went well.
16:05:12 <jlaska> #info color management test day recap - http://lists.fedoraproject.org/pipermail/test/2010-February/088574.html
16:05:53 <jlaska> adamw: you had some updates on the privilege escalation front last week as well?
16:06:05 * jlaska doesn't see {{draft}} anymore
16:06:24 <adamw> yeah, it's now officially a policy. yay.
16:06:33 <adamw> so we can get started on writing up some privesc tests if we want.
16:06:47 <adamw> any volunteers? :D
16:07:15 <jlaska> adamw: let's add that to the future TODO list
16:07:25 <adamw> ok
16:07:32 <jlaska> #info privilege escalation is approved as an official policy - https://fedoraproject.org/wiki/Privilege_escalation_policy
16:08:10 <jlaska> #idea develop test cases to validate the privilege escalation policy
16:08:22 <jlaska> alrighty, let's dive into Alpha
16:08:28 <jlaska> #topic F-13-Alpha test status
16:08:50 <jlaska> So you probably saw Hurry's mail that there is an Alpha release candidate available for test
16:09:17 <jlaska> #info F-13-Alpha-RC1 available for test - http://lists.fedoraproject.org/pipermail/test-announce/2010-February/000023.html
16:09:25 <Oxf13> it seems kinda brown paper baggy
16:09:48 <jlaska> Oxf13: it sure does :(
16:09:49 <wwoods> oh no, reall?
16:10:06 <jlaska> so ... I just wante dto spend a few moments making sure we all know what's up ... and what the next steps are
16:10:14 * adamw doesn't really work over the weekend, so can you provide an executive summary?
16:10:27 * adamw claiming not executive rank but executive attention span
16:10:27 <jlaska> so, we got some weekend testing on the RC, which resulted in new additions to the blocker list
16:11:00 <jlaska> #info F13Alpha blocker bugs - https://bugzilla.redhat.com/showdependencytree.cgi?id=538273&hide_resolved=1
16:11:14 <jlaska> adamw: best spot to visit is the test result wiki pages ...
16:11:14 <adamw> gadzooks
16:11:24 <jlaska> https://fedoraproject.org/wiki/Test_Results:Fedora_13_Alpha_RC1_Install#Test_Matrix
16:11:28 <jlaska> has the highlights
16:11:55 <adamw> looks quite...failish.
16:12:09 <Oxf13> I'd say we've got our work cut out
16:12:12 <jlaska> yeah, I'm not at all clear why the change in fortune from TC2
16:12:26 <Oxf13> vastly newer anaconda no?
16:12:55 <jlaska> Oxf13: well, afaict it's not all anaconda
16:13:03 <Oxf13> sure
16:13:06 <adamw> jlaska: the input device breakage is the x server change from hal to udev for input device configuration
16:13:19 <adamw> i knew about that but I didn't know anaconda was actually using the hal system for some kind of configuration
16:13:31 <jlaska> that's https://bugzilla.redhat.com/show_bug.cgi?id=566948
16:13:52 <Oxf13> even the stuff that is "anaconda" might not all be anaconda
16:13:57 <jlaska> right
16:14:12 <jlaska> howabout desktop validation, anyone have input on that front?
16:14:29 <jlaska> kparal, you tried a live image earlier today?
16:14:47 <kparal> jlaska: I did try the today's image
16:15:04 <kparal> jlaska: there's some problem with selinux, just reported: https://bugzilla.redhat.com/show_bug.cgi?id=567319
16:15:21 <OldFart> I tried a desktop and after clicking the install to hard drive nothing happened, then tried in a terminal and nothing happened and then it was very late so went to bed.
16:15:30 <kparal> jlaska: I will do the desktop validation with selinux disabled, that should work
16:15:44 <jlaska> kparal: that sounds F13Alpha worthy
16:15:52 <OldFart> I had selinux disabled.
16:16:00 <jlaska> OldFart: Clyde, welcome!
16:16:10 <adamw> well
16:16:15 <adamw> first i'd want to know if the nightlies are f13 or f14
16:16:20 <adamw> since they diverge now
16:16:25 <kparal> adamw: nirik said it's f13
16:16:27 <adamw> ah k
16:17:17 <nirik> yes, f13
16:17:40 <jlaska> okay, so we have a lot of FAIL so far with RC1.  And it seems clear that we'll need an RC2 if we want to meet the alpha release criteria
16:17:48 <jlaska> sound accurate?
16:17:53 <kparal> yes
16:18:06 <Oxf13> absolutely
16:18:09 <adamw> yeah, it seems pretty no-brainery that we'll need to delay the alpha
16:18:18 <kparal> but we don't have much time to go/nogo meeting
16:18:29 <jlaska> is there any additional testing we can perform in the meantime?
16:19:08 <OldFart> Later today I will try the desktop and see if I can capture to data on whats happening.
16:19:17 <wwoods> I don't think we need a meeting if it doesn't meet basic acceptance criteria
16:19:26 <jlaska> wwoods: it'll certainly be a quick meeting :D
16:19:35 <wwoods> (does it?)
16:19:57 <adamw> i think we hold the meeting anyway and just say 'well, duh, no.' :)
16:20:08 <adamw> anyway, seems like everyone agrees
16:20:18 <adamw> oh, and i'd agree kparal's bug feels like a blocker
16:20:32 <jlaska> is there anything we can do in the meantime, or are we blocked awaiting new test images?
16:20:39 <kparal> adamw: it blocks this: https://fedoraproject.org/wiki/QA:TestCases/Install_Source_Live_Image
16:20:46 <kparal> which is an Alpha criterion
16:21:26 <adamw> kparal: well, more properly it blocks the criterion 'The installer must boot (if appropriate) and run on all primary architectures from default live image...' but yeah
16:21:26 <kparal> jlaska: the desktop validation testing should be possible, as I mentioned, just boot with selinux=0 or enforcing=0
16:21:35 <adamw> kparal: well, that could possibly invalidate the results
16:21:44 <adamw> if selinux caused problems with any of the tests, you wouldn't be able to tell that way
16:21:46 <kparal> hmm
16:21:55 <adamw> so i'm not sure it's good testing to do
16:22:06 <kparal> that sounds valid
16:22:26 <kparal> the testing should be done with selinux enabled, right
16:22:34 <jlaska> unless otherwise specified, yes
16:22:36 <kparal> even desktop testing
16:23:02 <Oxf13> it should, unless selinux is so horked that you get all failures
16:23:13 <Oxf13> failing every test because of selinux doesn't help as much
16:23:23 <jlaska> imo ... it's okay to workaround it temporarily to see what else lurks
16:23:28 <jlaska> but not something to rely on for the release
16:23:44 <adamw> Oxf13: as i said, in that case, i'd rather wait till selinux is fixed then test
16:23:47 <adamw> Oxf13: than test without it
16:23:55 <adamw> Oxf13: because that's the state we would release in
16:24:05 <Oxf13> adamw: I agree but disagree
16:24:10 <kparal> :)
16:24:14 <adamw> well i'm worried this would happen
16:24:17 <jlaska> adamw: it should be tested that way prior to release
16:24:24 <adamw> jlaska: yeah
16:24:26 <Oxf13> if we waited to test, only to find other things that were obviously broken, we've just lost that much time in getting them fixed
16:24:32 <kparal> it's true we can find more errors not related to selinux, if we test now even without selinux
16:24:41 <adamw> Oxf13: fair point
16:24:43 <adamw> so, how about this
16:24:53 <adamw> if you do the desktop validation with selinux disabled you can log *failures* but not successes
16:25:07 <OldFart> I would like to get by the x server issue by using the desktop to test other anaconda features.
16:25:11 <Oxf13> while positive results could not be taken as authoritative, negative results have value.
16:25:11 <jlaska> adamw: oh definitely
16:25:23 <Oxf13> adamw: jynx (:
16:25:28 * adamw call of nature, brb
16:25:32 <kparal> adamw: we can provide PASS results, but we just have to know not to rely it until re-tested again with selinux on
16:26:03 <jlaska> kparal: let's use the WARN or FAIL state for that now
16:26:17 <jlaska> that'll let us know it was tested, and it doesn't meet the stated objectives in the test
16:26:25 <kparal> alright
16:26:32 <jlaska> I think I described WARN for that scenario
16:26:52 <jlaska> either way ... I think we all agreed we can't rely on testing with selinux disabled
16:27:03 <jlaska> alright ... so next steps on the Alpha time ...
16:27:09 <jlaska> #topic F-13-Alpha - next steps
16:27:30 <jlaska> we have a batch of F13Alpha blocker bugs, we have 2 days until the go/no_go meeting
16:28:19 <jlaska> A reminder of the goal ... the F13Alpha list should contain no unMODIFIED bugs
16:28:34 <jlaska> is that accurate?
16:28:43 <Oxf13> I'm on hand to make trees like a madman as we get fixes
16:29:00 <Oxf13> we're going to want to keep a good mapping of potential fixes to bodhi tickets
16:29:05 <skvidal> Oxf13: trees are good - I like oak trees personally
16:29:06 <jlaska> Oxf13: how would you like that type of  feedback?  email/irc/tickets?
16:29:15 * skvidal stops being silly
16:29:16 <Oxf13> jlaska: any of hte above works.
16:29:19 <jlaska> Oxf13: okay
16:29:30 <Oxf13> jlaska: doing it in the ticket is probably a good canonical location for tree compose requests
16:30:03 <jlaska> #info requests for new trees should take place in the open RC ticket (https://fedorahosted.org/rel-eng/ticket/3319)
16:30:38 <adamw> back
16:30:41 <kparal> so, F13Alpha blocker list may contain modified bugs (not fixed) to let F13Alpha GO?
16:31:01 <Oxf13> kparal: yes, if they are modified and there is a build in koji for them
16:31:13 <Oxf13> we can compose the tree before the build makes it all the way through bodhi and shows up in a public repo
16:31:18 <kparal> Oxf13: but the F13Alpha will contain all the fixes, right?
16:31:22 <kparal> ok
16:31:24 <adamw> jlaska: i thought we were trying to avoid it this cycle?
16:31:53 <adamw> the criteria says there shouldn't be any open bugs. really that'd just involve closing them if a build known to fix the bug was in place for the compose, though.
16:31:55 <Oxf13> adamw: I thought we were trying to avoid bugs this cycle too (:
16:32:03 <adamw> i mean, avoid having open ones when releasing :)
16:32:16 <jlaska> adamw: that's correct
16:32:20 <Oxf13> by the time we release, bodhi could have automunged the bugs
16:32:29 <jlaska> we can do builds and composes ... but at release ... these puppies need to be CLOSED
16:32:30 <Oxf13> since we're using bodhi this time around we can let it decide when to close the bug(s)
16:32:52 <adamw> okay anyway, cross that bridge when we get to it
16:33:04 <jlaska> so ... let's talk about that bridge
16:33:16 <jlaska> we don't have any blocker review meetings between now and go/no_go
16:33:29 <jlaska> is there anything QA can do to help the blocker bugs along?
16:33:53 <adamw> jlaska: well, a ton of them are 'needsretesting' anaconda aren't they?
16:34:19 <jlaska> yup, I think we can knock some of those out with the current set (using VNC or text-mode installs)
16:34:51 <jlaska> #help Bugs listed as MODIFIED can be verified using RC1 - https://bugzilla.redhat.com/showdependencytree.cgi?id=538273&hide_resolved=1
16:35:33 <jlaska> anything else to consider on these bugs?
16:36:20 <jlaska> otherwise, I'll need to turn the #chair over to someone else for the remainder of the agenda
16:36:26 <jlaska> any takers on #chair?
16:36:47 <adamw> i'm not so sure about https://bugzilla.redhat.com/show_bug.cgi?id=566995
16:36:59 <adamw> if it's not actually causing any problems in itself
16:37:14 <jlaska> I wasn't either ... until it spits out to the console during install a *LOT*
16:37:32 <jlaska> so I raised it to get some feedback on what's impacted by this ... if it's just noise or something other
16:37:59 <adamw> ah k
16:38:05 <adamw> well, dwalsh is cc'ed so not much we can do there
16:39:38 <adamw> so, I dunno, what's the plan from here?
16:39:52 <adamw> when do we envisage doing an rc2? when the big bugs from the blocker list are fixed?
16:40:01 <jlaska> that's my impression
16:40:03 <adamw> are we working on the assumption we'll be delaying the alpha now?
16:40:24 <jlaska> the phrase I used is ...  the alpha is 'at risk'
16:40:25 <jlaska> :)
16:40:40 <jlaska> #chair adamw
16:40:41 <zodbot> Current chairs: adamw jlaska
16:41:07 <jlaska> I don't have anything else on this topic ... just wanted to make sure all the issues were raised and we had an idea how to proceed
16:41:09 <adamw> okay
16:41:17 <kparal> on Live image the Rawhide repository is enabled, instead of Fedora repo. is that a bug?
16:41:17 <jlaska> so to summarize ...
16:41:22 <adamw> anyone else have anything to add on the alpha topic?
16:41:55 <adamw> kparal: seems like one to me, yeah - could be similar to https://bugzilla.redhat.com/show_bug.cgi?id=566991 ?
16:42:04 <adamw> but definitely file it and escalate as a blocker
16:42:20 <kparal> ok
16:42:25 <jlaska> kparal: good catch
16:43:25 <jlaska> the only other topics I had for today were the open-discussion items
16:43:36 <adamw> yeah
16:43:45 <maxamillion> w00t! I'm late but just in time for open-discussion
16:43:47 <maxamillion> :P
16:43:58 <adamw> hey maxa
16:44:01 <jlaska> adamw: can you walk meetbot through those items?
16:44:01 <maxamillion> I just wanted to point out ---> http://fedoraproject.org/wiki/SecuritySpin:QA_Brainstorm
16:44:08 <adamw> okay okay everyone calm down!
16:44:11 <adamw> deep breaths!
16:44:18 <adamw> #topic open floor
16:44:21 <wwoods> harrumph! etc!
16:44:22 <maxamillion> adamw: hi hi :)
16:44:23 <adamw> now, children
16:44:28 <adamw> form an orderly queue
16:44:36 <adamw> jlaska suggests:
16:44:45 <adamw> #topic open floor: fedora 13 bug filing procedures
16:44:53 <adamw> but notes 'possible BugZapper topic'
16:45:02 <adamw> which, for me, it is - I have something about this on the BZ agenda for tomorrow
16:45:02 <jlaska> adamw: you raised this last week, just wanted to make sure it was captured somewhere
16:45:13 <jlaska> great!
16:45:34 <adamw> i'm just concerned about what we do with bugs filed on rawhide between a release and a branch point, since it's now not clear whether we should treat those as rawhide or f13 bugs, but we'll discuss that in bz tomorrow
16:45:37 <maxamillion> I do like me some procedures
16:45:40 <adamw> anyone have any comments on that / related concerns?
16:46:20 <adamw> okey dokey
16:46:34 <adamw> #topic open floor: defining FAS qa membership
16:46:49 <maxamillion> oooo, good one
16:46:53 <adamw> not sure where this one comes from; every time it's come up in the past we've noted that we don't actually use the FAS system for anything much so it's a no-op
16:47:01 <adamw> i guess someone has new concerns / proposals?
16:47:17 <jlaska> well, we have a real use case now where FAS qa membership provides a benefit
16:47:19 <maxamillion> adamw: its being used for packages in critical path for updates to F13
16:47:25 <jlaska> ^^^ yup
16:47:43 <maxamillion> adamw: those packages have to get karma from members of the FAS group in order to make it through bodhi
16:47:47 <jlaska> so, I just wanted to start this thought process ... and add to the collective team TODO list
16:48:13 <adamw> ahhh
16:48:24 <adamw> so, the question is, how do we define who gets on the list
16:48:31 <maxamillion> bingo
16:48:45 <maxamillion> also, what requirements need to be met for newcomers who want to join the FAS group
16:48:48 <maxamillion> ?*
16:48:50 <adamw> we could take a cue from the bugzappers system for new members
16:49:01 <adamw> and grandparent in existing ones, i guess
16:49:27 <maxamillion> is anyone even a member of that group?
16:49:42 <adamw> i think like three people or something :)(
16:49:46 <Oxf13> well, I'd say flush the group membership except for those who are regularly showing up at this meeting
16:50:01 <wwoods> actually the QA group was flushed a while ago
16:50:02 <wwoods> wasn't it?
16:50:08 <Oxf13> from that point, those in the group can serve as a proxy for those who want to be in the group
16:50:11 <adamw> not everyone who contributes usefully on the list shows up to the meetings
16:50:13 * wwoods tried to flush the QA group about a year ago
16:50:18 <jlaska> it might need a re-flush
16:50:25 <wwoods> who's been adding people to it?
16:50:30 <Oxf13> those who want can do the testing and provide the karma, a QA member can "sign-off" on that testing
16:50:40 <Oxf13> eventually we'll grant full membership to those who prove that they are doing the right thing
16:50:54 <maxamillion> +1
16:51:03 <kparal> we should also create a short wiki document saying how and when to do ACK and when not to do it
16:51:06 <maxamillion> now the question is ... how to put that into verbage for the wiki?
16:51:08 <wwoods> you can't view *approved* group members? feh
16:51:09 <kparal> for QA members
16:51:12 <jlaska> kparal: +++1
16:52:01 <kparal> another policy task :)
16:53:03 <Oxf13> we're going to need to do something very similar for the releng group
16:53:40 <adamw> so what's the next step on this? who wants to draft up some wiki pages?
16:54:02 <maxamillion> I can, but I imagine there will be some heavy editing needed
16:54:10 <maxamillion> but I wouldn't mind putting together the first draft
16:54:14 <kparal> great
16:54:15 <adamw> excellent
16:54:28 <adamw> #action maxamillion to draft up some proposed policy docs for qa FAS group membership
16:54:48 <adamw> that's everything on the list...
16:54:54 <adamw> so, maxamillion, your time to shine :P
16:54:59 <adamw> #topic open floor: security spin
16:55:09 <maxamillion> heh :D
16:55:22 <maxamillion> http://fedoraproject.org/wiki/SecuritySpin:QA_Brainstorm
16:55:53 <maxamillion> I'm trying to work out some bits here since almost all the tools on the spin are command line, I want to get some people to help write some test cases to make sure those bits aren't broken
16:56:27 <maxamillion> then once that piece is done and we can successfully do some manual testing I was wanting to move to a scripted test and *hopefully* toss it in the AutoQA ring
16:56:46 <maxamillion> but that's the large scale hope .... right now its just going to be an information gathering piece
16:57:20 <adamw> yep, like you wrote to the list
16:57:30 <adamw> so, anything in particular to discuss or is this just a call for help?
16:57:52 <maxamillion> more or less just a call for help if anyone has a few spare cycles and knows anything about any of the tools listed
16:58:11 <maxamillion> I don't know about all of them which is really my main motivation behind extending out to the group for help
16:58:28 <adamw> yeah, so get off your lazy butts people :)
16:58:31 <maxamillion> lol
16:58:34 <jlaska> maxamillion: there was a sectool test day a few releases back ... lemme remember who hosted that event so maybe they have ideas
16:58:48 <jlaska> https://fedoraproject.org/wiki/Test_Day:2009-09-01_Sectool
16:58:48 <maxamillion> jlaska: oh, that'd be awesome!
16:59:48 <jlaska> looks like ... tmraz, mbarabas and pvrabec
17:00:18 <maxamillion> oh goodness ... we don't appear to have sectool on the security spin :X
17:01:10 <adamw> #info maxamillion looking for people to help draw up test cases for tools on the security spin
17:01:13 <maxamillion> anyhoo, as far as QA efforts go there really wasn't a whole lot to add to what I posted on the list, I just thought I'd bring it up in case some of the people in the meeting hadn't gotten a chance to read through their email (I know I get backed up quite a bit at times)
17:01:31 <adamw> #action jlaska / maxamillion to co-ordinate with sectool test day hosts tmraz, mbarabas and pvrabec who may be able to contribute
17:02:31 <adamw> okay, that's that one
17:02:35 <adamw> anyone else have an open floor topic?
17:04:04 <jlaska> I already chewed up my allowance :)
17:04:15 <maxamillion> oh, I had a random side question that *might* have been covered and I'm just a tardy turd ... but what's the current libvirt/KVM QA plans, etc.? and where can I find docs on how to help with that? (I have hardware that does KVM now and I'm excited about it)
17:04:42 <adamw> i think our test plans are basically 'we do most of our testing in virtual machines so if it's broken we probably know about it' :P
17:04:46 <adamw> but I may be missing something
17:05:00 <kparal> and we have it as F13Beta criteria
17:05:02 <maxamillion> adamw: fair enough ... that's pretty much what my work flow is at this point as well
17:05:42 <maxamillion> ok, just thought I'd ask :)
17:06:02 <Oxf13> I'm stepping out, I have a local engagement
17:06:07 <adamw> proper formal testing would be useful i guess
17:06:11 <adamw> i don't think we have any planned atm
17:06:48 <adamw> #topic open floor
17:07:07 <adamw> anything else?
17:08:17 <maxamillion> not I
17:08:26 <jlaska> nothing here
17:08:41 * adamw is busy watching youtube videos about insane japanese robot suits
17:09:01 <maxamillion> adamw: LOL
17:09:33 <adamw> alrighty then - thanks for coming everyone
17:09:44 <adamw> http://www.youtube.com/watch?v=G4evlxq34og , btw
17:09:46 <jlaska> adamw: thanks for the #chair assist
17:10:02 <adamw> aka HOLY SHIT IT'S THE FUTURE
17:10:12 <adamw> #endmeeting