15:00:30 <adamw> #startmeeting Fedora QA meeting
15:00:30 <zodbot> Meeting started Mon May 12 15:00:30 2014 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:30 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:34 <adamw> #meetingname fedora-qa
15:00:34 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:37 <adamw> #topic Roll call
15:00:41 <adamw> ahoyhoy folks, who's here?
15:01:43 * adamw watches the dust balls
15:02:11 * satellit listening
15:02:53 * tflink is here
15:03:27 <adamw> let us contemplate the dust balls, together
15:04:38 <adamw> roshi: ahoy?
15:05:00 * pwhalen is here
15:05:14 * kparal joins
15:05:55 * roshi is here
15:06:18 <adamw> ahoyhoy folks
15:06:45 <adamw> welcome to the most exciting ride of your life - oh no, wait, just the qa meeting.
15:07:04 <adamw> #topic Previous meeting  follow-up
15:07:28 <adamw> #info "adamw to synthesize server, desktop and cloud test outlines to give a rough idea of required fedora.next test coverage and kickstart a discussion of how it should be organized" - not done yet, sorry. will work more on it this week
15:07:40 <adamw> #action adamw to synthesize server, desktop and cloud test outlines to give a rough idea of required fedora.next test coverage and kickstart a discussion of how it should be organized
15:07:52 <adamw> #topic Rawhide validation testing
15:08:10 <adamw> so Gene's mailed this morning to say current rawhide images may be badly broken, anyone else seeing that?
15:08:15 <adamw> i'm planning to take a look after the meeting
15:08:34 * kparal looking for bug description
15:08:57 * kparal found it
15:08:57 <brunowolff> In what sense. I successfully rebooted two rawhide machines this morning, including the one I am typing this on?
15:10:16 <brunowolff> (That was after updating to this morning's rawhide updates.)
15:10:45 <adamw> brunowolff: looks like he was testing boot of daily images
15:10:53 <satellit> I get aualligned poiner ..... INIT18 BOOT FAILURE with liveinst
15:11:14 <satellit> unalligned*
15:11:20 <pwhalen> minimal arm image from today boots okay, suffers from existing bugs, but boots
15:11:31 <adamw> hum, okay. sounds like we have something to look into at least
15:11:35 <satellit> need 2 cores to boot lives
15:12:35 <adamw> satellit: you were having that 'unaligned pointer' problem last week too though, right?
15:12:39 <adamw> i don't think gene was having trouble then
15:12:46 <adamw> perhaps your issue is specific to virtualbox?
15:13:10 <adamw> oh, hum, you reported one case of it with metal. hrm.
15:13:11 <satellit_e> https://fedoraproject.org/wiki/Test_Results:Fedora_21_Rawhide_2014_05_Install   yes   no also in bare metal install with DVD
15:13:33 <adamw> and gene says "after about May 7th', which ties in with your timeframe
15:13:33 <adamw> ok
15:13:42 <adamw> so probably you are both hitting the same thing
15:13:55 <satellit> looks like it.....
15:14:07 <adamw> so we should definitely look into that as a priority...i'll hit it after the meeting
15:14:15 <satellit> thanks
15:14:20 <brunowolff> I remember Josh needing to move the iso9660 driver back to core last week. Could that be what they're seeing?
15:14:37 <brunowolff> (At least I think that was the driver.)
15:14:47 <adamw> um. possibly? need more info, i guess =)
15:15:04 <brunowolff> I'll see if I can find the message and come back here with a reference.
15:15:34 <adamw> thanks
15:16:29 <adamw> #topic Fedora.next status and where we go next
15:16:51 <adamw> so I was hoping to have the 'skeleton test plan' for fedora.next that i'm working on available as a sort of jumping-off point for discussion this week...but we can float the topic without it, for sure
15:17:31 <adamw> this topic is kinda meant to cover the ongoing discussion about how far we should be attempting to cover testing for the new Products and so on
15:17:34 <adamw> hey, viking, good timing
15:17:57 <adamw> Viking-Ice: you just missed: "<adamw> so I was hoping to have the 'skeleton test plan' for fedora.next that i'm working on available as a sort of jumping-off point for discussion this week...but we can float the topic without it, for sure"
15:18:46 <adamw> does anyone have any particular thoughts on the topic, or any specifics to discuss, or anything? or would folks rather wait for me to launch the trial balloon so we have something specific to shoot at? :)
15:19:16 <roshi> I got nothing against waiting for you
15:19:27 <brunowolff> This is the start of the devel thread I rememebered: https://lists.fedoraproject.org/pipermail/devel/2014-May/199058.html
15:19:32 <adamw> brunowolff: thanks
15:21:45 <adamw> i know viking strongly feels like 'fedora qa' should cover only base components and leave product QA to the relevant WGs / SIGs / whatever, my current inclination is that 'fedora qa' should probably be responsible for at least directing / overseeing testing on the major 'fedora products', but i really want to get the 'test plan' down to see just how much work that's going to be
15:21:48 <roshi> adamw: with this 'skeleton test plan,' is it going to be similar to the old test plans that used to be written or are you talking matrices?
15:22:31 <adamw> roshi: similar to the old test plans - the textbook definition of 'test plan', with scopes and responsibilities and blah blah, but the idea is really to sort of provide a concrete reference of how much work it's going to be
15:22:56 <roshi> kk
15:23:11 <adamw> i don't think we'd wind up actually using this as the working document for fedora 21 testing, i think it'll be more useful as a way to visualize what fedora-as-a-whole is going to need to get done.\
15:23:52 <Viking-Ice> adamw, my opinion is becoming irrelevant since I'm leaving the project but as I go I still think Fedora QA should be focusing more/better on the core/baseOS part be it comps groups or the specific output from the baseWG, ( the end result is the same ) if and when we would have manage to covered that properly we could extend providing resources to other products given that there exist enough QA community resources and expertise to do so ( as
15:23:52 <Viking-Ice> in covering core/base comps grousp and or baseWG +. whatever else )
15:25:20 <adamw> roger
15:25:30 <Viking-Ice> I have always felt how we done things ( past/present ) been somewhat inefficient
15:25:48 <Viking-Ice> or in subtle terms have a huge room for improvement ;)
15:25:56 <adamw> i like putting it that way better ;)
15:26:04 <roshi> lol
15:26:06 <adamw> there's always room for improvement! :)
15:26:26 <adamw> well, sounds like i should come up with a more concrete basis for discussion, anyhow
15:26:41 <adamw> i have the action item for that already
15:27:34 <adamw> oh, also: big vote of thanks to roshi and franciscod for the cloud and workstation 'outlines' respectively
15:27:59 <adamw> so, we'll come back to that when i have the plan ready...
15:28:04 <adamw> #topic Open floor
15:28:08 <Viking-Ice> the fact remains the same the best people to possess the in-depth knowledge to properly test are the ones producing the product ( in current case ) and the community surrounding that
15:28:13 <adamw> oops, sorry
15:29:13 <brunowolff> But the people doing the testing would almost certainly benefit from consulting with the people who have been QAing Fedora a long time.
15:29:23 <Viking-Ice> right
15:30:05 <adamw> i think however we wind up constituting things exactly, development and testing should always be as collaborative as possible...
15:30:10 <Viking-Ice> I envisioned both the roles of QA and releng and other service sub community's in the project being more of an guiding force rather then implementing and or testing one
15:30:37 <Viking-Ice> historically we have been faced with the same problem regarding test day's as we are now with products
15:31:14 <Viking-Ice> these are target spesific testing which often requires specific skill set even hardware to complete
15:31:39 <adamw> i do find it's sometimes useful on test days when people who know almost nothing about the thing under testing come and try to test things
15:31:56 <roshi> it's a good place to become familiar
15:31:57 <adamw> often the devs find they've made assumptions about how people will do stuff or how they'll have their systems set up that are not correct
15:32:20 <adamw> but you're right that often you need specialist knowledge or config/hardware to really test out something extensively
15:32:53 <adamw> whoever's administratively 'in charge' we certainly need to be getting the folks developing the .next products to be actively involved in testing them, i agree completely on that
15:33:13 <roshi> none of this will remotely work otherwise :)
15:33:35 <Viking-Ice> right but limiting it to test day   which we never intended to do at least not me and James ) as opposed to days or weekends weeks even months then you wind up spending resources assisting the people not in the know how, teaching them about the software they are dealing with
15:34:17 <Viking-Ice> you can take 389ds for example
15:34:25 <adamw> we did try running longer 'test day' type events a few times...i found it was hard both to dedicate the time / effort to running  them effectively, and keep testers' interest over the longer term
15:34:47 <adamw> even graphics test week is just a heck of a lot of work to run
15:35:08 <Viking-Ice> which is one of the bigger problem we are dealing with in the project
15:35:08 <roshi> true - but it was good, imo
15:35:10 <Viking-Ice> lack of proper tools
15:35:39 <roshi> like what?
15:35:59 <Viking-Ice> like everything
15:36:19 <Viking-Ice> ranging from feature process handling to release cycle to proper documentation tool and even bugzilla
15:36:33 <adamw> you've got to start somewhere
15:37:00 <roshi> I've only known what we currently have - which is why I was asking
15:37:01 <Viking-Ice> we equally less prepared infrastructure wize to deal with multiple product as we are policy wise etc
15:37:44 <Viking-Ice> @dayjob we have infrastructure in place which surpasses the project ones in decades
15:37:48 <roshi> proper documentation tool == not the wiki, is what you mean, right?
15:38:03 <Viking-Ice> right
15:38:12 <roshi> FOSS infrastructure we could use/emulate?
15:38:13 <Viking-Ice> linking for example to documentation from within bug reports
15:39:24 <adamw> you're also probably building a smaller product with more people. i often get the same feeling that fedora is run on duct tape and wd40, but sometimes you have to work mostly with what you have. we have folks working on better tooling, but you know the realistic timescales for it.
15:39:57 <Viking-Ice> we have 2000 projects here with ca 20000 customers in 3 different country
15:40:18 <Viking-Ice> ranging from service support to development, to project management etc
15:40:33 <adamw> fedora has, what, 10k+ SRPMs, with millions of users in X different countries =)
15:40:57 <Viking-Ice> our clients are everywhere but we have companies in three different countries
15:41:05 <adamw> (well, counting users is hard, but at least hundreds of thousands.)
15:41:17 <Viking-Ice> we sold what we owned in Denmark since it gave us bad ebita
15:41:45 <adamw> i'm just saying, it's not realistic to apply the standards of commercial software development to a project like fedora; if we sit around waiting for an infrastructure of that type to show up, it's just not going to work
15:41:55 <adamw> we're a shoestring-and-dreams kind of project, we have to function on that level
15:42:05 <tflink> and there's always the question of cost-benefit
15:42:08 <adamw> (first 'we' there is fedora, second 'we' is fedora qa)
15:42:48 <roshi> and we can always make it better
15:43:28 <Viking-Ice> if I was involved in RH management of the project
15:43:42 <Viking-Ice> I would be investing in proper tools to gain as much out of the project as I could
15:44:05 <Viking-Ice> regardless of license and given that the foundations are under constant attack these days
15:44:05 <adamw> Viking-Ice: fwiw, we - the RH fedora qa folks - make that argument as often as we can
15:44:32 <Viking-Ice> what's the difference of just give up on the whole idealism of "free and open"
15:44:36 <adamw> Viking-Ice: tflink has a list as long as your arm of things he'd like to have the resources to get built
15:44:45 <adamw> many the kind of infrastructure things you're talking about
15:45:13 <Viking-Ice> yes but that requires man power to build ourselves
15:45:23 <Viking-Ice> which we dont have
15:45:24 <adamw> what we have now in terms of manpower and hardware etc is what RH is willing to provide
15:45:26 <adamw> yeah.
15:45:46 <Viking-Ice> now and indefinitely
15:45:48 <adamw> what i'm saying is, we do go out and ask for the resources to build all that stuff, and what you see is what we get
15:46:01 <Viking-Ice> I've been under the illusion we could somehow change that for the past 8 years
15:46:04 <Viking-Ice> dam was I wrong
15:46:11 <adamw> so we have to prioritize the work according to the resources...and that means right now, taskotron is priority #1, then we have a list of stuff for after that
15:46:28 <Viking-Ice> it's easier to buy existing solution that building it from ground up ( if possible )
15:46:40 <Viking-Ice> s/that/then
15:46:51 * kparal looks up
15:46:58 <roshi> well sure - it's 'easier'
15:47:09 <adamw> i'd love to have RH approve five new full-time fedora qa tool hires next week, but it's probably not gonna happen. i'd love to have more folks with the ability and motivation to work in that area come along from the community, and maybe we could push harder for that too...
15:47:10 <roshi> but so is buying a sandwich than making one
15:47:22 <Viking-Ice> and the Atlantis products ( jira/confluence ) most certanly can and are cost free for open source projects
15:47:38 <tflink> for the licenses, sure
15:47:48 <tflink> but they have to be hosted and maintained
15:48:00 <Viking-Ice> I know I maintain the one here
15:48:00 <kparal> we can hardly run anything non-open-source inside Fedora infra
15:48:02 <adamw> and you have to put in the huge amount of work to reorient the entire fedora workflow around them
15:48:15 <Viking-Ice> that's a full time job including handling workflows etc
15:49:02 <Viking-Ice> it would take team of no larger then four persons by me guesstimate for the project to maintain
15:49:17 <Viking-Ice> all products
15:49:19 <tflink> which is one of the concerns I have - we'd be increasing our (fedora's) cost significantly if we shift away from what RH is already using
15:50:04 <adamw> Viking-Ice: how do you mean 'maintain all products'?
15:50:06 <Viking-Ice> well RH BRNO has XP running in their receptions so I honestly dont think they care us much as they want to believe in open
15:50:08 <misc> adamw: wel, the new hire doesn't have ti be strictly from RH
15:50:25 <Viking-Ice> adamw, update/upgrades built workflows post functions etc
15:50:45 <Viking-Ice> we have a complete automated triage process in place
15:50:58 <adamw> Viking-Ice: that gets into inside baseball, but let's say at least that the question of *fedora's* F/OSS principles is somewhat separate from RH's.
15:51:22 <Viking-Ice> from what's been taken place used to be
15:51:42 <kparal> Viking-Ice: really? I have to see that. AFAIK all PCs run Fedora or RHEL
15:51:42 <adamw> misc: do you know of someone else interested in hiring people to work on fedora QA? if so, for mercy's sake, send me their contact details :)
15:51:57 <Viking-Ice> we dont have the resources to built this, we have been under the illusion could since the project began
15:51:59 <kparal> Viking-Ice: oh, there's one reception with XP - the one that's not ours
15:52:19 <adamw> misc: like i said we could renew efforts to get more community help with tooling...it's always good to take a fresh push at that
15:52:22 <Viking-Ice> kparal, and the first computer you see when you enter RH HQ in EU that XP ;)
15:52:44 <misc> RH HQ in EU is not Brno, but aht's I think not relevant to the topic of the meeting, no ?
15:53:21 <Viking-Ice> Oh interesting I thought you guys where the biggest and most bad ass but yeah it's irrelevant
15:53:30 <adamw> Viking-Ice: at least one of RH's brno offices is in a shared building - the reception you're thinking of is the *building* reception. we can't really go and tell the building management to use RH. although perhaps we should send in a salesperson. :P
15:53:42 <kparal> Viking-Ice: it's landlord's
15:54:19 <Viking-Ice> anyway lack of tooling
15:54:28 <number80> please ladies and gentlemen, keep focused (and forget that RH even exist)
15:54:36 <adamw> number80: thanks
15:54:52 <roshi> what kinds of gains do you think we'd see with different tooling?
15:55:02 * adamw notes 5 minute warning
15:55:23 <Viking-Ice> automation, better oversight of various things we implement, better management in general you name it
15:55:42 <Viking-Ice> we are in the shit hole with all of this
15:55:49 <roshi> because I've almost always seen that people always work *in spite* of their tooling regardless of how good the tooling is
15:56:07 * adamw certainly doesn't work 'in spite of' git.
15:56:09 <Viking-Ice> we are suffering from massive resources leakage in terms of contribution
15:56:20 <Viking-Ice> and oversight how and where things stand
15:56:27 <adamw> okay, new rule: all tools have to be as good as git. get on it, roshi/tflink/kparal
15:56:43 <pingou> ^^
15:56:46 * pingou ducks
15:56:51 <roshi> we'll just put / in a repo - and go from there
15:56:52 <kparal> i.e. really hard to use for newcommers?
15:56:58 <Viking-Ice> I have yet to see us properly keep tab on how well our ideas are working out
15:57:15 <Viking-Ice> global wide in the project
15:57:45 <Viking-Ice> name me one before and after measurement us QA have ever done or just somewhere in the project?
15:58:41 <Viking-Ice> how many people used it, did it improve things or make them worse, we have no fracking idea
15:58:45 <adamw> Viking-Ice: depends what you mean. i've done numbers on anaconda and GNOME memory usage a few times./
15:59:10 <roshi> that's also really hard to get a good measure of
15:59:18 <adamw> we do stats on graphics test weeks, we do stats on bodhi.
15:59:20 <Viking-Ice> those are memory usages not ideas, features or proesses implemented in the project or distribution
15:59:36 <roshi> to even establish a baseline for - but I would love to be able to get those metrics
16:00:15 <Viking-Ice> we are not keeping track on any of this and the whole change process seems to be done more for the sake of change and Matt's gut feeling then any actually statistic in usage droppings for example
16:00:23 * satellit_e afk
16:00:31 <Viking-Ice> think about it we are at our largest component base so far
16:00:32 <adamw> well, now we're outside QA scope again.
16:01:01 <Viking-Ice> just pointing out global things we as in QA are as bad at measuring things as anyone else in the project
16:01:32 <roshi> any suggestions on how to measure them within QA? because I would love to
16:01:43 <adamw> what is it you think we should be measuring?
16:01:50 <adamw> measuring 'quality' is notoriously hard.
16:01:59 <roshi> bah - we're over time anyhow at this point, can continue in #fedora-qa
16:02:06 <roshi> exactly
16:02:28 <adamw> i'll wrap up here in a minute, can continue in -qa as roshi said...
16:02:38 <adamw> though i need to go get some breakfast (belly rumble)
16:03:55 <adamw> thanks for the discussion, folks
16:03:58 <adamw> #endmeeting