15:00:10 <adamw> #startmeeting Fedora QA meeting
15:00:10 <zodbot> Meeting started Mon Mar 10 15:00:10 2014 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:10 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:12 <adamw> #meetingname fedora-qa
15:00:12 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:14 <adamw> #topic Roll call
15:00:20 <adamw> morning folks, who's around?
15:00:24 * adamw goes back to making pancakes
15:00:34 * satellit listening
15:00:56 * cmurf is almost lucid
15:00:59 * tflink is around
15:01:01 * kparal is here for once
15:01:32 * roshi is here
15:01:38 * brunowolff is here
15:01:58 * roshi will take a pancake
15:02:05 * mkrizek arrives
15:03:47 * pwhalen is here
15:04:48 <adamw> mmmm...pncakes
15:04:51 <adamw> NO PANCAKES FOR ROSHI
15:04:55 <adamw> #chair roshi cmurf
15:04:55 <zodbot> Current chairs: adamw cmurf roshi
15:05:06 <adamw> #topic previous meeting follow-up
15:05:20 <adamw> lessee
15:05:23 <roshi> w/e I prefer waffles anyway - they're like pancakes, but with character
15:05:24 * jreznik is around
15:05:35 <jreznik> (for pancakes)
15:05:41 <adamw> #info roshi is a charlatan
15:05:49 <cmurf> haha
15:05:55 <roshi> it's been weeks since I've been called that
15:06:03 <roshi> :)
15:06:07 <adamw> #info "adamw and cmurf and anyone else interested to keep watching and driving the product WG discussions on filesystems" - that one was just a sort of watching brief, more solid info coming up
15:07:02 <adamw> #info "adamw to ask fesco to consider KDE release blocking status" - I've done that, https://fedorahosted.org/fesco/ticket/1243 - FESCo punted for now but will discuss it when appropriate
15:07:10 <adamw> (this week)
15:07:40 <jreznik> btw. tomorrow kde sig meeting, we will discuss our own product
15:07:52 <jreznik> (on tomorrow's)
15:07:54 <adamw> #info "adamw to propose disabling of autokarma on depcheck fail (to FESCo) as a stopgap till more reliable depcheck with taskotron" - I did that, https://fedorahosted.org/fesco/ticket/1242 , and the proposal was accepted, just need lmacken and tflink to do it now :)
15:08:13 <adamw> #info KDE SIG will discuss a KDE Product at meeting tomorrow
15:08:32 <kparal> I'm a bit afraid some of the maintainers won't like us, because it will disable autopush completely for their packages
15:08:51 * pschindl is here
15:08:51 <kparal> (some packages always fail depcheck)
15:09:18 <adamw> it won't disable anything completely, as designed
15:09:29 <adamw> the intent is that the maintainer can flip it back on if the test failure is false, and it'll stay on
15:09:36 <jreznik> kparal: autokarma is bad concept, so I'll be happier to be hated
15:09:46 <adamw> in other words, all bodhi does is flip the bit.
15:09:58 <kparal> adamw: but only for that particular update, right? he needs to flip it back on another update again
15:10:00 <adamw> yeah
15:10:10 <kparal> jreznik: I totally agree here
15:10:16 <adamw> but this is going to be relatively short term, anyhow, it doesn't need to be perfect
15:10:22 <kparal> ok
15:10:29 <adamw> if it winds up taking longer than expected to deploy taskotron, and we get lots of hate, we can always adjust
15:10:43 <kparal> ok, just wanted to mention it
15:10:58 <adamw> #info kparal notes that the autokarma change may annoy some maintainers, we will keep an eye on feedback and taskotron readiness and stand ready to try and make adjustments if it seems necessary
15:11:09 <adamw> any other follow-up on follow-up? :)
15:12:44 * roshi has nothing
15:13:18 <adamw> okey dokey
15:13:31 <adamw> #topic Fedora.next plans
15:13:50 <adamw> So, the desktop and server tech specs seem to be fairly firmed up now
15:13:54 <adamw> those links again:
15:13:58 <adamw> https://fedoraproject.org/wiki/Workstation/Technical_Specification
15:14:00 <adamw> https://fedoraproject.org/wiki/Server/Technical_Specification
15:14:49 <Martix> https://fedoraproject.org/wiki/Fedora_Plasma_Product/Technical_Specification
15:14:59 <Martix> WIP :-)
15:15:04 <adamw> Martix: thanks  :)
15:15:27 <adamw> of note, I don't think it's written into them exactly, but I believe we came to an agreement to drop the Filesystem Foot Shooter from the non-custom install path, which is great
15:15:57 <roshi> you should trademark that - "Filesystem Foot Shooter (tm)"
15:16:00 * kparal is not sure what that means
15:16:08 <Martix> kparal: +1
15:16:16 <adamw> I also checked with current F21 and noticed that anaconda team actually already *did* that, so unless someone mounts a desperate rear guard action, it looks like we have some improvement in the filesystem testing situation for f21
15:16:36 <adamw> kparal: martix: the drop-down on Installation Options which let you pick LVM, btrfs, LVM thinp or plain ext4.
15:16:37 <roshi> limiting the choices for non-custom installs - getting rid of the drop down
15:16:47 <kparal> what's the default now?
15:16:54 <adamw> kparal: right now, ext-on-LVM.
15:17:05 <kparal> ok
15:17:06 <adamw> however, for the products, server wants xfs-on-LVM, workstation wants ext4-on-LVM.
15:17:12 <cmurf> I think it's XFS on LVM for Server and ext4 on LVM for Workstation.
15:17:14 <cmurf> right
15:17:16 <adamw> so we'll still have two variants to test, unfortunately, but it's better than two.
15:17:19 <adamw> ....better than four.
15:17:20 * satellit it is in custom now (dropdown ) f21.24-1
15:17:26 <adamw> satellit: yes, like I said.
15:17:29 <cmurf> And Server might choose LVM thinp on top of that.
15:17:31 <Martix> adamw: custom partitioning gui wont be available?
15:17:36 <jreznik> btw. for next - I prepared initial F21 schedule (goes to FESCo table this week) - I'd like to have your thoughts on it
15:17:57 <jreznik> https://fedorahosted.org/fesco/ticket/1178#comment:31
15:17:59 <cmurf> Martix: custom partitioning remains
15:18:23 <Martix> cmurf: ok
15:19:04 <adamw> jreznik: i'll put that in open floor if it's OK
15:19:29 <adamw> so, I guess we can take another cut at partitioning test revision now we have this plan in place
15:19:40 <jreznik> adamw: ok, np - I just tried to avoid to forget to mention it here :)
15:19:40 <sgallagh> jreznik: I think "pre-next" may have to go on the hall of fame of terms :)
15:19:49 <adamw> should I take another cut at my matrix? cmurf, did you want to come up with something? anyone else want to take a swing?
15:20:22 <jreznik> sgallagh: :D
15:20:26 <cmurf> It's probably a good idea to look at revision in a newui context since, as you've pointed out, a lot of the tests are oldui based.
15:20:59 <adamw> cmurf: that was the original point, really...my current draft is already newui-based
15:21:23 <cmurf> Maybe as the matrix is being done, to rank them in the categories previously discussed: critical, nice, bonus
15:22:19 <roshi> I'll help with that effort
15:22:47 <cmurf> While I still think options in custom partitioning should be tested or be removed, there appears to not be sufficient political will to cull features from the custom partition path.
15:23:13 <adamw> cmurf: we already get 'critical' and 'not-critical' by the 'release level' attribute
15:23:24 <adamw> we *can* split nice/bonus...but i'm not sure if there's value in it...
15:23:34 <cmurf> release level attribute?
15:23:39 <adamw> Alpha, Beta, Final.
15:23:45 <adamw> if it has one of those, it's critical.
15:23:49 <adamw> if it doesn't, it's not.
15:23:52 <cmurf> OK
15:23:56 <adamw> this is already followed in the current matrix
15:24:03 <cmurf> So ext3 is critical, ext2 isn't, e.g.
15:25:09 <adamw> oh, yeah, given the format of the current draft it becomes a bit tricky to identify. but i'll see what I can do about that in the new draft
15:25:13 <cmurf> Which we can haggle over later, I just want to be sure I'm understanding what you're talking about.
15:25:47 <cmurf> There's a matrix entry for testing rootfs on ext3, there isn't one for ext2, ergo, ext3 is "critical" and ext2 is "not-critical".
15:26:00 <adamw> yeah, in the format of the current draft you couldn't indicate the same test being 'critical' for one filesystem but 'non-critical' for another. i was thinking more of the current actual matrix.
15:26:06 <adamw> cmurf: no, that's not quite what i meant
15:26:18 <adamw> the ones that are non-critical have the release level "optional"
15:26:23 <cmurf> So what I was thinking of for "bonus" were all the more esoteric things that maybe we don't even really want to test, but to demonstrate that they can be done in the installer, yet we really have no practical way to test them - sort of thing.
15:26:25 <adamw> (used to just be empty, i guess someone went and filled them in)
15:26:40 <kparal> that might have been me
15:26:53 <cmurf> But the distinction is probably not that useful - probably just adds more bureaucracy.
15:28:29 <adamw> cmurf: let's keep it in mind and see how the draft looks
15:29:02 <cmurf> Meh - I'm in favor of just keeping the test matrix as concise as we can. If it's not listed, but can be done, it's ipso facto a bonus.
15:29:20 <adamw> true
15:29:41 <Martix> is there any progress in testing automation? especially storage? beaker lab with machines with various HW storage setups would be nice
15:29:51 <adamw> #action adamw to come up with another new draft storage test matrix based on new-new storage UI (no filesystem dropdown) and product filesystem choices
15:30:16 <tflink> Martix: can you be a bit more specific?
15:30:18 <adamw> Martix: i don't have any specific news, i know anaconda team is 'working on it'
15:30:32 <adamw> and some things we could maybe make taskotron testing a long-term goal for
15:30:34 <tflink> we have no plans to have anything other than virt clients for beaker
15:32:15 <Martix> tflink: virt could be enough for start
15:32:34 <adamw> you can do quite a lot of storage stuff in virt, sure. we're a way down the line there, though.
15:33:27 <tflink> beaker hasn't been the highest priority as of late but dcallagh made some changes to the beaker server that should help workaround the problems we were having
15:33:27 <Martix> adamw: cmurf on what is Anaconda team exactly working regarding automatic testing?
15:33:41 * tflink needs to follow up on a secondary problem he was seeing, though
15:33:52 <cmurf> Martix: dunno
15:34:15 <adamw> Martix: i don't know in detail. really they're looking at it from the point of view of CI/unit testing for the installer code, AIUI.
15:34:19 <adamw> but best talk to them about it
15:35:40 <Martix> unit testing is one thing, but what takes valuable QA time is blackbox testing, any efforts on improving automation in this area?
15:35:55 <Martix> in Anaconda
15:37:15 <adamw> i don't think anyone is working on it specifically
15:37:26 * satellit is there any difference in live (liveinst) and boot.iso anaconda installs and using wireless vs wired connections?
15:37:31 <tflink> Martix: any suggestions?
15:37:44 <adamw> it was always a goal for autoqa, i'm assuming it's still a goal for taskotron
15:37:57 <adamw> in which case the answer is that we need to finish building taskotron first (or, well, finish making it usable at least)
15:38:10 <tflink> I'm not sure blackbox testing fits well into taskotron
15:38:20 <tflink> or autoqa, for that matter
15:38:28 <adamw> oh, sorry, as in actual mystery hardware
15:38:35 <tflink> ?
15:39:03 <adamw> grf, sorry, i'm multitasking and missing bits.
15:39:36 <kparal> Martix: nothing has really changed in the last 2 weeks that you haven't asked
15:40:01 <adamw> tflink: well, i kinda remembered when we had RATS in autoqa that the plan was to expand that out into a set of different installation paths
15:40:10 <adamw> though it never happened
15:41:07 <tflink> that's still something that I'd like to see automated but unless I'm misunderstanding, it's a bit orthagonal to what Martix was asking about
15:41:26 <kparal> I think we should move on to the next topic
15:41:49 <adamw> tflink: i'm not entirely sure what martix was asking either :)
15:42:04 <adamw> kparal: well, the next topic was another subtopic of this one
15:42:16 <adamw> kind of open-ended, though
15:42:42 <adamw> i was wondering whether we should try looking at the server/workstation tech specs as a framework for test planning - try and design tests around the specs
15:42:56 <cmurf> makes sense
15:43:06 <kparal> manual or automated? both?
15:43:09 <adamw> pretty clearly going to be more stuff there than we have time to test regularly, but we can at least consider drawing up test plans
15:43:19 <adamw> kparal: both, i guess
15:43:23 <cmurf> hard to argue with "incongruent with the spec"
15:43:29 <danofsatx-work> oh crud....network issues here. I'm here, really....
15:43:37 * danofsatx-work slinks off to the corner in shame
15:43:49 <kparal> it can be a good source of what to focus at
15:43:57 <cmurf> and conversely "meets the spec therefore not a bug" etc
15:44:16 <adamw> danofsatx-work: heh, don't sweat it
15:44:19 <cmurf> And if there's vagueness in the spec, that's cause to update the spec to make it clear.
15:44:32 <adamw> cmurf: mostly the specs aren't quite that detailed, but sure
15:45:04 <adamw> i'm thinking stuff like 'if it's listed in the spec, we should aim to have a basic test plan for it'
15:45:15 <cmurf> It's fine for them to not have certain details, but if they're totally vague such that it's not helpful in plotting some direction forward, then the spec isn't a spec.
15:45:36 <cmurf> A spec is "a detailed working description"
15:46:40 * handsome_pirate stumbles in
15:46:47 <adamw> well, take a look and point up any bits you think are too vague...
15:46:48 <cmurf> Plus it's an "appeal to a higher authority" :-D
15:47:26 <cmurf> adamw: I'm not suggested there are now parts that are vague. I'm just saying it's a good idea to use the specs as the basis for test planning.
15:47:37 <adamw> cmurf: sure
15:49:13 <adamw> OK, so in general we're happy with that idea? cool. if people want to look at the specs and draw up draft tests for things we don't have covered yet...:)
15:49:21 <adamw> that Base matrix has always looked a little small ;)
15:49:40 <kparal> :)
15:50:27 <adamw> #topic Open floor
15:51:03 <roshi> ping jreznik
15:51:04 <adamw> #info jreznik points out a draft F21 schedule he'd like comments on: https://fedorahosted.org/fesco/ticket/1178#comment:31
15:51:41 <adamw> looks fairly reasonable to me
15:51:49 <adamw> gives us plenty of time before alpha (yay) and a reasonable alpha->beta gap
15:51:52 <jreznik> thanks
15:51:58 <adamw> anyone else?
15:52:07 <jreznik> for ping and for reply
15:52:50 <cmurf> looks reasonable to me also
15:53:08 <handsome_pirate> +1
15:53:20 <jreznik> could you #info it, so I can put it into the ticket?
15:54:08 <adamw> already did.
15:54:13 <adamw> or you mean my response?
15:55:52 <adamw> anything else for open floor, folks?
15:56:16 <jreznik> adamw: response
15:56:51 <jreznik> (or better response for pancakes - next time you'll be in Brno ;-)
15:57:23 <adamw> #info adamw, cmurf and handsome_pirate thought the schedule looked good - plenty of time before Alpha, but still a decent Alpha->Beta gap
15:58:05 <jreznik> thanks!
15:58:33 <adamw> welp, let's set the Quantum Fuse then
15:58:39 <adamw> and maybe even finish within an hour - shocking!
16:00:12 <adamw> #endmeeting