15:00:10 #startmeeting Fedora QA meeting 15:00:10 Meeting started Mon Mar 10 15:00:10 2014 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:10 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:12 #meetingname fedora-qa 15:00:12 The meeting name has been set to 'fedora-qa' 15:00:14 #topic Roll call 15:00:20 morning folks, who's around? 15:00:24 * adamw goes back to making pancakes 15:00:34 * satellit listening 15:00:56 * cmurf is almost lucid 15:00:59 * tflink is around 15:01:01 * kparal is here for once 15:01:32 * roshi is here 15:01:38 * brunowolff is here 15:01:58 * roshi will take a pancake 15:02:05 * mkrizek arrives 15:03:47 * pwhalen is here 15:04:48 mmmm...pncakes 15:04:51 NO PANCAKES FOR ROSHI 15:04:55 #chair roshi cmurf 15:04:55 Current chairs: adamw cmurf roshi 15:05:06 #topic previous meeting follow-up 15:05:20 lessee 15:05:23 w/e I prefer waffles anyway - they're like pancakes, but with character 15:05:24 * jreznik is around 15:05:35 (for pancakes) 15:05:41 #info roshi is a charlatan 15:05:49 haha 15:05:55 it's been weeks since I've been called that 15:06:03 :) 15:06:07 #info "adamw and cmurf and anyone else interested to keep watching and driving the product WG discussions on filesystems" - that one was just a sort of watching brief, more solid info coming up 15:07:02 #info "adamw to ask fesco to consider KDE release blocking status" - I've done that, https://fedorahosted.org/fesco/ticket/1243 - FESCo punted for now but will discuss it when appropriate 15:07:10 (this week) 15:07:40 btw. tomorrow kde sig meeting, we will discuss our own product 15:07:52 (on tomorrow's) 15:07:54 #info "adamw to propose disabling of autokarma on depcheck fail (to FESCo) as a stopgap till more reliable depcheck with taskotron" - I did that, https://fedorahosted.org/fesco/ticket/1242 , and the proposal was accepted, just need lmacken and tflink to do it now :) 15:08:13 #info KDE SIG will discuss a KDE Product at meeting tomorrow 15:08:32 I'm a bit afraid some of the maintainers won't like us, because it will disable autopush completely for their packages 15:08:51 * pschindl is here 15:08:51 (some packages always fail depcheck) 15:09:18 it won't disable anything completely, as designed 15:09:29 the intent is that the maintainer can flip it back on if the test failure is false, and it'll stay on 15:09:36 kparal: autokarma is bad concept, so I'll be happier to be hated 15:09:46 in other words, all bodhi does is flip the bit. 15:09:58 adamw: but only for that particular update, right? he needs to flip it back on another update again 15:10:00 yeah 15:10:10 jreznik: I totally agree here 15:10:16 but this is going to be relatively short term, anyhow, it doesn't need to be perfect 15:10:22 ok 15:10:29 if it winds up taking longer than expected to deploy taskotron, and we get lots of hate, we can always adjust 15:10:43 ok, just wanted to mention it 15:10:58 #info kparal notes that the autokarma change may annoy some maintainers, we will keep an eye on feedback and taskotron readiness and stand ready to try and make adjustments if it seems necessary 15:11:09 any other follow-up on follow-up? :) 15:12:44 * roshi has nothing 15:13:18 okey dokey 15:13:31 #topic Fedora.next plans 15:13:50 So, the desktop and server tech specs seem to be fairly firmed up now 15:13:54 those links again: 15:13:58 https://fedoraproject.org/wiki/Workstation/Technical_Specification 15:14:00 https://fedoraproject.org/wiki/Server/Technical_Specification 15:14:49 https://fedoraproject.org/wiki/Fedora_Plasma_Product/Technical_Specification 15:14:59 WIP :-) 15:15:04 Martix: thanks :) 15:15:27 of note, I don't think it's written into them exactly, but I believe we came to an agreement to drop the Filesystem Foot Shooter from the non-custom install path, which is great 15:15:57 you should trademark that - "Filesystem Foot Shooter (tm)" 15:16:00 * kparal is not sure what that means 15:16:08 kparal: +1 15:16:16 I also checked with current F21 and noticed that anaconda team actually already *did* that, so unless someone mounts a desperate rear guard action, it looks like we have some improvement in the filesystem testing situation for f21 15:16:36 kparal: martix: the drop-down on Installation Options which let you pick LVM, btrfs, LVM thinp or plain ext4. 15:16:37 limiting the choices for non-custom installs - getting rid of the drop down 15:16:47 what's the default now? 15:16:54 kparal: right now, ext-on-LVM. 15:17:05 ok 15:17:06 however, for the products, server wants xfs-on-LVM, workstation wants ext4-on-LVM. 15:17:12 I think it's XFS on LVM for Server and ext4 on LVM for Workstation. 15:17:14 right 15:17:16 so we'll still have two variants to test, unfortunately, but it's better than two. 15:17:19 ....better than four. 15:17:20 * satellit it is in custom now (dropdown ) f21.24-1 15:17:26 satellit: yes, like I said. 15:17:29 And Server might choose LVM thinp on top of that. 15:17:31 adamw: custom partitioning gui wont be available? 15:17:36 btw. for next - I prepared initial F21 schedule (goes to FESCo table this week) - I'd like to have your thoughts on it 15:17:57 https://fedorahosted.org/fesco/ticket/1178#comment:31 15:17:59 Martix: custom partitioning remains 15:18:23 cmurf: ok 15:19:04 jreznik: i'll put that in open floor if it's OK 15:19:29 so, I guess we can take another cut at partitioning test revision now we have this plan in place 15:19:40 adamw: ok, np - I just tried to avoid to forget to mention it here :) 15:19:40 jreznik: I think "pre-next" may have to go on the hall of fame of terms :) 15:19:49 should I take another cut at my matrix? cmurf, did you want to come up with something? anyone else want to take a swing? 15:20:22 sgallagh: :D 15:20:26 It's probably a good idea to look at revision in a newui context since, as you've pointed out, a lot of the tests are oldui based. 15:20:59 cmurf: that was the original point, really...my current draft is already newui-based 15:21:23 Maybe as the matrix is being done, to rank them in the categories previously discussed: critical, nice, bonus 15:22:19 I'll help with that effort 15:22:47 While I still think options in custom partitioning should be tested or be removed, there appears to not be sufficient political will to cull features from the custom partition path. 15:23:13 cmurf: we already get 'critical' and 'not-critical' by the 'release level' attribute 15:23:24 we *can* split nice/bonus...but i'm not sure if there's value in it... 15:23:34 release level attribute? 15:23:39 Alpha, Beta, Final. 15:23:45 if it has one of those, it's critical. 15:23:49 if it doesn't, it's not. 15:23:52 OK 15:23:56 this is already followed in the current matrix 15:24:03 So ext3 is critical, ext2 isn't, e.g. 15:25:09 oh, yeah, given the format of the current draft it becomes a bit tricky to identify. but i'll see what I can do about that in the new draft 15:25:13 Which we can haggle over later, I just want to be sure I'm understanding what you're talking about. 15:25:47 There's a matrix entry for testing rootfs on ext3, there isn't one for ext2, ergo, ext3 is "critical" and ext2 is "not-critical". 15:26:00 yeah, in the format of the current draft you couldn't indicate the same test being 'critical' for one filesystem but 'non-critical' for another. i was thinking more of the current actual matrix. 15:26:06 cmurf: no, that's not quite what i meant 15:26:18 the ones that are non-critical have the release level "optional" 15:26:23 So what I was thinking of for "bonus" were all the more esoteric things that maybe we don't even really want to test, but to demonstrate that they can be done in the installer, yet we really have no practical way to test them - sort of thing. 15:26:25 (used to just be empty, i guess someone went and filled them in) 15:26:40 that might have been me 15:26:53 But the distinction is probably not that useful - probably just adds more bureaucracy. 15:28:29 cmurf: let's keep it in mind and see how the draft looks 15:29:02 Meh - I'm in favor of just keeping the test matrix as concise as we can. If it's not listed, but can be done, it's ipso facto a bonus. 15:29:20 true 15:29:41 is there any progress in testing automation? especially storage? beaker lab with machines with various HW storage setups would be nice 15:29:51 #action adamw to come up with another new draft storage test matrix based on new-new storage UI (no filesystem dropdown) and product filesystem choices 15:30:16 Martix: can you be a bit more specific? 15:30:18 Martix: i don't have any specific news, i know anaconda team is 'working on it' 15:30:32 and some things we could maybe make taskotron testing a long-term goal for 15:30:34 we have no plans to have anything other than virt clients for beaker 15:32:15 tflink: virt could be enough for start 15:32:34 you can do quite a lot of storage stuff in virt, sure. we're a way down the line there, though. 15:33:27 beaker hasn't been the highest priority as of late but dcallagh made some changes to the beaker server that should help workaround the problems we were having 15:33:27 adamw: cmurf on what is Anaconda team exactly working regarding automatic testing? 15:33:41 * tflink needs to follow up on a secondary problem he was seeing, though 15:33:52 Martix: dunno 15:34:15 Martix: i don't know in detail. really they're looking at it from the point of view of CI/unit testing for the installer code, AIUI. 15:34:19 but best talk to them about it 15:35:40 unit testing is one thing, but what takes valuable QA time is blackbox testing, any efforts on improving automation in this area? 15:35:55 in Anaconda 15:37:15 i don't think anyone is working on it specifically 15:37:26 * satellit is there any difference in live (liveinst) and boot.iso anaconda installs and using wireless vs wired connections? 15:37:31 Martix: any suggestions? 15:37:44 it was always a goal for autoqa, i'm assuming it's still a goal for taskotron 15:37:57 in which case the answer is that we need to finish building taskotron first (or, well, finish making it usable at least) 15:38:10 I'm not sure blackbox testing fits well into taskotron 15:38:20 or autoqa, for that matter 15:38:28 oh, sorry, as in actual mystery hardware 15:38:35 ? 15:39:03 grf, sorry, i'm multitasking and missing bits. 15:39:36 Martix: nothing has really changed in the last 2 weeks that you haven't asked 15:40:01 tflink: well, i kinda remembered when we had RATS in autoqa that the plan was to expand that out into a set of different installation paths 15:40:10 though it never happened 15:41:07 that's still something that I'd like to see automated but unless I'm misunderstanding, it's a bit orthagonal to what Martix was asking about 15:41:26 I think we should move on to the next topic 15:41:49 tflink: i'm not entirely sure what martix was asking either :) 15:42:04 kparal: well, the next topic was another subtopic of this one 15:42:16 kind of open-ended, though 15:42:42 i was wondering whether we should try looking at the server/workstation tech specs as a framework for test planning - try and design tests around the specs 15:42:56 makes sense 15:43:06 manual or automated? both? 15:43:09 pretty clearly going to be more stuff there than we have time to test regularly, but we can at least consider drawing up test plans 15:43:19 kparal: both, i guess 15:43:23 hard to argue with "incongruent with the spec" 15:43:29 oh crud....network issues here. I'm here, really.... 15:43:37 * danofsatx-work slinks off to the corner in shame 15:43:49 it can be a good source of what to focus at 15:43:57 and conversely "meets the spec therefore not a bug" etc 15:44:16 danofsatx-work: heh, don't sweat it 15:44:19 And if there's vagueness in the spec, that's cause to update the spec to make it clear. 15:44:32 cmurf: mostly the specs aren't quite that detailed, but sure 15:45:04 i'm thinking stuff like 'if it's listed in the spec, we should aim to have a basic test plan for it' 15:45:15 It's fine for them to not have certain details, but if they're totally vague such that it's not helpful in plotting some direction forward, then the spec isn't a spec. 15:45:36 A spec is "a detailed working description" 15:46:40 * handsome_pirate stumbles in 15:46:47 well, take a look and point up any bits you think are too vague... 15:46:48 Plus it's an "appeal to a higher authority" :-D 15:47:26 adamw: I'm not suggested there are now parts that are vague. I'm just saying it's a good idea to use the specs as the basis for test planning. 15:47:37 cmurf: sure 15:49:13 OK, so in general we're happy with that idea? cool. if people want to look at the specs and draw up draft tests for things we don't have covered yet...:) 15:49:21 that Base matrix has always looked a little small ;) 15:49:40 :) 15:50:27 #topic Open floor 15:51:03 ping jreznik 15:51:04 #info jreznik points out a draft F21 schedule he'd like comments on: https://fedorahosted.org/fesco/ticket/1178#comment:31 15:51:41 looks fairly reasonable to me 15:51:49 gives us plenty of time before alpha (yay) and a reasonable alpha->beta gap 15:51:52 thanks 15:51:58 anyone else? 15:52:07 for ping and for reply 15:52:50 looks reasonable to me also 15:53:08 +1 15:53:20 could you #info it, so I can put it into the ticket? 15:54:08 already did. 15:54:13 or you mean my response? 15:55:52 anything else for open floor, folks? 15:56:16 adamw: response 15:56:51 (or better response for pancakes - next time you'll be in Brno ;-) 15:57:23 #info adamw, cmurf and handsome_pirate thought the schedule looked good - plenty of time before Alpha, but still a decent Alpha->Beta gap 15:58:05 thanks! 15:58:33 welp, let's set the Quantum Fuse then 15:58:39 and maybe even finish within an hour - shocking! 16:00:12 #endmeeting