15:01:54 #startmeeting Fedora QA meeting 15:01:54 Meeting started Mon Jul 1 15:01:54 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:54 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:02:00 #meetingname fedora-qa 15:02:00 The meeting name has been set to 'fedora-qa' 15:02:03 #topic Roll call 15:02:10 * pschindl is here 15:02:14 * tflink is here 15:02:34 hi 15:02:59 * mkrizek is here 15:03:06 * Cerlyn is here 15:03:08 set phasers to FUN 15:03:40 * jreznik is there, would like to leave a bit earlier, so go on, go on! 15:03:54 * brunowolff is here 15:04:32 #topic Fedora 19 retrospective and wrap-up 15:04:56 so, we got F19 tested and out the door with only a one week slip - big congratulations to everyone! 15:05:15 in case anyone missed the ML message: 15:05:22 yep, thanks everyone! 15:05:27 #info I put the retrospective page up recently at https://fedoraproject.org/wiki/Fedora_19_QA_Retrospective 15:05:36 you can add feedback on the *QA* process to that page 15:05:51 adamw, my question is how the sudo switch is missing from the gnome install 15:06:05 Southern_Gentlem: the GNOME team does not want it to be optional. 15:06:13 it's by design for that tool. 15:06:36 that doesn't quite tie in with anaconda views it, but then i don't think we're quite at the point where GNOME gets to define the UI of *the entire distribution* :) 15:06:54 so first boot you have to give admin priviledges to the first user 15:06:56 if GNOME was designing the installer, it wouldn't have a switch either: the first user account would always be an admin. 15:07:03 something wrong with that 15:07:08 if you install GNOME and don't create a user in anaconda, yes. 15:07:09 * dan408\ sips coffee 15:07:49 so let's see, as well as general f19 stuff, some leading questions... 15:08:09 anyone particularly worried about any specific bugs that slipped through the net and have ideas to do anything about them? 15:08:36 other than the mac bug? 15:09:13 no 15:09:15 nvm, already on the list 15:09:15 including that, if we have anything productive to contribute 15:09:28 though personally i'm not hugely concerned about that one 15:09:55 it's good ol' fedora 15:10:12 yeah, I don't have anything to contribute that hasn't already been said 15:10:42 can anyone think of any issues we need to document that aren't documented already? 15:11:01 I'm minorly concerned about what criteria the cloud folks want to add, but that can wait until they're proposed 15:11:54 i'm already seeing bugs 946964/955779 on one of my newly installed F19 boxes. i think it may be a common issue 15:11:57 nothing really, same usual questions popping up in #fedora 15:12:12 grub questions, video drivers, getting up and running 15:12:20 these are pretty much documented 15:12:47 cool 15:12:54 so no big new stuff 15:13:14 no. 15:13:35 i really dont see anything major at the moment 15:13:46 robatino: if you can clarify that one it could certainly be a commonbugs candidate, but I haven't had issues using gdm in KVM virt machines 15:13:56 thanks dan 15:14:10 np 15:14:18 i'm seeing it on bare metal now (on a machine with blacklisted 82865 video) 15:14:32 i'll be investigating more 15:15:19 for the record, i've done most of the obvious commonbugs entries, the ones remaining are mostly ones that aren't entirely clear, but i'll take another pass through the list today 15:15:30 anything more for f19 or shall we move on to the next one? :) 15:15:35 thanks adamw for commonbugs 15:15:54 the next one ? 15:15:58 * dan408\ gets scared 15:16:02 f20! 15:16:06 NO 15:16:09 :( 15:16:45 2 weeks vacation for all 15:16:48 #info Fedora 20 is cancelled, dan408's orders 15:16:59 not cancelled 15:17:00 the official QA dictionary does not know the meaning of the word 'vacation' 15:17:10 I added a note about getting a TC for the minor spins. 15:18:18 alright 15:18:46 brunowolff: that's really up to releng 15:18:50 we don't order them what to build 15:18:55 we just ask for a tC 15:18:56 adamw: vacation? is it somehow related to vaca in spanish? :D 15:19:10 #topic Fedora 20 planning 15:19:19 It affected QA of those spins. 15:19:22 so F19 is SO last week 15:19:28 brunowolff: well, true. 15:19:28 * nirik would like to clearly propose dgilmore's spins critera for the f20 cycle. 15:19:40 nirik: can you expand on what you mean by that? 15:20:29 for tc/rc's we compose all approved spins, but spins must get 2 testers to ack that they passed basic tests. Those that do, are released for that milestone. Those that don't aren't. 15:20:37 for f20 schedule reference - https://fedoraproject.org/wiki/Releases/20/Schedule (see, no earlier than) 15:20:45 if something doesn't get any acks for a cycle, it's dropped. 15:21:13 can these acks be done via email? 15:21:15 * nirik was going to write up something for devel/test/spins 15:21:35 dan408\: good question 15:21:38 whatever folks would prefer, sure. 15:21:40 nirik: well, we planned it in milano with cwickert too (similar, even for the big spins) - so I'm +1 for that 15:21:44 but a wiki might be easier to track 15:21:53 +1 to a wiki on that 15:21:54 we have matrices for desktop spins, we don't print a matrix for every other spin 15:21:55 nirik: or real ticket... 15:22:08 yeah, could get confusing tho... with all the spins... 15:22:10 or matrix... 15:22:16 a page that just had 'does it boot?' for every spin would be easy to write, i guess. 15:22:21 * nirik doesn't care how we track it really. 15:22:32 does anyone see a major problem with the policy in general? 15:22:40 from a QA perspective? doesn't seem like a problem for us, to me 15:22:43 I was thinking: boots, can login, network works? (or any other basic things) 15:22:51 as long as we aren't dropping spins left and right due to lack of acks, im +1 15:23:15 well, I suspect we will, but we will see. 15:23:26 i suspect the *two* tester requirement may be too many 15:23:33 we don't get two testers for KDE sometimes, enver mind anything else 15:23:34 should it be 1/ 15:23:35 ? 15:23:39 1 seems more realistic 15:23:42 hold on 15:23:59 It depends on what you want to test 15:24:02 and for a really basic functionality check, i don't see the need for double testing 15:24:11 anyhow, I am not saying we approve anything. I am just saying I intend to start a clear dialog on this for f20. ;) 15:24:19 cwickert: " I was thinking: boots, can login, network works? (or any other basic things)" 15:24:28 nirik: sure, understood 15:24:40 adamw: my thought was the spin maintainer and one other 15:24:41 most spins are based on other spins, so they should have pretty good coverage already 15:24:53 #info nirik and dgilmore will be proposing a new policy for deciding which spins to ship for F20 15:25:03 like the jam-kde spin inherits testing from KDE 15:25:11 cwickert: the problem is that just adding packages can bust stuff sometimes, we can't assume a pass for the KDE spin means a pass for all KDE-derived spins 15:25:12 dgilmore: spin maintainer + one other makes sense for me 15:25:15 except when it doesn't compose or has other issues. 15:25:23 jreznik: i think sometimes, 'spin maintainer' may be all we can get. 15:25:34 all we then need to test is: 1) it composes, 2) it boots, 3) one can login and 4) all applications run as expected 15:25:44 adamw: i know in the past they have not tested 15:25:50 'all applications run as expected' is a large bear trap 15:25:50 and this should IHNO be done by two people 15:25:54 owner and one more 15:25:58 it takes about three hours to run that test for KDE 15:26:00 +1 cwickert 15:26:02 and it's rarely a 100% pass 15:26:03 even spin owner is better than what we have now. 15:26:06 s/IHNO/IHMO 15:26:16 it takes 3 hours to run that test for kde? 15:26:19 yup 15:26:26 is kde really that slow? 15:26:28 yeah, all applications is a bit crazy... 15:26:31 just launching every damn app in the menus and testing it can do _something_ 15:26:37 or are you talking about KDE + derivative spins? 15:26:39 adamw: I don't expect it to pass 100, I just expect it to be tested 15:26:40 dan408\: no, it just has a lot of apps 15:26:44 dan408\: manually, yes - there are a lot of apps, you have to try to click over, load document 15:26:51 dan408\: Desktop has rather fewer, still takes 1-2hrs though 15:26:55 testing all apps on something like the security spin would be difficult 15:26:57 adamw: why would it take 3 hours to test kde? 15:27:06 jreznik: that's what we have you for 15:27:06 definitely time to talk to vhumba about this one 15:27:12 I'd be fine with that as an optional. 15:27:13 adamw: you mean according to the criteria I just outlined? 15:27:28 well KDE is a "release blocking" desktop 15:27:30 cwickert: "all applications run as expected" 15:27:32 anyhow 15:27:38 we don't need to get bogged down in the details 15:27:41 but for a basic spin you dont need to "launch every app" 15:27:53 yeah, that's kinda what I was saying, i don't think that test is even required 15:27:59 maybe check the key apps for the spin, sure 15:28:03 yes. 15:28:10 so long as people can get in, get online and open a browser is a good start 15:28:17 i.e. can you open a "konsole" or a "terminal" 15:28:21 that the apps work as expected is a plus 15:28:23 * nirik is with dgilmore. We should walk before we run. ;) 15:28:42 okay 15:28:54 i think we are all talking about the same thing here 15:29:00 yeah, i think dan's right 15:29:15 no-one seems to think the idea is a screaming failure, so let's just wait for the formal proposal and we can bikeshed the details then 15:29:23 everyone get your preferred tin of paint ready 15:29:30 adamw: so, how many applications are there in the menu? say 50, starting one takes 10 seconds. that makes it less than 10 minutes. you just fire them up, and see if they start, you don't test each and every menu option 15:30:13 There are other details regarding arch and whether the image is optical, dd'd to usb or lcti'd to usb that should be considered. 15:30:17 cwickert: if you literally just want to see if they run, it might take a bit less. if you want to test at least that you can do some basic operation without it crashing, much longer. 15:30:23 * nirik notes f19 xfce fails on several apps in that test right now. ;) 15:30:36 * cwickert knows 15:30:37 heh 15:30:48 LXDE has two fails 15:31:00 * adamw waves the 'next topic' hammer threateningly 15:31:31 maybe there should be a recompose of the live spins 15:31:43 but i think someone is waving a hammer so i should run 15:31:50 a reduction in live spins 15:32:13 desktops and provide ks for rest 15:33:04 Southern_Gentlem: i don't think there's anything to be gained by arguing about that in a QA meeting. 15:33:07 alright 15:33:18 so other than this proposal, what do we need to do for F20? 15:33:27 uh 15:33:46 nothing, lets ship it! :) 15:33:48 i have a few things lined up: I need to continue revising the final criteria (which got stalled during final validation), and i'd like to go through the whole validation test case set 15:33:49 how are we pracitcally dealing with "features" 15:33:49 * nirik runs 15:33:57 nirik: you have to compose it first 15:34:04 there are no more features. There are now changes. ;) 15:34:17 #info adamw will continue with Final criteria rewrite and look at updating the validation test case set 15:34:25 nirik: right 15:34:39 dan408\: is the question 'how do things change for QA with the new Changes process'? 15:34:44 if so, excellent question 15:34:48 yes 15:34:52 sure 15:35:07 nirik: can you help us with that? 15:35:22 * dan408\ is trying to help with a "change" for 20 15:35:38 I could try. ;) I don't think things change too much aside from that the 'standalone' ones vs system wide might help with what you need to test? 15:36:50 i guess for me it was more of a better understanding, and now i guess it's more for adamw to write criteron for standalone vs systemwide changes? 15:37:13 i don't know if there's some kind of obvious criterion 15:37:19 and i'm not the only one who can write criteria :) 15:37:19 there is 15:37:23 if you think we need one, draft one up 15:37:41 so i guess there should be a decision of whether certain systemwide changes should be "release blocking" or not 15:37:44 the distinction between 'standalone' and 'systemwide' should give us some clues of what to test 15:38:01 fesco is no longer voting on these things anymore 15:38:04 well, what would it mean for a Change to be 'release blocking'? 15:38:04 they are just proposed 15:38:25 adamw: Well, do we plan any more anconda "improvements" for 20? 15:38:49 Gnome 3.10 would be a systemwide change wouldn't it? 15:39:03 * jreznik would be glad so see qa involved in change process - especially what we miss now is the final ON_QA validation step - especially for the bigger system wide ones... 15:39:37 Well what fesco used to vote on, who votes on it now? QA? 15:39:55 as in voting on whether features are 'approved'? 15:39:58 dan408\: FESCo still votes on both 15:40:01 * nirik gets back. what? 15:40:08 thank god 15:40:16 nirik: who's right, jreznik or dan? 15:40:22 just for systemwide - FESCo wants change by change, for selfcontained - in batch 15:40:29 right. 15:40:47 actually I thought we were just approving selfcontained unless someone asked for a vote. 15:40:50 but either way. 15:41:08 if somebody is not ok with selfcontained being selfcontained - it could be raised to fesco to vote on as nirik says 15:41:27 * dan408\ thinks all systemwide changes should be approved by fesco 15:41:32 all of them go to devel list for comment. 15:41:33 i think that's clearly the case 15:41:41 all systemwide ones must be passed by fesco 15:41:47 okay 15:41:53 then self contained can just go to devel list 15:41:58 so we don't need to try and take that function over or anything, dan. but thanks for raising it 15:42:16 right because there's obviously confusion there 15:42:36 dan408\: i think everyone's clear that systemwide Changes are reviewed, though. which is the important thing 15:42:48 so i guess once that's done 15:42:51 we should certainly read the threads on devel@ and point out concerns we have with Changes from a QA perspective, btw 15:42:54 we can get to testing these things 15:42:59 i've been doing that, but it'd be great if others can too 15:43:14 well jreznik needs to send out the list.. 15:43:41 i kind of made a copy for 20 out of panic 15:43:43 adamw: yes, that's one important thing from qa perspective 15:44:03 dan408\: each systemwide Change should show up as its own thread on devel@ 15:44:06 adamw: and that other part is validation of at least of systemwide changes 15:44:18 http://fedoraproject.org/wiki/Releases/20/FeatureList 15:44:22 aka is it really implemented? 15:44:22 there is nothing up there 15:44:29 well, that gets back to the question I asked Dan above 15:44:32 dan408\: it's not used anymore 15:44:42 what does 'validation' of a feature mean exactly? 15:44:47 we should remove/fix that page 15:44:52 jreznik: which is bad imo 15:44:57 that the feature works 100%? that it works at all? that it doesn't cause breakage in anything else? what? 15:44:58 how do we track this now 15:45:29 there will be another page without the word "Feature" in it? 15:45:30 good question dan 15:45:33 dan408\: it's going to be replaced by https://fedoraproject.org/wiki/Releases/20/ChangeSet - being, sorry, only two accepted changes and I have to change my script to generate right list 15:45:46 nirik: yes 15:45:46 nirik: i'd expect the 'FeatureList' location to re-direct to the new list for at least a few releases 15:45:53 why are we confusing the hell out of people? 15:45:56 i don't know about anyone else, but XX/FeatureList is in *my* muscle memory 15:45:58 adamw: of course I can do it 15:45:59 and possibly not in wiki... up to jreznik 15:46:17 adamw: +1 15:46:40 that is why i basically copied the wiki page at the time 15:46:47 http://fedoraproject.org/wiki/Releases/19/FeatureList 15:46:50 this is beautiful 15:46:56 adamw: the idea was to extend the list with more info - not only a list but a place with details - see the ChangeSet template... there should be docs, marketing and I'm more than happy to track QA status there too 15:46:58 redirect to Releases/20/ChangeSet would be fine. 15:47:20 adamw: ok, I redirect from old policy and yeah, I'll do it for a few releases too 15:47:24 so my take on 'validating features' has always been that, well, we validate the release 15:47:27 features are a part of the release 15:47:28 for FeatuesList/ChangeSet 15:47:40 if they break something important, our validation tests should catch that 15:47:52 adamw: release is validated, does it mean some changes are really implemented? no 15:47:54 I guess we will repropose E for f20 as a "self contained" feature 15:48:04 but i'm not sure if that philosophy is water-tight; we might want to reconsider 15:48:06 dan408\: do it please and let me knwo 15:48:21 let's make sure that whatever is listed under the features category is moved to the changes category 15:48:46 jreznik: so, you want us to test that the Changes all actually work? that's another bunch of work for us to do, i guess 15:48:54 #link http://fedoraproject.org/wiki/Features/Enlightenment 15:49:05 adamw: one thing is - we know release is ok, but we are never 100% sure if all changes are ok and you know, we do release announcements (and many times we realized hey, it's actually not there etc.) 15:49:13 #info jreznik would like to see functional validation of Changes for F20 15:49:22 volunteers welcome... 15:49:26 adamw: that's the question if it's doable... 15:49:36 well, we can look at that 15:49:41 for now let's move on as time is getting short 15:49:45 yeah, that sounds like a lot of additional work 15:50:07 jreznik: I guess you just need to check http://fedoraproject.org/wiki/Category:FeatureReadyForWrangler 15:50:09 tflink: it is but it's something we miss now :( 15:50:10 #action adamw to add an item for Change validation discussion to next week's agenda 15:50:20 let's discuss it in more depth next week 15:50:42 dan408\: I'm in touch with owners and we're migrating it from features to changes 15:50:42 #topic Taskbot 15:50:52 jreznik: okay 15:51:14 wanted to check in on the current status of taskbot 15:51:14 #chair tflink 15:51:14 Current chairs: adamw tflink 15:51:24 what's the news, flinkonian one? 15:51:31 not much to say, getting ramped up now that F19 is mostly over 15:52:05 I want to get something in place before F20 starts up 15:52:20 that'd be nice 15:52:30 but most of AutoQA is running on F17, so that may need fixing sooner than later 15:52:36 ah, yes 15:52:57 #info current AutoQA setup is running on F17 which will go EOL soon: we may need to spend time moving it to something newer 15:53:12 the fedora version doesn't really matter much for our current tests 15:53:30 but it seems like bad form to be running on an EOL version 15:53:57 well time to upgrade to 19 or "rawhide" ? 15:54:17 yeah, it's just a lot of work 15:54:28 i'm not sure if it's worth wasting a lot of work for 'bad form' 15:54:39 if there's no actual security issue to running an EOL'ed Fedora for the test machines... 15:54:52 they're all behind a firewall 15:55:02 do they have selinux enabled? :P 15:55:03 everything publicly accessible is running el6 15:55:20 dan408\: not sure I see how that matters 15:55:29 they don't even have public ips 15:55:29 just a joke, i think 15:55:30 kind of a joke 15:55:37 so, i'd consider that question before spending a bunch of time on it 15:55:51 especially if there is a realistic road ahead to taskbot in a reasonably short timeframe and this would delay that 15:55:58 yeah, I was hoping to avoid upgrading if we can 15:56:03 maybe work with infra to see what they think 15:56:19 about the upgrade? 15:56:22 yeah 15:56:28 that's all us 15:56:30 if they can think of an actual practical reason we may need to do it 15:56:32 ah okay., 15:56:58 but it would be worth asking if they can see an issue with the plan 15:57:40 right, more input always valuable 15:57:45 as for taskbot, what were you hoping to get in place? 15:57:53 I 15:58:09 I'd like to get something mostly equivalent to AutoQA soon 15:58:09 reinstalling thing should be pretty easy I would think... but happy to discuss details. 15:58:45 nirik: there are some complications in doing that which lead back to the mess that is the method we use for sysadmin 15:58:46 well, that'd sure be nice if we can do it 15:59:07 is there stuff that people can help out with? I see john dulaney has been trying to help with depcheck testing 15:59:10 tflink: yeah, lets talk in admin later and hash out a plan. 15:59:30 * tflink has another meeting @ the top of the hour 15:59:44 at the moment, not much - there are still way too many variables 15:59:50 okay 15:59:55 * nirik nods. Later is just fine. 16:00:02 we can move the test day topic to next week 16:00:06 ideas for tests, or even better - code 16:00:11 adamw: I'm here 16:00:22 adamw: but I'll be here also next week ;-) 16:00:34 #info we're hoping to have a functional taskbot in place during F20 cycle 16:00:42 Martix_: the date on the special request wasn't super soon, was it? 16:01:03 adamw: what special request? 16:01:09 it isn't, okay. 16:01:11 let's do that next week 16:01:15 ok :-) 16:01:15 #topic open floor 16:01:32 wait, I got the time wrong on the next meeting. oh well not much to say right now, anyways 16:01:45 anyone have anything important we didn't cover? 16:02:16 i think we're good 16:02:29 no blocker meeting follow up today? :) 16:02:38 f20 blockers, sure :) 16:02:44 * adamw found one yesterday 16:02:54 ack 16:03:41 Semi related to QA is that spin-kickstarts has been changed to be easier to build. 16:04:12 awesome, thanks bruno 16:04:24 yup saw your email on that 16:04:28 #info spin-kickstarts package build process has been improved, which may make it less of a pain to do s-k updates 16:06:09 anything else, folks? 16:06:22 * adamw attaches Quantum Fuse to Cat 16:06:41 that brings up bad images 16:07:00 =) 16:07:32 okay, thanks for coming out everyone 16:07:41 #endmeeting