16:00:40 #startmeeting Fedora QA Meeting 16:00:40 Meeting started Mon Feb 28 16:00:40 2011 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:40 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:45 #meetingtname fedora-qa 16:01:13 #topic Roll call 16:01:19 Hey kanarip 16:01:26 hey jlaska 16:01:26 * jskladan is here 16:01:28 * kanarip here 16:01:29 Anyone else ready for a QA checking? 16:01:32 * red_alert 16:01:36 Hey everybody 16:01:39 not really a part of the QA team but regardless ;-) 16:01:41 starting off on the right foot ... "check-in" 16:01:44 * kparal is here 16:01:58 * tflink is present 16:01:59 yo 16:02:02 * Alam in 16:02:16 hey all 16:02:31 * vhumpa here 16:02:57 will get started in 30 seconds 16:03:08 who else we waiting for ... robatino, Viking-Ice? 16:03:27 * robatino here 16:03:34 Hey there 16:03:59 I'll be walking through the agenda posted at https://fedoraproject.org/wiki/QA/Meetings/20110228 16:04:17 * Viking-Ice jumps in 16:04:23 alright, let's get movin 16:04:28 #topic Previous meeting follow-up 16:04:40 #info Viking-Ice - solicit feedback on test@lists.fedoraproject.org to see whether we need to require only bugzilla.redhat.com use during test days 16:05:08 Viking-Ice: any updates on your end ... is this something you'd like to continue tracking? 16:06:17 we can come back to that ... 16:06:19 next up ... 16:06:21 jlaska: it's on the agenda 16:06:36 it is? 16:06:50 but let's just get back to it after I post the list and get feed back from the RFE I mentioned to you 16:06:58 okay 16:07:06 jlaska: as in me continue on that topic 16:07:09 #info jlaska - review blocker bug meeting SOP and add include bug summary by way of #info for each bug 16:07:15 the agenda it's on my agenda ;) 16:07:25 no updates from me, I'll look at this sometime this week 16:07:27 Viking-Ice: okay :) 16:08:03 #topic F-15-Alpha RC2 status 16:08:11 adamw: what's the word? 16:08:44 yo 16:08:49 #link https://fedoraproject.org/wiki/Test_Results:Current_Desktop_Test 16:08:50 erm, bird? 16:08:55 #link https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test 16:08:58 adamw: don't you know about the word? 16:09:11 if the word happened during the weekend, hell no. 16:09:13 #info Accepted Alpha blockers - http://bit.ly/f15-alpha-blocker-accepted 16:09:24 #info Proposed Alpha blockers - http://bit.ly/f15-alpha-blocker-proposed 16:09:36 adamw: can you give an update on how things are looking please? 16:09:40 (outside of the links above) 16:09:52 the highlights ... good/bad/ugly 16:09:58 rc2 fixes the known blockers we were dealing with last week. 16:10:12 given that it was built late friday and i haven't done any real work over the weekend, i don't know much more about it. :) 16:10:24 oh, it also fixed two key (for me) nth bugs 16:10:31 splendid 16:10:34 the firstboot crash on certain key input bug, and the mesa 32-bit bug 16:10:53 * Viking-Ice only encountered 32 bit bug.. 16:11:00 we (we being me, toshio and dgilmore) declared the password-echoed-to-screen bug not a blocker based on the difficulty of reproducing it 16:11:03 #info firstboot crash and mesa 32-bit (aka mutter crashing) resolved in RC2 16:11:33 adamw: *cough* and others *cough* 16:11:34 I agree ... shall we move that off the proposed list? 16:11:41 https://bugzilla.redhat.com/show_bug.cgi?id=678720 16:12:16 oh you mean on the bug not the meeting.. 16:13:05 Viking-Ice: yeah :) 16:13:11 jlaska: sure 16:14:02 done ... good, I like to see 2 empty blocker lists 16:14:13 adamw: so what's the plan between now and the go/no_go meeting? 16:14:19 test test and more testing? 16:14:24 jlaska: i hope so 16:14:27 dgilmore: :) 16:14:35 jlaska: yup. 16:14:37 test and validate. 16:14:43 make sure there is no new blockers 16:14:50 do we have a list of bugs that need VERIFIED? 16:14:53 given that we didn't change anaconda, i would expect rc2 to wind up good. 16:15:10 technically speaking, the only one that needs to be VERIFIED is the gdm bug, i believe 16:15:12 er 16:15:15 keyboard layout bug 16:15:22 since that's the outstanding blocker 16:15:24 okay 16:15:42 note ... thanks to bruno, we have some queries to find UNTESTED blocker bugs 16:16:06 #info List of blocker bugs that were not VERIFIED -- https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting#Review_CLOSED.2C_but_not_VERIFIED_Blocker_Bugs 16:16:31 okay ... if nothing else, we'll move on to the next topic in 30s 16:17:15 thanks all for testing against the test composes and the RC's ... let's keep getting results in so we can have enough data to support the go/no_go decision 16:17:25 #topic AutoQA update 16:17:44 kparal: What's the latest on the autoqa front? 16:18:12 ok, here are the news: 16:18:30 #info Our new_koji_watcher was finally pushed to master. 16:18:34 Most of the work did jskladan. We also had to solve some issues with missing -pending tags in Koji and Bodhi not correctly marking updates. But lmacken pushed fix recently and therefore everything should be pretty ok now. The new watcher watches for both Koji and Bodhi events and fixes a lot of issues. It also sends "batch" events for tests we can to execute on many events at once. 16:19:02 very nice ... good to see that in master. Kudos jskladan 16:19:05 I cleaned up all of the stray builds in the pending tags over the weekend as well 16:19:17 lmacken: I noticed. thanks for that :-) 16:19:47 #info upgradepath was modified to conform to the new batched events type 16:19:54 The patch is now waiting in autoqa-devel. We will execute upgradepath on all updates waiting in -pending tag all over and over again, because we currently don't have any means how to re-schedule a test. When we are able to do that, we will change upgradepath to test only the new updates once again. 16:20:37 kparal: can you remind me on the need for rescheduling? 16:21:08 jlaska: sure. let's say upgradepath is broken because rawhide contains an old package 16:21:11 this is when we have a FAIL result for one update ... and an update to a newer release resolves the issue 16:21:20 then you fix it - push to rawhide most recent version 16:21:29 but the failed update won't be re-tested again 16:21:44 I see, thanks 16:22:11 ok 16:22:15 #info tflink created a new branch 'pytest' containing proof of concept (and even documentation!) of using py.test in autoqa 16:22:32 and fixed the shell script, so it should work now 16:22:41 I'm sorry, what is this documentation you speak of? :D 16:22:58 :) 16:23:11 Great work tflink, testing is largely unknown to me :) 16:23:36 What's the plan for that branch ... is it pending review, and then going into master ... or is more exploration planned? 16:23:36 it seems a little complicated, I'll have to study it more. I'm interested in comparing the approaches with standard unittest library 16:23:44 thanks, I'm just glad not to have to fight tooth and nail to get working on it 16:23:57 I am excited for the most simplest method :-) 16:24:02 +1 :) 16:24:07 I'm planning to do another proof of concept using nose/unittest this week 16:24:10 jlaska: just proof of concept 16:24:43 and update the documentation to reflect that proof of concept and do a comparison between the two tools 16:24:53 nice approach 16:24:57 in anaconda we're using nose 16:25:43 I think that covered the last week 16:26:12 did I forget some{thing,one}? 16:26:22 there's also an interesting helper module called 'mock' that lets you set up fake versions of real objects and dictate their behavior 16:26:58 hmm ... that sounds interesting for simulating bodhi/koji calls 16:27:00 wwoods: tflink introduced dinguses, it's a little confusing but seems interesting concept :) 16:27:01 so you can do unit testing without necessarily needing to set up whole complex environments to run them in 16:27:11 wwoods: yeah, I was using Dingus, which is a similar tool. 16:27:18 tflink: interesting, I'll check that out 16:27:27 anyway, thought that might be worth mentioning. also: hi! 16:28:02 wwoods: thanks, I'd be interested to see what you think of the proofs of concepts once they're done 16:28:54 kparal: how we looking for doing some more pre builds of 0.4.4? 16:29:37 jlaska: well, jskladan has now pushed some patches to depcheck to adjust it for new koji watcher. after that is accepted, I think we can do full build 16:29:55 tflink: sure 16:29:58 excellent ... I'll stay tuned and get ready to build 16:30:12 okay ... anything else to cover for AutoQA? 16:30:27 that's all from me 16:30:37 thanks! 16:31:05 okay ... a brief reminder of upcoming events ... 16:31:10 #topic Upcoming QA events 16:31:14 aaaaa 16:31:26 #info Tuesday, March 1 - l10n/i18n Installation Test Day - https://fedoraproject.org/wiki/Test_Day:2011-03-01_L10n_i18n_Installation 16:31:30 whoops 16:31:32 * ianweller hides 16:31:34 :)) 16:31:34 ianweller: no worries 16:32:16 Igor just recently announced the event ... the wiki looks good and it uses RC2 ... so that's an extra bonus 16:32:31 yup, looking nice 16:32:41 #info Wednesday, March 2 - F-15-Alpha - Go/NoGo meeting - https://fedoraproject.org/wiki/Go_No_Go_Meeting 16:32:46 it would be really good to get lots of participation in the week 16:33:00 Another go/no_go meeting is planned for this Wednesday 16:33:17 hopefully, not a repeat of last week ... but let's keep testing and stay tuned to the blocker lists posted earlier 16:33:34 #info Thursday, March 3 - i18n Desktop Test Day - https://fedoraproject.org/wiki/Test_Day:2011-03-03_I18n_Desktop 16:33:52 Part#2 of this weeks i18n test days will focus on the desktop 16:34:12 same as before ... good looking wiki and very concise test cases available 16:34:39 any other events folks would like to call attention to? 16:34:48 if not ... we'll move on to a topic I added for robatino 16:35:09 next blocker bug meeting? 16:35:16 red_alert: Wednesday 16:35:41 #topic Infrastructure hosting of delta ISO images 16:35:46 robatino: still there? 16:35:50 yes 16:36:06 Hey! So I added this to the agenda after we spoke on the subject last week 16:36:29 to summarize ... this is to find a more appropriate hosting solution for the delta ISOs you generate? 16:36:53 yes - i have space, but it's on crummy infrastructure (not a real HTTP server) 16:36:59 initially, you were thinking about asking for more quota on fedorapeople.org ... and I think this prompted some discussion on whether there might be a more appropriate solution 16:37:33 actually, i'd prefer fedorapeople if possible, since it's set up and has the necessary facilities 16:37:51 have you reached out to infrastructure already on this topic? 16:37:54 but unfortunately i'd like about 25 G which is much larger than the standard 2G quota 16:38:23 robatino: it would be nice to have a solution where others could help you 16:38:40 jlaska: not yet, you suggested i fill out an RFR ( https://fedoraproject.org/wiki/Infrastructure/RFR ) but i need to discuss that with you some more 16:38:49 okay 16:38:57 let's sync up after meeting and we can talk further 16:39:11 if anyone else has ideas or suggestions, feel free to shout 16:39:30 robatino: do you currently have a way to gather download metrics for your delta ISOs? 16:39:57 jlaska: adrive does show number of downloads, but i don't know how reliable it is 16:40:03 okay 16:40:22 re 16:40:32 is this topic on hosting delta images for live images as well? 16:40:40 I'm assuming these images are used frequently during testing, since I do see some regular feedback on the list about them 16:40:49 kanarip: afaik there's no good delta compression, so no 16:41:00 (for live images, i meant) 16:41:21 robatino, right, that's what i wanted to add, it's squashfs that randomizes the bits so much there's no delta 16:41:46 okay, so action here is for robatino & I to continue discussion 16:42:06 robatino: anything else you wanted to add before we move on? 16:42:23 not that i can think of 16:42:28 robatino: okay, thanks 16:42:32 #topic FreeIPA v2 Test Day feedback 16:42:43 This is an open topic from last week 16:42:58 dpal had mentioned hosting another FreeIPAv2 test day 16:43:25 which, I don't have any objections to ... but wasn't sure that the challenges listed would be resolved by hosting another event 16:43:39 anyone else have opinions/suggestions here? 16:43:52 robatino, you need about 25G? 16:43:54 i'll ask 16:44:03 i have the capability to grant it, but want to check before giving that large 16:44:08 #info dpal posted challenges/problems encountered from FreeIPA v2 test day -- https://fedorahosted.org/fedora-qa/ticket/163#comment:10 16:44:19 perhaps breaking it up to more test days specific to each part of freeipa would be better 16:44:45 Viking-Ice: so improve the scope? 16:44:50 s/improve/narrow/ 16:44:58 nb: actually the way that works out is about 10-15G for QA TCs/RCs, about another 10G for Alpha/Beta/Final disos, which would be nice 16:45:04 if possible 16:45:18 jlaska: yup host a directory server test day etc.. 16:45:31 adamw: thoughts? 16:46:13 #info Viking-Ice suggested a more narrow scope for the test day (e.g. only directory server) 16:46:26 if nothing else ... we'll move on and track this in the ticket 16:46:31 jlaska: series of each component test day followed by reusing vm images for a final freeipa2 test day? 16:46:58 Viking-Ice: I like the idea, but that's a lot of test days to manage 16:47:18 so? 16:47:20 :) 16:47:29 test days aren't free unfortunately 16:47:34 someone has to prepare and execute them 16:47:46 and :) 16:47:57 get more people involved 16:48:04 sounds like a great idea 16:48:08 hehe :) 16:48:16 this was one of the comments I had in the ticket 16:48:25 feel free to add your suggestions along those lines 16:48:29 robatino, who all should have access to write? we're putting it on /pub/alt/stage 16:48:41 would be nice to figure out how to drum up more interest in the subject 16:49:01 alright ... moving on .. 16:49:05 #topic Open discussion - 16:49:10 nb: afaik, just me 16:49:24 any topics folks would like to cover? 16:49:58 quick note - i'm sorry not to have followed up on X test week yet 16:50:03 got ambushed by the alpha rcs 16:50:21 another quick one, we need to go through the CommonBugs list and make sure everything gets documented before Alpha hits 16:50:58 Thanks for reminder ... I wasn't planning to visit those after the go/no_go meeting 16:51:27 #info adamw noted that the X test week recap is delayed, but will be coming to test-announce@ soon 16:53:55 bodhi unit test integration is live 16:54:01 #info Before F-15-Alpha is released, we need to document the current list of CommonBugs -- https://fedoraproject.org/wiki/Common_F15_bugs 16:54:07 anything else on the list 16:54:20 kanarip? 16:54:23 lmacken: wahay 16:54:30 here i am 16:54:30 lmacken: also the improved comments on bug reports, I noticed 16:54:36 adamw: indeed! 16:54:42 may i fire away? 16:54:48 lmacken: any example page? 16:55:07 kparal: I haven't seen any updates that have them yet :( 16:55:15 adamw and i briefly discussed two topics during fudcon in tempe, which i wanted to bring under your attention / consideration 16:55:27 #info lmacken announced that test case integration is now live in bodhi 16:55:40 #undo 16:55:40 Removing item from minutes: 16:55:43 #info lmacken announced that wiki test case integration is now live in bodhi 16:55:59 kanarip: okay, what did you have in mind? 16:56:01 one is Continuous Integration, which polls a source code management system for changes and can then build the sources on various platforms, as well as provide test case coverage 16:56:12 ah ... a timely topic :) 16:56:49 i know that many upstream parties just don't have the resources, man-power, and that of course also, fedora project is upstream for a variety of applications 16:57:07 kanarip: autoqa has some support for git post-receive (aka git push) event notification. We use this to trigger some anaconda tests 16:57:10 i think perhaps package maintainers would also be interested, or at least some of them 16:57:35 kanarip: it would be used for fedorahosted projects? or some broader scope? 16:57:46 kanarip: you're talking about providing CI for projects, right? 16:58:05 kparal, i'd think some broader scope actually 16:58:24 tflink, yes, fedora project projects and perhaps also upstream "external" projects 16:58:54 i suppose one or the other party interested in getting CI going on their project could trigger the project to be included in the CI environment 16:59:23 like if QA was interested in project X, then QA would request inclusion of project X, but other stakeholders include package maintainers, developers, and such and so forth 16:59:29 i'm hoping it makes sense ;-) 16:59:46 are there projects included in Fedora that already have tests intended for continuous int. execution? 16:59:49 robatino, can you join #fedora-admin please 17:00:08 jlaska, well, packages for sure; hudson.cyrusimap.org 17:00:11 kanarip: what was the example you showed me at fudcon? is it something public? 17:00:12 who would maintain the wrapper layer between the CI and non fedorahosted projects 17:00:30 many ruby gem projects also have CI, code coverage, and the likes 17:00:36 * jlaska nods 17:01:01 adamw, the example i showed you is not, but it's hudson.ogd.nl where a customer of mine has CI running for a ruby on rails web app 17:01:09 kanarip: "packages" ? 17:01:21 kanarip: any specific packages, or just package testing in general? 17:01:24 jlaska, applications packaged and included with Fedora 17:01:32 kanarip: that's too broad 17:01:48 kanarip: I think we needed to tighten that up first 17:01:49 jlaska: er, i think you're at cross-purposes 17:01:58 jlaska, not the point, you asked whether fedora project internal projects were set up to be used with CI environments 17:02:23 kanarip: hmm, that's a good question too ... but I was thinking of a different topic 17:02:32 jlaska: i don't think kanarip was saying 'we should do this for anything that's a package', he was giving a definition of the word 'packages' as he thought you asked for one :) 17:02:35 to which i responded that i didn't know, but i knew for a fact some packages included with fedora do have themselves set up for unit/functional testing, which could be utilized with a ci environment 17:02:45 adamw: gotcha 17:02:58 kanarip: cool, that's good to know 17:03:02 not in so many words though ;-) 17:03:05 heh :) 17:03:18 I'm certainly not opposed to the idea ... who doesn't want more testing :) 17:03:30 Oh I know, the people who like to club baby seals 17:03:35 anyway, perhaps we all ponder on this one a little while... 17:03:52 the other thing i showed off to adamw is testopia as a test case management suite 17:03:53 so is the proposal to have CI available to fedorahosted projects? or to set it up for them? 17:03:54 are we kind of looking at a way to maybe glue together an example package/project which is set up for CI, and autoqa? 17:04:07 One of the open questions I have are if/how we can intersect this with the goals of autoqa 17:04:18 adamw: bingo 17:04:32 I'm not sure that the CI idea isn't orthagonal to autoqa 17:04:37 right 17:04:39 tflink: right, that was my question 17:04:45 don't get me wrong, I'm all for CI 17:04:50 does it make sense to bring it under autoqa or is it something different 17:05:12 remembering that i'm probably the dumbest lunk in this particular discussion =) 17:05:13 I think we need to better understand the concept/goals that kanarip is discussing 17:05:16 I just tend to think of it as something that is useful for developers instead of the functional integration testing that we tend to do 17:05:53 ci is mostly towards unit testing and code coverage reporting, if you will 17:06:00 we have preliminary support for commit (well git-push) time testing in AutoQA now 17:06:04 kanarip: could you maybe write up a detailed proposal (ideally of something with reasonable bounds that make it not a big, scary, open-ended project at first) for the list, with mockups and stuff? 17:06:09 or is that too much to ask? 17:06:10 that doesn't make it mutually exclusive with functional testing, but it is something different 17:06:35 I'm not articulating incredibly well at the moment, so that isn't helping 17:06:36 yeah ... I think we'll need some vision to start working towards 17:06:37 * adamw trying to think of ways to move the process forward 17:06:41 adamw, i could whip something up, but i can't tell you when it'll be done ;-) 17:06:52 my day only has 24 hours and i'm short another 24 if you will 17:07:09 understood :) 17:07:15 kanarip: yeah, i know what you mean :) 17:07:24 jlaska: maybe we could task someone to try and work with kanarip on this? tflink? 17:07:29 do we have someone with some developing / continuous integration experience perhaps? 17:07:35 * tflink is interested 17:07:39 (sorry, just threw you under the bus there, tim) 17:07:39 awesome 17:07:40 sounds like we have a candidate ;) 17:07:50 kanarip: did you get to meet tim at fudcon btw? 17:07:59 not sure, sorry! 17:08:05 oh well :) 17:08:09 no worries, I'm not sure either :) 17:08:11 kanarip: AutoQA has some *basic* continuous integration support ... but I know tflink understands formal CI better and can help me understand the differences 17:08:27 jlaska, sure, that's cool 17:09:06 i suppose we get a publictest instance up with jenkins and something with autoqa and look how these things fit together exactly, or not at all, and we'll churn out a nice list of bullet points from there 17:09:18 that's one option 17:09:31 should we revisit after you and tflink have had time to discuss? 17:09:34 there's lots of work after the "me like yummie yummie" point either way, so... 17:09:42 yes please 17:09:52 awesome. 17:09:59 cool 17:09:59 sounds like a plan to me, I'm still trying to understand the scope and integration of the proposal 17:10:05 the other thing then, if we still have some time? 17:10:19 #info kanarip asked about continuous integration. tflink volunteered to discuss further and the two would return with some ideas on how to proceed 17:10:32 i showed off testopia to adamw for test case management 17:10:44 well, this will be a quick topic :) 17:10:50 it seems it's pretty much covered with mediawikiwiki fu for the moment... 17:10:59 duct tape, yes :) 17:11:10 but i was wondering whether testopia has some features that you're interested in 17:11:16 yes and no 17:11:25 we tried *many* releases ago to use testopia 17:11:33 noted of course a major blocker in any testopia case would be the fact that it is bugzilla.*redhat.com* ;-) 17:11:35 and had to stop due to licensing issues with the javascript library used 17:11:49 and some other maint. issues with keeping it in sync with bugzilla HEAD (like we do with bugzilla.redhat.com) 17:11:50 i mentioned this to kanarip at the time, but for the meeting, we are already evaluating our needs in a tcms with reference to mediawiki vs. nitrate - http://fedoraproject.org/wiki/Tcms_Comparison 17:11:57 kanarip: https://fedorahosted.org/nitrate 17:12:19 the nitrate project uses the same db schema as testopia but removes the parts that just didn't work for us 17:12:26 my takeaway from discussion with kanarip was along the lines of we should consider other candidates and make it a more general comparison; the way rhe has it set up now, this wouldn't be such a change 17:12:29 notably the unusable UI and the license conflict 17:12:32 jlaska, the reason i'm bringing it up is... i'm using it and i'm shipping packages for it, so the maintainance overhead can be very, very low 17:12:34 but of course the licensing issue with testopia removes it as a candidate 17:12:53 (unless it gets fixed, i guess) 17:13:04 sure ... you can definitely add your feedback to the comparison pages that rhe is maintaining 17:13:14 i'll look into the licensing thing, do we have some reference perhaps? 17:13:19 I'm not overly thrilled about the UI of testopia, it's really really unpleasant 17:13:28 a review request in bugzilla < does anyone know whether it had been udner review? 17:13:40 https://fedoraproject.org/wiki/QA/Testopia_Evaluation 17:13:52 jlaska, agreed, yet again, there's a balance to be sought if not inspiration to be distilled from it ;-) 17:14:00 kanarip: right on 17:14:30 there are other challenges that we'll have with testopia, but we can talk about those later 17:14:34 alright, so that's another quick topic then ;-) 17:14:46 tflink, your fas account name is... tflink? 17:14:54 kanarip: yep 17:15:11 then i know your email address... scary huh? 17:15:23 #info kanarip asked whether testopia is being considered - Will integrate his work with https://fedoraproject.org/wiki/Tcms_Comparison 17:15:27 thanks kanarip 17:15:30 okay ... I've got nothing else 17:15:34 and we are 15 mins over 17:15:39 thanks everyone for your time 17:15:45 as always, I'll send minutes to the list 17:15:46 well, its not like I hide it much :) If you knew my home address, I'd be more worried 17:15:58 #endmeeting