15:00:17 #startmeeting Fedora QA Meeting 15:00:17 Meeting started Mon Aug 30 15:00:17 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:17 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:23 #meetingname fedora-qa 15:00:23 The meeting name has been set to 'fedora-qa' 15:00:38 #topic Waiting for critical mass 15:00:43 * adamw gets critical 15:00:58 * kparal enjoys brand new QA meeting :) 15:01:22 * jskladan tips his hat 15:01:26 adrianr: not sure why that makes me think of an Olivia Newton-John song :) 15:01:29 * vaschenb is here 15:01:33 adamw: ^^^ 15:01:39 kparal: jskladan vaschenb welcome gang 15:02:12 red dwarf: still, you know what they say - 'better to have loved and to have lost than to have listened to a record by Olivia Newton-John'. But then, anything's better than listening to a record by Olivia Newton-John... 15:02:40 kparal: adamw: you both may be spared my grammar and spelling mistakes for half of the meeting today 15:02:44 #chair kparal adamw 15:02:44 Current chairs: adamw jlaska kparal 15:02:52 * wwoods is here 15:02:58 I may need to drop out halfway through the meeting 15:03:01 hey wwoods 15:03:24 I saw robatino and Viking-Ice-Home join earlier ... you guys lurking? 15:03:43 yup 15:03:48 welcome 15:03:49 yes 15:03:54 double welcome :) 15:04:08 okay, I think we all feel critical (or massy) ... let's get started 15:04:18 #topic Previous meeting follow-up 15:04:32 #info adamw and jlaska to propose artwork final release criteria 15:05:00 nothing big to report back yet on this ... I sent out an email to design-team this morning to collect some thoughts 15:05:17 #link http://lists.fedoraproject.org/pipermail/design-team/2010-August/003190.html 15:05:54 Feel free to weigh in, otherwise I'll collect some feedback and we can make a more precise criteria proposal 15:06:15 thanks for handling that 15:06:26 certainly, sorry took so long :) 15:06:33 you might want to add copyright clause 15:06:38 definitely relevant as it came up again in the blocker meeting 15:06:43 displaying correct year :) 15:07:20 Viking-Ice-Home: heh, good point 15:07:26 #info jlaska to publish F-14-Alpha QA retrospective page 15:07:44 the wiki page is available for anyone to record thoughts about the Alpha ... 15:08:02 it's not formatted to my liking yet, but I'll get to that soon 15:08:11 https://fedoraproject.org/wiki/Fedora_14_QA_Retrospective 15:08:16 #link https://fedoraproject.org/wiki/Fedora_14_QA_Retrospective 15:08:19 * jsmith stumbles in, a bit late 15:08:26 jsmith: welcome 15:08:35 jlaska: Thanks :-) 15:08:51 the 2 other action items were autoqa specific, so I'll save those for that topic 15:08:59 anything else to discuss from last week? 15:09:40 alrighty ... moving on 15:09:55 #topic F14 Test Days 15:10:05 if you haven't noticed, test days have started in full swing again 15:10:16 yaaay test days 15:10:26 hoooray! 15:10:26 :) 15:10:36 w00t! 15:10:54 I was hoping to spend just a few minutes rounding the bases for previous and upcoming events 15:11:09 * fenris02 smirks 15:11:11 for those new to test days ... there is a great introduction to the process at https://fedoraproject.org/wiki/QA/SOP_Test_Day_management 15:11:31 #topic F14 Test Days - Aug 26 OpenSCAP 15:12:04 kparal, you and wrabco collaborated in setting up this event. How'd it go last week? 15:12:25 #link https://fedoraproject.org/wiki/Test_Day:2010-08-26_OpenSCAP 15:12:48 #link http://kparal.wordpress.com/2010/08/25/openscap-test-day-thursday-2010-08-26/ 15:12:53 the test day wast quite successful, as opposed to what test matrix might look like :) 15:13:05 sorry i didn't get on the publicity wagon for this one, got it a late mention in fwn but that's all 15:13:21 well, the red fields are just best results from QA perspective, aren't they? :) 15:13:36 quite a lot of problems were identified 15:13:54 it seems so ... a fair number of attendees as well 15:14:08 and they were fixed. the new openscap release, which is just waiting for pushing into updates, should fix all those issues 15:14:27 thats great work 15:14:35 * kparal will send test day recap into list shortly 15:14:50 kparal, does this replace sectool ? 15:15:31 fenris02: I am not sure I can answer that, openscap developers might be a better choice :) I believe it should 15:15:59 to tell the truth, I've never heard of openscap nor sectool before this test day 15:16:10 yay, learning all around :) 15:16:22 but let's hope it got at least some attention and more people learned what it is good for 15:16:57 kparal: nicely done on the setup with wrabco ... look forward to the recap later this week 15:17:11 kparal: feel free to note any good/bad/ugly on the retrospective page 15:17:31 * kparal will try 15:17:41 #topic F14 Test Days - Sep 02 PreUpgrade 15:18:02 Hurry (rhe) isn't here at the moment, but it seems she has a good handle on the setup for this weeks preupgrade event 15:18:08 #link https://fedoraproject.org/wiki/Test_Day:2010-09-02_Preupgrade 15:18:44 as mentioned on the list, the test day will request testing against the command-line interface 15:18:51 we've not previously done much with it 15:19:14 this should be a good event ... nice to shake out/debug any preupgrade issues in advance of the beta 15:19:29 is there any way to determine what percentage of users actually use the cli vs. gui for preupgrade? 15:19:29 robatino: you've done some preupgrades recently, are there any show stoppers we need to address before the event? 15:19:43 i'll get some publicity out for this one as we'll want to have as many people as possible test 15:19:46 fenris02: good question ... I'm not aware of a good method 15:20:30 jlaska: i haven't tested preupgrade for F14 (problems with VMs not working) 15:20:39 I always thought that those that used cli would just run yum upgrade.. 15:20:43 KVM works better but just getting used to it 15:20:55 Viking-Ice-Home: they might, that is however a completely different method for upgrading a system 15:21:16 i wonder how preupgrade would compare now that yum has dist-sync ... 15:21:35 fenris02: I've had enough of your good questions! 15:21:35 * Viking-Ice-Home just keeps home on separate partition and does a fresh install.. 15:21:44 Viking-Ice-Home: +1 to that method :) 15:21:46 jlaska, that's what i'm here for :) 15:22:00 #help i wonder how preupgrade would compare now that yum has dist-sync 15:22:06 #help is there any way to determine what percentage of users actually use the cli vs. gui for preupgrade? 15:22:07 okay 15:22:16 * jlaska readies for next #topic 15:22:19 so preupgrade just preps the system and downloads the pkgs for an anaconda install 15:22:39 so that if there is a significant change that only anaconda can do - then it can be done 15:22:49 if the change is more iterative then yum can do it 15:23:33 #info skvidal clarified the difference between preupgrade and yum upgrade 15:23:36 skvidal, you do not think yum could dist-sync your system from say, f13 to f14 after you updated the -release package? 15:23:59 fenris02: there have been updates to major system pieces before that were impossible to do ON the running system 15:24:05 fenris02: it's an impossible question, it depends on the upgrade 15:24:07 you had to be standing outside of the system to perform the upgrade 15:24:16 the lvm->lvm2 transition as a case in point 15:24:30 point 15:24:33 you couldn't be RUNNING the kernel from the previous release and hope to survive it 15:24:36 or upgrading ext3->ext4, or (someday perhaps) ext4->btrfs 15:24:52 wwoods: istr ext3->ext4 wasn't the end of the world but you had to reboot RIGHT AWAY 15:24:55 there have also been incompatible differences in the rpmdb, if memory serves 15:25:01 yeah 15:25:11 wwoods: yes - most of those required: yum update yum rpm ; yum upgrade 15:25:23 or the metadata switchover, but you could handle those probably by upgrading yum/rpm/etc first 15:25:26 yeah 15:25:29 wwoods: nod 15:25:30 okay, so it seems preupgrade has everything lined up for this week 15:25:31 this seems somewhat ot 15:25:50 good info, but yeah, we can move this outside the meeting 15:25:50 for now, we're testing preupgrade, not debating its philosophical / practical merits =) 15:25:56 it's a tremendously complex problem that anaconda has already solved, and preupgrade takes advantage of that, and therefore, yes, there is still a useful niche for preupgrade 15:26:09 #topic F14 Test Days - Sep 09 Systemd 15:26:21 ooh, this is one of mine 15:26:26 adamw: Viking-Ice-Home: I think you two are the closest to the state of systemd 15:26:37 how's this event shaping up? 15:26:48 i have the test day mostly set up, i would appreciate review of the monster test cases (which are heavily based on notting's systemd 'acceptance criteria') 15:27:03 i may add a few other test cases, i'll check in with lennart on what he thinks so far 15:27:12 ooh, I'd like to review ... I'm in need of improved systemd knowledge :) 15:28:04 I think we play close attention to upgrade case to see if that's ironed out 15:28:08 adamw: post here a link 15:28:16 Viking-Ice-Home: F13 -> F14 upgrades? 15:28:27 yup and systemd 15:28:41 kparal: they're on the page 15:28:54 there was set back with 8.1 that got fixed in 8.3 15:29:04 kparal: https://fedoraproject.org/wiki/QA:Testcase_initialization_basic , https://fedoraproject.org/wiki/QA:Testcase_initialization_tools 15:29:14 adamw: ah, there's no summary page yet? 15:29:26 kparal: a category page? 15:29:35 well, a test day page 15:29:41 #link https://fedoraproject.org/wiki/Test_Day:2010-09-09_Systemd 15:29:41 yes 15:29:45 jlaska linked to it already i thought 15:29:47 oh no :) 15:29:49 thanks 15:29:52 it would just be good to test systemd when preupgrade tests are performed 15:30:10 adamw: it's not available from https://fedoraproject.org/wiki/QA/Fedora_14_test_days 15:30:15 Viking-Ice-Home: okay 15:30:22 kparal: I just updated that page 15:30:28 kparal: oh yeah, i'll fix that 15:30:34 thanks 15:31:24 at one point we were working on a debugging page for systemd ... is that still in progress? 15:31:53 think that's viking's department 15:32:12 adamw: there should be clear instructions how to setup F14 machine, since installing F14 Alpha and updating will break the boot process. to avoid confusion and reporting that as bugs, it should be covered in installation instructions 15:32:46 not a bad idea ... from the list at least, it seems there is significant confusion around what to test, and how to get there 15:32:54 this could help clear that up 15:33:16 kparal: it shouldn't by the time we run the event 15:33:31 in fact it shouldn't now, since updates-testing is enabled by default and 8-3 is in updates-testing 15:33:40 #info Test cases available, will request review shortly 15:33:46 adamw: alright, you're always ahead of me :) 15:34:13 actually, it's gone stable now...https://admin.fedoraproject.org/updates/systemd-8-3.fc14,initscripts-9.17-2.fc14 15:34:49 though /me wonders about that unpush by lennart before the push to stable...better check with him on that 15:35:07 anything else to consider to prepare for this event? 15:35:21 not that i can think of 15:35:25 feel free to #info,#action,#{favorite-tag} as needed 15:35:40 cool ... I'm sure this will be another well attended event given the buzz :) 15:36:00 #topic Proventester (metrics and sponsor upgrade) 15:36:09 I wanted to raise this for brief discussion 15:36:49 I know this has come up previously, but I received a request from a proventester to upgrade them to a sponsor 15:37:05 Wanted to ask if we've done this before, what we'd need to consider for this etc... 15:37:13 jlaska: Ah yes I need to finish that one up 15:37:22 systemd debugging page that is 15:37:22 and it seemed like a good flow into the previous topic of gathering proventester metrics 15:37:34 what I wrote in the policy was just "Any proven tester can become a mentor. Simply let any existing mentor or group administrator - those listed as administrator or sponsor in the group member list - know you would like to become a mentor, and they will upgrade you to sponsor level, which will allow you to accept applications to the group. " 15:37:36 Viking-Ice-Home: cool thanks, I'm happy to review if you need input 15:37:43 so basically the process to become a mentor is...ask to be a mentor 15:38:00 adamw: this is too simple! 15:38:01 going with the priniciple of least resistance we agreed on for the whole proven tester process 15:38:03 no trac ticket? 15:38:13 * adamw hates bureaucracy 15:38:13 we need more obfuscation in the way :) 15:38:37 a trac ticket would give a record of the process i guess. i just made it up as i was going along. 15:38:39 fenris02: ah, yeah I can recommend the trac ticket route just for transparency 15:38:49 but for now according to what we have written down, you can go ahead and make 'em a mentor, jlaska 15:39:04 if someone wants to revise the policy to say 'file a trac ticket' i'm fine with that 15:39:11 jlaska, CYAP 15:40:58 okay, thanks 15:41:09 anything else we want to talk through on the metrics side 15:41:09 on metrics, um, not done anything yet. 15:41:20 adamw: you reached out to lmacken, I forget what the plan is here 15:41:21 it should probably go on my todo list 15:41:30 this is something included in an upcoming bodhi release? 15:41:44 luke asked me to file a trac ticket with a list of desired metrics 15:41:49 i think i did that...lemme see 15:42:00 yeah, https://fedorahosted.org/bodhi/ticket/456 15:42:15 seems we're waiting on luke there, we could give him a poke 15:42:25 cool, I forgot about the ticket, thanks adamw 15:43:35 okay, we can check-in with lmacken later 15:43:55 okay, I need to move my focus to another meeting :( 15:43:57 #action adamw contact lmacken about proventester metrics https://fedorahosted.org/bodhi/ticket/456 15:44:09 kparal or adamw can you guys push throug the remaining autoqa and open discussion topics? 15:44:31 kparal: do you want to take autoqa? it's your area 15:44:38 #topic AutoQA 15:44:40 adamw: ok 15:44:42 kparal: wwoods: go for it 15:45:06 whoosh 15:45:20 is it a bird? is it a plane? no, it's...autoqa! 15:45:23 last week there were some big documentation updates 15:45:36 the main culprit is jskladan, so I'll let him talk about it 15:45:42 praise yourself, jskladan! :) 15:45:52 hooray! 15:46:04 okay 15:46:18 last week, i finished the "how to write autoqa tests" wiki draft 15:46:29 autoqa should be a massive WIN for fedora :) 15:46:40 jskladan: got link 15:46:44 https://fedoraproject.org/wiki/User:Jskladan/Sandbox:Writing_autoqa_tests 15:46:51 #link https://fedoraproject.org/wiki/User:Jskladan/Sandbox:Writing_autoqa_tests 15:47:13 so, feel free to comment/edit... 15:47:32 nice stuff 15:47:41 and at the end of this week, i'll replace the "official" page with the content of mine sandbox 15:47:57 might be worth mentioning to the docs team? see if they have any special skills to improve it? 15:48:17 adamw: great idea? could you throw any names? 15:48:29 (so i can abuse their knowledge and creativity :-D) 15:48:34 I think looks good 15:48:38 #info jskladan updated AutoQA documentation 15:48:41 #info jskladan will replace the official page with the content of his sandbox at the end of the week 15:48:44 jskladan: mailing list - docs@lists.fp.o 15:48:53 * Viking-Ice-Home sneaks in 'it' 15:48:56 jskladan: or irc #fedora-docs i think 15:49:08 ok, will try, thx adamw 15:49:18 other than that - i have nothing 15:49:35 i started to dig into the mash 15:49:40 but that thing is crazy :-D 15:49:50 yeah, mash is kind of.. black magic 15:49:54 but it's black magic we need 15:50:15 so that's a nice segue to depcheck! 15:50:19 i was thinking - can we (maybe) add/change tags on packages in koji? 15:50:45 you know - add a custom (unique) tag on the one package, create repo from that that tag 15:50:52 I updated the autoqa depcheck tickets to give a bit more information - including the "make it use mash" ticket 15:50:53 and then remove the respective tag? :) 15:51:14 that's https://fedorahosted.org/autoqa/milestone/Package%20Update%20Acceptance%20Test%20Plan%20-%20depcheck 15:51:28 and the ticket: https://fedorahosted.org/autoqa/ticket/201 15:51:49 #info wwoods added comments to depcheck tickets to explain more about mash support 15:52:02 jskladan: yes, we could do something like that - but remember that we need to run depcheck using *all* the packages that have requested moving 15:52:05 not just the latest one 15:52:37 sure, but that could be arranged :) (you know, tagging one package, or several packages... same thing...) 15:52:42 so the simplest process would be: when a maintainer requests an update get moved to stable/testing, the builds get a new tag (like -blah-pending) 15:53:06 then when we run the test, we use that tag to get ourselves a proper multilib repo of proposed packages 15:53:14 I seem to remember Oxf13 made tags for this purpose 15:53:24 but bodhi doesn't tag packages like that 15:53:44 so, summary here - you need to work with mash for the depcheck tests? 15:53:51 so either we can a) make the autoqa bodhi watcher handle the tagging, or b) figure out a way to run mash without needing a tag 15:53:51 so, maybe the "post koji build" watcher could fire some "tag this package" routine? 15:53:54 adamw: yes. 15:53:54 and you're working on how to do that 15:53:58 okay 15:54:08 or, at least, we need the multilib-solving bits of mash 15:54:17 we got 6 minutes left so maybe the practical aspects can go outside of the meeting :) 15:54:21 ok ok 15:54:23 * jskladan out 15:54:28 meetings aren't for practical stuff, everyone should know that! 15:54:30 ;) 15:54:39 * jskladan lurks into the shadows :) 15:54:44 wwoods: you could declare multilib forbidden and never touch it again, that would also work :) 15:54:47 * skvidal kids 15:54:56 heh. the point I'm making is a) we require this for proper multilib testing, and b) it's kind of tricky and we're still trying to determine the best way to accomplish that 15:55:06 yes, we could also just decide we aren't going to test multilib 15:55:20 and/or have depcheck 0.1 just.. not handle multilib 15:55:28 #info depcheck tests need to use mash to handle multilib cases, team is investigating how best to do this 15:55:45 #info possibility to release initial set of depcheck tests without multilib handling 15:55:52 and work on handling multilib/mash in parallel with setting up depcheck to handle karma etc 15:56:12 wwoods: probably unwise 15:56:17 once you get it mostly working 15:56:28 no one will want to disturb it only to make the whole world break with multilib 15:56:32 at this point I'm thinking about the latter, but I *REALLY DO NOT* want depcheck to be active and enforcing policy until we're really sure it's doing everything right 15:56:39 skvidal: yeah, exactly 15:56:54 once it's in Production nobody will want to change it, so it can't go into Production until it's feature-complete 15:57:20 Google solution - endless beta! 15:57:33 hah yes. 15:57:58 * kparal notes there is one more update from vaschenb about upgradepath 15:58:03 yeah, let's get to that 15:58:08 vaschenb: go! 15:58:10 onward! 15:59:23 i updated upgradepath, the new test... now it has improved output and is automaticaly started with post-bodhi-update hook... 15:59:48 that means it's now included in the set of tests run on always bodhi update 16:00:03 always->every 16:00:49 and that reminds me one more thing, I asked jlaska whether we should deploy a new version of autoqa to our production server 16:01:02 #info vaschenb updated upgradepath - now it has improved output and is automaticaly started with post-bodhi-update hook 16:01:19 but I guess jlaska is busy right now 16:01:30 wwoods: jskladan: what do you think? 16:01:47 I think the current autoqa version is quite outdated 16:02:01 If memory serves there are some big changes and bugfixes etc. that we'll want on the production systems soon 16:02:09 kparal: I'm happy to build and deploy as needed 16:02:18 I can do a autoqa-0.4-pre1 or something? 16:02:49 it will take some time until depcheck is ready, right? I think now it quite a good time to make a snapshot 16:02:58 jlaska: cool :) i'd like to know, if my puppy can survive in the real world of production servers :) 16:03:10 agreed - it'd be good to get some testing on the new code before we start adding new tests on top of it 16:03:19 if need be we can take this to the list and review what (if anything) we need to fix/finish before pushing new code 16:03:31 but I think kparal's right and this is an opportune time 16:04:44 I feel like it's ready for a snapshot - if there are no concerns let's just do it 16:04:48 #info autoqa team agrees now is a good time to make a new autoqa release and deploy it 16:04:53 * jskladan +1 16:04:57 otherwise, send send mail to the list and we'll talk about what we need 16:05:22 anyone have any concerns about trying to push the new autoqa code live? 16:05:46 * wwoods assumes silence == "let's do it!" 16:05:53 so.. let's do it 16:05:55 #action jlaska build and deploy new autoqa release 16:06:15 I'm just going to build from master ... unless there are specific things you guys want me to wait for 16:06:18 * kparal hopes he's not too daring to assign action items like that :) 16:06:27 kparal: assign away! :) 16:06:39 yes, from master, that should be fine 16:07:17 I think that's all from autoqa for today, don't you think? 16:08:08 okay then 16:08:22 #topic open discussion 16:08:27 any other business? 16:09:29 otherwise, closing meeting in 3... 16:09:38 2.. 16:09:47 1... 16:09:57 thanks, everyone! 16:10:01 #endmeeting