16:00:05 <adamw> #startmeeting Fedora QA meeting
16:00:05 <zodbot> Meeting started Mon Feb 18 16:00:05 2013 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:05 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:09 <adamw> #meetingname fedora-qa
16:00:09 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:11 <adamw> #topic roll call
16:00:16 <adamw> morning folks, who's around?
16:00:28 * kparal waves
16:00:29 * satellit listening
16:00:34 * mkrizek is here
16:01:06 * nirik is lurking
16:01:52 * pschindl is here
16:02:08 * tflink is here
16:02:36 <adamw> hey, the gang's all here, now- OH NO WATCH OUT FOR THAT METEOR DRIVEN BY A RAPTOR
16:03:53 <kparal> jskladan will survive and free us from the raptor dictatorship
16:03:54 <adamw> wow, tough crowd.
16:03:59 <adamw> heh
16:06:08 * jskladan lurks
16:06:17 * adamw waves from under meteor
16:06:49 <adamw> hum, we don't seem to have a viking-ice yet
16:07:21 <adamw> tflink: do you know what it was he wanted to discuss about the review process?
16:07:51 <kparal> oh no, we're one viking short
16:09:33 <adamw> if no-one knows what it was he wanted to talk about, we'll skip that item
16:09:35 <tflink> adamw: wasn't it on the agenda for last week
16:10:00 <tflink> nvm
16:10:32 <tflink> oh, it was about the changes we kept making in F18
16:10:48 <adamw> oh, the "QA:TestCase" topic from 0128?
16:10:49 <tflink> keeping a static IRC channel, capping meetings @ 3hrs etc.
16:10:54 <adamw> oh, i see.
16:11:06 <adamw> well, let's do the other topic first
16:11:19 <adamw> #topic Automatic blocker proposal
16:11:41 <adamw> seems like most of the feedback on the 'automatic blocker' idea is +ve, i'll adjust it to incorporate andre's suggestions, any other thoughts?
16:12:01 <adamw> #info the proposal is https://lists.fedoraproject.org/pipermail/test/2013-February/113840.html
16:12:05 <tflink> it seems like a good idea to me
16:12:22 <tflink> but it will increase the average time that we spend on bugs in meetings though :)
16:12:32 <kparal> tflink: will it?
16:12:35 <tflink> since we won't have the really easy ones bringing the average down
16:12:40 <kparal> ah
16:12:44 <kparal> the _average_ time
16:12:53 <kparal> yes, bad for statistics! nack!
16:12:58 <kparal> :)
16:13:13 <adamw> heh
16:13:22 <adamw> lies, damn lies, and tflink statistics
16:13:27 <robatino> will there be automatic freeze exceptions as well (for oversized non-blocking desktops, for example)?
16:13:39 <adamw> robatino: i didn't reply to that mail yet, but it seems reasonable
16:13:57 <adamw> i'll try and come up with a new draft soon; i might emphasize the rules a bit harder too
16:14:10 <adamw> so no-one can claim they misread it and just start slapping acceptedblocker on everything they propose
16:15:03 <tflink> yeah, hopefully this won't be abused
16:15:11 <tflink> but we won't know until we try
16:16:02 <adamw> ah, the viking's here
16:16:13 <adamw> Viking-Ice: we're on the 'automatic blocker' proposal - any further thoughts on that?
16:16:32 <Viking-Ice> nope I agree to it fully
16:17:12 <Viking-Ice> ( I needlessly  worried a bit about that gray area )(
16:18:11 <adamw> cool
16:18:19 <adamw> ok, i'll send out a second draft with andre's suggestions soon then
16:18:31 <adamw> #info group generally supports the automatic blocker proposal
16:18:55 <adamw> #action adamw to write a second draft with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list
16:19:32 <adamw> #topic Blocker review process
16:19:51 <adamw> Viking-Ice: we held this one in case you showed up - so, you said you wanted some discussion about this?
16:21:11 <Viking-Ice> nothing comes to mind at the moment
16:21:38 <Viking-Ice> so I got nothing new to add atleast
16:22:01 * adamw checks log
16:22:26 <adamw> Viking-Ice 	well what did we learn about the blocker bug meetings
16:22:29 <adamw> Viking-Ice 	well we should set a fixed channel and keep with the 3hour max limit
16:22:37 <adamw> tflink 	#info discussion around the blocker review process for F19 would be wise before we get into testing
16:22:41 <adamw> okay, that's where we were coming from.
16:23:02 <adamw> so i guess this is about whether we want to formalize any of the f18 changes to the blocker bug meeting process
16:24:01 <Viking-Ice> I think the 3 hour limit turn out working well but still I doubt ( or let's say I hope )  that we wont be experiencing that again this release cycle
16:24:21 <tflink> we can hope :)
16:24:22 <adamw> we all hope so :)
16:24:29 <adamw> but it does seem like a reasonable rule indeed
16:24:54 <tflink> but I suspect that it's going to keep happening every once in a while until/if we redo the process
16:25:00 <tflink> but that's not happening for F19
16:25:08 <Viking-Ice> and perhaps we should introduce new channel dedicated just for this ( not qa as some people wanted and not bugzappers and not meeting )
16:26:19 <Viking-Ice> I also think it's better not to do blocker bug meetings in the midst of qa meetings or atleast I think it's better to just end the qa meeting and move to another channel
16:26:47 <adamw> we did that once in the last cycle and it worked out fine
16:26:55 <adamw> so it's a decent idea
16:27:04 <tflink> yeah, I don't have any objections
16:27:06 <adamw> it does get a bit messy having to look in qa meeting logs for blocker review
16:27:21 <adamw> i can draft up a few changes to the sop
16:27:35 <tflink> not sure about the dedicated channel, though unless we re-purpose #fedora-bugzappers
16:27:39 <adamw> eh
16:27:42 <adamw> i don't mind it
16:27:46 <Viking-Ice> the reason I personally favor moving/using qa channel is likely hood of more participation
16:27:48 <adamw> not like channels cost anything
16:28:06 <Viking-Ice> tflink, we cant kill bugzappers if we continue to use it
16:28:06 <tflink> it's just one more channel to join and keep an eye on :)
16:28:07 <nirik> channels actually do cost. ;)
16:28:08 <adamw> i can see viking's argument that using -bugzappers is kinda weird
16:28:20 <adamw> the only reason to use -bugzappers any more is for these meetings though
16:28:20 <nirik> they cost in attention of people...
16:28:31 <adamw> so the net cost of a new channel is 0, as if we used one, everyone could quit -bugzappers...
16:28:40 <tflink> that'd work for me
16:28:48 <tflink> I don't much care about what the channel is named
16:29:08 <tflink> but using a dedicated channel does open some interesting possibilities with irc bots in the future
16:29:17 <adamw> Viking-Ice: i think we convinced him ;)
16:30:12 <Viking-Ice> everyone is familiar and usually on the qa channel ( devs/qa community members ) but we might be interrupted like happened that one time if we use it in the midst of the meeting
16:30:16 <adamw> okay, so how about this, i'll draft sop changes for all the above ideas and we can kick it around further on list
16:30:20 <adamw> yeah, that's the problem with using -qa
16:30:24 <adamw> it's a pretty active channel
16:31:39 <Viking-Ice> I assume we want as much activity on that channel
16:32:08 <Viking-Ice> ( which usually means more vibrant and active community )
16:32:10 * nirik is happy with another channel as long as we kill bugzappers. net 0 is good.
16:32:21 <adamw> okay.
16:32:43 <adamw> Viking-Ice: sure, we want -qa to be active, but as you said, it gets awkward if we're having a two-hour blocker meeting and someone shows up wanting to chat about something else.
16:33:36 <tflink> yeah, agreed that #fedora-qa is not the right place for review meetings
16:33:40 <Viking-Ice> just throw it on the test list new channel any suggestion for the name of that channel and or use the qa channel and see how the community reacts/wants it
16:33:46 <adamw> sounds good.
16:34:15 <tflink> it sounded like a good idea when it was first proposed but in reality, it caused more problems than it solved :-/
16:34:21 <adamw> #action adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion
16:36:08 <adamw> okay then
16:36:14 <adamw> looks like that's all we had on the agenda, so...
16:36:16 <adamw> #topic open floor
16:36:51 <satellit> anymore koji builds to test f19?
16:37:17 <adamw> koji builds?
16:37:31 <satellit> lives to test
16:37:38 <Viking-Ice> I've been wondering a bit about that do we really need iso files ?
16:37:52 <Viking-Ice> ( other then alpha beta final )
16:37:59 <Viking-Ice> as in nightly's
16:38:01 <nirik> satellit: I have been holding off doing them while the mass rebuild is running. Should resume tomorrow or so.
16:38:04 <adamw> they're useful, sure.
16:38:21 <Viking-Ice> aren't we usually using them only to test anaconda?
16:38:26 <satellit> I rely on the .iso's for soas
16:38:28 <adamw> in f18 cycle we didn't use them a lot as we were making TCs almost constantly
16:38:44 <adamw> but in previous cycles they've gotten a decent amount of use. not that weird to ask someone to check something with a nightly.
16:39:12 <kparal> also tflink's composes lowered our usage of nightlies
16:39:51 <Viking-Ice> do we have download stats on the iso's
16:40:44 <adamw> not sure koji tracks that...nirik?
16:41:09 <adamw> kparal: we're aiming to do fewer smoke builds for f19, to save tflink all the work.
16:41:12 <nirik> I don't know that it does off hand...
16:41:16 <nirik> there's probibly http logs.
16:41:23 * nirik could look if you like.
16:41:49 <Viking-Ice> I've briefly been touching/pondering the idea if we somehow can use Colin Walters OStree to our advantage (  https://live.gnome.org/OSTree )
16:42:29 <adamw> i remember reading his blog post on it and thinking 'hmm, that's interesting', but i didn't really have any concrete ideas
16:43:16 <nirik> so, httpd logs are kept for iso downloads. What info from there would you find useful?
16:43:57 <adamw> i think viking was curious about how much the nightlies are downloaded?
16:44:07 <nirik> 4569 downloads in 2012-12
16:44:11 <Viking-Ice> adamw, yeah that's where I'm at came across it looks interesting wondering if we can take some kind of advantage of it but nothing concrete yet
16:44:29 <Viking-Ice> nirik, each release?
16:44:31 <nirik> 5117 in 2013-01
16:44:32 <Viking-Ice> or total
16:44:34 <nirik> total
16:44:41 <nirik> any ".iso " download
16:45:10 <nirik> nightlys are only kept for a week or so tho.
16:45:43 <Viking-Ice> I'm just trying to asses the benefit of using it
16:45:47 <Viking-Ice> vs overhead
16:45:53 <adamw> the overhead's pretty tiny
16:45:57 <adamw> i think it's just nirik firing a script
16:46:05 <nirik> yep.
16:46:06 <adamw> if it doesn't build, we don't try and fix it
16:46:10 <nirik> and we keep wanting to automate it.
16:46:21 <Viking-Ice> makes sense
16:46:26 <nirik> it's useful also for spins folks to test if they ever do
16:46:36 <satellit> +1
16:47:26 <Viking-Ice> technically gnome users should be testing the gnome spin as well while we try to focus our energy on the core function
16:47:47 <Viking-Ice> but yeah
16:48:46 <Viking-Ice> how much testing did other then the *DE spin get
16:49:17 <Viking-Ice> I think those might be getting little to no testing even from their maintainers
16:49:19 <nirik> I don't think there's any reasonable way to quantify that. ;)
16:49:27 <adamw> in f18 not a huge lot, for f15->f17 i tried to get decent amount of testing for the non-blocking spins
16:49:28 * nirik did in fact test the Xfce spin a number of times.
16:49:40 <adamw> we at least made sure the whole desktop matrix was done once or twice at each milestone
16:49:57 <adamw> for xfce and lxde
16:50:00 <adamw> satellit tests sugar quite a lot
16:50:40 <Viking-Ice> I'm not worried about the *DE spins ( and sugar ;) )  they all have active communities it's the other ones that concern me
16:50:50 <satellit> I also do VirtualBox installs from spins to test yum installs of other DE's with sugar
16:51:19 <adamw> outside of the desktops and sugar, hell if i know.
16:51:26 <adamw> you may well be right that they don't get much of a look.
16:51:29 <Viking-Ice> I'm wondering if we should not come up with a test matrix for those that the spin maintainers have to walk through and "pass" before release
16:51:52 * nirik nods. Suggested as much to the spins list a while back.
16:51:58 <adamw> i don't mind the idea in theory, as it does kinda suck when we ship stuff that's utterly borked
16:52:07 <adamw> even if it's a spin we explicitly don't support
16:52:26 <adamw> maybe you two could get together and re-propose it to spins?
16:52:50 <Viking-Ice> what's releng take on something like that
16:53:22 <Viking-Ice> ( anything we might handout at various events needs to be thoroughly tested  )
16:53:35 <nirik> the spins setup is disfunctional, but attempts to fix it haven't met with anything concrete.
16:53:50 <nirik> cwickert would be the one to involve in those discussions
16:54:18 <cwickert> ?
16:54:36 <nirik> cwickert: spins process... didn't go so well last cycle. ;(
16:54:44 <cwickert> yes, I know
16:55:12 <Viking-Ice> cwickert, to bring you up to speed qa/releng requesting test matrix spin has to pass before being released
16:55:13 <cwickert> but I'm afraid it will become worse when we kill the spins
16:55:16 <adamw> i don't think we hand out anything but the multi-install and multi-desktop
16:55:22 <adamw> and the regular install / desktop of course
16:55:56 <cwickert> Viking-Ice: that means what exactly?
16:56:00 <nirik> I was thinking a 2 person checkoff of a test matrix for each spin we want to promote on spins.fedoraproject.org. The rest can exist, just in a corner of alt.
16:56:42 <Viking-Ice> cwickert, test matrix spin maintainers have to walk through which ensures atleast no surprises
16:56:48 <Viking-Ice> for their spins
16:56:51 <adamw> this is the proposal
16:57:04 <cwickert> Viking-Ice: argh
16:57:38 <Viking-Ice> cwickert, the *de spins are not much worries since those have active community's it's the other spins
16:57:45 <cwickert> Viking-Ice: please consider me as an idiot who doesn't have a clue what a "test matrix spin maintainer" is
16:57:58 <adamw> cwickert: there should've been some punctuation or grammar in there :)
16:58:10 <Viking-Ice> uhum yes
16:58:24 <cwickert> I know what a test is, I know what a test matrix is, I know what a maintainer is
16:58:27 <adamw> the idea is that there would be *a* test matrix (basically just a test plan) that spin maintainers have to run through to have their spin 'approved' or whatever for a release, just a very basic 'does it boot?' smoke test
16:58:34 <nirik> I think we are talking about "it boots, selinux is enforcing and works, it lets you login, etc"
16:58:36 <cwickert> but who is supposed to maintain the test matrix for a spin?
16:58:58 <adamw> cwickert: it'd be a generic one i think
16:59:00 <cwickert> can there be specific tests for a spin?
16:59:06 <cwickert> who is to maintain them and so on
16:59:17 <cwickert> there is tons of questions
16:59:19 <adamw> i think this idea would just be a very basic generic 'smoke test'
16:59:32 <adamw> spin-specific tests are possible but would be a different thing
16:59:43 <Viking-Ice> qa would maintain the matrix I suppose we already have criteria for "core" the rest is just packages on top of that
16:59:44 <adamw> we actually already have one such matrix for the security lab spin (though no-one ever runs it)
16:59:45 <cwickert> ok, I'm sorry, I need to stop here, FAMSCo meeting
16:59:52 <cwickert> but we DO need to talk about this
16:59:54 <adamw> anyway, it seems like a decent idea
17:00:06 * nirik nods.
17:00:12 <adamw> #action viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert
17:00:12 <cwickert> I know the spins went badly
17:00:29 <nirik> yeah, I don't think anyone disagrees... just how we improve them. ;)
17:00:32 <cwickert> but on the other hand I am very frustrated about getting little or no feedback from QA about my requests
17:00:45 <adamw> sorry, which requests?
17:01:11 <Viking-Ice> yeah I missed those to
17:01:32 <cwickert> adamw: changelogs in the announcements for the differenc milestones, better browsability in the wiki, meaningful renaming of the tracker bugs
17:01:51 <adamw> oh, those
17:02:19 <cwickert> are you coming to devconf?
17:02:27 <adamw> 1) i talked to andre about that one and we edited the text of the announcements somewhat to make it clearer that the 'changelog' is in the trac ticket
17:02:36 <adamw> 2) unfortunately didn't get to that one yet
17:02:39 <adamw> 3) we did that
17:02:44 <cwickert> I'd appreciate if we can discuss some things
17:02:45 * Viking-Ice still lost...
17:02:49 <adamw> cwickert: nope
17:03:01 <Viking-Ice> I will be there kparal as well
17:03:05 <adamw> Viking-Ice: these are requests from some time back, i don't recall exactly what the forum was but i recall the discussion now
17:03:19 <adamw> cwickert: right, you can talk to viking and kparal there (probably also jskladan)
17:03:33 <Viking-Ice> perhaps this should end up in our trac instance
17:03:34 <adamw> cwickert: did you miss the tracker bug renaming thing? cos that was a whole thing a few weeks back.
17:03:37 <kparal> in a pub :)
17:03:40 <adamw> i think it may well be there
17:03:52 <Viking-Ice> kparal, with rotten shark bits ;)
17:03:53 <adamw> i'm pretty sure i filed tickets at the time
17:04:13 <kparal> Viking-Ice: that's not really a czech speciality
17:04:19 <adamw> https://fedorahosted.org/fedora-qa/ticket/273 is the ticket for the 'browsability' thing
17:05:20 <adamw> https://fedorahosted.org/fedora-qa/ticket/272 is for the 'changelog' thing...it's not that we didn't give you any feedback, really, but andre didn't entirely agree with the proposal...
17:05:21 <Viking-Ice> 307 for some of it?
17:05:44 <adamw> that wasn't part of cwickert's request, no. but i do need to finish that up. sigh
17:05:54 <adamw> so much stuff to do
17:06:07 <adamw> anyhoo, we're a bit over time
17:06:07 <Viking-Ice> kparal, I will be bringing a box of bits for people to try as requested ;)
17:06:11 <Viking-Ice> yup
17:06:16 <adamw> so let's wrap up
17:06:21 <cwickert> adamw: 3) we did that?
17:06:28 <adamw> cwickert: the tracker bug renaming,.
17:06:43 <cwickert> adamw: I don't think so
17:06:51 <cwickert> what are the names now?
17:07:12 <adamw> cwickert: https://lists.fedoraproject.org/pipermail/test/2013-January/113405.html
17:07:22 <kparal> Viking-Ice: let's hope it's not an attempt to wipe out Brno's Red Hat office :-)
17:07:24 <cwickert> I am searching the wiki for 5 minutes now for the NTH have bugs :(
17:07:32 <adamw> cwickert: they're called FreezeException now
17:07:41 <adamw> cwickert: and they're always listed at https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers
17:08:36 <adamw> which is linked from https://fedoraproject.org/wiki/Blocker_Bug_FAQ and https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process
17:09:53 * adamw sets fuse
17:10:16 * cwickert needs to bail out for the FAmSCo meeting
17:11:03 <adamw> cwickert: let us know what you think about the new names
17:12:04 <cwickert> adamw: I made a proposal, so I probably prefer what I proposed, right?
17:12:21 <adamw> well i'd *hope* not everyone thinks that way :)
17:12:34 <adamw> i had a proposal too, and so did tflink, but we both prefer the final scheme
17:12:34 <cwickert> I mean, the new names are better than the old ones, but still I consider mine better :P
17:12:43 <adamw> anyhoo
17:12:46 <adamw> time to end this nightmare!
17:12:52 <adamw> thanks for coming folks
17:12:54 <adamw> #endmeeting