16:00:35 <adamw> #startmeeting Fedora QA Meeting
16:00:35 <zodbot> Meeting started Mon Nov 28 16:00:35 2011 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:35 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:00:38 <adamw> #meetingname fedora-qa
16:00:38 <zodbot> The meeting name has been set to 'fedora-qa'
16:00:50 <adamw> #chair tflink j_dulaney
16:00:50 <zodbot> Current chairs: adamw j_dulaney tflink
16:00:56 <adamw> #topic roll call
16:01:10 * nirik is lurking around if he can help with anything.
16:01:12 * j_dulaney is ready to rock-and-roll!
16:01:14 <adamw> good morning everybody, welcome to another stonking week of fedora qa
16:01:34 * pschindl is here
16:01:45 * jskladan enters with evil laughter
16:02:20 <j_dulaney> Uh oh...
16:02:55 <adamw> hey hey hey, that's my job
16:03:11 * tflink is here
16:03:14 <adamw> #topic muahahahahahahaahahahah!
16:03:14 <adamw> ahem
16:03:18 * brunowolff is here
16:03:24 <adamw> #topic previous meeting follow-up
16:03:41 <adamw> hellooo, everybody. okay, let's see what we had hanging around from last week
16:04:12 <adamw> "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code"
16:04:17 <adamw> well, it looks like kparal isn't around
16:04:21 <adamw> does anyone know his status on that?
16:04:45 <j_dulaney> #info I think adamw fired kparal last week...
16:04:52 <jskladan> I know that kparal sent out some email to the anaconda guys
16:04:57 <adamw> heh
16:05:14 <jskladan> but I did not see any reply, apart of the ...
16:05:16 <adamw> jskladan: thanks
16:05:23 * jskladan is looking for it
16:05:31 <adamw> aha, here he is
16:05:33 * kparal joins in last
16:05:38 <kparal> at last
16:05:42 <adamw> kparal: just in time for your starring role!
16:05:51 <adamw> we're on your action item from last week: "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code"
16:05:57 <kparal> oh yes
16:06:04 <adamw> where'd you get with that?
16:06:05 <kparal> I sent an email to anaconda-devel
16:06:13 * kparal looking for link
16:06:37 <pschindl> jskladan is out for a moment (gnome-shell is fell)
16:06:49 <adamw> what? gnome shell crashed?! never!
16:06:50 <pschindl> *fallen :)
16:07:10 <j_dulaney> Not Gnome Shell!
16:07:14 <kparal> #link https://www.redhat.com/archives/anaconda-devel-list/2011-November/msg00112.html
16:07:19 <pschindl> yep, today for second time (propably some poblem with weather applet)
16:07:39 <kparal> pschindl: there are no applets, don't you know?
16:07:50 <kparal> anyway, I started the discussion and a few people responded
16:07:52 <pschindl> sry. extension
16:08:04 <kparal> but it doesn't seem like a huge discussion
16:08:34 <kparal> I hoped for some knowledgeable people to respond, like wwoods or jlaska, because they had experience with the previous test
16:08:35 <adamw> okay
16:08:46 * j_dulaney only sees clumens
16:08:49 <adamw> if you're worried the right people aren't  replying, probably best poke them directly
16:08:54 <kparal> anyway at least we know the current rats-install test doesn't work :)
16:08:59 <adamw> the thread may have just skipped under their radar
16:09:07 <kparal> I CCed them directly I believe
16:09:10 <adamw> ah okay
16:09:28 <adamw> #info kparal mailed anaconda-devel-list and wwoods / jlaska directly, but response so far seems thin
16:09:50 <j_dulaney> Should we get Máirín Duffy in on this?
16:10:03 <kparal> they know we can respond if they have some questions regarding autoqa
16:10:05 <adamw> not sure mo would have much input?
16:10:06 <j_dulaney> Since she's doing a lot of the design work
16:10:11 <kparal> and that's probably the most important
16:10:14 <adamw> j_dulaney: yeah, but that's at a very high level
16:10:27 <j_dulaney> True
16:10:31 <tflink> I think she's on anaconda-devel but I'm not sure that testable hooks into anaconda is so much of a design level
16:10:38 <kparal> nope
16:10:46 <adamw> j_dulaney: essentially mo draws workflow diagrams, it's still the anaconda team that writes the actual code :) (no insult intended either side)
16:11:12 * j_dulaney understands now
16:11:16 <adamw> kparal: well, having a line of communication open here is great, but this doesn't sound like something anaconda team will be actively asking us about
16:11:26 <adamw> it's more a 'we'll put in anything you need but we need you to tell us what it is' deal
16:11:34 <tflink> yeah
16:11:47 <adamw> i'd feel more comfortable if we did at least have a decent idea whether we need stuff in there or not
16:12:26 <kparal> I don't know and finding out is quite an extensive task
16:12:28 <tflink> I'm still not clear on what it is that we would want to do
16:12:38 <adamw> hrm.
16:12:43 <kparal> I hoped just to start the discussion
16:12:46 <adamw> yeah
16:13:07 <tflink> but I'm not sure my questions about purpose are as appropriate for here
16:13:10 <adamw> i guess all we can really do if we don't have any clear ideas is re-ping it
16:13:12 * jskladan aaaaand I'm back
16:14:32 <adamw> let's try putting it this way: is the uncertainty here most likely to be a symptom of us *not* needing any special code in anaconda for (auto)qa purposes?
16:14:52 <tflink> no, I don't think so
16:14:56 <tflink> at least not for me
16:14:57 <adamw> well, shucks. :)
16:15:17 <j_dulaney> I'm thinking it would make our life easier
16:15:24 <tflink> my question is "what are we going to try to automate?"
16:15:27 * j_dulaney is just not sure *what* code is needed, not that it *is* needed
16:15:34 <kparal> ideally anaconda guys will write their own test suite and hook it up in autoqa. so they should know best
16:15:51 <j_dulaney> kparal:  Do they know that they need to do this?
16:15:51 <adamw> did you ask them specifically if they're planning to do that?
16:15:56 <tflink> it sounds like they aren't really sure what would be helpful
16:16:14 <kparal> they already have some test suite in autoqa
16:16:21 <tflink> at least from clumens' response
16:18:15 <j_dulaney> kparal:  Won't the new Anaconda render that obsolete?
16:18:33 <kparal> j_dulaney: clumens responded something like they will need to update it
16:18:40 <tflink> j_dulaney: I don't think so. AFAIK, there is no GUI interface in those tests
16:18:46 <kparal> once they start to receive emails with test results they will be more active about it I hope
16:19:19 <adamw> so i'm still kinda looking for a clear way forward here: is the best we can do wait and hope? or re-ping specific people and hope they reply this time?
16:19:21 <kparal> oh, and I don't think we are talking about GUI testing, right? we are talking about anaconda testing
16:19:27 <j_dulaney> How much besides GUI/Workflow is changing?
16:19:53 <tflink> kparal: I thought we were talking about GUI testing since that's the major change for F17, I think
16:19:58 <j_dulaney> How much will that change the backend stuff?
16:20:58 <adamw> i think a lot of it is looking like re-organization rather than change exactly
16:21:11 <adamw> they're moving to a hub / spoke design rather than a serial one
16:21:14 <adamw> (aiui)
16:22:05 <kparal> there might be some misunderstandings here. I believe the new GUI is best tested by humans, and autoqa can be used to test the plumbing
16:22:29 <kparal> if you think we should automate GUI testing, well, I don't think we have the manpower to do that ATM
16:22:34 <adamw> no, that's not it
16:22:56 <tflink> I think we might need to add some clarification to the discussion, then
16:23:05 <adamw> yeah.
16:23:16 <tflink> I'm reading clumens' responses as talking about automated GUI testing
16:23:17 <adamw> gimme a sec to back up and check the history
16:23:44 <adamw> oh, kay
16:23:55 <adamw> sorry, my mistake: we are on UI stuff here
16:23:55 <kparal> as I wrote to clumens, the thing they can do is to allow to control the whole workflow programatically, without clicking buttons. then it can be tested
16:24:21 <adamw> but the thing is that even though we can't automate UI testing *now*, if we need fundamental code in there to make it possible in future, we should ask now
16:24:26 <adamw> because they aren't going to retrofit it.
16:24:47 <tflink> kparal: that's still testing the GUI even if it isn't through screenshots and clicking
16:24:55 <adamw> so yeah, we are specifically on UI here, and i guess part of the problem is that we don't have any UI tests so it's hard to know what we'd need
16:25:17 <kparal> that means we have to have someone who understands UI testing
16:25:44 <tflink> specifically, automated and scripted UI testing
16:25:47 <adamw> yeah. i think what we're missing here is the experience of trying to write automated UI tests against an existing codebase and going 'damn, wouldn't it be great if i could XXX'
16:26:14 <j_dulaney> Indeed
16:26:31 <adamw> kparal: tflink: do you know of anyone who fits the bill?
16:27:12 <kparal> I can talk to the RH desktop team and ask
16:27:26 <adamw> yeah, that might be a good way to go.
16:27:31 <kparal> we have a few guys sitting here in Brno
16:27:37 <adamw> sorry to drag this item out so far, but it'd be nice to have a definite plan.
16:27:58 <tflink> adamw: no not really, I've heard about building in testable interfaces but have never dealt with them myself and I don't know of anyone who has
16:28:30 <j_dulaney> Would anyone outside of Fedora have this experience?
16:29:03 * wwoods reading scrollback
16:29:07 <adamw> j_dulaney: well, probably? but we'd need someone who'd talk to us and be able to explain what we'd need
16:29:09 <adamw> oh hey, wwoods
16:29:54 <adamw> wwoods: executive summary: clumens has told us that if we want hooks for automated testing in the new UI code we should ask for them now so they get baked in at the start, but we're really not sure what would be useful as we don't really have any automated UI testing experience
16:30:05 <adamw> wwoods: kparal was hoping you might have a few thoughts
16:31:01 <wwoods> generally UI automation seems to use a11y
16:32:04 <adamw> (accessibility, for those playing along at home)
16:32:23 <wwoods> right. I can't remember what the current libraries are for this stuff
16:32:44 <wwoods> but from the anaconda side - we just have to make sure we write a proper accessibility-enabled UI
16:33:05 <wwoods> with the UI rewrite we're taking every step to use the standard widgets where possible
16:33:08 <adamw> okay
16:33:16 <wwoods> and we're trying to make sure the UI can be run on a normal system in a nice test mode
16:33:35 <wwoods> so it should be pretty easy to mess around and see how/if you can automate it
16:33:52 <wwoods> I think we can commit to making sure the UI is a11y-enabled (whatever that might entail)
16:34:09 <adamw> okay
16:34:31 <adamw> i think we're not necessarily going to get much farther in here
16:34:35 <wwoods> but I'll definitely bring it up with the team and we'll talk further on the mailing list(s) and/or put something in the UI rewrite docs
16:34:40 * j_dulaney thinks that it comes down to what tests we want to run
16:34:46 <adamw> kparal: okay if i action this back to you to try and follow up this week?
16:35:34 <kparal> yes
16:35:49 <adamw> okay
16:35:52 <wwoods> basically what it comes down to is: nearly automated GUI testing uses the a11y stuff, so if you want to be able to automate the UI itself, we need to have a11y support
16:36:12 <kparal> wwoods: just please respond with your thoughts to my email in anaconda-devel, thanks
16:36:15 <adamw> #action kparal to continue to look at whether we need hooks in anaconda UI code for testing, will see if anyone in RH's desktop QA team has experience
16:36:17 <wwoods> kparal: will do.
16:36:36 <adamw> thanks wwoods
16:36:53 <adamw> okay, let's try and blow through the rest of this fast :)
16:36:58 <adamw> "j_dulaney to take a shot at co-ordinating F17 test days, thanks! "
16:37:09 <adamw> j_dulaney: I sent you a sort of 'getting started' guide, did you get that?
16:37:21 <j_dulaney> I did
16:37:29 <adamw> cool
16:37:31 * j_dulaney has been on vacation
16:37:40 <j_dulaney> I'll get to it this afternoon
16:37:43 <adamw> that's fine
16:37:53 <adamw> just wanted to make sure communications are functioning properly etc :)
16:38:00 <adamw> alright, so nothing further necessary there
16:38:23 <adamw> #info j_dulaney is still happy with the test day co-ordination role, adamw is available for help
16:38:36 <adamw> "adamw to get retrospective tickets filed and summary mailed this week"
16:38:42 <adamw> i'm not quite done with this, but close
16:39:24 <adamw> i'm not gonna go into huge detail because it'd just duplicate the summary mail, but i've been filing tickets on our trac and releng's, and sending out a few ml discussions
16:39:39 <adamw> most of the tickets are just up for grabs, so do feel free to grab any you feel like taking a shot at
16:40:22 <adamw> QA tickets so far are 256 through 260
16:40:58 <adamw> #info adamw has nearly finished the retrospective wrap-up, several tickets filed so far
16:41:24 <adamw> anyone have any concerns / questions there or should we roll along?
16:42:11 <j_dulaney> Rolling on the river
16:42:24 <adamw> rolling rolling rolling
16:42:26 <adamw> rawhiiiiiiiiide!
16:42:37 <adamw> "adamw to start an ml thread proposing changes to the tc dates for f17 schedule to be sent to rbergeron "
16:42:43 <adamw> we've got an agenda item for that later, so let's table it
16:42:59 <adamw> aaand here it comes!
16:43:02 <adamw> #topic Revised F17 schedule confirmation
16:43:11 <j_dulaney> Ponies!
16:43:21 <adamw> so i did indeed start a thread proposing revisions to the TC drop dates for f17
16:43:38 <adamw> https://lists.fedoraproject.org/pipermail/test/2011-November/104583.html
16:43:59 <adamw> it looks like everyone was happy with that, so i'm planning to ask the berge to put it in place - just wanted to check if there were any last-minute issues anyone saw
16:44:15 <j_dulaney> Not here
16:44:31 <adamw> speak now or forever hold thy peace
16:45:14 <adamw> okay!
16:45:31 <adamw> #agreed proposed revisions to f17 schedule look good
16:45:40 <adamw> #action adamw to send the proposal along to rbergeron
16:45:55 <adamw> #topic F16 retrospective activity
16:46:30 <adamw> oh, yeah, i did want to take a few minutes just to flag up the QA retrospective tickets i filed so far so everyone knows the kinds of stuff we need to take care of ahead of f17 alpha cycle starting up
16:47:28 <adamw> it might be best to shelve this for next week though as there's some stuff that might start up interesting discussions but we're kinda long already
16:47:46 <adamw> is that okay? or was anyone really hoping for a nice long meeting? :)
16:48:00 <tflink> works for me
16:48:07 * j_dulaney is good
16:48:43 <adamw> #agreed this topic shelved till next week since not all tickets are yet filed and this meeting is running long
16:48:50 <adamw> #topic AutoQA update
16:48:57 <adamw> anything exciting from the world of autoqa, folks?
16:49:23 <tflink> still looking to get the new version of autoqa deployed
16:49:43 <kparal> we are just in the process of reviewing staging results from the past week and the weekend
16:49:49 <kparal> we found some weird issues
16:50:08 <kparal> it seems mainly autotest related, but are not sure
16:50:12 <kparal> *we
16:50:42 <kparal> I updated documentation about autoqa configuration
16:50:51 <adamw> #info team is looking into some issues found in testing of the new autoqa version
16:51:17 <kparal> if we don't find anything critical I would like to deploy _soon_
16:51:39 <kparal> who's got more updates?
16:51:42 <jskladan> pschindl & I are once again working on the ResultsDB (querying api and the frontend)
16:52:00 <tflink> most of the production infrastructure has been updated to not run on F14 any more
16:52:22 <jskladan> and IIRC, after we deploy the new version, we'll start testing ResultsDB on our Staging setup
16:52:39 <adamw> #info most infrastructure has now been upgraded from F14
16:52:55 <adamw> #info resultsdb to begin testing in staging after autoqa 0.7 is deployed
16:53:28 <adamw> sounds good
16:54:05 <adamw> okay, that's all from autoqa?
16:54:29 <tflink> I think so
16:54:46 <adamw> cool
16:54:47 <adamw> thanks
16:54:50 <adamw> #topic open floor
16:54:57 <adamw> aaand we made it - any other stuff?
16:55:16 * adamw notes that you can always add agenda topics to the wiki page ahead of time, which will likely give them more time than the 'open floor' slot
16:55:18 <adamw> PSA ends!
16:55:21 <j_dulaney> No one was fired?
16:55:41 <j_dulaney> PSA?
16:56:39 <tflink> public service announcement
16:56:48 <adamw> j_dulaney: i can fire you if you think it would help.
16:56:58 <j_dulaney> I've already been fired
16:58:19 <adamw> hey, that doesn't stop me. ask tflink, he must've been fired dozens of times now.
16:58:34 * tflink has lost count
16:58:49 <brunowolff> FYI rawhide is a bit rough now. gnome-shell has a soname issue, so it will be hard to test fresh builds. Hopefully this gets fixed soon.
16:59:17 <adamw> usually does.
16:59:30 <adamw> okay, sounds like we're about done here
16:59:38 <adamw> thanks for coming, everyone
16:59:50 <adamw> #endmeeting