15:00:30 <adamw> #startmeeting Fedora QA meeting
15:00:30 <zodbot> Meeting started Mon Oct 20 15:00:30 2014 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:30 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:34 <adamw> #meetingname fedora-qa
15:00:34 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:38 <adamw> #topic Roll call
15:00:41 <adamw> ahoy hoy, folks
15:00:44 * pschindl_ is here
15:00:47 <adamw> who's around for some meet-y goodness?
15:00:49 * kparal is here
15:01:13 * roshi is here o/
15:01:14 * jreznik is here
15:01:24 * satellit listening
15:02:51 <adamw> #chair satellit pschindl_
15:02:51 <zodbot> Current chairs: adamw pschindl_ satellit
15:03:55 <adamw> no-one else wants tasty tasty meet? :)
15:04:37 <roshi> danofsatx ^^
15:04:55 <danofsatx> aqui
15:05:12 <adamw> ahoy dan
15:05:15 * danofsatx is coffee deficient
15:05:31 <adamw> emergency coffee transfusion for danofsatx, STAT
15:05:39 <adamw> #topic Fedora 21 status
15:06:16 <adamw> so we're on TC4, we'll probably want a new compose soon because of the issue with fedora-release/generic-release in TC4 -  that's https://bugzilla.redhat.com/show_bug.cgi?id=1154235
15:07:24 <adamw> do we want to do a quick blocker review after this meeting, btw?
15:07:33 <jreznik> I'd say yes
15:07:38 <adamw> there are 6 proposed blockers and it's go/no-go this thursday
15:07:41 <roshi> works for me
15:08:07 <pschindl_> I can't be here for so long.
15:08:11 * jreznik will send go/no-go and readiness reminders later today/early tomorrow
15:08:20 <brunowolff> There is another sort of related bug where someone had trouble using a generic-release version of workstation. It was pulling in fedora-release even though they didn't want it to.
15:08:37 <brunowolff> It might be worth taking that into consideration when designing a fix.
15:09:11 <adamw> brunowolff: possibly, but it's less critical for now. did they report a bug?
15:10:12 <brunowolff> Yes. I asked about it on the desktop list and people were OK with a generic-release-workstation. Though it was still an odd dependency causing the issue.
15:10:45 <adamw> ok
15:10:52 <brunowolff> I didn't want to throw that in before beta because of the potential to break things.
15:11:12 <brunowolff> https://bugzilla.redhat.com/show_bug.cgi?id=1154154
15:11:51 <adamw> so let's try for a blocker review meeting after we're done here. aside from the potential blockers we still have one accepted blocker without a fix - #1120964
15:12:21 <adamw> i asked anaconda folks if anyone other than dlehman could work it while he was away, but seems like he's back today, so i'll ask him to take another look at it
15:12:45 <adamw> we have a test fix for the fedup issue, I was trying to get releng to build a test upgrade initramfs for that, i'll try again today
15:14:15 <adamw> 21 test coverage is still missing a few things, notably cloud tests
15:14:49 <roshi> I'll be on that with TC4 once I get some images
15:14:50 <adamw> that is, the cloud tests on the Installation page
15:15:05 <adamw> i thought we had some now?
15:15:12 <roshi> still no cloud images that I can find (though I haven't looked in koji today)
15:15:49 <roshi> none on dl.fp.o
15:16:03 <roshi> I'll ping pbrobinson about it and take care of that today
15:16:04 <adamw> hum
15:16:05 <adamw> yup
15:16:18 <adamw> #info cloud images for TC4 are again missing, roshi will follow up with pbrobinson
15:16:21 <roshi> there are *isos* but those aren'
15:16:26 <roshi> t useful
15:16:28 <roshi> :)
15:16:29 <adamw> right
15:16:54 <danofsatx> yup, the iso's don't work.
15:16:59 * danofsatx found out the hard way
15:18:36 <adamw> do we have any other notes on f21 status?
15:19:42 <kparal> I have looked at the renewed testcase_stats and it doesn't look that bad
15:20:03 <kparal> there are some blank spots
15:20:37 <adamw> kparal: there aren't a *lot* of them, but we need to get to 'em all, like the annoying kickstart tests everyone hates doing :)
15:21:35 <kparal> yeah
15:22:11 <adamw> ok, so let's move on so we can get to blocker meeting and then (shock) actual testing!
15:22:20 <adamw> #info blocker review meeting will be held after this meeting
15:22:38 <adamw> #topic Taskotron next steps
15:23:05 <adamw> so just in case anyone hadn't noticed yet, Taskotron is now in production deployment and AutoQA has retired to a farm upstate
15:23:16 <adamw> congratulations tools folks :)
15:23:27 <roshi> \o/
15:23:52 <kparal> yay
15:25:31 <kparal> also, someone fixed something in bodhi/fedora infra and our tests are no longer crashing so often. which is always good news
15:25:41 <adamw> there was a fesco meeting last week where a proposal of mine to require update bundling (so updates don't require things in *other* updates) as part of the Updates Policy got passed
15:25:45 <adamw> awesome
15:26:15 <kparal> should we adjust our checks to accommodate to that?
15:26:21 <adamw> so on that specific front i was looking at whether we should keep an eye on taskotron accuracy to see if it's potentially good enough for its tests to be 'enforced' via karma or update blocking in future
15:26:27 <adamw> kparal: depcheck is already the test to cover that
15:26:30 <adamw> my interest was ^^^
15:26:36 <kparal> well
15:26:57 <adamw> we know autoqa depcheck was not accurate enough for that, but taskotron depcheck seems better
15:27:09 <kparal> the current approach of both depcheck and upgradepath, afaik, is to consider all updates in -pending together. so we don't actually require all updates to be standalone, just to be pushed together
15:27:24 <kparal> the question is whether we should more strict about it
15:27:36 <adamw> oh, i didn't realize that part.
15:27:52 <adamw> so it'll only catch cases when the required update hasn't been submitted when the other package is tested, for e.g.?
15:28:23 <kparal> submitted, as in requested to be pushed to stable
15:28:43 <kparal> so if update A requires update B, but you only submit A to stable, and not B, depcheck will complain
15:28:55 <kparal> if you submit A and B at the same time, it's ok from our POV
15:29:05 <kparal> current POV
15:29:32 <kparal> we can be totally strict and say "A needs to bundle B"
15:29:50 <adamw> that's what i had in mind with the policy, yeah
15:29:58 <kparal> of course I'm not sure about all the edge cases, so it might not be totally simple to change
15:30:08 <adamw> having them submitted for stable push at the same time is a lot better than not, though
15:30:13 <kparal> :)
15:30:38 <adamw> do we have a plan for monitoring taskotron's results? or was it more a 'throw it out there and see who complains' thing?
15:30:52 <kparal> what monitoring do you have on mind?
15:31:56 <adamw> well, it was an open ended question :) just wondered if it was something that had been considered at all
15:32:19 <kparal> I already did a short comparison of AutoQA's depcheck vs Taskotron's depcheck, of course on just a limited sample. we've found some cases where it's better (not producing false negatives), and some cases where it's worse (not catching some use cases)
15:32:25 <kparal> we'll try to improve that
15:32:49 <roshi> iirc, tim was talking about monitoring and also working on the front end for said monitoring
15:32:49 <kparal> as for monitoring crashed tests etc, we receive an email on every crash
15:33:06 <roshi> but I don't know for sure - it was spitballing IRL
15:33:12 <adamw> OK
15:33:22 <kparal> ah, like front-end and back-end monitoring, to be sure nothing has broken
15:33:33 <kparal> tim has to provide an update on that, I don't really know
15:33:43 <adamw> kparal: i was mostly meaning checking the quality of the results, but the other one's an interesting topic too
15:34:36 <kparal> as for quality of results, it seems that the new depcheck has improved the level of  trustworthiness
15:34:46 <kparal> it might not catch everything, but when it does, it usually doesn't lie
15:34:54 <adamw> that's definitely an improvement
15:35:04 <roshi> for sure
15:35:25 <adamw> ok, so maybe keep this one in mind for now, obviously not as important as f21 testing, just wanted to get it out there
15:35:29 <kparal> I'm sure we plan to check on that more in the future
15:35:41 <adamw> next steps in terms of development are presumably to get disposable test clients going?
15:35:56 <kparal> I believe so
15:36:54 <adamw> coolbeans.
15:37:22 <adamw> alrighty, moving on...
15:37:25 <adamw> #topic Open floor
15:37:37 <adamw> anyone have anything to kick around? we have a bit of time if we need it, or we can move on to blocker review
15:38:01 * roshi has nothing
15:38:03 <satellit_e> anyone confirmed liveusb-creator?
15:38:24 <danofsatx> wasn't fedmsg integration part of taskotron?
15:38:48 <roshi> taskotron gets triggered by fedmsg danofsatx, iirc
15:38:53 <kparal> adamw: I wonder, do you plan to keep https://www.happyassassin.net/testcase_stats/21/ at the current URL? I just want to reference to it in some documents
15:39:17 <adamw> kparal: i can leave it there but we can also move it to the old URL if you like
15:39:33 <adamw> kparal: I don't know if I have any access to that system, or who does?
15:39:47 <danofsatx> ahh, ok. I thought it was to spit out the test results via zodbot, like the status messages injected into #releng
15:39:51 <kparal> adamw: if you're fine with keeping it there, let's just keep it there for F21 cycle. that will be easiest
15:39:54 <adamw> anyone who has access could set it up, it's pretty simple
15:39:56 <kparal> just wanted to confirm
15:41:09 <kparal> also I wonder if we could link to it at the top of the release validation pages. it's very useful and I guess most people don't know about it
15:41:11 <adamw> satellit: oh right, what was the problem you had with luc again? i forgot
15:41:35 <satellit> fails to create a bootable USB boots to cursor
15:41:45 <adamw> kparal: it'd be easy enough to add, I guess - the glory of templates ;)
15:42:01 <adamw> kparal: you'd just have to add it to https://fedoraproject.org/wiki/Template:Release_validation_instructions and it'd appear in all validation pages future and (recent) past
15:42:17 <adamw> satellit: what was the bug #?
15:42:18 <satellit_e> https://bugzilla.redhat.com/show_bug.cgi?id=1149782
15:42:38 <adamw> kparal: we could add a quick note about the Summary and testcase_stats pages to the instructions i guess for sure
15:42:59 <satellit_e> may be due to naming too long
15:43:11 <adamw> #info satellit_e has a potential blocker with liveusb-creator, https://bugzilla.redhat.com/show_bug.cgi?id=1149782 - if other people could test that'd be useful
15:43:44 <kparal> adamw: that sounds great. will you add it, or should I?
15:43:52 <adamw> i can do it
15:43:56 <adamw> thanks for the idea
15:44:15 <kparal> also, I'd like to add a short legend howto to the top of the testcase_stats results page
15:44:55 <kparal> at the moment, it's not clear that those dots is a time-view of different RC/TC builds
15:44:59 <kparal> just the colors are explained
15:45:19 <kparal> a few extra sentences would make it clearer
15:46:07 <kparal> I was working today on a short how-to-do-testing document for some interns and volunteers and this struck me
15:47:14 <adamw> sure, that'd be great
15:47:19 <adamw> send a patch ;)
15:47:25 <kparal> ok :)
15:47:35 <adamw> one idea I had was to have tooltips for the bitmap, this would complement that
15:47:52 <adamw> right now i'm in the middle of teaching relval to report results, though, which is teetering right on the edge of complete absurdity
15:48:07 <kparal> tooltips could also help
15:48:16 <adamw> (at the point i re-do testcase_stats to be dynamic and have 'report results' link we will have comprehensively fallen over the edge)
15:48:35 <kparal> :D
15:48:49 <adamw> well, it already reports results in the code i have, i just don't really like the design i came up with.
15:49:11 <adamw> and i didn't write the little interactive 'here are all the results, pick one' thing i wanted yet.
15:49:48 <adamw> brb, call of nature
15:50:15 <adamw> we probably should move wikitcms/relval to the fedora-qa repo and add it to phab i guess
15:51:42 <kparal> it will make bug reporting easier
15:54:19 * adamw back
15:54:27 <adamw> ok, sounds like we're done here, let's wrap up and move on to blocker review
15:54:36 * adamw lights fuse
15:56:58 <roshi> you running the review adamw or do you want me to?
15:57:31 <adamw> roshi: go ahead
15:57:33 * danofsatx votes for roshi
15:57:37 <adamw> thanks folks!
15:57:39 * adamw is offended
15:57:54 <danofsatx> you sholdn't be - it was a reward for the coffee transfusion
15:58:02 <roshi> now you've done it danofsatx
15:58:18 <roshi> thanks for running the meeting adamw
15:58:32 <danofsatx> I'm busy trying to build a router. be quiet over here.
15:59:09 <adamw> #endmeeting