15:00:48 <adamw> #startmeeting Fedora QA meeting
15:00:48 <zodbot> Meeting started Mon Jul 31 15:00:48 2017 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:48 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:48 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
15:00:50 <adamw> #meetingname fedora-qa
15:00:50 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:58 <adamw> #topic Roll Call
15:01:09 * tflink is here, finishing up another IRC meeting
15:01:31 * Sumantro is traveling
15:02:00 * tenk is here
15:02:08 <adamw> morning folks, who's around for meeting funtimes?
15:02:32 <Sumantro> Morning adamw
15:02:48 <adamw> hi sumantro
15:02:50 <adamw> how's it going?
15:03:08 <Sumantro> Visa approved :) .. so can't complain
15:03:28 * kparal is here
15:04:02 <Sumantro> Kparal hi :)
15:04:10 <adamw> visa for flock?
15:04:26 <Sumantro> Adamw yes!
15:04:30 <adamw> great!
15:04:56 <Sumantro> Adamw yes :)
15:05:23 <adamw> alrighty
15:05:27 <adamw> looks like that's everyone
15:06:01 <adamw> #topic Previous meeting follow-up
15:06:05 <adamw> let's see what I forgot this week
15:06:41 <adamw> "sumukher to go ahead and send out the call for Test Days, as this is a short cycle and there's likely a lot of change to test" - you did send that, right?
15:06:54 <Sumantro> Yes adamw
15:07:09 <Sumantro> Approved and will announce lxqt test day soon
15:07:11 <adamw> roger
15:07:15 <Sumantro> With tenk
15:07:20 <adamw> #info "sumukher to go ahead and send out the call for Test Days, as this is a short cycle and there's likely a lot of change to test" - this was done
15:07:21 <tenk> yep
15:07:53 <adamw> #info "adamw to draft up some proposals for adjusting QA processes, policies etc. to lack of Alpha release" - i'm still working on this, been thinking it through for a while, hoping to have actual drafts this week. there's a topic coming up later with a bit more detail
15:08:07 <adamw> #info "adamw to look into adjustments to Server validation for Modular Server" - still on my todo list :/
15:08:28 <adamw> #info "adamw to add lxqt to desktop test matrix" - this one is done
15:08:41 <adamw> "sumukher and tenk to look into setting up an lxqt spin test day" - sounds like you're moving along with that one?
15:09:00 <Sumantro> Yes tenk and me are on it
15:09:30 <tenk> perhaps it will help tester if lxqt is added on the fedora media writer
15:09:37 <Sumantro> Will announce somewhere 20th of Aug .. ticket on Pagure is filed but I can't pull that as I am on mobile
15:10:08 <Sumantro> Tenk , mbriza is the person to reach out for that
15:10:21 <tenk> Sumantro: ok
15:10:31 <adamw> agreed, it sounds like a bug to me (though not 100% sure)
15:10:58 <adamw> but for a test day we may want to use a new image with latest bits anyhow
15:11:19 <tenk> ok
15:12:00 <adamw> anything else for follow-up?
15:12:34 <Sumantro> Adamw to review storage drafts for validation matrix which roshi and me drafted
15:12:40 <mbriza> tenk: yeah, somebody already reported that to me, sorry for not including that
15:12:57 <adamw> Sumantro: yeah, indeed, i never got to that last week :( sorry
15:13:17 <adamw> at first i thought it'd be quick so i started looking at it late one day, then realized it'll probably take a bit longer and didn't have the time
15:13:32 <adamw> will do it this week, i promise
15:13:35 <Sumantro> That's okay :)
15:13:54 <Sumantro> You reviews helps a *lot* :)
15:14:14 <adamw> once i actually get it done :P
15:14:17 <adamw> okay, moving along
15:14:21 <adamw> mbriza: thanks
15:14:58 <tenk> mbriza: no problem, thx to have take into account this
15:15:02 <adamw> #topic 'No more Alpha' status update
15:15:17 <mbriza> tenk: it may take a while, i'll have to get the icon for it somewhere
15:15:18 <adamw> so just figured i'd give a bit more detail on what i'm thinking for this currently, before i start sending out drafts, so you folks can chime in
15:16:02 <Sumantro> Mbriza thanks :)
15:16:40 <adamw> broadly my best idea atm is to turn the Alpha release criteria into something like "Branched quality target", and document that our aim during the post-branch, pre-beta phase is to ensure every published compose meets those requirements
15:17:05 <adamw> and obviously require Beta and Final to meet those requirements (as at present)
15:18:10 <adamw> the alternative would be to merge the alpha criteria into Beta and write up some new stuff about the pre-beta automated testing requirements, but that seems like more work and potentially more confusion...
15:19:36 <kparal> so what happens when those reqs are not met
15:19:56 <adamw> kparal: yeah, that's a good question :P
15:20:39 <adamw> obviously, any violation that makes it into a compose is a beta blocker
15:21:15 <adamw> but i think, practically speaking, we can only block the 'release' of nightly composes on things we can automate testing for
15:22:04 <adamw> i don't think there's any value to this scenario: compose 20170731 passes automated testing and is released, then manual testing finds a 'Branched quality target' violation in it, so we block the release of all further nightly composes till that violation is fixed
15:22:45 <kparal> the benefit is that people following latest branched don't get broken composes
15:22:52 <adamw> but it doesn't achieve that
15:23:02 <adamw> because the original broken compose would still be the 'current' compose in that case
15:23:19 <kparal> ah, I see. I was already in the gated world
15:23:51 <adamw> okay, so let me reset: i'm thinking about the *awkward* case where we fail to catch a violation initially
15:24:19 <adamw> of course the straightforward case is that a compose fails the automated tests and is rejected for release
15:24:48 <adamw> the corner case is where a compose passes the automated tests, but turns out to have a 'Branched requirements'-violating bug which the automated tests missed
15:25:25 <adamw> and really, i can't think of a lot we can do in that case, unfortunately. so for now i'm gonna say we don't try too hard and just say, that's a thing that could happen.
15:25:36 <tenk> can we add some karma point to the release, more karma more tested the compose is
15:25:56 <tenk> karma not by tested user but by tested module
15:26:25 <tenk> like automatique test ok: 5 karma,  pre beta test ok: 5 karma
15:26:30 <adamw> well, i know the devs are working on 'generalizing' bodhi right now (so it can be used on things other than RPMs, the intent being to use it for the 'modules' of modularity)
15:26:41 <adamw> so i guess in theory we could
15:27:07 <adamw> but the problem with putting any kind of manual testing in the 'nightly' compose loop is that people are slower than robots :P
15:27:34 <adamw> i can see possible tweaks we could carry out down the road, like giving a four hour window for pitiful humans to download the compose and flag any obvious issues before it gets synced, something like that
15:27:38 <adamw> but not sure we want to *start* there
15:29:53 <tenk> the karma could help people to know easely how far the compose have been tested before install
15:30:16 <tenk> just not to have people complain about compose quality
15:30:19 <adamw> tenk: ah, i see where you're going. yeah, for people wanting to install from a nightly it'd be handy
15:30:30 <adamw> tenk: but the main thing we're concerned about isn't actually that
15:30:48 <adamw> what it means for a nightly compose to be 'released' is that it gets synced to the location on the mirrors that installed systems use as their main repo
15:31:00 <adamw> so that people who already have the release installed get the packages from it as updates
15:31:23 <adamw> what we want to avoid is sending out a 'bad' compose that breaks existing installs
15:32:33 <adamw> so that's pretty much where i'm at with it right now, aiming for the most obvious and straightforward change initially...does it sound broadly OK to everyone? any other notes before we move on?
15:34:23 <kparal> sounds reasonable
15:34:48 <kparal> except for humans racing with robots, obviously :)
15:35:16 <adamw> well, of course, we had to use a *simulated* robot...
15:35:23 <adamw> (/discovery channel reference)
15:36:16 <adamw> okay, so i'll aim to get drafts out this week. thanks for the notes
15:36:21 <adamw> #topic Fedora 27 Change review and test planning
15:36:32 <adamw> sooo, this one's gonna be fun as i didn't actually plan it in advance :P
15:36:42 <adamw> so let's just look at the f27 change list and cry a bit, shall we?
15:37:00 <adamw> https://fedoraproject.org/wiki/Releases/27/ChangeSet
15:37:32 <adamw> and yes, this *is* for a release scheduled to happen in october
15:37:38 <adamw> perhaps manic laughter can accompany the crying
15:38:09 <kparal> seems like a piece of cake!
15:38:52 <adamw> poisoned cake containing rusty nails and very small sharks?
15:39:20 <kparal> that must be some kind of a British pudding
15:39:35 <adamw> hey, i resent that remark
15:39:43 <adamw> pudding is the one thing british food actually gets *right*
15:39:53 <kparal> :D
15:40:12 <Sumantro> Adamw it's actually gonna be a roller coaster ride :D
15:40:40 <adamw> anyway, who are you to talk? czech dessert is just another piece of meat on a smaller plate
15:40:58 <kparal> exactly as it should be :)
15:41:09 <adamw> Sumantro: for sure
15:41:26 <adamw> so, obviously some of these are pretty 'conventional' changes that we can handle with test days as usual
15:42:13 <adamw> sumantro, i guess you're ready to take the lead on that? identifying changes that can benefit from a test day and reaching out to the owners to co-ordinate?
15:43:50 <Sumantro> Yes already started ;) arbitary branching and modularity mainatiners have been contacted
15:44:17 <Sumantro> Awaiting response and branch which is 15 aug
15:44:27 <adamw> heh, see, those are ones i'm not sure we can cover so well with test days
15:44:46 <adamw> although, the change owners might see things we could usefully do at one
15:46:00 <Sumantro> Yup. Also, will be looking at the changes Tomo and mail the mainatiners :)
15:46:13 <adamw> i guess we can do a test day *for package maintainers* to run through the arbitrary branching steps, for e.g.
15:46:39 <adamw> but yeah, i'm actually interested in the question of what we should do with regard to the bigger changes
15:47:29 <adamw> specifically 'modular release', 'modular server', 'host and platform', and 'arbitrary branching'
15:48:00 <adamw> so far i'm on the spot to do some work related to 'modular server', but should we be doing anything else? any thoughts?
15:49:03 <Sumantro> I need to go through the changes  to suggest something
15:50:47 <adamw> i kinda get a feeling like we ought to be Doing Something, but i'm not entirely clear on what Thing that should be :)
15:51:51 <tenk> I can help you Sumantro on that to if you want
15:52:28 <adamw> yeah, it'd be great if folks can read through and suggest anything that comes to mind
15:52:41 <adamw> more than anything, we should be on top of what's happening and trying to actively keep up with it
15:55:40 <Sumantro> Since the release cycle is small which means we will have less time to run test days tol
15:55:44 <Sumantro> Too*
15:55:51 <adamw> yep, another great thing about the release cycle :/
15:56:08 <Sumantro> Maybe prioritize the major test days
15:56:28 <adamw> Sumantro: yeah, please try and make sure there are definitely test days early enough for the major changes
15:56:48 <Sumantro> Done.
15:57:05 <adamw> so we'll maybe revisit this at a later meeting, for now let's:
15:57:07 <adamw> #topic Open floor
15:57:13 <adamw> any other business while we still have a couple of minutes?
15:57:48 <Sumantro> On boarding call to happen soon , voting for timings ends on 2017-08-03
15:57:59 <adamw> right, new folks, please vote on that
15:58:25 <adamw> #info next onboarding call will happen soon, there's a vote for preferred time/date which ends on 2017-08-03
15:58:26 <Sumantro> Fedora QA classroom community blog is up and we will have a session there too.
15:59:02 <adamw> awesome
15:59:42 <adamw> when you say 'up', do you mean submitted for review? I don't see it at https://communityblog.fedoraproject.org/
16:00:57 <Sumantro> No no https://fedoramagazine.org/fedora-classroom-sessions/
16:01:07 <Sumantro> My bad it's on fedora mag
16:01:12 <adamw> oh whoops :)
16:01:58 <adamw> #info there will be a Fedora Classroom session on QA around the end of August, see https://fedoramagazine.org/fedora-classroom-sessions/
16:02:04 <adamw> okay, anything else, folks?
16:02:40 <tenk> not for me
16:03:11 <Sumantro> Nothing from my side
16:03:24 <adamw> kparal must be busy eating his dessert meat
16:03:47 <adamw> alrighty, thanks for coming, everyone
16:03:54 <tenk> thx for the meeting
16:03:55 <adamw> see you next time
16:03:55 <kparal> yum yum
16:04:16 <adamw> #endmeeting