15:00:48 #startmeeting Fedora QA meeting 15:00:48 Meeting started Mon Jul 31 15:00:48 2017 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:48 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:48 The meeting name has been set to 'fedora_qa_meeting' 15:00:50 #meetingname fedora-qa 15:00:50 The meeting name has been set to 'fedora-qa' 15:00:58 #topic Roll Call 15:01:09 * tflink is here, finishing up another IRC meeting 15:01:31 * Sumantro is traveling 15:02:00 * tenk is here 15:02:08 morning folks, who's around for meeting funtimes? 15:02:32 Morning adamw 15:02:48 hi sumantro 15:02:50 how's it going? 15:03:08 Visa approved :) .. so can't complain 15:03:28 * kparal is here 15:04:02 Kparal hi :) 15:04:10 visa for flock? 15:04:26 Adamw yes! 15:04:30 great! 15:04:56 Adamw yes :) 15:05:23 alrighty 15:05:27 looks like that's everyone 15:06:01 #topic Previous meeting follow-up 15:06:05 let's see what I forgot this week 15:06:41 "sumukher to go ahead and send out the call for Test Days, as this is a short cycle and there's likely a lot of change to test" - you did send that, right? 15:06:54 Yes adamw 15:07:09 Approved and will announce lxqt test day soon 15:07:11 roger 15:07:15 With tenk 15:07:20 #info "sumukher to go ahead and send out the call for Test Days, as this is a short cycle and there's likely a lot of change to test" - this was done 15:07:21 yep 15:07:53 #info "adamw to draft up some proposals for adjusting QA processes, policies etc. to lack of Alpha release" - i'm still working on this, been thinking it through for a while, hoping to have actual drafts this week. there's a topic coming up later with a bit more detail 15:08:07 #info "adamw to look into adjustments to Server validation for Modular Server" - still on my todo list :/ 15:08:28 #info "adamw to add lxqt to desktop test matrix" - this one is done 15:08:41 "sumukher and tenk to look into setting up an lxqt spin test day" - sounds like you're moving along with that one? 15:09:00 Yes tenk and me are on it 15:09:30 perhaps it will help tester if lxqt is added on the fedora media writer 15:09:37 Will announce somewhere 20th of Aug .. ticket on Pagure is filed but I can't pull that as I am on mobile 15:10:08 Tenk , mbriza is the person to reach out for that 15:10:21 Sumantro: ok 15:10:31 agreed, it sounds like a bug to me (though not 100% sure) 15:10:58 but for a test day we may want to use a new image with latest bits anyhow 15:11:19 ok 15:12:00 anything else for follow-up? 15:12:34 Adamw to review storage drafts for validation matrix which roshi and me drafted 15:12:40 tenk: yeah, somebody already reported that to me, sorry for not including that 15:12:57 Sumantro: yeah, indeed, i never got to that last week :( sorry 15:13:17 at first i thought it'd be quick so i started looking at it late one day, then realized it'll probably take a bit longer and didn't have the time 15:13:32 will do it this week, i promise 15:13:35 That's okay :) 15:13:54 You reviews helps a *lot* :) 15:14:14 once i actually get it done :P 15:14:17 okay, moving along 15:14:21 mbriza: thanks 15:14:58 mbriza: no problem, thx to have take into account this 15:15:02 #topic 'No more Alpha' status update 15:15:17 tenk: it may take a while, i'll have to get the icon for it somewhere 15:15:18 so just figured i'd give a bit more detail on what i'm thinking for this currently, before i start sending out drafts, so you folks can chime in 15:16:02 Mbriza thanks :) 15:16:40 broadly my best idea atm is to turn the Alpha release criteria into something like "Branched quality target", and document that our aim during the post-branch, pre-beta phase is to ensure every published compose meets those requirements 15:17:05 and obviously require Beta and Final to meet those requirements (as at present) 15:18:10 the alternative would be to merge the alpha criteria into Beta and write up some new stuff about the pre-beta automated testing requirements, but that seems like more work and potentially more confusion... 15:19:36 so what happens when those reqs are not met 15:19:56 kparal: yeah, that's a good question :P 15:20:39 obviously, any violation that makes it into a compose is a beta blocker 15:21:15 but i think, practically speaking, we can only block the 'release' of nightly composes on things we can automate testing for 15:22:04 i don't think there's any value to this scenario: compose 20170731 passes automated testing and is released, then manual testing finds a 'Branched quality target' violation in it, so we block the release of all further nightly composes till that violation is fixed 15:22:45 the benefit is that people following latest branched don't get broken composes 15:22:52 but it doesn't achieve that 15:23:02 because the original broken compose would still be the 'current' compose in that case 15:23:19 ah, I see. I was already in the gated world 15:23:51 okay, so let me reset: i'm thinking about the *awkward* case where we fail to catch a violation initially 15:24:19 of course the straightforward case is that a compose fails the automated tests and is rejected for release 15:24:48 the corner case is where a compose passes the automated tests, but turns out to have a 'Branched requirements'-violating bug which the automated tests missed 15:25:25 and really, i can't think of a lot we can do in that case, unfortunately. so for now i'm gonna say we don't try too hard and just say, that's a thing that could happen. 15:25:36 can we add some karma point to the release, more karma more tested the compose is 15:25:56 karma not by tested user but by tested module 15:26:25 like automatique test ok: 5 karma, pre beta test ok: 5 karma 15:26:30 well, i know the devs are working on 'generalizing' bodhi right now (so it can be used on things other than RPMs, the intent being to use it for the 'modules' of modularity) 15:26:41 so i guess in theory we could 15:27:07 but the problem with putting any kind of manual testing in the 'nightly' compose loop is that people are slower than robots :P 15:27:34 i can see possible tweaks we could carry out down the road, like giving a four hour window for pitiful humans to download the compose and flag any obvious issues before it gets synced, something like that 15:27:38 but not sure we want to *start* there 15:29:53 the karma could help people to know easely how far the compose have been tested before install 15:30:16 just not to have people complain about compose quality 15:30:19 tenk: ah, i see where you're going. yeah, for people wanting to install from a nightly it'd be handy 15:30:30 tenk: but the main thing we're concerned about isn't actually that 15:30:48 what it means for a nightly compose to be 'released' is that it gets synced to the location on the mirrors that installed systems use as their main repo 15:31:00 so that people who already have the release installed get the packages from it as updates 15:31:23 what we want to avoid is sending out a 'bad' compose that breaks existing installs 15:32:33 so that's pretty much where i'm at with it right now, aiming for the most obvious and straightforward change initially...does it sound broadly OK to everyone? any other notes before we move on? 15:34:23 sounds reasonable 15:34:48 except for humans racing with robots, obviously :) 15:35:16 well, of course, we had to use a *simulated* robot... 15:35:23 (/discovery channel reference) 15:36:16 okay, so i'll aim to get drafts out this week. thanks for the notes 15:36:21 #topic Fedora 27 Change review and test planning 15:36:32 sooo, this one's gonna be fun as i didn't actually plan it in advance :P 15:36:42 so let's just look at the f27 change list and cry a bit, shall we? 15:37:00 https://fedoraproject.org/wiki/Releases/27/ChangeSet 15:37:32 and yes, this *is* for a release scheduled to happen in october 15:37:38 perhaps manic laughter can accompany the crying 15:38:09 seems like a piece of cake! 15:38:52 poisoned cake containing rusty nails and very small sharks? 15:39:20 that must be some kind of a British pudding 15:39:35 hey, i resent that remark 15:39:43 pudding is the one thing british food actually gets *right* 15:39:53 :D 15:40:12 Adamw it's actually gonna be a roller coaster ride :D 15:40:40 anyway, who are you to talk? czech dessert is just another piece of meat on a smaller plate 15:40:58 exactly as it should be :) 15:41:09 Sumantro: for sure 15:41:26 so, obviously some of these are pretty 'conventional' changes that we can handle with test days as usual 15:42:13 sumantro, i guess you're ready to take the lead on that? identifying changes that can benefit from a test day and reaching out to the owners to co-ordinate? 15:43:50 Yes already started ;) arbitary branching and modularity mainatiners have been contacted 15:44:17 Awaiting response and branch which is 15 aug 15:44:27 heh, see, those are ones i'm not sure we can cover so well with test days 15:44:46 although, the change owners might see things we could usefully do at one 15:46:00 Yup. Also, will be looking at the changes Tomo and mail the mainatiners :) 15:46:13 i guess we can do a test day *for package maintainers* to run through the arbitrary branching steps, for e.g. 15:46:39 but yeah, i'm actually interested in the question of what we should do with regard to the bigger changes 15:47:29 specifically 'modular release', 'modular server', 'host and platform', and 'arbitrary branching' 15:48:00 so far i'm on the spot to do some work related to 'modular server', but should we be doing anything else? any thoughts? 15:49:03 I need to go through the changes to suggest something 15:50:47 i kinda get a feeling like we ought to be Doing Something, but i'm not entirely clear on what Thing that should be :) 15:51:51 I can help you Sumantro on that to if you want 15:52:28 yeah, it'd be great if folks can read through and suggest anything that comes to mind 15:52:41 more than anything, we should be on top of what's happening and trying to actively keep up with it 15:55:40 Since the release cycle is small which means we will have less time to run test days tol 15:55:44 Too* 15:55:51 yep, another great thing about the release cycle :/ 15:56:08 Maybe prioritize the major test days 15:56:28 Sumantro: yeah, please try and make sure there are definitely test days early enough for the major changes 15:56:48 Done. 15:57:05 so we'll maybe revisit this at a later meeting, for now let's: 15:57:07 #topic Open floor 15:57:13 any other business while we still have a couple of minutes? 15:57:48 On boarding call to happen soon , voting for timings ends on 2017-08-03 15:57:59 right, new folks, please vote on that 15:58:25 #info next onboarding call will happen soon, there's a vote for preferred time/date which ends on 2017-08-03 15:58:26 Fedora QA classroom community blog is up and we will have a session there too. 15:59:02 awesome 15:59:42 when you say 'up', do you mean submitted for review? I don't see it at https://communityblog.fedoraproject.org/ 16:00:57 No no https://fedoramagazine.org/fedora-classroom-sessions/ 16:01:07 My bad it's on fedora mag 16:01:12 oh whoops :) 16:01:58 #info there will be a Fedora Classroom session on QA around the end of August, see https://fedoramagazine.org/fedora-classroom-sessions/ 16:02:04 okay, anything else, folks? 16:02:40 not for me 16:03:11 Nothing from my side 16:03:24 kparal must be busy eating his dessert meat 16:03:47 alrighty, thanks for coming, everyone 16:03:54 thx for the meeting 16:03:55 see you next time 16:03:55 yum yum 16:04:16 #endmeeting