15:01:45 #startmeeting Fedora QA meeting 15:01:45 Meeting started Mon Sep 26 15:01:45 2016 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:45 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:45 The meeting name has been set to 'fedora_qa_meeting' 15:01:48 #meetingname fedora-qa 15:01:48 The meeting name has been set to 'fedora-qa' 15:01:51 afternoon afternoon 15:01:52 #topic Roll call 15:02:00 ahoyhoy folks 15:02:03 who's around? 15:02:06 hi garret 15:02:08 \me is here 15:02:11 * kparal is here 15:02:21 * tenk is here 15:02:26 ha, just like in Windows 15:02:32 * pschindl is here 15:02:32 * coremodule is here! 15:02:32 garretraziel: review my patches damn you :P 15:02:35 c:\me 15:02:49 * tflink is here 15:03:02 adamw: do you know how long it takes to run whole test suite on my laptop :-D? 15:03:33 .fire garretraziel longer than it takes me to fire you! 15:03:33 adamw fires garretraziel longer than it takes me to fire you! 15:03:51 * cmurf thinks he's here 15:04:23 cmurf: sorry, but no. 15:04:23 :P 15:04:28 alrighty, let's get rolling! 15:04:40 #topic Previous meeting follow-up 15:04:42 adamw: you need to wait longer to see if that sinks in and what effect it has 15:05:03 next week on Doctor Who, be baffled and bored as the Doctor goes back in time to fix some embarrassing bugs in Fedora releases before they go out 15:05:13 (hi Pharaoh) 15:05:18 oh really? 15:05:25 * satellit listening 15:05:40 so i guess sumantro didn't make it - he let me know he'd be on the road today 15:05:49 i'll try and cover his topics 15:06:04 #info "sumantro to check testdays webapp is working ahead of test days starting up" - pretty sure that got done, since we've run a Test Day using the app since then :) 15:06:26 #info "adamw to do something about a wayland test day" - not done yet, still on my todo list (i'll do that first thing this week) 15:06:31 #action adamw to arrange a Wayland test day 15:07:13 hmm, next one is "sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport" 15:07:23 i don't see a ticket for that yet, or anything; anyone know the status? 15:08:18 no 15:08:41 I could ping one of the stakeholders if they're listed in the change, but I thought sumantro had done that already 15:09:07 yeah, let's wait till he's around' 15:09:13 i'll put the item back on the list for next time 15:09:16 * handsome_pirate shows up for a meeting for once 15:09:19 it might be best to leave it to sumantro rather than add confusion 15:09:35 #info "sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport" - we're not sure on the status of this and sumantro is not here today, will re-list 15:09:42 #action sumantro to organize a test day for https://fedoraproject.org/wiki/Changes/BetterSwitchableGraphicsSupport 15:10:37 #info "sumantro and a2batic to work on planning migration from trac" - looks like a2batic sent an email on that topic this morning, so this is moving along: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/WYNY2LISKV7S3QVEGQGTD6IGSLKOAO5H/ 15:11:08 do we want to dive into that now, or shall we follow it up on the lisT? 15:11:47 pagure transition? 15:11:53 yeah 15:12:21 i'm fine with followup on the list, i'm not sure what to chat about it yet anyway 15:12:59 alrighty 15:13:09 then i think that's all the follow-up 15:13:15 infra moved to pagure over the weekend, so we'll let them shake things out :D 15:13:26 ;) 15:13:31 #topic Fedora 25 Beta status 15:14:07 so, freeze is 09-27 00:00 UTC, which - if my math is correct - is like 16 hours away 15:14:14 so we're getting down to the business end of Beta 15:14:31 I'm slightly surprised to not see many bugzilla emails early Monday morning 15:14:44 we've got 4 beta blockers ATM 15:15:01 i got about a dozen, but yeah, it's a light monday... 15:15:24 Whoo! 15:15:34 I guess I'm not cc'd on enough bugs haha 15:16:37 cmurf: you should make like robatino 15:16:42 he CCs himself on every blocker bug 15:16:53 every time i fail a blocker the next mail I get is 'robatino CCed himself', heh 15:16:57 so freeze causes things in u-t to stay in u-t, that's the gist? 15:17:03 yes 15:17:31 so if at freeze time something doesn't have enough karma to go stable, it stays in u-t 15:17:53 after freeze nothing goes from u-t to 'stable' (the 'fedora' repo used for the composes) unless it fixes a freeze exception or blocker bug 15:18:18 but more importantly it means we have two weeks until go/no-go :) 15:18:26 * adamw looks at https://www.happyassassin.net/testcase_stats/25 15:18:40 holy k rap 15:18:48 we're missing the RAID tests, so I guess I'll try and get them done this week 15:18:54 #action adamw to run firmware/hardware RAID tests 15:19:51 so there's a raid bug that's a rather odd duck, in that it's affecting CentOS7, Fedora 25, 24, 23, and maybe 22? 15:20:17 I'm like - that stuff got tested so has something else changed? Anyway, can talk about it on #fedora-qa but it's sort of a WTF bug 15:20:27 RAID bugs are always wacky 15:20:34 and quite frequently depend on the previous state of the disk... 15:20:45 so yeah, test coverage doesn't look bad but there's a few tests that haven't been run since Alpha 15:20:54 right that's what I'm thinking is something new touched the drive and then something old is doing the WTF 15:21:04 #info https://www.happyassassin.net/testcase_stats/25 looks pretty good but there's a few tests that haven't been run since Alpha: please help out with covering those 15:21:34 anyone have any specific bugs/issues/situations to cover for Beta, or are we just kinda rolling along? 15:22:04 roll along more on the agenda 15:22:08 tflink: coremodule: which of us is supposed to be looking after Cloud, again? 15:22:25 the Cloud tests don't seem to have been run since Alpha, it'd be good to avoid another last-minute 'oh crap, the cloud images don't work' situation 15:22:28 oh yeah good point 15:22:33 I can do it. I'll do it today! 15:22:39 that's the spirit 15:22:41 * adamw sends gin 15:22:48 Delicious! 15:22:48 #action coremodule to run Cloud tests 15:22:58 I keep trying to test the atomic ISO but until there's a new anaconda in a compose it blows up 15:23:05 coremodule: i'm not sure if nightlies go to EC2, best ask in releng if you want to try 'em there 15:23:28 cmurf: yeah, openQA knows that one is broken, but it's not the blocking image 15:23:42 Cloud folks want to make atomic iso their official release, i.e. release blocking image 15:23:44 i really need to add a space to the matrix for that image in fact... 15:23:55 I'm kinda like, let's maybe do that for F26... 15:23:56 cmurf: yep, but i think still the installer ISO wouldn't be blocking, and it's not till 26 at least. 15:23:59 kinda short notice 15:24:08 adamw, Okay, I will. 15:24:21 The impression I got is they want it release blocking for F25... 15:24:40 There are other ducks to get in a row before that though IMO 15:24:42 #info freeze is in 16 hours from this meeting, go/no-go is in two weeks (2016-10-07) 15:24:56 cmurf: wow, that sounds...radical...do you have references? is there a ticket or something? 15:25:07 i thought the blocking list was supposed to be set in stone some time before Alpha 15:25:22 right me too, just on the face of it 15:25:37 it's on the cloud list and in meeting logs 15:25:48 I don't know if there's a ticket, it's a discussion 15:26:06 And maybe I threw cold water on it when I said, OK all the stuff in custom partitioning needs to work for you to do that 15:26:18 heh...well, only if they wanted the ISO. 15:26:19 pretty much nothing in custom partitioning works on atomic ISO 15:26:25 but thanks for the heads up, let us know if it goes any further... 15:26:37 yep 15:27:01 any other Beta topics? or shall we move on? 15:27:16 move on more interesting stuff to cover 15:27:28 #topic Test Day status 15:27:34 well, this one will be a bit tricky with sumantro not around 15:27:41 but let's just throw him lots of action items while he's away ;) 15:27:46 * cmurf snickers 15:27:58 MediaWriter for mac and windows? work for me but slow downloading .isos FedoraMediaWriter-mac-3.97.1 https://bugzilla.redhat.com/show_bug.cgi?id=1377825#c6 15:28:14 +1 on Windows 15:28:22 very slow download and very very slow check 15:28:24 satellit: weird you get it to even launch, the Mac version stopped working for me entirely 15:28:43 nice on windows also even did wks f25 in other 15:28:50 #info the Fedora Media Writer test day went ahead and we had great attendance and test coverage, thanks everyone: http://testdays.fedorainfracloud.org/events/9 15:29:04 looks like a lot of bugs were found and there's already been a big fmw update 15:30:03 Is it normal that there is a new version of the tested product during the test day? 15:30:28 there were several new ones.. 15:30:42 tenk: not super common, but fmw is a fairly new and quite important thing 15:31:18 i'd say the most common pattern is that the maintainers of whatever we're testing try to get all the latest bits for testing lined up a few days before the test day, we test them during the event, and then they fix the bugs we find in the next week or two 15:31:30 --best in linux is 3.97.0 15:31:42 it has happened before though, if there turns out to be some really obvious bug in the thing-being-tested or whatever and we need to fix it rapidly. 15:32:03 * satellit no 3.97.1 15:32:33 #info the i18n Test Day will be on Wednesday 2016-09-28, please do come along if you can help test non-English input etc. - https://fedoraproject.org/wiki/Test_Day:2016-09-28_I18N_Test_Day 15:32:55 #action adamw and sumantro to blog, mail, social media etc. i18n Test Day 15:33:02 oh yeah, another one for sumantro... 15:33:09 #action sumantro to send post-event report for FMW Test Day 15:33:19 * adamw loves giving people work =0 15:33:46 #info Wayland test day does not yet exist because adamw is bad and should feel bad 15:34:42 any other test day notes before we move along? thoughts on the FMW event, especially from newer members - did you find it easy/fun to take part? 15:35:11 That could be good to have official UTC time of the start of the test 15:35:19 end the end 15:36:13 we usually try not to put a specific time frame on it so as not to discourage people from testing (as really, we can take test results from as soon as we have the test materials available up until a few days later) 15:36:31 but maybe we can find a middle ground :) 15:36:42 yes the second point is to know what version we should test 15:36:51 (in the past we put specific times on, and people who wanted to test but happened not to be available at those times would just be discouraged) 15:37:13 It would be nice to alert testers to do download just before test...as it did change versions 15:37:28 yeah, sounds like that caused some problems 15:37:35 yep 15:37:53 i think i saw kparal or someone asking the maintainer to put versions in the filenames... 15:38:04 they are there now 15:38:22 #info update of FMW during test period caused confusion, we should try to avoid changes to test materials during test days as much as possible, and message clearly when they're needed 15:38:29 It's just that the result page we don t know on which version the test have be made 15:38:29 cool 15:38:38 I noted version in comments on result page 15:38:52 tenk: yeah, i think when sumantro set up the page we didn't expect there would be multiple versions 15:39:15 thanks for the feedback, it's helpful :) 15:39:20 and no possibility to edit result also 15:39:47 I did add 2nd result by clicking top header though 15:40:52 20m remaining 15:41:06 tenk: yeah, the app is a fairly quick job that could do with some improvements 15:41:11 thanks for the heads up cmurf 15:41:31 #info people are still missing the obvious possible improvements for the testdays webapp (e.g. result editing) 15:41:48 yipes, two more topics...let's get along :) 15:41:53 #topic Workstation Atomic / Flatpak testing 15:42:13 so this is kind of a heads-up, as i realized workstation WG have been talking to me about this but i didn't really pass it along 15:42:28 they're planning to make a Workstation Atomic image, and they're interested in getting testing for it 15:43:02 it would be non-release-blocking at first, of course, so testing would be on the same level as the Xfce/LXDE/Cinnamon images etc 15:43:26 it is a different world that is for sure 15:44:01 so far i've basically been suggesting we could add it to the Desktop validation page and run mostly the same tests on it as we run on the regular Workstation image, with the update and upgrade tests adjusted for Atomic-ness, and maybe add a Flatpak install test which we could run on both images 15:44:06 does that sound sensible? 15:44:10 user facing behavior should be similar or identical to now, but for QA people needing to see things under the hood, it's radically different 15:44:42 adamw: yes but I also think this starts to relate to your Project Ambition 15:45:06 because the old way of testing just doesn't scale as you're pointing out 15:45:07 haha 15:45:30 well, this would only grow an extra column on the 'non-blocking desktops' table really so it's not super horrible 15:45:34 but yeah 15:45:56 we can only stuff *so* many columns into wiki tables 15:46:01 right I get that, but it's also fitting a square peg into a round slot so... 15:46:28 better than nothing so yea sure do it 15:46:56 this would be whenever they actually manage to get the image built as part of official composes by releng, that's kinda the point i was looking at. 15:47:16 yes i just saw releng's meeting notes 15:47:48 "ostree repo management needs redesigned and unified in pungi and bodhi" 15:48:21 #info Workstation team are planning a Workstation Atomic image, expecting Flatpak to be the primary app delivery mechanism, and would like testing for it; we plan to add it to the Desktop matrix as a column in the non-blocking desktops table 15:48:22 releng changes, installer changes, file system changes... 15:48:31 CHANGE ALL THE THINGS 15:48:37 change it now! 15:48:40 i don't know if that's specifically related to workstation 15:48:43 aaaaanyho 15:48:58 it's related to all atomic stuff (rpm-ostree really) 15:49:00 did that get everyone up to date? any questions/objections before we move on to Project Ambition? :P 15:49:02 right 15:49:11 Project Ambition 15:50:11 alrighty, project ambition it is 15:50:18 so we only have ten minutes, but quickly... 15:50:31 #topic Release validation NG 15:50:57 so in case anyone didn't see, i sent a mail with my initial thoughts on this: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/message/XO5WAZ6CVXUPD2W5DACXOFA3A257DJBV/ 15:51:19 what's NG? 15:51:20 the executive summary is: let's build a release validation testing submission/display webapp that's better than doing it in mediawiki 15:51:25 Next/New Generation 15:51:28 gotcha 15:51:30 just a silly name 15:51:32 pity that jskladan is not here. but I think one of the major issues with using resultsdb frontend (e.g. similar to current testday app) is lack of auth support 15:51:42 not just frontend 15:51:48 for testdays, we simply disallowed editing and removal of results 15:51:52 kparal: well, if we're using a webapp as an intermediary is that really a problem? 15:52:10 FWIW, this is a usecase we've been talking about for years 15:52:18 not that any progress has been made on it :-/ 15:52:19 we can do the auth there and just grant the webapp rights to do whatever it likes to resultsdb... 15:52:30 tflink: yeah, this is indeed Episode ~8 in an ongoing story 15:52:39 adamw: how do you track who submitted what, then? 15:53:14 i suppose we could have a convention for a key/value pair attached to human-submitted resultsd 15:53:15 tflink: fair point, but it would be easier to just add a field to resultsdb that it doesn't have to actually authenticate (it just trusts the webapp told it the right thing), wouldn't it? 15:53:15 I'm a bit afraid we will need to implement quite a lot of stuff - what about edit history, do we care? 15:53:46 I'd rather not allow result changing in resultsdb 15:53:50 kparal: while i've been thinking about it i've been trying *not* to plan too much stuff, there's a lot we *could* do, but i think if we just focus first on making 'something better than the wiki' it wouldn't be too big 15:54:08 there's lots to think about, i guess 15:54:36 I'm definitely for the idea of using a more appropriate tool. otoh with wiki we have a lot of features for free 15:54:44 but i think if we want something a) really tuned to our workflow and b) using resultsdb it's kinda gonna have to be 'build it yourself'...i think moztrap was our best shot at 'use something else' and it still didn't quite fly 15:54:57 yeah 15:55:04 I think it's reasonable to have an ideal goal in mind that's also realistic, and then break that up into v1, v2, v3 milestones 15:55:06 I'm about to use the s-word but it shouldn't be too bad to throw something together as a PoC assuming someone has the time to do it 15:55:10 kparal: we *could* build a webapp which acts as an intermediary to *the wiki*, but that's getting dangerously close to insanity...:P 15:55:24 adamw: you're on your own for that one :-P 15:55:28 we need more layers 15:55:34 * satellit need to keep it simple for nubies to use still not up to speed in editing in git and pagure...:/ 15:55:35 tflink: i was basically volunteering to at least *try* and do that. you know, with my extensive webapp development background 15:55:46 satellit: oh, it'd be much easier than that. hopefully. 15:55:48 satellit: that's the idea 15:55:54 +1 15:55:56 satellit: make it easier for people to contribute 15:56:09 satellit: i'd hope it'd be pretty much like the wiki only easier: see a gap, click it, click pass/fail/warn, type your comment, profit 15:56:10 adamw: that's the hardest I've laughed for a while :-D 15:56:10 adamw: there has to be some useful related code somewhere on stackoverflow, no worries :P 15:56:21 kparal: right? how hard can it be 15:56:30 i googled flask yesterday, i've totally got this 15:56:33 bah, I can't read 15:56:40 well, you need to figure out where to steal the underpants from first i guess 15:57:13 * tflink laughed because he thought adamw was volunteering someone else to do the webapp work 15:57:22 tflink: oh, well, i can do that too :P 15:57:55 so i'm guessing it'd be too much to hope there's a webapp development ninja idling here who just thought 'hey, this would be a great project for my huge amount of spare time'? 15:57:58 ... 15:58:07 .... 15:58:15 thought so! 15:58:17 hence, where to steal underpants from 15:59:05 he wouldn't be ninja if you could see him, would he 15:59:15 so i'm gonna try and find some cycles to at least start fiddling with this and try to come up with some kind of mock-up (and teach myself webapp development at the same time!), but if anyone wants to help out / do it instead because they'd be better at it, please say so 15:59:16 maybe put the idea on devel? 15:59:34 cmurf: good point, might be people on other lists interested in helping out; i'll send out a mail shot 15:59:50 yeah if it has a design writeup, that might help someone better understand what scope and action they'd be getting into 15:59:59 #action adamw to troll for interest in release validation app development project outside of test@ list 16:00:11 step 1. steal underpants 16:00:15 maybe i can talk to mizmo about it and come up with a design... 16:00:24 * tenk find the edit of the wiki for result quite simple 16:00:37 tenk: yikes, really? i think you're the first person who ever said that :P 16:00:53 adamw: yes really 16:00:54 easy for me also... 16:01:02 adamw: it'd be worth looking at the fancy js frameworks (eg angular.js) for the app instead of flask 16:01:17 oh christ please don't make me code things in js 16:01:23 anyhoo, we're over time 16:01:26 :) 16:01:26 i think i busted the wiki the first three times i edited it, adamw has just blocked from memory having to fix it 16:01:36 cmurf: haha, maybe a lil bit 16:02:02 * satellit can always roll back 16:02:04 tenk: satellit: still that's interesting feedback, maybe it's not as big of a problem as i thought...maybe i'll talk to mo about doing some kind of user survey for it 16:02:15 satellit: it gets tricky when someone else edited since the break, though 16:02:21 alrighty, we should be over in #fedora-blocker-review right now 16:02:29 food and coffee time 16:02:30 so let's wind things up here, and i'll start that meeting up... 16:02:45 adamw: I just talk about to add test result on the wiki, not to add a new test or something else (not try yet) 16:02:54 #action adamw to do some very preliminary work on validation app planning, maybe with design WG 16:03:14 tenk: please do poke me if you're interested in that stuff, it'd be great to have someone else who knows how the bits work 16:03:30 tenk: be warned, though, if you start getting into how the pages are made, it gets into some crazy wikicode :P 16:03:38 * adamw doesn't even remember how it all works any more... 16:03:49 alrighty, thanks for coming along everyone 16:04:01 adamw: thanks for running the meeting 16:04:04 release blocker review meeting starting in #fedora-blocker-review right now 16:04:14 sorry for missing open floor - please send any urgent topics to the list 16:04:31 (or bring them up in blocker review open floor, but usually only two people are alive by that point) 16:05:15 #endmeeting