15:01:40 #startmeeting Fedora QA meeting 15:01:40 Meeting started Mon Jul 10 15:01:40 2017 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:40 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01:40 The meeting name has been set to 'fedora_qa_meeting' 15:01:43 #meetingname fedora-qa 15:01:44 The meeting name has been set to 'fedora-qa' 15:01:47 #topic Roll Call 15:01:52 ahoyhoy qa folks, who's around for a meeting? 15:03:12 * sumantro is here 15:03:15 * garretraziel is here 15:03:16 * satellit listening 15:03:17 * kparal is here 15:04:26 hi cmurf 15:04:47 how's everyone this sunny/not sunny (delete as appropriate) morning/afternoon/evening (ditto)? 15:05:31 60F and somewhat hung over 15:05:55 yowch 15:06:07 in mycase moonlit evening :D 15:06:17 i guess i'd better turn off my pneumatic drill then, huh 15:06:35 sumantro: nice! 15:06:35 * roshi is here 15:06:50 15C is nice 15:06:52 alrighty, let's get started with... 15:07:06 #topic Previous meeting follow-up 15:07:07 at least we're not on fire 15:07:32 you know, that's a useful mantra for almost any situation. 15:07:37 * cmurf is momentarily in this realm 15:07:39 when you're on fire, substitute "at least we're not drowning". 15:07:45 lol 15:08:12 haha well California is on fire, and ealier this spring they were drowning 15:08:27 if you're on fire *and* drowning, check if you're wearing a red shirt and work for Starfleet. 15:08:51 that *does* fit with the headcannon 15:09:19 we need another volunteer 15:09:21 headcannon sound like a fine idea, but how do you deal with the recoil? 15:09:38 welp, it appears we had no action items last time 15:09:43 #info no action items from last meeting 15:09:54 can anyone remember anything other than action items that might've needed follow-up? 15:10:12 * roshi can't think of any 15:10:42 * sumantro concurs with roshi 15:10:43 I have: find bakery near adamw that will make and deliver unicorn poop 15:10:46 kinda weird 15:11:02 seems legit 15:11:07 status? 15:11:24 well, *that* explains what all those mystery deliveries were. 15:11:37 couldn't have been me I haven't made the attempt yet 15:11:52 i know, blame kparal 15:12:20 never heard of him 15:12:21 aaallllrighty then 15:12:24 moving on 15:12:55 #topic Fedora 26 status / retrospective 15:13:30 soo, we signed off RC-1.5 as the Fedora 26 release on Thursday - many thanks for all the testing, everyone 15:13:41 #info RC-1.5 was signed off as the Fedora 26 release on Thursday, thanks for all the work 15:13:55 as always, that will be released tomorrow 15:14:02 super job folks 15:14:35 a few things wound up getting fudged, a bit 15:14:38 when will updates start flowing? 15:14:43 yeah... 15:14:47 there should be a 0-day update push today, i guess 15:14:53 but check with releng to be sure 15:16:49 so the blocker meetings wound up with a couple of bugs that 'should have' been release blockers getting pushed out to be f27 blockers on the basis that they were discovered very late and weren't showstoppers, more or less 15:17:24 this isn't really something that's documented in the release process, but it *is* implied by the accepted idea that Fedora's release cycle is not based entirely on time or quality but a combination of both 15:17:37 thank the maker 15:17:45 so we also agreed to draft up some kind of formal encoding of this possibility in the release process docs somewhere 15:17:58 the hand waive clause? 15:18:01 so, first action item is: doing that. i'm willing to take it, or does anyone else want to? 15:18:07 roshi: let's call it that! 15:18:21 * roshi helped! 15:19:00 * sumantro will help! 15:21:05 ok, i'll draft the changes, and loop sumantro in - sounds OK? 15:21:11 ack 15:21:22 ack :) 15:22:18 #action adamw to draft 'hand waive clause' (thanks roshi) for release process documentation covering late-discovered blockers and loop sumantro in on the process 15:23:29 so there were two other things i'd say were obvious from the fudging: we aren't covering all the desktop testing early enough in the cycle, and we still have a few tests which don't get run often or at all due to people not having the prerequisites for the tests 15:23:49 it'd be good to do something about both of those problems, if we can 15:23:56 anyone have any thoughts or suggestions, first of all? 15:25:50 Maybe there should be a flag day if an included desktop app hasn't been tested, it gets pulled from composes? 15:26:13 maybe an alert if certain desktop tests haven't been ran in N composes? 15:26:15 If it's not getting testing, maybe it doesn't really need to be on the install media? (Exceptions no doubt exist.) 15:26:54 well, we don't really have per-app testing granularity anywhere 15:27:12 talk about the same in onboarding calls to help the new people start with desktop testing , if they are willing to. Maybe writing a post on how to get started with desktop testing.Also , have a fixed test day in the cycle. I dont know if these will be helpful , but these are my thoughts 15:27:14 or even a list of all the apps on the different DE's :p 15:27:16 how would we know which individual apps had been tested and which hadn't? would we have to start tracking that somewhere? 15:27:31 I suppose so 15:27:35 but we need a list first 15:27:39 sumantro: we did used to run a GNOME test day each cycle, we've never done KDE I don't think but we could 15:28:12 roshi: i kinda like the 'alert' idea, generalized out... 15:28:38 roshi: the testcase_stats code could be extended to a bot that sends out mails calling out tests that haven't been run in the cycle, or something 15:28:45 not 100% sure on the "how," but back of napkin math says we could do something with testcase_stats 15:28:51 yeah, it's possible 15:29:53 Yeah I guess assuming something is not tested until proving otherwise takes tracking, whereas assuming it works unless there's a bug that's also set as a blocker. 15:29:59 roshi: you feel like volunteering? you know you want to work with wikitcms ;) 15:30:00 * roshi can take a stab at that (though I'm on PTO two days this week, so time is limited) 15:30:14 just as a warning, might not get to it this week 15:30:34 cmurf: yeah - at present all we know is whether or not the desktop_menus test has been run, basically, so we can't say 'these X apps have been tested but these Y haven't' 15:30:39 roshi: cool 15:30:46 i'll file a ticket to monitor it 15:30:52 sounds good to me 15:31:09 #action roshi to look at setting up a 'missing test alert system' of some kind (for validation tests that are not getting run) 15:31:13 openqa doing a basic launch app and quit test? 15:31:20 everything listed in Activities 15:31:21 * satellit bare metal testing is important here also 15:31:43 Activities > Show Applications rather 15:32:09 cmurf: it's...difficult to implement 15:32:19 i've thought about it, but i don't really see how to do it 15:32:37 one problem is teaching openqa 'just open the menus and click on everything' 15:32:44 it's hard to infer the list, it'd be easier if we had a list of things 15:32:48 Seems quit is harder. 15:33:11 if we had a list, it could be "open overview, type these letters, hit enter" to ensure that the apps launch 15:33:25 the other problem is distinguishing between 'the app started normally', 'the app started then crashed', 'the app didn't start at all', and 'this menu entry is some weird special case' 15:33:32 Every icon'd application has a .desktop file right? 15:33:32 then there's the screen grab of any/every app successfully launching (aiui) 15:34:14 you know what we need 15:34:17 it's possible this could be automated with *something*, i'm just not sure openqa is that something... 15:34:18 a fuzzer for openqa 15:34:23 just start clicking on shit at random 15:34:44 but then you still have to have the needle to check that what's displayed is valid 15:34:45 bet we'd get a buttload of crashes 15:35:09 it doesn't know what right looks like unless we seed it with that first, is that right adamw ? 15:35:15 by definition any UI that can be clicked is valid or it should be grayed out / out of focus 15:35:34 it can't see anything, know if it's working correctly, only knows if it crashes 15:35:50 working correclty is going to take a human for now is my bet 15:36:27 plus there's cosmetic broken vs does not work at all broken 15:36:37 you could do...something like that, but we're a bit off track now 15:36:42 ok 15:37:13 so, any other thoughts? though we're running a bit short on time 15:37:16 * satellit link to a wiki page for each DE with list of applications to test with those 3 results? 15:37:35 the WG/SIG involved needs to keep that list up to date 15:37:50 for manual testing 15:37:53 it did occur to me that I used to send the manual compose announcements to mailing lists other than test-announce@ , but the bot doesn't do that, it only sends to that list (which is automatically forwarded to devel@) 15:38:15 the problem with having a 'list of applications to test' is keeping it from going stale 15:38:27 the desktops *do* change their application loadouts, quite often 15:38:33 yeah 15:38:51 and any time the method is 'someone should update the list manually', you can bet your bottom dollar it won't happen 15:38:53 that's why I think they should keep the list up to date themselves :p 15:39:08 as rbergeron says, wiki stands for 'where information kills itself' 15:39:13 lol 15:39:22 except in Archland 15:39:31 nerds 15:39:38 :p 15:40:17 roshi: i agree, maybe the list of apps should be split out as a group, so that the list is canonical 15:40:33 i.e. dnf group 15:40:39 * roshi wonders if it can be autogenerated post compose based on .desktop files 15:40:48 or that 15:40:48 that'd be even easier! 15:40:48 #action adamw to look at enhancing revalconsumer to send announcement mails to other lists 15:41:03 roshi: i was thinking about that, but it's a tad problematic 15:41:13 isn't everything? 15:41:14 you'd have to effectively rewrite a .desktop menu parser 15:41:38 because there are properties of .desktop files which can prevent them appearing in menus 15:41:39 not just `locate *.desktop` ? 15:42:02 there is, for instance, an item for 'desktops in which this app should appear' 15:42:09 (and one for 'desktops in which this app should not appear') 15:42:10 you might want to talk to pschindl, he experimented with a dogtail task that he stole from RHEL Desktop QE and that did all this and we tried to run it in taskotron. it mostly worked 15:42:28 there are .desktop files which aren't "really" for making something show up in the menus at all 15:42:39 yeah i see that, caribou-autostart.desktop 15:42:40 kparal: ah yeah, they have something like that, don't they 15:42:41 ah, that's fun :) 15:42:44 but it didn't work under wayland, because wayland doesn't support it yet, which was a serious showstopper 15:42:58 still, it'd be interesting to know what its approach was 15:43:08 flatpaks also do weird things with locate *.desktop 15:43:11 /var/lib/flatpak/runtime/org.gnome.Platform/x86_64/3.24/917abdce38d8852606b5ad7311052cdbd083b223f8c39d8567880b23af3c3e52/files/share/gnome/autostart/libcanberra-login-sound.desktop 15:43:35 anyway, time's a tickin' 15:43:41 please do think about this problem, folks 15:44:01 and if you can think of an idea that seems like it would help going forward, please send a mail to the list or bring it up in a future meeting 15:44:02 roshi.enqueu(other_problems) 15:44:49 if anyone can come up with a convincing way of producing a list of apps that are in the menus on each blocking desktop that'd be *great*, for instance, we could absolutely use that to make the desktop_menus results more granular 15:46:23 as far as 'tests that are hard to run' goes, i have a couple of ideas. we (the RH fedora QA team) may get some of the problematic hardware expensed and bought for the brno office 15:47:01 we can also look at just removing some of the tests (like i already proposed for xen, though it seems like it'll get a reprieve for now), and we could also look at a general-purpose clause about test coverage in the release process documentation 15:47:22 so, one more thing: 15:47:56 do we want to do a formal 'retrospective' on the wiki for f26? 15:48:02 we did one for f25 - https://fedoraproject.org/wiki/Fedora_25_QA_Retrospective - but no-one filled anything in 15:48:17 so it seems a waste to set one up for f26 if no-one's going to use it 15:48:49 honestly, between the 'hand waive clause,' the hardware expense, and desktop testing alerts, I think we've covered everything 15:48:52 did I miss something? 15:49:02 adamw, true 15:49:13 those are all the things I would bring up, but we've kinda covered those 15:49:46 anyone else? 15:50:41 alrighty then 15:51:00 #info for now we'll skip the wiki retrospective for f26 15:51:04 #topic Fedora 27 planning 15:51:14 so, we're a little short on time, so maybe we can come back to this in more detail next time 15:51:33 but let's just say F27 is shaping up to be a major cycle, with some really major changes proposed 15:51:50 and a short timeframe, iirc 15:51:58 is there any specific part of the f27 plans anyone wants to discuss urgently? 15:52:21 * roshi hasn't looked at it closely enough yet 15:52:53 for the record, i'm definitely planning to start looking at the necessary changes for Server right away...Server is planned to be built on Modularity for Fedora 27, which will be a big change 15:53:05 no kidding :p 15:53:07 * sumantro looks for changeset and test day planning 15:53:55 sumantro: a lot of the changes are still under review at present - a good place to follow them is on the devel@ list 15:54:03 i believe they won't show up on the ChangeSet page until approved 15:54:24 following devel already :) 15:54:49 but f27 modularity is surely a major one! 15:55:01 #info You can keep up with the proposed changes for Fedora 27 by following the devel@ list, look for mails with 'System Wide Change' and 'Self Contained Change' in the subject 15:55:04 it is for sure 15:55:09 there are others too 15:55:21 #info we will dive into F27 planning in more detail next week 15:55:33 #topic Test Day status 15:55:49 sumantro, where are we up to with test days? all complete for f26? 15:56:11 yes all complete , gnome 3.24 is happening right now :D 15:56:41 awesome 15:57:06 did you make sure to transfer results from all recent test days from the app back to the wiki? 15:57:27 roshi: are you doing heroes of fedora atm? i forget who is 15:57:37 yes I did and closed all the tickets on pagure :) 15:57:40 sumantro was going to take a crack at it this time around 15:57:49 iirc 15:58:08 yes adamw I wanted to take a shot at it :) 15:58:14 alrighty 15:58:21 can you two look at doing f26 HoF together, then? 15:58:33 yep, that's the plan 15:58:34 yes :D 15:58:35 #info Fedora 26 Test Day cycle is pretty much closed out, results have been transferred back to wiki 15:58:38 awesome 15:58:46 #action roshi and sumantro to write Heroes of Fedora posts for Fedora 26 15:59:13 call for test days for f27 will go out at the usual time, i guess? 15:59:22 I presume so 15:59:40 yes adamw :) 15:59:42 alrighty 15:59:46 #topic Open floor 15:59:50 any other business, very quickly? 16:00:01 #info No Blocker Review today 16:00:04 Whee! 16:00:17 yaaay 16:00:24 yaaaaaaaaaaaay 16:00:50 yaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaay 16:01:32 print 'y' + 'a' * one_more_than_anyone_else + 'y' 16:01:36 =) 16:01:47 from future import oneupper 16:01:48 :p 16:01:49 roshi: let's both run that, it doesn't seem like it could cause any problems 16:02:11 alrighty then, sounds like we're done here, folks. many thanks again for all the work on fedora 26! great job everyone 16:02:12 should probably use the cloud, so we can scale :p 16:02:27 kparal: of course we should make sure commonbugs is updated...have you worked on it yet? 16:03:01 * adamw sets fuse 16:03:15 * roshi goes to refill coffee 16:03:51 * sumantro goes out to click pictures of moon 16:04:00 that's no moon! 16:04:15 ! 16:04:52 Sorry, now that F26 is ready, can I ask how I request testing for a remix (hopefully future spin)? 16:04:59 alrighty, thanks again for all the f26 work and thanks for coming, folks 16:05:01 #endmeeting