15:01:40 <adamw> #startmeeting Fedora QA meeting
15:01:40 <zodbot> Meeting started Mon Jul 10 15:01:40 2017 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:40 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
15:01:43 <adamw> #meetingname fedora-qa
15:01:44 <zodbot> The meeting name has been set to 'fedora-qa'
15:01:47 <adamw> #topic Roll Call
15:01:52 <adamw> ahoyhoy qa folks, who's around for a meeting?
15:03:12 * sumantro is here
15:03:15 * garretraziel is here
15:03:16 * satellit listening
15:03:17 * kparal is here
15:04:26 <adamw> hi cmurf
15:04:47 <adamw> how's everyone this sunny/not sunny (delete as appropriate) morning/afternoon/evening (ditto)?
15:05:31 <cmurf> 60F and somewhat hung over
15:05:55 <adamw> yowch
15:06:07 <sumantro> in mycase moonlit evening :D
15:06:17 <adamw> i guess i'd better turn off my pneumatic drill then, huh
15:06:35 <adamw> sumantro: nice!
15:06:35 * roshi is here
15:06:50 <cmurf> 15C is nice
15:06:52 <adamw> alrighty, let's get started with...
15:07:06 <adamw> #topic Previous meeting follow-up
15:07:07 <cmurf> at least we're not on fire
15:07:32 <adamw> you know, that's a useful mantra for almost any situation.
15:07:37 * cmurf is momentarily in this realm
15:07:39 <adamw> when you're on fire, substitute "at least we're not drowning".
15:07:45 <roshi> lol
15:08:12 <cmurf> haha well California is on fire, and ealier this spring they were drowning
15:08:27 <adamw> if you're on fire *and* drowning, check if you're wearing a red shirt and work for Starfleet.
15:08:51 <roshi> that *does* fit with the headcannon
15:09:19 <cmurf> we need another volunteer
15:09:21 <adamw> headcannon sound like a fine idea, but how do you deal with the recoil?
15:09:38 <adamw> welp, it appears we had no action items last time
15:09:43 <adamw> #info no action items from last meeting
15:09:54 <adamw> can anyone remember anything other than action items that might've needed follow-up?
15:10:12 * roshi can't think of any
15:10:42 * sumantro concurs with roshi
15:10:43 <cmurf> I have: find bakery near adamw that will make and deliver unicorn poop
15:10:46 <cmurf> kinda weird
15:11:02 <roshi> seems legit
15:11:07 <roshi> status?
15:11:24 <adamw> well, *that* explains what all those mystery deliveries were.
15:11:37 <cmurf> couldn't have been me I haven't made the attempt yet
15:11:52 <cmurf> i know, blame kparal
15:12:20 <kparal> never heard of him
15:12:21 <adamw> aaallllrighty then
15:12:24 <adamw> moving on
15:12:55 <adamw> #topic Fedora 26 status / retrospective
15:13:30 <adamw> soo, we signed off RC-1.5 as the Fedora 26 release on Thursday - many thanks for all the testing, everyone
15:13:41 <adamw> #info RC-1.5 was signed off as the Fedora 26 release on Thursday, thanks for all the work
15:13:55 <adamw> as always, that will be released tomorrow
15:14:02 <cmurf> super job folks
15:14:35 <adamw> a few things wound up getting fudged, a bit
15:14:38 <cmurf> when will updates start flowing?
15:14:43 <roshi> yeah...
15:14:47 <adamw> there should be a 0-day update push today, i guess
15:14:53 <adamw> but check with releng to be sure
15:16:49 <adamw> so the blocker meetings wound up with a couple of bugs that 'should have' been release blockers getting pushed out to be f27 blockers on the basis that they were discovered very late and weren't showstoppers, more or less
15:17:24 <adamw> this isn't really something that's documented in the release process, but it *is* implied by the accepted idea that Fedora's release cycle is not based entirely on time or quality but a combination of both
15:17:37 <cmurf> thank the maker
15:17:45 <adamw> so we also agreed to draft up some kind of formal encoding of this possibility in the release process docs somewhere
15:17:58 <roshi> the hand waive clause?
15:18:01 <adamw> so, first action item is: doing that. i'm willing to take it, or does anyone else want to?
15:18:07 <adamw> roshi: let's call it that!
15:18:21 * roshi helped!
15:19:00 * sumantro will help!
15:21:05 <adamw> ok, i'll draft the changes, and loop sumantro in - sounds OK?
15:21:11 <roshi> ack
15:21:22 <sumantro> ack :)
15:22:18 <adamw> #action adamw to draft 'hand waive clause' (thanks roshi) for release process documentation covering late-discovered blockers and loop sumantro in on the process
15:23:29 <adamw> so there were two other things i'd say were obvious from the fudging: we aren't covering all the desktop testing early enough in the cycle, and we still have a few tests which don't get run often or at all due to people not having the prerequisites for the tests
15:23:49 <adamw> it'd be good to do something about both of those problems, if we can
15:23:56 <adamw> anyone have any thoughts or suggestions, first of all?
15:25:50 <cmurf> Maybe there should be a flag day if an included desktop app hasn't been tested, it gets pulled from composes?
15:26:13 <roshi> maybe an alert if certain desktop tests haven't been ran in N composes?
15:26:15 <cmurf> If it's not getting testing, maybe it doesn't really need to be on the install media? (Exceptions no doubt exist.)
15:26:54 <adamw> well, we don't really have per-app testing granularity anywhere
15:27:12 <sumantro> talk about the same in onboarding calls to help the new people start with desktop testing , if they are willing to. Maybe writing a post on how to get started with desktop testing.Also , have a fixed test day in the cycle. I dont know if these will be helpful , but these are my thoughts
15:27:14 <roshi> or even a list of all the apps on the different DE's :p
15:27:16 <adamw> how would we know which individual apps had been tested and which hadn't? would we have to start tracking that somewhere?
15:27:31 <roshi> I suppose so
15:27:35 <roshi> but we need a list first
15:27:39 <adamw> sumantro: we did used to run a GNOME test day each cycle, we've never done KDE I don't think but we could
15:28:12 <adamw> roshi: i kinda like the 'alert' idea, generalized out...
15:28:38 <adamw> roshi: the testcase_stats code could be extended to a bot that sends out mails calling out tests that haven't been run in the cycle, or something
15:28:45 <roshi> not 100% sure on the "how," but back of napkin math says we could do something with testcase_stats
15:28:51 <adamw> yeah, it's possible
15:29:53 <cmurf> Yeah I guess assuming something is not tested until proving otherwise takes tracking, whereas assuming it works unless there's a bug that's also set as a blocker.
15:29:59 <adamw> roshi: you feel like volunteering? you know you want to work with wikitcms ;)
15:30:00 * roshi can take a stab at that (though I'm on PTO two days this week, so time is limited)
15:30:14 <roshi> just as a warning, might not get to it this week
15:30:34 <adamw> cmurf: yeah - at present all we know is whether or not the desktop_menus test has been run, basically, so we can't say 'these X apps have been tested but these Y haven't'
15:30:39 <adamw> roshi: cool
15:30:46 <adamw> i'll file a ticket to monitor it
15:30:52 <roshi> sounds good to me
15:31:09 <adamw> #action roshi to look at setting up a 'missing test alert system' of some kind (for validation tests that are not getting run)
15:31:13 <cmurf> openqa doing a basic launch app and quit test?
15:31:20 <cmurf> everything listed in Activities
15:31:21 * satellit bare metal testing is important here also
15:31:43 <cmurf> Activities > Show Applications rather
15:32:09 <adamw> cmurf: it's...difficult to implement
15:32:19 <adamw> i've thought about it, but i don't really see how to do it
15:32:37 <adamw> one problem is teaching openqa 'just open the menus and click on everything'
15:32:44 <roshi> it's hard to infer the list, it'd be easier if we had a list of things
15:32:48 <cmurf> Seems quit is harder.
15:33:11 <roshi> if we had a list, it could be "open overview, type these letters, hit enter" to ensure that the apps launch
15:33:25 <adamw> the other problem is distinguishing between 'the app started normally', 'the app started then crashed', 'the app didn't start at all', and 'this menu entry is some weird special case'
15:33:32 <cmurf> Every icon'd application has a .desktop file right?
15:33:32 <roshi> then there's the screen grab of any/every app successfully launching (aiui)
15:34:14 <cmurf> you know what we need
15:34:17 <adamw> it's possible this could be automated with *something*, i'm just not sure openqa is that something...
15:34:18 <cmurf> a fuzzer for openqa
15:34:23 <cmurf> just start clicking on shit at random
15:34:44 <roshi> but then you still have to have the needle to check that what's displayed is valid
15:34:45 <cmurf> bet we'd get a buttload of crashes
15:35:09 <roshi> it doesn't know what right looks like unless we seed it with that first, is that right adamw ?
15:35:15 <cmurf> by definition any UI that can be clicked is valid or it should be grayed out / out of focus
15:35:34 <cmurf> it can't see anything, know if it's working correctly, only knows if it crashes
15:35:50 <cmurf> working correclty is going to take a human for now is my bet
15:36:27 <cmurf> plus there's cosmetic broken vs does not work at all broken
15:36:37 <adamw> you could do...something like that, but we're a bit off track now
15:36:42 <cmurf> ok
15:37:13 <adamw> so, any other thoughts? though we're running a bit short on time
15:37:16 * satellit link to a wiki page for each DE with list of applications to test with those 3 results?
15:37:35 <roshi> the WG/SIG involved needs to keep that list up to date
15:37:50 <satellit> for manual testing
15:37:53 <adamw> it did occur to me that I used to send the manual compose announcements to mailing lists other than test-announce@ , but the bot doesn't do that, it only sends to that list (which is automatically forwarded to devel@)
15:38:15 <adamw> the problem with having a 'list of applications to test' is keeping it from going stale
15:38:27 <adamw> the desktops *do* change their application loadouts, quite often
15:38:33 <roshi> yeah
15:38:51 <adamw> and any time the method is 'someone should update the list manually', you can bet your bottom dollar it won't happen
15:38:53 <roshi> that's why I think they should keep the list up to date themselves :p
15:39:08 <adamw> as rbergeron says, wiki stands for 'where information kills itself'
15:39:13 <roshi> lol
15:39:22 <roshi> except in Archland
15:39:31 <adamw> nerds
15:39:38 <roshi> :p
15:40:17 <cmurf> roshi: i agree, maybe the list of apps should be split out as a group, so that the list is canonical
15:40:33 <cmurf> i.e. dnf group
15:40:39 * roshi wonders if it can be autogenerated post compose based on .desktop files
15:40:48 <cmurf> or that
15:40:48 <roshi> that'd be even easier!
15:40:48 <adamw> #action adamw to look at enhancing revalconsumer to send announcement mails to other lists
15:41:03 <adamw> roshi: i was thinking about that, but it's a tad problematic
15:41:13 <roshi> isn't everything?
15:41:14 <adamw> you'd have to effectively rewrite a .desktop menu parser
15:41:38 <adamw> because there are properties of .desktop files which can prevent them appearing in menus
15:41:39 <roshi> not just `locate *.desktop` ?
15:42:02 <adamw> there is, for instance, an item for 'desktops in which this app should appear'
15:42:09 <adamw> (and one for 'desktops in which this app should not appear')
15:42:10 <kparal> you might want to talk to pschindl, he experimented with a dogtail task that he stole from RHEL Desktop QE and that did all this and we tried to run it in taskotron. it mostly worked
15:42:28 <adamw> there are .desktop files which aren't "really" for making something show up in the menus at all
15:42:39 <cmurf> yeah i see that, caribou-autostart.desktop
15:42:40 <adamw> kparal: ah yeah, they have something like that, don't they
15:42:41 <roshi> ah, that's fun :)
15:42:44 <kparal> but it didn't work under wayland, because wayland doesn't support it yet, which was a serious showstopper
15:42:58 <adamw> still, it'd be interesting to know what its approach was
15:43:08 <cmurf> flatpaks also do weird things with locate *.desktop
15:43:11 <cmurf> /var/lib/flatpak/runtime/org.gnome.Platform/x86_64/3.24/917abdce38d8852606b5ad7311052cdbd083b223f8c39d8567880b23af3c3e52/files/share/gnome/autostart/libcanberra-login-sound.desktop
15:43:35 <adamw> anyway, time's a tickin'
15:43:41 <adamw> please do think about this problem, folks
15:44:01 <adamw> and if you can think of an idea that seems like it would help going forward, please send a mail to the list or bring it up in a future meeting
15:44:02 <roshi> roshi.enqueu(other_problems)
15:44:49 <adamw> if anyone can come up with a convincing way of producing a list of apps that are in the menus on each blocking desktop that'd be *great*, for instance, we could absolutely use that to make the desktop_menus results more granular
15:46:23 <adamw> as far as 'tests that are hard to run' goes, i have a couple of ideas. we (the RH fedora QA team) may get some of the problematic hardware expensed and bought for the brno office
15:47:01 <adamw> we can also look at just removing some of the tests (like i already proposed for xen, though it seems like it'll get a reprieve for now), and we could also look at a general-purpose clause about test coverage in the release process documentation
15:47:22 <adamw> so, one more thing:
15:47:56 <adamw> do we want to do a formal 'retrospective' on the wiki for f26?
15:48:02 <adamw> we did one for f25 - https://fedoraproject.org/wiki/Fedora_25_QA_Retrospective - but no-one filled anything in
15:48:17 <adamw> so it seems a waste to set one up for f26 if no-one's going to use it
15:48:49 <roshi> honestly, between the 'hand waive clause,' the hardware expense, and desktop testing alerts, I think we've covered everything
15:48:52 <roshi> did I miss something?
15:49:02 <sumantro> adamw, true
15:49:13 <roshi> those are all the things I would bring up, but we've kinda covered those
15:49:46 <adamw> anyone else?
15:50:41 <adamw> alrighty then
15:51:00 <adamw> #info for now we'll skip the wiki retrospective for f26
15:51:04 <adamw> #topic Fedora 27 planning
15:51:14 <adamw> so, we're a little short on time, so maybe we can come back to this in more detail next time
15:51:33 <adamw> but let's just say F27 is shaping up to be a major cycle, with some really major changes proposed
15:51:50 <roshi> and a short timeframe, iirc
15:51:58 <adamw> is there any specific part of the f27 plans anyone wants to discuss urgently?
15:52:21 * roshi hasn't looked at it closely enough yet
15:52:53 <adamw> for the record, i'm definitely planning to start looking at the necessary changes for Server right away...Server is planned to be built on Modularity for Fedora 27, which will be a big change
15:53:05 <roshi> no kidding :p
15:53:07 * sumantro looks for changeset and test day planning
15:53:55 <adamw> sumantro: a lot of the changes are still under review at present - a good place to follow them is on the devel@ list
15:54:03 <adamw> i believe they won't show up on the ChangeSet page until approved
15:54:24 <sumantro> following devel already :)
15:54:49 <sumantro> but f27 modularity is surely a major one!
15:55:01 <adamw> #info You can keep up with the proposed changes for Fedora 27 by following the devel@ list, look for mails with 'System Wide Change' and 'Self Contained Change' in the subject
15:55:04 <adamw> it is for sure
15:55:09 <adamw> there are others too
15:55:21 <adamw> #info we will dive into F27 planning in more detail next week
15:55:33 <adamw> #topic Test Day status
15:55:49 <adamw> sumantro, where are we up to with test days? all complete for f26?
15:56:11 <sumantro> yes all complete , gnome 3.24 is happening right now :D
15:56:41 <adamw> awesome
15:57:06 <adamw> did you make sure to transfer results from all recent test days from the app back to the wiki?
15:57:27 <adamw> roshi: are you doing heroes of fedora atm? i forget who is
15:57:37 <sumantro> yes I did and closed all the tickets on pagure :)
15:57:40 <roshi> sumantro was going to take a crack at it this time around
15:57:49 <roshi> iirc
15:58:08 <sumantro> yes adamw I wanted to take a shot at it :)
15:58:14 <adamw> alrighty
15:58:21 <adamw> can you two look at doing f26 HoF together, then?
15:58:33 <roshi> yep, that's the plan
15:58:34 <sumantro> yes :D
15:58:35 <adamw> #info Fedora 26 Test Day cycle is pretty much closed out, results have been transferred back to wiki
15:58:38 <adamw> awesome
15:58:46 <adamw> #action roshi and sumantro to write Heroes of Fedora posts for Fedora 26
15:59:13 <adamw> call for test days for f27 will go out at the usual time, i guess?
15:59:22 <roshi> I presume so
15:59:40 <sumantro> yes adamw :)
15:59:42 <adamw> alrighty
15:59:46 <adamw> #topic Open floor
15:59:50 <adamw> any other business, very quickly?
16:00:01 <roshi> #info No Blocker Review today
16:00:04 <roshi> Whee!
16:00:17 <adamw> yaaay
16:00:24 <sumantro> yaaaaaaaaaaaay
16:00:50 <adamw> yaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaay
16:01:32 <roshi> print 'y' + 'a' * one_more_than_anyone_else + 'y'
16:01:36 <adamw> =)
16:01:47 <roshi> from future import oneupper
16:01:48 <roshi> :p
16:01:49 <adamw> roshi: let's both run that, it doesn't seem like it could cause any problems
16:02:11 <adamw> alrighty then, sounds like we're done here, folks. many thanks again for all the work on fedora 26! great job everyone
16:02:12 <roshi> should probably use the cloud, so we can scale :p
16:02:27 <adamw> kparal: of course we should make sure commonbugs is updated...have you worked on it yet?
16:03:01 * adamw sets fuse
16:03:15 * roshi goes to refill coffee
16:03:51 * sumantro goes out to click pictures of moon
16:04:00 <adamw> that's no moon!
16:04:15 <x3mboy> !
16:04:52 <x3mboy> Sorry, now that F26 is ready, can I ask how I request testing for a remix (hopefully future spin)?
16:04:59 <adamw> alrighty, thanks again for all the f26 work and thanks for coming, folks
16:05:01 <adamw> #endmeeting