15:00:04 <adamw> #startmeeting Fedora QA Meeting
15:00:04 <zodbot> Meeting started Mon Mar 22 15:00:04 2021 UTC.
15:00:04 <zodbot> This meeting is logged and archived in a public location.
15:00:04 <zodbot> The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:04 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:04 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
15:00:12 <adamw> #meetingname fedora-qa
15:00:12 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:17 <adamw> #topic Roll call
15:00:23 <tflink> morning
15:00:25 <adamw> morning folks! time for super qa fun
15:00:35 <adamw> as always, fun is not guaranteed
15:00:43 <copperi_> .hello2
15:00:45 <zodbot> copperi_: Sorry, but you don't exist
15:00:46 <tflink> there's always a disclaimer
15:01:00 <adamw> existence is also not guaranteed
15:01:03 <coremodule> .hello2
15:01:04 <zodbot> coremodule: coremodule 'Geoffrey Marr' <gmarr@redhat.com>
15:01:13 <adamw> if you find yourself struggling with existence, please consult a philosopher immediately
15:01:14 <Southern_Gentlem> .hello jbwillia
15:01:16 <zodbot> Southern_Gentlem: jbwillia 'Ben Williams' <vaioof@gmail.com>
15:01:20 <copperi_> I like being a ghost
15:01:36 <adamw> who ya gonna call
15:01:48 <tablepc> .hello2
15:01:48 <zodbot> tablepc: tablepc 'Pat Kelly' <pmkellly@frontier.com>
15:02:11 * kparal is here
15:02:46 <tablepc> Top of the Mornin' everyone!
15:03:39 <adamw> good to see everyone
15:04:05 <tablepc> We've got a Sunny day in the 70s later
15:04:10 <cmurf> .hello
15:04:10 <zodbot> cmurf: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
15:04:17 <cmurf> .hello chrismurphy
15:04:18 <zodbot> cmurf: chrismurphy 'Chris Murphy' <bugzilla@colorremedies.com>
15:04:46 <cmurf> maybe fas replacement will sort out aliases better
15:04:46 <copperi_> .hello copperi
15:04:48 <zodbot> copperi_: copperi 'Jan Kuparinen' <copper_fin@hotmail.com>
15:05:15 <adamw> alrighty, let's get going
15:05:24 <adamw> #topic Previous meeting follow-up
15:05:41 <adamw> i got no action items here
15:05:43 <adamw> anyone have anything?
15:08:06 <adamw> alrighty then
15:08:19 <adamw> #topic Fedora 34 Beta prep and status
15:08:27 <Southern_Gentlem> got a report this morning in #fedora someone dnf upgraded from f33 to f34 had grub issues
15:08:30 <adamw> #info Beta RC3 was signed off as the Beta release last week
15:08:44 <adamw> #info it will be released tomorrow morning
15:08:56 <adamw> Southern_Gentlem: was there any detail?
15:09:41 <tablepc> From my testing it's look lots better now. Only the SELinux big still hanging around.
15:09:49 <Southern_Gentlem> lvm
15:10:03 <cmurf> \o/ congratulations! we're having a beta!
15:10:14 <Southern_Gentlem> i pointed them to #fedora-qa and i dont see where they posted anything
15:10:54 <adamw> Southern_Gentlem: hmm, so potentially concerning but not a lot we can do
15:11:01 <adamw> i'll poke around bug reports and forums a bit
15:11:09 <Southern_Gentlem> <n3p> luks+lvm+ root (ext4) + home (btrfs) (its a mess bc it's an install fc32
15:11:10 <cmurf> needs more info
15:11:25 <Southern_Gentlem> cmurf i talking to them in #fedora now
15:11:32 <adamw> people are gonna hit some big updates on first update after install (inc. GNOME 40 RC1 and systemd rc4) but afaict those are working okay
15:11:36 <adamw> anyone had any trouble with them?
15:11:54 * sumantro is here
15:12:18 <tablepc> Yeah over 300 packages this morning
15:12:51 <tablepc> they ran fine and rebooted okay, haven't check all my usual stuff yet.
15:14:52 <adamw> we have a few bugs to document in commonbugs, i'll work on that today
15:15:13 <adamw> if you're aware of anything likely to be visible to many users which isn't tagged yet, please add the keyword CommonBugs to it
15:16:57 <cmurf> oh need to remove commonbugs in that btrfs /boot bug since we got the newer anaconda after all
15:17:17 <adamw> that's easy enough, i always check whether the issue's fixed when i go through the list
15:17:36 <adamw> so, sounds like we're pretty good on the f34 front?
15:19:08 <Southern_Gentlem> atm
15:19:36 <tablepc> I'd say so.
15:20:14 <adamw> roger roger
15:20:29 <adamw> #topic Test Day / community event status
15:20:32 <sumantro> Kernel Test Week ended up recording 415 test runs across 154 testers on 5.11 which is shipping with F34 Beta this week.
15:20:33 <adamw> sumantro, are you here?
15:20:41 <linuxmodder> .hello linuxmodder
15:20:42 <zodbot> linuxmodder: linuxmodder 'Corey W Sheldon' <sheldon.corey@gmail.com>
15:20:44 <adamw> yes he is!
15:20:59 <adamw> #info Kernel Test Week ended up recording 415 test runs across 154 testers on 5.11 which is shipping with F34 Beta this week.
15:21:00 <sumantro> GNOME 40 test day was successful with a good amount of coverage with 40+ testers testing
15:21:10 <sumantro> Shout out to the downstream desktop-qe team for participating as well!
15:21:11 <adamw> #info GNOME 40 test day was successful with a good amount of coverage with 40+ testers testing
15:21:19 <adamw> i saw several bug reports for that too
15:21:22 <adamw> (and helped fix a couple :>)
15:21:25 <sumantro> I18n Test week was successful with a lot of testers testing the new changesets https://testdays.fedoraproject.org/events/101
15:21:42 <adamw> #info I18n Test week was successful with a lot of testers testing the new changesets
15:21:44 <sumantro> OOMd test week is ongoing till 2021-03-25 https://testdays.fedoraproject.org/events/105
15:22:49 <adamw> #info systemd-oomd test week is ongoing till 2021-03-25: https://fedoraproject.org/wiki/Test_Day:2021-03-18_Systemd-OOMd_Test_Week
15:22:52 <sumantro> GRUB test day might happen on 25th https://pagure.io/fedora-qa/issue/663
15:23:04 <cmurf> oh yeah need to check on that
15:23:10 <adamw> what's with "might happen"? waiting on devs?
15:23:13 <linuxmodder> sumantro: I may reach out to you on the oom stuff post meeting
15:23:21 <cmurf> no signed shim yet
15:23:28 <sumantro> linuxmodder, sure :)
15:23:49 <cmurf> do we want to have a grub 2.06 only test day? *shrug* and a separate shim test day once it's signed?
15:24:09 <cmurf> or combine them since they're both bootloaders
15:24:21 <sumantro> Combining them sounds like a plan
15:24:55 <adamw> yeah, i'd agree with combined
15:24:56 <sumantro> adamw, can you please help me nail a date for upgrade test day?
15:25:13 <cmurf> once there's a signed shim incoming we'll schedule the test day
15:25:35 <adamw> suman: sure, here and now? in the ticket?
15:25:53 <sumantro> adamw, in the ticket :) I have tagged you :D
15:26:04 <adamw> thanks
15:26:22 <sumantro> adamw, note that we will have another IoT and Audio Test day sometime soon
15:26:42 <sumantro> also, I am planning to nail down virt test day as well between now and final
15:27:09 <tablepc> Pipe wore's been working fine for me Even with OBS Studio
15:27:21 <tablepc> wire
15:27:22 <adamw> should we talk a bit about nitrate here?
15:28:05 <sumantro> adamw, Kiwi but yes,
15:28:40 <sumantro> lruzicka[m], if you are around, we would love to hear your feedback on experience during the last IoT Test Day
15:28:48 <adamw> oh right. names! names are hard
15:29:21 <adamw> #info we're looking into using the Kiwi TCMS system for test days (instead of wiki/testdays tool), there was a limited trial at the IoT Test Day
15:29:38 <adamw> https://kiwitcms.org/
15:29:43 <sumantro> adamw, you summed it for me :)
15:29:47 <sumantro> thanks :)
15:31:45 <sumantro> adamw, one more thing, lbrabec is mentoring for "QA dashboard" alongwith Josef, for Outreachy and we have people introducing themselves in the @test list
15:31:51 <sumantro> https://www.outreachy.org/communities/cfp/fedora/
15:31:52 <adamw> right, indeed
15:32:18 <adamw> please welcome the outreachy folks
15:33:04 <sumantro> thats all from my side :)
15:35:32 <coremodule> after porting the IoT tests over manually to KiwiTCMS, I will say that the editor has less formatting options that the current wiki system does; things like italics, bold font, etc... so *if* we chose to go with Kiwi, the current testcases could not be copy/paste ported over, they would each need special attention and perhaps not have all of the fun features we currently have (notes, warnings, etc)
15:35:59 <coremodule> it was more limited than the wiki
15:36:02 <coremodule> in that regard
15:36:08 <tflink> yeah, it's a markup
15:36:17 <adamw> you'd think at least basic text formatting could be implemented
15:36:19 <tflink> er, markdown. brain not yet working this morning
15:36:22 <adamw> that at least would be worth a feature request, i think
15:36:26 <tflink> it is, from my experience
15:36:29 <adamw> markdown supports bold and italics...:D
15:36:40 <adamw> the big thing that would be hard to get is dynamic templating.
15:36:47 <u9000[m]> did you write it like _this_ or like *this*
15:37:17 <coremodule> sorry, that was a bad example, bold/italics are supported, as well as hyperlinks, but that was about it. a single list feature and a single left/right/center feature and that was all.
15:37:23 <coremodule> I have some feedback from lruzicka[m]
15:37:31 <coremodule> "What bothered me most was, when you create a test run to test the tests, you cannot provide more results to the test run, just one. When you attempt to provide more, the later result will overwrite it."
15:37:32 <tflink> another one is numbering. it doesn't seem to support the 1. auto numbering - you have to number all the steps which while annoying, probably isn't quite a deal breaker
15:37:56 <coremodule> "So for each tester, there would have to be a way to create their own test run and fill-in the results."
15:38:07 <coremodule> "If that should replace the matrices, also a different test run would have to be created for each user that wants to report their result, otherwise only one test result will be recorded."
15:38:16 <tflink> coremodule, lruzicka[m]: I don't understand why that's a problem. can you elaborate a bit more (if we want to discuss it here)
15:38:32 <tflink> the part about only allowing one result per user per version
15:39:35 <adamw> i think it's one result per test
15:39:38 <adamw> per run
15:39:43 <adamw> so we have to create lots of runs
15:39:49 <sumantro> adamw, yes
15:40:07 <adamw> and the system envisages a run as being created by some sort of admin/supervisor, right? not self service
15:40:23 <coremodule> adamw, yes, an "admin" creats the runs
15:40:25 <tflink> hrm, I must be misunderstanding something. that's not how I was using it when I poked at a local instance
15:41:05 <tflink> it looked like anyone with testing privileges can create a run that consists of multiple test cases
15:42:01 <tablepc> Why switch? What we have seems to work.
15:42:02 * tflink shrugs - if I'm the only one who saw that, I'm proably misunderstanding something
15:43:06 <coremodule> tflink, I bet you are right on user privileges. I can't recall that specifically, and I think each of the user accounts that were made all had the same access, so we didn't run into that
15:43:06 <adamw> tablepc: the idea is to use a 'proper system' rather than the kinda hacked-together approach of wiki pages plus tooling around them that we maintain
15:43:07 <tflink> tablepc: I think the question is whether kiwi would work better
15:43:47 <adamw> limitations of the wiki system are that it can be hard to work with, easy to make mistakes in, hard to query without writing stupid tools to do the querying, plus at this point infra kinda wants to not have a wiki any more and "but qa built an entire testing system in it" is #1 reason it's still around :P
15:43:53 <tflink> coremodule: from what I saw, there were pretty fine-grained permissions that could be granted to groups of users
15:44:09 <tablepc> Okay I didn't realize that the current system was a maintneance problem.
15:44:12 <coremodule> that makes sense
15:44:22 <adamw> but we've tried this a few times now and it's never worked out, because it does always seem to turn out that the stupid hacked together wiki system works better than the replacements
15:44:39 * tflink spent a bit of time poking at how hard it would be to port our existing test cases/matrices into kiwi
15:44:51 <sumantro> tflink
15:44:53 <tflink> port via code
15:44:54 <sumantro> tflink++
15:44:54 <zodbot> sumantro: Karma for tflink changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:45:51 <adamw> tablepc: you should see the code behind the testcase_stats pages
15:45:54 <adamw> it's one of those things where even the people who wrote it don't remember how it works any more :P
15:46:20 <adamw> https://pagure.io/fedora-qa/relval/blob/master/f/relval/testcase_stats.py#_228 is my favorite bit
15:46:49 <tablepc> That would be a problem.
15:46:52 <tflink> FWIW, it doesn't look like it'd be too hard to move everything if we chose to do so. kiwi's API is a little old-feeling and the documentation isn't terribly intuitive but it does seem to work
15:47:54 <u9000[m]> omg there's seven paragraphs of comments explaining what it does
15:47:56 <tflink> but I didn't get too deep into it - there are probably corner cases that I didn't hit
15:48:21 <adamw> u9000: yup, that's always a good sign
15:48:29 <adamw> anyhow, yeah, that's the situation :)
15:48:36 <adamw> so there'll probably be wider trials of kiwi coming up
15:48:37 <sumantro> adamw, tflink also the current test day app cant handle result submission for the same test case on different arch .. until we create weird redirects on wiki
15:49:28 <linuxmodder> sumantro:  so you can't do several at once ?
15:49:55 <tflink> sumantro: isn't that a limitation of testdays, not the wiki?
15:50:20 <u9000[m]> worth noting is the paragraphs begin with "Crazy magic voodoo", meaning any vulnerabilities in there are unlikely to be noticed/understood
15:50:22 <linuxmodder> maybe both tflink
15:50:47 <tablepc> Sounds like kiwi was designed for testing of relatively small sofware sets
15:50:51 <linuxmodder> valid point u9000[m] albeit scary to think about
15:51:03 <tflink> kiwi was originally designed to test RHEL, actually
15:51:19 <adamw> u9000: to be clear, i wrote most of it, and i more or less remember how it works :P but it'd be nice not to have to any more
15:51:45 <tflink> as far as I know. it was developed inside of RH and open sourced. some of the original authors forked it into kiwi and created a new company around it
15:51:48 <tablepc> Things seem to have changed
15:52:06 <adamw> the chance of security vulns in that specific code is low as it's just reasoning about internal representations it already constructed there. there's much more chance of security vulns in the next bit that constructs html (we already fixed some there).
15:52:37 <adamw> tablepc: not really, it's more that rhel qa works much like, well, most industry 'qa'
15:52:49 <adamw> there is a small team of professional testers with a few supervisors/managers deciding what they'll do
15:53:09 <adamw> so the original design of this system was based around the test managers creating work lists for the testers, essentially
15:53:23 <adamw> then giving the test managers visualization tools to see the combined results
15:53:37 <tablepc> Okay tight focused testing with emplouee crew.
15:54:19 <adamw> still, it is open source, and the devs are actively working with us on it, so we can get things opened up, potentially.
15:55:22 <tablepc> Soulds like a project worth doing to get something more maintenance friendly
15:55:45 <adamw> anyhow, yeah, just wanted to make sure folks know about that
15:55:48 <adamw> let's do a quick:
15:55:51 <adamw> #topic Open floor
15:55:53 <adamw> any other business?
15:56:18 <tablepc> Well testers are used to suffering through development cycles.
15:57:47 <tablepc> Waiting for my new PC to come back. I'll keep you posted on the list about the results.
15:57:49 <bcotton> i was going to bring up the g-i-s proposal, but i don't think there's time to discuss it. the consensus seemed generally fine with it?
15:58:42 <adamw> what's the proposal?
15:59:02 <bcotton> #link https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org/thread/OSN4LTQ2BSUJQL7KIB67YNIDHSLD7I5B/
15:59:19 <adamw> oh, the criterion proposal. yes, broadyl
15:59:26 <cmurf> yes
15:59:42 <tablepc> Yes, I'm for it.
15:59:46 <adamw> i don't remember if i did a detailed reply yet, but if not i'll try to do one soon
15:59:52 <adamw> if anyone else has opinions on that, please chip in
16:00:36 <adamw> alrighty folks, that's time
16:00:41 <adamw> thanks a lot for coming everyone
16:01:11 <tablepc> Have a Great Day Everyone
16:01:17 <adamw> i didn't announce a blocker review meeting this week, but please do vote in the blocker review tickets - you can find the ticket links in the blockerbugs app at https://qa.fedoraproject.org/blockerbugs/milestone/34/final/buglist
16:01:40 <adamw> #endmeeting