15:00:18 <adamw> #startmeeting Fedora QA Meeting
15:00:19 <zodbot> Meeting started Mon Jul  6 15:00:18 2020 UTC.
15:00:19 <zodbot> This meeting is logged and archived in a public location.
15:00:19 <zodbot> The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:19 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:19 <zodbot> The meeting name has been set to 'fedora_qa_meeting'
15:00:22 <adamw> #meetingname fedora-qa
15:00:22 <zodbot> The meeting name has been set to 'fedora-qa'
15:00:27 <adamw> #topic Roll call
15:00:32 <adamw> top o' the morning to you too, tablepc
15:01:17 <tablepc> .hello 2
15:01:18 <zodbot> tablepc: Sorry, but you don't exist
15:01:29 <tablepc> .hello2
15:01:30 <zodbot> tablepc: tablepc 'Pat Kelly' <pmkellly@frontier.com>
15:01:34 <bcotton> .hello2
15:01:37 <zodbot> bcotton: bcotton 'Ben Cotton' <bcotton@redhat.com>
15:01:40 * cmurf is here and there and where and near
15:02:25 <adamw> how's everyone apocalypsing
15:02:40 <tablepc> there isn't one here
15:02:46 <cmurf> this is an exciting and morbid question
15:03:14 <adamw> you're welcome
15:03:30 <cmurf> Coloradoans went crazy with Wyoming's fireworks on Saturday
15:03:36 * bcotton wonders if that's a reference to These Uncertain Times or just the nano and btrfs threads on devel
15:03:58 <adamw> it can be two things!
15:04:04 <cmurf> btrfs is tame, nano made me run for the hills
15:04:05 <tablepc> I had fireworks in my driveway. The neighbors enjoyed it.
15:04:25 <adamw> i am a fan of william gibson's theory that the apocalypse happens in slow motion and is made up of many things
15:04:33 <adamw> btrfs and nano threads among them
15:04:55 <tablepc> Forgive me but wasn't btrfs the subject of much discomfort and discussiopn recently
15:05:05 <adamw> discussion, sure
15:05:14 <adamw> we will get to that later in the meeting
15:05:18 * cmurf thought it was docile
15:05:53 <adamw> #topic Previous meeting follow-up
15:05:58 <adamw> #info there were no action items
15:06:01 <tablepc> Why do we need the new editor is it "new and exciting features?"
15:06:08 <adamw> tablepc: we'll get to that in a minute
15:06:09 <adamw> hold up
15:06:26 <adamw> if anyone remembers the last meeting, do they have any other follow up on it? :)
15:06:39 <tablepc> Was there a last meeting?
15:06:46 <adamw> https://meetbot-raw.fedoraproject.org/teams/fedora-qa/fedora-qa.2020-06-15-14.59.txt
15:07:15 <adamw> so far as the data center move goes, we're still looking fine there, all services running...still waiting on the openqa secondary arches.
15:08:56 <adamw> okey dokey then
15:08:57 <adamw> #topic Fedora 33 status and Changes
15:09:19 <adamw> #info current Rawhide continues to work fairly well, proposed blocker count is low
15:09:32 <adamw> but we do indeed have lots of potentially disruptive Changes lined up
15:09:40 <tablepc> 0703 WS doesn't seem to have any new misery
15:10:09 <adamw> #info there are several potentially disruptive Changes proposed at present, in addition to the already-accepted ones we've looked at in previous meetings
15:11:04 <adamw> #info Make btrfs default: https://fedoraproject.org/wiki/Changes/BtrfsByDefault
15:11:49 <adamw> #info compiler policy change (allow building with upstream preferred C compiler, not requiring GCC in almost all cases): https://fedoraproject.org/wiki/Changes/CompilerPolicy
15:12:08 <tablepc> Wasn't is a few months back when lots of people here were complaining about they couldn't get it to work?
15:12:25 <tablepc> btrfs
15:12:35 <adamw> there are also glibc and binutils version bumps, which can always be fun
15:13:36 <tablepc> When will be start seeing them in Rawhide?
15:13:41 <adamw> #info this one's not likely very 'disruptive', but significant: nano as default console editor: https://fedoraproject.org/wiki/Changes/UseNanoByDefault
15:13:50 <adamw> tablepc: these are all still proposed changes ATM
15:13:52 <adamw> fesco has not approved them yet
15:13:58 <adamw> they should not land till at least after fesco approves them
15:14:02 <adamw> (if it does)
15:14:22 <adamw> anyone have thoughts or concerns on any of these? btrfs is the most significant, i guess
15:14:31 <adamw> i'm not sure if we need to Take A Position on it as a group
15:14:38 <cmurf> there's an Advanced-Custom (blivet-gui) btrfs bug, when asking it to put btrfs on /boot but i think it might already be fixed
15:15:22 <cmurf> seems sort of edge case to me - it ought to work - but it's not a hill for me
15:15:37 <adamw> not really related to having it as the default either, i guess?
15:15:38 <cmurf> btrfs test day tomorrow
15:15:53 <cmurf> right
15:15:59 <adamw> cmurf: what's the status on getting an image patched to actually try and use btrfs by default for that?
15:16:49 <cmurf> the discussion over whether /boot will be on btrfs is on-going with bootloader folks who i'm deferring to, they're OK with it, but i'd like them to be a bit more than just ok with it
15:16:53 <tablepc> Will there be a drop to install WS with btrfs for test day?
15:17:14 <cmurf> i doubt it, i need to ask ngompa
15:17:28 <cmurf> i think the test day will use custom partitioning's btrfs scheme drop down
15:17:43 <cmurf> making the images totally out of band make it hard to test
15:18:03 <tablepc> so can I test by reinstalling 0703 but use custom with btrfs
15:18:04 <cmurf> but the custom partition layout is the same as what's proposed in the base change
15:18:12 <cmurf> yes
15:19:27 <bcotton> cmurf: is Workstation still considering going to bare ext4 instead of ext4-on-lvm as a contingency plan?
15:20:05 <Southern_Gentlem> .hello jbwillia
15:20:06 <zodbot> Southern_Gentlem: jbwillia 'Ben Williams' <vaioof@gmail.com>
15:20:25 <cmurf> the contingency is lvm+ext4
15:20:32 <adamw> cmurf: i'm more concerned about catching odd bugs in the exact 'switch btrfs to be the default in non-custom workflow' cases...
15:20:38 <Southern_Gentlem> btrfs -100
15:20:49 <cmurf> i don't think it makes sense to do plain ext4 as the contingency
15:21:14 <bcotton> cmurf: cool, thanks :-)
15:21:40 <cmurf> adamw: go on?
15:21:42 <Southern_Gentlem> cmurf, well at least with ext4 the tools are there to recover whereas btrfs is still very much lacking
15:21:55 <adamw> cmurf: that was it. i was finished. :P
15:22:14 <adamw> cmurf: i definitely agree with the contingency being status quo, btw
15:22:22 <adamw> the contingency being another change never made any sense
15:22:33 <cmurf> if the change is approved then it's straightforward to flip the switch in rawhide and do another test day or week or months
15:22:45 <adamw> yeah, i think we would want to do that
15:24:10 <cmurf> Southern_Gentlem: the problem with btrfs is there are many recovery tools
15:24:35 <cmurf> with ext4 you've got fsck.ext4 and either it works or doesn't and you're done, with btrfs it's a few layers deep with scary UX
15:25:10 <cmurf> but 'btrfs restore' is seriously professional grade scraping, Josef says he's only had 1 case in 10 years he didn't get something out of a btrfs file system
15:25:17 <tablepc> So a recovery tool needs to be chosen and tested.
15:25:43 <cmurf> i don't think i'm gonna ask people to try and sabotage file systems and then recover them - seems odd
15:26:00 <adamw> sounds like testing!
15:27:01 <tablepc> What the gain in switching to btrfs?
15:27:18 <cmurf> see the change proposal - list of benefits to Fedora
15:27:21 <adamw> it's btr
15:27:27 <adamw> thank you, thank you, i'm here all week
15:27:56 <cmurf> the main thing for desktops is it solves the competition for free space between /home and / while also gaining us some features like
15:28:22 <tablepc> I'm on the list of changes page but a search for btrfs null.
15:28:24 <cmurf> reflinks for container usage; compression; and IO isolation
15:28:29 <Lailah> Hello! Sorry I'm late!
15:28:48 <Lailah> .fas lailah
15:28:49 <cmurf> https://fedoraproject.org/wiki/Changes/BtrfsByDefault
15:28:49 <zodbot> Lailah: lailah 'Sylvia Sánchez' <BHKohane@gmail.com>
15:29:52 <tablepc> Okay that's  a good gainn. thanks.
15:31:34 <adamw> so, i mean, it's pretty obvious that the btrfs proposal comes with risks. we can state that on the ticket, though it's not gonna be news to anyone. does anyone think we should Officially Support or Oppose it, or shall we trust fesco to weigh the pros against the cons?
15:34:31 <adamw> (i'm kinda inclined to leave it to fesco, though i do wish this had come up earlier in the cycle)
15:34:43 <tflink> +1 to trusting fesco
15:35:26 <tflink> I wish that the change proposal outlined some of the risks - that seems to be relevant
15:35:50 <cmurf> i'm all ears on what risks to add to the proposal
15:36:08 <adamw> tflink: what specific risks would you want to add?
15:36:33 <tflink> the recovery tool part seems relevant
15:36:33 <Lailah> +1 trusting FESCO
15:36:43 <tablepc> Need to select (as in one) recovery tool and test it too.
15:37:06 <tablepc> Risk that is.
15:38:06 <cmurf> even XFS repair is different than ext4
15:38:15 <cmurf> but btrfs is even more so
15:39:19 <cmurf> because repair is really hard, the general strategy is to (a) get it to mount read-only, and get important data off
15:39:21 <tablepc> Going with new FS without a known reliable recover tool would not be good.
15:39:43 <tflink> maybe the other risks don't need to be mentioned, general "if stuff doesn't work" things like if anaconda has issues with btrfs
15:39:50 <cmurf> (b) scrape the data off if that doesn't work
15:40:10 <cmurf> (a) is the common outcome; (b) is uncommon
15:40:20 <adamw> how does 'scraping' it work exactly
15:41:02 <adamw> also that sounds like kind of a pain if it affects a key system partition as then you get to reinstall, presumably?
15:41:37 <cmurf> scraping is byzantine - it's a significant conversation to tell you "how exactly"
15:41:43 <tablepc> Reinstall is not a popular solution.
15:42:07 <cmurf> one of the things being looked at is automating that process, but it's so uncommon that the resources to make it work that way haven't been a priority
15:42:23 <adamw> okay.
15:42:28 <cmurf> Josef is focused first on making an even more tolerant read-only mount command for kernel 5.9
15:42:57 <cmurf> also note, that the only time btrfs is falling over like this, is if the hardware is having issues
15:43:31 <cmurf> it could be firmware bugs, or uncorrectable errors (by the firmware's ECC)
15:43:40 <tablepc> Are there any statics on that?
15:43:46 <cmurf> yes
15:43:52 <Southern_Gentlem> cmurf,  next time i have people in #fedora with issues with btrfs i will pm you to come in
15:44:07 <cmurf> millions of computers at Facebook
15:44:12 <Southern_Gentlem> so
15:44:17 <cmurf> using consumer grade drives, both HDD and SSD
15:44:35 <Southern_Gentlem> they are not our users, we need to stop shooting our user base in the foot
15:44:40 <cmurf> Southern_Gentlem: that's fine, i've always been around for that
15:44:45 <tablepc> sWell is storage hardware dies I guess a reinstall is expected. In that case I'm fine.
15:45:02 <Lailah> What does  "consumer grade drives"  mean?
15:45:10 <Lailah> What sort of device is that?
15:45:18 <Lailah> How is it different from the rest?
15:45:19 <cmurf> Samsung 840 EVO
15:45:24 <Southern_Gentlem> tablepc, we have been seeing non hardware failures of btrfs which causes reinstalls
15:45:47 <Lailah> cmurf:  I don't use Samsung at all.
15:46:27 <Lailah> I'll google the model
15:46:30 <tablepc> Then I quess the recovery tool is still an issue
15:46:37 <cmurf> Southern_Gentlem: unlikely - here's why. Josef Bacif, btrfs kernel developer who also works at Facebook, has done a deep dive into hundreds of Btrfs fall over cases
15:46:52 <cmurf> he says they've all turned out to be hardware sourced failures
15:47:12 <cmurf> btrfs is an early detection system, it is more sensitive because it's checksumming everything not just the file system
15:47:14 <Southern_Gentlem> cmurf, i have no issue with offering btrfs, but i do if it is the default
15:47:40 <Lailah> Same here.
15:47:41 <Southern_Gentlem> ext4 has been around and the tools are there for issues
15:47:41 <tablepc> Oh I like checksumming everything.
15:47:42 <cmurf> if the hardware is doing spurious things, data is way more likely to get hit
15:48:23 <Lailah> Spurious things like what?
15:48:48 <cmurf> yep user to user support will need to be on board with how to help people figure this out, that's a legit risk
15:48:57 <Southern_Gentlem> Lailah, people plugging in their laptop to the power adaptor or unplugging
15:49:26 <cmurf> ok so in my resource control tests, i've had to yank the power cord often because the test gets stuck
15:49:28 <Southern_Gentlem> can cause things to change (on power machine runs 2x quicker)
15:49:41 <cmurf> i've done this over 100 times, while compiling webkitgtk
15:49:58 <cmurf> it's writing to the disk, using btrfs, during the power being cut
15:50:16 <cmurf> from cold boot, it just starts up normally
15:50:31 <cmurf> no errors, no fsck, no recovery commands - nothing special at all
15:50:33 <cmurf> for a year
15:50:43 <cmurf> i know for a fact that SSD has firmware bugs
15:50:45 <Southern_Gentlem> until it does happen
15:50:56 <cmurf> i *will* get hit by one eventually
15:51:06 <Southern_Gentlem> everything man made has bugs especially software
15:51:06 <cmurf> but then so does any other file system
15:51:21 <adamw> ok, we're running short on time
15:51:47 <Southern_Gentlem> like i said btrfs on fedora no issue, not as default
15:52:08 <adamw> do folks think we should register specific concerns as a group? you can register concerns as an individual on the list or in the issue ticket
15:52:15 <adamw> Southern_Gentlem: well, that's the proposal
15:52:21 <adamw> Southern_Gentlem: it's already included, the proposal is making it default
15:52:27 <Southern_Gentlem> one of the foundations of fedora is first but we dont have to jump immediately
15:52:28 <adamw> so it sounds like you have a concern with the proposal :)
15:52:37 <Southern_Gentlem> yeah
15:53:01 <Southern_Gentlem> i dont see it as a selling point to our userbase
15:53:15 <Southern_Gentlem> i see it more as a point of people running away
15:53:46 * tflink is still +1 to leaving it to fesco, doesn't see any testing-specific reason to go against it as a group
15:53:59 <cmurf> Southern_Gentlem: this has been discussed for 9 years
15:54:03 <cmurf> not sure what's immediate about it
15:54:06 <tflink> so long as we do test days and poke at stuff which I imagine we'd do :)
15:54:10 <tablepc> I like all the files being check summed, but that's not a biggie for me.
15:54:40 <cmurf> FESCo originally approved it in 2011; the Workstation wg approved it in their tech spec in 2014; it's been discussed most of this year in working group meetings
15:55:08 <Lailah> Lailah, people plugging in their laptop to the power adaptor or unplugging
15:55:08 <adamw> i'm +1 leaving it to fesco too
15:55:10 <Southern_Gentlem> cmurf and majority of fedora users will not care until something goes wrong and the tools are not there to recover easily like ext4
15:55:12 <Lailah> Shit
15:55:33 <adamw> sorry, we gotta move on for now
15:55:34 <Lailah> Southern_Gentlem:  You mean that the computer suddenly goes off?
15:55:46 <Lailah> adamw:  Okay
15:55:50 <Southern_Gentlem> Lailah, or a change like i said
15:55:50 <adamw> #topic Test Day / community event status
15:55:53 <adamw> mboddu: around?
15:56:58 <adamw> er\
15:57:04 * adamw needs more coffee
15:57:06 <adamw> let's try that again
15:57:15 <mboddu> adamw: yes
15:57:20 <adamw> mboddu: sorry, wrong person :P
15:57:25 <adamw> sumantro doesn't seem to be here
15:57:27 <mboddu> Ah okay
15:57:33 * mboddu is not around anymore :P
15:57:42 * tflink was wondering what testday mboddu was working on :)
15:58:08 <tablepc> We need a tester test day
15:58:28 <mboddu> tflink: Rawhide :P
15:58:30 <adamw> #info today we have SwapOnZRAM Test Day: https://fedoraproject.org/wiki/Test_Day:F33_SwapOnZRAM
15:58:39 <tablepc> Check for slepp status etc.
15:58:45 <adamw> (grrr, that test day page name is irregular, someone remind me to yell at someone else later)
15:59:21 <adamw> #info Wednesday is the first(?) btrfs Test Day: https://fedoraproject.org/wiki/Test_Day:F33_btrfs_by_default_2020-07-08
15:59:37 <adamw> so if folks can please help out with those, that'd be great. over in #fedora-test-day
15:59:53 <adamw> that's all i'm aware of
15:59:55 <adamw> #topic Open floor
16:00:00 <adamw> anything for the shortest open floor ever? :D
16:00:09 <Lailah> I have one little thing,
16:00:16 <tablepc> Have a Good Day Everyone!
16:00:22 <Lailah> Well, two little things actually.
16:00:47 <Lailah> tablepc:  Have a good day you too!  :-D
16:01:06 <adamw> Lailah: fire away
16:02:34 <Lailah> There's a bug related to logging out and in in KDE
16:02:38 <Lailah> I filed it.
16:02:42 <Lailah> But no response so far.
16:02:49 <adamw> what's the bug link?
16:03:06 <Lailah> This bug:  https://bugzilla.redhat.com/show_bug.cgi?id=1851200
16:03:43 <Lailah> It's not even assigned.  Every other bug I reported has been fixed, and that makes me happy.  But this one is still there.
16:04:41 <adamw> Lailah: can you attach the logs from an affected boot to the bug? that may well help
16:05:38 <adamw> if you reboot after it happens, do 'journalctl -b1 > journal.log' as root and upload journal.log .. /var/log/Xorg.0.log may also be useful but you'd have to catch it before rebooting somehow, ssh maybe
16:06:51 <adamw> i think there must be something specific to your system in the bug, because the openqa test for this passes at the moment
16:07:08 <Lailah> Okay. I'll do that.
16:07:18 <Southern_Gentlem> (booting up f32 updated kde iso to test
16:09:28 <adamw> Lailah: what was the other thing?
16:09:28 <Southern_Gentlem> (works in the live instance, trying install)
16:11:11 <Lailah> No, I just looked. It's fixed now.
16:12:53 <adamw> ok :)
16:12:59 <adamw> welp, we're over time, so let's wrap it up
16:13:01 <adamw> thanks for coming, everyone
16:13:35 <Lailah> A pleasure, sorry for being late.
16:13:42 <Lailah> adamw++
16:13:48 <Lailah> adamw ++
16:13:58 <Lailah> How do I give you cookies?
16:13:59 <Southern_Gentlem> Lailah, can you grab the F32-KDE-LIVE-X86_64-20200701.iso from https:/tinyurl.com/Live-respins, put on a usb stick and boot your computer with said stick and test that it works in the live there
16:14:22 <Lailah> Sure1
16:14:25 <Lailah> Sure!
16:16:36 <Lailah> Southern_Gentlem:  It's going to take some time to download.
16:16:57 <adamw> #endmeeting