15:00:40 <pwhalen> #startmeeting Fedora ARM and AArch64 Status Meeting
15:00:40 <zodbot> Meeting started Tue Jul  2 15:00:40 2019 UTC.
15:00:40 <zodbot> This meeting is logged and archived in a public location.
15:00:40 <zodbot> The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:40 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:40 <zodbot> The meeting name has been set to 'fedora_arm_and_aarch64_status_meeting'
15:00:40 <pwhalen> #chair pwhalen pbrobinson jlinton
15:00:40 <pwhalen> #topic roll call
15:00:40 <zodbot> Current chairs: jlinton pbrobinson pwhalen
15:00:50 * pbrobinson o/
15:00:53 <pwhalen> good morning folks, who's here today?
15:00:54 * jlinton waves
15:01:48 <pbrobinson> hey jlinton
15:02:36 * pwhalen gives it a minute for others to join
15:05:06 <pwhalen> #topic 1) ==== Userspace Status  ====
15:05:20 <pbrobinson> nothing of real note here
15:05:23 <pwhalen> I don't we have anything new here
15:05:39 <pbrobinson> yesterday was the deadline for any features needing mass rebuild
15:05:44 <jlinton> Looks like PAC got put on hold
15:06:23 * jonmasters here
15:06:26 <pbrobinson> morning jonmasters
15:06:26 <pwhalen> howdy jonmasters
15:06:36 <jonmasters> figured I'd turn up for a change, morning!
15:07:01 <pwhalen> #info No issues reported.
15:07:07 <pwhalen> #topic 2) ==== Kernel Status ====
15:07:16 <pbrobinson> jonmasters: you may as well start here
15:07:18 <pwhalen> Just in time
15:07:31 <jonmasters> well, I can add a note about the builders if you like
15:07:43 <pwhalen> #info Fedorn 29 armv7 guests "pause" from time to time on rhel-alt 7.6 kernel-4.14.0-115.2.2.el7a.aarch64 kernel
15:07:43 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=1576593
15:07:49 <pbrobinson> jonmasters: yes, that's was I was intending
15:07:50 <pwhalen> That would be great
15:08:00 <jonmasters> yea so I looked at https://bugzilla.redhat.com/show_bug.cgi?id=1576593
15:08:18 <jonmasters> I am suspecting that it's actually a hardware errata in the X-Gene1 processor
15:08:43 <jonmasters> it happens when you have a guest that takes a trap while walking its page tables
15:09:07 <jonmasters> and the HPFAR_EL2 register that contains the faulting address (LAPE) seems truncated in *that* case
15:09:12 <jonmasters> so you'll only see this if:
15:09:20 <jonmasters> 1). You're running a guest with 32-bit LPAE
15:09:44 <jonmasters> 2). It takes a trap on a page table page while walking them AND that page is outside of the 4GB 32-bit range
15:09:58 <jonmasters> so you'll boot ok, and run fine for a while, but eventually, boom
15:10:14 <jonmasters> thanks pwhalen for setting up a builder, I'm still using it btw :)
15:10:38 <jonmasters> I ran multiple crash sessions on the guest/host and was going all day/night over the weekend, but I did report what I think is a hw errata yesterday
15:10:44 <jonmasters> if I'm right, there's a nasty workaround
15:10:46 <pbrobinson> so we're awaiting confirmation if it is a errata?
15:10:49 <jonmasters> two actually, or maybe more:
15:11:06 <jonmasters> 1). You could make sure guest page table pages were allocated low on LPAE (nasty)
15:11:38 <jonmasters> 2). During fault handling on the host, manually do an AT (address translation) and walk the page tables and look to see if any of the addresses almost match the reporting fault address
15:11:51 <jonmasters> if they do, assume it's an errata and handle
15:11:58 <jonmasters> waiting yea
15:12:07 <jonmasters> the silicon team are currently checking
15:12:26 <pbrobinson> coolio
15:12:28 <jonmasters> but there were known issues with 32-bit emulation at the time and I doubt LPAE ever really got validated
15:12:31 <jonmasters> thanks
15:12:37 <jonmasters> I'll poke possible hacks to workaround
15:13:05 <pwhalen> jonmasters, np, thanks for working on it (sorry it took your weekend)
15:13:07 <pbrobinson> jonmasters++ awesome, thanks for the work on digging into this, I know you love a good challenge ;-)
15:13:08 <zodbot> pbrobinson: Karma for jcm changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:13:13 <jonmasters> it was fun ;)
15:13:22 <pwhalen> awesome, nirik ^
15:13:39 <pwhalen> jonmasters++
15:13:39 <jonmasters> brb
15:13:41 <zodbot> pwhalen: Karma for jcm changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:13:56 <nirik> awesome! great news... thanks for the work!
15:14:46 <pwhalen> #info arm-smmu e0800000.smmu: Unexpected global fault, this could be serious
15:14:46 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=1724276
15:15:27 <pwhalen> opened this last week for the seattle, msalter offered to take a look (thanks Mark!)
15:15:32 <msalter> so I haven't been able to reproduce this
15:15:49 <msalter> but I'm using "different" firmware
15:16:10 <msalter> I have to find a copy of the old firmware and try that
15:16:18 <pwhalen> right, I was about to ask. Should I just update?
15:17:01 <msalter> but I think the answer is going to be a command line arg to avoid it on seattle
15:17:15 <pwhalen> I think there was some reason I stayed on the older firmware
15:18:01 <pwhalen> msalter, thanks
15:18:05 <msalter> np
15:18:19 <pwhalen> any other kernel issues?
15:18:32 <pwhalen> #info F31: kernel-5.2.0-0.rc7.git0.1.fc31
15:18:32 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1306512
15:18:55 <pwhalen> I've not started to test rc7
15:19:10 <pbrobinson> I've done some testing, no real change to rc6
15:19:30 <pbrobinson> there's a few minor fixes, both from upstream and from me, but nothing of huge note
15:20:38 <pwhalen> #info Please test and report any issues to the list or #fedora-arm.
15:20:52 <pwhalen> #topic 3) ==== Bootloader Status ====
15:21:03 <pwhalen> #info uboot-tools-2019.07-0.1.rc4
15:21:03 <pwhalen> #link https://koji.fedoraproject.org/koji/taskinfo?taskID=35612902
15:21:10 <pwhalen> #undo
15:21:10 <zodbot> Removing item from minutes: <MeetBot.items.Link object at 0x7f6d313ced10>
15:21:11 <pwhalen> #undo
15:21:11 <zodbot> Removing item from minutes: INFO by pwhalen at 15:21:03 : uboot-tools-2019.07-0.1.rc4
15:21:38 <pwhalen> #info uboot-tools-2019.07-0.2.rc4.fc31
15:21:45 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1300621
15:22:00 <pbrobinson> there's not a lot of difference between the two TBH
15:22:27 <pwhalen> ah, ok.. I see from the changelog..
15:22:39 <pbrobinson> I think we should have rc5 from upstream today/tomorrow
15:22:41 <pwhalen> any other bootloader news? issues?
15:23:00 <pbrobinson> I spent some time over the weekend looking at the Rockchips stuff
15:23:32 <pbrobinson> I did a few patches to improve primarily the Rock960, although others will be added to that too
15:24:19 <pbrobinson> I think we should have at least a bunch of rk3399 devices in reasonable shape for F-31, plus some others
15:25:18 <pwhalen> very nice, I know they've been painful for me
15:25:47 <pbrobinson> I think I have at least some of it sorted out, I'm trying to work out why the display isn't powering up in U-Boot
15:26:07 <pbrobinson> that would give us a reasonable experience for a bunch of devices
15:26:28 <jlinton> there are a few rk3399 edk2 ports people have been hacking too
15:27:02 <pbrobinson> jlinton: yea, I've seen a few around. I'm still avoiding edk2.... someone else can do that
15:27:28 <pbrobinson> jlinton: speaking of which, did you have any luck with the inconsistent console in ekd2 for virt?
15:28:40 <jlinton> I think when I changed the virtio/etc config (trying to remember what the solution was) it ran consistently
15:29:10 <jlinton> my initial graphics config wasn't quite right though
15:30:04 <jlinton> (I thought I mentioned in the defect the change between virtio/QLX or some such fixing it)
15:30:32 <pbrobinson> jlinton: let me know if you have a issue or something I can follow? Or a patch I can test
15:33:20 <pwhalen> #topic 4) ==== Fedora 31 ====
15:33:26 <pwhalen> #info Latest nominated nightly - https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test
15:34:10 <pwhalen> I've not had a chance to start testing this
15:35:07 <pwhalen> the previous nominated nightly had some issues on the rpi3 and the new firmware
15:35:33 <pbrobinson> yes, I think upstream is nearly to the bottom of the issue there, and the fact they damn well shouldn't regress
15:36:43 <pbrobinson> it actually works fine if you have a display plugged in
15:37:56 <pwhalen> aha
15:38:29 <pbrobinson> but I expect a new firmware later this week with the fixes with luck
15:38:42 <pwhalen> and it looks like you untagged the problem build?
15:38:49 <pbrobinson> yes
15:39:42 <pwhalen> the other issue was with the display on the rpi3, that has also been reported upstream
15:40:08 <pbrobinson> yea, it's upstream, I have not got to it further yet
15:40:17 <pwhalen> any other issues in recent f31 nightlies?
15:41:10 <pbrobinson> I think it's mostly looking OK with my testing
15:41:23 <pwhalen> agreed
15:41:26 <pbrobinson> there's a few bits and pieces here and there but nothing particularly frightful
15:41:51 <pwhalen> #topic 5)  == Open Floor ==
15:42:08 <jonmasters> I got one
15:42:25 <pbrobinson> jonmasters: go!
15:42:40 <jonmasters> on the builder issues - do we have any firm plans for other hardware lined up yet? I think not currently, right?
15:42:56 <jonmasters> so a workaround is something you're actually interested in seeing developed?
15:43:31 <pbrobinson> jonmasters: yes, there's budget for it, not sure the timeframe, if I remember Fedora gets 14 of the new Lenovo boxes
15:43:35 <jonmasters> if it won't be a waste of time (unlikely to get upstream, might be a way to finagle into RHEL for the host) then I'll make something
15:43:43 <pbrobinson> and then CentOS 6 for a total of 20
15:43:48 <jonmasters> ok, so we need to find out if eMAG has the same problem :)
15:43:53 <jonmasters> that's also on my todo
15:44:12 <jonmasters> and that silicon team is checking
15:44:17 <jonmasters> thanks, got what I need
15:44:17 <pbrobinson> jonmasters: I figure you have one to be able to test, and an install of presumably el8 to test?
15:44:23 <jonmasters> yea
15:44:28 <jonmasters> got one here at home
15:44:32 <jonmasters> thanks
15:44:46 <pbrobinson> so let us know if you need any config bits for testing
15:44:52 <jonmasters> will do
15:45:12 <jonmasters> I'll keep the existing builder for now that Paul setup, then setup my own this week at home to check eMAG
15:45:16 <jonmasters> that's it - thanks
15:45:34 <pwhalen> thanks jonmasters
15:46:16 <pwhalen> anything else for open floor?
15:47:05 <pwhalen> #endmeeting