15:00:18 <pwhalen> #startmeeting Fedora ARM and AArch64 Status Meeting
15:00:18 <zodbot> Meeting started Tue Sep 14 15:00:18 2021 UTC.
15:00:18 <zodbot> This meeting is logged and archived in a public location.
15:00:18 <zodbot> The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:18 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:18 <zodbot> The meeting name has been set to 'fedora_arm_and_aarch64_status_meeting'
15:00:18 <pwhalen> #chair pwhalen pbrobinson coremodule
15:00:18 <pwhalen> #topic roll call
15:00:18 <zodbot> Current chairs: coremodule pbrobinson pwhalen
15:00:30 * coremodule is here, good morning
15:00:31 <pwhalen> Good morning, who's here today?
15:01:22 * pbrobinson o/
15:02:10 <pwhalen> Alrighty, lets get started
15:02:16 <pwhalen> #topic 1) ==== Userspace Status  ====
15:02:57 <pwhalen> Anyone seeing any new user space issues?
15:03:24 <pbrobinson> none I'm aware of
15:03:40 <pwhalen> thank goodness.
15:03:49 <coremodule> yeah, nothing new
15:03:59 <pwhalen> We have the nano issue, but lets discuss that in the f35 section
15:04:10 <pwhalen> #topic 2) ==== Kernel Status ====
15:04:19 <pwhalen> #info kernel-5.14.3-300.fc35
15:04:19 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1830953
15:04:52 <pwhalen> 5.14 has been looking pretty good for me, however this morning my mustang flipped readonly on btrfs
15:05:13 <coremodule> it's those dang bad micro sd cards :)
15:05:19 * nirik has a dumb question for open floor.
15:05:34 <pwhalen> heh, thankfully mustang uses spinning disk
15:05:37 <pwhalen> https://paste.centos.org/view/01979980
15:06:19 <pwhalen> Seems we've been having some issues in general with that hw, still looking at it. But in the same lab we also hit this one
15:06:40 <pwhalen> https://bugzilla.redhat.com/show_bug.cgi?id=2000482
15:06:42 <coremodule> interesting, I'm not familiar with it
15:06:55 <pbrobinson> it's ancient HW
15:07:12 <pwhalen> it is, so it could be some hardware issues
15:07:13 <coremodule> oh okay
15:07:17 <pbrobinson> it was the original gen of aarch64 HW that was usable for building and servers etc
15:07:46 <pbrobinson> so it's not standards compliant, and after a buy out or two it's also unsupported as the vendor pretends they don't know it
15:08:07 <pbrobinson> so it's very much best effort but there's still a bunch of it about
15:08:43 <pbrobinson> mine has 2* spinning rust in raid-1/LVM with ext4
15:09:12 <pwhalen> Right. Concerned its an issue with btrfs though. Has anyone seen any similar issues?
15:09:27 <pwhalen> could definitely be the hw dying too
15:10:11 <pbrobinson> nothing to me, I'd maybe move the RHBZ to the btrfs-tools package so it gets actual btrfs maintainers assigned to it
15:10:20 <jlinton> I think most of my machines are xfs because they were installed from the server disk.
15:10:31 <pwhalen> good plan, will do
15:11:04 <pwhalen> Any other kernel news or issues?
15:11:21 <pwhalen> #info Please test and report any issues to the list or #fedora-arm.
15:12:06 <jlinton> I got pinged about a rockpro64 boot failure with rawhide, which appears to be dying fairly early in the kernel init process.
15:12:07 <pbrobinson> still catching up with kernel, between RHEL releases and PTO I'm behind but I think 5.14 mostly looks just fine
15:12:21 <pbrobinson> 5.15 looks to be quite a quiet release thankfully
15:12:37 <pbrobinson> jlinton: why were you pinged?
15:12:39 <jlinton> but other than that my 5.14 machines are ok.
15:13:11 <jlinton> (internal it was part of this other board bringup, the rockpro is on a cert track for systemready ir)
15:13:24 <pbrobinson> ah, cool
15:13:39 <pbrobinson> yes, I have a few bits for SR-IR to deal with
15:14:24 <pwhalen> #topic 3) ==== Bootloader Status ====
15:14:40 <pwhalen> #info grub2-2.06-5.fc35
15:14:40 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1830126
15:14:57 <pwhalen> #info Fixes kernel alignment issue on AArch64 and the boot issue on 32-bit arm (BZ#2000756).
15:15:06 <pwhalen> #link https://bodhi.fedoraproject.org/updates/FEDORA-2021-51f205d5af
15:15:50 <pwhalen> #info uboot-tools-2021.10-0.5.rc3.fc35
15:15:50 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1825997
15:16:30 <pbrobinson> will be a new U-Boot build likely tomorrow
15:16:49 <pbrobinson> rc4 is due today and I'm going to be testing a fix for the rpi issue
15:17:04 <pbrobinson> so once I have all that stuff aligned I'll push a build
15:17:29 <pwhalen> #info U-Boot RC4 should be ready for testing this week.
15:17:51 <pwhalen> Anything else?
15:18:27 <pwhalen> #topic 4) ==== Fedora 35 Beta ====
15:18:37 <pwhalen> #info Fedora 35 Beta Bugs
15:18:37 <pwhalen> #link https://qa.fedoraproject.org/blockerbugs/milestone/35/beta/buglist
15:18:57 <pwhalen> We have two beta blockers
15:19:06 <pwhalen> #info sdhci_setup_cfg: Hardware does not specify base clock frequency
15:19:06 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=1999180
15:19:39 <pbrobinson> The u-boot build should fix the sdhci_setup_cfg issue
15:20:09 <pwhalen> Nice! Thanks.
15:20:17 <coremodule> that's great
15:20:49 <pwhalen> #info [abrt] gnome-shell: cogl_texture_get_gl_texture(): gnome-shell killed by SIGSEGV
15:20:49 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=1989726
15:21:10 <pwhalen> This one is concerning, dont think anyone is looking at it
15:21:43 <pbrobinson> side effect of the fix is that it should also boot faster
15:22:01 <pwhalen> great side effect :)
15:22:32 <bcotton[m]> #info bcotton nudged ajax on BZ 1989726 this morning. no reply yet
15:22:59 <pwhalen> thanks bcotton[m]!
15:23:03 <pbrobinson> I'll ping Karol and see if he has some cycles to look too, he's looked at that HW before
15:23:27 <pwhalen> Awesome, and he has one which is always helpful.
15:23:58 <jlinton> the uboot is using the max clock, that makes it slower.
15:24:16 <pbrobinson> jlinton: there's patches to fix that was my point
15:24:21 <jlinton> because the firmware isn't actually reporting the clock rate
15:24:34 <jlinton> its still a huge bug in the rpi low level firmware
15:24:53 <jlinton> and it likely is causing problems outside of the bare metal report in that defect as well.
15:25:08 <pwhalen> thanks for digging into it as well, jlinton
15:26:01 <pwhalen> Any other F35 issues not mentioned?
15:26:04 <pbrobinson> jlinton: why am I not shocked at that
15:26:06 <jlinton> The only clock that appears to work is the system clock (id=4) the rest of them report 0 as the freq in some number of cases despite first requesting !0 clocks.
15:28:47 <pwhalen> #topic 5)  == Open Floor ==
15:28:57 <pwhalen> nirik: the floor is yours
15:29:08 <nirik> I lied... I had 2 items (but feel free to punt them to list or elsewhere):
15:29:42 <nirik> first, people keep telling me for our armv7 builders we should just run the 64 bit kernel and 32bit userspace. Is that even possible in our setup? What would need to happen to make that work?
15:30:03 <nirik> or I guess all 64bit installs and make mock do 32bit chroots?
15:30:54 <nirik> I know we specifically didn't support multilib for arm right?
15:31:43 <nirik> Anyhow, can bring that to list? or is there anywhere reasonable to discuss that?
15:33:16 <pbrobinson> nirik: we would need to build another kernel
15:34:07 <pbrobinson> I've never investigated it, although I saw an update about adjusting the kernel/userspace split and I was going to look at that, but that update was while I was on PTO and I've not got to it
15:34:23 <pbrobinson> that should be more straight forward
15:34:43 <nirik> for me, it would be ideal if we could just have 64 bit builders and could build either on them, but that would need mock work right? or rpm and mock and ?
15:34:54 <pbrobinson> nirik: I think the bug is the best sport to discuss TBH, it's on my list to follow up
15:35:10 <nirik> ok, fair enough.
15:35:10 <pbrobinson> nirik: we could do that if koji could use containers
15:35:39 <nirik> would systemd-nspawn be good enough? mock can use those (actually does by default), but we don't in koji
15:35:58 <pbrobinson> and making koji use containers instead of mock I know has been discussed going all the way back to when I was in rel-eng but I've never seen it really go anywhere
15:36:28 <pbrobinson> nirik: what does it use for the "source" in systemd-nspawn?
15:36:29 <nirik> yeah, it's been on the todo list. The only big thing to check would be secureboot signing
15:37:01 <nirik> not sure what you mean? I think it's it's own container implementation?
15:37:02 <pbrobinson> does it build it's own container or pull it from somewhere or where does that userspace come from?
15:37:39 <nirik> ah, I see... I think it builds it's own.
15:37:58 <pbrobinson> right, and I think it would get confused there about the uname reporting
15:38:02 <nirik> but mock optionally has a bootstrap container feature to use a premade one
15:38:36 <nirik> I can push this up on my todo list and try and see if I can get nspawn to work...
15:39:54 <pbrobinson> if nspawn could use the latest arch container build as the basis for the chroot I think that may work, we have done some testing of 32 bit containers on 64 bit
15:40:13 <pbrobinson> maybe that's a rfe for mock/koji
15:41:43 <nirik> ok, the second thing was from the mobility sig and pinephone kernel. :) Currently we make a remix because we have a 3rd party kernel to support a bunch of things. I asked the author of that about upstreaming things and got a very nice reply from him (which he said I could share): https://paste.centos.org/view/d1cdad12 basically not much more is going to go upstream. :( But the modem should be he said, so perhaps we can get that working. Any help
15:41:43 <nirik> to get any of this moving forward is very welcome from anyone. :)
15:42:21 <nirik> the wifi is 'not upstreamable' sadly
15:42:43 <pbrobinson> nirik: I replied to you on the arm channel yesterday, did you see that?
15:43:24 <pbrobinson> I missed the meeting as I had a conflict and it was full on enough I didn't have bandwidth to chime in
15:43:25 <nirik> yeah, if we can get the modem working we will be in a much better place. I thought it was needing a driver from him.
15:43:44 <pbrobinson> the support for that modem is upstream in the standard drivers
15:44:12 <pbrobinson> the stuff he has is around uber power savings and shut down and I doubt in it's current form it'll ever go upstream
15:44:20 <nirik> just having any kind of network is really the lowest bar to making a fedora image IMHO... without networking things are just too hard to deal with.
15:45:00 <pbrobinson> right, and ATM I'm the only one who's even looked at kernel stuff but unfortunately my bandwidth is extremely low for kernel stuff
15:45:01 <nirik> yeah, thats all gravy IMHO...I just want a fedora image that somewhat works that I can update and have be somewhat usable
15:45:24 <nirik> yeah, I was sad he said almost no one asks about upstreaming. :(
15:45:42 <nirik> anyhow, thats all, just more a FYI and hope that someone might be sparked to work on it. ;)
15:46:08 <pbrobinson> TBH I'm not surprised, people are happy to glue shit together and stick with one random kernel even if it's not updated
15:46:18 <pbrobinson> it's so damn weird to me too
15:46:43 <pbrobinson> I dug into the kernel enough I found the commits for the upstream kernel IDs
15:47:18 <pbrobinson> I think what we need to do is have a DT entry which allows us to poke to UART to bring up the USB interface and it should "Just work"
15:47:25 <jlinton> I saw a couple mentions that the oneplus6 can run a more normal linux env too.   Which makes sense given its just a q845.
15:47:49 <pbrobinson> I don't think the modem is a *LOT* of work to get it up
15:48:34 <pbrobinson> I also don't think the wifi is too hard either, I have started looking at that and I'm hoping to get some weekend cycles in mid Oct as I'm basically going to pause renovations for 6 months then
15:48:39 <nirik> https://megous.com/git/linux/log/?h=pp-5.15 is his patches repo
15:48:55 <nirik> pbrobinson: he mentions another existing driver might be made to work in his email
15:49:11 <pbrobinson> yep, I have that cloned already
15:49:32 <nirik> jlinton: I saw that, but... https://wiki.postmarketos.org/wiki/OnePlus_6_(oneplus-enchilada) has a lot of still broken/not working things. ;(
15:49:54 <pbrobinson> but given his completely unrelated modem driver it's not huge amounts of use
15:50:08 <nirik> if we could find a existing phone we can base things on, I'd be fine with that...pinephone is really underpowered, but... sadly
15:50:48 <pbrobinson> and the wifi I'm looking at crafting into the current upstream driver based on other work, but someone has been hammering that upstream with cleanups of late
15:51:16 <pbrobinson> nirik: I have a n900 I could send you ;-)
15:51:27 <pbrobinson> oh wait, that's single core.... even less powered
15:51:43 <nirik> I'll put it next to my openmoko ;)
15:51:53 <pbrobinson> it has more power than that!
15:52:47 <nirik> true, most things do...
15:52:49 <pbrobinson> anyway I *think* some form of network for the PinePhone isn't too hard, it's just getting someone to poke it
15:53:29 <nirik> phosh and plasma mobile are really doing a lot... so once we find some hw that works we might really pick up speed.
15:56:21 <nirik> thats all I had, thanks for listening to me pontificate. :)
15:57:05 <pwhalen> Anyone have anything else for today?
15:58:02 <pbrobinson> so karol is going to take a look at the nano bug tomorrow
15:58:18 <pbrobinson> he may have destroyed his nano
15:59:07 <pwhalen> Ouch, ok.
15:59:28 <pwhalen> #endmeeting