15:01:21 <pwhalen> #startmeeting Fedora ARM & AArch64 Status Meeting
15:01:21 <zodbot> Meeting started Tue Jan 20 15:01:21 2015 UTC.  The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:21 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:21 <pwhalen> #chair pwhalen jonmasters bconoboy pbrobinson dgilmore dmarlin hrw jsmith kyle ahs3 bemk
15:01:21 <zodbot> Current chairs: ahs3 bconoboy bemk dgilmore dmarlin hrw jonmasters jsmith kyle pbrobinson pwhalen
15:01:31 <pwhalen> mornign folks, thanks for joining
15:01:34 <pwhalen> .fas pwhalen
15:01:35 <zodbot> pwhalen: pwhalen 'Paul Whalen' <pwhalen@redhat.com>
15:01:43 <dmarlin> .fas dmarlin
15:01:44 <zodbot> dmarlin: dmarlin 'David A. Marlin' <dmarlin@redhat.com>
15:02:15 <oatley> .fas oatley
15:02:16 <zodbot> oatley: oatley 'Andrew Oatley-Willis' <andrew.oatley-willis@senecacollege.ca>
15:02:25 <ctyler> .fas ctyler@
15:02:26 <zodbot> ctyler: 'ctyler@' Not Found!
15:02:51 * ctyler is dead, obviously
15:03:37 <bconoboy> .fas blc@
15:03:37 <zodbot> bconoboy: blc '' <blc@redhat.com>
15:03:46 <ctyler> .fas ctyler.fedora
15:03:47 <zodbot> ctyler: ctyler 'Chris Tyler' <ctyler.fedora@gmail.com>
15:03:59 <pwhalen> alright, from last week..
15:04:05 <pwhalen> #topic 0) ==== Previous meeting action items ====
15:04:06 <pwhalen> #info AArch64 Build Host Maintenance Completed
15:04:33 <dgilmore> hola
15:04:36 <pwhalen> this was completed last week, hosts on the most recent version of Tianocore. no issues since.
15:04:59 <pwhalen> we did lose one host during the firmware upgrade, its being sent for replacement/repair
15:05:39 <pwhalen> #info Python blockage cleared
15:07:35 <pwhalen> as if he heard his name ..
15:07:44 <pwhalen> #topic 1) ==== Kernel Status ====
15:09:20 <pwhalen> testing 3.19 only major issue ive seen is on the Pandaboard - https://pwhalen.fedorapeople.org/testing/panda-3.19.0-0.rc3.git2.1.fc22.armv7hl.txt
15:09:35 <pwhalen> fails to boot at all, need to try the latest
15:10:29 <pwhalen> #info overview of kernel testing - https://fedoraproject.org/wiki/Architectures/ARM/Quality_Assurance/Kernel_Testing/Fedora_22#kernel-3.19.0-0.rc3.git2.1.fc22
15:11:18 <pwhalen> ..tap..tap... "Is this thing on?" (in honour of Joan)
15:11:39 <dmarlin> quiet crowd
15:12:32 * pbrobinson is here
15:12:53 <pwhalen> anything else we want to mention for kernel issues, etc?
15:13:23 <pwhalen> #topic 2) ==== Rawhide Testing Status ====
15:13:29 <pbrobinson> there's an issue with PandaES on 3.18
15:13:37 <pwhalen> #undo
15:13:37 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x4bb39050>
15:14:11 <pwhalen> #info issue with Panda ES on 3.18 needs investigation
15:14:20 <pwhalen> #topic 2) ==== Rawhide Testing Status ====
15:14:46 <pbrobinson> so aarch64 rawhide is quickly catching up
15:14:53 <pbrobinson> no major build issues atm
15:15:12 <pwhalen> as dgilmore mentioned on our mailing list there is an issue with ARM disk images, last disk images are from Jan 10th (iirc)
15:16:47 <pbrobinson> it's not just arm but also live images on x86 too
15:17:11 <pwhalen> yes, i noticed.. doing installation testing on armhfp i ran into some issues with both vnc and text installs
15:17:26 <pwhalen> #info network spoke listed as not connected despite having assigned IP and hostname
15:17:33 <pwhalen> #link https://bugzilla.redhat.com/show_bug.cgi?id=1183807
15:19:32 <pwhalen> also started to test the latest uboot on all platforms, havent run into any problems with default boot behaviour and pxeboots
15:20:00 <pbrobinson> excellent
15:20:13 <dgilmore> it is an issue with mock
15:20:21 <pbrobinson> I've been testing 3.19 rc5 on the BBB and it's looking OK
15:20:44 <pwhalen> pbrobinson, you mentioned an issue with USB(iirc), any change there
15:20:50 <pwhalen> on the bbb specifically
15:21:06 <pbrobinson> I've not got to the bottom of it
15:21:17 <pbrobinson> not sure it's specifically USB
15:21:29 <pwhalen> ah, ok..
15:22:02 <pbrobinson> I plug a usb storage device in and with 3.17.8 it can be mounted on the BBB, for 3.18+ it sees it's usb-storage device but I don't get a partition or device to mount
15:22:33 <pbrobinson> was cross checking configs today but while I fixed a bunch of other stuff I never got to the bottom of that issue :-/
15:22:35 <pwhalen> i'll try to reproduce
15:23:41 <yselkowitz1> .hellomynameis yselkowitz
15:23:42 <zodbot> yselkowitz1: yselkowitz 'Yaakov Selkowitz' <yselkowi@redhat.com>
15:24:52 <pwhalen> #topic 3) ==== AArch64 Updates  ====
15:25:54 <pwhalen> its been mentioned the aarch64 repos are a little behind, any eta on when they will be closer to koji?
15:26:04 <pwhalen> or whats been built anyways
15:26:07 <pbrobinson> yes, we're catching up
15:26:20 <pwhalen> awesome, wfm
15:26:24 <pbrobinson> there was a libffi bug which blocked python/pthon3
15:26:41 <pbrobinson> I've been watching and pushing it forward
15:26:56 <kylem> hmm, i thought rth had a handel on that
15:26:57 <pbrobinson> mostly things are building and we're catching up
15:27:07 <dmarlin> pbrobinson:  any eta when packages tagged for f21 updates will hit the repos?
15:27:13 <bconoboy> kylem: he provided the patch so it's fixed now
15:27:16 <pbrobinson> kylem: yup, it was a patch from him I applied to libffi
15:27:32 <kylem> ok, good.
15:27:33 <pbrobinson> dmarlin: this week with luck
15:27:48 * kylem goes back to blissful/willful ignorance.
15:27:57 <pbrobinson> it's been one of many things on my list I'm trying to resolve
15:27:58 <dmarlin> pbrobinson:  are there any alternate (koji) repos we can use in the mean time (to do updates) ?
15:28:04 <dgilmore> dmarlin: no
15:28:06 <pbrobinson> no
15:28:14 <dmarlin> :-(
15:28:22 <pbrobinson> I'll send an update to the list once i've got it done
15:28:30 <pwhalen> thanks pbrobinson
15:28:33 <dmarlin> thanks
15:28:55 <pwhalen> #topic 4) == Open Floor ==
15:29:24 <bconoboy> Anybody see smooge's proposal to drop 32 bit architectures to secondary status?
15:29:36 <yselkowitz1> yes
15:29:44 <yselkowitz1> it's a bit early I think
15:29:50 <bconoboy> Yes, much too early
15:30:02 <dgilmore> bconoboy: i dod not see that
15:30:05 <bconoboy> I'd rather address the underlying reasons for why people think this is a good idea
15:30:16 <dgilmore> bconoboy: but I think at thei spoint it would be 32 bit x86
15:30:21 <pwhalen> nor i
15:30:30 <dgilmore> a lot of people seem to ignore arm when talking about fedora
15:30:30 <bconoboy> http://smoogespace.blogspot.com/2015/01/devils-proposal-moving-fedora-to-64-bit.html
15:30:42 <bconoboy> It's not a fesco proposal, yet.
15:30:45 <dgilmore> and just consider i686 and x86_64
15:30:58 <bconoboy> He calls out armv7hl by name
15:31:00 <pbrobinson> yes, I'm fuming to say the least. I'm trying to work out what my personal response is going to me
15:31:05 <pbrobinson> s/me/be
15:31:23 <pbrobinson> he's not come to the people doing ARMv7
15:31:56 <pbrobinson> and I don't remember smooge ever having anything to do with ARMv7 so I don't see he is qualified to make any decision
15:31:57 <yselkowitz1> I think the response would be to give a timeline as to when it may be feasible
15:32:25 <yselkowitz1> how soon do we think consumer-level aarch64 boards will supplant armv7?
15:32:26 <dgilmore> bconoboy: it is one persons idea
15:32:37 <bconoboy> I can't really support dropping 32 bit arm until 64 bit arm completely usurps it, like when we dropped armv5 because everything was armv7
15:32:46 <dgilmore> everyone is entitled to it. will it come to anything who knows
15:32:52 <ctyler> +1 yselkowitz1 ... i686 -> secondary is inevitable, just a question of when, and someone has to start the discussion. I don't think armv7
15:33:06 <bconoboy> dgilmore: We need to dig into the question though, because it's going to come up when we try pushing aarch64 to primary.
15:33:09 <dgilmore> bconoboy: rigth and plenty of new 32 bit arm hardware is made today
15:33:11 <ctyler> I don't think armv7hl will be of interest for more than a couple years.
15:33:13 <pbrobinson> the difference between ARMv7 and v5 is that there were stuff all useful v5 devices on the market, there's 100s of v7 devices
15:33:19 <bconoboy> People are silly and think "let's swap one for the other"
15:33:50 <dgilmore> bconoboy: when we can buy aarch64 laptops and desktops. we can look at it
15:33:51 <pbrobinson> Intel is releasing a lot of new small maker style devices that people are using that are 32 bit
15:33:55 <bconoboy> pbrobinson: Exactly.  It's going to be a long, long time before there are hundreds of v8 devices
15:34:17 <dgilmore> but sure we need to put forward a good case for why armv7 makes sense still
15:35:01 <pbrobinson> yes
15:35:13 <dgilmore> I actually do not see anyone stepping up to maintain i686 as a secondary arch
15:35:36 <yselkowitz1> but there is still need for multilibs
15:35:45 <dgilmore> yselkowitz1: no there is not
15:35:52 <yselkowitz1> for i686??
15:36:09 <yselkowitz1> i686 multilibs for x86_64 iow
15:36:16 <dgilmore> yselkowitz1: the only use case that really makes sense now is 32 bit windows applications in wine
15:36:37 <dgilmore> yselkowitz1: we have defaulted to not doing multilib installs for years now
15:36:47 <ctyler> dgilmore: websphere etc. still need 32 bit libs on x86 :-(
15:36:52 <bconoboy> As an ARM person I don't think I'm informed about the use cases for i686
15:36:54 <kylem> yeah, no. there's still a lot of 32-bit only proprietary software. you're just wrong on this.
15:36:55 <dgilmore> and you will find most people do not have i686 rpms on a x86_64 box
15:37:15 <yselkowitz1> well if you're running foss only yeah
15:37:31 <yselkowitz1> but binary-only software, isn't there still a lot 32-bit only?
15:37:38 <dgilmore> anyway. that is pretty off topic for here
15:37:58 <bconoboy> Okay, so let's keep an eye on this.  If people are having problems because of armv7hl let's address the problems.  It's going to be at least 2 years before we start seeing a decline in armv7 devices, and even that is optimistic.
15:38:21 <pbrobinson> ctyler: since when so we support websphere in Fedora?
15:38:26 <dgilmore> bconoboy: biggest complaints are that there is no java jit and that the disk io on the arm builders sucks
15:38:45 <bconoboy> dgilmore: is it really disk io? we could buy ssds
15:39:05 <dgilmore> bconoboy: disk io i think is just an issue with the sata controller calxeda used
15:39:10 <ctyler> With the armv7hl stuff, we drop support for devices all the time, release-to-release - it's not like we've continuously supported Box X for years. I can imagine a pretty quick 32-bit phaseout, especially with 64-bit stuff coming in at the same price points.
15:39:21 <dgilmore> bconoboy: its disk io but ssd's do not really help
15:39:27 <pbrobinson> bconoboy: there's OMAP3 devices that are just coming out that are being supported and shipped for 10 more years!
15:39:42 <dgilmore> bconoboy: the sata controller seems to have some bottleneck in it
15:39:50 <bconoboy> ctyler: we're still supporting the panda board!
15:40:08 <ctyler> the original one?
15:40:13 <bconoboy> yes.
15:40:33 <bconoboy> Okay, so builder hardware needs a refresh.  Any other objections people are raising?
15:41:11 <ctyler> You're not going to be able to refresh the builders with 32-bit enterprise HW. VMs on armv8?
15:41:22 <kylem> that's our plan.
15:41:43 <bconoboy> Actually, this is a good topic for here, the armv8 32 bit compatibility option.  People interested?
15:41:54 <dgilmore> bconoboy: java jit
15:42:01 <bconoboy> dgilmore: Oh, right, I'll check on that
15:42:02 <ctyler> bconoboy: Only in VM.
15:42:20 <bconoboy> There are basically 3 ways we could in theory use armv8 to host armv7 builders.
15:42:41 <pbrobinson> bconoboy: you mean you've got a solution?
15:42:55 <pbrobinson> bconoboy: I thought we were going to be doing ARMv7 VMs on ARMv8 HW
15:43:12 <bconoboy> The first is to use qemu to boot armv7 guests.  This actually works right now.  Unfortunately it's not with KVM acceleration.  Adding KVM acceleration will take months of work and nobody is lined up to do it, not at Red Hat, not at Linaro.
15:43:42 <dgilmore> bconoboy: unaccelerated qemu vms will not be acceptable
15:43:55 <dgilmore> they will be too slow
15:44:06 <bconoboy> The second is to boot a 32 bit kernel on armv8 hardware.  While theoretically possible, none of the semiconductors have upstreamed such a kernel, so it would require kernel development.  Not on anybody's drawing board.
15:44:10 <pbrobinson> bconoboy: that will be slower than the highbank boxes
15:44:40 <pbrobinson> bconoboy: so basically you're saying that everything that jcm promised has been lies
15:44:47 <kylem> no.
15:44:52 <bconoboy> The third option is to boot a 64 bit kernel on armv8 hardware, use 4k pages, and run with the 32 bit compatibility mode turned on.  This works now, but will require some tooling changes to rpm and such, since uname's output is different.
15:45:15 <kylem> bconoboy, that can be worked around in the kernel.
15:45:16 <dgilmore> bconoboy: we could use a 32 bit personality, however that will take, mock, yum, dnf, rpm work that no one has scheduled and would not give us much benefit
15:45:19 <ctyler> 32bit on armv8 baremetal has the issue that aarch32 is a large subset of armv7 -- but still a subset.
15:45:46 <bconoboy> Those are the options- they all require work. No free rides here.  Of them, I think option 3 is the most tractable.
15:46:02 <dgilmore> it would take koji work also
15:46:14 <bconoboy> dgilmore: Indeed, several packages needing updates
15:46:32 <dgilmore> bconoboy: everything would need to learn about armv8l
15:46:51 <dgilmore> or we would need to always patch the fedora kernel to say its armv7l
15:46:51 <bconoboy> dgilmore: Or we could change the kernel config to lie and call it armv7
15:47:05 <dgilmore> bconoboy: that would be true for everyone
15:47:20 <ctyler> but aarch32 != armv7
15:47:25 <dgilmore> we do not run special kernels on the builders
15:47:29 <ctyler> there are some unsupported instructions
15:47:46 <bconoboy> ctyler: which ones? do we use them?
15:47:57 <ctyler> no idea! but we might
15:48:15 <ctyler> this came up @LCU14
15:48:17 <bconoboy> dgilmore: Clearly we would make it part of the standard kernel
15:48:28 <pbrobinson> there's an option in the kernel now to emulate them
15:48:37 <pbrobinson> ARMV8_DEPRECATED
15:48:38 <dmarlin> why not just use 32-bit arm to build 32-bit arm... it works now and as posted earlier, there are new devices that are just coming out that are being supported and shipped for 10 more years
15:49:03 <dmarlin> just update the 32-bit hardware, if we are going to support it as PA
15:49:12 <bconoboy> dmarlin: Because we don't want to  be demoted to secondary architecture status and our builder hardware is not getting any faster.
15:49:16 <pbrobinson> dmarlin: because the highbank hardware is getting to EOL and it's slow and the plan was always proposed to replace it with v8 HW
15:50:00 <dgilmore> dmarlin: there is no faster 32 bit hardware available
15:50:11 <bconoboy> We haven't seen high quality server grade 32 bit hardware come out since Calxeda went under.  Their assets have been purchased so it's possible Midway will rise again, but even that is only going to provide a 30%-50% performance boost.
15:50:12 <dmarlin> and won't be in the future?
15:50:14 <pbrobinson> well no faster "enterprise" HW
15:50:49 <ctyler> right, there are faster itsy bitsy boxes, but that's not easy to integrated into a datacentre context.
15:51:26 <bconoboy> Right, so with that in mind, we have the power to start making the changes needed to host armv7hl on aarch64 systems.
15:51:44 <bconoboy> But we need to agree that's the way to go, then how to go about doing it.
15:52:05 <ctyler> Sounds like it's a lot of work to get higher-than-CX1000 performance out of any of the 3 approaches.
15:52:13 <pbrobinson> anything else we need to cover? This is sort of going off topic
15:52:15 <dmarlin> ctyler:  +1
15:52:42 <bconoboy> ctyler: option3 provides an immediate 4x performance boost.
15:53:32 <ctyler> bconoboy: and is still a *lot* of work.
15:53:53 <bconoboy> Okay, something to revisit next week
15:53:58 <bconoboy> pwhalen: That was my only topic :-)
15:54:21 <pwhalen> *only* :)
15:54:33 <pwhalen> thanks bconoboy
15:54:48 <pwhalen> weve got six minutes, anything else?
15:55:25 <pbrobinson> where's jcm to sing when you need him?
15:55:46 <bconoboy> pbrobinson: he can't sing right now, drinking coffee
15:56:04 <dgilmore> speaking of I need some
15:56:15 <ctyler> ditto!
15:56:37 <pwhalen> ok, thanks for coming all!
15:56:46 <pwhalen> #endmeeting