15:00:19 <pwhalen> #startmeeting Fedora ARM and AArch64 Status Meeting
15:00:19 <zodbot> Meeting started Tue Aug 28 15:00:19 2018 UTC.
15:00:19 <zodbot> This meeting is logged and archived in a public location.
15:00:19 <zodbot> The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:19 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:19 <zodbot> The meeting name has been set to 'fedora_arm_and_aarch64_status_meeting'
15:00:19 <pwhalen> #chair pwhalen pbrobinson
15:00:19 <pwhalen> #topic roll call
15:00:19 <zodbot> Current chairs: pbrobinson pwhalen
15:00:29 <pwhalen> morning folks, who's here today?
15:00:43 * pbrobinson o/
15:01:09 <smooge> is here in case there are deployment items
15:03:16 <pwhalen> smooge, what sort of deployment issues?
15:03:21 <pwhalen> or items
15:03:22 * jlinton waves
15:03:27 <pwhalen> howdy jlinton
15:03:59 <smooge> hardware being sent to locations. hardware needing to be brought online for testing
15:06:16 <pwhalen> ok, lets get started
15:06:18 <pwhalen> #topic 1) ==== Userspace Status  ====
15:06:42 <pwhalen> any new issues with user space?
15:07:08 <pbrobinson> nothing here that I'm aware of
15:07:34 <pbrobinson> smooge: we can discuss that towards the end
15:08:19 <pwhalen> #info No new issues.
15:08:25 <pwhalen> #topic 2) ==== Kernel Status ====
15:08:45 <pwhalen> #info F29: kernel-4.18.5-300.fc29
15:08:45 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1139115
15:08:51 <pwhalen> #info F30: kernel-4.19.0-0.rc1.git0.1.fc30
15:08:51 <pwhalen> #link https://koji.fedoraproject.org/koji/buildinfo?buildID=1139720
15:09:06 <pbrobinson> there looks to be a bunch of random problems with 4.19rc1
15:09:24 <pbrobinson> pine64 is broken
15:09:45 <pbrobinson> and Jetson TX2 has regressed to the bad state it was in before
15:09:56 <pbrobinson> at the first tests I've done
15:10:35 * pwhalen needs to test
15:10:48 <pwhalen> how does 4.18 look?
15:11:00 <pbrobinson> I needed to for a couple of reasons and it was all fire
15:11:00 <pwhalen> is anyone seeing any issues there?
15:11:24 <pbrobinson> 4.18 looks fine AFAICT, jlinton have you had a chance to test any bits?
15:11:38 <jlinton> No, I'm busy trying to automate the whole thing..
15:12:03 <jlinton> I expect in the next few days to be able to deploy fedora into our (maas) infrastructure here.
15:12:25 <jlinton> which should give us better coverage than the adhoc testing i've been doing.
15:12:26 <pwhalen> nice
15:13:00 <pbrobinson> jlinton: that's awesome, will tests etc be public?
15:13:16 <jlinton> No, but I will be opening defects/working issues as they appear
15:13:24 <pbrobinson> good news
15:14:02 <pwhalen> #info Please test and report any issues to the list or #fedora-arm.
15:14:18 <pwhalen> anything else for kernel news?
15:15:03 <pwhalen> #topic 3) ==== Bootloader Status ====
15:15:21 <pwhalen> we still have an issue with grub2 on aarch64
15:16:48 <pwhalen> #info Bug 1615969 - AArch64 - Error when booting "error: out of memory."
15:17:20 <pwhalen> anything else?
15:17:43 <pbrobinson> new u-boot rc3 should be out today I think
15:17:49 <pwhalen> nice
15:18:10 <pwhalen> will let you know if that fixes the arndale
15:19:39 <pwhalen> #topic 4)  == F29 ==
15:20:02 <pbrobinson> so beta freeze is today
15:20:55 <pwhalen> we havent had a new nominated compose in a while due to an issue with the bot, but one is expected shortly
15:21:17 <pwhalen> #info F29 Beta freeze is today (Aug 28th).
15:23:31 <pwhalen> #info New nominated compose expected this week.
15:23:40 <pwhalen> #topic 5)  == Fedora Infra ==
15:24:16 <pbrobinson> smooge: so infra size we had some cavium boxes for cloud/copr, what's the latest status of those?
15:24:55 <smooge> you were going to get and update the firmware
15:25:40 <smooge> or walk one of us in doing it at FLOCK (or that is my recollection on it)
15:25:48 <pbrobinson> smooge: I don't remember that, someone had already updated it
15:26:47 <smooge> mgalgoci did an update when the box was installed. last we checked you had some other updates you thought were needed but had not shown up.
15:26:51 <pbrobinson> they're basically the same as most x86 boxes, I think Zi said he would create you a login for the support site in the last couple of days to that thread for the gen 2 devices
15:27:41 <pbrobinson> my understanding is the latest version of el7 needs a new firmware but I've not been involved in anything aorund el7 so it's just rumours I heard, I can try and find out
15:28:53 <smooge> ok sorry.. we don't know anyone except you on this so when you said it was needed, I figured it was a hard requirement
15:29:07 <smooge> if you can get us some connections also that would be useful.
15:29:42 <smooge> and I meant connectiosn internal. The email to Zi will be very helpful for the cavium hardware
15:29:43 <pbrobinson> smooge: there's been enough problems with the Cavium that my understanding is that there's basically a new firmware with each el7 release
15:30:29 <smooge> once that box is up.. we can start interfacing it with the new cloud that is being worked on
15:31:00 <smooge> that will get it on the copr and other cloud items
15:31:04 <pbrobinson> smooge: YAY! I would try with the current firmware it has, let me know if I need to poke people
15:31:05 <smooge> boxes
15:32:05 <smooge> ok next up.. the HP moonshot goes out of warranty soonish... and has some serious hardware issues which would require a complete chassis replacement from the tech last sent
15:32:30 <pbrobinson> URGH
15:32:31 <pwhalen> :(
15:32:35 <smooge> we need to look at getting a replacement and lining up funding for it for next fiscal year (which starts in March)
15:32:57 <pbrobinson> we don't yet have a replacement that will run armv7 accelerated
15:33:31 <pbrobinson> I will likely know more in Sept though, I can sync when I know more
15:33:43 <jlinton> Right all the "serious" servers are 64-bit only.
15:33:59 <jlinton> But there are some xgene'ish class machines using ARM cores that have 32-bit
15:34:10 <smooge> this is more of a 'we need to think of a a plan C for arm'
15:35:11 <pbrobinson> smooge: well it's around timing
15:35:24 <pbrobinson> we can move the arm64 workload over to other boxes
15:35:46 <smooge> at the moment 1 of the 3 power supply sockets do not work and require a new chassis... and 1 of the sockets in the chassis makes whatever is plugged into afterwords unbootable but they don't know why
15:36:21 <pbrobinson> smooge: can the chassis not be swapped under warrantee before it expires?
15:39:51 <smooge> hopefully. the tech was of the opinion that was what was needed. the people in HP stopped responding on the ticket once it came up. What it looks like it will be is: Put in tickets, get a tech out to make the same report after replacing the parts which still don't work. Get them to send out a new chassis. Take out the old hardware and reset up the 'new' chassis (we are the only moonshot outside of silicon valley in the techs area),
15:39:51 <smooge> reset every cartridge as they are locked ot the old one
15:40:19 <smooge> we are looking at a couple days downtime and probably needing someone out there to be focused on it.
15:40:48 <pbrobinson> oh joy
15:41:22 <smooge> I don't know if that downtime would affect all of Fedora because of how koji does jobs
15:41:59 <pbrobinson> smooge: we could bring the mustangs online in the interim, I'm pretty sure we have a bunch that are unused
15:42:18 <smooge> I believe they are all in use as hardware aarch64 builders
15:42:35 <smooge> buildhw-aarch64 vs buildvm-aarch64
15:43:06 <pbrobinson> well that should give us interim capacity then depending on when it's done
15:43:18 <smooge> I was wondering about the arm side of things
15:43:48 <pbrobinson> smooge: we can put ARMv7 virt on some of those
15:44:01 <pbrobinson> they're the same CPU as in the moonshot
15:44:20 <smooge> oh ok. phew
15:44:51 <smooge> I was worried they were like the other hardware you had mentioned
15:45:03 <pbrobinson> smooge: they should be provision able in exactly the same ansible manner
15:45:49 <smooge> in that case I will put together a workplan on this, get your ok on it and see how hard HP is going to be this time
15:46:39 <pbrobinson> smooge++ thanks!
15:46:41 <zodbot> pbrobinson: Karma for smooge changed to 17 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:47:16 <smooge> no problem. I will also be back next week with update on cavium in cloud
15:47:38 <pwhalen> thanks smooge
15:47:43 <pwhalen> #topic 6) == Open Floor ==
15:47:50 <pwhalen> anything else for today?
15:48:12 <pbrobinson> nothing from me
15:48:18 <smooge> we could also plug in and turn on the old old arm32 hardware. it was still int he racks
15:48:46 <smooge> I will put that in as plan Z
15:49:37 <pwhalen> #endmeeting