20:01:31 <pwhalen> #startmeeting
20:01:31 <zodbot> Meeting started Wed Jul 11 20:01:31 2012 UTC.  The chair is pwhalen. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:01:31 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
20:01:31 <pwhalen> #chair pwhalen jonmasters bconoboy ctyler pbrobinson dgilmore
20:01:31 <zodbot> Current chairs: bconoboy ctyler dgilmore jonmasters pbrobinson pwhalen
20:01:41 <pwhalen> #topic 1) F18 Mass rebuild - What needs to be done in preparation?
20:01:53 <jcapik> .fas jcapik
20:01:53 <zodbot> jcapik: jcapik 'Jaromír Cápík' <jcapik@redhat.com>
20:01:59 <djdelorie> .fas djdelorie
20:01:59 <zodbot> djdelorie: djdelorie 'DJ Delorie' <dj@redhat.com>
20:02:05 <pwhalen> .fas pwhalen
20:02:05 <zodbot> pwhalen: pwhalen 'Paul Whalen' <pwhalen@redhat.com>
20:02:08 <dmarlin> .fas dmarlin
20:02:10 <zodbot> dmarlin: dmarlin 'David A. Marlin' <dmarlin@redhat.com>
20:02:50 <djdelorie> is there a feature page for the F18 rebuild?  Aside from the ARM-specific issues?
20:03:14 <bconoboy> .fas blc@
20:03:15 <zodbot> bconoboy: blc '' <blc@redhat.com>
20:03:18 * pbrobinson is here
20:03:25 <pbrobinson> sorry board meeting is still running
20:03:50 <bconoboy> We need the armhfp gcc patch to land before the F18 mass rebuild.  GCC team is at gcc cauldron, but will hopefully get to it early next week.
20:04:52 <pbrobinson> as far as I'm concerned from the ARM side of things that's the only thing really blocking the mass rebuild from the ARM perspective
20:05:10 <djdelorie> provided we have sufficient builder hardware by then
20:05:15 <dgilmore> hola
20:05:30 * djdelorie does NOT want us to fall behind again
20:05:34 <pbrobinson> djdelorie: we've already done one mass rebuild
20:05:51 <dgilmore> djdelorie: https://fedorahosted.org/rel-eng/ticket/5222 is the ticket to releng requesting it
20:05:53 <pbrobinson> and we're in a much better shape than we were when we were doing F-17 mass rebuild
20:05:54 <bconoboy> We should get the builders in tip-top shape before the mass rebuild as well.
20:06:40 <pbrobinson> we also need the koji team to monitor it closely so we don't lose HW / nfs mounts / hsv builders etc for 24 hours at a go
20:06:44 <djdelorie> it would be sweet to just say "we're done too" shortly after PA is done :-)
20:06:48 <dmarlin> bconoboy: move them to F17?
20:07:09 <bconoboy> Moving them to F17 seems like a good idea if we can do it early.  Definitely for the pandas since they are unstable F15 builders.
20:07:22 <pbrobinson> dmarlin: they've been pretty stable on what we have, is there a specific fix/need to move them?
20:07:48 <dmarlin> bconoboy: we have two trimslice builders in testing now (in arm.koji)
20:07:49 * djdelorie thinks we should move builders to F17 "just because"
20:08:01 <pbrobinson> and we have around 10 days in which to do that because mainline is kicking off the mass rebuild at the end of the month
20:08:20 <pbrobinson> I would prefer not to have breakages "just because"
20:08:44 <bconoboy> we'll have more breakages because f15's kernels aren't tremendously stable.
20:08:54 <dmarlin> pbrobinson: our previous rootfs was cobbled together and the kernels we have still produce errors and have to be restarted sometimes
20:09:14 <pbrobinson> if there is direct improvements we get by moving them, sure but I don't want to be in the situation where we lose koji for a week as a result of it
20:09:16 <jcapik> bconoboy: How many F15 failures have you experienced?
20:09:32 <dmarlin> pbrobinson: I can't verify that f17 is more stable, but we are running builders to test this
20:09:33 <bconoboy> jcapik: dozens and dozens.
20:09:55 <pbrobinson> dmarlin: exactly my point, there's no proof at the moment that F17 is more stable as a builder
20:10:10 <dmarlin> pbrobinson: that's why we're testing
20:10:19 <dmarlin> and have been for over a week
20:10:20 <jcapik> Haven't we already agreed that we'll move them to F17 one by one?
20:10:35 <bconoboy> we have.  I propose we move pandas this week.
20:10:53 <bconoboy> the trimslices have been solid
20:11:02 <dmarlin> bconoboy: I'd prefer to start with trimslice, since that's what we've tested
20:11:12 <dmarlin> bconoboy: and test a couple of pandas
20:11:12 <pwhalen> how about a test panda then?
20:11:17 <bconoboy> dmarlin: Sure, we can finish trimslice transition... but let's start on pandas
20:11:27 <dmarlin> bconoboy: sounds good
20:11:44 <bconoboy> Okay, so going back to the question: More F17-builders for F18 mass rebuild is a good thing
20:11:51 <djdelorie> +1
20:12:00 <dmarlin> +1
20:12:01 <jcapik> If there's a small chance, that F17 builders could be more stable .... I would give it a try
20:12:16 <jcapik> +1
20:12:20 <pwhalen> I'll work with Seneca on that one, and ensure there is no impact on koji availability
20:12:36 <bconoboy> I'll see about moving more of the HSV Builders to UPS power.
20:12:38 <dmarlin> has anyone seen any issues with the hsv trimslice builders -7 and 8?
20:13:15 <dgilmore> dmarlin: not that ive seen
20:13:16 <dmarlin> those are the test systems (one sfp, one hfp)
20:13:21 <jcapik> we can move them back to F15 if we encounter any issues ..... can't we?
20:13:26 <bconoboy> we can
20:13:32 <bconoboy> or we can fix the issues :-)
20:13:45 <djdelorie> moving forward is better than moving backwards
20:13:47 <jcapik> bconoboy: ok .... then I see no reason for staying at F15
20:13:57 <bconoboy> Okay, so the plan for this week is: Move the remaining trimslice builders to F17, and make a panda test image for 1 hfp, 1 sfp
20:14:10 <pwhalen> we'll keep the f15 cards just in case we need to switch back quickly
20:14:20 <jcapik> pwhalen: +1
20:14:24 <dmarlin> pwhalen: +21
20:14:32 <dmarlin> +1
20:14:35 <jcapik> dmarlin: woooooow
20:14:50 <bconoboy> dmarlin's been banking approval ;-)
20:14:53 <jcapik> dmarlin: thats +22
20:15:09 <bconoboy> Anything else that needs to land for F18 mass rebuild?
20:15:12 <pbrobinson> 22 SD cards?
20:15:23 <jcapik> pbrobinson: :D :D :D
20:15:34 <djdelorie> are we space-constrained enough to *not* want MiniDebugInfo ?
20:15:36 <jcapik> pbrobinson: good one
20:15:42 <djdelorie> OLPC is considering dropping it
20:16:01 <djdelorie> because their flash is so small
20:16:18 <pbrobinson> olpc doesn't want it at all and wants a means of easily removing it which bundling in the base rpm doesn't provide
20:16:27 <dgilmore> djdelorie: thats not an option we have
20:16:38 <pbrobinson> so if its there they want it in another rpm or similar
20:17:56 <pwhalen> #action pwhalen to work with Seneca to update two Pandaboards, remainder of the Trimslices to F17
20:18:24 <pwhalen> anything else we need to discuss for the rebuild? or move on?
20:19:04 <pwhalen> #topic 2) F18 build status - problem packages?
20:19:07 <pbrobinson> what's happening with the other non panda/TS devices?
20:19:18 <bconoboy> pbrobinson: Stay F15 for now I imagine
20:19:28 <pbrobinson> ok
20:19:39 <bconoboy> I'd suggest systems that are F15 today stay F15.  New systems (highbank?) might be provisioned F17.
20:20:11 <pwhalen> not sure if there is anything to discuss here, but added it in.. pbrobinson?
20:20:13 <dgilmore> anything new should get f17 installs
20:20:17 <dmarlin> bconoboy: +1 to provisioning new builders as f17
20:20:26 <bconoboy> Problem packages: gcc (armhf), anaconda (needs external patches), openmpi (needs atomics), llvm (needs who knows what)...
20:20:54 <pbrobinson> llvm needs a patch for clang to properly see the correct triplet
20:21:22 <pbrobinson> the java-1.7.0-openjdk package that has been causing massive problems was confirmed fixed today
20:21:30 <pbrobinson> but has only has light testing
20:21:45 <bconoboy> oh, and kernel.. we need kernel testing
20:21:53 <pbrobinson> the other two problem packages are libdb which is suddenlt ftbfs
20:21:59 <dgilmore> pbrobinson: though it should be fine since for secopndary arches its the old version
20:22:24 <pbrobinson> and there's a problem with systemtap which is currently in a disagreement between kernel and glibc for a proper fix
20:23:54 <pbrobinson> so at the moment the two current F18 build related issues that are blocking most builds at the moment are libdb and systemtap. All the rest don't block builds majorly (or any more than they currently do in F17)
20:24:01 <pbrobinson> EOF
20:24:32 <bconoboy> is there a bz for each?
20:24:52 <pbrobinson> there is and they're linked against the ARM tracker bug :-P
20:25:09 <pbrobinson> systemtap https://bugzilla.redhat.com/show_bug.cgi?id=837641
20:25:19 <pbrobinson> libdb https://bugzilla.redhat.com/show_bug.cgi?id=838334
20:25:39 <pbrobinson> java-1.7.0-openjdk as reference https://bugzilla.redhat.com/show_bug.cgi?id=837403
20:27:09 <pwhalen> #topic 3) When do we begin testing Rawhide?
20:27:15 <dgilmore> now
20:27:34 <dgilmore> ive been trying to run rawhide on my pandaboard
20:27:40 <dgilmore> and im hitting kernel issues
20:28:01 <dgilmore> the sooner we run rawhide the quicker we test kernels and find and fix bugs
20:28:02 <bconoboy> we need images to test
20:28:10 <jcapik> dgilmore: what issues?
20:28:13 <dgilmore> you can yum update
20:28:13 <pwhalen> dgilmore, you did a yum update?
20:28:18 <dgilmore> pwhalen: i did
20:28:20 <pbrobinson> rawhide should be testing now. I would love to see the anaconda bits done so that we can be building nightly images with that
20:28:26 <pwhalen> will do so now too
20:28:27 <dgilmore> jcapik: the sdcard doesnt show up
20:28:45 <dgilmore> jcapik: and im seeing issues very much the same as the issue we had with audit
20:28:47 <jcapik> dgilmore: ? like .... you had no rootfs?
20:28:49 <bconoboy> we can already make anaconda-based images with some not-quite-upstream bits
20:28:55 <dgilmore> jcapik: yes,
20:29:07 <jcapik> dgilmore: ..... and you were unable to boot
20:29:22 <dgilmore> jcapik: dracut drops to a recovery shell
20:29:23 <pbrobinson> excellent so can we start doing that and can someone document how to do that somewhere?
20:29:35 <dgilmore> modprobing omap doesnt make the sdcard show up
20:29:58 <jcapik> dgilmore: maybe it's caused by dracut .... it's still broken :]
20:29:58 <pbrobinson> and what's the time frame to get the not-quite-upstream bits upstream?
20:30:19 <bconoboy> working on it right now.
20:30:25 <jcapik> dgilmore: I submitted 2 bugs yesterday and today
20:30:25 <bconoboy> Aiming for before mass rebuild.
20:31:27 <dgilmore> jcapik: its not, i have installed the rawhide kernel on a otherwise f17 system and the result is the same
20:31:42 <jcapik> dgilmore: oh .... ok then
20:32:34 <pwhalen> so should we make some images to begin testing f18 now? or just yum update our existing systems?
20:32:53 <pbrobinson> pwhalen: both
20:33:01 <dgilmore> pwhalen: both
20:33:18 <pbrobinson> dgilmore: jinx ;-)
20:33:18 <jcapik> +1
20:33:29 <jcapik> =both
20:33:30 <bconoboy> who is going to make images?
20:33:46 <dgilmore> bconoboy: who wants to do arm releng?
20:33:49 * pbrobinson bags not!
20:33:55 * bconoboy is not it
20:34:18 <bconoboy> dgilmore: It seems like it should be automateable
20:35:07 <dgilmore> bconoboy: yes, it should be
20:35:24 <dgilmore> but its the kind of thing that falls into releng category
20:35:26 <jcapik> and is it? :]
20:35:43 <dgilmore> jcapik: should be :)
20:35:55 <dgilmore> needs someone to do the work
20:36:03 <bconoboy> once it's in place I'd be happy to retire the F17 nightlies.
20:36:03 <jcapik> dgilmore: should be, could be, wants to be
20:36:17 <bconoboy> pwhalen: Can you do it?
20:37:04 <pwhalen> bconoboy, sure can
20:37:18 <djdelorie> +1 to pile all the problems on pwhalen :-)
20:38:17 <pbrobinson> pwhalen: please yell and scream if you need help and assistance
20:38:21 <dmarlin> bconoboy: what type of images are we looking to build?
20:38:30 <pwhalen> pbrobinson, will do
20:38:47 <bconoboy> dmarlin: like the nightlies
20:39:06 <dmarlin> bconoboy: we have only tested livemedia-creator on highbank and trimslice.  I think dgilmore has been working on panda.
20:39:09 <bconoboy> we can start with just trimslice, move on to panda... until it's all done
20:39:23 <dmarlin> bconoboy: ok
20:39:39 <pbrobinson> I think to start with if we do a couple of minimal builds for the core platforms
20:40:04 <dgilmore> we should start with the work ive done in spins-kickstart
20:40:12 <dmarlin> pbrobinson: the trick is with the systems that require u-boot on the rootfs
20:40:23 <dgilmore> dmarlin: thats not hard
20:40:33 <dmarlin> dgilmore: it was for me.
20:40:43 <bconoboy> we can workout details later
20:40:44 <dgilmore> dmarlin: works for me on a pandaboard
20:41:09 <bconoboy> next topic?
20:41:13 <dmarlin> dgilmore: great.  please point me to the bits
20:41:16 <jcapik> bconoboy: +1
20:41:23 <pwhalen> #topic 4) Your topic here
20:41:34 <bconoboy> pi update? maxam? ctyler_onroad?
20:41:40 <pwhalen> that was all I had
20:42:02 * dgilmore is going to be in australia for 6 weeks from July 22 through Sept 2.
20:42:26 <dgilmore> this meeting starts at 6am so there is probably a good chance ill make it
20:42:35 * djdelorie is redoing the stage1-3 bootstrapping scripts, if anyone wants to try them
20:42:47 <dgilmore> but im going to be available at different times and will be taking a bit of time off
20:43:08 <dgilmore> i likely will only take a few arm systems with me also
20:43:19 <pbrobinson> general rawhide build update: I'm doing the perl 5.16 rebuild at the moment so the core koji-shadow is off. Once i have the core bits in place needed for the build side of things I'll tag it all in and let the main f18 koji-shadow deal with it but just a heads up that over the next day or so there will be some churn in the rawhide repos
20:46:08 <pwhalen> anything else for today or shall we wrap it up?
20:46:14 * jcapik is still waiting for his RPi :'( RS components are soooo sloooooooowwwwwwww
20:46:28 <bconoboy> sounds like a wrap
20:46:34 <jcapik> yup :]
20:46:40 <pwhalen> #endmeeting