16:31:25 <dgilmore> #startmeeting RELENG (2016-02-29)
16:31:25 <zodbot> Meeting started Mon Feb 29 16:31:25 2016 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:31:25 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:31:25 <zodbot> The meeting name has been set to 'releng_(2016-02-29)'
16:31:25 <dgilmore> #meetingname releng
16:31:25 <zodbot> The meeting name has been set to 'releng'
16:31:25 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion
16:31:25 <zodbot> Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll
16:31:28 <dgilmore> #topic init process
16:31:32 <maxamillion> .hello maxamillion
16:31:33 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
16:31:38 <nirik> morning everyone
16:31:57 * sharkcz is here
16:33:26 <dgilmore> lets get started I guess
16:33:37 * pbrobinson is here
16:33:44 <dgilmore> #topic Pungi status
16:33:59 <dgilmore> last week was a mess
16:34:16 <nirik> lets hope the light at the end of the tunnel is not a train. ;)
16:34:27 <maxamillion> nirik: +1
16:34:36 <dgilmore> It did get a little better as the week went on
16:34:55 <dgilmore> right now the mail to devel@ is going to the ethers somewhere
16:35:30 <dgilmore> there has been no feedback on the contents of the compose
16:35:44 <nirik> I've got abompard looking into the email issue.
16:35:44 <dgilmore> there has been some on the compose report
16:36:03 <dgilmore> mostly complaints about things some people do not like
16:36:14 <pbrobinson> I've been meaning to look closer at the contents of the compose but I've been drowning in a bunch of other issues
16:36:17 <dgilmore> still need to get atomic working
16:36:28 <dgilmore> need to get secondaries moved
16:36:38 <dgilmore> we broke mirrormanager
16:37:40 <pbrobinson> I plan on dealing with secondaries this week
16:37:58 <maxamillion> dgilmore: how'd pungi break mirrormanager?
16:38:12 <dgilmore> secondaries are going to need forks of the configs :(
16:38:22 <nirik> there's now a bunch of trees... and it didn't know which one to point everyone at
16:38:22 * linuxmodder hanging in back
16:38:34 <dgilmore> maxamillion: mirrormanager picked Workstation for f24 and did not update the rawhide location
16:38:52 <pbrobinson> dgilmore: yes, I'm aware of that
16:38:57 <dgilmore> right now most of the compose is broken due to the glibc changes
16:39:36 <nirik> is that hopefully going to get fixed today?
16:40:19 <dgilmore> nirik: maybe
16:40:30 <pbrobinson> it looks like Carlos is on it, he's been replying to threads, we can probably work around it by explicitly pulling in all the langpacks
16:40:43 <dgilmore> pbrobinson: its nothing to do with him
16:40:47 <nirik> seems like anaconda should just require all
16:40:56 <dgilmore> anaconda needs to make sure that what it needs is available
16:40:59 <pbrobinson> dgilmore: Oh, want to extrapolate more?
16:41:35 <pbrobinson> dgilmore so why would that be worked around by pulling in all the langpacks?
16:41:37 <dgilmore> https://bugzilla.redhat.com/show_bug.cgi?id=1312951
16:41:38 <dgilmore> https://bugzilla.redhat.com/show_bug.cgi?id=1312607
16:42:00 <dgilmore> pbrobinson: anaconda fails to run because it can not set the locale
16:42:06 <maxamillion> dgilmore: you said there was a mailing list thread on this? ... I either missed it or just haven't gotten to it yet, would you mind linking it?
16:42:40 <dgilmore> https://www.redhat.com/archives/anaconda-devel-list/2016-February/msg00039.html
16:42:53 <dgilmore> i sent an email to anaconda-devel but have not yet got a response
16:43:35 <maxamillion> dgilmore: +1 thanks
16:44:01 <dgilmore> the arm images build, but I expect that initail-setup fails to run due to the locales being missing
16:44:01 <sharkcz> https://github.com/rhinstaller/lorax/pull/93
16:44:11 <sharkcz> ?
16:44:12 <pbrobinson> it looks like dshea has some pulls to help it out a little
16:44:17 <dgilmore> the livecds and cloud images fail to build because anaconda can not run
16:45:06 <dgilmore> sharkcz: thats pretty woefully incomplete
16:45:20 <dgilmore> but would allow the installer isos to work again
16:45:29 <dgilmore> but would not fix any other use case
16:45:37 <maxamillion> all in stride :)
16:45:46 <nirik> this is fine.
16:45:50 <maxamillion> nirik: +1
16:45:54 <maxamillion> nirik++
16:47:11 <dgilmore> anyway it has been rough
16:47:22 <maxamillion> dgilmore: sounds like it :/
16:47:36 <dgilmore> there is unanticipated complications
16:47:56 <dgilmore> and the usual people shoving all their shit in last minute
16:48:19 <pbrobinson> sadly this always seems to be the case for both points :-(
16:48:24 <dgilmore> yeah
16:48:32 <maxamillion> pbrobinson: +1 - I was just typing basically that
16:49:02 <dgilmore> I fully expect that glibc split will have a lot more fallout yet
16:50:17 <dgilmore> #topic Secondary Architectures updates
16:50:17 <dgilmore> #topic Secondary Architectures update - ppc
16:50:25 <dgilmore> lets move on
16:50:32 <dgilmore> we could discuss pungi all day
16:50:42 * pbrobinson would rather not ;-)
16:50:48 <dgilmore> pbrobinson: how is ppc?
16:50:50 <pbrobinson> so PPC is getting there
16:51:03 <pbrobinson> we got a new gcc that fixed some maths issues
16:51:20 <pbrobinson> and that allowed me to spend a bunch of the weekend on fixing up builds there
16:51:34 <dgilmore> cool
16:51:42 <pbrobinson> so we've got around 4K builds left to catch up now on the mass rebuild, so around 75% done
16:52:04 <pbrobinson> there's some kernel issues and a few other bits
16:52:21 <dgilmore> okay
16:52:37 <pbrobinson> depending on when nirik can get me some cables plugged into the P8s there will be a bunch of movement soon on ppc infra
16:52:48 <pbrobinson> but it should not overly affect most people
16:52:53 <nirik> yeah, ticket is in on it...
16:53:06 <pbrobinson> just my sleep patterns but not my life as I don't have one
16:53:31 <nirik> those cloud p8's should be in now, need to get them racked and setup. ;)
16:53:38 <pbrobinson> I'm going to rebuild the compose boxes tomorrow either way
16:53:49 <sharkcz> nirik: good news :-)
16:54:05 <pbrobinson> nirik: that's cool, let me know when they're there, although they're a lower priority than getting f24 composing
16:54:11 <nirik> (I don't have confirm of that, but will try and get it)
16:54:44 <nirik> once they are up I want to add some ppc64{be|le} instances for test machines for package maintainers... let them debug things.
16:54:48 <pbrobinson> so basically PPC for me this week is 1) close out mass rebuild 2) pungi4 3) infra maintenance
16:55:16 <dgilmore> sounds like a plan
16:55:27 <pbrobinson> nirik: the plan is to put them both in the OpenStack cloud so they should be deployable in the same means as the x86 stuff from that PoV
16:55:42 <maxamillion> pbrobinson: what about PPC needs pungi4 improvements? are the composes different?
16:55:44 <nirik> yeah, that was my understanding... which will be pretty slick. ;)
16:56:01 <pbrobinson> maxamillion: nothing with luck, but none of it's configured yet
16:56:02 <maxamillion> pbrobinson: (that's mostly for my education, I don't know much about the ppc stuff)
16:56:06 <maxamillion> pbrobinson: ahhh, +1
16:56:13 <dgilmore> maxamillion: we will need some builders in teh compose channel only, with /mnt/koji moounted rw
16:56:23 <dgilmore> then just the configs
16:56:27 <maxamillion> dgilmore: rgr
16:57:43 <dgilmore> anything else on ppc?
16:57:53 <pbrobinson> not from me
16:57:57 <dgilmore> #topic Secondary Architectures update - s390
16:58:03 <dgilmore> sharkcz: how is s390?
16:58:34 <sharkcz> not bad :-) we should finish first full cycle over the mast rebuild today/tomorrow
16:58:39 <nirik> we are going to have to come up with some pungi4 plan for s390. :(
16:58:49 <sharkcz> but have at least 2 gcc6 bug candidates
16:59:10 <dgilmore> nirik: indeed
16:59:48 <sharkcz> yep, it will be the hard part ...
17:01:02 <dgilmore> anything else?
17:01:11 <sharkcz> nope
17:01:22 <dgilmore> #topic Secondary Architectures update - arm
17:01:30 <dgilmore> how is aarch64 pbrobinson ?
17:01:57 <pbrobinson> not looking too bad, about 2500 packages left in mass rebuild
17:02:17 <pbrobinson> nothing else major blocking
17:02:23 <dgilmore> cool
17:02:26 <nirik> moonshot is up and networked now BTW. I am able to pxe boot...
17:02:30 <pbrobinson> similar to PPC will be looking at pungi4 this week
17:02:34 <nirik> but somehow it didn't install right.
17:02:41 <pbrobinson> nirik: yes, it's also on my todo list
17:02:54 <pbrobinson> nirik: there's likely a number of issues need
17:03:02 <pbrobinson> nirik: what were you trying to install?
17:03:23 <nirik> I don't recall... whatever was setup on noc01 in grub config...
17:03:24 <maxamillion> aarch64 moonshot?
17:03:42 <nirik> maxamillion: HP moonshot. It's a big thingie with cartridges that are aarch64 machines.
17:03:42 <dgilmore> maxamillion: yes
17:03:52 <nirik> we have 30 carts
17:04:09 <nirik> I might try and poke at it some more if I have time too.
17:04:20 <pbrobinson> maxamillion: it's basically a faster mustang, same SoC, 64 GB RAM and 512Gb SSD on each blade
17:04:31 <maxamillion> nirik: I'm familiar with moonshot, I remember it was going to be arm 32bit but then intel atom something or other, didn't realize they switched to aarch64 .... that's awesome
17:04:44 <maxamillion> pbrobinson: rgr, swank
17:04:52 <nirik> it was f23 I tried...
17:04:58 <pbrobinson> nirik: so what we currently have won't install directly because ACPI
17:05:07 <nirik> ah whee. ;(
17:05:17 <pbrobinson> nirik: don't waste your time with F-23
17:05:31 <nirik> ok. I can leave it to you too... but also happy to assist
17:06:00 <pbrobinson> nirik: that's why I've said I'll look at moonshot once I have everything else done for f24 composing because it'll be down the rabbit hole for a bit
17:06:04 <nirik> I really would love to get armv7 builders up on them and power off the calexida stuff. ;)
17:06:11 <nirik> yeah.
17:06:38 <nirik> the carts are actually all installed from their time in internal testing... if that helps any.
17:06:43 <pbrobinson> nirik: the ARMv7 on aarch64 still will be a month or so away as there's a bunch of upper layer work to do
17:07:16 <pbrobinson> nirik: yep, I need to get on and have a look, I have a number of fall back plans, I just need more than 24 hours in a day in the short term
17:07:26 <nirik> yeah, I hear that. ;)
17:07:33 <pbrobinson> nirik: about 6 levels of fall back actually!
17:09:20 <dgilmore> anything else that needs covered?
17:09:29 <pbrobinson> nope
17:09:45 <dgilmore> #topic open floor
17:10:04 <dgilmore> I had something 10 minutes ago
17:10:09 <dgilmore> but forget now
17:10:19 <dgilmore> maxamillion: hows the osbs work?
17:10:32 <dgilmore> when do you expect it to be deployed?
17:10:43 <maxamillion> dgilmore: I hope to have it deployed today
17:10:49 <nirik> maxamillion: oh, speaking of, I added some questions to the RFR ticket on there, I guess Smooge filed it and you aren't cced tho...
17:10:52 <maxamillion> dgilmore: it's all in ansible to the best of my knowledge, now it's just deploy and test
17:11:13 <dgilmore> maxamillion: okay cool
17:11:15 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/5126#comment:3
17:11:21 <maxamillion> nirik: +1 - will update
17:11:35 <nirik> thanks. Should have asked before, but forgot
17:12:00 <maxamillion> dgilmore: I need to figure out why the role I added isn't executing when I run the playbook but I'll look into that after the meeting
17:12:10 <maxamillion> nirik: no worries, stuff's been a bit crazy
17:12:21 <nirik> always. ;)
17:13:02 <maxamillion> the mottos of Fedora are basically "everything is crazy" and "this is fine"
17:13:06 <maxamillion> ;)
17:13:34 <dgilmore> maxamillion: you forgot our new one "Faster, Faster, Faster"
17:13:43 <maxamillion> dgilmore: oh yes, I did indeed ... the new one
17:13:50 <maxamillion> we should start a running list somewhere
17:15:00 <dgilmore> :)
17:15:31 <dgilmore> okay I can not remeber what I wanted to bring up
17:15:34 <maxamillion> but yeah, basically at this point for osbs stuff I just need to make sure it deploys right and test and then get a couple security things evaluated before we go to prod ... just to make sure our i's are dotted and t's crossed
17:15:37 <dgilmore> does anyone have anything?
17:15:53 <dgilmore> cool
17:16:57 <dgilmore> oh one thing
17:17:33 <dgilmore> we need to sit down with adamw and come up with a list of fixes for productmd
17:18:15 <maxamillion> dgilmore: just out of curiosity, what's that list for?
17:18:16 <dgilmore> we also need to sit down with threebean and figure out some RFE's for PDC
17:18:19 <adamw> i think the payload thing is all i'm really missing
17:18:52 <dgilmore> maxamillion: it has a bunch of hardcodedness that does not make sense in our world
17:19:06 <maxamillion> ahh, rgr
17:19:07 <adamw> oh wait, pdc, i was thinking of productmd.
17:19:24 <dgilmore> we also need to make sure that and productmd changes are handled in pdc
17:19:40 <dgilmore> since inmporting composes is brokenb in who knows what way
17:19:47 <dgilmore> adamw: well both
17:20:25 <adamw> everything is broken! :)
17:20:31 <dgilmore> adamw: it is
17:20:53 <dgilmore> looking at pdc I am not sure that it actually defines anything
17:21:04 <dgilmore> it just seems to record datsa
17:21:06 <dgilmore> data
17:21:10 <adamw> i didn't really get to the point where i have a laundry list for PDC yet, but i'm sure I will. for now the only thing I've run into is not being able to tell what you can do with the API, and if you can't do stuff like specify ranges for dates, i'd like that
17:21:23 <dgilmore> but I do not see how we can use it to define rings
17:22:51 <maxamillion> almost seems like rings need to be defined in pungi and then PDC would just be the central information store that everything else could query from
17:23:10 <maxamillion> threebean would likely have more insight into that though
17:23:20 <adamw> from what i've heard it *does* seem like they just want PDC to be more or less a dumb information store
17:23:21 <dgilmore> maxamillion: that is not how things should work
17:23:45 <adamw> i get the feeling they want us to make new...things...to do policy stuff like "based on this information in PDC we can do X, Y and Z"
17:24:04 <dgilmore> ::sigh:: okay
17:24:20 <adamw> just my impression, though.
17:24:27 <dgilmore> I guess I will have to talk with stano more
17:24:58 <dgilmore> anyway, thats next weeks problem
17:25:37 <pbrobinson> that's the impression I had got from PDC
17:25:53 <dgilmore> if no one has anything else lets wrap up
17:27:37 <dgilmore> #endmeeting