15:50:30 <dgilmore> #startmeeting RELENG (2015-10-12)
15:50:30 <zodbot> Meeting started Mon Oct 12 15:50:30 2015 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:50:30 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:50:41 <dgilmore> #meetingname releng
15:50:41 <zodbot> The meeting name has been set to 'releng'
15:50:41 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion
15:50:41 <zodbot> Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll
15:50:44 <dgilmore> #topic init process
15:50:54 * sharkcz is here
15:50:55 <tyll> Hi there
15:50:57 <nirik> .hello kevin
15:50:57 * maxamillion is here
15:50:58 <zodbot> nirik: kevin 'Kevin Fenzi' <kevin@scrye.com>
15:51:00 <maxamillion> .hello maxamillion
15:51:01 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
15:56:14 <maxamillion> ?
15:56:29 <nirik> is this thing on? ;)
15:56:51 <maxamillion> nirik: I *think* so ...
15:57:10 <dgilmore> so we have no tickets to go over
15:57:19 <nirik> cool.
15:57:22 <dgilmore> sorry was just eyeing up the tickets
15:57:40 <dgilmore> before we do secondary arches, I wanted to cover the change to dnf
15:57:50 <dgilmore> #topic rawhide buildroot changed to dnf
15:57:54 <nirik> I closed https://fedorahosted.org/rel-eng/ticket/6276 because it's a expected dnf change. We might want to note it to devel-announce?
15:58:15 <dgilmore> somehow there was one x86 and about 5 arm boxes that did not get the needed dnf update
15:58:28 <dgilmore> there is a few small number of issues reported
15:59:31 <nirik> dgilmore: I did reinstall a few of them the other day.
15:59:42 <nirik> buildhw11 was down I think, I had to power cycle it.
15:59:52 <dgilmore> nirik: dnf is in updates-testing
16:00:06 <nirik> ok. I may have not pulled that in, so thanks for fixing that.
16:00:10 <dgilmore> we should give it some karma
16:00:13 <pbrobinson> what's needed to get it into updates
16:00:23 <dgilmore> pbrobinson: karma
16:00:38 <nirik> the plan is still to move builders to f23 after release right?
16:00:39 <pbrobinson> what's the NVR?
16:00:51 <dgilmore> https://bodhi.fedoraproject.org/updates/FEDORA-2015-9bcdcdc46b
16:00:55 <dgilmore> thats the update
16:01:00 <dgilmore> nirik: yeah
16:01:04 <nirik> cool
16:01:37 <pbrobinson> have we tested things like EPEL-6 on f23 with dnf installing the chroot?
16:02:47 <dgilmore> pbrobinson: no because that will never happen
16:03:00 <nirik> it should use yum for that. ;)
16:03:14 <nirik> only f23+ buildroots will it use dnf
16:03:14 * pbrobinson is lost
16:03:19 <dgilmore> only f24 and up will use dnf to install teh buildroot contents
16:03:22 <nirik> sorry, f24
16:03:23 <nirik> yeah
16:04:05 <pbrobinson> yes, but on a F-23 builder doesn't mock use the native dnf to install the EL6 buildroot first?
16:04:07 <nirik> this dnf version is needed on the builders to make f24+ buildroot making work
16:04:10 <dgilmore> only rawhide/f24 and newer fedora's will have thier buildroots created by dnf
16:04:19 <dgilmore> pbrobinson: no
16:04:43 <nirik> koji tells mock to use yum or dnf as desired
16:04:57 <dgilmore> you will only have dnf used if it is specified in the mock config
16:04:58 <nirik> is that patch upstream BTW, someone was asking about it on devel this weekend
16:05:32 <dgilmore> nirik: the patch is upstream, but all it does is species the yum command to yum-deprecated if it exists on the host
16:05:59 <pbrobinson> so mock uses yum-deprecated on a f23 host for el6?
16:06:06 <dgilmore> pbrobinson: yes
16:06:17 <dgilmore> and el7
16:06:22 <pbrobinson> ok
16:07:23 <pbrobinson> we have that in staging yet?
16:07:40 <dgilmore> I suspect that there will be more issues filed, and most of them will be due to changes in behaviour in dnf
16:07:50 <dgilmore> f23 builders?
16:08:07 <dgilmore> nirik: what would it take to put a f23 builder or two in staging?
16:08:27 <nirik> we could sure.
16:08:32 <nirik> should be pretty easy
16:08:38 <pbrobinson> yes, f23 builders to test all the various senarios
16:09:37 <dgilmore> #info we will get some f23 builders in staging koji
16:09:37 <nirik> I can try and do that later this week...
16:10:24 <dgilmore> thanks
16:10:40 <dgilmore> lets all try keep on top of and answer tickets quickly
16:11:18 <nirik> the reporter of 6276 was not happy and wants to reopen it.
16:11:18 * maxamillion needs to get better at checking the ticket queue and being useful tehre
16:11:21 <maxamillion> there*
16:11:24 <nirik> but there's pretty much 0 we can do about it.
16:12:55 <dgilmore> nirik: I think its actually a bug in f21's dnf
16:13:23 <dgilmore> nirik: as on my laptop "dnf install /bin/tcsh" works
16:13:33 <nirik> I don't think so. They have explicitly said thats not supported
16:13:45 <dgilmore> it does work in f23
16:14:19 <nirik> they may have changed the isntall case.
16:14:23 <nirik> but this is builddep
16:14:39 <pbrobinson> nirik: I just commented on that actually
16:14:39 <tyll> dgilmore: afaik it might work if there is already a symlink installed on your local system
16:14:43 <dgilmore> rdieter reported in the dicket builddep worked also
16:15:00 <dgilmore> tyll: if that is the casde it is horribly broken in dnf
16:15:28 <pbrobinson> tried to comment even, fuck trac
16:16:15 <nirik> no idea then. I thought it was expected....
16:16:25 <dgilmore> I am not sure
16:16:48 <dgilmore> #action dgilmore to find who in the dnf team to work with on dnf issues
16:17:01 <dgilmore> I will find a contact person to work with
16:17:09 <pbrobinson> I thought it was as expected too, afterall usrmove feature was years ago
16:17:37 <maxamillion> dgilmore: +1
16:17:37 <dgilmore> pbrobinson: yeah. In this case I think that its bugs in the packaging
16:19:02 <maxamillion> dgilmore: I feel like when we file BZs it's just http://screamintothevoid.com/ ... so having a point of contact would be nice for at least rel-eng impacting issues in dnf
16:19:06 <nirik> dnf builddep also works in rawhide.
16:19:14 <maxamillion> (or at least when I file BZs ... YMMV)
16:19:14 <nirik> so I guess you are correct that it's a f21 thing.
16:19:26 <dgilmore> maxamillion: indeed, it took quite a few weeks to fix the bugs so we could make this change
16:20:20 <dgilmore> does anyone want to talk about this more?
16:20:49 <dgilmore> #topic Secondary Architectures updates
16:20:50 <dgilmore> #topic Secondary Architectures update - ppc
16:21:04 <dgilmore> pbrobinson: how is ppc going?
16:21:53 <pbrobinson> looking pretty good, spun a TC1 to test anaconda and some installer stuff today
16:22:27 <pbrobinson> I've built out a large chunk of new build VMs all anisblised etc
16:22:49 <pbrobinson> there's a few bits I need to debug, it's on the list for tomrrow to clean the last bits of that up
16:22:57 <dgilmore> nice
16:23:27 <pbrobinson> just awaiting VLAN changes from nirik (or int infra via nirik) for new EPEL builders on the same HW
16:23:41 <nirik> yeah, I put in the ticket, but no word yet.
16:24:11 <pbrobinson> two hosts built, will build 3rd tomorrow, and associated VMs
16:24:26 <dgilmore> sounds good
16:24:34 <pbrobinson> need to work out a few ansible bits and pieces to clean up some stuff and a few koji bits
16:24:42 <sharkcz> pbrobinson: yep, good job
16:24:52 <pbrobinson> but mostly there (finally!!)
16:25:06 <dgilmore> ) then to move the hub and compose box etc
16:25:25 <pbrobinson> dgilmore: yes!
16:25:50 <pbrobinson> and kill off old shitty P6 hardware and stuff!
16:26:02 <dgilmore> and that also
16:26:14 <dgilmore> #topic Secondary Architectures update - s390
16:26:20 <dgilmore> sharkcz: how is s390
16:27:08 <nirik> I'll remove those redirects for now so signing can move forward and we can fix that properly after freeze?
16:27:09 <sharkcz> it's not that good as ppc, but I have created Beta RC1, there is one issue in the partitioning, fix exists
16:27:26 <dgilmore> okay
16:27:48 <sharkcz> and still working on the hub as you can see
16:27:56 <dgilmore> yeah
16:28:07 <dgilmore> I think we need to setup a vhost for s390pkgs
16:28:07 * sharkcz got dragged more into the enterprise world now ...
16:28:12 <nirik> whats the status of stuff in https://fedorahosted.org/fedora-infrastructure/ticket/4840 ? mostly fixed? some still left? would be good to add a summary of whats left...
16:28:17 <dgilmore> or we need to setup a second vm for it
16:28:25 <sharkcz> nirik: will look into it
16:28:44 <nirik> dgilmore: yeah, but I'd prefer to do that after freeze, not in a big hurry today.
16:28:53 <sharkcz> the thing with s390pkgs is that this name is used for the builder for a local squid cache
16:29:10 <sharkcz> used by the builders
16:29:13 <dgilmore> comment 1 it shoudl all just be releng
16:29:49 <dgilmore> sharkcz: and that should not matter in any way shape or form
16:30:17 <sharkcz> I think the main issue on the hub is that hardlinking doesn't work for the branched/rawhide mashes
16:30:32 <dgilmore> sharkcz: that is an issue on primary also
16:30:59 <dgilmore> it needs some major changes to how we do things that I am working on
16:31:02 <sharkcz> dgilmore: any solution in sight?
16:31:10 <sharkcz> ok
16:31:12 <dgilmore> sharkcz: changing how we do things
16:31:20 <dgilmore> it will happen
16:31:59 <sharkcz> in the meantime s390 would need more space on the storage for /mnt/koji (and keep just few mashes)
16:32:49 <dgilmore> sharkcz: we should only be keeping a few mashes anyway
16:33:34 <nirik> note that the netapp also does de-dupe...
16:34:09 <nirik> wow.. you are at 97%?
16:34:12 <sharkcz> dgilmore: is there already a cron job to remove the old ones? IIRC I could see such function in the build-branched/rawhide/functions.sh
16:34:21 <dgilmore> sharkcz: no
16:34:26 <dgilmore> it has to be done manually
16:34:27 <pbrobinson> nirik: yes, but you still need to allocate space to the mount, even if it's lazy allocation
16:34:29 <sharkcz> could not
16:34:43 <nirik> pbrobinson: sure.
16:35:02 <nirik> anyhow, we can add more space if needed, but of course want to try and conserve as much as we can.
16:35:58 <sharkcz> nirik: I'm running hardlinking manually when possible
16:36:23 <nirik> so, how much will you think you will need now? should I just +1TB?
16:36:44 <sharkcz> if you can add 1T, it would be ok
16:37:34 <nirik> done
16:37:59 <sharkcz> thx :-)
16:38:03 <dgilmore> anything else for s390?
16:38:10 <sharkcz> hope not
16:38:19 <dgilmore> #topic Secondary Architectures update - arm
16:38:27 <dgilmore> pbrobinson: how does arm look?
16:38:41 <pbrobinson> looking really good in terms of builds
16:38:49 <dgilmore> I know nirik made a new ansibilised db and hub box last week we need to poke at
16:39:00 <nirik> I made a new hub/db instances, but haven't yet copied a db dump over to it... I'll try and do that sometime... but if someone else has time before me, please go for it
16:39:13 <pbrobinson> I cleaned up some old ARMv7 only packages that some how kept getting tagged in
16:39:27 <dgilmore> pbrobinson: :( boo
16:39:45 <pbrobinson> well they're things like fc20 builds of mono so of no use
16:39:54 <pbrobinson> and it gives a clearer picture of where we are
16:40:17 <pbrobinson> but we've fixed up a bunch of other stuff too and it's mostly looking pretty good
16:40:50 <pbrobinson> I need to look at a kernel patch for the builder lock up issue, it's on the list for this week once the power infra is cleaned up some more
16:40:59 <dgilmore> okay
16:41:05 <dgilmore> reminds me
16:41:18 <nirik> theres some builders showing down again...
16:41:24 <dgilmore> I did not look at what we will need to do to make sure that dnf is used on secondaries
16:41:38 <pbrobinson> nirik: yep, on my list
16:41:50 <nirik> cool.
16:42:15 <pbrobinson> dgilmore: I was going to ask about that actually, wasn't sure if we got it by default with the new version of koji you pushed out
16:42:26 <pbrobinson> or quite how it was enabled
16:42:36 <dgilmore> pbrobinson: its a flag that has to be set on the tag
16:43:10 <pbrobinson> dgilmore: OK, I'll sync with you about that so I can co-ord with the various arches to get it done
16:43:13 <dgilmore> koji edit-tag -x mock.package_manager=dnf f24
16:44:05 <pbrobinson> we should be OK for aarch64 right up, on PPC I'll need to get all the new builders in, and disable the old ones once we know the new ones are good and then flip, sharkcz will know better on s390
16:44:22 * pbrobinson takes note of command
16:44:27 <dgilmore> may need to patch koji-shadow to look for it and use it when needed
16:44:43 <pbrobinson> dgilmore: AH
16:44:48 * pbrobinson goes and cries in the corner
16:45:04 <dgilmore> pbrobinson: but we should add it to the f24 tags on secondaries
16:45:39 <pbrobinson> although most of what koji-shadow does is actually custom createrepo so it might be OK
16:46:28 <pbrobinson> dgilmore: yep, I'll play with it this week, probably on aarch64
16:47:00 <sharkcz> dgilmore: patch shadow like for adding the dnf option to the SHADOWBUILD-f24* tags shadow creates on the fly?
16:47:00 <dgilmore> cool
16:47:08 <dgilmore> sharkcz: yes
16:47:14 <sharkcz> ok
16:48:11 <dgilmore> anything else?
16:48:27 <pbrobinson> not from me
16:49:04 <pbrobinson> nirik: actually, when is infra freeze?
16:50:18 <dgilmore> pbrobinson: tomorrow
16:50:27 <pbrobinson> that's what I though
16:50:43 <dgilmore> it is the same as change freeze
16:51:13 <pbrobinson> I'll aim to clean up the last of the power builder stuff tomorrow morning my time and get it out of the way
16:51:16 <nirik> yeah.
16:51:20 <dgilmore> cool
16:51:29 <dgilmore> #topic Open Floor
16:51:32 <pbrobinson> and then the rest of that can wait until post freeze
16:51:33 <nirik> we should perhaps start doing it at 00:00 UTC, but we have typically done it the next morning.
16:51:40 <maxamillion> I have one thing for open floor
16:51:49 <dgilmore> maxamillion: go for it
16:52:18 <maxamillion> I was just wanting to request that everyone try to find some time to review my proposal for the new docs workflow and provide feedback in the not too distant future https://lists.fedoraproject.org/pipermail/rel-eng/2015-October/020804.html
16:52:45 <maxamillion> mostly just because I wanted to get to working on it :)
16:52:49 <maxamillion> that's all I had, thanks
16:53:16 <dgilmore> #info maxamillion wants to update docs workflow. please provide feedback
16:53:23 <dgilmore> https://lists.fedoraproject.org/pipermail/rel-eng/2015-October/020804.html
16:53:30 <dgilmore> anything else?
16:54:03 <dgilmore> #endmeeting