16:35:33 <dgilmore> #startmeeting RELENG (2016-01-04)
16:35:33 <zodbot> Meeting started Mon Jan  4 16:35:33 2016 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:35:33 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:35:33 <zodbot> The meeting name has been set to 'releng_(2016-01-04)'
16:35:40 <dgilmore> #meetingname releng
16:35:40 <zodbot> The meeting name has been set to 'releng'
16:35:40 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson pingou maxamillion
16:35:40 <zodbot> Current chairs: bochecha dgilmore masta maxamillion nirik pbrobinson pingou sharkcz tyll
16:35:43 <dgilmore> #topic init process
16:35:57 <maxamillion> \o/
16:36:02 * pbrobinson waves
16:36:05 <maxamillion> .hello maxamillion
16:36:06 <zodbot> maxamillion: maxamillion 'Adam Miller' <maxamillion@gmail.com>
16:36:10 * sharkcz is here
16:36:12 <nirik> morning.
16:36:20 <masta> hello
16:37:00 <dgilmore> #topic Secondary Architectures updates
16:37:02 <dgilmore> #topic Secondary Architectures update - ppc
16:37:07 <dgilmore> pbrobinson: how is ppc?
16:37:43 <pbrobinson> we're a little behind, overall not bad.
16:38:01 <dgilmore> cool
16:38:11 <pbrobinson> the builders should be bought up to F-23 shortly, just awaiting firmware and one or two other bits
16:38:20 <dgilmore> :) cool
16:38:35 <dgilmore> I kinda expect all se condaries will be a bit behind after the holidays
16:38:36 <pbrobinson> possibly some other infra changes, I'm trying to align it all to do together
16:38:47 <dgilmore> okay
16:39:05 <dgilmore> #info ppc slightly behind currently
16:39:17 <dgilmore> #info updating builders to f23 asap
16:39:25 <dgilmore> anything else?
16:39:59 <pbrobinson> nope
16:41:30 <dgilmore> #topic Secondary Architectures update - s390
16:41:35 <dgilmore> sharkcz: how is s390?
16:42:11 <sharkcz> it looks good, should be up-to-date with primary
16:42:35 <dgilmore> #info s390 up to date with primary
16:42:36 <sharkcz> I'm now working on updating the builders to f23
16:42:46 <dgilmore> #infor builders being updated to f23
16:42:52 <dgilmore> #info builders being updated to f23
16:43:20 <dgilmore> sharkcz: how is the hub? is tehre anything left with teh ansible migration?
16:44:45 <sharkcz> I think there is not, after the shared "shadow" vm will be deployed, the hub function itself is ok
16:45:03 <sharkcz> but have one question - are you pruning the signed rpms from EOLed distros?
16:45:11 <sharkcz> to free some disk space
16:45:14 <dgilmore> sharkcz: yes
16:45:54 <dgilmore> we purge the signed rpms for EOL releases
16:46:09 <sharkcz> ok
16:46:16 <dgilmore> the signed headers still exist and the signed rpm can be regenerated
16:46:53 <dgilmore> has that caused some issue?
16:47:00 <pbrobinson> in theory it shouldn't make much difference if the netapp has de-dupe enabled
16:47:04 <maxamillion> sharkcz: are there ansible playbooks to deploy shadow koji instances now?
16:47:28 <sharkcz> dgilmore: nope, just thinking about space consumption
16:47:29 <pbrobinson> maxamillion: what do you mean by shadow koji instances
16:48:32 <maxamillion> pbrobinson: maybe I don't know, I've still never seen good documentation on just what the hell "shadow koji" is and how to set it up
16:48:33 <sharkcz> maxamillion: shadow is a script from koji-utils and a config and lot of know-how :-)
16:48:46 <maxamillion> yeah, nvm me
16:49:19 <dgilmore> maxamillion: shadow is the process of mirroring builds from primary to secondary koji
16:49:27 <pbrobinson> maxamillion: there is koji-shadow, which is different from the arch specific koji instances, the later can be deployed with ansible and all of them should be done by the end of the month
16:49:35 <dgilmore> maxamillion: I think you mean secondary koji
16:49:47 <sharkcz> with the shared shadow vm we will setup a git repos and some processes around it
16:50:07 <maxamillion> yeah, we don't need to get into it right now ... this is just another area where koji badly needs documentation
16:50:19 <sharkcz> config is eg. http://ppc.koji.fedoraproject.org/shadow/koji-shadow-f24.conf (the long excludelist is generated)
16:50:23 <pbrobinson> maxamillion: the VM to run koji-shadow for all of those will be ansibled once I've got to it, also hopefully in the next couple of weeks
16:50:31 <dgilmore> maxamillion: its nothing to do with koji honestly
16:50:40 <maxamillion> dgilmore: how isn't it?
16:50:42 <dgilmore> maxamillion: its a fedora specific thing
16:51:10 <dgilmore> the whold secondary and koji-shadow work
16:51:13 <dgilmore> whole
16:51:28 <dgilmore> anyway
16:51:43 <dgilmore> #topic Secondary Architectures update - arm
16:52:24 <dgilmore> pbrobinson: how is arm?
16:52:45 <nirik> We didn't get to migrating the hub/db before the break... should be ready to do that anytime we want to schedule it this week.
16:52:48 <dgilmore> I know nirik got a new arm hub setup
16:53:04 <pbrobinson> arm is pretty much up to date in terms of builds
16:53:21 <dgilmore> #info arm is up to dat builds wise
16:53:40 <pbrobinson> nirik: yea, I'm pretty much ready to go, I've got 1 or 2 small things I need to do, how's Wed for you
16:53:50 <nirik> sure, works for me.
16:53:55 <nirik> what time?
16:54:32 <pbrobinson> nirik: I'll cross check stuff tomorrow but once I've done those bits pretty much when ever is good for you
16:54:55 <dgilmore> #info arm hub to be migrated to ansible version this week
16:54:55 <pbrobinson> likely could do it tomorrow or any time after actually if it's better
16:54:56 <nirik> cool. now that fesco is fridays my wed should be pretty open
16:55:19 <nirik> so, just ping me when ready and we can get it done. ;)
16:56:15 <pbrobinson> nirik: I moved the rawhide nightly compose over to a rel-eng ansibled aarch64 box before xmas, so it should have zero special case bits and basically be like primary in that regard now
16:56:42 <nirik> very nice. ;)
16:57:39 <pbrobinson> nirik: yep, the hub will be the last bit of non ansible arm.koji infra, and hub/compose boxes the last for PPC and I plan to have them crossed off RSN
16:57:40 <dgilmore> cool. anything else?
16:57:55 <nirik> excellent.
16:58:25 <pbrobinson> none from me, just awaiting the outcome of the live-media-creator image stuff in koji outcome for aarch64 stuff from my side
16:58:53 <pbrobinson> dgilmore: not sure if you've any update on that, i'm likely a bit behind
16:59:01 <dgilmore> okay, I will cover that in a sec
16:59:21 <dgilmore> #topic rawhide changes
16:59:23 <pbrobinson> cool
16:59:42 <dgilmore> so before christmas I spent a lot of time working on changes to make rawhide be a full compose
16:59:52 <dgilmore> I have yet to test secondary arches at all
17:00:12 <dgilmore> so it turned out that livemedia-creator is not at all suitable for our needs
17:00:27 <pbrobinson> I can test that once we have it enabled and documented for primary
17:00:33 <dgilmore> we do need to move off of livecd-tools and appliance-tools
17:00:52 <dgilmore> so we are scaling back the changes for f24
17:01:11 <dgilmore> as a sign of goodwill we will move to livemedia-creator
17:01:50 <dgilmore> I am going to either write a patch for livemedia-creator or write a wrapper
17:02:02 <pbrobinson> so what changes will be postponed, what changes will happen, what's missing?
17:02:04 <dgilmore> the output from building livecds is not suitable for delivering
17:02:18 <nirik> bummer. ;( I assume livemedia-creator authors aren't interested in adjusting things much?
17:02:20 <dgilmore> pbrobinson: arm disk images are not possibel without virt
17:02:51 <dgilmore> there is a meeting lined up between releng and anaconda teams to go over requirements
17:03:12 <pbrobinson> dgilmore: from a link you provided me before, wasn't it a possibility it could be done some other way but needs some changes (dm-mapper comes to mind?)
17:03:27 <masta> huh... I've used LMC to generate arm disk images for aarch64 (without virt)
17:03:31 <pbrobinson> when is the meeting?
17:03:35 <dgilmore> https://www.redhat.com/archives/anaconda-devel-list/2015-December/msg00019.html is where I started communicating with teh anaconda team
17:03:52 <masta> the pxe live images might be another story
17:03:57 <dgilmore> pbrobinson: we are going to reach out to the lvm team also to see what we can do there
17:04:07 <dgilmore> maxamillion: no you did not
17:04:16 <dgilmore> masta: ^^^
17:04:17 <maxamillion> dgilmore: what?
17:04:19 <dgilmore> sorry maxamillion
17:04:20 <maxamillion> oh
17:04:21 <maxamillion> rgr
17:04:25 <maxamillion> :)
17:04:48 <dgilmore> masta: you did not do it in a clean environment provided my a mock chroot
17:05:18 <masta> that is correct
17:05:25 <dgilmore> masta: so you did not test it
17:05:38 <dgilmore> because it is not possible to do what we need
17:06:08 <dgilmore> anaconda blows up trying to setup the storage for a arm disk image when run in a chroot
17:06:35 <dgilmore> which we need to do to ensure we run the correct version of lorax (provider of livemedia-creator)'
17:06:55 <dgilmore> pxe-to-live is super special
17:07:10 <masta> right, that I thought was to be solved with special care for populating /dev in the chroot
17:07:11 <dgilmore> you have to actally run lmc twice
17:07:15 <maxamillion> doesn't pxe-to-live need virt?
17:07:42 <dgilmore> you need to build a disk image in one run, then run it a second time to convert it to teh live environment
17:07:58 <masta> dgilmore: that is correct
17:08:20 <masta> it needs to chain
17:08:59 <dgilmore> pxe-to-live really needs to go back to the drawing board and be redesigned to work on a sane manner and provide the output in one run
17:09:50 <dgilmore> so for f24 we will only use lmc to make live media
17:10:01 <maxamillion> does anyone in our group really understand the process to produce the pxe-to-live well enough to work with those teams to get that done? (I certainly don't)
17:10:32 <dgilmore> maxamillion: not sure we need to.
17:10:53 <dgilmore> though atomic entirely breaks our rules for how to define things
17:11:55 <pbrobinson> dgilmore: so appliance-creator will be used for arm images, and livecds will move to lmc?
17:11:57 <dgilmore> pungi going forward breaks the rules also
17:12:17 <dgilmore> pbrobinson: yes, and we will continue to not do pxe-to-live
17:12:37 <maxamillion> dgilmore: which rules are those and is this something where the rules should be revised or something needs fixing?
17:13:04 <pbrobinson> what is pxe-to-live used for? Is that atomic stuff?
17:13:04 <dgilmore> maxamillion: we have stated that all deliverables need to be defined in comps and kickstarts
17:13:14 <dgilmore> pbrobinson: yes it is atomic stuff
17:13:40 <dgilmore> maxamillion: pungi4 uses xml and atomic is mostly in json files
17:14:40 * pbrobinson awaits the next build tool using yaml .....
17:14:57 <maxamillion> dgilmore: ah
17:14:57 <dgilmore> pbrobinson: shut up, it is bound to happen
17:15:01 <dgilmore> do not give ideas :P
17:15:05 <pbrobinson> :)
17:15:06 <nirik> ha
17:15:23 * maxamillion starts writing koji 2.0 yaml support
17:15:28 <maxamillion> >.>
17:15:34 <dgilmore> so that is the general status of lmc
17:15:42 * dgilmore kicks maxamillion
17:15:56 <dgilmore> pungi I have been hgitting bug after bug
17:16:10 <dgilmore> I have been filling issues in pagure as I hit them
17:16:13 <maxamillion> pungi is "fun"
17:16:27 <pbrobinson> maxamillion: sicko!
17:16:32 <masta> it's fragile
17:16:34 <maxamillion> pbrobinson: note the quotes!
17:17:13 <dgilmore> I was hoping to switch rawhide before the break, but I was not confident enough in the output
17:17:50 <dgilmore> https://kojipkgs.fedoraproject.org/compose/rawhide/ is the list of composes we currently have
17:18:26 <dgilmore> if people could dig through and see anything that they think is wrong please do
17:18:40 <dgilmore> the Everything repo currently has no multilib
17:19:25 <dgilmore> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-24-20151218.n.3/ is the last complete compose
17:20:17 <dgilmore> we pretty much have 2 weeks to get rawhide switched
17:20:58 <dgilmore> Time is running out fast
17:20:59 <pbrobinson> dgilmore: do you have any notes, anything we should look at specifically, any particular concerns etc?
17:21:33 <dgilmore> pbrobinson: some of them like the naming there is a patch that blew up applied
17:22:17 <dgilmore> pbrobinson: I still need to ensure that we do things like split out the Cloud install tree from the images
17:23:01 <dgilmore> pbrobinson: at this point just having another set of eyes looking over to see if things look odd
17:23:17 <dgilmore> some of the oddities we are stuck with
17:23:29 <pbrobinson> OK, I'll do my best to have a look over tomorrow
17:23:34 <dgilmore> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-24-20151218.n.3/compose/Cloud/i386/iso/ like the manifest file
17:23:47 <dgilmore> not sure its a bad thing
17:23:57 <dgilmore> just not something we have had before
17:24:18 <dgilmore> it is a list of the contents of the iso
17:24:29 <nirik> what does the .n.3 denote?
17:24:39 <dgilmore> the .n is nightly
17:24:50 <dgilmore> the .3 is the 4ths run for the day
17:24:53 <dgilmore> 4th
17:25:03 <dgilmore> the first run starts with .0
17:25:12 <nirik> ah, ok. Wonder if it might be better to just be -hh-mm instead, but whatever. ;)
17:25:15 <pbrobinson> so generally it should be n.0
17:25:21 <dgilmore> generally
17:25:36 <dgilmore> unless we need to restart for a blowup
17:26:16 <dgilmore> each compose has a composeid
17:26:27 <dgilmore> https://kojipkgs.fedoraproject.org/compose/rawhide/Fedora-24-20151218.n.3/COMPOSE_ID
17:26:37 <dgilmore> Fedora-24-20151218.n.3 for that one
17:27:17 <pbrobinson> any chance we could have a newer 2016 compose done?
17:27:35 <dgilmore> pbrobinson: no. pungi is currently busted completely
17:27:45 <pbrobinson> ah, useful :-D
17:28:03 <dgilmore> I need to work with lsedlar to get his enhancements fixed and working
17:28:27 <nirik> hopefully we will get one soon, or else we might need to punt to 25. ;(
17:28:37 <dgilmore> https://pagure.io/docs/pungi/ has some docs
17:28:44 <dgilmore> they still need work
17:29:38 <dgilmore> but as new things are added docs come with them
17:29:51 <dgilmore> nirik: we have a small window :(
17:30:04 <dgilmore> any other questions?
17:30:14 <pbrobinson> not from me
17:30:15 <dgilmore> not sure I totally covered everything?
17:30:38 <pbrobinson> dgilmore: any other koji enhancement planned for f24?
17:30:53 * nirik has nothing on rawhide changes, have a few small things for open floor.
17:31:01 <dgilmore> pbrobinson: just maxamillion's osbs support
17:31:26 <maxamillion> and that's just a plugin
17:32:04 <dgilmore> okay lets move on
17:32:09 <dgilmore> #topic open floor
17:32:14 <dgilmore> nirik: you are up?
17:32:22 <nirik> just 2 quick things:
17:32:53 <nirik> 1. I see there's a gcc change proposed and they are asking for a mass rebuild, but we have no mass rebuild scheduled, so it might need schedule adjustment.
17:33:02 <nirik> (I guess thats more a fesco thing, but wanted to mention it)
17:33:14 <dgilmore> nirik: I filed a ticket with FESCo already
17:33:29 <nirik> 2. Next week smooge and I will be out at the phx2 datacenter... so if there's any hands on work people need, please ask this week so we can schedule things. ;)
17:33:33 <dgilmore> https://fedorahosted.org/fesco/ticket/1522
17:33:41 <nirik> great. :)
17:34:22 <dgilmore> nirik: sorry https://fedorahosted.org/fesco/ticket/1519
17:34:34 <nirik> ok. cool. Anyhow, thats all I had.
17:34:46 <dgilmore> the first ticket was the change
17:35:13 <dgilmore> okay, I am not sure 100% on DC work
17:37:05 <dgilmore> does anyone else have anything?
17:37:26 <dgilmore> #info if you need something done in the Data Centre next week please talk to nirik this week
17:40:32 <dgilmore> if nothing else I will close the meeting
17:41:14 <maxamillion> oh
17:41:17 <maxamillion> I kind of do
17:41:52 <maxamillion> I'm going to be updating the docker base image for Fedora 23 and Rawhide soon for https://github.com/docker-library/official-images/issues/1235 ... we don't have an official policy around this, so it's a bit ad-hoc
17:42:20 <maxamillion> I'm not sure if there's any motivation to craft an official policy around the docker image or not but I thought I'd at least mention it
17:42:54 <dgilmore> maxamillion: in my ideal world, when we make a change to any package in it, we update it and push the update
17:43:25 <dgilmore> with PDC we should be able to get the knowledge of when something in it changes
17:43:38 <maxamillion> dgilmore: I'm all for that, but we need testing ... right now I basically just fire it up and do a couple basic things, call it good enough, and submit a request for docker to pull the latest image
17:44:29 <dgilmore> maxamillion: sure, that would be the Base WG (which is basically dead) working with QA on it
17:44:37 <maxamillion> fun
17:44:41 <maxamillion> alright
17:44:58 <dgilmore> the Base WG owns the docker base image
17:45:11 <dgilmore> maxamillion: and there are two members here, masta and me
17:45:16 <maxamillion> I'll reach out to them on the mailing list to discuss that
17:45:17 <maxamillion> oh
17:45:19 <maxamillion> o/
17:45:36 <masta> =)
17:46:57 <dgilmore> lets wrap up
17:47:02 <dgilmore> #endmeeting