16:40:51 <dgilmore> #startmeeting
16:40:51 <zodbot> Meeting started Mon Feb 10 16:40:51 2014 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:40:51 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
16:41:08 <dgilmore> #meetingname releng
16:41:08 <zodbot> The meeting name has been set to 'releng'
16:41:19 <dgilmore> #topic rollcall
16:41:28 * masta is here
16:41:31 <janeznemanic> hi
16:41:31 <dwa> here
16:41:49 <masta> dgilmore: plz chair me
16:42:14 <dgilmore> #chair masta
16:42:14 <zodbot> Current chairs: dgilmore masta
16:42:42 * nirik is sort of around
16:42:49 <masta> #chair dwa nirik
16:42:49 <zodbot> Current chairs: dgilmore dwa masta nirik
16:42:57 * sharkcz is here
16:43:02 * tyll_ is here, too
16:43:23 <masta> #chair tyll_ sharkcz
16:43:23 <zodbot> Current chairs: dgilmore dwa masta nirik sharkcz tyll_
16:43:52 <masta> Monday monday monday!
16:44:04 <tyll_> sharkcz: Are you missing on https://fedoraproject.org/wiki/ReleaseEngineering#Composition intentionally?
16:44:28 <dwa> masta: do not taunt happy fun ball
16:45:15 <masta> dwa: =)
16:45:22 <Viking-Ice> why does fesco have to approve members?
16:45:44 <masta> Viking-Ice: huh?
16:46:03 <tyll_> Viking-Ice: FESCo has delegated this power to the Release Team itself.
16:46:11 <Viking-Ice> "Release Team members are approved by FESCo"
16:46:14 <sharkcz> tyll_: I represent secondary arches here, which means I'm not direct member of the releng team
16:46:18 <tyll_> Viking-Ice: it's explained in the next sentence
16:46:27 <masta> FESCo has delegated this power to the Release Team itself.
16:46:29 <Viking-Ice> perhaps update the wiki page ;)
16:46:37 <sharkcz> but I can add myself there :-)
16:47:14 <tyll_> sharkcz: I was just wondering, I just pinged everyone from the wiki list in the releng channel and missed you :-)
16:47:19 <dgilmore> okay off teh phone
16:47:27 <masta> Viking-Ice: the one does kinda negate the other, but there is a context so it makes sense to leave it there...
16:48:21 <masta> perhaps someday our delegated powers would be revoked...
16:48:43 <dgilmore> sharkcz: we probably should add you
16:48:51 <dgilmore> or add yourself ;)
16:48:54 <masta> +1
16:48:59 <sharkcz> dgilmore: done ;-)
16:49:01 <Viking-Ice> then the wiki page would be updated it's just a bit confusing fesco handles this and fesco says releng handles this
16:49:28 <dgilmore> #topic tickets
16:49:45 <dgilmore> https://fedorahosted.org/rel-eng/report/10
16:49:47 <masta> Viking-Ice: I'll AI myself to update the wiki, thanks for pointing out the confusion.
16:49:59 <dgilmore> currently no tickets
16:51:23 <masta> no tickets sounds good... inbox zero, etc..
16:51:26 <tyll_> I believe last meeting we discussed to try out XZ compression - are there some news? - https://fedorahosted.org/rel-eng/ticket/5362
16:51:48 <masta> tyll_: oh I'm glad you brought that up
16:52:14 <dgilmore> it should have had the meeting keyword added
16:52:42 <dgilmore> tyll_: we will need to patch mash and pungi to do it
16:52:45 <masta> So it was found out that of the two options a) update the default in create repo or b) configure mash to set the option to create repo.. that option B is the wya to go because the builders are on rhel6
16:53:05 <dgilmore> createrepo has said they wont update the default
16:53:18 <dgilmore> so we have to patch all the tools to change it
16:53:37 <dgilmore> masta: builders are not rhel6
16:54:11 <masta> dgilmore: oh, my bad... but I believe there was a rhel6 dimension to this, perhaps in regards to epel then?
16:54:20 <dgilmore> so someone will need to write patches for pungi and mash
16:54:38 <dgilmore> the boxes that compose updates are rhel6 today
16:54:55 <dgilmore> the boxes where we compose rawhide is rhel6
16:55:17 <dgilmore> but the builders are f20 except for the kernel builders
16:55:28 <tyll_> dgilmore: do we need to get the patches into the Fedora Rawhide packages?
16:55:28 <dgilmore> builders != compose boxes
16:55:34 <dgilmore> tyll_: yes
16:55:46 <dgilmore> tyll_: pungi and mash are run in rawhide chroots
16:55:54 <dgilmore> tyll_: I am upstream for both packages
16:56:06 <nirik> buildhw and buildppc are still rhel6 also. ;)
16:57:40 <tyll_> dgilmore: which is the system where the packages need to be updated?
16:57:56 <tyll_> dgilmore: i.e. the rawhide compose box(?)
16:58:08 <dgilmore> tyll_: nowhere
16:58:11 <nirik> tyll_: it's done in a chroot
16:58:17 <nirik> so they need to be updated in rawhide
16:58:19 <dgilmore> tyll_: its all in a rawhide chroot
16:58:50 <dgilmore> we will need to update the updates compose boxes before f21 branches so that updates would have xz compression
16:59:05 <dgilmore> it will need to be a config option to mash
16:59:36 <dgilmore> like we have with dirctory hashing
17:00:13 <dgilmore> as updates are mashed on rhel6
17:00:25 <dgilmore> though maybe by then it will be rhel7
17:00:25 <tyll_> Is there no mash git repo? Or is it no fedorahosted project?
17:00:40 <dgilmore> tyll_: there is a mash repo, it doesnt have a gtrac
17:01:07 <nirik> http://pkgs.fedoraproject.org/cgit/mash.git/
17:01:07 <dgilmore> https://git.fedorahosted.org/cgit/mash
17:01:37 <tyll_> Ah, I see
17:02:33 * nirik has an item for open floor when we get to it.
17:02:33 <dgilmore> tyll_: patches go to the buildsys list
17:02:41 <dgilmore> nirik: :) okay
17:02:44 <tyll_> So both mash and pungi need a new option to support xz compression?
17:02:51 <dgilmore> tyll_: yep
17:03:06 <dgilmore> tyll_: and both need a different solution
17:03:20 <dgilmore> mash execs createrepo and pungi uses its python api
17:03:40 <dgilmore> any other tickets?
17:04:05 <dgilmore> #topic secondary arch status (ppc)
17:04:14 <dgilmore> dwa: where is ppc at?
17:04:20 <dwa> dgilmore: I'll add the meeting keyword to it shortly, but have you looked at the f18/-updates tags on secondaries? I think they're still empty.
17:04:42 <dgilmore> dwa: ill look at them again
17:05:30 <dwa> for general ppc status, we caught up on rawhide from not having a working llvm but then rsyslog broke :) we have a BZ filed
17:05:49 <dwa> we're also going to be receiving KVM and little endian-capable ppc64 systems shortly
17:06:39 <dgilmore> cool
17:06:52 <dgilmore> so then we will have a ppc64le bootstrap
17:06:54 <dgilmore> ?
17:06:57 <dwa> yup
17:07:25 <dwa> our systems though are pretty much the only ones that will exist outside of IBM right now though, so there won't be much opportunity for community involvement at first :/
17:07:46 <dgilmore> :( okay
17:08:01 <dwa> (I'm sure other OS vendors also have them, but they're not exactly jumping in to help with Fedora anyways :) )
17:08:25 <dgilmore> kinda weird we will be on the third bootstrap in about as many years after none for 10 or so
17:09:00 <dgilmore> anything else you want to bring up ppc wise?
17:09:29 <dgilmore> #topic secondary arch status (s390)
17:09:30 <dwa> I've also asked the ppc team to look at the finished Fedora.next PRDs to see if any of them particularly speak to them aside from server
17:09:43 <dgilmore> #undo
17:09:43 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x2b0f7510>
17:09:48 <dwa> but we haven't decided yet if we're going to try to follow more than one product or what yet
17:10:14 <dgilmore> okay, there is going to be a major ovrhaul in compose tooling
17:10:20 <dwa> yeah
17:10:48 <dgilmore> #topic secondary arch status (s390)
17:10:48 <dwa> also if we (Fedora Releng) want to set policies for what secondaries can do with products (or not) we should probably do that soon before any of the secondaries get deep into it
17:10:59 <dgilmore> gahh
17:11:01 <dgilmore> #undo
17:11:01 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x2ad02bd0>
17:11:07 <dgilmore> im getting ahead of myself
17:11:13 <dwa> I can take a hint ;)
17:11:19 <dgilmore> dwa: will bring that up in open floor
17:11:26 * dwa is done now :)
17:11:29 <dwa> (with ppc)
17:11:32 <dgilmore> #topic secondary arch status (s390)
17:11:38 <dgilmore> sharkcz: your turn
17:11:40 <sharkcz> ok, now:-) not much happening, f19 and f20 updates are being build regularly, rawhide blocked by few relatively small packages (perl-net-sslay,...)
17:11:54 <dgilmore> okay
17:12:03 <dgilmore> and you recently updated the builders right
17:12:10 <sharkcz> yep
17:13:00 <sharkcz> it's all much faster, higher count and more memory
17:13:10 <dgilmore> nie
17:13:11 <dgilmore> nice
17:13:31 <dgilmore> anything you want to bring up s390 related?
17:14:17 <sharkcz> one point - when designing the new releng tools and processes, keep in mind that no all koji builders will have /mnt/koji mounted
17:14:48 <dgilmore> sharkcz: right. that does need to be in the design
17:15:15 <dwa> if we're making the new releng tools be koji-based, maybe have a 'compose' channel similar to the createrepo channel?
17:15:36 <dgilmore> dwa: they will all be koji based
17:15:39 * sharkcz would like to know whether the composes still must be run natively
17:15:54 <dgilmore> and likely will use channels to send to desiganted hardware
17:15:57 <sharkcz> but probably yes, due the scripts
17:16:03 <sharkcz> scriptlets
17:16:26 <dgilmore> sharkcz: running dracut etc in the compose chroot will need it
17:16:58 <dgilmore> okay if nothing else lets move to open floor
17:17:16 <dgilmore> #topic open floor
17:17:19 <dgilmore> nirik: you have something?
17:17:35 <nirik> oh yeah, actually 2 small things that might be something new folks would want to look into...
17:18:02 <nirik> a) When trying to debug an issue with dhclient being broken on rawhide boot.iso I noticed that the pungi output has a bunch of small junk we should clean up...
17:18:18 <nirik> like things that don't exist anymore
17:18:30 <dgilmore> thats all in lorax
17:18:50 <dgilmore> well I think it is
17:18:57 <nirik> let me get an example, it might be.
17:19:00 <dgilmore> groups to install and packages to install etc
17:19:35 <nirik> installpkg firstaidkit-plugin-passwd failed: No package(s) available to install
17:19:35 <nirik> installpkg firstaidkit-plugin-key-recovery failed: No package(s) available to install
17:19:35 <nirik> installpkg firstaidkit-plugin-mdadm-conf failed: No package(s) available to install
17:19:52 <nirik> installpkg *-fedup-dracut failed: No package(s) available to install
17:20:19 <sharkcz> it's lorax afaik
17:20:22 <dgilmore> thats all lorax
17:20:23 <nirik> ok.
17:20:29 <nirik> so, file lorax bugs on it?
17:20:33 <sharkcz> the templates need a cleanup
17:20:50 <dgilmore> patch should be small
17:20:50 <sharkcz> or send patches
17:20:50 <nirik> turned out that was the cause of dhclient breakage too. ;)
17:20:55 <nirik> (lorax removing a lib it needed)
17:21:07 <nirik> ok, the second item then..
17:21:14 <dgilmore> lorax tries to be clever
17:21:55 <sharkcz> or trying to save space, but blindly
17:22:07 <dgilmore> yeah
17:22:08 <nirik> b) at the end of one of the last cycles (might have been f18?) we had livecd composes where we didn't change much and the size jumped around a bunch. We talked about adding a step to run 'fstrim' in there. Perhaps someone could look into doing that? it would clear the unused space and hopefully make the image sizes more consistent.
17:22:48 <dgilmore> thats likely needed to be done in livecd-creator
17:22:52 <nirik> yep.
17:23:07 <dgilmore> but it wouldnt hurt
17:23:07 <nirik> just might be somethig for someone who wants to dig into live creation. ;)
17:23:29 <dgilmore> would be nice to look at a new tool for live creation
17:23:51 <nirik> well, there's livemedia-creator...
17:23:54 * nirik runs
17:24:31 <dgilmore> there is
17:24:41 <dgilmore> if it can be made to run sanely then sure
17:25:13 <nirik> well, we could revist that... I don't have any idea if it's changed.
17:25:18 <nirik> but now would be a great time.
17:25:19 <dgilmore> its not
17:25:29 <dgilmore> it ould need to be able to run in a chroot
17:25:46 <dgilmore> and I don't know what that would take
17:25:57 <nirik> well, we can ask. ;)
17:25:58 <dgilmore> but now is a good time to look
17:26:18 <nirik> thats all I had off hand, just wanted to toss out some good stuff for new folks to dig into. ;)
17:26:22 <dgilmore> I did speak briefly with dave cantrell and he said they want to do it
17:26:55 <dgilmore> okay thanks nirik
17:27:12 <dwa> #link https://fedoraproject.org/wiki/ReleaseEngineering/GettingStarted I started a skeleton of a 'getting started' page for Releng. If people think this is a good idea we should try to flesh out the projects section and start marking tickets as easyfix (or some other descriptor - I just stole it from infra :) )
17:27:37 <dwa> (speaking of 'stuff for new folks to dig into')
17:27:38 <dgilmore> dwa: so secondary arch product requirements.
17:27:46 <dgilmore> okay that first
17:28:24 <dgilmore> I will try do a bit of work on that page this week
17:28:27 <tyll> dwa: using easyfix is probable the best marker to match http://fedoraproject.org/easyfix/
17:28:43 <dgilmore> yeah. making some easyfix tickets will be best
17:29:07 <dwa> tyll: oh neat, didn't know about that page at all
17:29:41 <tyll> dwa: there is always one project in Fedora one does not know :-)
17:30:21 <dgilmore> theres lots of projects that many dont know about
17:30:39 <masta> ok so going back to the repo data compression stuff... what does the bzip2 compression of the sqlite database files ?
17:30:51 <dgilmore> masta: createrepo
17:30:59 <dgilmore> it supports xz
17:31:04 <tyll> masta: some algorithm is hardcoded in createrepo
17:31:11 <dgilmore> but its an option and they wont change the default
17:31:24 <tyll> masta: for some files, not sure if it is the sqlite files or the other files
17:31:46 <masta> ok, I was imagining that was a seperate step, because I was baffled that two diff compression would be selected
17:32:04 <dgilmore> not a separate step
17:32:14 <masta> thanks for answering...
17:32:28 <dgilmore> gz was used for the xml and bz2 for teh sqlite they were added at different times
17:32:42 <dgilmore> initially it was xml only
17:32:47 <nirik> createrepo --compress-type=xz
17:33:00 <dgilmore> it was to do with not breaking existing clients
17:33:07 <tyll> the primary xml file is always gz
17:33:37 <dgilmore> shouldnt have to be always
17:33:37 * masta would like to XZ all the things
17:33:44 <dgilmore> anyay
17:33:48 <dgilmore> anyway
17:33:55 <nirik> before we throw the switch we should also ping dnf maintainers... to make sure there's nothing there that will breat
17:33:56 <nirik> break
17:34:11 <dgilmore> lets move on to secondary arches and products
17:34:16 <dgilmore> nirik: yep
17:34:17 <tyll> line 444: http://createrepo.baseurl.org/gitweb?p=createrepo.git;a=blob;f=createrepo/__init__.py;h=85f2a3d666aaaff24736d78e4a78bb5d12146efd;hb=HEAD
17:34:19 <tyll> masta: ^
17:34:32 <dwa> so for ppc at least, we haven't decided if we want to follow a single product or multiple or what
17:35:13 <nirik> dwa: or you could just not do products... keep the same status... but server should work fine on ppc/s390 I would think?
17:35:46 <dwa> BUT if primary releng wants to have policies, e.g. "A secondary arch MUST follow [at least one / no more than one] product", or "if secondaries want to say they're creating Server they can't modify it" or whatnot we should do that sooner than later
17:35:53 <dgilmore> dwa: I think its something that for the most part should be worked out with FESCo, the working groups and secondary arches
17:36:09 <dgilmore> I guess the working group could request you support a product
17:36:35 <dwa> I think right now we're operating under the assumption that whatever we want to do is up to us, as long as we follow the usual source/build restrictions that are in place now
17:36:49 <dgilmore> dwa: i think from a releng perspective we dont care whats produced.
17:36:52 <dwa> and we could either continue as we are now, or choose to follow a product, or do our own thing.
17:37:04 <dgilmore> its kinda been up to the secondaries
17:37:11 <dwa> ('own thing' as in 'We want these pieces of product X but not those pieces')
17:37:20 <dgilmore> we have not said that you have to make lives of appliances for instance
17:37:29 <dwa> nod, just want to throw it out there that we're starting to think about products
17:37:50 <dgilmore> I think secondaries should deliver things based on the products
17:37:52 <masta> dwa: I think the secondaries should pick at least one of the products, at least base, and the others be optional... but this is probably a FESCo kinda thing
17:37:53 <dwa> so if primary cares about how secondaries realize them, something should be said sooner rather than later.
17:38:00 <dgilmore> but they dont have to do all the products
17:38:15 <dgilmore> dwa: i think thats something to ask FESCo
17:38:54 <dgilmore> my thinking is they need to use the same tooling and be produced in the same way
17:38:59 <dwa> dgilmore: ok. I think I'll wait for fedora for power to decide what we want to do then.. if it's something more than "We'll follow Server [except for the stuff that doesn't build on ppc]", I'll bring it to fesco.
17:39:16 <dgilmore> dwa: might be worth bring up
17:39:23 <dgilmore> they may have differenet ideas
17:40:00 <masta> we could raise the issue, put on the next FESCo docket for their meeting.
17:40:08 <dwa> 'k, I'll write something up
17:40:10 <dgilmore> likely should
17:40:11 <sharkcz> dwa: please put me on cc for the fesco ticket
17:40:18 <dgilmore> dwa: and me too
17:40:24 <masta> and me too
17:40:42 <masta> I've been wonder about this for aarch64 also
17:40:42 <dwa> right, I'll just add releng as a whole to the CC :)
17:40:53 <dgilmore> anyonw have anything else to bring up?
17:41:10 <dwa> masta: is fedora/aarch64 going to be a secondary arch? I guess I assumed once it was bootstrapped it'd go right to primary.
17:41:11 <nirik> should we start asking for aarch64 updates?
17:41:16 <nirik> jinx. ;)
17:41:17 * dgilmore needs to learn to type
17:41:35 <dgilmore> dwa: it will be secondary for some period of time
17:41:47 <dwa> gotcha
17:42:04 <tyll> What is the docs tag setup for which tickets were file recently? I did not find anything about it in the SOPs.
17:42:05 <dgilmore> when there is hardware available and its moving along to some point it will need to ask for promotion
17:42:08 <masta> dwa: I believe aarch64 will be secondary for a period of time, and will not be primary until it is promoted by FESCo (if ever)
17:42:31 <dwa> dgilmore: hardware other than iPhone 5Ss, right? ;)
17:42:38 <dgilmore> dwa: right
17:42:48 <dgilmore> tyll: the docs team wants to build their things in koji
17:43:02 <dgilmore> tyll: there as a long standing ticket to setup koji for it
17:43:19 <dgilmore> they have a policy allowing them to build into that target using srpms
17:43:47 <dgilmore> that was the packages they needed to be able to make docs
17:44:01 <dgilmore> I need to write up a SOP on it
17:44:08 * nirik has all the history on that if anyone wants to know the full story and workflow
17:44:38 <tyll> nirik: If you have notes, maybe you can add them to the wiki?
17:45:07 <dgilmore> there will be tickets from time to time
17:45:12 <dgilmore> adding new packages
17:45:25 <dgilmore> i guess at some point they will want el7-docs
17:45:42 <nirik> tyll: let me look if there's an infra ticket... but it's not a short note. ;)
17:46:02 <dgilmore> nirik: i think there is
17:46:18 <nirik> https://fedorahosted.org/fedora-infrastructure/ticket/3860
17:46:22 <dgilmore> tyll: they wanted a second instance of koji setup to use for making docs
17:46:32 <dgilmore> I said lets just do it in the koji we have
17:46:36 <nirik> anyhow. Basically it's changing their workflow to be better.
17:47:17 <dgilmore> tyll: does that make sense?
17:47:25 <dgilmore> we do need to document it better
17:47:30 <tyll> Are they building RPMS with documentation in it?
17:48:21 <dgilmore> tyll: https://fedorahosted.org/rel-eng/ticket/5214
17:48:24 <tyll> dgilmore: I guess I understand it enough now to handle incoming tickets since you provided command lines when you handled them
17:48:25 * nirik tries to do a short answer...
17:48:40 <dgilmore> tyll: they are for installation on the app servers to deliver via the web
17:49:12 <dgilmore> instead of a weird workflow they had
17:49:20 <nirik> current workflow is that they make changes to docs git repos, then the person making the changes runs publican and checks that output into git. then we git pull it on our proxy servers. This makes the repo gigantic and requires everyone to run publican and causes locking problems if multiple people do, etc.
17:50:02 <nirik> they instead want: check into git, make a src.rpm, build that in koji to produce output rpms, then we install them on a single instance and sync the files to proxies.
17:50:24 <nirik> so, no more output in git, we have packages we can upgrade/downgrade, etc.
17:50:59 <tyll> Thank you, I guess I understand it now
17:51:09 <dgilmore> :)
17:51:14 <dgilmore> I have one thing
17:51:19 <dgilmore> http://ausil.fedorapeople.org/DevConf-2014-Agile-releng.odp
17:51:21 <nirik> it took me a lot of discussion with them to figure out what they wanted. ;)
17:51:32 <dgilmore> thats the slides from my talk on Sunday at devconf
17:51:51 <dgilmore> Id like everyone to look at them and provide feedbck ask questions
17:51:51 <tyll> dgilmore: ah, yes, I looked at them - I guess tickets need to be filed for the current tasks
17:52:03 <dgilmore> tyll: they do
17:52:36 <dgilmore> ove the next two weeks while im in Europe I plan to see what help I can get from folks here
17:52:54 <dgilmore> and start working on an overview for new tooling etc
17:52:58 <masta> dgilmore: want me to handle the updates push this week while you're traveling?
17:53:24 <dgilmore> masta: if you want to sure, If not I can ill be in Brno this week and next
17:53:25 <dwa> oh yeah, can we add updates push schedule to a calendar somewhere or a webpage or something?
17:53:40 <dgilmore> yeah need to make a releng calendar in fedocla
17:53:44 <dgilmore> fedocal
17:54:02 <dgilmore> if no one has anything else lets wrap up im hungry and dinner calls
17:54:04 <masta> I have asked the fedocal developer to add a 4 week scheduling option to fedocal, to support our current rotation
17:54:13 <dgilmore> masta: okay
17:54:15 <masta> we now how our own calendar too
17:54:22 * dwa seconds dgilmore
17:54:27 <nirik> I made a calendar
17:54:42 <nirik> sysadmin-releng/releng should be admins of it.
17:54:43 <dgilmore> nirik: awesome thanks
17:54:55 <masta> right now we cannot schedule reoccuring reminds for each person because they are only two week max cycles, until my request enhancement is finished
17:54:58 <dgilmore> okay lets wrap up
17:55:05 <dgilmore> #endmeeting