15:31:54 <dgilmore> #startmeeting RELENG (2014-09-29)
15:31:54 <zodbot> Meeting started Mon Sep 29 15:31:54 2014 UTC.  The chair is dgilmore. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:31:54 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:32:00 <dgilmore> #meetingname releng
15:32:00 <zodbot> The meeting name has been set to 'releng'
15:32:00 <dgilmore> #chair dgilmore nirik tyll sharkcz bochecha masta pbrobinson
15:32:00 <zodbot> Current chairs: bochecha dgilmore masta nirik pbrobinson sharkcz tyll
15:32:02 <dgilmore> #topic init process
15:32:08 <dgilmore> hola all who is here
15:32:11 * nirik is here
15:32:11 <bochecha> hi everyone
15:32:14 * sharkcz is here
15:32:16 * pbrobinson is
15:34:43 <tyll> hi
15:35:05 <dgilmore> lets get moving
15:35:17 <dgilmore> #topic #5931 [Proposal] Move new branch and unretire requests to pkgdb2
15:35:22 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5931
15:35:35 <tyll> #info nothing new from pingou
15:35:36 <dgilmore> pingu said in #fedora-releng that there was nothing new
15:35:55 <dgilmore> #topic #5967 Enable source repo generation
15:36:00 <dgilmore> I have still failed here
15:36:24 <tyll> #action dgilmore still needs to ask for more info
15:37:04 <dgilmore> #topic #4071 Block pushes to origin/ in gitolite ACLs
15:37:09 <dgilmore> https://fedorahosted.org/rel-eng/ticket/4071
15:37:14 <nirik> bochecha: has a patch here I think...
15:37:20 <dgilmore> he does
15:37:21 <bochecha> yeah
15:37:25 <dgilmore> its in the ticket
15:37:50 <bochecha> both for installing the hook in new repos, and the check to run on older repos so they also get the hook
15:38:05 <tyll> #info patches in the ticket need review
15:38:09 <dgilmore> it looks okay to me
15:38:21 <bochecha> can someone merge it then, so we test in in staging?
15:38:41 <dgilmore> sure
15:38:53 <dgilmore> #action dgilmore to apply patch
15:39:38 <bochecha> I'll note that distgit staging starts diverging from production, as production is still in puppet
15:40:11 <bochecha> do we want to move distgit production to ansible before F21 at all, or should we postpone that until F22?
15:40:38 <dgilmore> I thought we would move to gitolite3 and distgit at once?
15:40:44 <dgilmore> or is that not the plan?
15:40:48 * bochecha is just worried that we need to push a change to production, but we can't because staging is too different
15:40:58 <bochecha> dgilmore, we already moved to gitolite3 :)
15:41:00 <bochecha> in staging
15:41:04 <dgilmore> bochecha: right
15:41:08 <bochecha> staging is in ansible, on rhel7, with gitolite3
15:41:14 <nirik> right.
15:41:15 <dgilmore> so staging is already a lot different to prod
15:41:16 <nirik> lots of changes
15:41:25 <bochecha> yeah
15:41:29 <nirik> I was thinking we would do that after f21
15:41:35 <nirik> since it could be a lot of changes/bugs.
15:41:45 <dgilmore> nirik: seems like the right time to do it
15:41:54 <bochecha> and if we need to send a change to prod, then we might not be able to test it in staging?
15:41:57 <dgilmore> January sometime
15:42:01 <nirik> I don't know that this change is urgent enough to want to push to prod before then
15:42:10 <dgilmore> bochecha: we already can't test in staging
15:42:20 <nirik> bochecha: yep... but in the past our stg instacnce was not very testable anyhow.
15:42:25 <bochecha> ah, ok
15:42:58 <bochecha> well, no worries then :)
15:43:41 <dgilmore> I do not think this is urgent enough to need to go into prod straight away
15:44:18 <tyll> #info move to production after f21 with lot of changes
15:44:25 <tyll> #undo
15:44:25 <zodbot> Removing item from minutes: INFO by tyll at 15:44:18 : move to production after f21 with lot of changes
15:44:27 <tyll> #info move to production after f21 with lots of changes
15:44:32 <bochecha> dgilmore, that wasn't what I meant, I was just worried about testability of something else that might be urgent ;)
15:44:50 <nirik> we can always spin up another puppet based stg instance if needed.
15:44:57 <nirik> would be a pain, but can do it. :)
15:44:59 <bochecha> but if we didn't really test in staging in the past, then the current situation doesn't make things worse :)
15:45:06 <dgilmore> bochecha: sure but we can not test today as its way too different already
15:45:33 <dgilmore> bochecha: right, just some firefighting if things go bad :(
15:45:52 <bochecha> dgilmore, right, I was speaking more generally, not this change in specifics (I probably shouldn't have conflated the two in the same #topic, then)
15:45:53 <dgilmore> or setup a puppet based stg if we think its really needed
15:46:01 <dgilmore> bochecha: indeed
15:46:09 <dgilmore> okay lets move on
15:46:15 <dgilmore> #topic #5878 noarch packages corrupted on import on ppc.koji.fp.o
15:46:21 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5878
15:46:27 <dgilmore> so this has nothing to do with us
15:46:31 <dgilmore> its a bug in koji
15:46:46 <dgilmore> koji-shadow needs to verify the rpms it imports
15:47:18 <nirik> move to koji-shadow bug?
15:47:22 <nirik> (is that even packaged?)
15:47:28 <dgilmore> nirik: its part of koji
15:47:29 <tyll> #info bug in koji
15:47:38 <sharkcz> it is, part of koji-utils
15:47:49 <nirik> k
15:49:44 <pbrobinson> feel free to add me as a cc to that bug pls
15:50:20 <dgilmore> #topic #5959 Enable keep-alive connections for koji (primary and secondaries)
15:50:27 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5959
15:50:39 <dgilmore> I thought we agreed to enable everywhere lastw eek
15:50:50 <dgilmore> seems we just forgot to update the ticket
15:50:56 <dgilmore> and make sure its enabled everywhere
15:51:22 <tyll> #info updating ticket was forgotten, it should be enabled everywhere
15:51:32 <dgilmore> #topic #5963 Orphaned vulnerable packages in EPEL
15:51:35 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5963
15:51:46 <tyll> there is still tooling missing
15:52:09 <tyll> unless there is a volunteer now, I guess we can skip it
15:52:27 <dgilmore> #action volunteer needed for toolingh
15:52:31 <dgilmore> #undo
15:52:31 <zodbot> Removing item from minutes: ACTION by dgilmore at 15:52:27 : volunteer needed for toolingh
15:52:34 <dgilmore> #action volunteer needed for tooling
15:52:52 <dgilmore> #topic #6000 automate creation of excludelist for koji-shadow
15:52:58 <dgilmore> https://fedorahosted.org/rel-eng/ticket/6000
15:53:24 <dgilmore> I added the lists being built nightly. I need to debug why its not quite working right
15:53:53 <sharkcz> where should the output land?
15:54:09 <dgilmore> nightly branched and rawhide log files
15:54:15 <sharkcz> ok
15:54:19 <dgilmore> http://kojipkgs.fedoraproject.org/mash/branched-20140929/logs/excludearch-arm.log
15:54:21 <tyll> #info scripts were adjusted, but need debugging
15:54:22 <dgilmore> for instance
15:54:39 <dgilmore> its only doing the first letter and not all of them
15:55:49 <dgilmore> sharkcz: http://kojipkgs.fedoraproject.org/mash/branched-20140929/logs/excludearch-s390.log
15:55:54 <dgilmore> is the s390 one
15:56:10 <sharkcz> I think I see it - --path /mnt/koji/mash/branched-$DATE/$BRANCHED$EXPANDARCH/source/SRPMS/*/
15:56:24 <dgilmore> they are in the format of excludearch-<koji instance>.log
15:56:32 <sharkcz> it takes only the first dir here
15:56:46 <dgilmore> sharkcz: we have to use a wildcard
15:57:37 <dgilmore> ./srpm-exlcuded-arch.py -a "ppc64,ppc64le" --path /mnt/koji/mash/rawhide/source/SRPMS/\*/
15:57:45 <sharkcz> but it gets expanded when ./srpm-exlcuded-arch.py is called
15:57:52 <dgilmore> last time i manually ran it i had to escape the *
15:57:52 <sharkcz> not inside it
15:58:04 <dgilmore> easily fixed
15:58:15 <sharkcz> yep
15:58:34 <dgilmore> #action dgilmore to fix up the calls
15:58:59 <dgilmore> sharkcz: so from tomorrow we should have them created nightly
15:59:58 <sharkcz> perfect :-)
16:01:11 <dgilmore> #topic #5886 need method for distributing urgent fixes... urgently
16:01:17 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5886
16:01:30 <dgilmore> so we had a need for this last week again with bash
16:01:44 <dgilmore> twice
16:03:00 <dgilmore> I do not think that we need to have a testing repo for these updates
16:03:39 <tyll> I was wondering, will we create one for Rawhide as well? Then it can be used for testing
16:03:42 <dgilmore> in the examples we have had people grabbing from koji and testing has been sufficient
16:03:54 <dgilmore> we could have a rawhide one
16:04:11 <sharkcz> the person who adds the builds to the repo can call for testing before they are added
16:04:13 <dgilmore> though there would never be any rawhide bodhi integration
16:04:40 <tyll> nevertheless, it is a lot easier for testers to test if everything is just in a testing repo
16:04:58 <tyll> it also makes sure that everyone is testing the same set of updates and not accidently some other koji build
16:05:45 <dgilmore> I also want to not have any of this rely on fedmsg
16:05:59 <dgilmore> just because fedmsg has no reliabiliy guarantee
16:06:14 <nirik> so, this would start for f21? or a fedora-release update for older releases to enable it/
16:06:33 <dgilmore> nirik: we could do both
16:06:41 <dgilmore> f21 ships by default with it
16:06:42 <nirik> perhaps we look at doing it for rawhide/f21 now and once it's all setup do the older
16:06:45 <tyll> #info do not rely on fedmsg
16:06:49 <dgilmore> but we backport to f20
16:07:34 <tyll> fedmsg is not really necessary
16:07:40 <nirik> and we need to write up exactly how it works/policy
16:07:47 <nirik> a wiki page perhaps
16:08:04 <dgilmore> it definitely needs documenting
16:08:20 <dgilmore> likely we should come up witha  policy and take it to FESCo for ratification
16:08:22 <nirik> ie, update MUST be a security update, MUST be submitted to bodhi as normal?, file releng ticket, etc
16:08:26 <nirik> yep
16:08:41 <dgilmore> anly security updates
16:08:50 <dgilmore> they have to be of a critical priority
16:09:00 <dgilmore> i would say remotely exploitable only
16:09:17 <nirik> if we just require a real update too we can have that go as normal and only clean out the crit one once the real update is stable
16:09:18 <dgilmore> we will want a releng ticket to track it
16:09:20 <nirik> (for a bit)
16:09:34 <dgilmore> nirik: yeah, clean up after a week
16:09:42 <sharkcz> should the security team be involved in defining whether this is the real critical fix?
16:09:47 <dgilmore> it will give heaps of time for it to go stable via normal channels
16:09:59 <tyll> btw. it might be more difficult than it is outlined in the ticket  because of multilib, so I guess that createrepo is not enough anymore
16:10:06 <dgilmore> sharkcz: we could require the request comes from SRT
16:10:11 <nirik> tyll: good point.
16:10:19 * nirik wishes we could just drop multilib. ;(
16:10:28 <bochecha> probably also have a criteria that the update can not depend on other updated packages, only the one being fixed?
16:10:37 <dgilmore> we only have multilib on s390 and x86_64 now
16:10:59 <dgilmore> bochecha: no, if it requires another package we will have to push both
16:11:33 <tyll> Bodhi updates shoul already contain all dependent builds in one update
16:11:42 <dgilmore> bochecha: say heartbleed fix needed a fix in another library it was using we would want both
16:11:45 <dgilmore> and to get it out
16:11:56 <bochecha> dgilmore, right, if both need to be fixed, sure
16:12:06 <dgilmore> but having it contained to a single update is okay
16:12:21 <nirik> I can whip up a wiki page.
16:12:34 <dgilmore> bochecha: if the other packages fixes are optional that is a different problem
16:12:34 <nirik> and we can draft on it for a week and see what we can come up with?
16:12:42 <tyll> #info multilib needs to be taken care of
16:12:44 <bochecha> what I meant is if we build a bash update against a version of a lib newer than what's in the repository, then we'd have to push both bash and that lib, even though the lib isn't necessary for the critical security fix
16:12:47 <dgilmore> nirik: sure that would be awesome
16:12:53 <tyll> #action nirik create a wiki page
16:13:04 <dgilmore> we will need to make an asjustment to mm I think
16:13:06 <bochecha> and in that case, I would think we'd want to only push bash, so we'd need a build that doesn't require that new lib, only stuff currently in the repos
16:13:18 <dgilmore> fedora-repos change is simple
16:13:54 <tyll> #info mirrormanager needs changes
16:14:28 <dgilmore> bochecha: would be a case by case thing I think
16:15:52 <dgilmore> lets get the wiki page up, comment and adjust and discuss in next weeks meeting before taking to FESCo for ratification
16:16:24 <dgilmore> #topic https://fedorahosted.org/rel-eng/ticket/5775
16:16:27 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5775
16:16:38 <nirik> https://fedoraproject.org/wiki/Urgent_updates_policy is the draft I put up. Please edit/add.
16:16:50 <dgilmore> afaik this is what oddshocks's upload service does
16:18:20 <dgilmore> there is really nothing here for us to discuss
16:18:21 <nirik> so, close? asign to oddshocks ? :)
16:18:50 <dgilmore> well i think we should leav it open. we need to properly get the upload service deployed
16:19:00 <dgilmore> we need to make sure we do it
16:19:06 <dgilmore> I guess assign to oddshocks
16:19:07 <tyll> #info this is what oddshocks's upload service does
16:20:19 <dgilmore> #topic #5819 Need tags deleted in python-heatclient repo
16:20:22 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5819
16:20:28 <dgilmore> this needs closed as wontfix
16:20:32 <dgilmore> or invalid
16:20:37 <dgilmore> its not something we do
16:21:28 <tyll> why do we not delete tags?
16:21:46 <dgilmore> we do not delete anything from git
16:22:05 <sharkcz> or what is the problem when the tags are there, there is no harm AFAICT
16:22:16 <dgilmore> there is no harm in the tags
16:22:23 <bochecha> there is also no harmin deleting them
16:22:31 <dgilmore> I think you can actaully doa  build from a tag
16:22:45 <kalev> I'm pretty sure this is possible, yes
16:22:50 <dgilmore> so we would need to make sure that a build wasnt done
16:23:12 <bochecha> oh, like building from git://pkgs.fedoraproject.org/module#TAG instead of #HASH ?
16:23:13 <dgilmore> it doesnt cause any harm to have the tags
16:23:19 <dgilmore> bochecha: yeah,
16:23:28 <kalev> I think this should be a self-service where packagers can delete their own tags and stuff, but have a git hook that rejects the push if a build was done out of that tag / whatever
16:23:36 <dgilmore> koji clones teh repo can checks out what was passed
16:23:38 <bochecha> fedpkg alays uses the commit hash, so I didn't know it was doable to build from a tag
16:23:41 <tyll> as long as all tagged commites are still referenced by a branch, nothing is lost
16:23:42 <dgilmore> so the hash or tag
16:23:49 <bochecha> if that's the case, then yeah, deleting them is much harder
16:24:10 <dgilmore> we use the hash in fedpkg as it is immutable
16:24:10 <bochecha> tyll, if the build sourceurl contains the tag but not the commit hash, how do you know which commit was referenced by the tag?
16:24:29 <dgilmore> bochecha: you don't which is why we use the hash
16:24:49 <bochecha> dgilmore, I know, I was replying to tyll's « nothing is lost »
16:25:30 <tyll> bochecha: does koji allow to build from a tag?
16:25:39 <bochecha> tyll, that's what we were just saying :)
16:25:48 <dgilmore> tyll: yes
16:25:50 <bochecha> tyll, I learned it a few sentences above, just like you :)
16:26:09 <kalev> I'm pretty sure koji allows building out of everything, can even say tell it to build HEAD and it just goes and builds it
16:26:11 <tyll> it scrolled to fast here, I have only a small screen
16:26:15 <tyll> too fast*
16:27:29 <dgilmore> I see more reasons to keep the tags than remove them
16:27:49 <sharkcz> +1
16:27:59 <bochecha> yeah, if we can build from tags, then we shouldn't remove them
16:28:04 <tyll> nevertheless, this means we also should make sure in koji that it does not allow to build from HEAD
16:28:11 <dgilmore> tyll: we should
16:28:25 <nirik> ideally make it only build from hash. ;)
16:28:32 <dgilmore> I know some people regularly do scratch builds from HEAD
16:28:56 <tyll> #info koji should be changed to create non-scratch builds only from full commit hashes
16:29:02 <dgilmore> the plugin we need to check the branch is valid should probably also check that its a hash and not a tag or HEAD
16:29:08 <kalev> ideally it should build from a hash that's reachable from one of the fxx branches and verify that the build target matches with the fxx branch in git
16:29:19 <dgilmore> kalev: correct
16:29:23 <dgilmore> :)
16:29:26 <kalev> :)
16:29:31 <tyll> kalev: this is an open ticket in trac :-)
16:29:49 <dgilmore> its a open ticket, it will need updating with this extra info
16:30:30 <tyll> https://fedorahosted.org/rel-eng/ticket/5843
16:30:41 <tyll> https://fedorahosted.org/koji/ticket/281
16:31:15 <dgilmore> tyll: the last two tickets are rawhide signing and moving the blocking service. do we have any updates there? or should we quickly cover secondary arches?
16:31:32 <tyll> #info tags cannot be removed, because they might have been used to build something
16:31:40 <tyll> there is an update to the blocking service
16:31:48 <dgilmore> okay
16:32:04 <dgilmore> anything else here?
16:32:27 <dgilmore> #action dgilmore to close the ticket as wontfix with and explanation as to why
16:32:28 <tyll> not from me
16:32:35 <dgilmore> #topic #5914 Move fedmsg based blocking service to Fedora Infrastructure
16:32:41 <dgilmore> https://fedorahosted.org/rel-eng/ticket/5914
16:32:53 <dgilmore> the floor is yours tyll
16:33:01 <tyll> So I noticed that koji and pkgdb is a lot faster to use from autosign than frommy local system
16:33:35 <tyll> e.g. checking whether all currently retired packages are blocked takes only a couple of seconds for all relevant branches
16:34:07 <tyll> therefore I guess using fedmsg does not make sense unless we need to block packages fast after they are retired
16:34:19 <nirik> cool.
16:34:20 <tyll> but afaik blocking them within one day is usually enough
16:34:43 <dgilmore> tyll: :) well ideally they get blocked the day they are retired
16:34:49 <tyll> therefore it could just be a cron job (which we needeed anyhow)
16:34:58 * nirik nods. cron job sounds good.
16:35:03 <dgilmore> but perhaps we can run the blocking script at the start of buildbranched and buildrawhide
16:35:27 <dgilmore> we would still need to deal with epel
16:35:36 <nirik> could do that. Would also mean perhaps we could add that info to the report.
16:35:45 <dgilmore> nirik: yeah
16:35:45 <nirik> (well, I guess it already is in repodiff)
16:35:49 <tyll> the script can just run always for all branches
16:36:02 * nirik doesn't care much
16:36:09 <tyll> if there is nothing to do, it is done really fast
16:36:15 <dgilmore> tyll: can we put the script in the releng git repo
16:36:23 <dgilmore> then we could have buildrawhide run it
16:36:41 <tyll> yes, I am planning to do this
16:36:44 <dgilmore> and buildbranched to catch last minute straglers
16:37:00 <tyll> but we also need to deploy a cert for it on the affected systems
16:37:07 <dgilmore> we will need to make sure the masher user can blokc packages
16:37:13 <dgilmore> tyll: therer already is one
16:37:15 <dgilmore> there
16:37:31 <dgilmore> tyll: as koji needs to work to create the lives and cloud images
16:37:50 <tyll> ok, is it still releng04 the script would run on?
16:38:48 <dgilmore> tyll: no
16:39:15 <dgilmore> there is two compose boxes, one for rawhide and one for branched
16:39:31 <nirik> compose-{rawhide|branched}.phx2.fedoraproject.oorg
16:39:33 <nirik> org even
16:39:59 <tyll> I see
16:40:14 * nirik goes to get more coffee. I have an item for open floor tho.
16:40:32 * kalev has an item too.
16:40:33 * sharkcz has one open floor too
16:40:36 <sharkcz> :-)
16:40:42 <dgilmore> nirik: no problem I have some generaly secondary arch stuff to cover before going into arch specifics
16:40:52 * bochecha also has an item for open floor
16:40:53 <tyll> So how/who deploys it? I am not sure whether I have sudo permissions there
16:41:25 <dgilmore> #action till to get teh script in to the releng git repo and wire up buildbranched and buildrawhide to run it
16:41:54 <dgilmore> #action dgilmore to make sure the masher user can block the packages and that till wires thinsg up correctly
16:42:07 <dgilmore> tyll: it just needs added to teh releng git repo
16:42:15 <dgilmore> we need to tag the repo
16:42:22 <dgilmore> then it will get run automatically
16:42:23 <sharkcz> one idea about buildbranched/buildrawhide - what about making them like wrappers calling individual functions form other scripts? they start to get complicated
16:42:28 <dgilmore> no need fot sudo
16:42:29 <dgilmore> for
16:42:34 <sharkcz> s/form/from/
16:42:41 <tyll> ok
16:42:54 <dgilmore> sharkcz: good task fro rewriting to a unified set of scripts
16:43:26 <dgilmore> sharkcz: I do wan to make the nightly composes be full ocmposes
16:43:32 <dgilmore> composes
16:43:52 <dgilmore> so the scipts for nightly and TC/RC composes would be the same
16:44:01 <tyll> It would be nice to have a releng python module that contains all the common code, I usually need the same methods in several scripts
16:44:24 <dgilmore> tyll: there is a lot of things we could do better
16:44:33 <dgilmore> it is something we could look at doing
16:44:53 <nirik> python-fedorareleng ! :)
16:45:10 <kalev> python3-fedorareleng ! :)
16:45:26 <dgilmore> kalev: python3 is not even started yet
16:45:35 <dgilmore> it is something we need to look at
16:45:36 <tyll> kalev: for this we need python3 koji and yum or dnf
16:46:20 <dgilmore> anything else here?
16:46:36 <tyll> not from me
16:46:43 <dgilmore> #topic Secondary Architectures updates
16:46:56 <dgilmore> so as a general secondary arch update
16:47:24 <dgilmore> I am going to be transitioning secondary arch leadership over to pbrobinson
16:47:45 <nirik> cool.
16:47:51 <dgilmore> he is going to lead making sure that secondary arches follow policy and all use the same procedures
16:48:27 <dgilmore> I still have a long term goal of unifying secondary arches and primary and making the whole secondary arch process transparent
16:48:38 <dgilmore> and so I wont be not doing secondary arch work
16:48:55 <dgilmore> I won't be trying to do it all
16:49:20 <nirik> sounds good. ;)
16:49:31 <sharkcz> ack :-)
16:49:42 <nirik> I'd also over time like to pull the secondary arch stuff into infrastructure so it's all setup the same way, gets updates the same way, etc, etc,
16:50:01 <dgilmore> nirik: yep, fas groups for shell access
16:50:09 <dgilmore> and sudo etc controleed through ansible
16:50:48 <dgilmore> pulling secondary arch infra into primary infra is somewhere we should go
16:51:02 <dgilmore> I know that the s390 builders will be hard to do
16:51:35 <pbrobinson> nirik: yes, I want to do that too so I'm extremely happy to work with you on that :)
16:51:51 <sharkcz> but s390 should be able to share at least something
16:51:54 <nirik> cool. :)
16:52:17 <pbrobinson> nirik dgilmore: ultimately I don't see any reason from an infra PoV  to be any different from mainline
16:52:18 <dgilmore> they should be able to run fas and pull from ansible git
16:52:24 <nirik> yeah.
16:52:25 <sharkcz> and some time we might move to KVM :-)
16:52:34 <dgilmore> the bits in ansible-private will be hard
16:52:51 <pbrobinson> sharkcz: I can work with you on that.... we'll leave it until the end ;-)
16:53:09 <dgilmore> sharkcz: that owuld be lovely
16:53:45 <dgilmore> anything else generic secondary arch related?
16:54:02 <dgilmore> #topic Secondary Architectures update - ppc
16:54:13 <pbrobinson> the 2nd TC went out earlier
16:54:13 <dgilmore> pbrobinson: sharkcz: how are things?
16:54:41 <sharkcz> and we are testing them, and it looks good
16:54:41 <dgilmore> excellent
16:54:42 <pbrobinson> I'm awaiting reports on that. Need to look at the signing side of things so we're ready to move to RC -> Alpha
16:55:48 <dgilmore> #info ppc Alpha TC2 out and being tested, hopefully RC soon
16:56:00 <dgilmore> #topic Secondary Architectures update - s390
16:56:07 <dgilmore> sharkcz: how goes s390?
16:56:39 <sharkcz> I'm going to do first compose this week, builds looks good, eclipse will need a rebootstrap as it wasn't built for a long time
16:57:50 <dgilmore> :( okay
16:58:35 <sharkcz> not sure if I have access to f21 key, so I'll check and let you know
16:58:49 <dgilmore> sharkcz: pretty sure you do, if not we will fix it
17:00:03 <dgilmore> sharkcz: you have key access
17:00:08 <dgilmore> just double checked
17:00:13 <sharkcz> ok, thx for info
17:00:28 <dgilmore> #topic Secondary Architectures update - arm
17:00:41 <dgilmore> pbrobinson: how goes the arm world?
17:02:42 <dgilmore> guess we lost pbrobinson
17:02:50 * pbrobinson is herre
17:02:56 <dgilmore> I know he was working to get a TC done
17:03:13 <pbrobinson> so I'm working on the TC, didn't do much over the weekend, working on it now
17:03:17 <pbrobinson> hoping to get something out today
17:03:30 <pbrobinson> builds are going great with very few missing
17:03:42 <pbrobinson> over all aarch64 is looking pretty decent
17:05:01 <pbrobinson> although some changes for the compose host WRT ACLs would make things much easier. eg arm.koji isn't resolvable and not accessible on the public IP so I have to hack around using internal addresses. Is nirik the best person to assist with that?
17:05:49 <nirik> hum, I thought it did have a public ip...
17:06:03 <nirik> can work with after meeting to sort that out.
17:06:15 <dgilmore> pbrobinson: yeah, ansible uses /etc/host entries to make things work
17:06:53 <pbrobinson> OK, I can't telnet to the public http(s) IP but can the 10.x addr.
17:07:19 <pbrobinson> similar I can't mount the /mnt/koji nfs mount
17:07:55 <pbrobinson> but I suspect the later needs a storage change request of some type to update the ACL because I can see the nfs server
17:08:23 <nirik> where are you trying to do those things? some other compose machine?
17:08:32 <dgilmore> pbrobinson: nirik can change the exports file on the netapp
17:08:41 <nirik> yep. :)
17:08:45 <pbrobinson> nirik: let's take if offlinee
17:08:45 <dgilmore> nirik: one of the aarch64 boxes in phx2
17:08:52 <nirik> ah ha. ok.
17:08:59 <pbrobinson> or post meeting
17:09:03 <nirik> yeah, can update... see me in #fedora-releng post meeting. ;)
17:09:11 <nirik> or anytime
17:09:24 <pbrobinson> cool
17:10:34 <dgilmore> pbrobinson: anything else?
17:11:36 <dgilmore> #topic Open Floor
17:11:39 <dgilmore> guessing no
17:11:44 <dgilmore> nirik: you had something
17:11:44 <nirik> I had one item
17:12:02 <nirik> https://lists.fedoraproject.org/pipermail/releng-cron/2014-September/002672.html and https://lists.fedoraproject.org/pipermail/releng-cron/2014-September/002673.html
17:12:06 <nirik> can someone fix these cron jobs?
17:12:14 <nirik> they are using dwa's old cert that is expired.
17:12:28 <nirik> I don't know if they are doing something important, but they should be fixed or removed.
17:12:30 <dgilmore> masta set those up to go to the list
17:12:43 <nirik> they are going to the list.
17:12:44 <nirik> they are just busted.
17:12:46 <dgilmore> we should evaluate what they are as they are not part of teh standard processes
17:12:50 <pbrobinson> looks like they're using dwa's cert
17:13:10 <nirik> yep. They need fixing.
17:13:11 <pbrobinson> I can look at them and work with people to learn the proper process and migrate them to it
17:13:16 <dgilmore> the sync-tagged-primary is run from a central location
17:13:24 <tyll> #info cron jobs fail because of dwa's expired koji certificate
17:13:28 <dgilmore> the ppc specific one should not be needed
17:13:40 <dgilmore> same with sync-override
17:13:48 <dgilmore> both of those cronjobs should be removed
17:13:52 <pbrobinson> dgilmore: I'll review the difference between them and kill them as appropriate
17:13:55 <tyll> #action pbrobinson takes care to fix them
17:14:04 <nirik> great. :)
17:14:10 * nirik doesn't like looking at tracebacks.
17:15:01 * dgilmore notes secondary arches shouldnt go inventing processes just for themselves
17:15:15 <nirik> anyhow, that was it. :)
17:15:24 <tyll> I want to propose to remove the epel component from trac as we do not really use it (afaik it was used earlier when there was a distinction between Fedora and EPEL releng)
17:15:28 <pbrobinson> dgilmore: it's historical.... it won't be like that moving forward!
17:15:32 <dgilmore> if you think you need to do something do it with all secondary arches in mind and get it part of the standard processes
17:15:40 <dgilmore> pbrobinson: right :)
17:15:46 <dgilmore> just pointing it out
17:15:52 <pbrobinson> dgilmore: you can get off your high horse now ;-)
17:16:03 <bochecha> I have to go very soon, so I'll throw it out there, I sent two mash patches to the rel-eng list today, https://lists.fedoraproject.org/pipermail/rel-eng/2014-September/018424.html and https://lists.fedoraproject.org/pipermail/rel-eng/2014-September/018425.html
17:16:08 <dgilmore> tyll: ive updated the epel component to go to teh fedora-releng list
17:16:11 <nirik> tyll: ok with me now that there is an epel trac... but are there any tickets open in it?
17:16:22 <bochecha> second one is something Remi asked for, the first one is just something I found while testing for it
17:16:31 <dgilmore> the epel trac is not for releng tasks
17:16:42 <dgilmore> its for epel management
17:16:47 <tyll> the epel task can just go into the koji component
17:16:54 <tyll> the relen epel tasks
17:16:57 <tyll> releng*
17:17:09 <dgilmore> epel was setup like it is in releng trac because JEsse wanted nothing to do with epel
17:17:57 <dgilmore> I do plan to shutdown the epel-releng list
17:18:09 <tyll> this sounds also good
17:18:45 <sharkcz> mine item - there are 20+ builds found by check-latest-build.py in f21 where the latest NVR isn't the highest NVR, tyll already fixed one because a ticket was opened, I'll create a new ticket for the rest
17:19:25 <dgilmore> sharkcz: thats due to bodhi bugs
17:19:48 <sharkcz> not all, some are "intentional" in ghc* IIRC
17:20:15 <dgilmore> sharkcz: not sure what you are talking about
17:20:41 <tyll> so can we remove the trac componend? Will someone do it? I cannot do it afaik.
17:20:48 <tyll> trac epel component
17:20:54 <dgilmore> packages must never go backwards nvr wise
17:21:14 <sharkcz> some of the problematic builds are in ghc* packages, and they are intentional becuase they reverted some commits after mass rebuild or something like that, you will see from the full list
17:21:30 <dgilmore> sharkcz: so if ghc packages went out at a higher nvr at some point and they taged older builds in they violated policy and need to go about fixing it the right way
17:21:56 <sharkcz> I just recall some discussion on the devel list (not sure)
17:22:02 <dgilmore> sharkcz: that sounds like they did the wrong thing and get to deal with it the right way
17:22:04 <sharkcz> I understand your point
17:23:05 <sharkcz> or can someone run the check-latest-build.py script on a releng host? it would be faster than on mine wkst
17:23:42 <dgilmore> sure
17:23:45 <bochecha> I really have to go, so I'll take feedback on my mash patches on the list :)
17:23:47 <sharkcz> as I said I'll open a ticket, so it's not forgotten
17:23:49 <dgilmore> we need to clean things up
17:23:58 <nirik> bochecha: thanks
17:24:51 <kalev> I had an item too, can I go?
17:25:18 <dgilmore> kalev: sure
17:25:28 <kalev> ok, some quick background first
17:25:37 <kalev> at flock, we had a meeting where dgilmore and pbrobinson and hughsie and I discussed how to move on with appstream data generation in koji
17:25:47 <kalev> and came up with an action plan with a bunch of items
17:25:59 <kalev> one thing we wanted to do was to move things around so that instead of http://alt.fedoraproject.org/pub/alt/screenshots/f21/failed.html , we'd get the warnings / errors at package build time in koji instead
17:26:14 <kalev> to move forward with this, I've been looking into how to make desktop file and appdata file validation automatic during koji builds
17:26:26 <pbrobinson> kalev: yes, you were going to add the functionality to the desktop files package thing
17:26:54 <kalev> what I wanted to discuss here today was how to do it -- the most straightforward way would be to add desktop-file-utils and the appdata validation tools to the minimal koji buildroot and run validation from redhat-rpm-config for everything that gets built in koji
17:27:06 <kalev> bochecha did some quick tests to see how much new deps these two would drag in to the minimal builroot
17:27:11 <dgilmore> kalev: thats not how to do it
17:27:15 <kalev> bochecha: do you still have the fpaste links from earlier at hand or did you leave already?
17:27:39 <dgilmore> kalev: people already have to add a BuildRequires to validate the .desktop file
17:27:39 <pbrobinson> kalev: does it need to be run for 16k source packages as opposed to just desktop apps?
17:28:12 <dgilmore> kalev: there is zero reason for adding the extra packages to every buildroot
17:28:34 <pbrobinson> or running on every build as opposed to just desktop apps
17:28:43 <kalev> ok, that's a good enough answer, that's exactly what I wanted to know: if it would make sense to add it to the minimal buildroot
17:29:14 <kalev> well, it wouldn't _run_ on every build if it sees that there are no files, but the additional deps in every buildroot could be a problem, agreed
17:29:38 <pbrobinson> kalev: but if you're running it in desktop-file-utils it would have the deps as part of that package anyway
17:29:51 <kalev> yes
17:29:52 <dgilmore> http://fedoraproject.org/wiki/Packaging:Guidelines#Desktop_files
17:30:13 <dgilmore> the plan was to have desktop-file-validate do the appdata validation
17:30:36 <dgilmore> packages that ship it should already be doing the validation
17:31:00 <kalev> mhm, but a bunch have probably forgotten to add the validation and it might be nice to make this automatic
17:31:20 <pbrobinson> and not have it fail the build for F-21 and only warn but for F-22+ to fail the build
17:31:30 <kalev> yep, definitely
17:31:33 <dgilmore> kalev: having a way to check broken packages is very useful
17:32:02 <dgilmore> kalev: but every time we add something to the minimal buildroot we extend teh build time
17:32:17 <pbrobinson> kalev: if you have a list of packages that have app data I can grep the entire source tree check out and see if any don't have desktop-file-utils as a dep
17:32:59 <kalev> pbrobinson: that should actually be even easier, should be able to just get the filelists from the metadata and cross-check that
17:33:17 <kalev> anyway, I have my answer now, thanks -- I won't pursue adding it to the minimal buildroot any more
17:34:31 <kalev> I guess I'll hook it up so that redhat-rpm-config runs the checks when the appropriate package(s) are installed
17:34:57 <kalev> and then make sure to cross-check as pbrobinson suggested to make sure everything has the needed build deps to actually run this
17:35:09 <kalev> thanks, nothing more from me
17:35:12 <nirik> yeah, there's a guideline requiring it.
17:35:27 <nirik> so would be good to fix any broken packages
17:35:28 <sharkcz> kalev: you probably should also go via packaging committee with such change
17:35:59 <dgilmore> kalev: such a change will require packaging guideline changes
17:36:23 <nirik> https://fedoraproject.org/wiki/Packaging:Guidelines#Desktop_files
17:36:34 <nirik> there already is a guideline
17:36:34 <dgilmore> kalev: that's part of why we said you should do it the way we said
17:36:52 * sharkcz needs to leave now
17:37:42 <kalev> right, thanks, I'll make sure the guidelines get updated as well
17:37:47 <dgilmore> desktop-file-install or desktop-file-validate have to be run according to the guidelines
17:37:59 <dgilmore> we should make both validate the  appdata
17:38:17 <dgilmore> they are supposed to already be run
17:38:25 <kalev> I want to make this automatic so that BR: desktop-file-utils takes care of the validation
17:38:28 <dgilmore> there should be no need for guideline changes
17:38:42 <kalev> copy-pasting validation lines to each and every spec is just crazy, and a lot of packages forget those out
17:38:45 <dgilmore> kalev: I do not think that is the right things to do
17:38:51 <kalev> so the guidelines are just going to get simpler
17:39:44 <dgilmore> kalev: i think things would get overly complex in redhat-rpm-config trying to do it automatically
17:40:10 <kalev> nah, it's not hard at all, just a few additional lines
17:40:11 <dgilmore> quite often desktop-file-install is used to change or set some variables
17:40:21 <kalev> I've already posted a patch and got tentative buy in from the redhat-rpm-config maintainers
17:40:47 <dgilmore> kalev: where is the patch?
17:41:00 <nirik> devel list I think it was?
17:41:14 <kalev> dgilmore: https://lists.fedoraproject.org/pipermail/devel/2014-September/202711.html
17:42:49 <dgilmore> kalev: okay, that's not dealing with installing the .desktop file at all
17:43:03 <dgilmore> you made it sound like you were trying to automate that
17:43:05 <kalev> right, just validation
17:43:12 <dgilmore> that was not clear
17:43:18 <kalev> oh now, I just want to make sure that every desktop file that is installed also gets validated automatically
17:43:23 <kalev> oh no *
17:44:02 <dgilmore> the patch will need some revision
17:44:13 <dgilmore> but automatically validating is fine
17:44:13 <kalev> yes, or more like complete rewrite :)
17:44:51 <kalev> cool, glad that you agree :)
17:45:11 <kalev> next rpm version should also make a lot of other stuff easier, like killing most of the copy-pasted scriptlets
17:45:14 <mbooth> Hi all, I saw "eclipse" mentioned here! For getting eclipse built on the platform where it doesn't build, the instructions here should apply: https://bugzilla.redhat.com/show_bug.cgi?id=1141225
17:45:14 <dgilmore> I would rather not add the extra deps to every buildroot
17:45:42 <dgilmore> mbooth: it was mentioned for s390 sharkcz is on it afaik
17:46:23 <dgilmore> anything else for openflor?
17:46:25 <dgilmore> floor
17:46:26 <mbooth> dgilmore: Okay great. I'd be happy to help if there are deeper problems with it :-)
17:47:02 <dgilmore> if not ill wrap up we are over an hour over
17:48:20 <dgilmore> #endmeeting