18:00:13 #startmeeting EPEL 18:00:13 Meeting started Fri Sep 4 18:00:13 2015 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:13 Useful Commands: #action #agreed #halp #info #idea #link #topic. 18:00:14 #meetingname EPEL 18:00:14 The meeting name has been set to 'epel' 18:00:14 #chair smooge avij bstinson dgilmore nirik Evolution 18:00:14 #topic Init process / agenda gathering 18:00:14 Current chairs: Evolution avij bstinson dgilmore nirik smooge 18:00:31 hello 18:00:36 hi all 18:01:35 morning everyone 18:01:57 good 9pm everyone 18:01:57 There's folks interested in talking other arches... any other topics for today aside from that? 18:02:14 hello 18:02:55 dgilmore: you around? 18:03:00 I have an item 18:03:10 RHEL-7.2 beta (and then final) 18:03:26 hi 18:04:01 sure. 18:05:08 nirik, want me to run the meeting since I am here now? 18:05:17 if you like. ;) 18:05:24 I think dgilmore is out getting a cheque or something so won't be around 18:05:45 Evolution, you here for this meeting? 18:05:49 I am 18:05:52 for once. 18:06:01 ok so we have 4 out of 5 so have super quorum 18:06:16 penultimate quorum? 18:06:25 anyway first topic since it is short 18:06:39 #topic RHEL-6.7 and RHEL-7.2 beta 18:07:12 RHEL-6.7 was released a while ago.. a weeding of packages that were moved in and out of the distro probably needs to be done. 18:07:43 to throw a monkey wrench in that slightly.... 18:07:49 sure, just needs someone to go over the list and file bugs/ticket. 18:07:56 Evolution, ? 18:07:59 do we care about server vs workstation package differences? 18:08:05 I seem to recall a bug or two about that. 18:08:24 yeah there is a problem in that we build against server but there are packages in workstation 18:08:33 yes. we don't use/care about workstation. ;) 18:08:53 I filed a number of bugs for the packages to be retired; http://fpaste.org/263713/ 18:09:33 avij: cool. ;) perhaps we should use a tracking bug also and then we can go force retire if maintainers don't respond 18:10:21 just curious: if something is dropped from the distribution, would EPEL pick it up again? 18:10:34 bryanpkc, if someone wants to take it 18:10:42 I'm not sure if those cover all the packages to be retired, but those were the ones that I filed 18:11:05 bryanpkc, we don't pick up packages.. we just give people room to support those packages using our hardware :). [Or something like that.] 18:11:09 they are usually good about listing new ones in release notes. 18:11:44 Evolution, does that cover what you wanted to monkey with? 18:12:03 in any case, I think I checked that there's a bug for each package to be retired, either filed by me or someone else 18:12:11 yup. 18:12:26 I've not looked recently, but in the past when we looked at workstation vs server there were things like them shipping different versions of the same packages, which would make our lives more crazy 18:12:28 smooge, thanks for the explanation 18:12:31 #info RHEL-6.7 was released a while ago.. a weeding of packages is commencing thanks to avij 18:12:45 #info RHEL-7.2 beta was released this week. 18:12:58 * nirik grumbles about the pidgin situation too. 18:13:13 There were broken deps due to ImageMagick soname bump in 6.7, not sure if that's all be sorted 18:13:24 RHEL-7.2 is a HUGE rebase of a lot of the graphics stack and some other items. I have noticed we have some packages which are moved in and some packages which are moved out (and 1-2 packages we have already in) 18:13:29 orionp: I rebuilt all the ones I could see and submitted updates on them. 18:13:41 have those updates been pushed past testing? 18:13:43 nirik - great! 18:13:54 Evolution, if anyone tested them 18:13:55 Evolution: don't think so yet, need karma or more time 18:14:15 smooge: of course we can't really act on it since it's not final yet, but good preview of fun ahead. 18:14:43 smooge: perhaps a ticket of anticipated breakage/changes? 18:15:18 https://admin.fedoraproject.org/updates/drawtiming-0.7.1-2.el6 https://admin.fedoraproject.org/updates/psiconv-0.9.8-6.el6 https://admin.fedoraproject.org/updates/perl-Image-SubImageFind-0.03-2.el6 https://admin.fedoraproject.org/updates/spectrum-1.4.8-6.el6 18:15:33 I am trying to get an idea of all the src.rpms which have changed majorly so we can see. Since some of this is in workstation and some of this is in server and they may not overlap 18:15:42 yes, thanks nirik, looks like all the packages depending on ImageMagick have now been taken care of 18:16:16 if all goes well maybe next meeting? 18:16:25 sounds like a plan. 18:17:01 anything else anyone wants to talk about RHEL releases? [Maybe RHEL-5 EOL sometime in 2029? :)] 18:17:37 2017 18:17:38 in *theory* we should have a 32bit build for 7 done soon. 18:18:06 orionp, sure sure.. I will believe it when I see it. [It is still 35+% of EPEL load :)] 18:18:18 we need to spin and test install media, and that's it. 18:18:38 Evolution, cool. At that point we can look at what is possible... 18:18:49 ok next topic then 18:19:00 #topic Alternative Archictectures 18:19:12 which would be i386 and s390 18:19:35 and ppc64le and mips :) 18:19:45 * Evolution shoots mips in the face 18:19:46 and armv7 and aarch64 18:20:09 anyhow, we talked about i386/i686 at flock. 18:20:16 aarch64 is gold and released already, but dgilmore wanted to begin with x86 18:20:16 Evolution, if we do mips32 we could be on every router :) 18:20:30 dgilmore said he thought we could get things to work building that against centos and still building x86_64 against rhel... 18:20:49 I didn't think koji would let us, but if he says it could work I trust him. ;) 18:20:59 aarch64 is not 2ndary arch? 18:21:25 or am I misunderstanding evolution 18:21:50 bryanpkc, I believe it is gold in CentOS 18:21:59 oh i see 18:22:04 bryanpkc: different terminology. 18:22:10 alternate vs secondary 18:22:25 some folks said 'secondary' made it feel like they mattered less. 18:22:48 I am not using secondary on purpose because our build system has conotations for secondary which seemed to get confusion occuring 18:22:51 there's talk of changing what that means in fedora as well.. (although nothing has been officially proposed) 18:23:10 smooge: yeah. I don't care one way or the other, just pointing it out. 18:23:25 secondary usually has a different koji, different rules for releases and such 18:23:33 anyhow, I think 32bit intel is the thing we would/should try first and move on from there. 18:23:36 x86 is built but not assembled. aarch64 is out the door with a shiny gold star. 18:23:41 In that case, EPEL is not set up to deal with "secondary" in that form. 18:23:56 the logistics of other arches could be difficult, since we may not have builders for them... we would have to figure out what we want to do then. 18:24:14 we have several options s390 builders... but they'll be remote 18:24:28 re x86: sounds like a job for a set of targets in koji.stg :) 18:25:20 bryanpkc: yeah. We have some remote ones that fedora s390 secondary arch uses, could use those, there was talk about some at syracuse, and some other sites... 18:25:28 bstinson: yeah 18:26:45 Syracuse is not officially GA until November but we can start getting you some VMs now 18:27:18 if physical access is a must... someone has to got pay for the box :-) 18:28:31 I think I'm late to the party 18:28:50 we were just talking about s390 builders 18:28:50 hey Brian_TS 18:29:05 Sorry about my lateness....got pulled into a security meeting. 18:29:15 bryanpkc: we should likely talk to the s390 fedora secondary arch maintainer... I don't know much about those builders. 18:30:30 Anyhoo....I've got access to s390x/zvm hardware and was wanting to compile RPMs but I understand that I'm an outsider (not in a bad way) and there's a trust relationship that would be hard to overcome because the fedora project doesn't have access to Z architecture to ensure that nastiness doensn't get injected into the RPMs. I completely understand. 18:30:38 perhaps before we do anything official folks who do have access to the hw might do a rebuild and put it up somewhere so we can see issues? 18:31:10 Brian_TS: there's that, also there's a matter of control... if we depend on someone elses infra we are down if they are down, etc. 18:31:22 nirik: agreed. 18:31:26 hi Brian_TS 18:31:44 nirik, we have done rebuilds for a bunch of EPEL SRPMs 18:31:46 which may paritially be mitigated with control over vms (or whatever s390x calls them) :) 18:31:52 cool. 18:32:14 last time i checked, 75% of SRPMs could be built out of the box 18:32:32 we just used a stupid shell script that iterated over the SRPMs 18:32:33 The builder issue is my major worry mainly because in the past that was the bigger complaining "We can't build F-nn because EPEL is down (whether true or not)" 18:32:42 bryanpkc: I've had higher numbers than that in my own non-scientific testing. 18:32:57 Brian_TS: EPEL6 or EPEL7 18:32:58 ? 18:33:08 bryanpkc: both...but mostly EPEL6 at this point. 18:33:29 bryanpkc: However, I've only built a small subset of things so far. Perhaps 200 RPMs from EPEL. 18:33:40 oh 18:33:53 Brian_TS: we were trying 4400 SRPMs 18:33:57 * nirik runs to get more coffee, back in a few. 18:34:06 Brian_TS: all of EPEL7 18:34:23 bryanpkc: The only stuff I've had issue with is things that require dmidecode and other intel-ish stuff. 18:35:12 however 50% of what I compile ends up being noarch RPMs. 18:35:24 I think we had issues with ghc-related stuff, some python-related stuff 18:35:41 ghc is often a problem. ;) 18:35:50 I didn't look too deeply at the issues. I have zero knowledge about ghc 18:36:16 i have a list somewhere... need to find where it is 18:36:30 I have next to zero knowledge about rpmbuilding. You guys do a great job of making the specs quite portable. 18:36:43 anyway, in IBM we could build the packages and try to report on the issues 18:36:50 but our hands are tied around publishing binaries 18:37:23 and i'm not certain that our method (a shell script that iterates on the SRPMs) is correct 18:38:05 i use mock to do the builds, but i didn't set up koji... not sure if that's ok 18:38:43 mock should cover most of the issues 18:39:03 bryanpkc, you invented copr-0.9 18:39:41 smooge, lol 18:40:54 so for me, if we can get some sort of way to build the RPMS via the fedora s390 2ndary builder without breaking Fedora then I don't see why not. However those are a couple of big ifs 18:42:01 until those get answered... we are in a hold pattern 18:42:04 why would EPEL build failures break Fedora? 18:42:16 it is more about builders not being available 18:42:29 well, you mean other arch epel builds? 18:43:23 nirik, yeah. I know in the far past when EPEL ppc builders went down, koji would sometimes stop processing Fedora builds 18:43:39 until someone went in and did some magic 18:43:42 that must have been back when ppc was a fedora primary arch or something. 18:44:37 build failures could cause builders to go down? 18:44:42 I can't remember.. I thought it was after ppc was primary and the only builders were for EPEL. We have had such good uptimes in the last 4 years that I don't know if it still matters 18:45:19 what if we add net new s390 builders for EPEL 18:45:25 anyhow, I think next steps here are to work on i686 first... and also talk with fedora s390 secondary arch maintainer(s) to see how they handle builders and if epel could do something similar. 18:45:27 and stay away from the fedora 2ndary builders 18:45:56 bryanpkc: i think the issue is more with remote builders that go down for $REASONS (unrelated to the builds themselves) 18:46:10 right. 18:47:39 new builders should still help right? 18:47:41 The way it was explained to me by a koji developer several FUDcons ago was koji etc are built on the assumption that the builders are local. that it works when they aren't.. is not a planned feature :) 18:47:54 if these new builders go down, they would affect EPEL but not Fedora 18:48:13 bryanpkc: sure... but blocking epel builds is still anoying and bad. 18:48:43 nirik, agreed... and that's the other concern... how can we experiment with a new arch without blocking other arches 18:49:18 if koji had the ability to not care about failures on some arches that would be handy for that. 18:49:40 but I don't think it does... perhaps thats on the roadmap for 2.0 18:49:54 bryanpkc: if you are already building bits of EPEL internally, and are prepared to report bugs/patches (even if you don't post any results) that would be helpful 18:50:10 failing that there's setting up a new koji with just that arch and building the builds as they come on primary 18:50:48 perhaps something could be setup to listen on fedmsg for epel builds and fire those off on s390 18:50:55 bstinson: yeah, we can do that. would i report issues on bugzilla.redhat.com? or somewhere else? 18:50:58 mailing list? 18:51:45 bugzilla should be good. 18:52:58 nirik, setting up a new koji sounds like a solution 18:53:19 it could host all the new arches as issues are ironed out 18:54:41 yeah, not sure how easy it would be to shadow the main one... it would be different than the fedora secondary arch ones... 18:55:34 coming up on the hour soonish. are we good to keep going on this, or does anyone have anything else burning to address? 18:55:56 I don't have anything more to address on this. 18:56:31 nirik: you will talk to the fedora s390 secondary arch maintainer(s)? 18:56:37 I can do so yeah. 18:57:07 ok i'll take a look at our build results from a month or so ago (haven't done re-builds recently) 18:57:15 and start combing through the failues 18:59:01 #action nirik will contact s390 arch maintainer about EPEL 18:59:07 or is that item... 18:59:24 anyway.. going to close out the meeting in case another group has it in 1 minute 18:59:30 #endmeeting