18:00:25 <smooge> #startmeeting EPEL (2019-05-22)
18:00:25 <zodbot> Meeting started Wed May 22 18:00:25 2019 UTC.
18:00:25 <zodbot> This meeting is logged and archived in a public location.
18:00:25 <zodbot> The chair is smooge. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:25 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
18:00:25 <zodbot> The meeting name has been set to 'epel_(2019-05-22)'
18:00:25 <smooge> #meetingname epel
18:00:25 <zodbot> The meeting name has been set to 'epel'
18:00:26 <smooge> #topic Chair and Introductions
18:00:26 <smooge> #chair bstinson Evolution nirik smooge pgreco tdawson
18:00:26 <zodbot> Current chairs: Evolution bstinson nirik pgreco smooge tdawson
18:00:52 <bstinson> wednesday again
18:01:05 <pgreco> hello
18:01:29 <smooge> hello everyone
18:01:36 <smooge> this will be a short lights on meeting
18:02:04 <smooge> #topic Open Floor
18:02:53 <pgreco> smooge, I have a request, more than anything
18:02:55 <smooge> Mainly this week I ahve been working on grobisplitter to work with the cdn. I am looking at some changes sgallagh is landing in libmodulemd2 and I can pull in default modules to the build
18:03:06 <smooge> pgreco, sure
18:03:09 <nirik> moring...
18:03:16 <tdawson> Hi, sorry I'm a bit late
18:03:23 * nirik wanted to bring up the ppc64 thing and get folks thoughts on it.
18:03:38 <pgreco> I'd like to know when you do some of the tests for epel8, even in stg
18:03:50 <pgreco> so I can replicate those in armhfp in "almost real time"
18:04:07 <pgreco> and I don't have surprises
18:05:20 <smooge> testing.. that is an interesting word you use there
18:05:48 <pgreco> trying and failing counts as a test to me :D
18:05:49 <smooge> I am sure i have heard it before.. but not sure I can place it with anything we do
18:06:20 <smooge> so one test I was going to look at adding was a repoclosure test.. but I found repoclosure does not understand modules
18:06:58 * nirik is happy to share anything we do for epel8... for feedback or if it's helpfull to others.
18:07:06 <smooge> I would be happy to work with you on what tests we can do
18:07:25 <smooge> I am planning on making all of the scripts and tools open
18:08:01 <pgreco> ack, I'll keep an eye on those scripts
18:09:02 <smooge> nirik, want to take over on the PPC proposal
18:10:38 <nirik> yeah, so fedora koji builders are running fedora.
18:10:52 <nirik> In order to get the latest rpm, etc for changes.
18:11:08 <nirik> fedora dropped ppc64 arch in Fedora 29.
18:11:29 <nirik> so, in order to keep building epel6/7 ppc64, we would need to make some rhel7 builders that only do that.
18:11:33 <nirik> which we can.
18:11:51 <nirik> but, is it really worth it for ppc64? it has few users I suspect
18:12:12 <nirik> but on the other hand we sort of told people we would keep doing it by adding it.
18:12:24 <nirik> so, drop? keep? investigate more? something else?
18:12:46 <pgreco> how complicated is it to migrate the builders?
18:13:13 <pgreco> wrt installation, and wrt using a different OS than the other builders
18:13:44 <nirik> it will need probibly a fair bit of work on our ansible playbooks which have moved to being fedora specific.
18:13:56 <nirik> also it means we will have builders that just sit there most of the time doing nothing.
18:14:20 <nirik> and it means we will need to keep koji-builder working on epel7
18:14:45 <pgreco> you need 2 sets of everything
18:15:17 <smooge> personally I would be ok with saying we are stopping producing PPC64 builds after RHEL-7.7
18:15:44 <pgreco> after the release or after the EOL of 7.7?
18:15:58 <smooge> we archive off PPC64 for EL6 and EL7 into /pub/archive/epel/el{6,7} and point the people to it
18:16:04 <smooge> after the point release
18:16:16 <pgreco> ok, so ~4 months
18:16:46 <smooge> F28 will EOL in a week or so I think so we are a bit late on this
18:17:26 <mizdebsk> nirik, ppc64 builders could still be used for building SRPMs, createrepo etc, or various lightweight parent tasks
18:17:28 <smooge> but I think the point release meets our commitments
18:18:38 <smooge> nirik, does that sound doable? Keep the builder until the point release?
18:19:24 <nirik> mizdebsk: they could, provided the tools on them are new enough.
18:19:32 <nirik> smooge: not really.
18:19:39 <nirik> they are currently fedora 28
18:19:56 <nirik> I don't want to keep them around after eol (even tho of course our armv7 ones are older)
18:20:06 <nirik> if we have to convert them to rhel7, the work will be done
18:20:28 <pgreco> and then we could just keep on going
18:20:42 <nirik> right.
18:21:10 <smooge> ok so we stop next week?
18:21:33 <nirik> or we deploy rhel7 ones... I guess I can look into how much ansible work will be needed.
18:23:54 <pgreco> nirik, I think you'll need a new rpm package
18:24:13 <nirik> I don't think so... what for?
18:24:16 <pgreco> among other things, so it understands "recommends" and others...
18:24:40 <nirik> the rhel7 builders should only ever build rhel6/7 packages.
18:24:46 <nirik> which shouldn't have that.
18:25:23 <pgreco> yeap, right, I have too much 8 in my head :)
18:25:55 <nirik> I suppose we could try making rhel8 builders...
18:26:00 <nirik> wait, no
18:26:20 <pgreco> looks like nobody wants ppc64
18:26:38 <nirik> it's old and crfty... but if you have hardware that only supports it...
18:27:24 <pgreco> at this point, I think there are more armhfp users than ppc64
18:28:31 <nirik> yeah.
18:28:39 <nirik> I don't know how to reach them or tell tho...
18:28:58 <nirik> I suppose we could send to epel-announce about it...
18:29:17 <nirik> or... we could disable it on f28 eol day, and wait a few weeks to see if anyone complains.
18:30:21 <smooge> I would say a little of both. I put it on the list already and it didn't get much reaction. I will blog about it and we disable it on f28 eol day. If we need it again after that we turn it back on
18:31:54 <nirik> yeah, until we nuke the f28 builders. ;)
18:32:20 <pgreco> can those builders be reused for something else?
18:32:27 <nirik> we can just disable it in pungi... so koji would still build them, but not collect them in repos
18:32:42 <nirik> pgreco: yeah, they are just vm's... the capacity could be reused for more ppc64le builders
18:36:10 <smooge> Proposal: We send one more thing to the list and blogs. We turn off ppc in pungi on f28 EOL day. We archive off ppc to /pub/archive/epel to make sure nothing gets lost.. if there is significant interest we turn them back on with the work of moving them to el7
18:36:35 <nirik> +1
18:36:48 <pgreco> right, and it could be done as nirik said, doing it gradually
18:37:03 <pgreco> 1) pungi, 2) turn off, 3) nuke vms
18:37:28 <pgreco> +1
18:37:33 <smooge> +1
18:37:39 <smooge> bstinson, ?
18:37:45 <tdawson> Sounds like a solution to me.  +1
18:37:52 <smooge> thanks tdawson
18:38:22 * nirik can blog on it, have been slacking on blogs lately.
18:39:11 <tdawson> It didn't come up, but don't we have numbers on the amount of users.  Wasn't it something like 100 to 1.5 million ?  (ppc64 vs x86_64)
18:39:36 <smooge> yes
18:40:01 <bstinson> sorry, got pulled in elsewhere
18:40:05 <bstinson> +1 belatedly
18:40:05 <pgreco> yeap and most of those were from ibm, right?
18:40:07 <tdawson> Yep, that just increases my +1 ... but I can't vote twice :)
18:40:19 <smooge> on a good day it is 90
18:42:36 <pgreco> does the fact that they come from IBM change anything?
18:42:44 <smooge> #agreed  We send one more thing to the list and blogs. We turn off ppc in pungi on f28 EOL day. We archive off ppc to /pub/archive/epel to make sure nothing gets lost.. if there is significant interest we turn them back on with the work of moving them to el7 (5+1, 1 0, 0 -1)
18:43:13 <smooge> pgreco, not at the moment
18:44:29 <smooge> 88 ppc64, 217 ppc64le, 38659 x86_32,  1591787 x86_64 on average for april
18:45:15 <nirik> thats per day? or over the month?
18:45:19 <smooge> that is per day
18:45:27 <nirik> nice
18:45:29 <pgreco> smooge, x86_32 means it includes el6, right?
18:45:39 <pgreco> can you split 6/7 somehow?
18:45:46 <smooge> x86_32 means its only el6
18:46:01 <nirik> can you tell the ppc64's which are which?
18:46:06 <nirik> I guess it doesn't matter
18:46:09 <smooge> pgreco, I can do so with some awk later
18:46:24 <smooge> nirik, what do you mean?
18:46:40 <pgreco> because if most of those are el6, eol is just around the corner anyway...
18:46:45 <nirik> I was asking the same thing as pgreco. which of the ppc64 ip's were 6 and which were 7
18:46:49 <nirik> right
18:46:51 <smooge> ah let me see
18:49:26 <smooge> 16 of them are el6 and 62 of them are el7
18:51:06 <nirik> alright.
18:51:18 <smooge> some overlap because some ask for both
18:51:18 <nirik> but of course it's the same work to support both...
18:51:53 * smooge thinks of an epel koji to just support epel
18:52:20 <nirik> thats... more work
18:53:12 <smooge> well if we are going to work...
18:53:27 <smooge> anyway.. my migraine is getting worse so I would like to call this meeting to an end
18:53:32 <smooge> thank you all for coming
18:53:45 <smooge> #endmeeting