21:00:43 <tdawson> #startmeeting EPEL (2020-03-13)
21:00:43 <zodbot> Meeting started Fri Mar 13 21:00:43 2020 UTC.
21:00:43 <zodbot> This meeting is logged and archived in a public location.
21:00:43 <zodbot> The chair is tdawson. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:43 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
21:00:43 <zodbot> The meeting name has been set to 'epel_(2020-03-13)'
21:00:45 <tdawson> #meetingname epel
21:00:45 <zodbot> The meeting name has been set to 'epel'
21:00:46 <tdawson> #chair nirik tdawson bstinson Evolution pgreco merlinm carlwgeorge
21:00:46 <zodbot> Current chairs: Evolution bstinson carlwgeorge merlinm nirik pgreco tdawson
21:00:54 <tdawson> #topic aloha
21:01:00 <nirik> hey
21:01:04 <carlwgeorge> howdy
21:01:22 <tdawson> Hi nirik and carlwgeorge
21:02:56 * tdawson gives it a few more minutes before starting.
21:03:22 <pgreco> hello
21:03:31 <tdawson> Hi pgreco
21:04:32 <pgreco> pidgin decided to duplicate my chat windows, so suddenly I was using the wrong one... :(
21:05:10 <tdawson> bad pidgin
21:05:20 <pgreco> ok, let me correct, triplicate.... :)
21:05:43 <tdawson> #topic Old Business
21:05:45 <tdawson> #info Discussion and/or voting on EPEL8 module addition to EPEL GuidelinesAndPolicies
21:05:46 <tdawson> #info https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraproject.org/thread/JX5FKXJXZN24GTZI7DSR67JVKVRHSFSX/
21:06:16 * nirik re-reads the last draft
21:06:36 <tdawson> I believe the last draft is in the lsat email I sent.
21:07:25 <pgreco> tdawson, I'm +1 on the draft. My only question is if this is affected by the new -devel packages being added in CentOS
21:07:56 <nirik> the last draft only talks about modules...
21:08:15 <pgreco> right, but -devel coming from modules may be affected
21:08:45 <pgreco> I'm ok with "shut up, it doesn't affect" :D
21:08:59 <nirik> I think thats a seperate issue/discussion...
21:09:09 <bstinson> modular devel packages are not yet covered by the thing we're doing in CentOS either
21:09:42 <nirik> I'm ok with the last draft of this in the thread, but would like to include it to also talk about non modular rpms.
21:09:47 <tdawson> pgreco: I believe those missing -devel packages would fall under the same category as -devel packages in CRB
21:10:51 <tdawson> nirik: So specifically say that non-module RHEL packages can be replaced with modular EPEL packages?
21:12:14 <nirik> no, the other case... can EPEL have a non modular package that is the same as one in a module (IMHO, no for default rhel modules, yes for non default?)
21:12:50 <nirik> perhaps we need a chart. ;)
21:12:59 <tdawson> Ohhh .... I hadn't thought of that scenario.
21:13:19 <nirik> modular (default) | modular (non default) | non modular
21:13:24 <nirik> epel | | |
21:13:27 <nirik> rhel | | |
21:13:37 <nirik> or something.
21:14:49 <tdawson> nirik: that's a good idea ... I wonder if the wiki will accept it.
21:15:07 <nirik> it can do tables, but not sure if thats the best way to show it.
21:15:25 <nirik> there's basically 3 cases for rhel and 3 for epel, so 9 possible combos?
21:16:16 <nirik> but perhaps more if you do 2 things at once... module and non module. Bah, giving myself a headache
21:17:10 <tdawson> Well, chart or not, your idea of non-modular EPEL packages replacing non-default RHEL packages ... I guess that would work.
21:17:31 <tdawson> Because if someone using RHEL were to enable a non-default module, it would override the EPEL package.
21:17:42 <nirik> yeah
21:17:55 <carlwgeorge> intriguing
21:18:05 <nirik> if the rhel module was default, the non modular version in epel would never get seen
21:18:54 <tdawson> So, does it really matter if it's default or not?
21:20:17 <nirik> oh, it might get used in the buildsystem... not entirely sure.
21:20:20 <tdawson> Well, I guess we've already seen, with libssh2, that our tooling currently will not let you branch for a package if it's in a default module.
21:21:02 * nirik nods
21:21:19 <tdawson> So, let's keep it simple and say non-default modules.
21:22:28 <tdawson> I like that idea ... but it's going to require a re-write of the proposal ... so I don't think we'll be voting on it this week.
21:23:11 <nirik> so, "In EPEL8 or later is also permitted for epel to provide a alternative non modular package to any packge in a non default RHEL module'
21:23:15 <nirik> or something like that
21:23:20 <nirik> yeah, we can poke it on list
21:23:39 <tdawson> Issues will handle tables.   And the email thread is a little messy.  Everyone ok if I make an issue for it?
21:24:06 <nirik> +1
21:24:20 <pgreco> +1
21:24:23 <carlwgeorge> i'll always vote for less email
21:24:38 <carlwgeorge> +1
21:24:52 <tdawson> OK, I'll start and issue, and we can work on it through the week.
21:26:34 <tdawson> #info After discussion, we will hold off on voting for the proposal.  Will open an issue and track it in there instead of email.
21:26:50 <tdawson> OK, that brings us to the next point of discussion.
21:26:56 <tdawson> #info Discussion about using CentOS Devel repo packages in EPEL8
21:26:57 <tdawson> #info https://lists.centos.org/pipermail/centos-devel/2020-March/036644.html
21:27:15 <tdawson> bstinson: ping - we're discussing the CentOS Devel repo
21:27:42 <tdawson> I guess the first question, Do we want to use that repo for our missing -devel packages?
21:28:05 <bstinson> so we have a general shape for this repo in CentOS
21:28:30 <tdawson> My opinion is yes ... but let's hear bstinson on the subject.
21:29:10 <nirik> I am... not convinced it will work, but I am willing to try it. :)
21:29:23 <bstinson> i think it has the potential to cause a few problems if we consume it in EPEL, but like many things to do with unshipped packages, i don't see how to have our cake and eat it too
21:30:58 <pgreco> what about s390x?
21:31:10 <tdawson> What are the things it solves, vs the things that might go wrong (pro's and con's)
21:31:11 <bstinson> see the "few problems"
21:31:43 <pgreco> the cons I see is s390x, and divergence in time between centos and rhel
21:32:02 <tdawson> Yep
21:32:09 <bstinson> and these are completely open to the wild west
21:32:19 <bstinson> RHEL is reserving the right to do anything or nothing with these packages
21:32:20 <nirik> epel builds from a split/massagued RHEL, adding a CentOS repo next to it will mean that when they are even slightly out of sync things will blow up...
21:32:40 <smooge> another problem will be where timestamps and other items diverge. koji gets grumpy the -devel and the main package are different
21:32:50 <nirik> that too.
21:33:11 <smooge> it was the reason we could not build aarch64 on el7 with the centos tools
21:33:23 <pgreco> smooge, no, that was different
21:33:34 <pgreco> that was because we were using 2 version of the same file
21:33:52 <pgreco> xxxx.noarch.rpm from rhel, and the same xxx.noarch.rpm from centos
21:33:59 <pgreco> koji hates that :D
21:34:14 <tdawson> Let's say we didn't use the CentOS Devel repo, and we made our own.  It would fix the s390x, but  would we still have the timestamp issue?
21:34:20 <smooge> ok I thought koji also complained about other mismatches
21:34:47 <smooge> tdawson, we would need to replace all-da-packages made from a srpm
21:34:56 <pgreco> smooge, it may complain about other things too, I'm fairly familiar with the one I described, just that ;)
21:35:30 <smooge> understood.
21:36:12 <smooge> so if RHEL ships foo-goo-bin, foo-goo-lib and doesn't ship foo-goo-devel we need to have all 3 in this repo and not the others
21:36:27 <smooge> or at least my attempts to try it otherwise failed horribly
21:36:43 <tdawson> The timestamp thing sorta makes this a non-starter, or at least not easy.  Is there any way to override that.
21:36:48 <tdawson> And you just answered that.
21:36:57 * nirik is happy to try things in stg and see what blows up... given time to do so
21:37:41 <tdawson> I wouldn't mind trying in stg either ... see the blow up for myself ... if people don't mind setting it up.
21:38:13 <nirik> should be mirroring it and adding it as another external repo.
21:38:17 <bstinson> some of the other problems won't show up until point-release time, either
21:38:25 <bstinson> one of them being the trouble with lag
21:38:46 <tdawson> True, but even if EPEL built them all themselves, we'd still have some of that lag.
21:39:00 <smooge> well my original hope was that at point release time we would stop our grobisplitter from moving to EL8.N+1
21:39:07 <bstinson> the other being that Devel is specifically curated content (you need to ask for packages to be included) so if a package moves into a released variant, then we have to monitor that by hand
21:39:17 <bstinson> that's by design for reasons(TM)
21:39:53 <bstinson> but we may need to craft the externalrepo order carefully to compensate
21:40:05 <pgreco> stream may help with that too
21:40:21 <pgreco> cherry picking from stream instead of 8.x may help with the delay
21:40:57 <pgreco> again, manual labor, but I could help automating this if we wanted to
21:42:17 <nirik> I'm not sure stream helps... it would mismatch the devel and non devel rpms more wouldn't it?
21:42:47 <bstinson> if cherry-picking is something that epel wants to do, sure, but that's a no from me for us in CentOS
21:42:49 <pgreco> not if we do what smooge said
21:42:59 <pgreco> nirik ^^
21:43:16 <pgreco> bstinson: yes, I think we're talking about epel cherry-picking
21:43:37 <nirik> pgreco: but then you force everyone to use and build against stream instead of 'stable'
21:44:01 <pgreco> only during that period, yes
21:44:25 <pgreco> we could do that "only" if the build matches exactly
21:44:37 <pgreco> and we still don't have the same version from centos 8.x
21:44:39 <tdawson> So, my mind is thinking priorities (as in inhertance priority in koji).   If we had not only the RHEL repo's, but also all the CentOS repo's, with the CentOS repo's at a lower priority.  If rhel bumps up to 8.3, and CentOS is still at 8.2, and a package needed a -devel package that was at the lower CentOS version, would koji see that there was an older version and be able to use it?
21:45:10 <tdawson> Or is koji only seeing the one, newer RHEL 8.3 version?
21:45:17 <nirik> only the newer
21:45:23 <pgreco> I think so too
21:45:33 <pgreco> in fact, I think it would never see the -devel from centos
21:45:38 <tdawson> That's what I was afraid ... that it just takes the newest.
21:45:45 <bstinson> koji resolves by source package name
21:45:46 <nirik> koji operates on src packages. Once it decides where it is getting package A from, it ignores everything about package A in all other repos
21:46:04 <bstinson> and takes whatever is in the highest priority repo, no matter what the NVR is
21:46:10 <bstinson> right, what nirik said
21:46:12 <pgreco> tdawson: not even the newest, the one from the repo with the better priority
21:46:16 <nirik> (although this is not the case with epel8 probibly)
21:46:54 <tdawson> Ya, I hope hoping it acted more like regular dnf. :(
21:47:31 <nirik> epel8 uses external repos and has a flag set to merge them in a different way... ignoring overlaps... but I still don't think it would work...
21:47:31 <smooge> in the case of EPEL-8, koji is relying on DNF to set up the repo data but after that does its normal 'i choose you pikachu'
21:47:40 <pgreco> tdawson, I was thinking about doing the exact opposite  actually
21:48:06 <pgreco> putting cherry picked rpms from centos at a better priority and then "full rhel"
21:48:40 <bstinson> that causes problems the other direction if there happens to be a rebase in RHEL
21:49:04 <pgreco> I'm using better to describe the priority because lower number means higher priority....
21:49:12 <bstinson> right
21:49:13 <pgreco> bstinson, yes, that is where Stream "could" help
21:49:56 <nirik> then people would yell that we missed security updates... possibly
21:49:57 <bstinson> could, but is unequipped for at this stage
21:50:05 <smooge> so different question.. if EPIC was built inside of koji.mbox instead of koji.fedoraproject.org would it be easier to deal with these issues?
21:50:25 <pgreco> yes
21:50:35 <bstinson> if packages were built in koji.mbox.centos.org, they would have access to the entirety of the unshipped content
21:50:39 <nirik> some of them.
21:51:13 <smooge> ok I am not going to go down that rabbit hole any further
21:51:22 <tdawson> Did you mean to say EPEL?  Or am I missing something about EPIC?
21:54:06 <smooge> I said EPIC because EPEL usually means built on RHEL and I use a different name ot make it clear it wasn't
21:55:40 <tdawson> Ah, ok
21:56:15 <tdawson> It's getting towards the end of the meeting.  Do we have any sort of plan?
21:56:45 <pgreco> play with stg until we decide one way or the other?
21:57:03 <tdawson> To me it sounds like we want to try setting somethng up in stg, just to see if we can get around the various problems. ... what pgreco said.
21:58:24 <nirik> yeah
21:58:33 <tdawson> Do we have volunteer(s) to do that?  Or should we open a ticket for it?
21:58:43 <tdawson> Or both
21:59:05 <pgreco> I don't have permissions over stg, but I volunteer to break things
21:59:05 <nirik> well, it needs someone with perms... I can try and find time, but a ticket would be best.
21:59:24 <nirik> I'd prefer perhaps to do the setup with someone and then have them test. :)
21:59:48 <bstinson> i'm can put some time into answering questions about Devel for whoever is doing the actual work. and if we need packages added to 8 Devel or Stream Devel i'm happy to do so
22:00:34 <tdawson> OK, so start with a ticket, anyone want to open it and send it to the group?  Or should I?
22:01:01 <nirik> bstinson: we have some ticket(s) with requests I think... might be not all those tho
22:01:25 <bstinson> nirik: yeah sorting through the backlog is next on my list for Devel
22:01:32 <pgreco> tdawson: please create it if you can, we can link everything from there
22:01:42 <tdawson> OK, will do.
22:01:50 <bstinson> i'm paying attention to the epel project in pagure, bugs.centos.org, and in some cases bugzilla
22:02:36 <tdawson> #info We will open a ticket to have the stg epel build use the CentOS Devel repo.  We will look at what breaks, and try to fix the problems.
22:03:07 <tdawson> #assign tdawson to open the ticket for the stg epel koji
22:03:24 <tdawson> OK, looks like we are ove ra little bit.  Thanks for coming and the good discussions.
22:03:38 <tdawson> #endmeeting