17:00:02 <StephenGallagher> #startmeeting ELN SIG 2021-11-19
17:00:03 <zodbot> Meeting started Fri Nov 19 17:00:02 2021 UTC.
17:00:03 <zodbot> This meeting is logged and archived in a public location.
17:00:03 <zodbot> The chair is StephenGallagher. Information about MeetBot at https://fedoraproject.org/wiki/Zodbot#Meeting_Functions.
17:00:03 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
17:00:03 <zodbot> The meeting name has been set to 'eln_sig_2021-11-19'
17:00:03 <StephenGallagher> #meetingname eln
17:00:03 <StephenGallagher> #topic Init Process
17:00:03 <StephenGallagher> .hello sgallagh
17:00:03 <zodbot> The meeting name has been set to 'eln'
17:00:04 <zodbot> StephenGallagher: Something blew up, please try again
17:00:06 <zodbot> StephenGallagher: An error has occurred and has been logged. Please contact this bot's administrator for more information.
17:00:29 <StephenGallagher> .hello sgallagh
17:00:30 <zodbot> StephenGallagher: Something blew up, please try again
17:00:33 <zodbot> StephenGallagher: An error has occurred and has been logged. Please contact this bot's administrator for more information.
17:00:39 <dcavalca> .hi
17:00:40 <zodbot> dcavalca: Something blew up, please try again
17:00:43 <zodbot> dcavalca: An error has occurred and has been logged. Please contact this bot's administrator for more information.
17:00:49 <davdunc> .hi
17:00:50 <dcavalca> oh well
17:00:50 <zodbot> davdunc: Something blew up, please try again
17:00:52 * tdawson doesn't try anything fancy
17:00:53 <zodbot> davdunc: An error has occurred and has been logged. Please contact this bot's administrator for more information.
17:00:57 <dcavalca> I guess zodbot is on strike again
17:01:04 <tdawson> But Hi to ya'll
17:01:07 <jforbes> .hello2
17:01:08 <zodbot> jforbes: Something blew up, please try again
17:01:11 <zodbot> jforbes: An error has occurred and has been logged. Please contact this bot's administrator for more information.
17:02:58 <jforbes> Oh, hmm. Guess I should have read backscroll first
17:02:58 <cyberpear> .halp zodbot is unhappy
17:02:58 * nirik is fixing it. but it's only fas lookup, nothing else is wrong.
17:02:58 <StephenGallagher> #topic Fedora Python and the Mysterious %{eln} variable
17:02:58 <StephenGallagher> https://github.com/fedora-eln/eln/issues/73
17:03:46 <StephenGallagher> A while back, we set a policy (now encoded in the docs, thanks tdawson ) that any usage of the `%eln` macro needed approval.
17:04:03 <StephenGallagher> Mostly so we could avoid diverging from RHEL constantly.
17:04:08 <MichelAlexandreS> .hi
17:04:09 <zodbot> MichelAlexandreS: Sorry, but user 'MichelAlexandreS' does not exist
17:04:18 <MichelAlexandreS> .hello salimma
17:04:18 <zodbot> MichelAlexandreS: salimma 'Michel Alexandre Salim' <michel@michel-slm.name>
17:04:39 <StephenGallagher> It turns out that Python is one such case where Fedora, ELN and RHEL/CentOS Stream all need to behave differently at this time
17:05:16 <StephenGallagher> I don't want to recap the summary that Miro put into the ticket, since he did a much better job than I can at explaining the reasons.
17:06:01 <tdawson> I've read it, and I give their reasons a +1 and/or approve, whichever is needed.
17:06:29 <StephenGallagher> Yeah, I propose that we approve (well, rubber-stamp) the current state
17:06:58 <StephenGallagher> I'd like to revisit it down the road, because diverging Python like this will cause issues when we hit the RHEL 10 branch.
17:07:15 <StephenGallagher> But I don't have any workable short-term solutions.
17:07:18 <cyberpear> I'm not a fan but haven't spent lots of time considering it.
17:07:56 <StephenGallagher> It's the best out of a lot of bad options for the moment, in my opinion.
17:08:00 <jforbes> I do understand the reasoning, and while Fedora and ELN could be changed to do something similar to what RHEL is doing, it doesn't seem to make too much sense
17:08:41 <jforbes> Largely because the lifecycle of RHEL makes it an interesting solution, but for Fedora/ELN it would just be more work without the benefit
17:08:49 <StephenGallagher> Actually, I guess that *is* the real question. mhroncok if you happen to be around, is there a strong reason we can't do the same thing in Fedora?
17:09:09 <StephenGallagher> Is it more work, or is it macro tweaks? I don't know.
17:09:16 <cyberpear> ++ bringing the two closer
17:10:15 <StephenGallagher> But I still think that should be an ongoing discussion and we should grant approval for the current approach, if only so we continue on unbroken for a while.
17:10:45 <mhroncok> StephenGallagher: I am around
17:10:59 <mhroncok> StephenGallagher: I even have this meeting in y calendar, but somehow missed it :/
17:11:59 <StephenGallagher> mhroncok: Thanks for joining us!
17:12:01 <jforbes> I would agree with approval for the current approach, even if only temporary
17:12:11 <mhroncok> the strong reason is that it makes no sense in Fedora. the directory in Fedora is shared for all Pythons
17:13:03 <mhroncok> we could technically make it not shared and build idfferent versions of setuptools and pip and wheel for all the different Pythons we ship, but we don't build parallel Python stacks for Fedora
17:13:30 <mhroncok> and we keep pip setuptools wheel up to date, unlike rhel, where we keep them stable
17:13:46 <StephenGallagher> Sure, but would it be so bad if we had them that way on disk?
17:14:02 <mhroncok> to the problem rhel has is that when we introduce Python 3.24, pip from 1995 is no longer working with it
17:14:15 <StephenGallagher> i.e. `/usr/share/wheels3.10` instead of `/usr/share/wheels`?
17:14:18 <mhroncok> so we need a pip from this century
17:14:39 <mhroncok> StephenGallagher: so we install the wheels to `/usr/share/wheels3.10`and tahn what
17:14:44 <StephenGallagher> Even if we only ever have one installed at a time, can't we use the RHEL naming structure?
17:14:45 <salimma[m]> yeah, always using versioned directories seems to make sense.
17:14:55 <mhroncok> python 3.12 uses wheels from `/usr/share/wheels3.10` as well? How does it know?
17:14:58 <StephenGallagher> mhroncok: I'm asking, not telling.
17:15:06 <StephenGallagher> symlinks?
17:15:14 <salimma[m]> don't we rebuild all Python packages when we upgrade Python anyway?
17:15:16 <mhroncok> what package cretaes the symlinks?
17:15:28 <mhroncok> salimma[m]: yes, but that is completely unrelated to this problem
17:15:34 <jforbes> Again, that just becomes more work without any real benefit
17:15:39 <mhroncok> salimma[m]: we have python3.11 available today on fedora
17:16:02 <mhroncok> salimma[m]: it uses the same wheels as python3.10 and python3.9, becasue it works
17:16:05 <salimma[m]> mhroncok: ah, got it, on Fedora you want the alternate versions to share the wheels
17:16:12 <mhroncok> yes
17:16:22 <salimma[m]> it would make sense for the alternate Pythons to make that symlink then
17:16:27 <StephenGallagher> So those alternate versions could just own their symlink.
17:16:28 <mhroncok> becasue the life time of fedora release is shorter than python release
17:16:29 <StephenGallagher> Yeah
17:16:30 <salimma[m]> and they could opt not to if they know it's incompatible
17:16:59 <mhroncok> StephenGallagher: so all the laternate Python version would own symbolic links to one common location?
17:17:02 <copperi[m]> yes
17:17:11 <StephenGallagher> mhroncok: That's what I'm suggesting, yeah
17:17:19 <mhroncok> and what package owns the shared location?
17:17:35 <StephenGallagher> Whichever is the system-python for the release
17:17:37 <salimma[m]> the system Python maybe
17:17:38 <salimma[m]> yeah
17:17:42 <mhroncok> that's the one and only version of pip/setuptools/wheel package that puts stuff in there
17:17:52 <jforbes> This seems unnecessarily complex.
17:17:58 <mhroncok> but on RHEL, there is no such package
17:17:58 <StephenGallagher> Is that a question or a statement?
17:18:24 <mhroncok> we could bend the Fedora solution around the RHEL problem
17:18:34 <mhroncok> but for the sake of what exactly?
17:18:50 <mhroncok> on Fedora, we only have one common location for wheels
17:19:07 <StephenGallagher> Being able to rebuild the python packages for EPEL using the RHEL directory structure while still allowing Rawhide builds in the buildroot for ELN
17:19:16 <mhroncok> and as long as we will only have one common wheel "repository" I see no reason to namespace the directory with Python versions
17:19:41 <mhroncok> StephenGallagher: I do not nderatnd your last message
17:19:46 <mhroncok> *understand
17:20:27 <mhroncok> the builds in epel will use the rhel directory structure
17:20:32 <StephenGallagher> mhroncok: The reason for your usage of the `%eln` macro is because we tag Rawhide builds into the ELN buildroot and use those to build ELN builds, correct?
17:20:45 <mhroncok> yes
17:20:51 <StephenGallagher> So if we tried to have ELN build in the RHEL style, they would be unable to locate the dependent packages for ELN. Do I have that right so far?
17:21:39 <mhroncok> if the eln build of python tries to use the rhel-style directory and pulls the rawhide-built wheels into the buildroot, the build fails
17:22:03 <StephenGallagher> mhroncok: OK, maybe I'm misunderstanding the *why* of it.
17:22:09 <StephenGallagher> Why does it fail, exactly?
17:22:10 <mhroncok> not to mention if some the rawhide builds bubble up to the eln compose, in that case, it fails for the users
17:22:51 <mhroncok> StephenGallagher: becasue during the tests, it will try to load files form a directory that does not exist and it will end with FileNotFoundError and thta will fail the tests and thta will fail the %check and that will fail the build
17:22:58 <StephenGallagher> (I feel like there's probably some crucial nugget that I'm just missing here and that you're so familiar with that you are talking past it)
17:23:28 <StephenGallagher> OK, and how does adding a symlink not address that?
17:23:51 <mhroncok> what symlink? where does it go from, where does it go to? what pakcage is this symlink in?
17:24:25 <mhroncok> (I probably fail to understand what are you trying to acomplish here, sorry)
17:25:01 <StephenGallagher> I'm trying to avoid the FileNotFoundError without needing ELN to diverge wildly from RHEL
17:25:28 <StephenGallagher> So I'm asking if we can't just have a symlink for the root of the unversioned wheel directory to one that has the version.
17:25:55 <StephenGallagher> So ELN's python package could locate things where RHEL would expect them
17:26:12 <mhroncok> "symlink for the root of the unversioned wheel directory to one that has the version"
17:26:16 <jforbes> The simplified issue, is using eln works fine for now, but means when things branch for RHEL next, everything is broken until things are rebuilt
17:26:44 <mhroncok> so /usr/share/python-wheels would be a symbolic link *to* /usr/share/python3.10-wheels?
17:26:45 <StephenGallagher> mhroncok: What are the wheel directories called in RHEL and Fedora?
17:26:49 <mhroncok> or the other way around?
17:27:10 <mhroncok> that is listed in the ticket
17:27:17 <mhroncok> fedora: /usr/share/python-wheels
17:27:43 <StephenGallagher> If it's listed in the ticket, I'm not seeing it.
17:27:50 <mhroncok> rhel: /usr/share/python3-wheels for "python3", /usr/share/python3.X-wheels for any additional Python 3 version
17:28:11 <mhroncok> StephenGallagher: "The values of the macros are (for the prefix and for the dir):"
17:28:12 <StephenGallagher> ok
17:28:51 <StephenGallagher> OK... so why can't we have a symlink from `python3-wheels` to `python-wheels`?
17:29:18 <StephenGallagher> ELN doesn't need to care about the `python3.10-wheels` as far as I can tell
17:30:10 <mhroncok> Fedora doesn'T need to care about python3-wheels
17:30:51 <StephenGallagher> But would having that symlink be harmful (or even much effort to maintain) if it made things easier for ELN?
17:31:09 <mhroncok> I have no idea what package would even own that symolic link
17:31:40 <StephenGallagher> Why couldn't it just be python3.X-libs (where X is whatever the "system" python is for this Fedora release)
17:32:01 <StephenGallagher> Sorry, I mean python3-libs
17:32:04 <mhroncok> it would mena that the package that creates the symbolic link and owns it would needs additional logic to know whether it should do that or not
17:32:09 <StephenGallagher> I forgot the built RPMs don't include the .X
17:32:27 <StephenGallagher> When shouldn't it?
17:32:37 <mhroncok> when on rhel?
17:32:53 <mhroncok> that symbolic link woudl conflcit with the actual directory created on rhel
17:33:35 <StephenGallagher> ok, so we wouldn't entirely eliminate the need to have a %eln conditional, but the end result would still be much closer to what RHEL should eventually look like, yes?
17:33:57 <mhroncok> let me try to wrop this up in my head first...
17:34:00 <mhroncok> out loud
17:34:11 <StephenGallagher> Sure. I'll go silent until you EOM
17:34:22 <cyberpear> can't we just use python3-wheels in all cases?
17:35:20 <mhroncok> the "main" Python interpreter in Fedora would require wheel packages that install wheels to /usr/share/python-wheels, it would load the wheels from /usr/share/python-wheels, but it will also own a symbolic link /usr/share/python3-wheels -> /usr/share/python-wheels
17:35:46 <cyberpear> skip python-wheels entirely? (or ln -s python3-wheels python-wheels on Fedora)
17:36:19 <mhroncok> the "main" Python interpreter in ELN as well as in RHEL would require wheel packages that install wheels to /usr/share/python3-wheels, it would load the wheels from /usr/share/python3-wheels, and it would add no symbolic link at all
17:36:21 <StephenGallagher> cyberpear: Let him get his thoughts out first. Wait until his EOM, please.
17:37:20 <mhroncok> so when you build Python on ELN but it installs wheels from rawhide, it loads the wheels from /usr/share/python3-wheels, shich is a link to /usr/share/python-wheels and there are the wheels and tadah, it works, everyhting s nice and... oh waiyt
17:37:34 <mhroncok> if one of the wheel was build on rawhide
17:37:44 <mhroncok> and another wheel was built on ELN
17:38:19 <mhroncok> one of them is in /usr/share/python-wheels and the other one is in /usr/share/python3-wheels. but one fo those is a symbolic link to the other -- smells like RPM transaction error to me
17:39:12 <mhroncok> also, if python3-libs owns that symbolic link, we would not be able to build Python without having python3-libs installed in the buildroot (which we already do as the default, but it is bconded for bootstrap situations)
17:39:57 <mhroncok> as well as if we have the Python package built on rawhide and the wheels built on ELN, the wheels try to install to a directory which is a symbolic link
17:40:20 <mhroncok> this cerates so much compexity and only solves one order of the builds.
17:40:38 <mhroncok> EOM, but will answer cyberpear's questions
17:41:15 <mhroncok> "can't we just use python3-wheels in all cases?" we can, except that in theory, some wheels for Python 2 are in that directory as well (or for PyPy)
17:41:59 <mhroncok> oh, the second question seems just like a clarification of the first
17:42:24 <StephenGallagher> Does it matter if py2 and PyPy retain the use of `python-wheels`?
17:42:30 <StephenGallagher> Would that interact in any way?
17:42:53 <cyberpear> right. doing the opposite symlink to match RHEL only works once python2.7 is gone
17:43:23 <mhroncok> StephenGallagher: how do they reatin it if we install the wheels to a different place?
17:43:31 <mhroncok> it is 1 directory to rule them all
17:43:56 <mhroncok> it contains all wheels we need for all our Python interpreters (except the onces that we bundle in the Python interpreter pacakges when needed)
17:44:32 * StephenGallagher grasps at straws.
17:44:33 <mhroncok> that's wh there is no version in the directory name what so ever
17:44:59 <mhroncok> while on RHEL...
17:45:09 <StephenGallagher> Ignore that last; I thought I had another idea, but I saw a flaw.
17:45:10 <mhroncok> it contains wheels for 1 interpreter only
17:47:32 <StephenGallagher> Ok, I think we've beaten this topic to death for now. Anyone opposed to signing off on the current strategy for now? I'll try to sit down with mhroncok sometime in the next few months and see if we can go anywhere with this, but I don't think we need to solve it here today.
17:47:51 <StephenGallagher> mhroncok: Thank you very much for your input (and patience)
17:48:03 <davdunc> you have you hands full there.
17:48:13 <jforbes> I am fine with the current strategy at this time
17:48:25 <copperi[m]> +1
17:49:04 <mhroncok> note 1 - we have already done this, so if you don't feel like rubber stamping this now before trying alternatives, that is fine, as long as you don't "order" us to delete that macro
17:49:05 <cyberpear> +1 to avoid blocking stuff.
17:49:35 <StephenGallagher> Right, I'm aware. But it looks like we have consensus to proceed as-is for now
17:49:52 <cyberpear> i like just using python3-wheels possibly with a compatibility symlink. py2 uses built in
17:50:12 <salimma> +1
17:50:18 <mhroncok> not 2 - I will speak to StephenGallagher and see how the ELN > c10s rebuild bootstrap might work and if this difference between ELN and RHEL will cause toruble (I don't think it will, but I don't knwo the procedure)
17:50:21 <StephenGallagher> #agree The python macros are permitted to use the %eln macro as described in https://github.com/fedora-eln/eln/issues/73
17:50:26 <cyberpear> we ship only 3 wheels that I can see: pip, setuptools, wheel
17:50:37 <mhroncok> cyberpear: correct, for now
17:51:11 <StephenGallagher> #topic ELN Compose Issues: The Next Generation
17:51:28 <mhroncok> cyberpear: some fundamentalists want to have setuptools-less venvs, so it might be just 1 in the future, but other disagree. setuptools maintainers OTOH want to have real deps, so it might be 8 in the future
17:51:44 <mhroncok> *s/not 2 /note 2/
17:51:54 <StephenGallagher> I'll keep this brief: I've been trying to fight with a circular set of dependencies in the GCC/gdb stack for a week.
17:51:54 * mhroncok will shtut up already, you are on another topic
17:52:30 <StephenGallagher> mhroncok: Once again, thank you very much
17:54:13 <jforbes> StephenGallagher: I am guessing this is what is holding up composes and making mock eln targets broken?
17:54:29 <StephenGallagher> Essentially, there's a loop around gdb, gcc, doxygen and libstdc++
17:54:42 <StephenGallagher> jforbes: Yes, and it's driving me nuts.
17:55:04 <StephenGallagher> At this point, I could really use some help tracking it down.
17:55:15 <StephenGallagher> (Particularly since I'll be leaving for a week's vacation in four hours)
17:56:09 <salimma> most of the USians here are probably in similar situations
17:56:13 <jforbes> So I am not sure why this is a problem in ELN, but the bootstrap to get around those circular dependencies involves 2 different special gcc builds (maybe we are far enough along that we could skip the first)
17:56:45 <jforbes> But that is based on my work bootstrapping a distro, and it has been 12 years or so since I last did it
17:56:46 <StephenGallagher> jforbes: They shouldn't *need* the bootstraps, because the F36 final version is in buildroot
17:57:17 <StephenGallagher> The double bootstraps are done to make sure that we're shipping a compiler that compiled itslef.
17:57:25 <jforbes> Right
17:57:43 <StephenGallagher> But we can use the Fedora compiler to compile the ELN one without the bootstrapping steps
17:57:44 <StephenGallagher> Since it's all there
17:59:34 <StephenGallagher> Anyway, we're out of time. If anyone thinks they can help, please ping me over in #eln:fedoraproject.org
17:59:48 <salimma> enjoy your vacation Stephen Gallagher
17:59:55 <StephenGallagher> Thanks
17:59:58 <StephenGallagher> #endmeeting