15:00:24 <contyk> #startmeeting Modularity Team Meeting (2019-09-24)
15:00:24 <zodbot> Meeting started Tue Sep 24 15:00:24 2019 UTC.
15:00:24 <zodbot> This meeting is logged and archived in a public location.
15:00:24 <zodbot> The chair is contyk. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:24 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:24 <zodbot> The meeting name has been set to 'modularity_team_meeting_(2019-09-24)'
15:00:30 <contyk> #meetingname modularity
15:00:30 <zodbot> The meeting name has been set to 'modularity'
15:00:37 <contyk> #topic Roll Call
15:00:41 <sgallagh> .hello2
15:00:41 <contyk> .hello psabata
15:00:41 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
15:00:44 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
15:01:05 <contyk> it might be just the two of us today
15:01:41 <contyk> #chair contyk sgallagh
15:01:41 <zodbot> Current chairs: contyk sgallagh
15:01:51 <sgallagh> ...
15:02:24 <contyk> I'd propose skipping but we need to cover that FESCo thing
15:02:34 <sgallagh> Yes
15:02:40 <sgallagh> Is bookwar around?
15:02:51 <contyk> #topic #146 Add policy that all packages in default stream MUST be part of module API
15:02:57 <contyk> .modularity 146
15:02:58 <zodbot> contyk: Issue #146: Add policy that all packages in default stream MUST be part of module API - modularity - Pagure.io - https://pagure.io/modularity/issue/146
15:03:47 <contyk> doesn't seem to be
15:03:52 <sgallagh> ok
15:03:53 <contyk> this is going to be quite lame then
15:04:19 <contyk> so
15:04:26 <sgallagh> Well, let's give it a try and at least we can air our thoughts for the logs
15:04:49 <contyk> I understand you want to declare all exposed packages in the API section to mimic the non-modular content's guarantees
15:05:04 <sgallagh> That's not strictly true
15:05:08 <contyk> what I don't really understand is why
15:05:15 <sgallagh> Mind if I restate the problem?
15:05:19 <contyk> sure
15:05:50 <sgallagh> I'll type a bunch and then EOF when I've finished.
15:05:56 <contyk> okay
15:07:01 <sgallagh> First point: Modularity was designed with a principle that we could specify the portion of the packages that are carried as being "API" (hereafter defined as "other software is allowed to rely on any aspect of the exposed API from those packages"
15:07:39 <sgallagh> Second point: We did not (arguably: cannot) implement any technical enforcement of this API limitation. (No automatic namespacing, static linking, etc.)
15:08:14 <bookwar> .hello2
15:08:15 <zodbot> bookwar: bookwar 'Aleksandra Fedorova' <alpha@bookwar.info>
15:08:26 <sgallagh> Third point: All packages present in the default stream of a module are installable with dnf either directly or as a dependency for other package.
15:09:02 <sgallagh> Fourth point: there is no discoverable UX for a user or developer to determine whether a package they want to rely on is supported for their use-case.
15:09:43 <sgallagh> Fifth point: Some software commonly used as dependencies exists in default streams as non-API content today.
15:09:56 <sgallagh> (Example: much of the Java stack)
15:10:20 <sgallagh> EOF
15:10:30 <sgallagh> bookwar: Is that a satisfactory summary of the problem?
15:10:50 <bookwar> maybe to add modular non-api packages occupy the slot in the default namespace
15:11:03 <sgallagh> ack
15:11:12 <contyk> alright
15:11:36 <sgallagh> Sixth point: The default streams and the non-modular RPMs have the same namespace, with the default streams superseding a non-modular RPM of the same name.
15:11:48 <contyk> I'd probably prefer giving the maintainer a choice here
15:11:50 <sgallagh> (Numbering them so we can refer to them later)
15:12:10 <contyk> either make the package shareable and maintain it, i.e. make it the API
15:12:21 <sgallagh> contyk: Their choice would be "Maintain everything you are shipping or don't make it a default stream" in the current proposal
15:12:25 <contyk> or namespace it, i.e. rename the package
15:12:44 <sgallagh> Right, that's not incompatible with what I just said.
15:12:47 <contyk> that's more in line with the original idea about the module being an app and deciding on their own what they want to bundle
15:12:55 <contyk> indeed
15:13:38 <contyk> yeah, I think that would be acceptable, considering the points you brought up
15:13:47 <contyk> either support it or namespace it
15:14:42 <sgallagh> Right, so I guess there are three options they can take:
15:15:04 <sgallagh> 1) Maintain all of the packages in your default stream as if they were non-modular.
15:15:19 <sgallagh> 2) Namespace whatever packages you don't want to maintain for general use
15:15:33 <sgallagh> 3) Don't make the stream the default.
15:16:06 <bookwar> to give a practical use case: i am developer of a leaf application which uses java stack, to develop this application i need maven-plugin-*. Now i am running 'dnf install maven-plugin-X', it gets installed, my application works, i put it aside. But maven-plugin-X was actually a non-api part of a java-tools package, and 2 months later the maintainer of a java module rebuilds the maven-plugin-X disabling support for the featu
15:16:07 <bookwar> , because it is not needed by any of the packages in the java-tools module. Now my app is broken and can not be built anymore
15:16:24 <contyk> re:#2, it's not just about package names, of course -- all the provides and files
15:16:47 <sgallagh> contyk: Ack. Consider my statement an abbreviation.
15:17:00 <contyk> yes
15:17:31 <sgallagh> bookwar: Yes, if this policy is in place, that would constitute a violation of the Stable Updates Policy.
15:17:43 <sgallagh> And the maintainer would be required to revert it
15:17:56 <bookwar> and the thing is, i actually looked into packages in java-tools module, and i saw that those non-api packages from this module are built with extremely restrictive compile options, things like - disable svg support because we don't build docs and so on, so it is not a theoretical use-case, it is basically what we are dealing with right now
15:18:55 <asamalik> .hello2
15:18:56 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
15:18:58 <sgallagh> bookwar: I think contyk and I (and asamalik in the ticket) are on the same page for the policy decision
15:19:13 <sgallagh> So is that decision acceptable and sufficient to you?
15:19:20 * asamalik is on a phone, in a tram, and not yet decided how useful he will be
15:19:51 * sgallagh will be nice and not take the obvious opportunity for snark.
15:20:05 <bookwar> i don't understand how 2) is going to work
15:20:12 <bookwar> 1) and 3) are clear
15:20:14 * contyk also tried hard
15:20:43 <sgallagh> bookwar: 2) will be uncommon, because it will be difficult and custom
15:21:06 <sgallagh> It's essentially just "figure out how to bundle this so you don't own the primary namespace"
15:21:30 <bookwar> afaik there is no concept of namespaces officially, so it is going to be - introduce new package in Fedora which overlaps with the default package
15:21:32 <contyk> would be nice if we could make it easier
15:21:33 * asamalik just looks in the direction of sgallagh and contyk, but has no reaction for them
15:21:34 <sgallagh> Might be static linking, renaming libraries, etc.
15:21:49 <sgallagh> contyk: We went down that road before. I still have the tire tread marks on my back.
15:21:51 <contyk> perhaps people could come up with fancy macros, scl-like
15:22:22 <contyk> then define %privatedep or something and it would prefix all the paths and provides and package names... you know, that beauty
15:22:26 <contyk> basically scls, yes
15:22:33 <sgallagh> I don't think we need to define that solution here.
15:22:38 <contyk> we don't
15:22:42 <sgallagh> Merely acknowledge it as being an acceptable option
15:22:43 <bookwar> to be honest, i would push for option 3) anytime someone starts looking into 2), because "parallel installation is not what modules are for"
15:23:06 <asamalik> 1) and 3) are simple, yes... 2) will be tricky to do and not call it scl :P
15:23:06 <contyk> but bundling your deps is a feature and the reason for the API being explicitly defined, so that should be easy to do
15:23:07 <sgallagh> bookwar: Right, I'd discourage 2) for all but the most desperate of cases
15:23:36 <sgallagh> contyk: I agree, but the reality is that without such helpful tools, we really can't do that for the default stream.
15:23:56 <sgallagh> I think we can make a slightly stronger argument about the non-default streams, since the user had to make a conscious choice to enable it
15:23:59 <asamalik> sgallagh: wouldn't 2) be then covered by 1)? basically "act like there is no module"
15:24:09 <sgallagh> asamalik: Yes, it's a special case of 1)
15:24:15 <asamalik> on the RPM level of course
15:24:28 <bookwar> sgallagh: as a concept yes, it makes sense, so we can add it, just reoder the options, so the bad one is a last one, written in finest print possible..
15:24:41 <asamalik> yeah... in that case what about not even mentioning 2)?
15:24:44 <sgallagh> bookwar: +1
15:24:51 <sgallagh> asamalik: Also valid
15:25:03 <asamalik> it's kindof unrelated to modularity and people would blame modularity for it being complicated
15:25:12 <sgallagh> That's a really good point
15:25:33 <bookwar> yes, makes sense
15:25:58 <sgallagh> OK, I'm on board with striking 2) entirely
15:26:29 <bookwar> i think we can agree on this, we should still work on wording, but it is better to do it in a pull request to the docs
15:26:30 <sgallagh> contyk: Okay with you if we don't advertise it (but privately acknowledge it as a weapon of last resort)?
15:26:39 <sgallagh> bookwar++
15:26:40 <zodbot> sgallagh: Karma for bookwar changed to 3 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:26:41 <contyk> I'd rather keep it but whatever :)
15:26:59 <bookwar> the other question would be - how can we validate current default modules and new ones so they follow the policy?
15:27:07 <contyk> if you don't mention it here, the statement will imply that even namespaced packages in default streams are a problem
15:27:14 <asamalik> being correct vs. being effective — 2) is a valid case that will confuse people
15:27:16 <contyk> because there will be just two options
15:27:23 <contyk> everything in the API or don't be the default
15:27:52 <asamalik> yep, which is what we want, no?
15:28:02 <sgallagh> Hang on, I see the point you're making.
15:28:25 <sgallagh> I was thinking of "namespacing == bundling" and not considering "namespacing == renamed RPM"
15:29:21 <sgallagh> So if you namespaced "mymodule-java-plugin" to not own "java-plugin", it would technically still need to be API in our proposal right now
15:29:23 <contyk> yeah, basically the SCL approach
15:29:42 <asamalik> sgallagh: which is ok, no?
15:29:43 <sgallagh> OK, so I have a patch to 2)
15:29:52 <contyk> asamalik: it's not
15:29:56 <sgallagh> Or, really to 1)
15:30:03 <contyk> the point of a private dep is that you don't want to support it for other people
15:30:07 <asamalik> we want default streams to act as packages without a module I thought
15:30:24 <asamalik> to make it simple and compatible
15:30:55 <contyk> that already works; the problems were the points sgallagh was mentioning in the beginning
15:31:10 <sgallagh> 1) Maintain all of the packages in your default stream as if they were non-modular. Exception: if the package is prefixed with "private-modulename-", it is considered to be package internal and must not interfere with the operation of the non-prefixed package.
15:31:22 * asamalik feels like private deps are what makes modularity so confusing, but that's probably just a personal view
15:31:43 <asamalik> anyway, transferring to the subway and losing signal, see ya!
15:31:51 <bookwar> sgallagh: look like we actually creating an official "namespace" concept, which is probably a good thing
15:32:20 <sgallagh> We can bikeshed the prefix later, but does that seem reasonable?
15:32:35 <sgallagh> It'll be clear from the UX that you shouldn't expect that version of the package to work for you.
15:32:57 <contyk> yeah
15:33:02 <bookwar> sgallagh: do you want to propagate the prefix somehow deeper into file structure of a package, like scl does, or just keep it in the name?
15:33:29 <bookwar> should private-.. package have a private-.. binary?
15:33:33 <contyk> it needs to be happen on all levels
15:33:40 <sgallagh> bookwar: I haven't given it enough thought to answer that completely.
15:33:48 <contyk> but that's covered by the "interfere with the operation of the non-prefixed package"
15:33:50 <contyk> I think
15:34:04 <sgallagh> yeah, I tried to make that pretty catch-all
15:34:08 <contyk> basically you need to avoid provides conflicts, and fs paths
15:34:18 <contyk> so pretty much SCLs
15:34:29 * sgallagh nods
15:34:31 <bookwar> contyk: the problem with this solution is that it will require a lot of patches for packages which depend on this one
15:34:40 <contyk> yes, I know
15:34:49 <bookwar> but i guess it is acceptable
15:34:49 <contyk> and I think that needs to be made easier
15:34:57 <sgallagh> bookwar: Or they do the honorable thing and just maintain the package for real instead of privately
15:34:59 <contyk> but we're not solving that problem now
15:35:30 <bookwar> yes, it _is_ a bad option anyway, so if people choose to go that way - they struggle
15:35:43 * sgallagh peers down the road. "Can you still see the can?"
15:35:55 * sgallagh kicks it further
15:36:29 <sgallagh> Yeah, I'd call it a benefit if doing it *the bad way* took extra effort :-D
15:37:17 <contyk> okay, so I think we agree
15:37:22 <contyk> you have those three options
15:37:31 <sgallagh> It's just two still
15:37:43 <sgallagh> 1) Just now has the namespace exception
15:38:07 <contyk> okay, two and a half :)
15:38:08 <sgallagh> And we need to massage the way we document it
15:38:16 <sgallagh> But yes, we seem to be agreeing on how to proceed.
15:38:21 <sgallagh> \o/
15:38:35 <contyk> so do you want to block Ursa Prime on this being done?
15:38:45 <contyk> that is properly formulated and communicated
15:38:49 <contyk> and enforced for existing modules
15:39:03 <sgallagh> Properly formulated and communicated: yes
15:39:11 <sgallagh> enforced for existing modules: no
15:39:24 <sgallagh> (Though I think we may want to push for FESCo to block *F31* for that)
15:39:29 <bookwar> Ursa prime is going to be rawhide only at the beginning?
15:39:57 <contyk> yes; I wouldn't want such a major change in stable
15:40:14 <contyk> F32/Rawhide and onwards
15:40:44 <bookwar> i'd block on collecting the info, how many default modules are violating the policy as of now
15:41:25 <bookwar> and given the info make a final decision, which is most probably be non-block for rawhide
15:41:40 <sgallagh> That seems achievable. contyk ?
15:42:24 <bookwar> i am really worried that there will be smth big depending on java non-api things, and fixing it after we open the buildroot for modules will take a lot of time, mass rebuilds and so on
15:42:46 <contyk> yes, most likely
15:43:21 <contyk> okay
15:43:28 <contyk> would it be you collecting that information?
15:45:48 <contyk> bookwar: ^
15:46:28 <bookwar> honestly, i don't even know how to get the full list of default modules
15:46:40 <sgallagh> It shouldn't be too hard to script; compare the artifacts to the API
15:46:57 <sgallagh> bookwar: Why, what a coincidence: I added that to libmodulemd not long ago :-)
15:46:58 <bookwar> I can try to find this info, but i would be asking you how to do it, anyway
15:47:27 <sgallagh> bookwar: https://fedora-modularity.github.io/libmodulemd/latest/modulemd-2.0-Modulemd.ModuleIndex.html#modulemd-module-index-get-default-streams-as-hash-table
15:47:44 <sgallagh> Just read in the module metadata from F31
15:47:52 <bookwar> contyk: ok, you can put it on me, yes
15:48:11 <contyk> \o/
15:48:17 <sgallagh> Thank you
15:49:05 <contyk> okay; we can work on the wording and communication in the ticket, I think
15:49:14 <sgallagh> bookwar: libmodulemd 2.8.0 also supports reading the compressed YAML, FTR.
15:49:39 <contyk> we can say that bookwar will look into the state of the current default modules and overrides and we'll decide on whether that will block Ursa Prime deployment afterwards
15:50:03 <contyk> ack?
15:50:07 <bookwar> +1
15:50:53 <sgallagh> +1
15:51:34 <contyk> #info Default modules should declare all their exposed packages as API or bundle appropriately; alternatively they must stop being the default; the exact wording will be figured out in the ticket later
15:51:52 <sgallagh> ack
15:52:10 <contyk> #info bookwar will look into the currently default module streams to see how much would need to be changed; we'll decide whether this needs to block Ursa Prime deployment based on the status
15:52:32 <contyk> okay, awesome; in theory tere are mo topics but I'd close it today
15:52:40 <contyk> #topic Next week's chair
15:52:56 <contyk> I'll be out
15:53:29 <sgallagh> contyk: I actually have one thing I'd like to raise quickly: https://github.com/fedora-modularity/libmodulemd/issues/368
15:53:50 <sgallagh> Because it's potentially a blocker for EPEL 8.1 work
15:54:39 <contyk> I see
15:55:17 <contyk> yeah, +1
15:55:24 <contyk> will add a comment
15:55:48 <sgallagh> Thanks
15:55:52 <contyk> I'm going to make langdon run the next one since he volunteered for today originally
15:55:59 <contyk> #action langdon will chair the next meeting
15:56:00 <sgallagh> Sounds good
15:56:03 <contyk> #topic Open floor
15:56:25 <sgallagh> #info https://github.com/fedora-modularity/libmodulemd/issues/368 is a potential blocker for EPEL 8.1
15:57:04 * contyk nods
15:57:14 <contyk> thanks for coming
15:57:17 <contyk> #endmeeting