15:01:19 <langdon> #startmeeting Modularity Team Meeting
15:01:19 <zodbot> Meeting started Thu Dec  5 15:01:19 2019 UTC.
15:01:19 <zodbot> This meeting is logged and archived in a public location.
15:01:19 <zodbot> The chair is langdon. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:19 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:01:19 <zodbot> The meeting name has been set to 'modularity_team_meeting'
15:01:27 <langdon> #meetingname modularity
15:01:27 <zodbot> The meeting name has been set to 'modularity'
15:01:43 <contyk> hey
15:01:51 <Son_Goku> aloha!
15:02:06 <langdon> I am not sure how much it breaks mote if i change the meeting name etc
15:02:15 <Son_Goku> it shouldn't?
15:02:19 <langdon> so ill just leave it.. and ignore that it is weird
15:02:56 <langdon> Son_Goku: on the drop down for https://meetbot.fedoraproject.org/ it very easily ends up with multiple things .. and i am not sure of the logic
15:03:07 <Son_Goku> erk
15:03:20 <langdon> like try typing "mod" and you see you get two.. but they are the "same"
15:03:37 <langdon> .hello2
15:03:38 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
15:03:42 <langdon> #topic roll call
15:03:58 <contyk> .hello psabata
15:03:59 <Son_Goku> .hello ngompa
15:03:59 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
15:04:00 <langdon> #chair Son_Goku contyk ignatenkobrain
15:04:00 <zodbot> Current chairs: Son_Goku contyk ignatenkobrain langdon
15:04:02 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com>
15:04:05 <langdon> ignatenkobrain: you around?
15:05:50 <ignatenkobrain_> .hello ignatenkobrain
15:05:51 <zodbot> ignatenkobrain_: ignatenkobrain 'Igor Gnatenko' <i.gnatenko.brain@gmail.com>
15:05:56 <langdon> i know sgallagh is afk
15:06:02 <langdon> hey igor
15:06:07 <sct> .hello
15:06:07 <zodbot> sct: (hello <an alias, 1 argument>) -- Alias for "hellomynameis $1".
15:06:15 <langdon> #chair sct
15:06:15 <zodbot> Current chairs: Son_Goku contyk ignatenkobrain langdon sct
15:06:19 <Son_Goku> sct if your nick and fas id are the same, use .hello2
15:06:21 <langdon> hi sct
15:06:25 <sct> .hello2
15:06:27 <zodbot> sct: sct 'Stephen Tweedie' <sct@redhat.com>
15:06:35 <Son_Goku> oh hi new person :D
15:06:48 <langdon> Son_Goku: new person?
15:06:56 <Son_Goku> langdon, never met/talked to sct before
15:07:09 <langdon> ohh .. weird.. ok :)
15:07:10 <ignatenkobrain_> I think he was hiding pretty well inside RH :)
15:07:11 <contyk> he's a newbie
15:07:14 <langdon> ha
15:07:26 <langdon> some filesystem guy ;)
15:07:32 <sct> Hah
15:07:37 <sct> Hi all :)
15:07:52 <langdon> ok.. so .. switching topic
15:07:59 <langdon> #topic agenda
15:08:31 <langdon> #info this meeting is to discuss a proposal called "RFC: Modularity Simplified" @ https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/3EP3LXGGZABBCAHF5EPZCVEL4FQ23GS3/#VIX7R6QA6EH4DHHEPGMFEQ5MQ5FF7TBE
15:08:45 <langdon> #topic RFC: Modularity Simplified
15:08:51 <langdon> ok.. so let's get started..
15:09:27 <langdon> unfortunately gallagher couldn't be here and i know he had some thoughts.. but I had already sent all the invites before i knew
15:09:46 <langdon> contyk: do you want to start? ignatenkobrain_ do you?
15:09:56 <ignatenkobrain_> I don't know what to start from :)
15:10:06 <contyk> well, I've just read the whole thread
15:10:15 <contyk> as ignatenkobrain_ says in the original email, this is not new
15:11:13 <contyk> not sure where to start either :)
15:11:18 <sct> Agreed
15:11:30 <contyk> it's another approach of doing things; I don't think it addresses some of the use cases we have
15:11:34 <langdon> well.. my big thing here is .. it is implied that this is "easy" and "well know".. which .. a) if it is, why wasn't it regularly done before b) there are a lot of places where it says "if we just develop this software"
15:11:47 <contyk> I don't find it specifically simpler either
15:11:50 <sct> And I think "Modularity
15:11:51 <sct> suffers greatly from trying to encode everything into one document.
15:11:51 <sct> This greatly raises the complexity of the task and makes it hard to consider alternative proposals for various parts" is really telling
15:11:56 <langdon> sorry s/"well know"/"well known"
15:12:20 <langdon> sct: can you elaborate (or are you typing?)
15:12:32 <sct> Some of the complexity is perceived, rather than real,
15:12:34 <sct> because
15:12:49 <sct> a) it's change to *very* well established rules which people have come to expect
15:13:25 <ignatenkobrain_> langdon, a) because we don't have guidelines adopted for this things and side-tags on demand were implemented with rawhide gating and because some of our tooling is not fully implemented because no one thought the idea entirely b) well, there is no need for a new software, but rather about some changes and patches in *existing* software
15:13:45 <sct> b) there are many related issues: some (like ||el install) are not changed by modularity at all, and we can still use SCLs or compat-libs; but it makes it hard to see what modularity is and what it is not, from an outsider PoV
15:14:16 <langdon> ignatenkobrain_: b) most of modularity is "no new software".. basically just MBS.. which really is just a scheduler
15:14:39 <sct> and c) our documentation and tooling lags on detecting violations of some modularity assumptions, so we get exposed to issues that should not be that complex.
15:15:08 <langdon> to sct's b) i would point out the hatred many people have for the naming of rpms in the SCL world.. and it really should be worse to encode even more information in the names
15:15:26 <ignatenkobrain_> langdon, well, most of modularity is MBS which still does not work reliably (which is actually new software) and also it required hell lot of changes in DNF and after few years it still does not work correctly. And it is really hard to define behavior since everyone has different opinions and ideas.
15:15:35 <sct> There's a second part of c) which is that some of the tooling isn't quite there yet and it means we get to deal with complexity that ought to be better handled (I'm thinking particularly about rolling updates to the latest module default stream for rawhide)
15:16:03 <ignatenkobrain_> contyk, can you elaborate more to "I don't think it addresses some of the use cases we have" ?
15:16:36 <sct> langdon: Agreed; yet it's not always clear to people that modularity doesn't actually make that situation better in all cases.  If you need ||el install, you still need SCLs or something related.
15:16:41 <ignatenkobrain_> langdon, yes, because SCLs are not done in the proper way and it is ton of manual work (incl. dist-git repos and changing names everywhere)
15:17:21 <contyk> ignatenkobrain_: the first two that came to my mind when reading the thread were 1) stream expansion (basically anything that uses multiple levels of modules in a stack, not the platform), and 2) overriding base, non-modular packages with older versions provided by these modules
15:18:43 <langdon> and, to combine some points, "discoverability" was/is a big requirement which scls and other kinds of name mangling really drop the ball on.. how do you know where to find this particular combination of things when it is just encoded (partially/cryptically) in the rpm name
15:18:55 <ignatenkobrain_> contyk, 1) yes, I explicitly pointed this out as "do we really need this?" 2) if you create package with different name, and add conflicts + suggests into the fedora-release (or any alternative), then it will behave exactly as you want it.
15:19:13 <ignatenkobrain_> langdon, can we switch to more specific examples?
15:19:28 <ignatenkobrain_> like a how "python37-foo" is not discoverable
15:19:37 <ignatenkobrain_> or "libgit2_0.28"
15:20:22 <contyk> ignatenkobrain_: re:1, I think we do; providing multiple versions of language stacks, for instance, was something we definitely wanted, at least in RHEL; and you still need the apps built on top of it to work with your selection
15:20:41 <langdon> ignatenkobrain_: well.. a) i think the problem is that if you think of specific examples, you can often find a solution but, the general case is still lost; b) there are a bunch of examples in scls where "maria" has drivers for various versions of php, but which ones?
15:20:47 <contyk> Perl is mentioned and it's a great example as it doesn't support parallel installation and is not binary compatible
15:21:06 <langdon> contyk: i want mult versions of lang stacks in fedora too
15:21:27 <contyk> langdon: me too, but someone needs to maintain them :)
15:21:33 <Son_Goku> contyk, PHP is also a good example of this
15:21:33 <langdon> ha
15:21:47 <Son_Goku> it's even better than Perl, since it doesn't even have versioned paths for PHP extensions
15:22:06 <contyk> Son_Goku++
15:22:14 <ignatenkobrain_> langdon, a) sure, but you need to start by taking explicit items and then make solution more generic b) so why can't it be made php55-mariadb10?
15:22:24 <Son_Goku> and also is the literal bane of my existence at work :(
15:22:30 <langdon> re: php) and different, incompatible "drivers" for web server versions/types, databases, etc
15:22:49 <langdon> ignatenkobrain_: which web server does it support?
15:22:56 <Son_Goku> ignatenkobrain_, the amount of patching that's required to make that work makes it infeasible and not scalable
15:23:04 <langdon> php55-nginxX-mariadb10
15:23:22 <ignatenkobrain_> contyk, so we have multiple python interpreters. there is nothing which would prohibit us building all python modules against all of them generating prefix with py version automatically. fr example openSUSE does that
15:23:28 <langdon> php55-nginxX-fastcgiKK-mariadb10
15:23:33 <ignatenkobrain_> langdon, why do you need nginx and fastcgi in there?
15:23:52 <langdon> because it can (sometimes) conflict with the apache drivers
15:23:54 <ignatenkobrain_> Son_Goku, I'm not experienced with mariadb and the php modules, so I can't really say anything about that. so I am just guessing
15:24:08 <Son_Goku> php modules and mariadb drivers are now kind of annoying
15:24:17 <Son_Goku> because mariadb versions are protocol incompatible now
15:24:20 <langdon> but the point is. it doesn't matter.. you will need to encode a BUNCH of stuff in a field it doesn't belong in
15:24:51 <contyk> ignatenkobrain_: yes, in theory you could invent new macros and add more magic to achieve that
15:24:54 <ignatenkobrain_> how many of such cases exist in this reality?
15:25:06 <Son_Goku> not too many, but enough to be problematic
15:25:15 <langdon> all of the scls have this problem.. (or almost all)
15:25:36 <Son_Goku> PHP, Perl, C/C++, D, and Go are all like this
15:25:46 <Son_Goku> those are just the ones off the top of my head
15:26:21 <Son_Goku> Ruby, Python, Java (and JVM-hosted languages like Kotlin, Scala, etc.) are usually fine
15:26:33 <langdon> the point is that the "ecosystem" needs a way to be "encoded" .. and mangling the name field seems like a bad thing to continue doing
15:26:33 <Son_Goku> they use fully versioned paths throughout the stack by default
15:26:49 <ignatenkobrain_> yeah, but you don't need multiple versions of glibc, while with compiler you can provide multiple versions of gcc which would conflict with each other, but that is not a proble
15:27:07 <langdon> Son_Goku: well.. python has its own problems because it is used so much by linux
15:27:22 <contyk> so the current implementation tries to solve this in a generic way without the need of name mangling and touching spec files as much as possible
15:27:24 <Son_Goku> langdon, yes, but the problem there is so trivially solvable that I don't bother with it
15:27:35 <contyk> the proposal does the exact opposite
15:27:45 <contyk> I know I'm biased but it doesn't feel simpler to me
15:27:57 <langdon> and it is mostly manual work for very experienced people..
15:28:23 <Son_Goku> my major complaint about modularity as it currently stands is that the two level dependency resolution is hacky and brittle
15:29:01 <langdon> i have MANY complaints.. but they are mostly about tooling not being in place / being used to automate 95% of the work
15:29:03 <Son_Goku> and I think I'm more irritated by the fact that it's a feature that's supposed to work and it doesn't than if we didn't have it at all and modules just behaved like subrepos
15:29:11 <contyk> I feel all the complaints boil down to two things: 1) dnf's handling of the content is suboptimal, and 2) the infra tools and processes are unreliable
15:29:22 <langdon> contyk++
15:29:22 <zodbot> langdon: Karma for psabata changed to 1 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:29:30 <Conan_Kudo> contyk++
15:29:30 <zodbot> Conan_Kudo: Karma for psabata changed to 2 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:29:43 <sct> contyk: I'm not 100% sure about that
15:29:56 <sct> contyk: there may be deeper issues w/ rolling updates, context fragility etc
15:29:58 <sct> but
15:29:59 <Son_Goku> contyk, the other thing is that it's *really* hard to reason what is supposed to happen
15:30:20 <Son_Goku> I have a hard time figuring out what's supposed to happen with a given combination of content sometimes
15:30:25 <contyk> sct: there is still some design work ahead, yes; the rolling updates are an example
15:30:48 <Son_Goku> and heck, when I heard langdon's talk at DevConf, I'm pretty sure I discovered two more behavioral bugs as he described the behavior of it, one of which only applies in RHEL
15:30:50 * langdon is still not allowed to say  "i told you so" about having "recommended upgrade path", right? ;)
15:31:06 <sct> it's really hard to figure out if any deeper concerns are really significant or not, precisely because of those content handling and tool/process issues.
15:31:21 <langdon> sct: yeah.. this is the huge issue
15:31:34 <langdon> it is still very hard to tell if it is a "design flaw" or a "bug"
15:31:47 <langdon> where "bug" can also mean "just not implemented yet"
15:31:54 <sct> So for every alternative proposal, my first reaction is instantly to ask "do we really need to throw out what we have, or do we just need to *improve* it?"
15:31:56 <Son_Goku> langdon, I hesitated to say design flaw for one of them, because I'm not sure whether it was intended that way or not
15:32:03 <langdon> thats what I am trying to capture with a new status report thingie
15:32:23 <langdon> Son_Goku: right...
15:32:51 <Son_Goku> langdon, for example, the hotfix flag could actually make it so you transition from older modular locked package to non-modular newer package instead of a hotfix package
15:33:13 <Son_Goku> which is completely unexpected behavior, I think? but is probably by design, based on what I've read
15:33:14 <ignatenkobrain_> sct, I think for very limited amount of use-cases we can have what we have, but the alternate proposal would solve much more of the problems people have today.
15:33:30 <sct> Son_Goku: That's by design.  "hotfix" is just the name to reflect a common intended use case
15:33:42 <langdon> ignatenkobrain_: but you gloss over the problems you would introduce
15:34:06 <Son_Goku> sct, then that means if you have e.g. bash-5.0 and modular bash 4.4, and you introduce hotfix pkg bash-4.4+hotfix1, you go to bash 5.0 again
15:34:10 <sct> but what it is supposed to do is exactly that, flag it as "update to this content (subject to NEVRA) whether you expect these packages to be in a module or not."
15:34:19 <Son_Goku> (I used bash because that was langdon's example at devconf ;) )
15:34:19 <langdon> at its simplest, where do you get all the people to do all the work? i am not sure there are more than 10 people total who can do what you describe.. and they are all busy
15:34:22 <sct> Son_Goku: That's why it's a per-repo flag ---
15:34:31 <Son_Goku> sct, but that's not how dnf treats it
15:34:37 <Son_Goku> that's not how anything treats it
15:34:47 <Son_Goku> it's an unmask property for an name map
15:34:53 <sct> Son_Goku: if you have a bash-4.4 hotfix, stick it in a repo with the hotfix flag, but do *not* set hotfix on the repos containing the main bash.5.0
15:35:05 <ignatenkobrain_> langdon, no, because you are not changing everything cardinally. you improve things which were working for years.
15:35:37 <sct> Son_Goku: That's exactly how dnf treats it.  It disables the module filtering that would otherwise mask out the non-modular content, but otherwise leaves it up to NEVRA to determine the latest version.
15:35:42 <langdon> ignatenkobrain_: how is it an "improvement"? it just is becoming more manual and more opaque... yeah you can do more but it is so much harder..
15:35:54 <Son_Goku> sct, right, but dnf has no vendor locking enabled, so it factors from all sources
15:36:10 <sct> Son_Goku: Right... I'm not sure what the issue is?
15:36:23 <Son_Goku> sct, it's just a really weird case that can cause issues with hotfix packages
15:36:36 <sct> Son_Goku: We don't release hotfix package in mainline repos
15:36:37 <Son_Goku> in practice, I don't think RH would be dumb enough to issue hotfixes in these scenarios
15:36:44 <ignatenkobrain_> langdon, generating modulemd.yaml with 300+ packages in it is not manual and not opaque?
15:37:08 <sct> Son_Goku: Fedora doesn't, 3rd party repos don't... hotfixes are by definition exceptions, which are distributed in an exceptional manner,
15:37:28 <Son_Goku> sct, it's a problem when hotfixes are imported into Pulp or Spacewalk
15:37:32 <sct> so we can set an appropriate flag on the exceptional repos that get set up to carry them.
15:37:53 <Son_Goku> RH Satellite or SUSE Manager (if you want the commercial names)
15:38:00 <langdon> ignatenkobrain_: considering it can be completely generated, automatically, most of the time, yeah.. i don't think so.. and the modulemd actual does a very good job of explaining what you are trying acocomplish vs a spec file.. especially as the spec file will be for some special metapackage that the end user has to find first
15:38:02 <sct> Son_Goku: Now, *that* is definitely true.  If you flatten the content into a single repo, you become responsible for making sure that the updates work right in that context
15:38:21 <sct> Son_Goku: but to me this veers into what we were talking about before about tooling shortcomings ---
15:38:33 <ignatenkobrain_> langdon, the problem is: we have always had a lot of manual things. now we have added yet another complexity level and did not add any automation / simplification into any of these layers.
15:38:35 <sct> I completely agree we need better tooling for maintaining local custom repos with modular content
15:38:45 <Son_Goku> sct, yeah, there's some really odd consequences for this stuff
15:39:09 <sct> Son_Goku: and hotfixes are just a part of that issue; we don't really support custom repos of redistributed module RPMs _at all_, never mind the interactions with hotfixes.
15:39:11 <Son_Goku> I've been bitten by most of them now since I am trying to build stuff without a Koji (since my workplace uses OBS instead)
15:39:12 <langdon> ignatenkobrain_: well.. we meant too :) but the tooling isn't doing its thing correctly/at all/etc because the work isnt done
15:39:20 <sct> So I think that's just a special case of a wider need for tooling improvements.
15:39:20 <ignatenkobrain_> langdon, worse, we have made it so overcomplicated that nobody understands what it is and how it is supposed to behave.
15:39:44 <langdon> ignatenkobrain_: well.. that is a broad, somewhat untrue statement
15:39:44 <ignatenkobrain_> langdon, sure, but it has been *years* and even basic stuff does not work :)
15:40:08 <Son_Goku> langdon, I think the biggest souring point is that we *really* don't have a way to deal with modules disappearing
15:40:10 <langdon> ignatenkobrain_: rpm took a bit of time as well...
15:40:25 <langdon> ignatenkobrain_: but it is fair
15:40:27 <Son_Goku> and we don't have *anything* for dealing with modular<->non-modular transitions
15:40:31 <ignatenkobrain_> langdon, show me person who can describe what tooling is supposed to do in all cases (compose, dnf, MBS+koji), how upgrades would work, siwtching and so on
15:40:42 <langdon> we have not had anywhere near the resources we expected to have to point at the problem
15:40:48 <ignatenkobrain_> there is simply no document / person describing all possible behaviors
15:40:55 <langdon> ignatenkobrain_: me? contyk? sct? sgallagh?
15:41:13 <ignatenkobrain_> I am pretty sure there are tens (or hundreds) of them, but either they are not public or not complete
15:41:20 <langdon> ignatenkobrain_: it is too complex for 1 document.. have you actually read all of the docs on the modularity docs site
15:41:26 <sct> Several of us have the mental map, but I totally agree it's not easy to navigate
15:41:26 <langdon> ?
15:41:27 <sct> but
15:41:34 <sct> I don't think modularity is necessarily that complex
15:41:40 <sct> I think *the problem space* is complex
15:41:55 <contyk> +1
15:41:58 <sct> and it is easy to get confused trying to map out all the issues that modularity addresses, vs. the ones that it does not, and how it all fits
15:42:00 <langdon> sct: +1
15:42:00 <sct> together.
15:42:14 <langdon> and how we rely on existing solutions so much
15:42:20 <sct> And rpm is *really* not straightforward either if you look at some of the corner cases we have encountered over the years
15:42:33 <Son_Goku> well, modularity-the-system is at the core subrepos with dependency information and built-in priorities
15:42:40 <sct> but you don't need to know every possible corner case to have a solid understanding of the simple principles at work.
15:42:42 <langdon> e.g. modularity doesn't solve compat libs.. but people think it does.. and so they get stuck trying to figure out how it does compat libs
15:43:03 <sct> langdon: *exactly.*  It's mapping out the various interacting issues and technologies which gets complex.
15:43:05 <langdon> sct: nor is dnf/yum.. with lots of straight up *magic*
15:43:11 <Son_Goku> langdon, well, that was largely caused by people thinking we can do weird things like stuff desktop environments as modules
15:43:12 <contyk> Son_Goku: + the build side of things
15:43:16 <sct> Yep.
15:44:02 <ignatenkobrain_> langdon, sure. though it is not written anywhere what problems it is supposed to solve :)
15:44:08 <Son_Goku> contyk, langdon: I don't think the modulemd document is all that bad as a module input control
15:44:11 <langdon> ignatenkobrain_: have you read the docs?
15:44:18 <ignatenkobrain_> sure?
15:44:19 <langdon> i think they do a very nice job of that
15:44:48 <contyk> re docs: we recently reviewed those parts and figured all of that is covered
15:44:57 <Son_Goku> the stuff on https://docs.fedoraproject.org/en-US/modularity/ is a lot more fleshed out than it was four months ago
15:45:08 <langdon> yeah.. everytime we get a q we go look at the docs and we see the answer
15:45:21 <langdon> even though we forgot it was there 1/2 the time
15:45:44 <Son_Goku> I think it's disingenuous to say containers are a useful solution to have parallel installable modules
15:45:52 <langdon> ignatenkobrain_: for problems being solved: https://docs.fedoraproject.org/en-US/modularity/faq/
15:46:01 <Son_Goku> it's totally possible to have parallel installable modules, you just have to have RPMs that are parallel installable
15:46:08 <zbyszek> langdon: I don't think so. If you consider that modularity is not useful for libraries, and only for leaf applications, the docs at https://docs.fedoraproject.org/en-US/modularity/ are at least very misleading.
15:46:18 <langdon> Son_Goku: i kind of agree.. but, its also kind of.. don't bite off the whole world
15:46:36 <ignatenkobrain_> langdon, so I can put libgit2 back to a module?
15:46:39 <langdon> zbyszek: but you think that.. the modularity team doesn't nec. think that?
15:46:45 <Son_Goku> langdon, sure, but I think you shouldn't be mentioning containers as a parallel installability solution...
15:46:55 <langdon> Son_Goku: why? it works
15:47:00 <contyk> Son_Goku: but enabling parallel installation makes the problem even harder (build, runtime, test, all that)
15:47:09 <langdon> it may not cover every use case but it does work
15:47:10 <zbyszek> Let me rephrase that: the docs *imply* much more than is currently possible.
15:47:13 <sct> Son_Goku: Sort-of... we can ||-install modules, but we cannot ||-el install multiple *streams* of a single module.
15:47:15 <contyk> Son_Goku: that was one of the reasons we decided not to pursue that
15:47:30 <ignatenkobrain_> sct, what is || btw?
15:47:34 <sct> Sorry
15:47:35 <sct> old habit
15:47:38 <sct> parellel
15:47:58 <Son_Goku> sct, hah, that just kept making me think of OR-install, and I had no idea what you were talking about :)
15:48:15 <Son_Goku> contyk, I know, but it's possible to do so, as the python modules show, but it's confusing because it doesn't answer how to make parallel installable modules
15:48:21 * langdon wants to see the symbol for perpendicular!
15:48:37 <Son_Goku> langdon ⊥installable :P
15:48:38 <sct> |_ ?
15:48:47 <langdon> Son_Goku: we didn't want to open that can of worms until more of the pain points were resolved
15:48:51 <langdon> and lol
15:49:23 <Son_Goku> langdon, that's fair, but it's annoying because Fedora has not that many containers mapped to modules
15:49:37 <Son_Goku> you describe solutions you don't actually *have*
15:49:54 <Son_Goku> in the RHEL case, you have them, because all those modules *do* have containers for the whole OpenShift-y thing
15:49:56 <langdon> Son_Goku: yeah.. like so many things .. that work got stalled.. it was nicely getting generated in a nice prototype.. that just got abandoned
15:50:17 * Son_Goku still wants containers to get auto-rebuilt when repos change
15:50:30 <contyk> freshmaker :)
15:50:32 <langdon> Son_Goku: +1 .. but i would like all the output artifacts
15:50:42 <Son_Goku> langdon, yeah, I've stopped hoping for that :(
15:50:45 <langdon> contyk: its like we thought of a lot of these problems
15:50:53 <Son_Goku> I've been harping on that idea for almost four years now
15:51:04 <ignatenkobrain_> langdon, you are implying you have generic solution, but then we end up in a situation that you actually can't use modules for some of the things. so it is not generic anymore and does not really solve the problem (for those people who wanted to take an advantage of the technology)
15:51:10 <langdon> so.. one more thing Son_Goku.. i was really "hurt" by "Fortunately, there *are* some links among the distributions on the community side. But the modularity project started from within Red Hat, and I suspect there wasn't any talking to SUSE in the early stages of the development. There wasn't much community solicitation to Fedora in the beginning either, as we know..."
15:51:11 <Son_Goku> nobody cares... and my prototypes seem to generate no interest
15:51:16 <langdon> what made you say that?
15:51:42 <langdon> ignatenkobrain_: why do you say that? we do have a generic solution.. unless i am missing your point?
15:51:58 <Son_Goku> langdon, years of attempts to try to get involved in Factory 2.0 / Modularity, trying to do improvements in releng, trying to get help understanding modularity in the early efforts...
15:52:09 <langdon> Son_Goku: we talked to suse folks and we have been talking in fedora for *literal* years
15:52:10 <Son_Goku> I was rebuffed so many times
15:52:32 <langdon> Son_Goku: sorry.. my "what made you say that?" was to my prior
15:52:52 <Son_Goku> langdon, when I talked to the suse folks (OBS team, rpm team, sys management team), they were taken completely by surprise by the modularity stuff we were doing
15:52:58 <Son_Goku> that is why I suspected nobody talked to them
15:53:11 <langdon> Son_Goku: always a question of "who" :)
15:53:15 <langdon> and it was years ago
15:53:19 <ignatenkobrain_> langdon, except that I was pushed back with libgit2 thing, the conflicting modules are a problem and the tooling behavior is not defined.
15:53:26 <langdon> so.. probably some different people as well
15:54:07 <Son_Goku> langdon, I can give you names offline, if you want to know
15:54:08 <langdon> ignatenkobrain_: i think that particular issue was the massive arch change to not do a platform module.. we missed some corner cases
15:54:24 <langdon> Son_Goku: we should probbably have those conversations, yes
15:54:32 <Son_Goku> langdon, I think we need a way to support orphan modules
15:54:46 <langdon> so.. we don't do libs anywhere near as well as we are "supposed" to
15:54:55 <Son_Goku> tying modules to the platform at runtime causes far too many problems for upgrades
15:55:13 <langdon> Son_Goku: orphan = they have gone out of support for whatever reason?
15:55:14 <contyk> that libgit2 thing should be possible once we have the rolling updates and stream migration in place
15:55:25 <contyk> meaning the generic solution still works, just not right now
15:55:32 <Son_Goku> langdon, sure, or distro upgrade where module no longer exists
15:55:34 <langdon> i think this is in rpkg already but isn't plumbed through to fedpkg
15:55:51 <langdon> Son_Goku: ohh... two diff things.. sorry
15:55:58 <Son_Goku> yeah, no prob
15:56:17 <langdon> yes.. "upgrades" need to get finished.. in lots of ways..
15:56:26 <Son_Goku> in The Big Thread on Modularity(TM), I described in detail the upgrade and orphan/obsoletes cases that are missing
15:57:28 <langdon> let me ask this...  how can we communicate this stuff better?
15:58:28 <langdon> i honestly still think the vast majority of the problem is the feeling that people think we "missed stuff" or that are reqs are bad or whateever.. but rarely do we discover actual problems that aren't in the pipeline already
15:58:49 <langdon> s/that are reqs / that our reqs
15:59:05 <Son_Goku> langdon, so you guys are doing a better job, but I think in terms of communication, it would really help if we got some tutorials, walkthroughs, etc. on build and user side workflows
15:59:24 <langdon> i do not understand why my fingers swap "our" and "are" all the time
15:59:42 <Son_Goku> the RH Summit lab helped out my colleagues quite a bit in understanding the RHEL modules stuff, so something like that for Fedora too
15:59:51 <langdon> Son_Goku: the one i did?
15:59:53 <Son_Goku> yes
15:59:59 <langdon> well.. with lots of help ;)
16:00:15 <langdon> we do have a fedora-classroom asamalik did
16:00:17 <Son_Goku> the one I intentionally didn't go to because you were afraid I'd heckle you :)
16:00:28 <langdon> hahahahha
16:00:39 * tachoknight_ raises his hand to address communicating the stuff better (i can discuss later if now is not the right time)
16:00:45 <langdon> ignatenkobrain_: what do you think? do you really think we can't just keep improving this?
16:01:15 <langdon> tachoknight_: now is fine with me (or in #fedora-modularity if you like)
16:01:27 <Son_Goku> langdon, the other thing that I think hurt you guys more than anything else was that Java thing
16:01:31 <Son_Goku> nobody was ready for that
16:01:33 <tachoknight_> okay, i'll bring it up there
16:01:39 <Son_Goku> and that really hurt the perception of modularity
16:01:40 <zbyszek> langdon: I think it is fine to keep working on this, but in a way that does not break normal packaging
16:01:55 <langdon> Son_Goku: yeah.. the missing "modules in the build root" problem was a *monster*
16:02:22 <Son_Goku> langdon, I'm still not sure whether Ursa Prime is actually the right solution for this, because I don't think we have a solution for buildroot overrides in the modular for non-modular build thing
16:02:24 <zbyszek> I.e. things must remain optional as long as they are in preview mode, not the other way around.
16:02:38 <contyk> Son_Goku: we do
16:03:14 <ignatenkobrain_> langdon, personally I want to see a) commitment from DNF team on resolving modularity issues (and also some future plan how to integrate with libsolv), b) explicit design document which describes how dependency solving should work on upgrades, switching streams and all possible situations and tooling behavior c) commitment from people who work on MBS/Koji to implement features (I am not sure if my 1+ yr old buildonly and buildafter
16:03:15 <ignatenkobrain_> features were implemented in MBS
16:03:15 <langdon> on that problem.. we have, more than once, suffered that team-a is doing A and team-b is doing B and B lands and A gets delayed/blocked/etc .. and B breaks.. but we (modularity team) can't always stop B
16:03:16 <contyk> Son_Goku: the koji repo preserves the modular metadata and has its own generated defaults, so dnf just handles that
16:04:01 <Son_Goku> contyk, yeah, but external repos are composed by bodhi
16:04:17 <Son_Goku> buildroot overrides typically work through koji tag manipulation
16:04:26 <langdon> contyk: we have ignatenkobrain_'s "b" somewhere, right? or is it not done yet?
16:04:56 <contyk> Son_Goku: ah, right; that's not yet done at all
16:05:07 <contyk> langdon: no, that's still an open thing
16:05:47 <contyk> I'll have to drop
16:06:08 <langdon> yeah.. ok.. we are over time.. i was trying to figure out how to "close the meeting" ...
16:06:40 <langdon> maybe i #info zbyszek's ignatenkobrain_'s and Son_Goku's final (ish) items?
16:06:42 <Son_Goku> langdon, found the message about upgrades and obsoletes: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/VNAXNXBEQWGQQ3TQVOKO3QNMBW4GPVYY/
16:07:08 <langdon> ohh yeah
16:07:46 <langdon> i was gonna do the #info without attribution though.. just like "these are the things we think should be done"
16:08:26 <langdon> #info we should keep working on modularity but we need to be more diligent about not breaking existing/normal packaging
16:08:46 <Son_Goku> I think that, along with some of the improvements to the build-side tooling to make modulemds less awful to maintain (like what I was talking about at Tuesday's meeting), would go a _very_ long way to make this a much nicer experience
16:08:55 <sct> langdon: Thanks for wrapping up --- I need to drop now
16:09:05 <Son_Goku> langdon, I'm also increasingly coming to the mind that we should just not have BR-only content in Fedora
16:09:06 <langdon> thanks sct!
16:09:09 <langdon> thanks all!
16:09:16 * langdon still typing infos
16:09:35 <langdon> Son_Goku: i think that is a policy thing thugh not a tech thing
16:09:45 <langdon> then we can switch it when we decided the other way
16:09:55 <langdon> or back and forth
16:10:11 <Son_Goku> we need to have that laid down as a ground rule for modules going forward
16:10:14 <langdon> i also think the "everything ref repo" would help aleviate (sp?) that
16:10:18 <Son_Goku> maybe
16:10:26 <Son_Goku> depends on how usable it is for people to rebuild content
16:10:36 <Son_Goku> similar arguments are happening in CentOS-land too
16:11:10 <langdon> please make comments here: https://pagure.io/modularity/issue/166
16:11:46 <Son_Goku> sure
16:12:19 <langdon> #info if the modularity bits are optional, they should be actually optional
16:12:21 <langdon> #info if they are not production ready, they should be optional
16:12:40 <langdon> #info  commitment from DNF team on resolving modularity issues (and also some future plan how to integrate with libsolv)
16:12:51 <langdon> #info explicit design document which describes how dependency solving should work on upgrades, switching streams and all possible situations and tooling behavior
16:13:09 <langdon> #info commitment from people who work on MBS/Koji to implement features (I am not sure if my 1+ yr old buildonly and buildafter
16:13:16 <langdon> #undo
16:13:16 <zodbot> Removing item from minutes: INFO by langdon at 16:13:09 : commitment from people who work on MBS/Koji to implement features (I am not sure if my 1+ yr old buildonly and buildafter
16:13:24 <langdon> #info commitment from people who work on MBS/Koji to implement features
16:14:07 <langdon> #info upgrades and obsoletes need better documentation and implementation
16:15:13 <langdon> #info past ~6 infos are meant to try to indicate some consensus on the group's desire for changes in modularity
16:15:33 <langdon> ok.. i am gonna end meeting now.. hopefully that captures what we wanted captured..
16:15:37 <langdon> #endmeeting