14:00:00 <nils> #startmeeting modularity_wg
14:00:01 <zodbot> Meeting started Tue Apr 17 14:00:00 2018 UTC.  The chair is nils. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:01 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
14:00:01 <zodbot> The meeting name has been set to 'modularity_wg'
14:00:01 <nils> #meetingtopic Meeting of the Modularity Working Group (once every two weeks)
14:00:01 <nils> #chair dgilmore langdon tflink
14:00:01 <zodbot> Current chairs: dgilmore langdon nils tflink
14:00:09 <nils> #topic Roll Call
14:00:18 <nils> .hello nphilipp
14:00:19 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
14:00:20 <langdon> .hello2
14:00:22 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
14:00:31 <dgilmore> hi
14:00:51 <langdon> hi!
14:00:58 <nils> hi :)
14:01:05 <asamalik> .hello2
14:01:06 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
14:01:29 <asamalik> hey there
14:01:55 <nils> The agenda is still empty, do you have anything to add?
14:02:10 <dgilmore> I have nothing
14:02:44 <nils> langdon, asamalik?
14:02:49 <langdon> me neither..
14:03:12 <langdon> except i used the nodejs:6 module to get an app to run that wouldn't work on f27 with nodejs:8!
14:03:42 <nils> \o/
14:03:49 <contyk> o/
14:03:52 <contyk> .hello psabata
14:03:53 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
14:04:43 <asamalik> nothing here
14:04:55 <nils> contyk, sct, anything to add to the agenda?
14:05:12 <tflink> .hello tflink
14:05:13 <zodbot> tflink: tflink 'Tim Flink' <tflink@redhat.com>
14:05:22 <nils> or tflink? :D
14:05:52 <contyk> well, nothing I could really talk about
14:06:09 <contyk> but we've hit quite a few issues regarding compose metadata and dnf lately
14:06:18 <contyk> I believe all of them are tracked in bugzilla
14:06:22 <sct> nils: Don't think so.  There's a bunch of relevant work going on right now in dnf and on module defaults, but it's all still in the pipe and I don't think is blocked on anything at this point
14:07:00 <nils> Alright, if there's nothing to discuss you can have almost an hour of your life back. :)
14:07:55 <langdon> we need some blog posts!
14:08:05 <contyk> with cat pictures
14:08:18 <langdon> yes.. cats
14:09:09 <nils> I'm tempted to link pictures of lions eating.
14:09:13 <jkaluza> I actually have something to discuss :)
14:09:21 <nils> a ha!
14:09:22 <jkaluza> I want to move with https://pagure.io/fm-orchestrator/pull-request/917
14:09:25 <nils> #topic Agenda
14:09:50 <jkaluza> it's mainly thing for contyk probably, but since there is meeting he must attend and we have an hour to discuss that, we could fix this issue :)
14:10:02 <contyk> :))
14:10:03 <nils> #info [jkaluza] discuss reusing module build strategy
14:10:18 <nils> #topic discuss reusing module build strategy
14:10:23 <nils> #chair jkaluza
14:10:23 <zodbot> Current chairs: dgilmore jkaluza langdon nils tflink
14:10:23 <contyk> so you currently reuse components from other modules and/or streams and you think it's a problem?
14:10:30 <contyk> it's a feature we wanted :)
14:11:02 <nils> #link https://pagure.io/fm-orchestrator/pull-request/917
14:11:13 <jkaluza> Right now the default module rebuild strategy is "only-changed"
14:11:33 <jkaluza> Which is described as "All changed components will be rebuilt".
14:12:29 <jkaluza> The issue we have once we introduced module stream expansion (MSE) is that module build which buildrequires platform:f29 could reuse components from previous module build buildrequiring platform:f28
14:12:56 <contyk> well, yes, that sounds wrong
14:13:01 <langdon> contyk: +1
14:13:16 <contyk> I'm all in favor of reusing components from other streams or even modules
14:13:18 <jkaluza> I have hotfixed that in MBS prod by checking the commit hashes of all buildrequired modules and only if they are the same, I nominated this previous module as candidate to reuse components from
14:13:25 <contyk> but only from builds that share the same buildroot on stream level
14:13:44 <jkaluza> but mprahl thinks this is wrong for "only-changed" and it makes it the same as "changed-and-after".
14:14:07 <jkaluza> and contyk thinks (at least I think) that we should check the "stream" instead of "commit hash" when rebuild strategy is "only-change".
14:14:14 <contyk> yes
14:14:16 <dgilmore> contyk: well if a module had a buildrequires platform:f28 it should be expected to run on platform:29
14:14:30 <contyk> because if you translate the streams to commit IDs, what mprahl says is true in practice
14:14:43 <contyk> the odds that you will find a match are very low and you will always rebuild everything
14:15:10 <contyk> if you match on stream names and trust that they maintain some ABI stability, you will re-use most of your components
14:15:13 <dgilmore> but you do want to make sure that a module with a buildrequires platform:f29 is not a dependency for a module that has buildrequires platform:f28
14:15:24 <jkaluza> The goal of this discussion should be decision if we should keep the current rebuild-strategy (only-changed) and fix it somehow (how?) or move to different strategy.
14:15:24 <dgilmore> one way should be fine
14:15:54 <contyk> dgilmore: that's up to what the module author defines; they can say they want to build once and run everywhere; they can say they wan to be built for every platform
14:16:01 <dgilmore> using older platform modules on newer platforms should be okay, but using newer platform on older platforms should not
14:16:08 <nils> dgilmore, but that's an exception for platform, or isn't it? Normally, different streams aren't guaranteed to be compatible.
14:16:10 <contyk> dgilmore: we can't guarantee 28 and 29 abi compatibility
14:16:27 <contyk> it doesn't matter which way
14:16:32 <dgilmore> contyk: no. but should we outright block it?
14:16:38 <jkaluza> contyk: I'm also OK with changing my PR linked by nils to use stream instead of refs and continue using "only-changed", I only need +1 from you on that.
14:16:51 <dgilmore> the module owners should make sure that if there is not compatability they rebuild
14:16:53 <contyk> dgilmore: I don't see what's... blocking it?
14:16:59 <jkaluza> contyk: or from modularity-wg basically, but your +1 is OK for me :)
14:17:03 <dgilmore> maybe I am not expressing myself correctly
14:17:25 <mprahl> My suggestion is when using the `only-changed` rebuild strategy is we keep it as is but only allow reusing from module components built against the same platform.
14:17:42 <contyk> mprahl: +1
14:17:47 <jkaluza> ^ Hm, that's different thing that I was proposing
14:17:54 <asamalik> is dgilmore here? am I loosing messages again? can't see your messages dgilmore :(  (connected with Matrix)
14:17:54 <dgilmore> mprahl: I think the same or older is okay
14:17:55 <contyk> mprahl: which you would achieve by checking streams, not refs; do you see a problem there?
14:17:56 <mprahl> This would make it work like it did before f29 was introduced
14:18:06 <dgilmore> asamalik: I am here
14:18:07 <jkaluza> asamalik: he is here
14:18:13 <dgilmore> asamalik: you have me on ignore I guess
14:18:14 <nils> mprahl, same platform, or same "module build deps"?
14:18:20 <mprahl> nils: same platform
14:18:40 <jkaluza> mprahl: can "platform" be generalized as "base module"?
14:18:42 <nils> Because the compat considerations are the same w.r.t. different streams of other modules on the same platform, or are they not?
14:18:48 <jkaluza> mprahl: as we maintain list of base modules in MBS config
14:18:49 <contyk> they are
14:19:10 <mprahl> contyk: `only-changed` currently doesn't check build deps, it just checks if the commit hash of the component is the same. It's very liberal.
14:19:30 <mprahl> contyk: We would just add an additional check of making sure platform is the same stream from both components
14:19:31 <contyk> mprahl: it needs to check build deps but it should only do so on stream level
14:19:45 <jkaluza> mprahl: contyk: *currently == in the MBS master git repo branch
14:19:54 <jkaluza> only-changed, as deployed, checks refs
14:19:58 <jkaluza> since I hotfixed that
14:19:59 <nils> contyk++
14:20:00 <zodbot> nils: Karma for psabata changed to 3 (for the f27 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
14:20:20 <asamalik_> .hello asamalik
14:20:21 <zodbot> asamalik_: asamalik 'Adam Samalik' <asamalik@redhat.com>
14:20:24 <mprahl> contyk: Okay, do you want the context to be computed based on streams as well? It's currently based on commit hashes of deps.
14:20:30 <contyk> if you reuse components built against a different ABI, you create broken modules
14:20:34 <dgilmore> hi asamalik_
14:20:43 <asamalik_> hey dgilmore! finally see you
14:20:43 <jkaluza> Wait, can we decide that we want to reuse based on "branch"?
14:20:50 <jkaluza> Did we decide that now?
14:20:54 <contyk> if you base your reuse policy on git refs, you will almost always rebuilt everything because it's effectively changed-and-after
14:20:56 <jkaluza> *based on "stream"
14:20:57 <dgilmore> asamalik_: I assumed you were ignoring me :D
14:21:27 <asamalik_> dgilmore: I realized you might recognize it, so had to fake a reason :P
14:21:56 <contyk> jkaluza: well, keep those two facts in mind and tell me if there's anything wrong with tracking build deps by streams?
14:21:58 <mprahl> contyk: Yes but this is a different question.
14:22:16 <dgilmore> asamalik_: I was saying that I think if the module you buildrequire used an older platform it should be fine
14:22:30 <dgilmore> there will be cases you need to rebuild it against a newer platform
14:22:31 <mprahl> contyk: Currently, the context is based on build deps + runtime deps and their commit hashes. Instead of commit hashes, should that be streams? Then the context is more predictable
14:22:55 <jkaluza> contyk: it shouldn't be if streams thing built against older version of a module in the same stream will always work in the later version. And I think it should, so it should be OK.
14:23:26 <jkaluza> mprahl: That would mean the context will always be the same unless you really change the stream names. I think it would be nice.
14:23:39 <contyk> jkaluza: that's one thing we need to rely on
14:23:56 <asamalik_> dgilmore: thanks for repeating that for me!
14:24:05 <mprahl> jkaluza: Exactly. It seems people want to be able to do that.
14:24:15 <jkaluza> mprahl: So all the modules which buildrequire *only* platform:f29 would have the same context forever.
14:24:26 <mprahl> jkaluza: Yes exactly
14:24:27 <jkaluza> which might be useful
14:24:29 <contyk> mprahl: well, as long as changing that doesn't break your other policies...
14:24:41 <mprahl> contyk: Nope, should be fine.
14:24:59 <jkaluza> mprahl: anything else around this issue we want to ask/decide?
14:25:01 <contyk> with changed-and-after I want changes in refs to affect the rebuild decision
14:25:21 <mprahl> contyk: Yes, that would stay the same.
14:25:37 <mprahl> jkaluza: I think I'm all set. :)
14:26:43 <nils> Anything that needs to be #info'd?
14:27:06 <jkaluza> Factory-2.0 will change the "only-changed" MBS rebuild-strategy to reuse components from previous build of a module which buildrequires the same modules in the same streams as current module.
14:27:17 <jkaluza> contyk: mprahl: ^ can this be summarized like that?
14:27:36 <asamalik_> jkaluza: that sounds good to me!
14:27:49 <mprahl> jkaluza: Yes, but additionally, the "context" will now be computed from the buildrequires and requires and their streams.
14:27:53 <jkaluza> Factory-2.0 will compute the context hash based on the stream of buildrequires/requires instead of commit hash of buildrequires/requires.
14:27:57 <jkaluza> mprahl: ^
14:27:59 <mprahl> +1
14:28:17 <jkaluza> well, *will change the MBS to compute
14:28:18 <contyk> +1
14:28:20 <jkaluza> I won't compute it myself
14:28:29 <mprahl> lol
14:28:35 <contyk> jkaluza-as-a-service
14:28:37 <nils> slacker
14:28:38 <jkaluza> nils: that can be #info'd I think :)
14:28:41 <nils> :)
14:28:42 <jkaluza> those two
14:28:45 <jkaluza> and I'm happy :) thanks :)
14:29:03 <nils> #info Factory-2.0 will change the "only-changed" MBS rebuild-strategy to reuse components from previous build of a module which buildrequires the same modules in the same streams as current module.
14:29:04 <nils> Factory-2.0 will change the "only-changed" MBS rebuild-strategy to reuse components from previous build of a module which buildrequires the same modules in the same streams as current module.
14:29:05 <mprahl> happiness++
14:29:09 <nils> oops
14:29:22 * jkaluza does not have to keep that hotfix in his mind and can redo that PR and finally merge it
14:29:31 <nils> #info Factory-2.0 will change MBS to compute the context hash based on the stream of buildrequires/requires instead of commit hash of buildrequires/requires.
14:30:09 <asamalik_> Mprahl gave karma cookie in f28 to happiness (43)!
14:30:26 <jkaluza> contyk: One more thing, it is not blocker for you for F28 anyhow I presume, right?
14:30:37 <jkaluza> contyk: given we are in freeze again, we won't be able to deploy that right now
14:30:56 <jkaluza> so it will continue reusing only from modules which have the same commit hashes of all buildrequires
14:31:01 <jkaluza> until the freeze is over
14:31:10 <contyk> at this point it shouldn't have any catastrophic consequences because f28 and f29 aren't all that different
14:31:19 <jkaluza> good
14:31:23 <jkaluza> I thought so :)
14:31:25 <contyk> but the sooner you can fix this the better
14:31:43 * jkaluza checks when the freeze is over
14:32:30 <contyk> May, probably
14:32:32 <contyk> after GA
14:32:37 <jkaluza> yeah
14:32:46 <jkaluza> 2018-05-01 or  2018-05-08 or some dates around
14:33:03 <contyk> yep
14:33:03 <jkaluza> contyk: let's see, we might also have some important fix and get this in as well
14:33:12 <jkaluza> I don't believe we won't break the freeze with MSE :)
14:36:16 <contyk> so, are we done? :)
14:37:16 * jkaluza has no more questions
14:39:56 <nils> good
14:40:37 <nils> Thanks everybody! Thanks jkaluza for giving us something to talk about. :D
14:40:40 <nils> jkaluza++
14:40:40 <zodbot> nils: Karma for jkaluza changed to 8 (for the f27 release cycle):  https://badges.fedoraproject.org/tags/cookie/any
14:40:45 <nils> #endmeeting