15:00:00 <nils> #startmeeting modularity
15:00:00 <zodbot> Meeting started Tue Mar 12 15:00:00 2019 UTC.
15:00:00 <zodbot> This meeting is logged and archived in a public location.
15:00:00 <zodbot> The chair is nils. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:00 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:00 <zodbot> The meeting name has been set to 'modularity'
15:00:00 <nils> #meetingtopic Weekly Meeting of the Modularity Team
15:00:00 <nils> #topic Roll Call
15:00:06 <nils> .hello nphilipp
15:00:07 <zodbot> nils: nphilipp 'Nils Philippsen' <nphilipp@redhat.com>
15:01:31 <langdon> .hello2
15:01:32 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
15:01:52 <asamalik> .hello3
15:01:57 <asamalik> lame
15:01:59 <asamalik> .hello2
15:02:00 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
15:03:11 <contyk> .hello psabata
15:03:12 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
15:04:04 <nils> #topic Agenda
15:04:05 <nils> #info #112 Discussion: Module lifecycles
15:04:05 <nils> #info #126 Automatically track refs in a module yaml file
15:04:12 <nils> anything else?
15:04:45 <asamalik> I can't think of anything more now
15:05:00 <nils> ok
15:05:18 <nils> #topic #112 Discussion: Module lifecycles
15:05:18 <nils> #link https://pagure.io/modularity/issue/112
15:05:18 <nils> .modularity 112
15:05:18 <nils> #link https://pagure.io/fesco/issue/2027
15:05:20 <zodbot> nils: Issue #112: Discussion: Module lifecycles - modularity - Pagure.io - https://pagure.io/modularity/issue/112
15:05:23 <nils> #chair asamalik
15:05:23 <zodbot> Current chairs: asamalik nils
15:05:42 <nils> I've noticed some recent discussion in the ticket
15:06:35 <asamalik> yes, it's been updated to make it less focused on implementation like we talked about last time...
15:06:40 <asamalik> FESCo is reviewing it again
15:06:46 <asamalik> should have updates next time!
15:07:14 <nils> yeah and churchyard/sgallagh proposed regularly not setting the EOL, and only do it when an EOL is "in sight"
15:07:25 <nils> to avoid going back and forth with EOL dates
15:07:54 <asamalik> right, the idea is to leave it empty unless you know when you want to kill it
15:09:23 <nils> #info proposed to leave EOL field empty until maintainers know they want to retire a module, and when
15:09:46 <asamalik> but that was there since the beginning
15:09:47 <nils> #info still being reviewed by FESCo
15:10:12 <langdon> shouldn't there be a "minimum notice" then?
15:10:22 <nils> asamalik, AIUI the possibility, but the proposal is to make this the norm, isn't it?
15:10:28 <langdon> like you have to set it at least x months before the eol?
15:11:07 <asamalik> langdon: that's kind of implicit, basically the same as traditional packages are handled...
15:11:21 <asamalik> ... you can only kill it in an upcoming release
15:11:22 <nils> langdon, I think the assumption is, as with packages, that a module will normally be maintained in released versions and only retired in Rawhide (or Branched?)
15:11:54 <langdon> ok.. thats probably find for now.. but as the lifecycles get misaligned its gonna be rougher :)
15:11:58 <asamalik> so the effect will be very similar like retiring a package in rawhide
15:12:33 <nils> I mean, it's not as if we could force maintainers to work on something if they decide to drop a package (or module) at some point, even in stable
15:13:01 <nils> but the expectation is that you commit to maintaining your components for the released stable versions of Fedora
15:15:06 <asamalik> langdon: it shouldn't
15:15:18 <asamalik> you'll have at least the same notice as you do now
15:15:45 <nils> yeah, we've been going with approx. zero guarantees from upstream until now, there's no real change :)
15:15:52 <langdon> ok.. i don't believe it.. but ok :)
15:16:40 <nils> shell we go to the next topic?
15:16:47 <asamalik> langdon: try to think of an example where it's not true... I'm confident you won't find one
15:16:50 <langdon> it has already been beached
15:17:20 <nils> "beached"?
15:17:31 <langdon> my point is more about when / if we start to have less consistent "releases".. like a longer one for some editions or somehting
15:17:37 <langdon> nils: you said "shell"
15:18:07 <nils> oh.
15:18:23 <nils> Well, you're a dad. You're allowed to make dad jokes.
15:18:27 <asamalik> langdon: then we adjust this and other mechanics to enable it
15:18:41 <langdon> asamalik: right.. hence "fine for now" :)
15:18:53 <asamalik> langdon: many things are :)
15:20:32 <nils> ok, move on?
15:20:48 <langdon> =1
15:20:49 <langdon> +1
15:20:55 <asamalik> langdon: BTW it actually already supports the idea of misaligned Fedora releases — Fedora and EPEL — would that make you happier? :)
15:21:21 <asamalik> in other words, it counts with all [currently] existing scenarios
15:21:28 <langdon> asamalik: the "notice time" doesn't if it relies on rawhide..
15:22:43 <asamalik> langdon: but in addition to this, I have an extra card specifically about policies
15:23:08 <asamalik> this proposal is primarily tech-oriented
15:23:48 <asamalik> so yeah if we see that in EPEL for example we need a longer notice or something, we can add it as a policy
15:24:08 <langdon> yeah.. ok
15:24:33 <asamalik> and also one for docs etc...
15:24:49 <asamalik> OK, nils, I think we can get to the next topic
15:24:55 <nils> okie
15:25:06 <nils> #topic #126 Automatically track refs in a module yaml file
15:25:07 <nils> .modularity 126
15:25:07 <nils> #link https://pagure.io/modularity/issue/126
15:25:08 <zodbot> nils: Issue #126: Automatically track refs in a module yaml file - modularity - Pagure.io - https://pagure.io/modularity/issue/126
15:25:25 * sgallagh waves
15:25:36 <sgallagh> I have some background info on this topic
15:25:56 * asamalik waves at sgallagh
15:26:27 <contyk> do share then
15:26:31 <sgallagh> So, this is going to be a situation that hits EPEL (and RHEL internally):
15:27:43 <sgallagh> Essentially, the idea is that if you want to make a bugfix-only release of a module stream, you will probably want to cherry-pick in only the components that need to chaneg.
15:27:45 <sgallagh> *change
15:28:23 <nils> I remember we had cards way back when to create tools that did this
15:28:24 <sgallagh> There are two ways to do this, really:
15:28:39 <asamalik> sgallagh: wouldn't rebuild strategies fix this?
15:28:43 <contyk> asamalik: no
15:28:55 <asamalik> or you wanna rebuild only a specific package even though more have been changed, right?
15:28:55 <sgallagh> asamalik: The concern is that the branches they point at may have unrelated updates.
15:28:59 <nils> i.e. point it at a modulemd and an MBS build and freeze the components to the commit hash refs used in the build
15:29:00 <sgallagh> Right
15:29:07 <sgallagh> nils: Exactly
15:29:08 <asamalik> OK
15:29:17 <sgallagh> And then change only the hashes of the components getting an update.
15:29:37 <contyk> yeah
15:29:48 <sgallagh> The alternative is to create individual branches or tags for all components that match every build... but that's painful and manual and would not delight the packager
15:30:14 <contyk> I personally think that using tags is much nicer than maintaining hashrefs
15:30:22 <contyk> but the reporter doesn't think so
15:30:41 <sgallagh> contyk: Well, they are minimally different in this scenario.
15:31:04 <sgallagh> They'd be more readable, but ultimately you're doing a 1:1 match of them.
15:31:12 <contyk> yes
15:31:16 <asamalik> sgallagh: so they want to do a new stream that's basically a frozen version + just some updates?
15:31:39 <contyk> but it means I can tag the refs I want to use in my module with something nice like "mymodule:mystream" and move the tags as I wish
15:31:46 <contyk> and keep my modulemd unchanged
15:31:46 * asamalik is trying to understand the end-result
15:31:47 <sgallagh> asamalik: Right, and they'd like a tool to just read the data.xmd.mbs section for the hashes that were used in the existing build and have that update the component hashes.
15:31:52 <contyk> I want to fix one component? i move one tag
15:32:03 <sgallagh> contyk: No, incorrect.
15:32:08 <sgallagh> We cannot move tags in dist-git.
15:32:13 <contyk> yes we can
15:32:15 <contyk> tags are mutable
15:32:17 <sgallagh> Both RHEL and Fedora have rules against that for legal reasons.
15:32:41 <contyk> that doesn't make any sense
15:32:53 <sgallagh> I don't know the specific details why
15:33:00 <langdon> how does someone who is not-the-packager know what or how it was built then?
15:33:33 <sgallagh> Now, if we want to talk about having MBS automatically push tags matching a ref to a build into dist-git, I could see that as an option
15:33:44 <sgallagh> Since it wouldn't require packager effort.
15:34:13 <contyk> I wouldn't want MBS touch dist-git
15:34:15 <asamalik> why not just create a new branch?
15:34:22 <sgallagh> Then instead of the requested tool to auto-set the refs, they could just have ref: NSV maybe
15:34:39 <asamalik> branches are basically like tags, just movable with the current policies
15:34:43 <sgallagh> asamalik: If we're creating a new branch for every bugfix release, that's painful.
15:35:05 <langdon> sgallagh: why? or.. better question.. couldn't it be automatic?
15:35:07 <nils> using tags requires considering this while you write the modulemd
15:35:16 <contyk> and anyone could commit to those branches, just like the old ones
15:35:19 <nils> freezing refs could be done after the fact
15:35:46 * asamalik probably doesn't understand what a bugfix release is in this context
15:35:46 <nils> I want to put out a quick fix of only one thing, with the least amount of friction? Freeze stuff and only update one component, then build.
15:36:01 <sgallagh> asamalik: In internal terms, think "Z-stream" or "hotfix"
15:36:08 <nils> ^^
15:36:22 <langdon> is it really a lot more work to do git branch then the change?
15:36:34 <sgallagh> langdon: Yes
15:36:39 <asamalik> sgallagh: ah got it thanks
15:36:48 <nils> branching always involved releng, right?
15:36:56 <sgallagh> langdon: It would be a new branch for every component
15:37:04 <nils> ugh
15:37:09 <contyk> I'd like to note that getting the component hashrefs for a build is absolutely trivial and is one MBS API request
15:37:12 <langdon> i thought you could do your own branches now, right?
15:37:21 <sgallagh> contyk: Sure
15:37:24 <contyk> I just don't think it's the best way to do things
15:37:30 <langdon> sorry.. i meant couldn't you branch the modulemd and put in the git hash for the changed component?
15:37:41 <contyk> and I have doubts about dist-git tags being immutable
15:37:45 <contyk> which I'm going to try right now :)
15:38:50 <sgallagh> contyk: I don't know if there's a tech block on it or just a policy one
15:39:02 <contyk> sgallagh: the policy wouldn't make sense either
15:39:07 <sgallagh> langdon: that's backwards.
15:39:22 <langdon> yeah.. reading the ticket..
15:39:23 <sgallagh> langdon: The risk is that if you don't freeze the things *other* than the one you want to change, you may pick up unrelated fixes
15:39:56 <nils> contyk, I'm not even sure about trivial, queying MBS builds with verbose=true doesn't include the information, e.g. https://mbs.fedoraproject.org/module-build-service/2/module-builds/3305?verbose=true
15:40:03 <nils> Or maybe I'm just short-sighted :)
15:40:35 <asamalik> so I need a tool that takes "I need to rebuild nodejs:8:123456 but only with a fix in the ABC package"
15:40:35 <langdon> yeah.. i guess the reporter's original request would do that well, right? like if we updated the modulemd with the one that was produced from the build back in dist-git.. perhaps on a special "released" branch or something and not overwriting the pkgr maintained one
15:40:43 <asamalik> and builds it
15:40:46 <contyk> nils: yes, you are -- search for ref: :)
15:40:56 <sgallagh> asamalik: Well, outputs a modulemd that can build it
15:41:03 <asamalik> right
15:41:13 <asamalik> and thanks to that I only need a branch for the modulemd
15:41:33 <sgallagh> Right
15:41:38 <contyk> ok, sgallagh is correct, dist-git blocks it
15:41:39 <asamalik> and then I can use that tool again for the next update, giving it the module I'm updating
15:41:39 <nils> contyk, ahh it hides in xmd :)
15:42:18 <asamalik> I think that should be a pretty easy and useful tool if you know what you're doing :)
15:42:28 <sgallagh> nils: Right, that's literally what I said above; they're asking for a tool that reads the hashes from the XMD session and then sticks those hashes into the refs in the components section
15:43:08 * sgallagh could probably write it in python in about a day, but I was thinking it might fit best in fedmod
15:43:40 <sgallagh> Oh, fedmod is python. For some reason I thought it was C.
15:43:49 * sgallagh hasn't actually hacked on that
15:43:51 <langdon> shouldn't it be landing in "pdc" (or in lieu of pdc, dist-git) anyway? as a mark of what we built?
15:44:17 <sgallagh> langdon: Shouldn't what?
15:44:59 <langdon> the modulemd-with-the-refs
15:45:14 <nils> sgallagh, yeah fedmod seems a good git. But as I'll probably be working on Rawhide gating more soon, I don't know if I'll get around to it.
15:45:26 <asamalik> langdon: yes the build system saves that info for every build
15:45:59 <sgallagh> nils: I have a full schedule for the next week at least, but if we want it after that, I can probably do the core logic if you can plug it in.
15:46:10 <contyk> note that any libyaml/libmodulemd-based tool will break formatting and destroy comments
15:46:28 <sgallagh> 🔥
15:46:34 <asamalik> so you just say "Take module:stream:version with the hashes, override with package abc branch efg, and give me the modulemd"... and then you can either build a fix yourself, make a new stream in EPEL if that's needed, or whatever...
15:46:41 <asamalik> all the data is there
15:46:50 * nils remembers a PR to use some other YAML package which left order etc. in place
15:46:59 <nils> I wonder what came out of it. *whistles*
15:47:05 <sgallagh> asamalik: I think we'd probably just have the tool output as-is and expect the user to modify the changed refs manually
15:47:44 <contyk> you could also add a feature to MBS, "only build these components"
15:48:21 <asamalik> sgallagh: true
15:48:28 <contyk> fedpkg module-build --components foo,bar
15:48:47 <nils> how will that work with build order? if you tell it to rebuild a component with a lower buildorder than something else that should be left untouched?
15:49:09 <contyk> depends on the strategy
15:49:46 <contyk> or it could enforce rebuild_strategy=only-changed
15:50:01 <asamalik> contyk: would the old packages be still available?
15:50:09 <asamalik> not in Fedora
15:50:14 <contyk> if there was a previous build, then yes
15:50:30 <contyk> if this is the first build ever then your request doesn't make sense
15:50:34 <contyk> and should be an error
15:50:56 <asamalik> like there is a newer build in updates, where do you get the old one? but we might be getting too deep here...
15:51:00 <asamalik> ah! in Koji...
15:51:05 <asamalik> ok, forget that
15:51:24 <contyk> it's normal component re-use
15:51:30 <nils> if you wanted to reuse previous component builds, the process would have to change substantially, wouldn't it? Preload the build root with not just build dep modules, but also previous builds of the components from previous module builds.
15:52:06 <contyk> I don't think you need to change anything
15:52:12 <contyk> we already have these rebuild strategies
15:52:17 <asamalik> so it looks like we have two options? either the script that gives you a modulemd that you can use anywhere you want... or an MBS enhancement that would just produce a new build but without having the source modulemd
15:52:32 <contyk> only-changed only rebuilds components whose translated refs have changed since the last build
15:52:42 <contyk> with this you would just limit it further to the provided component list
15:53:19 <asamalik> we have 7 mins... what's the next step?
15:53:29 <asamalik> do we have to decide? or did we just brainstorm?
15:53:32 <contyk> I can update the ticket with these options
15:53:42 <contyk> and we can continue next week
15:53:48 <asamalik> contyk: cool!
15:53:48 <nils> sounds good
15:53:50 <contyk> maybe the reporter will like that too
15:53:54 <nils> :)
15:54:12 <asamalik> yeah we should let them decide / help us decide :)
15:54:20 <nils> #action contyk updates the ticket with the available options
15:54:38 <nils> #info revisit this issue next week
15:54:39 <contyk> I could probably write the ref update in perl within 15 minutes; I can even attach it to the ticket ;)
15:54:48 <contyk> but I'd prefer the MBS feature
15:55:00 <asamalik> contyk: these are the best
15:55:08 <asamalik> like my terrible repoquery for all streams
15:56:15 * asamalik needs to leave for another meeting and a short break
15:56:21 <asamalik> thanks all!
15:56:30 <nils> I think we're done for today anyway
15:56:44 <nils> So unless someone brings up something else...?
15:57:29 <nils> Thanks everybody!
15:57:36 <nils> #endmeeting