15:02:49 <langdon> #startmeeting Modularity Team Meeting
15:02:49 <zodbot> Meeting started Tue Dec  3 15:02:49 2019 UTC.
15:02:49 <zodbot> This meeting is logged and archived in a public location.
15:02:49 <zodbot> The chair is langdon. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:49 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:02:49 <zodbot> The meeting name has been set to 'modularity_team_meeting'
15:02:49 <zodbot> contyk: In #fedora-meeting-3 is Modularity Team (weekly) (starting in 13 days)
15:02:49 <contyk> see
15:02:51 <zodbot> contyk: In #fedora-meeting-3 is Modularity Team (weekly) (starting in 20 days)
15:02:54 <zodbot> contyk: - https://apps.fedoraproject.org/calendar/location/fedora-meeting-3%40irc.freenode.net/
15:03:04 <langdon> it doesnt tell you if the meeting should have started already
15:03:16 <langdon> #meetingname modularity
15:03:16 <zodbot> The meeting name has been set to 'modularity'
15:03:23 <langdon> #topic Roll Call
15:03:25 <contyk> .hello psabata
15:03:26 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
15:03:26 <langdon> .heelo2
15:03:29 <Son_Goku> .hello ngompa
15:03:29 <langdon> .hello2
15:03:29 <zodbot> Son_Goku: ngompa 'Neal Gompa' <ngompa13@gmail.com>
15:03:32 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
15:03:54 <Son_Goku> been a while since I've last been to one of these...
15:04:34 <contyk> you haven't missed much :)
15:04:39 <langdon> ha
15:04:43 <Son_Goku> welp
15:04:48 <langdon> #chair contyk sgallagh
15:04:48 <zodbot> Current chairs: contyk langdon sgallagh
15:04:59 <langdon> #topic agenda setting
15:05:05 <langdon> proposals?
15:05:24 <contyk> we wanted to go through the ticket queue, finally
15:05:33 <contyk> we did some cleanup a couple of weeks ago
15:05:39 <contyk> but there are some new topics
15:05:43 <contyk> or tickets
15:05:48 <langdon> right
15:06:06 <langdon> we also have the proposal floating in email. that I am not sure any of us are prepared to discuss yet
15:06:23 <langdon> are there any issues in particular people want to make sure we hit?
15:06:28 <sgallagh> I'm still processing through the thread, but I'm really concerned at the sheer number of handwaves in the initial poist
15:06:29 <sgallagh> *post
15:06:36 <Son_Goku> I'd like to talk about this a bit: https://pagure.io/fm-orchestrator/issue/1241
15:07:01 <Son_Goku> but if we've got other things, then we can do that
15:07:18 <langdon> contyk, sgallagh would you be interested in starting with that? or do you need time to think about it?
15:07:33 <langdon> fm-o 1241 i mean
15:07:42 <contyk> well
15:07:50 <contyk> my opinion on that is still the same
15:08:38 <langdon> ok.. well let's start with that and then go most recent to oldest in the ticket queue and see where we get?
15:08:43 <sgallagh> ok
15:08:50 <asamalik> .hello2
15:08:52 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
15:09:03 <langdon> #info starting with https://pagure.io/fm-orchestrator/issue/1241
15:09:14 <langdon> #info then moving to tickets newest to oldest
15:09:50 <langdon> #topic https://pagure.io/fm-orchestrator/issue/1241 : "Automatically determine build order and build clusters instead of making people figure them out"
15:10:00 <langdon> contyk: do you want to share your opinion?
15:10:13 <contyk> so, as I said before, while it could work for some people in most cases, I don't think it's a universal solution
15:10:32 <contyk> some packages have different build deps on different arches (or driven by other specfile conditions)
15:10:46 <contyk> the general state of build deps listed in spec files is... sad
15:10:56 <Son_Goku> you're not wrong :(
15:11:39 <Son_Goku> my QnD prototype uses rpmspec to pass in macro settings and evaluate the spec based on target arches
15:11:53 <Son_Goku> so I can _mostly_ divine the build deps based on those things
15:12:06 <Son_Goku> the main monkey-wrench now is genbr
15:12:23 <contyk> and that, yes, almost forgot
15:12:29 <Son_Goku> but that, I think, is mostly forcing build-loops now
15:12:33 <contyk> we also now have the bootstrap feature where you can build packages more than once
15:12:58 <contyk> maybe that doesn't matter much; the problem is the same there
15:13:09 <sgallagh> Sorry, genbr?
15:13:10 <contyk> genbr was my other major concern
15:13:17 <Son_Goku> sgallagh: generated build requires
15:13:17 <contyk> generated build deps
15:13:22 <sgallagh> ok
15:13:46 <Son_Goku> my thinking has evolved a bit on this... I think regardless of this implemented, buildafter makes sense
15:13:59 <Son_Goku> I think that if this were implemented, buildorder and potentially bootstrap may not be necessary
15:14:15 <Son_Goku> though I can still see cases where you might want to declare a bootstrap condition and have different settings for bootstrap
15:14:34 <sgallagh> buildorder and buildafter are mutually-exclusive
15:14:38 <sgallagh> You can only use one or the other
15:14:49 <Son_Goku> ah
15:14:54 <contyk> yep
15:15:01 <contyk> and bootstrap is solving a different problem
15:15:04 <Son_Goku> I didn't know that wound up getting set :P
15:15:08 <contyk> I don't see how this would help there
15:15:09 <Son_Goku> contyk, yeah, bootstrap I get
15:15:36 <sgallagh> And bootstrap is basically just a naming fudge so you can have the same component name but pointing at a different ref (or having different flags)
15:15:45 <Son_Goku> yeah
15:16:06 <sgallagh> s/flags/build options/
15:16:12 <sgallagh> Like macros
15:16:19 <mbooth> FWIW there are separate tickets requesting these features be implemented :-)
15:16:34 <sgallagh> mbooth: Most of these are actually there today.
15:16:38 <contyk> I very much prefer maintainers being in control over having some magic there but I probably wouldn't block this feature
15:16:57 <contyk> this feature: automatic buildorder resolution
15:17:07 <sgallagh> Can people stop saying "these" and "this feature" without describing it?
15:17:10 <sgallagh> ... thanks contyk
15:17:15 <contyk> if you opt in via some flag in buildopts or something
15:17:20 <langdon> would generating for the pkgr to include work?
15:17:33 <Son_Goku> langdon, huh?
15:17:38 <langdon> like just take the burden off the human but not nec. lose the "power"
15:17:42 <Son_Goku> yes
15:17:50 <sgallagh> Son_Goku: I think he means "offer a fedpkg command that generates a proposed order"\
15:17:55 <langdon> like do the same thing just not at build time at "spec creation time" or something
15:18:10 <langdon> ubut i suppoose that doesn't help it getting stale
15:18:18 <Son_Goku> langdon, it's possible, ignatenkobrain already did this for Rust
15:18:24 <sgallagh> langdon: The specs are YAML; editing it in place should be easy
15:18:27 <Son_Goku> but the staleness problem is much worse
15:18:49 <Son_Goku> because we're talking about a large dependency web instead of one level of dependencies
15:19:03 <sgallagh> Son_Goku: Sorry, now I'm lost
15:19:07 <langdon> i still lean toward we should do "everything" automatically and just have packagers intervene when the system does something wrong
15:19:16 <Son_Goku> langdon, I agree here :)
15:19:32 <Son_Goku> sgallagh, at the minimum, buildorder or buildafter means you need to be specifying 2 levels of deps
15:19:36 <Son_Goku> deps of the main package, and deps of deps
15:19:45 <contyk> for backwards compatibility we need an opt-in flag, I think
15:19:47 <Son_Goku> most people don't know how to reason that well
15:19:56 <contyk> but if you set it and MBS supports it, okay
15:20:11 <contyk> the generated modulemd can expand it into buildorder or buildafter
15:20:40 <contyk> of course MBS then should really rebuild the SRPM on all arches like depchase did
15:20:44 <Son_Goku> sgallagh, for example, your ReviewBoard module has the deps of ReviewBoard, but also handling the deps of old-ass Django
15:21:08 <sgallagh> Not exactly
15:21:14 <contyk> not sure how to solve situations where the build order differs on different arches but I don't know how to solve that with a local generator either
15:21:26 <Son_Goku> sgallagh, well, you say you need old django, then you need reviewboard built
15:21:34 <Son_Goku> that's a two level chain at least in itself
15:21:58 <Son_Goku> most modules are hopefully at least two levels (otherwise there's not a ton of value in them)
15:21:59 <sgallagh> Son_Goku: django is a separate module in this discussion...
15:22:17 * Son_Goku sighs
15:22:32 <Son_Goku> lets say you collapsed them into one, then?
15:22:38 <Son_Goku> and the internal django is "private" or something
15:22:41 <sgallagh> ok...
15:22:49 <Son_Goku> it's still showing you two levels
15:22:55 <Son_Goku> even as separate modulemds, that's the case
15:23:08 <Son_Goku> and when python2 goes away, your django 1.6 module will incorporate _that_ too
15:23:15 <Son_Goku> so that makes three levels
15:23:55 <Son_Goku> reasoning the necessary build sequences for all the stuff you're including is not easy for most people
15:24:03 <Son_Goku> you and I are probably special weirdos in this regard
15:24:08 <Son_Goku> we can figure it out relatively easily
15:24:09 <sgallagh> heh
15:24:44 <Son_Goku> one of the things I think is critical for making modules more successful is getting as close as we can back to the 90% case of only caring about first-level dependency specification (direct deps)
15:25:16 <sgallagh> Son_Goku: Isn't "buildafter" already doing that?
15:25:19 <Son_Goku> so if you specify a bunch of refs to be built, Koji should be able to figure out the correct sequence and Do The Right Thing(TM)
15:25:24 <Son_Goku> sgallagh, sort of
15:25:30 <sgallagh> I'll certainly admit that "buildorder" did not meet this requirement
15:25:45 <Son_Goku> it does allow the packager to reason ordering
15:26:12 <Son_Goku> and in a ton of cases, one or two buildafters might be enough
15:26:23 <Son_Goku> but I'm thinking more of modules that contain whole language stacks
15:26:49 <Son_Goku> it'd be much easier if those could be mostly computed and only buildafters need to be specified in ambiguous cases
15:26:59 <Son_Goku> (where circular dep cycles might exist, for example)
15:27:26 <Son_Goku> riffing off OBS for a bit, the "Order" stanza in the prjconf serves this purpose
15:27:35 <Son_Goku> and I think buildafter for modulemds would work essentially the same way
15:28:09 <Son_Goku> sgallagh, the reason I'm thinking about this now is that at $DAYJOB, I'm working on the modularity support for our OBS instance
15:28:11 <langdon> i would like to get to something else today.. so can we iron this in to a proposal/resolution?
15:28:50 * Son_Goku shrugs
15:29:04 <sgallagh> Son_Goku: Would you mind writing up a use-case document at least?
15:29:13 <Son_Goku> sgallagh, I could, where do you want it?
15:29:19 <sgallagh> Just to show how you think the user-experience side of things might work
15:29:27 <sgallagh> Then we can work out the tech details.
15:29:27 <Son_Goku> sure
15:29:39 <sgallagh> Son_Goku: Add it to the ticket for now.
15:29:43 <Son_Goku> okay
15:30:21 <langdon> #action Son_Goku to write up a use-case doc and attach it to https://pagure.io/fm-orchestrator/issue/1241
15:30:27 <langdon> ack? ^^
15:30:36 <contyk> ack
15:31:08 <langdon> ok... should we move to the email proposal? or a ticket? (i forgot the email proposal when doing the agenda :( )
15:31:27 <contyk> which proposal do you mean, to be sure?
15:31:29 <Son_Goku> langdon, email proposal as in ignatenkobrain's RFC?
15:31:33 <Son_Goku> or something else?
15:32:02 <langdon> "RFC: modularity simplified"
15:32:40 <sgallagh> I'm still trying to process through the responses, but we can discuss it.
15:32:45 <contyk> I still need to read that thread
15:32:59 <langdon> how about we set up a separate meeting for ~thurs to discuss it?
15:33:04 <sgallagh> I've read the first five or six mails
15:33:14 <langdon> cause i think it won't be a good discussion at the moment
15:33:22 <langdon> but i would prefer to do something before next meeting
15:33:23 <contyk> sure
15:33:32 <langdon> ok..
15:33:42 <sgallagh> My primary concern is that there are a bunch of places where ignatenkobrain basically says "and we add this feature" without any sense of the effort level required.
15:34:05 <langdon> #topic
15:34:11 <langdon> arrgh
15:34:12 <langdon> #undo
15:34:12 <zodbot> Removing item from minutes: <MeetBot.items.Topic object at 0x7fbf0752e090>
15:34:15 <langdon> #topic  "RFC: modularity simplified"
15:34:41 <Son_Goku> sgallagh, I suspect he's not mentioning effort because he doesn't know what would be done
15:34:46 <ignatenkobrain> I would like to discuss, but I'm on the train station waiting for the train
15:35:03 <Son_Goku> I'm sure ignatenkobrain would help implementing things
15:35:06 <langdon> #action langdon to setup meeting to discuss  "RFC: modularity simplified" (https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/3EP3LXGGZABBCAHF5EPZCVEL4FQ23GS3/#VIX7R6QA6EH4DHHEPGMFEQ5MQ5FF7TBE) for later this week
15:35:20 <contyk> this channel is generally free; it's just us using it
15:35:23 <langdon> ignatenkobrain: ill invite you to the meeting later in the week?
15:35:38 <ignatenkobrain> langdon: sure
15:35:49 <Son_Goku> langdon, invite me too :)
15:35:56 <langdon> ok.. so let's do 1 ticket!
15:36:13 <langdon> Son_Goku: ill actually send it to devel@ unless you all think it is a bad idea?
15:36:33 <langdon> but put some names in the to line too (like the peeps here)
15:36:49 <Son_Goku> langdon, that might be okay... I'm worried about potential SNR
15:37:02 <Son_Goku> but we're not _hiding_ these things, so it's fine
15:37:50 <langdon> point
15:37:58 <langdon> ook.. sorry.. got distracted for a second
15:38:15 <langdon> #topic https://pagure.io/modularity/issue/165
15:38:21 <langdon> how about this one?
15:38:34 <contyk> .modularity 165
15:38:35 <zodbot> contyk: Issue #165: What modular packages (build)require Python 2 on rawhide? - modularity - Pagure.io - https://pagure.io/modularity/issue/165
15:38:52 <langdon> ohh i forgot you could do that :)"
15:39:05 <Son_Goku> this should be fixed in dnf 4.2.17
15:39:13 <contyk> well
15:39:14 <Son_Goku> you can tell repoquery to query even disabled packages
15:39:23 <contyk> to do this properly you also need to consider the "buildroot only" modules
15:39:28 <contyk> not just what's in the repo
15:39:30 * Son_Goku grumbles
15:40:25 <contyk> so, hmm
15:40:37 <contyk> obviously you can download all the packages and run repoquery
15:40:46 <contyk> but does anyone have a better idea?
15:41:06 <Son_Goku> well, we can't download them either
15:41:11 <Son_Goku> because we don't know _where_ to get them from
15:41:20 <Son_Goku> pulling from koji is pulling from a bloody firehose
15:41:39 <langdon> i thought we had a proposal of a publicly addressable repo that had "everything" for this kind of use case (but not for installing, etc)
15:41:52 <contyk> yes, a proposal :)
15:41:55 <Son_Goku> langdon, I remember discussing that in March, but I don't see it anywhere :P
15:42:11 <langdon> Son_Goku: it is hiding from you! ;)
15:42:16 <Son_Goku> haha
15:42:24 <langdon> well.. is that the answer?
15:42:37 <langdon> or is it a two piece answer?
15:42:51 <contyk> it would be the anawer if it existed now
15:43:13 <langdon> can we file a "tbd ticket" and point this at that?
15:43:25 <contyk> py2 goes eol in a month
15:43:29 <sgallagh> Question: do we need to have the actual packages in that repo or could we just have the repodata?
15:43:31 <langdon> true
15:43:48 <contyk> I think we should have actual packages in there
15:44:02 <contyk> so that people can work with them, use them for local builds etc
15:44:12 <langdon> contyk: the repo? or the near term solution for churchyard?
15:44:46 <mbooth> Son_Goku: Weird, I found it trivial to write a script to cache locally buildroot only modules: https://github.com/mbooth101/modularity-misc/blob/master/cache_module.sh
15:45:07 <contyk> not including the packages would make the compose a little bit faster; seconds instead of minutes, I think?
15:45:43 <contyk> the majority of the work like setting it up and identifying (automatically) what goes in there still needs to happen
15:45:51 <sgallagh> hmm
15:46:12 <langdon> contyk: "it" in your prior is "the lookup everythign repo"?
15:46:12 <Son_Goku> mbooth, if you know what module you're working from, then it's easy
15:46:16 <sgallagh> I suppose it wouldn't need deltarpms (or signing?).
15:46:20 <Son_Goku> mbooth, if you don't, you're SOL
15:46:23 <contyk> perhaps a near term solution is really just identifying all the modules for f32, then which of those are not available in the repo and providing a simple way to download those packages from koji
15:46:30 <Son_Goku> sgallagh, we should never not sign packages
15:46:39 <Son_Goku> but we can probably cut deltarpms
15:46:47 <mbooth> Son_Goku: Don't you know the list of all modules? That also seems like something you can compute :-)
15:46:49 <contyk> sgallagh: probably not deltas but I suspect it would need signing
15:47:07 <contyk> Son_Goku++
15:47:08 <sgallagh> OK, just trying to figure out which corners we could cut.
15:47:24 <Son_Goku> sgallagh, we can also use cheap metadata generation
15:47:41 <Son_Goku> no xz, no sqlite3, etc.
15:48:10 * sgallagh nods
15:48:35 <Son_Goku> just doing only gzip + zstd chunk cuts down repodata generation time by 60%
15:48:40 <contyk> so mbooth's script could be the short term solution
15:48:54 <langdon> ok.. so i see two things to do.. a) answer churchyard with something hacky b) propose a new repo
15:49:09 <langdon> mbooth: could you answer churchyard with your script?
15:49:26 <mbooth> langdon: Where am I looking?
15:49:31 <langdon> maybe with review by someone here before you post it (if you aren't 100% sure you know it)
15:49:39 <langdon> mbooth: https://pagure.io/modularity/issue/165
15:49:54 <mbooth> (Sorry for interrupting, I just happened to be dropping some eaves today)
15:50:06 <langdon> the more the merrier!
15:50:14 <langdon> esp if we can assign you something! ;)
15:51:24 <langdon> does someone want to volunteer for the "b"?
15:51:57 <contyk> we should have a ticket to track it
15:52:11 <langdon> contyk: the writing of it? i can do that
15:52:42 <langdon> #action langdon to file a ticket about writing up/proposing "everything for reference repo" thingy
15:52:44 <contyk> the existence of additional repo with buildroot-only modules and, just remembered, filtered packages
15:53:09 <contyk> that was another thing; filtered packages not being available
15:53:19 <Son_Goku> yeah, that's a biggie
15:53:38 <sgallagh> Actually, I have a related point about that for EPEL 8
15:53:49 <contyk> in RHEL we have these in the CBR repo; generated *-devel modules that depend on the main module and provide the filtered content
15:54:05 <sgallagh> contyk: Except not all of it
15:54:11 <sgallagh> It's apparently opt-in.
15:54:16 <Son_Goku> wut
15:54:18 <contyk> yes, not all, but in Fedora it'd be everything, I hope
15:54:18 <Son_Goku> no...
15:54:18 <langdon> contyk: CRB, right?
15:54:25 <langdon> code-ready builder
15:54:27 <contyk> langdon: yes, sorry
15:54:46 <Son_Goku> Confusing Repo Branding :P
15:54:47 <sgallagh> So that's actually what I was going to bring up wrt EPEL 8
15:55:07 <langdon> Son_Goku: dont get me started
15:55:11 <sgallagh> Children R Burning!
15:55:16 <langdon> lol
15:55:41 <Son_Goku> langdon, you mean you *didn't* choose that name? :P
15:55:41 <sgallagh> I had a crazy idea for what we could do in EPEL 8 for dealing with missing filtered packages.
15:55:46 <Son_Goku> sgallagh, oh?
15:55:54 <sgallagh> (For example, the lack of libuv-devel is becoming a pain point for many)
15:56:11 <Son_Goku> there are a few missing devel packages blocking pagure deps in EPEL8 too
15:56:16 * sgallagh nods
15:56:17 <langdon> Son_Goku: hahahahaha.. thanks..
15:56:17 <Son_Goku> libmilter-devel from sendmail, for example
15:57:04 <sgallagh> So what I'm thinking about doing is creating a "missingpkgs" module (with no default stream) that builds from the centos git commit and does the reverse filtering.
15:57:05 <Son_Goku> though I think that one was fixed recently
15:57:12 <Son_Goku> sgallagh, oh, neat
15:57:21 <Son_Goku> there's a slight problem, though
15:57:32 <sgallagh> Then we'd use Ursa Prime's override feature to put those into the EPEL 8 non-modular buildroot
15:57:34 <Son_Goku> we do EVR mangling, and our packaging structures make it so those packages conflict
15:57:55 <sgallagh> Son_Goku: Sorry, can you go into greater detail? I can't follow from that.
15:58:22 <Son_Goku> sgallagh, foo (filtered foo-devel) in main module, and foo-devel in aux module have different "release" values
15:58:34 <Son_Goku> so foo-devel requiring foo = %EVR doesn't work
15:58:54 <Son_Goku> the package is not installable, because foo from aux module is filtered out
15:59:11 <Son_Goku> and we have no way of building modules with the same EVR twice
15:59:18 <sgallagh> Sure we do.
15:59:23 <Son_Goku> what?
15:59:28 <sgallagh> Because it wasn't built the *first* time in the Fedora MBS
15:59:34 <Son_Goku> ohhhh
15:59:35 <sgallagh> So the records aren't there with which to conflict
15:59:37 <sgallagh> :-D
15:59:47 <Son_Goku> oh my god, that's so cheeze right there
16:00:08 <sgallagh> Cheezy like a fox?
16:00:10 <Son_Goku> :D
16:00:17 <Son_Goku> it's very clever
16:00:22 <langdon> mbooth: so .. was that a "yes" on replying to the ticket?
16:00:27 <Son_Goku> if the EVRs match, then yeah, it'd work
16:00:39 <sgallagh> langdon: He just submitted a comment, so I'd assume so
16:00:39 <mbooth> langdon: https://pagure.io/modularity/issue/165#comment-614509
16:00:54 <langdon> ohh thanks.. i had the thought to go look but lazy ;)
16:01:19 <langdon> mbooth: thanks!
16:01:23 <mbooth> np
16:01:34 <langdon> ill add a comment about "and look over here for future" .. once i make the other ticket
16:01:42 <langdon> ok.. we are officially over time...
16:01:49 <langdon> should i close the meeting?
16:02:23 <langdon> #action mbooth to add a comment to https://pagure.io/modularity/issue/165 with help
16:02:39 <langdon> #action langdon to add comment to https://pagure.io/modularity/issue/165 pointing at the long term fix ticket
16:02:50 <langdon> #topic open floor
16:02:55 <langdon> anything else?
16:02:58 <langdon> going once?
16:03:07 <sgallagh> Thanks for chairing, langdon
16:03:32 <langdon> going twice!
16:03:35 <Son_Goku> I'm gravy
16:03:41 * langdon attempts to get all the punctuation
16:03:52 <mbooth> langdon: What‽
16:04:02 <langdon> mmmm gravy... all my leftoovers are already gone :(
16:04:04 <langdon> mbooth: nice!
16:04:10 <langdon> and ...
16:04:14 <langdon> #endmeeting