15:00:37 <langdon> #startmeeting Modularity Team Meeting
15:00:37 <zodbot> Meeting started Tue Jan 14 15:00:37 2020 UTC.
15:00:37 <zodbot> This meeting is logged and archived in a public location.
15:00:37 <zodbot> The chair is langdon. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:37 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:37 <zodbot> The meeting name has been set to 'modularity_team_meeting'
15:00:43 <contyk> O hai.
15:00:45 <sct> .hello2
15:00:45 <zodbot> sct: sct 'Stephen Tweedie' <sct@redhat.com>
15:00:46 <langdon> #meetingname modularity
15:00:46 <zodbot> The meeting name has been set to 'modularity'
15:00:49 <langdon> .hello2
15:00:50 <zodbot> langdon: langdon 'Langdon White' <langdon@redhat.com>
15:00:53 <sct> Hi!
15:00:55 <langdon> #topic Roll Call
15:01:06 <contyk> .hello psabata
15:01:07 <zodbot> contyk: psabata 'Petr Šabata' <psabata@redhat.com>
15:01:58 <langdon> is that all we have today?
15:02:14 <asamalik> .hello2
15:02:15 <zodbot> asamalik: asamalik 'Adam Samalik' <asamalik@redhat.com>
15:02:19 <asamalik> happy Tuesday!
15:02:22 <langdon> ooo.. hi asamalik!
15:02:35 <asamalik> こんにちわ langdon!
15:02:36 <contyk> And it was such a good day.
15:02:43 <langdon> ha
15:02:49 <mhroncok> .hello churchyard
15:02:50 <zodbot> mhroncok: churchyard 'Miro Hrončok' <mhroncok@redhat.com>
15:02:53 <langdon> ok.. switching to agenda
15:02:59 <langdon> hi mhroncok!
15:03:00 * zbyszek is lurking too
15:03:04 <langdon> #topic agenda
15:03:06 <contyk> o/
15:03:18 <langdon> do we want to list specific tickets to discuss? or just "all meeting tagged"
15:03:20 <langdon> ?
15:03:20 <mhroncok> langdon, contyk: hi
15:03:30 <contyk> Hi hi.
15:04:30 <contyk> Let's start with the tagged.
15:04:35 <langdon> ok...
15:04:40 <contyk> I hope I tagged the one where I pinged zbyszek and mhroncok.
15:04:45 <langdon> #info reviewing issues tagged with "meeting"
15:04:47 <langdon> ha
15:05:03 <langdon> how's reverse numerical order?
15:05:19 <contyk> Fantastic.
15:05:28 <langdon> ok.. let's go!
15:05:48 <langdon> can we do some trick to get the tickets expanded?
15:06:08 <langdon> #topic CI test to enforce no-replace-base-packages policy for default streams (170)
15:06:16 <langdon> #link https://pagure.io/modularity/issue/170
15:06:17 <asamalik> langdon: open them as new windows in your tiling window manager and make the windows stack on top of each other :P
15:06:18 <contyk> .modularity 170
15:06:19 <zodbot> contyk: Issue #170: CI test to enforce no-replace-base-packages policy for default streams - modularity - Pagure.io - https://pagure.io/modularity/issue/170
15:06:41 <langdon> thats it.. i blank all the time
15:06:52 <langdon> but i can't add those to the minutes that way.. i don't think
15:06:56 <langdon> and asamalik :P
15:07:10 <langdon> so.. contyk, what did you want to discuss?
15:07:44 <contyk> So here I was wondering who could drive this and if there's something we need to keep in mind.
15:08:06 <contyk> In summary, the goal here is to check that a module update is not overriding any content from the Everything repo.
15:08:33 <contyk> It makes sense to add generic test like that to Bodhi, I think; but we should also run similar tests for non-modular packages.
15:08:54 <contyk> Does anyone know who is responsible for these things?
15:09:00 <mhroncok> nobody
15:09:05 <langdon> adamw ?
15:09:16 <langdon> although "responsible" isn't quite the word i would use
15:09:50 <langdon> i thought the qa team was responsible for distro wide tests
15:10:25 <contyk> Hmm.
15:10:48 <asamalik> I first thought that could be done during package review, but then I realized that's a gateway "to the distro, somewhere" not where exactly (morule or non-module)
15:10:52 <asamalik> so this sounds like a CI?
15:11:00 <langdon> is the team working on continuous testing have a formal "thing" in fedora? or are they just under the qa umbrella?
15:11:14 <contyk> asamalik: Yeah, it's a CI thing.
15:11:22 <langdon> ohh right.. the objective
15:11:36 <langdon> i could take this to the council and see if i can get dperpeet to take it
15:11:58 <asamalik> we should know everything we need after a build finishes, right? probably not before because sub-packages... although maybe not even right after the build because of filters in modules...
15:12:04 <contyk> That doesn't sound like a Council matter to me :)
15:12:23 <langdon> shoot ..can someone take over.. i need 5-10
15:12:45 <asamalik> langdon: agree with contyk, this is purely a technical solution to a concept that should be in place as part of Modularity I'd say
15:12:47 <contyk> But let's try talking to dperpeet and bookwar if they have some ideas.
15:13:06 <contyk> asamalik: Currently we run these tests in Bodhi only, afaik.
15:14:01 <asamalik> yeah, bodhi is basically the gate that sends packages "to the distro"... although there might not be a need to do it this late...
15:14:13 <mhroncok> I have a question
15:14:15 <asamalik> although doing this in bodhi could, somehow, allow us to swap packages around as a single update
15:14:44 <mhroncok> suppose the CI is done and reliable. what happens with the offenders?
15:15:10 <mhroncok> if it only informs, is it worh it? and if it blocks, should we fix the current state before we automate a check?
15:15:12 <contyk> I'd say the update should either be blocked or the maintainer notified at least.
15:15:33 <contyk> After that they can decide to waive/ignore the test result and then be punished by FESCo or fix it.
15:15:35 <asamalik> mhroncok: before I thought about the possibility of potentially swapping packages around in bodhi, I'd say the build fails... but that's probably not the right answer
15:15:44 <mhroncok> aka we know our modules are breaking this, should we focus on CI that determines what we know, or on the fix?
15:15:48 <contyk> It depends on the specifics, I'd say.
15:16:24 <mhroncok> I'd argue it only makes sense to CI this once we fix the know offenders
15:16:27 <mhroncok> *known
15:16:38 <asamalik> it's probably not CI... it sounds more and more as Bodhi to me... and Bodhi would just prevent submitting that update, with an error message saying "this package is already in platform, you can't do this"
15:16:53 <contyk> The CI should check whether the module provides artifacts that are also provided by Everything. When it comes to fixes, it's case by case.
15:17:06 <asamalik> "... you can swap them, that's done this way <link>"
15:17:26 <contyk> asamalik: Bodhi is CI on the update level :)
15:17:33 <langdon> i am back.. and i realize i am the only chair
15:17:40 <langdon> #chair contyk asamalik sct
15:17:40 <zodbot> Current chairs: asamalik contyk langdon sct
15:17:59 <asamalik> contyk: ok, in that case we probably agree :)
15:18:15 <asamalik> .chair langdon
15:18:15 <zodbot> langdon is seated in a chair with a nice view of a placid lake, unsuspecting that another chair is about to be slammed into them.
15:18:16 <asamalik> :D
15:18:43 <contyk> mhroncok: In your view, does it matter if we start testing it before the current modules are fixed?
15:18:53 <contyk> Imagine someone implements it tomorrow; why would that be an issue?
15:19:15 <mhroncok> contyk: no, if it is magically done with 0 resources, there's no problem
15:19:20 <langdon> asamalik: at least i have my towel
15:19:25 <contyk> :)
15:19:31 <mhroncok> I simply think the resources should be allocated to fix stuff rather than automate checking if it is borken
15:19:35 <asamalik> langdon++
15:19:51 <mhroncok> especaially since we know already that it is
15:20:34 <asamalik> mhroncok: I'd say 1) automate the check and make it grant exception to things that "have been already broken before the update" and 2) make humans fix the problems
15:20:52 <asamalik> the exception doesn't suddenly block updates to existing stuff
15:21:17 <asamalik> and the automation frees up people to then just fix stuff without being slowed down by more and more checks
15:21:19 <mhroncok> I don't agree with hat exception. I think this needs to be solved for the existing stuff ASAP
15:21:31 <mhroncok> *hat -> that
15:21:57 <mhroncok> (either way, I am not here to argua about that, feel free to ingore me and move on)
15:22:59 <contyk> I think the next step is to get in touch with someone who knows how and can help implementing those checks.
15:23:01 <zbyszek> FTR, I think mhroncok is fully right. Allocating resources to first automatize the check is substracting resources from the fix for hte problem that we need quickly.
15:23:07 <contyk> That's the point of this ticket.
15:23:24 <langdon> just fyi, when i said "bring to council" i meant "talk to dperpeet at the council meeting about how they should take it over" not actually ask council "for a decision"
15:23:25 <contyk> People fixing the modules are not necessarily the people writing these tests.
15:23:30 <contyk> It's quite unlikely they are.
15:24:20 <asamalik> mhroncok, zbyszek: I completely trust your judgement on the urgency, and I believe I'm not alone
15:24:32 <mhroncok> if the people fixing the things don't fix the things, you are wasting the resources of the poeple writing the checks
15:24:49 <zbyszek> This is pretty much the opposite of how we normally do things: fix the user visible issue first, maybe using some scripting to localize, and then do long-term prevention at leasure.
15:24:49 <asamalik> fixing sounds like a provenpackager / FESCo member action to me though
15:24:49 <contyk> That I disagree with.
15:25:32 <zbyszek> *leisure even.
15:25:37 <asamalik> and implementing the check as a Bodhi / CI dev job
15:25:43 <contyk> I don't see why one would need to happen before the other and why we wouldn't want to prevent introducing more potential breakage via new updates before the existing stuff is fixed.
15:26:00 <contyk> And why introducing those checks while there is something broken in the distro would be waste of time and resources.
15:26:04 <contyk> Maybe I'm missing something.
15:26:41 <asamalik> contyk: I agree with you considering it's not the same person/people doing those two things
15:27:22 <zbyszek> OK, it seems like we have different views on this.
15:27:54 <mhroncok> imagine you have 2 armed strange men at the airport
15:28:05 <mhroncok> you tlak about building a gun detector at the entrance
15:28:21 <mhroncok> I talk about chasing the men and getting them out of there
15:28:31 <mhroncok> sure, you can build the detector first, it doesn't harm anything
15:28:40 <langdon> but if you have two people... can't you do both?
15:28:48 <mhroncok> sure
15:28:50 <mhroncok> you can
15:28:52 <contyk> You can; because it's typically different people.
15:28:56 <asamalik> mhroncok: what if FESCo / proven packager chase them, while a CI / Bodhi dev builds the gate?
15:29:10 <langdon> or.. really.. an expert in chasing aka bouncer and a bomb detector expert.. neither can actually help the other
15:29:28 <langdon> sorry.. *gun detector expert ;)
15:29:39 <zbyszek> "Hello, this is the security director. There are some armed attackers at the door. I'd like to inquire about opening a tender procedure..."
15:29:46 <asamalik> and the Modularity Team advises both on how to catch them and how to build the gate so it works properly
15:29:51 <contyk> The disagreement stems from the the idea that it's one person doing everything, according to mhroncok and zbyszek.
15:29:53 <langdon> zbyszek: did you fill out the correct form?
15:29:57 <contyk> I don't think that's the case.
15:30:02 <contyk> (it's actually no one, haha)
15:30:06 <langdon> contyk: ++
15:30:15 <langdon> contyk++
15:30:19 <langdon> lol
15:30:33 <contyk> Anyway, to move forward.
15:30:36 <asamalik> zodbot++
15:30:43 <contyk> I'll get in touch with the CI folks to see who could be implementing this.
15:30:51 <langdon> woot!
15:30:54 <contyk> It will take some time anyway.
15:30:58 <langdon> ok.. contyk info? action?
15:31:30 <contyk> #action contyk will get in touch with the CI team to see who could own and implement such checks, as well as provide test definitions later.
15:31:39 <langdon> thanks!
15:31:41 <langdon> next topic?
15:31:58 <langdon> .modularity 169
15:31:59 <zodbot> langdon: Issue #169: Security vulnerabilities (CVEs) are not properly tracked in modular packages - modularity - Pagure.io - https://pagure.io/modularity/issue/169
15:32:10 <langdon> #topic  Issue #169: Security vulnerabilities (CVEs) are not properly tracked in modular packages - modularity - Pagure.io - https://pagure.io/modularity/issue/169
15:32:31 <mhroncok> one of mine :)
15:32:54 <langdon> in some ways, i think this is very similar to the last one (IIRC)
15:33:15 <langdon> as in we need more "infra help" for modules
15:33:56 <langdon> sorry.. "infra" meaning "fedora mechanics" or something.. not the infra team
15:34:02 <contyk> Yeah, well.
15:34:25 <contyk> Not infra indeed; we need to get in touch with the security team and make sure their SOPs consider modules and modular packages.
15:34:51 <contyk> Which means they will also need some guidance on how to discover modular content, which is not trivial.
15:34:59 <contyk> I believe there's another ticket from mhroncok on that topic.
15:35:06 <mhroncok> indeed
15:35:20 <zbyszek> Before talking about mechanics, what is the desired workflow? If security team is supposed to check all the extra packages, how will we grow the team?
15:36:18 <contyk> I'm not sure I understand the question.
15:36:25 <contyk> Is it something we could influence?
15:36:35 <zbyszek> Does it even make sense to ask people to spelunk in what is essentially bundled code?
15:36:43 <langdon> zbyszek: is that modularity team's responsibility? sure it is important but we need the security team to think about that
15:36:44 <contyk> Is the number of extra packages overwhelming right now or is there a reason to think it would become overwhelming in the near future?>
15:37:18 <zbyszek> Yes, the security team is already stretched.
15:37:48 <langdon> zbyszek: i would say pretty much all the teams are already stretched.. we can't just stop because of that
15:37:57 <langdon> we = fedora
15:38:18 <asamalik> I see it as the same thing as getting new packages in the distro — it's not really a modularity thing, no? but maybe I don't understand the question
15:38:30 <contyk> asamalik: I was just going to say the same thing.
15:38:34 <zbyszek> OK, let me make an example
15:38:44 <langdon> there is a volume argument.. but i definitely don't agree with the "bundled" comment
15:39:35 <zbyszek> Let's say we wanted to extend the support period to 18 months. This has been discussed before a few times. And the strongest counterargument is the extra support burden on packagers.
15:40:01 <zbyszek> By flipping a number from 12 to 18 we would "magically" add a lot of work to a certain group of people.
15:40:43 <zbyszek> If we say "security team is now also responsible" for amy modular copy of an rpm, suddenly they might go from 3 copies (Fn, Fn+1, Fn+2), to e.g. 20.
15:40:51 <zbyszek> This is potentially substantial work.
15:40:55 <langdon> right.. but that is an argument for modularity = yes/no ... not how do we support the security team
15:41:15 <zbyszek> Well, it's also a question who is on the hook for this.
15:41:18 <langdon> and also solvable by policy
15:41:30 <zbyszek> The security team is not the only possible answer.
15:41:38 <zbyszek> Module owners are certainly another possible answer.
15:42:05 <langdon> sure.. but unlike why modularity teams review module mds vs fpc... the security review of rpms is the same in modules vs non-modules
15:42:13 <contyk> It's rather rare that people report security problems for their own packages.
15:42:32 <mhroncok> langdon: except discoverability
15:42:40 <zbyszek> ... and volume
15:43:13 <langdon> mhroncok: the discoverabilty problem is your other ticket(s) .. again, we need more infra changes to support "normal" operations for modules..
15:43:47 <mhroncok> or less modules
15:43:56 <mhroncok> (technically, that also works)
15:43:56 <langdon> zbyszek: like I said, i can agree with that .. but.. we can easily solve that with policy.. "we can only have 37 modules & 200 new rpms until 17 people have been added to the security team"
15:44:14 <langdon> but that would be more of a council call.. i would think
15:44:25 <contyk> That's why I was asking whether the current number is considered overwhelming, or whether there's a reason it to become so in the near future.
15:44:31 <langdon> when the security teams says OMG ALL THE RPMs
15:44:33 <asamalik> what about this... the the note about capacity is a valid point, but it's not this team who plans capacity for the project, right? council has decided that we want more content and put modularity in place, so I'd say they need to plan the capacity for that as well
15:44:39 <asamalik> like we won't solve it here
15:44:46 <langdon> right
15:44:50 <contyk> If simply more packages is being a concern, then we should forbid new packages in Fedora until we have more people.
15:44:55 <langdon> right?
15:45:03 <contyk> Or just stop pretending we do security checks.
15:45:24 <mhroncok> contyk: I'm removing packages from Fedora at great speed, so there si some room for more
15:45:32 <contyk> mhroncok++
15:45:32 <zodbot> contyk: Karma for churchyard changed to 9 (for the current release cycle):  https://badges.fedoraproject.org/tags/cookie/any
15:45:53 <langdon> hahaha
15:46:12 <asamalik> lol
15:46:44 <langdon> mhroncok: you need to work the word "alacrity" into more of your comments :)
15:47:06 <langdon> so.. what's the next step?
15:48:01 <langdon> sounds like it is a) reach out to security team and ask what help they need (aside from more people).. b) i go to council and say "hey we are afraid sec is going to be overwhelmed w/ modules, whats the plan"
15:48:04 <contyk> I'd say make sure there's an easy way to discover all the content that is shipped or used to build stuff that we ship.
15:48:42 <mhroncok> contyk: +1
15:48:42 <contyk> After that we can look for people who could work with that and hope they have the resources. If not, Fedora might just declare "not all of the distro is properly tested for security problems"
15:48:52 <mhroncok> you cannot expect sec to track packages they cannot see
15:48:53 <langdon> contyk: i would rather the security team provides a punch list for any of the other things we don't know about that they use
15:50:21 <langdon> kinda like mhroncok has been doing in the tickets
15:51:45 <langdon> 9m left... do we have an info or action we can apply?
15:51:56 <contyk> Well, I'd say this is blocked by #166 and #163
15:53:29 <langdon> probably...
15:54:00 <langdon> so maybe it is blocked by those and a comment about approaching the security team about needs after those two are complete?
15:54:32 <langdon> anyone disagree/have more to add?
15:55:46 <contyk> Yeah.
15:55:51 <contyk> Adding the comment now.
15:56:11 <langdon> contyk: "yeah" you disagree? or you agree with my comment?
15:57:06 <contyk> Exactly!
15:57:09 <langdon> lol
15:57:33 <contyk> 168 now?
15:57:51 <contyk> This is the one for which I invited mhroncok and zbyszek :)
15:57:54 <langdon> for 3m?
15:57:57 <langdon> ok
15:58:37 <langdon> #info #169 blocked by #166 and #163, when resolved, we will follow up with the security team
15:58:46 <langdon> .modularity 168
15:58:49 <zodbot> langdon: Issue #168: some thoughts on modularity docs - modularity - Pagure.io - https://pagure.io/modularity/issue/168
15:58:56 <langdon> #topic  Issue #168: some thoughts on modularity docs - modularity - Pagure.io - https://pagure.io/modularity/issue/168
15:59:08 <langdon> contyk?
15:59:20 <contyk> So I think what I wanted is basically some input what exactly you guys think is still wrong with the docs.
15:59:48 <zbyszek> The ticket has quotes and specific pointers. What more could I say?
15:59:48 <contyk> We recently reviewed the whole site and asamalik moved things around so that it's, hopefully, easier to discover.
15:59:56 <contyk> We also think it covers most if not all of the topics.
16:00:46 <mhroncok> the ticket has explciit examples of what needs more coverage
16:01:08 <contyk> Hmm, seems that way now :)
16:01:12 <mhroncok> e.g. how does the overriding of nonmodular packages actually work
16:01:24 <langdon> i think we need a third party to help write it.. i think all of us are too close to it if you (mhroncok & zbyszek) disagree
16:01:32 <langdon> but.. we should be sure you have looked at latest
16:01:51 <mhroncok> we have looked ta latest when reporting
16:02:09 <mhroncok> if you updated since, please so answer the specific points with specific links
16:02:15 <mhroncok> *go
16:02:22 <langdon> asamalik: thoughts?
16:02:54 <contyk> Okay, thank you.
16:03:18 <langdon> mhroncok: i think we are worried about it because they are right around the same time
16:03:26 <langdon> the ticket and docs changes
16:03:57 <asamalik> langdon: I'd say if there's something missing, we should add it for sure... or what did you mean?
16:04:24 <asamalik> there is a lot of things answered (people are somehow missing the FAQ for example) but there might be gaps
16:04:25 <langdon> asamalik: i meant do you feel like the items in the ticket are a) covered b) can you suggest/provide links
16:04:28 <mhroncok> "Each module defines its own life cycle which is closer to the upstream rather than the Fedora release." is still there, the ticket describes the latest version
16:05:49 <mhroncok> if you feel like the statements in the ticket are not valid, you should be able to provide reasons for that, not say that generally about the entire ticket. what am I missing?
16:06:07 <contyk> No, I think it's alright.
16:06:16 <contyk> We need to sit down and go through it :)
16:06:26 <contyk> Next week would be good.
16:06:32 * contyk actually has to run now.
16:06:39 <asamalik> langdon: oh I don't think I'll be able to check all of that now... it's been a long time since I wrote bunch of those things
16:06:56 <langdon> ok..
16:07:05 <langdon> contyk: do you want the action?
16:07:08 <langdon> or someone else?
16:07:09 <mhroncok> contyk: what was the reason you decided to invite us to this meeting about this ticket?
16:07:25 <zbyszek> Did anyone actually read the ticket before the meeting?
16:07:37 <contyk> mhroncok: We were going through the queue yesterday and I don't think we looked at this one properly at that time.
16:07:48 <contyk> mhroncok, zbyszek: Feels like I wasted your time; apologies for that.
16:07:57 <langdon> zbyszek: ha.. i did..
16:08:25 <langdon> ok.. so we will follow up with more targeted questions
16:08:30 <langdon> if there are any
16:09:08 <langdon> #info contyk and langdon to review #168 and associated docs next week (jan 20, 2020)
16:09:15 <contyk> Ack.
16:09:22 <langdon> ok.. closing the meeting down unless anyone has anything last minute
16:10:13 <langdon> #endmeeting