15:00:19 <sgallagh> #startmeeting FESCO (2021-01-20)
15:00:19 <zodbot> Meeting started Wed Jan 20 15:00:19 2021 UTC.
15:00:19 <zodbot> This meeting is logged and archived in a public location.
15:00:19 <zodbot> The chair is sgallagh. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:19 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:00:19 <zodbot> The meeting name has been set to 'fesco_(2021-01-20)'
15:00:19 <sgallagh> #meetingname fesco
15:00:19 <sgallagh> #chair nirik, ignatenkobrain, decathorpe, zbyszek, sgallagh, mhroncok, dcantrell, cverna, Conan_Kudo, Pharaoh_Atem, Son_Goku, King_InuYasha, Sir_Gallantmon, Eighth_Doctor
15:00:19 <sgallagh> #topic init process
15:00:19 <zodbot> The meeting name has been set to 'fesco'
15:00:19 <zodbot> Current chairs: Conan_Kudo Eighth_Doctor King_InuYasha Pharaoh_Atem Sir_Gallantmon Son_Goku cverna dcantrell decathorpe ignatenkobrain mhroncok nirik sgallagh zbyszek
15:00:30 <nirik> morning
15:00:34 <sgallagh> .hello2
15:00:35 <zodbot> sgallagh: sgallagh 'Stephen Gallagher' <sgallagh@redhat.com>
15:00:39 <decathorpe> .hello2
15:00:40 <zodbot> decathorpe: decathorpe 'Fabio Valentini' <decathorpe@gmail.com>
15:00:47 <nirik> .hello kevin
15:00:48 <zodbot> nirik: kevin 'Kevin Fenzi' <kevin@scrye.com>
15:01:39 <sgallagh> I'm glad to be here; half an hour ago I was banned from Freenode for pinging you all about the meeting. :-P
15:01:51 <nirik> ha. spammer!
15:02:05 <cverna> hello
15:02:12 * sgallagh has a shame
15:02:27 <zbyszek> .hello2
15:02:29 <zodbot> zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' <zbyszek@in.waw.pl>
15:03:17 <King_InuYasha> .hello ngompa
15:03:18 <zodbot> King_InuYasha: ngompa 'Neal Gompa' <ngompa13@gmail.com>
15:03:19 * King_InuYasha waves
15:04:44 <sgallagh> That makes quorum. Just.
15:05:07 <sgallagh> Oh, I missed cverna. So that's 6
15:05:16 <sgallagh> Shall we begin?
15:05:28 <zbyszek> Yeah!
15:05:37 <cverna> +1
15:05:44 <sgallagh> #topic #2547 F34 Change: Signed RPM Contents
15:05:44 <sgallagh> .fesco 2547
15:05:45 <zodbot> sgallagh: Issue #2547: F34 Change: Signed RPM Contents - fesco - Pagure.io - https://pagure.io/fesco/issue/2547
15:06:38 <puiterwijk> Hi
15:06:46 <sgallagh> So this has been somewhat controversial due to concerns that it increases disk usage for unclear (to us) benefit.
15:06:53 <sgallagh> puiterwijk: Thanks for joining us.
15:07:04 <zbyszek> Hi puiterwijk
15:07:18 <dcantrell> .hello2
15:07:19 <zodbot> dcantrell: dcantrell 'David Cantrell' <dcantrell@redhat.com>
15:07:20 <nirik> I'm in favor, but I understand the concerns about space...
15:07:22 <dcantrell> sorry I'm late
15:07:26 <sgallagh> In general, I'm all for improving trust and provenance.
15:07:30 * King_InuYasha shrugs
15:07:50 <King_InuYasha> when we can't get simple things people can use now, I'm a lot less enthusiastic about stuff like this
15:07:56 <sgallagh> But there also seem to be concerns that those aren't particularly improved by it either. I'm not sure I agree, but I don't understand the plan well enough to be certain.
15:08:14 <mhroncok> sorry for being late, I forgot to join the channel
15:08:15 <sgallagh> King_InuYasha: Let's not derail the topic, please.
15:08:24 * King_InuYasha shrugs
15:08:36 <nirik> well, this just empowers people to build their own policies... but I agree some examples might be nice but that could come after.
15:08:38 <puiterwijk> sgallagh: so, this would integrate with for example Keylime to enable (remote) attestation. Or just with a local IMA policy to require signed files
15:08:39 <King_InuYasha> there's no explanation of how to use IMA at all
15:09:02 <puiterwijk> We absolutely intend to add documentation for IMA policies and how to use it to docs.fp.o
15:09:03 <King_InuYasha> and there's no way to strip it at install-time either
15:09:16 <sgallagh> puiterwijk: Could you indicate that on the Change page?
15:09:17 <King_InuYasha> because it basically doubles the size of the content on disk
15:09:22 <puiterwijk> sgallagh: will do
15:09:24 <zbyszek> Also, no explanation why this couldn't be handled similarly to -debuginfo files.
15:09:26 <sgallagh> Thank you
15:09:28 <puiterwijk> King_InuYasha: well, it doesn't really
15:09:34 <dcantrell> are there any special case packages where this proposal won't work?
15:10:21 <decathorpe> or does this conflict with the btrfs dnf/rpm cow functionality that was already approved?
15:10:22 <King_InuYasha> well, custom packages won't work if IMA is active
15:10:24 <puiterwijk> King_InuYasha: only the RPMDB increases (sligthly) in size (numbers in the thread), the files itself don't, and the attr's aren't put in place unless you have rpm-plugins-ima. So if you don't have that installed, there's no extra disk space other than rpmdb
15:10:28 <sgallagh> Point of order: there are many of us and only one puiterwijk. For this meeting, I recommend we use the o/ "hand raising" approach to asking questions.
15:10:32 <puiterwijk> King_InuYasha: they absolutely will
15:10:45 <King_InuYasha> puiterwijk: that plugin is installed by default
15:11:17 <King_InuYasha> at least, it is on my systems
15:11:20 <puiterwijk> King_InuYasha: okay. In that case, the disk space still isn't that much more. It's a single (binary) xattr per file with 81 bytes
15:12:04 <puiterwijk> Also, custom packages totally still work. Theyj ust won't have signatures. And that then depends on the policy you put in place
15:12:39 <dcantrell> reading through the proposal here, I would like to see some more concrete examples especially around the benefits of this proposal
15:12:45 <puiterwijk> zbyszek: so, I have a trial that adds the signatures in a -file-signatures subpackage. The problem with that is that the data needs to be in an xattr, so that IMA can actually use it
15:12:50 <decathorpe> .o/
15:13:54 <zbyszek> puiterwijk: would it be possible to order the installation so that the -file-signatures package is installed earlier?
15:13:56 <puiterwijk> And honestly, having an RPM plugin that automagically injects things from a side-package as an xattr seems.... really ugly to me
15:14:44 <nirik> might it help for examples/use cases to more explain what iot would like to use this for?
15:14:45 <sgallagh> decathorpe: Go ahead
15:15:08 <zbyszek> .o/
15:15:09 * nirik didn't raise his hand, sorry.
15:15:15 <zbyszek> bad nirik
15:15:25 <decathorpe> I see that this affects rpmdb and install size only minimally, but how does it change binary RPM size? some numbers shared by panu make this seem really bad
15:15:30 <puiterwijk> nirik: I can absolutely give examples now and put them in docs. That is a next part. The reason I put in the change now is so we can get the signatures before the mass rebuild
15:15:43 <decathorpe> which makes me concerned about storage use in koji and on mirrors.
15:16:20 <puiterwijk> decathorpe: it adds +- 162 bytes per file in an RPM. The "100% increase" is the very worst case if you have an RPM with a single tiny file
15:16:24 <zbyszek> It'd be good to put the full information about size increases (and instalation time changes) in the wiki.
15:16:42 <puiterwijk> I actually had those numbers, and then dropped them from the research notes I guess, since I can't see them right now. Sorry
15:16:45 <sgallagh> . /o (after zbyszek)
15:16:57 <sgallagh> zbyszek: Go ahead
15:17:21 <puiterwijk> But basically, it was a +- 10-15% increase on the entirety of the base container, iirc.
15:18:11 <zbyszek> I'm coming up short on the motivation. The policy (the way I imagine it), cannot cover things like .bashrc and other scripts. So I wonder how useful it is in practice. The Motivation part is somewhat lacking in the Change page.
15:19:01 <puiterwijk> zbyszek: it could cover things like .bashrc, it just means that system owners need to deliver their own signatures for files they intend to change from the Fedora default
15:19:49 <puiterwijk> This is less for your personal workstation, and more for e.g. servers where you want to make sure that what's actually running is what you intended to put in place
15:19:53 <zbyszek> puiterwijk: does it cover systemd unit files and other "config"?
15:20:24 <puiterwijk> zbyszek: we sign them, and if you set an RPM macro, it puts them in place. And then you can define a policy on whether it will cover that, yes
15:20:25 <sgallagh> puiterwijk: Could you give us a specific example of a case that is being solved for IoT? Such as a market demand that this satisfies (not just a checkmark, but a real security concern).
15:21:26 <puiterwijk> sgallagh: in the case of IoT: I want to ensure that a system "out in the field" (i.e. no physical protection) only starts authorized code. Normally, the "trusted code" ends with the kernel (with secure boot). With IMA, you can extend the trust out to the actual binaries of all the daemons and their configuration
15:22:07 <zbyszek> puiterwijk: if you have a link to some docs about ima that I could look at. I don't quite see how this is supposed to cover a real system, but I don't want to waste everyone's time here.
15:22:29 <puiterwijk> For that, you can define a policy that says "any binary that runs needs to be fully signed, and all configuration files need to be fully signed". After which the kernel would refuse to load any of them that is not correctly signed (or measured)
15:22:34 <puiterwijk> zbyszek: yes, one minute.
15:22:58 <puiterwijk> zbyszek: https://sourceforge.net/p/linux-ima/wiki/Home/ is the main doc
15:24:35 <nirik> ✋
15:24:35 <sgallagh> puiterwijk: And the IMA configuration itself is signed by the Secure Boot key?
15:24:43 <King_InuYasha> hold up
15:24:47 <zbyszek> puiterwijk: right, but right now our systems have lots of files which are not distributed as static rpm contents. Basically anything that requires a scriptlet in %post. E.g. /ets/nsswitch.conf is generated on the end system, and it loads arbitrary dlls into the address space of every process.
15:24:54 <King_InuYasha> Secure Boot has to be involved?!
15:25:03 <puiterwijk> King_InuYasha: no. This is not tied in to secure boo.
15:25:05 <puiterwijk> boot*
15:25:15 <King_InuYasha> good
15:25:18 <sgallagh> nirik: go ahead
15:25:22 <puiterwijk> King_InuYasha: The secure boot part is just an example of how to get the trust of the system from the moment the power button is pressed
15:25:35 <sgallagh> (Also, I give up on the hand-raising idea; it's not working)
15:25:36 <King_InuYasha> yeah, no, that is a terrible strategy and we should not endorse that
15:25:38 <dcantrell> .o/
15:25:41 <puiterwijk> sgallagh: it is signed by a customer key that they enroll on the system (that is a patch that I'm discussing with the kernel upstream right now)
15:25:46 <mhroncok> sgallagh: you've tried
15:25:48 <nirik> ha, yeah, it doesn't tend to work. ;)
15:25:50 <dcantrell> sgallagh: sorry
15:26:48 <sgallagh> puiterwijk: Without secure boot, how would the customer key be protected?
15:27:01 <mhroncok> o/
15:27:10 <nirik> I was going to note that we worked with puiterwijk to confirm this was all working in infra... it's currently working on rawhide builds, so in fact you can see size differences in the rawhide report. Of course those are due to other changes in the packages, so it's not caused all by this, but in general it's not seemed to cause too much size issues
15:27:29 <puiterwijk> sgallagh: so, the current patch that we have is to use the TPM NV to enroll the keys on the system.
15:27:59 <sgallagh> OK, so regardless of secure boot, it must use some sort of HSM.
15:28:04 <puiterwijk> sgallagh: but without secure boot, you can add your own key to the .ima kernel keyring, and it will work from there. systemd also has support to automatically do that
15:28:15 <sgallagh> mhroncok: go ahead
15:28:24 <puiterwijk> sgallagh: no. If you don't have that, systemd will grab /etc/...something.pem and load that on boot
15:28:31 <puiterwijk> (in pre-pivot, so initrd)
15:28:53 <dcantrell> what does IMA give us that we can't get through additional SELinux policies?
15:29:18 <mhroncok> obviously, we have a lot of questions. the change proposal detailed description is one sentence. maybe we should step back and actually have the change proposal updated with actual details, before we have meetings like this?
15:29:28 <puiterwijk> dcantrell: that you can be sure the files were not modified since they were shipped from Fedora, i.e. an attacker can't extract the disk, change a file, and shove it back in.
15:29:30 <zbyszek> sgallagh: yeah, I don't think hsm is *required*. This provides (some) runtime security as long as the machine is not compromised. It doesn't provide offline security.
15:29:43 <King_InuYasha> puiterwijk: wouldn't rpm -V work for that?
15:30:00 <puiterwijk> zbyszek: there's no hsm rewquired. But the use without that is limited. The signatures help
15:30:08 <King_InuYasha> we already have cryptographically strong checksums of all files, and we can derive trust from GPG signed payloads
15:30:39 <puiterwijk> King_InuYasha: if you trust rpm, sure. IMA pulls the trusted chain into the kernel, which you can use secure-boot with to trust.
15:30:44 <dcantrell> King_InuYasha: right
15:30:46 <nirik> that doesn't work for things like runtime tho...
15:31:14 <sgallagh> It also doesn't prevent you from running binaries that weren't delivered by RPM
15:31:25 <puiterwijk> Yes, that too
15:31:28 <nirik> rpm -V foo, looks ok, /usr/bin/foo, but during 'looks ok' attacker swaps in different one
15:31:36 <sgallagh> OK, we've been on this topic for 30 minutes.
15:32:20 <zbyszek> mhroncok: +1
15:32:44 <decathorpe> mhroncok: +1
15:32:54 <sgallagh> I'm going to raise the proposal for acceptance (with the understanding that puiterwijk will be adding more details to the Change page).  A majority of +1 votes accepts the Change, any other result will be revisited next week after the Change Proposal is updated.
15:32:55 * King_InuYasha shrugs
15:33:11 <zbyszek> sgallagh: I'm not ready to vote. Not enough info.
15:33:15 <dcantrell> same
15:33:20 <nirik> +1 here
15:33:22 <dcantrell> I want to read the revised change proposal and the sf page
15:33:28 <puiterwijk> Also, note that the intention of this change is just to get the files signed with keys under Fedora control. The thought being that after that, users can use it if they want to. Or ignore it if they don't.
15:33:29 <mhroncok> I am ready to vote -1 for the current one sentence proposal
15:33:43 <King_InuYasha> I'm +1 to this with the proviso that we *don't* endorse rooting it in secure boot trust *and* rpm-plugin-ima is not included on media by default
15:33:45 <nirik> but note if we move to next week, it will likely mean it will have to move to f35
15:34:00 <sgallagh> nirik: Good point.
15:34:27 <zbyszek> Right now it's not clear what the overhead is. "10-15% in the base image" sounds like a lot.
15:34:35 <sgallagh> FTR, I am +1 to adding this feature, but I would agree to King_InuYasha 's conditions.
15:34:36 <puiterwijk> zbyszek: base container*
15:34:53 <puiterwijk> And yes, King_InuYasha's conditions sound reasonable to me.
15:34:57 <zbyszek> puiterwijk: right, "base container". And it's the image where size matters a lot.
15:35:15 <King_InuYasha> we'll need to block rpm-plugin-ima from Mock bootstrap image builds
15:35:17 <puiterwijk> zbyszek: also, note that the 10-15% is on the rpmdb size of that
15:35:34 <zbyszek> puiterwijk: ah, OK. That's not an issue then.
15:35:47 <sgallagh> puiterwijk: meaning not 10-15% of the rpmdb, not 10-15% of the whole image size?
15:35:52 <sgallagh> (Just to be clear)
15:35:55 <puiterwijk> sgallagh: 10-15% rpmdb size
15:36:15 * nirik doesn't have rpm-plugin-ima here... workstation install
15:36:26 <King_InuYasha> nirik: I think we block it already for live media composes
15:36:47 <King_InuYasha> because I have been netinstalling my systems and I have it for some reason
15:37:05 <King_InuYasha> unless something else is pulling it in
15:37:06 <sgallagh> OK, I'll rephrase the formal proposal.
15:37:52 <sgallagh> Proposal: Signed RPM Contents for IMA are accepted, contingent upon additional details added to the Change Proposal and that the implementation may not require secure boot nor include rpm-plugin-ima in a default install.
15:37:54 <sgallagh> +1
15:38:04 <mhroncok> -1
15:38:08 <zbyszek> -1
15:38:09 <dcantrell> -1
15:38:17 <decathorpe> I stand by my -1 vote as well
15:38:37 <cverna> 0
15:38:38 <nirik> +1
15:38:51 <sgallagh> cverna: a 0 vote is equivalent to a -1 in this situation.
15:39:12 <nirik> nothing seems to require or mention rpm-plugin-ima that I can see.
15:39:22 <cverna> I have to catch up on the change, so I can't really vote now
15:39:34 <cverna> so I stand by my 0
15:39:44 <mhroncok> nirik: neither does the change proposal mention rpm-plugin-ima
15:39:45 <King_InuYasha> sgallagh: +1
15:40:22 <sgallagh> I count (+3, 1, -4) with no ignatenkobrain present.
15:40:30 * decathorpe also grumbles about this being enabled in production before it was approved
15:40:39 <mboddu> I think all the -1's are for lack of information, can we create a hackmd doc with all the questions that they may have and vote on the ticket by end of the today, provided puiterwijk answers those questions?
15:40:53 <mhroncok> not for me
15:41:10 <dcantrell> that won't work for me either
15:41:16 <sgallagh> Let's ask the question differently: what would be needed to change your vote to a +1?
15:41:42 <mboddu> Or that, thanks sgallagh
15:41:44 <zbyszek> mboddu: it's hard. I expect the document to be ready at best by 20:00 european time.
15:41:55 <decathorpe> Significantly shift the cost-benefit balance towards benefit, which I do not see happennig
15:41:56 <nirik> sadly our stg env isn't really up to testing this sort of thing...
15:42:08 <mhroncok> sgallagh: an actual detailed description being in the proposal, going trough the coomunity review on the devel mailing list
15:43:08 <zbyszek> sgallagh: what decathorpe said. A careful analysis of costs (not just "no effect", which is clearly not true, even if the effect is small), and an analysis of the benefits. Checking off "have ima" is not a benefit.
15:43:35 <sgallagh> puiterwijk: Do we have any partners lined up planning to take advantage of this?
15:43:46 <sgallagh> That would count heavily towards a benefit.
15:43:50 <mboddu> I can tell from releng/mass rebuild POV, mass rebuild isn't going to start today (gcc issues), do you guys think having two days is good enough for answering all the concerns?
15:44:02 <puiterwijk> sgallagh: yes. We have several projects ongoing that want to use this.
15:44:03 <nirik> well, I think it's clear (but perhaps not from the change) that iot wants to use this to secure iot devices and make it available for other parts of fedora to use
15:44:17 <puiterwijk> Among which Keylime, for which Luke Hinds also commented on the devel-list
15:44:18 <dcantrell> sgallagh: what the others have said.  examples and links to where I can find information on this upstream.  I am hesitant to believe a change like this won't have a huge impact somewhere, so I'd like to understand it more
15:44:25 <zbyszek> puiterwijk: then this should be part of the description!
15:44:28 <puiterwijk> And we really want this for IoT
15:44:54 <nirik> so, perhaps update change, ask for more feed back on list and vote in ticket?
15:45:01 <puiterwijk> (which was on the change page, in fact, as "aligns with IoT")
15:45:01 <zbyszek> mboddu: it's hard to say
15:45:01 <decathorpe> puiterwijk: IoT is not mentioned once in the Change proposal :)
15:45:07 <zbyszek> nirik: +1
15:45:09 <puiterwijk> decathorpe: pretty sure it is mentioned there
15:45:14 <cverna> nirik: +1
15:45:34 <puiterwijk> huh... It was in my draft
15:45:40 <decathorpe> ok, it's "Internet of Things", but not IoT
15:46:08 <puiterwijk> Right. Since it's the objective name that I think I copied
15:48:08 <sgallagh> nirik: OK, but given that this is heavily dependent upon the mass rebuild, can we also get the ticket treated under the FastTrack process?
15:48:11 <zbyszek> puiterwijk: I think the feedback seems more negative then it really is. The problem is that this change has a distro-wide impact, and people like to have time to dig through details for something like that.
15:48:40 <nirik> sgallagh: that sounds good to me.
15:49:18 <nirik> I would personally like to get this in... I hope everyone with -1's can review/ask questions as soon as things are updated and that puiterwijk can update things soon. ;)
15:49:34 <sgallagh> puiterwijk: Can you get that updated ASAP (like, within the next couple hours?)
15:49:36 <dcantrell> zbyszek: true.  it's usually not a good idea to rush distro-wide changes.  kind of part of the whole reason we do the change process is to think through all the impact scenarios and arrive at plans on how to deal with them.
15:49:40 <puiterwijk> sgallagh: yep, absolutely.
15:50:16 <mhroncok> to clarify: I cannot vote +1 until this change proposal has some details and until the details go thru community feedback
15:50:22 <mhroncok> that won't happen today
15:50:25 <sgallagh> OK, then please send a message to devel@ when it's ready and I'll open the FastTrack voting on the ticket.
15:50:44 <decathorpe> and if you find the data for size increase regarding FS / rpmdb / binary RPMs again, that would be great
15:50:44 <puiterwijk> sgallagh: will do
15:50:45 <dcantrell> I agree with mhroncok here.  This is the kind of change that really does need some time on the devel list for wide feedback
15:50:59 <puiterwijk> decathorpe: absolutely, will do.
15:51:00 <mhroncok> that's why system wide changes have deadlines
15:51:26 <sgallagh> mhroncok: If this doesn't go in this week, it misses F34 and, by extension, likely RHEL 9
15:51:41 <mhroncok> if we want this for f34 mass rebuild, the only option would be to delay the mass rebuild
15:52:00 <mhroncok> sgallagh: not good reasone nough to vote +1
15:52:05 <mhroncok> *reason enough
15:52:34 <puiterwijk> mhroncok: well, or delay the signing of the mass rebuild, technically. We could easily just turn off signing for it until we have votes, and then enable it when a decision is made. It would allow at least the mass rebuild itself to continue.
15:52:35 <sgallagh> mhroncok: That's worth exploring as well.
15:52:35 <nirik> well, the mass rebuild is already delayed a bit...
15:52:43 <puiterwijk> (since it's at sign moment)
15:52:44 <sgallagh> Also with GCC issues, we may have some delay anyway,
15:52:57 <sgallagh> But I'd prefer not to push it a full week and wait until the next meeting.
15:53:09 * nirik nods. sgallagh +1
15:53:17 <dcantrell> sgallagh: I would rather see us focusing on what the proposal needs and when the discussion will land on the mailing list than holding ourselves hostage to various deadlines
15:53:32 <mhroncok> my +1 requires a full week communtiy review of a detailed change proposal
15:54:00 <mhroncok> you are obviously free to rush things, but without my +1
15:54:07 <sgallagh> mhroncok:understood
15:55:24 * nirik notes it has been on the list... perhaps people didn't ask questions they had? or did they not get answers?
15:56:05 <decathorpe> AFAICT the concerns that were raised during today's meeting were not addressed on the list.
15:56:16 <mhroncok> well, the proposal has not been updated at all since announcng
15:56:20 <zbyszek> so... the deadline for changes requires mass rebuild was 29/12, and the change page was submitted in early January. Thus the time squeeze is not unexpected.
15:56:33 <mhroncok> zbyszek: for system wide changes even sooner
15:56:54 <zbyszek> mhroncok: according  https://fedorapeople.org/groups/schedule/f-34/f-34-key-tasks.html it's the same
15:57:06 <mhroncok> appologies
15:57:18 <mhroncok> I've mixed it with Changes requiring infrastructure changes
15:57:33 <mhroncok> which arguably this one is, but the changes were ready, so meh
15:58:06 <nirik> decathorpe: not addressed on the list or not raised on the list?
15:58:16 <decathorpe> the former.
15:59:03 <mboddu> I just wanna add that the mass rebuild is delayed and the full rebuild takes about 3-4 days, so we sorta have about a week for this
15:59:42 <sgallagh> mboddu: How so?
15:59:43 <nirik> mboddu: I did not think the delay was a week...
15:59:53 <sgallagh> It needs to be ready before the rebuild *starts*
16:00:05 <mboddu> sgallagh: Nope, it needs to be ready before we mass tag them
16:00:35 <sgallagh> oh?
16:00:39 <nirik> mboddu: oh, you are saying to wait and sign them all at the end? thats gonna make it slow, no?
16:00:47 <mhroncok> technically, this can land in RHEL 9 even if it gets delayed to F35, if that's the reason to rush this
16:01:00 <mboddu> So, we generally sign the rebuilds while we rebuild, but if we sign after the rebuild it might be slow, but it gives time
16:01:06 <mboddu> What nirik said
16:01:15 <King_InuYasha> but IMA is signed during the build
16:01:21 <nirik> yeah, that would add a day or so.
16:01:22 <King_InuYasha> since it's the contents of the RPM
16:01:31 <nirik> or mroe
16:01:40 <nirik> puiterwijk: ^ can you address the above?
16:01:51 <puiterwijk> King_InuYasha: no, IMA signatures are added to the rpm during rpm-sign operation
16:01:58 <King_InuYasha> how?
16:02:03 <puiterwijk> It's in the RPM signature header, and RPM puts them in place during install
16:02:12 <King_InuYasha> X_X
16:02:19 <puiterwijk> The file signatures are in the rpm headers
16:02:30 <King_InuYasha> wait, that means robosignatory and sigul need adjustments
16:02:40 <puiterwijk> They already have those. But yes
16:02:53 * King_InuYasha grumbles
16:03:11 <King_InuYasha> that means this requires infra changes too to enable it, where are those?
16:03:26 <sgallagh> OK, we're an hour into this topic and we have enough votes to table it for the moment.
16:03:55 <sgallagh> #action puiterwijk to update the Change Proposal with more details
16:04:21 <sgallagh> #info Once the Change is updated, FESCo will re-evaluate and may vote on the ticket following the FastTrack process
16:04:35 <zbyszek> ... and notify fedora-devel
16:04:46 <mhroncok> proposal: Once the Change is updated, it goes for one week review on devel
16:04:57 <decathorpe> sgallagh: we'd have had enough votes to reject it, too, but apparently people still want this in F34 at all cost ...
16:05:04 <sgallagh> zbyszek: +1
16:05:13 <nirik> zbyszek: +1
16:05:15 <nirik> mhroncok: -1
16:05:29 <cverna> zbyszek: +1
16:05:51 <sgallagh> @action puiterwijk will notify fedora-devel when the Change is updated
16:05:54 <sgallagh> #action puiterwijk will notify fedora-devel when the Change is updated
16:06:17 <sgallagh> mhroncok: -1
16:06:38 <dcantrell> I give mhroncok +1 here
16:06:57 <dcantrell> if we don't give the community time to review the revised proposal, then what's the point of updating the change proposal?
16:07:18 <nirik> We have not in the past required a week additional feedback on other changes, why here?
16:07:24 <King_InuYasha> we could just delay mass rebuild by another week
16:07:30 <King_InuYasha> it's not like it'll make a difference
16:07:32 <nirik> we have often asked people to update things and re-reviewed them...
16:07:36 <mhroncok> nirik: becasue this change is (close to) empty now
16:07:38 <dcantrell> nirik: I would say the distro impact by this change warrants the additional time
16:07:53 <mhroncok> it needs to be written, not slightly adapted
16:07:58 <zbyszek> I'll vote -1 for mhroncok's proposal: I think fedora-devel *might* be fine with a shorter discussion period. Let's see where it goes. If there are outstanding questions, I'll vote -1 preventing fasttrack approval.
16:08:12 <sgallagh> zbyszek: Exactly as I'd hope
16:08:16 <King_InuYasha> okay then
16:08:25 <King_InuYasha> sgallagh: +1
16:08:25 <dcantrell> ok, I think that's a reasonable compromise
16:08:31 <mhroncok> so, let me clarify
16:08:50 <mhroncok> the change is updated, posted to devel and immediately voted by fesco
16:09:12 <mhroncok> in fast track. so we'll need +7 to approve it, otherwise it waits for the whole week anyway
16:09:32 <sgallagh> I think it's reasonable to assume we'll give it at least 24 hours to soak
16:09:43 <sgallagh> But we can state that explicitly if you wish
16:09:46 <mhroncok> so the +1s will be added by people who think the discussion on devel was ssignficat enough already
16:11:08 <mhroncok> but as long as at least one fesco member posts -1, we are back at "wait an extra week for the next meeting"
16:11:27 <sgallagh> mhroncok: And you plan to do so to force the delay?
16:11:48 <mhroncok> sgallagh: it has crossed my mind, but I don't want to be that person
16:12:06 <mhroncok> but obviously, if I see that the proposal lacks details, I'll vote against
16:12:18 <sgallagh> I wouldn't want you to do otherwise.
16:12:45 <mhroncok> honestly, my opinion now is that this would better be retargetted to f35
16:13:23 <sgallagh> mhroncok: I trust you to use good judgement on whether, after the proposal is updated, it should be deferred.
16:14:17 <mhroncok> anyway, we know what happens next
16:14:27 <sgallagh> I move on to the next topic>?
16:14:31 <mhroncok> yes
16:14:36 <zbyszek> yes
16:14:42 <sgallagh> #topic #2545 F34 Change: Localization measurement and tooling
16:14:42 <sgallagh> .fesco 2545
16:14:43 <zodbot> sgallagh: Issue #2545: F34 Change: Localization measurement and tooling - fesco - Pagure.io - https://pagure.io/fesco/issue/2545
16:15:01 <zbyszek> This case is similar to a large extent.
16:15:38 <zbyszek> It's a noticable commitment of resources, and it's not entirely clear how it fits into the overall picture of Fedora infrastructure.
16:16:50 <zbyszek> Releng has been shedding projects, for good reasons. So before we add one more, we should be clear on the benefits and long term support.
16:16:51 <sgallagh> I'm frankly inclined to give them approval to work towards it and leave the final decision to Infra to determine if they have the resources to deploy it.
16:17:04 <sgallagh> deploy+maintain
16:17:48 <zbyszek> Should we open a ticket to ask infra?
16:18:03 <mhroncok> we can approve conditionally
16:18:07 * nirik is in another meeting,
16:19:04 <zbyszek> I'm not ready to vote +1 yet. There has been additional talk on the mailing list and I haven't had time to process it yet.
16:19:30 <sgallagh> This isn't as urgent, so I'm okay with deferring this for a week. Anyone else?
16:19:36 <sgallagh> Proposal: defer for a week
16:19:38 <sgallagh> +1
16:19:39 <mhroncok> sgallagh: ack
16:19:41 <dcantrell> +1
16:19:47 <zbyszek> +1
16:19:57 <decathorpe> ack
16:20:19 <sgallagh> OK, let's move on
16:20:21 <mhroncok> to clarify: my ack == +1, I just didn't think such decisions require a counted vote
16:20:25 <sgallagh> #topic #2543 F34 Change: Unify the GRUB configuration files location across all supported architectures
16:20:25 <sgallagh> .fesco 2543
16:20:26 <zodbot> sgallagh: Issue #2543: F34 Change: Unify the GRUB configuration files location across all supported architectures - fesco - Pagure.io - https://pagure.io/fesco/issue/2543
16:20:56 <sgallagh> mhroncok: They don't
16:21:08 <sgallagh> Just a lack of dissent
16:21:23 <mhroncok> sgallagh++
16:21:30 <mhroncok> about this grub change
16:21:53 <mhroncok> I abstain, 0
16:21:56 <sgallagh> I'll be honest, my eyes tend to glaze over when trying to read bootloader Changes.
16:22:07 <mhroncok> that
16:22:09 <decathorpe> I'm not sure about this one. While making things consistent, it doesn't address the worst issues of our grub setup
16:22:39 <King_InuYasha> the real reason this was filed was to unblock efforts to migrate from lorax to osbuild
16:23:10 <King_InuYasha> osbuild produces hybrid BIOS+UEFI images like other distros do, and in order for that to work properly, we need unified config locations
16:23:19 <zbyszek> I'm willing to defer to grub developers on this. I don't have a deep understanding of the tradeoffs involved.
16:23:41 <dcantrell> javier is one of our grub devs.  I'm +1
16:23:47 <nirik> I am +1 here
16:23:55 <King_InuYasha> I don't like it, but I'm +1
16:24:22 <sgallagh> Yeah, my +1 in the ticket was pretty much entirely due to it coming from GRUB developers.
16:24:22 <King_InuYasha> I think this is the wrong way to do it, but there's basically no way to fix this since GRUB is half-dead upstream
16:24:43 <sgallagh> King_InuYasha: Is there a replacement for it?
16:24:54 <King_InuYasha> we're not allowed to consider replacements
16:25:04 <King_InuYasha> because of secure boot shackles
16:25:04 <sgallagh> Who is "we"?
16:25:10 <King_InuYasha> Fedora
16:25:33 <King_InuYasha> no other bootloader in the ecosystem implements secure boot verification
16:25:34 <sgallagh> OK, topic for another day
16:25:54 <King_InuYasha> and there's the whole BIOS/UEFI thing
16:25:59 <King_InuYasha> but anyway
16:26:17 <King_InuYasha> they say this is the only thing they can do, even if it is not necessarily a good strategy
16:26:20 <King_InuYasha> so meh, +1
16:26:25 <sgallagh> zbyszek: Did your deference equate to +1?
16:26:35 <decathorpe> I agree. meh, +1
16:26:55 <King_InuYasha> (I guess refind would be another option, but we'd need a BIOS->UEFI shim)
16:27:17 <zbyszek> sgallagh: yeah, I voted +1 in the ticket
16:27:42 <sgallagh> #agreed Unification of GRUB config files is accepted (+6, 1, -0)
16:27:50 <sgallagh> #topic Next week's chair
16:28:07 <sgallagh> Who wants it?
16:28:20 <King_InuYasha> I'll do it
16:28:25 * mhroncok promises to chair some meetings in april+, after he is moved to the new house
16:30:11 <sgallagh> #action King_InuYasha to chair the next meeting
16:30:19 <sgallagh> #topic Open Floor
16:30:24 <sgallagh> Watch your step
16:30:29 * King_InuYasha falls
16:31:13 <zbyszek> .o/
16:31:20 <zbyszek> https://pagure.io/fesco/fesco-docs/pull-request/41
16:31:55 <nirik> lets get it merged. ;)
16:32:19 <zbyszek> OK, I'll merge it tomorrow if nobody votes -1 in the pull request.
16:32:34 <zbyszek> *** Last chance ***
16:32:52 <sgallagh> I nearly merged it last night just to end the discussion
16:33:02 <sgallagh> It's not as if we can't fix it later
16:33:02 <mhroncok> :)
16:33:05 <zbyszek> (And then we can start discussing actual changes to the policy :))
16:33:32 <nirik> thanks for taking that over the finish line. :)
16:33:41 <sgallagh> Anyone opposed to just merging it *right now*?
16:33:54 <nirik> there is one change in these changes...
16:33:54 <mhroncok> I'd like to discuss if pushing obviously broken or WIP commits to dist git should be disallowed, I'll probbaly open a topic on devel first
16:33:59 <decathorpe> crickets
16:34:16 <zbyszek> mhroncok: I think that's a separate issue.
16:34:33 <mhroncok> zbyszek: I agree, but I wonder whther it belong shere, or to packaging guidelines, or where
16:34:38 <nirik> the bit about allowing releng to untag packages that have gone out in composes if deemed needed. We could drop that if people think we should discuss it more
16:34:49 <sgallagh> mhroncok: FWIW, I agree. Pushes to dist-git should follow the same rules as Bodhi updates
16:35:02 <sgallagh> Nothing goes in except that which is expected to be pushed stable
16:35:03 <zbyszek> I think the policy should be "it's discouraged. If you need to and do it, and somebody rebuilds your package in an unwanted way, you're responsible for fixing things."
16:35:35 <zbyszek> (it == WIP commmits)
16:35:35 <mhroncok> zbyszek: obviously, side tag builds are bad with this, becasue you ned to push before building it
16:35:44 <nirik> "always have your package in a ready to build state"
16:36:13 <mhroncok> aka if I commit to python3 to update to 3.10 and do a mass side tag rebuild, a provenpackager can rebuild python for a libfoo bump and boom
16:36:21 <decathorpe> mhroncok: obviously, we need to wrap packages in Mutexes :)
16:36:23 <sgallagh> nirik: Was your question a request for me not to hit the merge button?
16:36:29 <mhroncok> decathorpe: exactly :D
16:36:30 <sgallagh> Because I've got my finger on the trigger
16:36:44 <mhroncok> hit it
16:36:58 <mhroncok> it's not like we cannot revert it entirely if there is a typo
16:37:04 <nirik> I'm fine with that change. ;) It was a chance for others to not want it. ;)
16:37:15 <sgallagh> Time's up, it's merged.
16:37:24 <sgallagh> 🥳
16:37:32 <mhroncok> \o/
16:37:40 <nirik> if our tooling was better, side tags could build from a branch, then merge to main after the side tag is merged.
16:37:53 <decathorpe> not even joking, I thought about proposing something like "do not touch this package" locks exactly for things like making side-tag builds race-free
16:38:00 <mhroncok> nirik: it is possible to do this mnaually
16:38:06 <nirik> mhroncok: yep.
16:38:18 <mhroncok> nirik: but a merge conflict happens if somebody bumps it in the meantime
16:38:19 <nirik> just will leave you with a bunch of branches or releng requests to remove them
16:38:28 <nirik> that too yeah.
16:38:46 <nirik> because changelog/release... and so the circle is complete
16:38:50 <mhroncok> decathorpe: I can already see the problems :D
16:39:13 <decathorpe> dreaded deadlocks and starving packagers
16:39:29 <sgallagh> starving packagers with dreadlocks?
16:39:51 <sgallagh> OK, any other topics or can we call it a day?
16:40:31 <decathorpe> call it
16:40:42 <mhroncok> release us...
16:41:32 <sgallagh> #endmeeting