15:00:32 #startmeeting FESCO (2018-03-09) 15:00:32 Meeting started Fri Mar 9 15:00:32 2018 UTC. The chair is zbyszek. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:32 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:32 The meeting name has been set to 'fesco_(2018-03-09)' 15:00:32 #meetingname fesco 15:00:32 The meeting name has been set to 'fesco' 15:00:38 .hello2 15:00:39 zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' 15:00:46 #topic Roll Call 15:00:54 Oh, I'm not chaired. 15:01:00 morning 15:01:02 #chair maxamillion dgilmore nirik jsmith sgallagh bowlofeggs tyll jwb zbyszek 15:01:02 Current chairs: bowlofeggs dgilmore jsmith jwb maxamillion nirik sgallagh tyll zbyszek 15:01:06 #topic Roll Call 15:01:11 .hello2 15:01:12 sgallagh: sgallagh 'Stephen Gallagher' 15:01:14 .hello2 15:01:15 zbyszek: zbyszek 'Zbigniew Jędrzejewski-Szmek' 15:02:13 .hello2 15:02:14 maxamillion: maxamillion 'Adam Miller' 15:02:24 dgilmore jsmith bowlofeggs tyll jwb: fesco meeting in -2 15:02:36 apologies for my absence the last few meetings, I have a newborn and things are a bit crazy at my house right now 15:02:38 hi 15:02:53 i'm double booked, but i should be able to pay attention 15:02:54 .hello2 15:02:55 jsmith: jsmith 'Jared Smith' 15:03:34 Hi 15:03:46 Hello everyone, we have quorum 15:03:57 I'll wait a minute more, and start 15:04:19 Quorums are good :-) 15:04:35 spelling is a bitch though ;) 15:04:57 words are hard 15:05:13 OK, let's go. 15:05:16 = Followups = 15:05:16 #topic #1853 F29 System Wide Change: Enable dbus-broker 15:05:17 .fesco 1853 15:05:17 https://pagure.io/fesco/issue/1853 15:05:18 zbyszek: Issue #1853: F29 System Wide Change: Enable dbus-broker - fesco - Pagure - https://pagure.io/fesco/issue/1853 15:05:39 I think we need to postpone this again. There has been no reply from Change Owners. 15:05:59 One is on PTO... 15:06:32 +1 to defer another week 15:06:33 is there anything by way of the schedule that would make it problematic to postpone again? 15:06:34 * nirik is fine with more (any?) discussion on it. 15:06:38 There's plenty of time to decide on it before F29 15:06:47 Yeah, no hurry whatsoever. 15:06:52 .hello2 15:06:54 (sorry for my tardiness - lost track of time) 15:06:55 bowlofeggs: bowlofeggs 'Randy Barlow' 15:07:08 Hi! 15:07:13 #topic #1845 389-ds-base and freeipa on 32 bit arches 15:07:13 .fesco 1845 15:07:13 https://pagure.io/fesco/issue/1845 15:07:14 zbyszek: Issue #1845: 389-ds-base and freeipa on 32 bit arches - fesco - Pagure - https://pagure.io/fesco/issue/1845 15:07:31 Do we have any news here? 15:07:39 i don't think so 15:07:45 zbyszek: i tried to follow up on this one but i don't feel like i have the info 15:07:47 I haven't heard anything :-( 15:07:50 * nirik checks bugs 15:07:50 they also still did not file a change 15:07:54 i asked them to 15:07:55 :/ 15:08:04 note that there's a bug linked from the bug 15:08:19 bugs on bugs on bugs 15:08:26 https://bugzilla.redhat.com/show_bug.cgi?id=1530832#c9 has more info 15:09:12 At least they think it's just i686 that is affected, and not 32-bit ARM 15:09:14 So it sounds like A) only i686 and B) only in newly-written code since F27 15:09:26 So there's no way to know yet if it has real-world impact 15:09:37 But their tests leave them without a warm fuzzy feeling 15:09:37 we have the i686 sig 15:09:43 can we ask them to care about this and move on? 15:09:50 or was that just about the kernel? 15:10:14 That's a good idea. If i686-sig does not care, then maybe the issue is overblown (or i686-sig does not exist ;)) 15:10:28 either way a change should be filed 15:11:18 proposal: kick the issue to i686 sig, and simultaneously get an f28 change filed. the change can be withdrawn if i686 sig can fix this in time 15:11:26 hm 15:11:37 bowlofeggs: +1 15:11:46 bowlofeggs: +1 to your proposal, but what if they don't file a change? 15:11:47 bowlofeggs: +1 15:11:47 well, it's still murky to me where the issue is... 15:11:47 if it's just i686, why did they disable arm? 15:11:52 or did they not do that 15:11:59 jwb: Exactly what I was trying to get to the root of... 15:12:25 If they have to write up a change page, it'll have to include those things. 15:12:35 jsmith: can we just decree that the packager has to file the change? 15:12:55 i don't want to kick an issue to the i686 sig if it isn't truly i686-only. that would be mean, for a sig that is rather understaffed/ethereal 15:12:55 also note that they blocked the x86 arch blocker bug...which if there is a i686 sig, they should be watching. ;) 15:12:59 basically say they either have to file the change or bring 686 back? 15:13:08 jwb: My gut feeling was that they thought it was a problem with 32-bit atomics and just disabled all 32-bit arches 15:13:11 jwb: they are saying it is 686 only 15:13:17 bowlofeggs: Sounds fair :-) 15:13:25 jwb: https://bugzilla.redhat.com/show_bug.cgi?id=1530832#c9 15:13:32 "apparently" 15:13:39 i guess that's not the strongest statement 15:14:15 this is all "we have a problem. we don't really care about the problem where it surfaces, let's punt on it entirely" 15:14:16 they only excluded i686 from the rawhide package 15:14:18 jwb: i think we can kick it to i686 with no penalty if they don't fix it 15:14:33 basically alert them of the problem - if they want to let it get dropped, fine 15:14:45 bowlofeggs, yes, but they don't actually understand the issue, do they? they're just dropping 32-bit 15:14:47 but still give them an opportunity to get involved and keep it working 15:14:50 let it be their choice 15:15:08 jwb: it does sound like the issue isn't fully understood to me 15:16:14 jwb, nirik, vote on bowlofeggs proposal? 15:16:22 bowlofeggs, right. and since it isn't understood and they've punted and they also included ARM, i don't think we can say "it's the i686 SIGs" problem 15:16:28 -1 15:16:57 proposal: FESCo leaves this to the maintainers of the packages and architectures to sort out. 15:16:58 I'm not all that in favor... I think we should have the tools folks press them for the real issue so it gets fixed... but we can't really force that 15:18:07 jwb: so ARM is included? i thought they were saying it was only i686 15:18:21 oh but i guess arm is why dgilmore filed in the first place? 15:19:12 https://src.fedoraproject.org/rpms/389-ds/blob/master/f/389-ds.spec: 15:19:18 # Exclude 32 bit arches 15:19:18 ExcludeArch: %{ix86} 15:19:33 So the comment does not match the line below, but it's just i686 right now 15:19:39 haha 15:19:55 I think it did also filter out armv7 before 15:19:59 i think they did include arm, then when people complained they dropped it. again, because they don't have root cause 15:19:59 But they might've re-added that later 15:20:10 i would be more worried if all 32-bit were excluded. not to sound mean to 386, but if it's just 386 that does seem less bad to me 15:20:36 which is why i said just leave it to the maintainers. FESCo has nothing they can do here 15:20:39 OK, since they only seem to be excluding i686, I'm all for letting them fiture it out. 15:20:59 yeah, I'm with bowlofeggs in that I thought it to be i686 only but it's possible I'm lacking context 15:20:59 zbyszek: huh, 389-ds-base has i686... 15:21:12 nirik: lol? 15:21:28 yeah i can be ok with that - and i think it could be good ot specifically alert the 686 sig just for their info (i.e., not requiring anything of them, just a friendly highlight of the issue) 15:21:46 they do need a change i think 15:21:48 389-ds has no ExcludeArch in F28 15:22:43 But 389-ds-base has ExcludeArch: i686 15:22:51 (as nirik said) 15:23:02 my worry about this entire thing is that there is a subtle toolchain bug, which ideally would be found and fixed, but it seems this is hard to isolate. 15:23:11 yeah it does look like the exclude is only on rawhide 15:23:22 https://src.fedoraproject.org/rpms/389-ds/blob/f28/f/389-ds.spec 15:23:28 no exclude there 15:23:35 but it is here: https://src.fedoraproject.org/rpms/389-ds/blob/f28/f/389-ds.spec 15:24:06 bowlofeggs: those are the same links, no? 15:24:12 * jsmith just got confused 15:24:14 whoops 15:24:15 haha 15:24:20 https://src.fedoraproject.org/rpms/389-ds/blob/master/f/389-ds.spec 15:24:21 sorry 15:24:25 bowlofeggs: it's excluded in f28 for 389-ds-base 15:24:30 https://src.fedoraproject.org/rpms/389-ds-base/blob/f28/f/389-ds-base.spec#_5 15:24:30 that was a middle click vs. ctrl-v error 15:24:41 But 389-ds Requires 389-ds-base so that cannot work 15:24:53 oh different package 15:24:53 right 15:25:17 I guess 389-ds is not important at all, it's a noarch package. 15:25:44 389-ds-base is what counts, and that has ExcludeArch in F28 and rawhide 15:26:00 we've spent almost 20 minutes on this. we have nothing we can actually do about it. why are we still discussing here? 15:26:47 proposal: FESCo requires a change for f28, the packagers are otherwise free to sort it out 15:27:42 +1 15:27:52 zbyszek: +1 15:28:20 sure, +1. 15:29:00 note that there might be also releng changes here at some point... since the i386 server images will stop composing. They aren't blocking, but if they don't work we should remove them/stop trying to make them at some point. 15:29:09 +1 15:29:10 +1 15:29:30 nirik: Good point, but I don't know what else we can do 15:29:31 nirik: oh true. that should be noted in the change, which should be system-wide 15:30:03 well, we can remove them after beta, just means some work 15:30:18 #agree FESCo requires that a Change be filed for F28, and involved are otherwise free to sort out the issue (+6, 0, -0) 15:30:21 #info If the change is accepted, releng changes will be required 15:30:47 Another "nice" topic: 15:30:52 #topic #1820 Adjust/Drop/Document batched updates policy 15:30:52 .fesco 1820 15:30:52 https://pagure.io/fesco/issue/1820 15:30:56 zbyszek: Issue #1820: Adjust/Drop/Document batched updates policy - fesco - Pagure - https://pagure.io/fesco/issue/1820 15:30:59 fun! 15:31:08 Yay 15:31:10 note that there were comments this morning just right before the meeting. 15:31:21 I made a proposal in the ticket. 15:31:21 nirik: can you summarize the results of the experiment over the last week? 15:31:38 and in fact some just now _during_ the meeting 15:31:41 zbyszek: I can tell you that I've only seen stable be pushed out on Monday if I recall correctly 15:31:58 (not counting Tuesday, and the Monday one I did because of the dhcp one) 15:32:30 zbyszek: as puiterwijk said (he was on push duty)... 15:33:10 yeah we can def make the changes requested by sgallagh pretty easily 15:33:19 I think we might step back a second and look at our goals here. My understanding is there were 2: 1) less "flood" of updates for users, 2) less metadata changes for users. 15:33:33 i also found a few deficiencies around severity in bodhi this week which i've been working on 15:33:49 like the CLI not supporting them at all (PR filed for that yesterday) 15:33:58 nirik: Also "less frequent syncs to mirrors" 15:34:11 nirik: and also 3) a way for QA to test the batch 15:34:19 sgallagh: sure, I guess that falls under 2 somewhat... 15:34:22 QA doesn't do that yet, but it would be possible now 15:34:56 i don't know if QA wants to do that 15:35:00 haha 15:35:08 yeah, that perhaps should be something we find out. ;) 15:35:29 I know the OSCI people really want that to happen and are working towards it 15:35:49 towards testing batches? 15:35:57 I believe so, yes 15:36:01 Going back to nirik's goal #1 and goal #2, I think sgallagh's proposal moves us towards those two goals 15:36:05 sgallagh: news to me 15:36:27 well, the delta repodata would help 2 a fair bit (but that is only in development) 15:36:31 hmm, I thought I remembered dperpeet and rsibley mentioning it. But it's been a while. 15:36:41 Dunno, I have to agree with kkofler that batched is not really working right now. 15:36:53 zbyszek: I agree it's not working in its current incarnation 15:37:06 I postulated that the reason for that is because it's implementation didn't take best advantage 15:37:12 If the delta repodata solves #2, then #1 is not enough of motivation, because it is better done on the client side. 15:37:13 I think a user end batching might make more sense for users, barring us testing them... which I think couldbe very valuable 15:37:29 But we would be able to test them the same way 15:37:45 do note that all/most workstation users don't see the 'flood of updates' already, since it's batched on client end. 15:38:01 one nice thing i've noticed this week is that i get more drpms 15:38:11 +1 15:38:11 that's another subtle benefit of batching 15:38:13 I mean we could install a default dnf-install-batched.timer that fires on (say) Tuesday, and QA would test that batch. 15:38:16 I do like drpms 15:38:18 bowlofeggs: +1 15:38:18 (i don't update every day) 15:38:26 (so i used to not get very many drpms) 15:38:33 bowlofeggs: I did notice more drpms too 15:38:36 (same, I update weekly) 15:38:42 That's an advantage we haven't made clear. 15:38:54 yeah it dind't occur to me until just yesterday 15:38:55 It's a significant decrease in download usage 15:39:05 I used to update every day (and still do on my rawhide laptop), but I don't on my other machines anymore... I just update once a week or so. 15:39:07 Useful for those on metered or slow connections, certainly 15:40:02 i like the idea of a fasttrack repo, i just don't know if we can do it infra wise 15:40:04 like, resource wise 15:40:12 or if the mirrors would bear it storage wise 15:40:16 zbyszek: the timer gets into 'do we auto apply updates' also? 15:40:29 sgallagh: yeah, on that note apparently metered connections are making a comeback ... I've noticed it in the small print of a lot of recent local offers, so that might become a rising concern for users 15:40:47 which honestly surprised me 15:40:57 I think the fasttrack thing adds a lot of complexity and resources for... probibly not too many people. 15:41:19 nirik: yeah that's a good counter point 15:41:23 it's hard to say for sure of course 15:41:56 ultimately on the note of things, I think "bugfixes batch" and "security updates go out immediately" is a good stance, regardless of security severity 15:42:13 i actually think most security updates aren't important 15:42:17 some security fixes are very minor. 15:42:18 The hardest part of a decision like this is that we will *never* hear from the people it benefits. 15:42:18 maxamillion: I am not really sure I'd agree with that general a statement on security bugs.. 15:42:21 most updates aren't important 15:42:32 But we will never *stop* hearing from the people who disagree. 15:42:36 sgallagh: yep 15:42:46 most of them are very difficult to exploit and also give very little benefit when they are exploited 15:42:56 they might not be, but who's going to "grade" them and make that decision for all packages across the distro at all times? 15:42:57 maxamillion: I definitely disagree with that statement on security bugs. 15:43:06 also, it's not just secuirty that can be worthy of skipping to stable - think of a real bad data eating bug 15:43:08 maxamillion: RH prodsec does so... 15:43:14 That's the data I've been using last week 15:43:18 so a bugfix would also warrant going out right away if it's real bad 15:43:19 can you use the gnome-software batching without installing a pile of gui? 15:43:20 If only because we have those at least every 2-3 days. It would render batching basically meaningless. 15:43:27 puiterwijk: RH prodsec does that for all of Fedora? 15:43:32 Look in RHBZ's for ecurity bugs, and tehre's a whiteboard with severity=(low|moderate|important|critical) 15:43:43 maxamillion: No, only those things RH also ships. 15:43:52 then I think it's a moot point 15:43:54 But honestly, that's 99% of the stuff that *really* matters. 15:43:58 sgallagh: not per se. I've seen enough prodsec bugs for things that aren't in products 15:44:06 interesting 15:44:10 They follow external things like oss-security and NVD 15:44:18 Let's settle for "only guaranteed for" 15:44:23 True on that 15:44:50 the CVE itself usually has a score too 15:45:04 But there's also other sources, like the CVE database and NVD, which have the CVSS score... 15:45:10 so as another datapoint: 15:45:31 So tehre's a lot of people checking the importance of security issues and publishing them in useful databases 15:45:34 fedora infra has all fedora installs we do setup with dnf autoupdate for security stuff. 15:45:49 But looking at the list from this week, I think that clamav is something that should not be batched, and php def too 15:46:24 to pick a random machine, it's applied them 13 times so far this year. 15:46:36 nirik: that is interesting 15:46:49 * misc also have autoupdate on most servers 15:46:51 That averages to one a week 15:47:02 sgallagh: a possible amendment to your proposal from the ticket: we could also require a justification from packagers when they pick "urgent" on the severity 15:47:04 err, a bit more than that, sorry 15:47:11 I guess my main point is that these entities that may or may not "score" things to give us a metric by which we can mark a security update as "important" or "not important" (or some variant of those) are not Fedora contributors, which means that the responsibility will eventually fall on us or others in the community and I don't think anyone just has a pile of spare cycles sitting around 15:47:12 but that was bodhi-backend01, machines with more packages will have more 15:47:14 zbyszek: note that for the clamav one, there was a few hours between batched and stable, so I didn't even get the time to look at that 15:47:15 bowlofeggs: I could be fine with that. 15:47:17 sgallagh: like a text box where they type why it's important, and releng can read it 15:47:24 zbyszek: same for the PHP one... 15:48:33 proposal: implement the severity stuff proposed in bodhi, ask QA about batch testing plans, go back to normal pushes, revisit next week and keep discussing plans ? 15:48:36 bowlofeggs: who's going to audit the justifications? who has authority to say "yes, that's a valid justification" or "no, that's not" 15:48:58 or perhaps if we don't know what we want to do in bodhi yet, drop that 15:49:02 maxamillion: perhaps the push duty person? 15:49:04 nirik: I think that's reasonable. Consider me +1 15:49:29 bowlofeggs: I think that's infeasible in the long term, just too much work 15:49:40 maxamillion: I'd say we trust the packagers but if the updates they submit consistently get marked as Urgent, maybe someone can take a look and if in doubt, submit to fesco? 15:49:43 nirik: i can +1 with the exception that i can't do that myself in the next week (too much modularity and container and other things on my plate) 15:49:52 alright 15:49:57 (and the data on "who marks things as urgent a lot" is easy to gather) 15:50:03 bowlofeggs: fair, we can drop the bodhi part until we decide more anyhow 15:50:08 I think the whole thing is silly, but consensus says otherwise so I'm going to back away from this item 15:50:42 the bodhi changes aren't too hard, though they technically are backward incompatible so would require an x release bump, unless we carry a fedora-specific patch 15:50:53 puiterwijk: I think that's a fair middle-ground. 15:50:58 or maybe i could make it a setting which severity levels are allowed... 15:51:13 maxamillion: the whole thing being batched updates? or requiring some kind of justification for every update? 15:51:17 bowlofeggs: Let's leave the justifications out for now until and unless it becomes obvious they're needed 15:51:22 sure 15:51:36 Or, in other words, "assume positive intent" :-) 15:51:45 yes 15:51:47 yeah that's good 15:52:04 I also think that reducing the number of choices to "batched" and "urgent" makes it more clear to our users what the fields mean 15:52:30 we should probably just tie urgency to whatever the urgency of the bug is 15:52:30 true 15:52:35 I think that, given the choice, the vast majority of our packagers will do the right thing 15:52:37 and not let it be selectable in bodhi 15:52:46 one thing to keep in mind is that yum and dnf also consume this urgency field 15:52:48 jwb: Not every update has BZs 15:52:53 sgallagh, then it isn't urgent 15:52:57 nirik: requiring justification and the idea that some security updates should be batched instead of sent out as soon as they hit stable 15:53:01 so i actually can't use "standard" i don't think 15:53:04 jwb: Fair point 15:53:10 but i could probably do medium and urgent or something 15:53:21 i assume that yum/dnf expect certain specific strings there 15:53:24 (not sure) 15:53:37 bowlofeggs: Well, at the UI layer you could translate those 15:53:39 maxamillion: I'm with you on the first part... but having seen some of the things that get CVE's... some security updates are super duper stupid. :) 15:53:40 nirik: I agree that some security updates don't need everyone to drop everything they're doing and get them tested and pushed out immediately, but it seems silly to me to let them sit and wait for a batch push 15:53:41 And just store them differently 15:53:48 sgallagh: true 15:54:42 proposal: ask QA about batch testing plans, go back to normal pushes, discuss bodhi/other plans on list, revisit next week 15:54:50 nirik: that's fine, but I don't think we realistically have the people to really monitor and/or police that on a daily basis across all updates so it just makes more sense to me to not follow the Red Hat Errata model 15:54:54 nirik: +1 15:55:01 maxamillion: some security updates can't be exploited unless other security measures are also broken. in some cases they just can't be exploited at all, but should still be fixed. i think those are obvious to wait 15:55:05 maxamillion: I've handled plenty of CVEs that boiled down to "first have sudo access on the console"... 15:55:08 maxamillion: ime, most CVEs are in that category 15:55:21 bowlofeggs: I disagree, I say push them out ... what harm does it do? 15:55:28 i read oss-security and so many CVEs really are like that 15:55:39 maxamillion: well it triggers everyone to download 40 MB 15:55:40 maxamillion: Unnecessary churn on the mirrors/metadata 15:55:51 and more flood of useless updates. ;) 15:55:52 nirik: +1 15:55:54 sgallagh: how many people copy and paste "sudo curl foo.com | bash" ? 15:56:30 like I said, I'm stepping away from this item 15:56:46 maxamillion: My point was that the actual exploit often requires enough access that the exploit itself is meaningless 15:57:05 sgallagh: but I think that's really up to maintainer to decide 15:57:12 Sure, I agree 15:57:23 But it's why I think "security != urgent" in all cases 15:57:23 We should trust maintainers to do their job, no need to secondguees. 15:57:27 zbyszek: agreed. Which is why I think that not all security marked bugs should be treated equal 15:57:32 zbyszek: We're in violent agreement :) 15:57:36 Yes! 15:57:41 Yeah. That 15:57:54 sgallagh: I agree that not all security updates are urgent, I attempted to clarify that 15:57:57 and urgent != security too (bad bugs) 15:57:59 nevermind, stepping away from this one 15:58:02 OK, so we have +3 to nirik, anyone else? 15:58:07 OK, I think we're all agreed there 15:58:09 +1 to nirik 15:58:10 urgent just means... urgent :) 15:58:25 i do think newpackage and enhancement updates should not be urgent 15:58:28 nirik: +1 15:58:35 +1 to nirik's proposal, in case my vote wasn't clear 15:59:14 +1 i suppose 15:59:41 #agree Ask QA about batch testing plans, go back to normal pushes, discuss bodhi/other plans on list, revisit next week (+6, 0, -0) 15:59:44 Oh, can I get one clarification, seeing that I'm still on push duty? 15:59:51 once f28 calms down i can get bodhi to be more in line with these ideas 15:59:59 #undo 16:00:00 Removing item from minutes: AGREED by zbyszek at 15:59:41 : Ask QA about batch testing plans, go back to normal pushes, discuss bodhi/other plans on list, revisit next week (+6, 0, -0) 16:00:02 Do people want me to push out everything today, or is the urgent-only push I did an hour ago good enough? 16:00:19 puiterwijk: whats done is fine. 16:00:20 zbyszek: sorry. Wasn't meant to revert the vote. Just clarification on when "back to normal pushes" starts 16:00:26 puiterwijk: i think today's push was fine - i think we just go back to normal tomororw 16:00:28 nirik: okay. Just wanted to make sure 16:00:32 Yeah, but I thihk I miscounted 16:00:39 I just wanted to go back to normal so we don't have to explain what to do to every other releng person on push duty... 16:01:00 nirik: well, we have a simple operation to do it now, so that's simple :) 16:01:00 #agree Ask QA about batch testing plans, go back to normal pushes, discuss bodhi/other plans on list, revisit next week (+7, 0, -0) 16:01:02 yeah if we want that behavior it'd be better if bodhi did it 16:01:16 puiterwijk: oh yeah, truuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuue. 16:01:18 bowlofeggs: funny enough, it kinda does. I didn't want to do it manually. 16:01:21 argh. 16:01:27 nirik: wayland? 16:01:32 puiterwijk: yeah that --urgentonly patch :) 16:01:34 bodhi-push --urgent-only :) 16:01:47 OK, let's move on. 16:01:49 #topic #1745 F27 System Wide Change: Switch OpenLDAP from NSS to OpenSSL 16:01:50 .fesco 1745 16:01:50 https://pagure.io/fesco/issue/1745 16:01:50 my sound card power saving likes to cause a nice delay when it loads in to beep on a highlight. 16:01:52 zbyszek: Issue #1745: F27 System Wide Change: Switch OpenLDAP from NSS to OpenSSL - fesco - Pagure - https://pagure.io/fesco/issue/1745 16:02:18 uh... F27? 16:02:19 F27 still? I think that title might need updating 16:02:32 it's 28 now. 16:02:37 it got deferred 16:02:39 Yeah. The title needs updating 16:03:02 It's ON_QA. I'm not sure if there's anything to discuss here. 16:03:23 oh, and it was orig a f26 one that got delayed to f27. ;) 16:03:26 Well, I think the technicality is that we may not have approved it for F28 16:03:28 but yeah, ack +1, move on 16:03:30 But I'm +1 16:03:44 (fyi my f27-20180302 updated isos now show 115M of updates) 16:04:12 +1 16:04:18 +1 from me 16:04:28 +1 16:04:31 +1 16:05:21 #agree F28 System Wide Change: Switch OpenLDAP from NSS to OpenSSL is reapproved (+6, 0, 0) 16:05:31 #topic FESCo meeting time 16:05:48 http://whenisgood.net/fesco2018roundtwo/results/ss5khfp 16:06:21 We only have 8 answers, but there's isn't much more choice. 16:06:32 proposal: keep current time, stop discussing 16:06:43 +1 16:06:44 i was +1 to the OpenLDAP change 16:06:54 i'm also +1 to keeping the current time 16:06:56 note: Daylight stealing time starts in the US this weekend. 16:07:03 +1 - that works for me, my availability is pretty open 16:07:12 so, do we keep to the same UTC time? or ? 16:07:17 * jsmith is fine with the current time, but would prefer Monday or Tuesday 16:07:20 nirik: oof that'll be real early for you, but not relative to the sun :) 16:07:31 it'll be 9 AM for me (it's 10 now) 16:08:06 i do think it's better for meetings to not change with DST 16:08:21 because DST is not observed in all places and in the places it is observed, it changes on different dates 16:08:25 too complicated 16:08:34 and too US-centric to change with the US schedule 16:08:35 bowlofeggs: +1 16:08:40 Yeah, EU changes I think two weeks later 16:08:55 bowlofeggs: well, it's 7am for me now... 16:09:29 nirik: we could move it to 16:00 UTC 16:09:37 that works for me 16:10:01 works for me as well 16:10:21 that would be nice... but I am happy to do whatever works for everyone else. I can get up super early... it's not the end of the world. 16:10:45 nirik: OK, if you can then let's keep current time, because otherwise we run into problems if the meeting are too long. 16:11:05 fair 16:11:17 Let's move on, I think there's general agreement. 16:11:29 = New business = 16:11:30 #topic #1861 F28 Changes not in ON_QA status (<100% completed) 16:11:30 .fesco 1861 16:11:30 https://pagure.io/fesco/issue/1861 16:11:32 zbyszek: Issue #1861: F28 Changes not in ON_QA status (<100% completed) - fesco - Pagure - https://pagure.io/fesco/issue/1861 16:13:26 i'm good with retiring the django packages 16:14:06 propoposal: wait one more week for maintainers of django packages to reply, and retire after 16:14:16 zbyszek: +1 16:14:18 +1 16:14:32 I suppose. I'd be fine with just doing it... it's been a while coming, but +1 16:14:55 +1 16:15:03 i could also +1 just doing it 16:15:09 it's been a real long time 16:15:29 bowlofeggs: one more week doesn't really change anything, since we're in freeze 16:15:44 true 16:16:21 jwb ? 16:16:27 Honestly, I could go either way. 16:16:52 it will also cost us a few more minutes x 8 people in next week's meeting if we don't do it now 16:17:01 and it's really unlikely they'll respond after this long 16:17:10 but +1 to either 16:17:17 No, let's vote now, and retire without talking about it any more 16:17:34 OK, then consider me +1 for "retire now" 16:17:47 It's just that I opened two more bugs asking maintainers for update/retire, and they need a few days to answer. 16:17:56 (Even though I don't expect an answer.) 16:18:34 So, we have +5 to retire in a week. Do you want to vote on retiring now? 16:18:57 (I'd be against, so it won't pass ;-] ) 16:19:11 Then let's just move on :) 16:19:16 sure 16:19:23 #agree Django packages which are incompatible with Django2.0 will be retired after one more week (+5, 0, 0) 16:19:36 #topic #1859 Review of release blocking deliverables for F28 16:19:36 .fesco 1859 16:19:36 https://pagure.io/fesco/issue/1859 16:19:37 zbyszek: Issue #1859: Review of release blocking deliverables for F28 - fesco - Pagure - https://pagure.io/fesco/issue/1859 16:19:55 do we not need to talk about the others from the last ticket? 16:20:21 #undo 16:20:21 Removing item from minutes: 16:20:34 bowlofeggs: of course, sorry, I got carried away 16:21:08 I think they're all moving forward, so we can just sit back and relax for now 16:21:32 (Unless you're one of the people helping move it forward, of course!) 16:21:34 i have seen patches for the krb in python one go by recently 16:21:40 so i know that one is making progress 16:22:02 i assume sgallagh can tell us about modularity? 16:22:22 bowlofeggs: I put an update into the ticket this morning 16:22:24 the s390x one had some performance issues with composing iirc? 16:22:25 Getting closer... 16:22:45 i have no idea about the crypto one 16:22:48 ignatenkobrain says "Replace glibc's libcrypt with libxcrypt" is done 16:22:56 oh, hmm.. another ticket maybe 16:23:13 No, it was this one. 16:23:23 oh i see the comments in the fesco ticket 16:23:23 * sgallagh got confused by the quick jump to the next topic 16:23:24 haha 16:23:30 Still another pungi change needed today... the fix yesterday caused another issue. Hopefully we will get a f28 / rawhide compose with modular right soon. 16:24:06 there are some bodhi problems that puiterwijk discovered yesterday too :/ 16:24:08 (well, 28, rawhide currently has no modules actually made for it) 16:24:10 Note that we have f28-modular-updates-testing, but last night we found fundamental problems wiht modularity and bodhi which we are resolving 16:24:21 and then there's update issue... yeah, that 16:24:40 rpm -q libxcrypt ;) 16:24:45 I have a patch for that that might work, but will proceed working on that on Monday, so then we'll know more 16:24:53 anybody know what the deal with s390x one is? 16:25:28 bowlofeggs: we disabled ostree because it was painfully slow. I think the rest were re-enabled and are waiting for us to have a working compose otherwise. 16:25:28 oh there is a comment on the bz fo rthat one 16:25:35 cool 16:25:51 "Only item remaining is building successfully cloud-base image. Root cause of this failure has been identified. It is failing because time required by imagefactory to finish building image exceeds specified timeout (which is 7200 seconds in koji). Increasing timeout value should give us successful cloud-base image." 16:26:14 ah yeah, there's a patch for that pending 16:26:15 zbyszek: sorry I can not make the current meeting tim for the next few weeks 16:26:17 i'm +1 to waiting another week on these 16:26:36 bowlofeggs: +1 16:26:40 bowlofeggs: +1 16:26:41 Do note that 7200 seconds is already 2 hours. 16:26:49 So this means that the composes will become significantly longer 16:27:12 (just pointing it out, since "shorter compose times" was also something people wanted) 16:27:21 puiterwijk: indeed 16:27:35 puiterwijk: it seems like we should investigate why they are so slow 16:27:45 dgilmore: can you answer the whenisgood poll? 16:27:50 zbyszek: no 16:28:02 ? 16:28:03 Note that we also had another suggestion on improving the IO by trying to virtualize an s390x composer. 16:28:06 zbyszek: I currently do not know my availability 16:28:16 dgilmore: oh, OK. 16:28:25 zbyszek: I am in the process of changing jobs and teams 16:28:53 I will know my new schedule and availability soon 16:29:07 dgilmore: OK, we should revisit then. 16:29:45 zbyszek: I expect I will have a bit more free time in the US mornings 16:29:47 puiterwijk: virtualize or emulate? 16:30:07 maxamillion: emulation I guess. Basically, run s390x composes on an x86_64 builder in PHX2 16:30:21 yeah i suggested that in a prior fesco meeting 16:30:25 Since the main problem with s390x is just the I/O (the data needs to go across the country from West coast to East coast twice) 16:30:33 not sure if it would be faster or slower though :) 16:30:48 puiterwijk: yeah, rgr .... I don't think the clarification matters outside my personal curiosity 16:30:53 the CPU tasks would be real slow, but the B/W would be local. so which one wins? :) 16:30:53 :) 16:30:53 bowlofeggs: yeah. I think we should just try it once... 16:30:54 and may have bugs/issues working on real hw then 16:31:11 maxamillion: yeah, totally fine. I was being too vague. 16:31:19 yeah the emulation itself could be error prone too 16:31:34 in general we have been very against emulated stuff or cross compiling. 16:31:45 nirik: yeah. Though it is not the actual binary building, "just" the building of images. 16:31:50 it was just a hair brained wacky idea, but could be worth trying if even just to know which is faster 16:32:11 why does the image building require the arch? 16:32:18 bowlofeggs: scriptlets 16:32:20 sure. Perhaps we could ask the change owner to test. ;) 16:32:21 (primarily) 16:32:28 ah 16:32:41 Everything else in rpm install is just "extract cpio archive" 16:32:44 nirik: yep, agreed 16:32:47 yeah makes sense 16:33:02 dgilmore, jwb, sgallagh, maxamillion, nirik, vote on bowlofeggs proposal? 16:33:20 (which is to give another week) 16:33:25 wait, I missed it ... sorry 16:33:26 +1 16:33:31 +1 16:34:28 +1 16:34:30 +1 16:35:08 #agree Wait another week for changes not in ON_QA status (+6, 0, -0) 16:35:30 OK, I think now we can move on. 16:35:31 #topic #1859 Review of release blocking deliverables for F28 16:35:59 .fesco 1859 16:35:59 https://pagure.io/fesco/issue/1859 16:36:00 zbyszek: Issue #1859: Review of release blocking deliverables for F28 - fesco - Pagure - https://pagure.io/fesco/issue/1859 16:36:16 Note that I replied to bowlofeggs' question (and added one of my own) a few minutes ago 16:36:32 (I had not seen a notification from getting pinged on it) 16:36:54 Server SIG voted on and updated our deliverables this week 16:37:30 i had to drop for a bit, apologies 16:37:34 We had previously voted to block on issues on three aarch64 boards, so we updated the wiki to reflect this 16:37:49 yeah, it looks fine for me except for Modules... are they blocking? 16:38:04 puiterwijk's suggestion is interesting - should we say that a repository is a release blocking deliverable? 16:38:10 My understand was that they weren't blocking... 16:38:20 ... but I could be wrong 16:38:26 i mean, we don't say that about other repos, but i'd think not having the ability to deliver updates would be "release blocking" if that happened for rpm? 16:38:54 like if bodhi suddenly couldn't update f28 normal rpms for some reason, would that be release blocking? 16:39:27 bowlofeggs: yeah, there's some criterion about being "able to install updates" 16:39:30 well, the release critera is kinda about... the release... 16:39:30 bowlofeggs: well, there's "Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image. 16:39:49 I guess that to be able to notify of updates, there must be an ability to ship updates? 16:39:50 alternately, since modules are now "add on", should they be release blocking? couldn't they easily be added after the release if necessary? 16:40:29 like, fedora server will still work if modules aren't ready at release day, right? 16:40:58 right. 16:41:30 we don't really have any release critera around modules or even testing guidelines right? 16:41:55 Not that I could find in any of the official pages 16:42:19 proposal: approve proposed list, and if modules need to be added, we can do that later when there are testing guidelines and release criteria for them 16:42:24 it does sound like it shouldn't block the release to me, based on my assumption that server can still work without it if it's not ready in time 16:42:33 zbyszek: +1 16:42:39 zbyszek: +1 16:42:39 Also, I don't think QA has tests for them, at least in openqa 16:42:47 we should ask about those things on lists... 16:43:26 zbyszek: +1 16:43:29 Any volunteer to ask start the thread? 16:44:33 jwb, jsmith, maxamillion, vote please 16:44:39 sgallagh: since you work on modules, you wanna start the thread? 16:44:48 i can do it too, but am less familiar with the subject 16:44:58 +1 16:45:21 bowlofeggs: I can do that next week, probably. 16:45:24 I'm overloaded today. 16:46:31 should be devel and test lists most likely 16:46:36 +1 16:47:11 #agree Proposed list is approved. Modular deliverables may be added to the list later when there are testing guidelines and release criteria for them (+6, 0, -0) 16:47:15 #action sgallagh to start a thread on the mailing lists (devel and test) about the criteria 16:47:16 i kinda feel like modularity shouldn't even delay the beta 16:47:27 imo, it shouldn't be a release blocker since it's an add on now 16:48:07 OK, let's move on. This we can discuss in the thread ;) 16:48:12 #topic #1858 Proposed Fedora 29 schedule 16:48:12 .fesco 1858 16:48:12 https://pagure.io/fesco/issue/1858 16:48:12 sure 16:48:13 zbyszek: Issue #1858: Proposed Fedora 29 schedule - fesco - Pagure - https://pagure.io/fesco/issue/1858 16:48:31 +1 16:48:36 +1, I'll go with the crowd 16:48:44 +1 -- looks sane 16:48:50 note that with f28 we are trying the 3 week freezes, we don't know if this will help or not yet... :) 16:49:03 but sure, +1 and we can revisit later if needed 16:49:24 0 (I haven't had a chance to examine it closely and I'm multi-tasking right now) 16:49:53 +1 16:49:59 jwb, dgilmore? 16:50:15 I would actually suggest putting one day between bodhi activation and freeze start (at least releng/infra freeze). But that might be colored by this time around's results 16:51:44 puiterwijk: were there issues with it being the same day that that would help? 16:51:51 puiterwijk: people asked them to be on the same day... :) 16:52:17 bowlofeggs: the configuration for f28(m) pushes. 16:52:19 but yeah, this cycle was rough with all the modularity attempts 16:52:58 OK to tally the vote and move on? 16:53:02 The code fixes wouldn't have been helped, but configuration updates had to be made. People always forget parts, even if it's in the SOPs. And getting one additional day to fix those might be good.. 16:53:30 nirik: well, regardless of that, there's the "fedora_28_primary_arches" setting and more that we only found out when we tried to push. 16:53:40 puiterwijk: could we adjust that by just having the infra freeze be one day after the releng freeze (i.e., not in this schedule?) 16:53:44 I'm reasonably sure we'll always find a new primary_arches flag 16:53:59 bowlofeggs: yeah, thats possible indeed. 16:54:05 bowlofeggs: well, releng freeze is for all releng boxes I think. 16:54:37 At least, that's how I read it. Which, in my mind, includes bodhi settings. But if that's a misinterpretation on my account, then sure 16:55:16 you mean the infrastructure freeze? 16:55:30 yeah i don't know where the line is - i assumed that bodhi cahnges were frozen by infra more than by releng 16:55:30 there's not a seperate releng one that I know of... 16:55:43 i thought the releng one was more about which packages can go to stable 16:55:50 i haven't had time to look at the f29 schedule 16:55:55 I thought I've seen references to a releng freeze. But yeah, that's what I mean with misinterpretation 16:56:14 proposal: one more week so people can look at this schedule 16:56:19 So yeah, let's change that to "proposal: infra freeze 1 day later", but that's infra 16:56:21 (since two of us haven't had time yet) 16:56:40 bowlofeggs: +1 16:56:55 There isn't any hurry at this point. 16:57:28 I assume sgallagh is +1 to bowlofeggs 16:57:35 Yes 16:57:44 +1 to bowlofeggs 16:57:52 jwb, jsmith, nirik ? 16:57:55 +1 sure 16:58:01 +1 to waiting a week 16:58:15 +1 16:58:16 please 16:58:56 #agree Punt on Proposed Fedora 29 schedule for a week to discuss moving "infra freeze" one day after bodhi activation (+7, 0, 0) 16:59:02 #topic #1860 allow major version bump of rpkg utility 16:59:03 .fesco 1860 16:59:03 https://pagure.io/fesco/issue/1860 16:59:05 zbyszek: Issue #1860: allow major version bump of rpkg utility - fesco - Pagure - https://pagure.io/fesco/issue/1860 16:59:10 +1 16:59:11 +1 16:59:48 +1 17:00:00 +1 17:00:54 jwb, jsmith, sgallagh? 17:01:00 +1 17:01:06 I was +1 in the ticket 17:01:37 Oh, jwb and dgilmore too 17:01:42 we got lots of ticket votes on this one actually 17:01:49 (which is nice!) 17:01:59 #agree Proposed allow major version bump of rpkg utility is accepted (+8, 0, 0) 17:02:02 #topic Next week's chair 17:02:28 Volunteers? 17:02:57 i did it two times ago and you did it the last two times - time for someone else ot step up i say :) 17:03:04 For US people, this will be an hour earlier, right? 17:03:38 sgallagh: Correct. 17:03:46 sgallagh: (at least I *think* I'm correct) 17:03:50 Yes 17:03:52 * jsmith will volunteer if nobody else does 17:03:53 That's going to be a conflict for me for a couple weeks until Europe catches up 17:03:58 according to thunderbird, it's an hour later actually 17:04:00 The same time was kept relating to UTC 17:04:04 I'll probably be able to join, but not run the meeting. 17:04:07 we switch from UTC-5 to UTC-4 17:04:12 (in EDT) 17:04:48 s/relating/relative/ 17:05:22 so 15:00 UTC is 10:00 fo rme, but at UTC-4 that becomes 11:00 17:05:25 If it's an hour later, I can do it 17:05:41 because we shift to become closer to UTC 17:05:59 sgallagh: Why don't I take next week, and you can take the week after that if you're able? 17:06:16 unless you live in arizona (unless you live in one of the many parts of Arizona that *do* observer DST!) 17:06:30 jsmith: Sounds good 17:06:38 or possibly unless you live in florida, depending on how they vote (current proposal to drop DST there) 17:06:48 bowlofeggs: or hawaii. :) 17:06:52 oh right 17:07:01 MA is also voting to move to Atlantic time with no DST 17:07:18 florida is actually voting to move to permanent DST 17:07:28 so always UTC-4 (which is actually kinda weird) 17:07:38 well, "voting on whether" i mean 17:07:44 bowlofeggs: That's the same thing as MA; moving to the Atlantic TZ. 17:08:09 i think it makes more sense for the sun to be approximately overhead at 12:00 but that's just me :) 17:08:21 sgallagh, when is that happening? 17:08:26 #action jsmith will chair next meeting 17:08:36 jwb: It's on the queue; no idea when it'll come up for a vote. 17:08:44 #action jsmith will get the world to stop doing DST and have everyone just switch to UTC 17:08:48 I think it might be a ballot initiative in Nov. actually 17:09:03 Wow... so much pressure :-) 17:09:08 haha 17:09:08 sgallagh, that will be nice for MA. it will destroy everyone's calendars :) 17:09:14 OK, let's move on. 17:09:15 #topic Open Floor 17:09:17 mwahahahaha 17:09:29 * jsmith has nothing for the open floor 17:09:44 Same 17:09:46 * zbyszek neither 17:09:54 * bowlofeggs crickets 17:10:03 I'll close in a minute 17:10:38 #endmeeting