17:00:01 #startmeeting FESCO (2011-09-12) 17:00:01 Meeting started Mon Sep 12 17:00:01 2011 UTC. The chair is nirik. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:01 Useful Commands: #action #agreed #halp #info #idea #link #topic. 17:00:02 #meetingname fesco 17:00:02 The meeting name has been set to 'fesco' 17:00:02 #chair notting nirik ajax cwickert mjg59 mmaslano t8m pjones sgallagh 17:00:02 #topic init process 17:00:02 Current chairs: ajax cwickert mjg59 mmaslano nirik notting pjones sgallagh t8m 17:00:08 Afternoon 17:00:11 hello sports fans 17:00:12 hello 17:00:29 notting said he would be a bit late. 17:00:33 hello party people. 17:01:17 looks like thats 5 folks... so I guess we can get started. 17:01:34 #topic ticket #663 Late F16 Feature Java7 17:01:36 .fesco 663 17:01:37 nirik: #663 (Late F16 Feature Java7) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/663 17:01:44 did we have anything else to do on this one? 17:01:48 or was it all taken care of? 17:02:02 * nirik checks the repo 17:03:53 http://fedoraproject.org/wiki/Features/Java7 17:04:18 not sure thats been updated. 17:04:38 the java-1.7.0-openjdk package has been updated. 17:04:46 Sorry I'm late folks 17:04:53 no worries. 17:04:57 so, we still need: 17:05:04 * make sure all packages are built against 1.6.0 17:05:20 * update the feature page to reflect the new target 17:05:35 I can ping dbhole on those in the ticket? 17:05:50 sure. 17:06:23 #action nirik to ping in ticket for remaining action items. 17:06:29 anything else on java? 17:07:30 ok, moving on. 17:07:34 #topic #667 Request to fix CRITPATH update process 17:07:37 .fesco 667 17:07:39 nirik: #667 (Request to fix CRITPATH update process) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/667 17:08:10 so, several proposals around this. 17:08:16 also the next ticket might be related. 17:08:52 Certainly the requirements for critpath updates should be lowered - and I think the dledford proposal is fine. 17:09:09 t8m: which one? all of them? 17:09:24 * nirik thinks 3 is perhaps reasonable. 17:09:30 I have no comprehension of how the appropriate response to "We're not getting enough testing" is "Remove the requirement for testing" 17:10:03 nirik, the 3) 17:10:23 how about 'the lack of possibly unrealistic levels of required testing is materially affecting our ability to release important security updates in a timely manner'? 17:10:39 i mean, i should be ra-ra more testing, but there clearly is an issue here. 17:10:39 mjg59: does seem to be the question, doesn't it? 17:10:46 adamw: The unrealistic levels of required testing being "Someone tested it"? 17:11:06 mjg59: many updates are not tested, I have same problem as dlenford 17:11:16 do we wish to also discuss #664 Older releases need a different approach for updates with this? 17:11:17 for critpath, the requirement is 'two people who aren't the maintainer tested it on two different releases, and one of them is a proven tester' 17:11:21 mjg59, the problem is - there are probably some people who test but they do not give karma to updates 17:11:26 mmaslano: I agree. But the answer here is to figure out why testing isn't happening. 17:11:39 nirik: I'd rather discuss it separately; in my mind near-death distro is a different question 17:11:42 so that's four tests for any bug that affects both currently supported releases, disregarding N+1 17:11:49 pjones: fair enough. 17:11:56 mjg59: Testing isn't happening because there is an insufficient number of subject matter experts in the proventester groups 17:12:08 sgallagh: So how do we fix that? 17:12:15 sgallagh: so that sounds like the problem we should attempt to address 17:12:20 well, I think there are more reasons than that... 17:12:33 pjones, will you pay someone for testing? 17:12:34 well, every time this comes up, everyone says that, then no-one does anything, and we wind back back here two weeks later. 17:12:34 for example, with mdadm, I suspect there are less people using raid day to day on a alpha release. 17:12:34 (okay, "a problem" yadda yadda) 17:12:36 mjg59: less popular applications are not tested, because it's hard/boring. And we have only small group of testers 17:12:53 t8m: we're looking for creative ideas, not sarcastic responses. 17:12:55 I also don't think "Let it in after two weeks" is a good plan, because again that doesn't actually solve the problem - it's not reasonable for security updates to be delayed 2 weeks 17:12:58 i can tell you right now i'm not going to have a lot of cycles to dedicate to getting more proven testers until f16 is done at least. 17:12:59 nirik: Well, the proventester concept is rather broken if you can't trust that a proventester is actually capable of validating a fix. 17:13:06 or, if you're me, you do a driver update for something ancient and nobody _can_ test it 17:13:08 would having a weekly proventester meetup/testfest possibly help? 17:13:20 pjones, I'd really like to implement at least the option 3) from the dledford's proposal 17:13:37 sgallagh: I don't follow. 17:14:12 we are speaking about not enough testers for a year. So we had to change rules 17:14:18 https://admin.fedoraproject.org/updates/FEDORA-2011-9064 for instance 17:14:18 nirik: The idea behind "proventester" with critpath is that for something to go stable in critpath it must be tested by someone considered to be capable of validating it properly. 17:14:24 sgallagh: you don't need to validate a specific fix to +1 a update.. 17:14:26 so while we are looking for more proventesters/testers willing to add karma to updates, can we at least workaround the problem we have 17:14:37 nirik: Then what's the value of a proventester? 17:14:55 t8m: It doesn't work around the problem. 17:14:56 sgallagh: we know they have read and agreed to the rules/docs 17:14:56 sgallagh: the intent for critpath is to check that the update doesn't break the critical path: i.e., doesn't stop you booting and installing updates 17:15:01 nirik: And I didn't mean a specific fix. I meant regression testing. 17:15:04 mjg59, it does 17:15:15 t8m: The problem includes the requirement that security updates ship in a timely manner. 17:15:29 Two weeks is not timely. 17:15:34 mjg59, well probably two weeks is much better than never 17:15:39 adamw: And in some cases this is a nuanced issue. (Example: the libtalloc errata that just finally pushed this morning after much twisting of arms) 17:15:42 No, two weeks is already a failure of process 17:15:54 If we make it easier to ignore that process failure then we have less incentive to fix the underlying problem 17:16:13 aargh 17:16:19 There is a genuine problem here. We need to work out the best way to fix it, not make it easier to ignore it. 17:16:20 #info there are currently 21 updates that are older than 2 weeks in critpath 17:16:25 there is no fix apart from paying people for testing 17:16:40 Then we need to pay people to do testing 17:16:54 perhaps there are other incentives than just money. 17:16:56 Is it really such a problem to go back to the old method of pushing updates and resorting to public shaming to keep people from making the same mistakes over and over again? 17:17:04 sgallagh: Yes, because it clearly didn't work 17:17:07 if people were interested in getting critical security updates in timely manner they woud be doing the testing already 17:17:09 (though I'm not against exploring paying people for testing.) 17:17:23 mjg59: I'm not sure it was any less effective than the way things work now. 17:17:28 sgallagh: note my comment on the bug: the reason we developed this policy in the first place was a glibc update, and glibc team is *still* submitting utterly broken updates. 17:17:46 sgallagh: It's now rarer for people to have their machines broken by stable updates 17:17:47 one thing that hasn't been brought up yet: there is the idea of 'implied testing' 17:17:55 adamw: So maybe instead of having global policy we start implementing a per-package policy. 17:18:00 I see no problem with public shaming people if they push broken updates in cases where they could find out the brokennes by themselves 17:18:06 i don't have hard numbers here, but i think it's true to say we usually get more negative karma for broken updates than we get positive karma for working but boring ones 17:18:07 So that packages with a known history of breakage end up on probation 17:18:32 sgallagh: that's not much better; the glibc package you use as example has entirely switched maintainership since then 17:18:32 you can see the timeout approach as 'ignoring the problem', but you can also look at it as, if this update broke the world, someone would probably have said something in the last two weeks 17:18:36 adamw: Again, any approach that involves some sort of timeout is just papering over the underlying problem 17:18:39 ... and then we're meeting every week about which packages are on probation and which aren't? 17:18:44 adamw: I agree 17:18:45 I've totally stopped updating it, but: http://fedoraproject.org/wiki/Updates_Lessons 17:18:45 the fact that it still has problems is coincidence at best. 17:18:49 notting: indeed. 17:18:52 adamw: I think part of the problem with that is that we have no mechanism for notifying interested parties of updates in testing 17:19:10 sgallagh: that's not true, we do. 17:19:11 mjg59: condition 1 of my proposal is what you want to meet for security updates 17:19:26 sgallagh: there is a daily mail to two mailing lists, and bug reports are automatically notified when an update goes into updates-testing. 17:19:35 sgallagh: you might want there to be *more* mechanisms, but there certainly are already mechanisms. =) 17:19:37 dledford: No, because we also want to ensure that security fixes don't cause regressions 17:19:53 adamw: I would prefer that there would be mechanisms accessible to end-users rather than developers 17:19:58 dledford: Security fixes that break something else are worse than not having security fixes 17:20:09 dledford: Because users will simply start not applying security fixes 17:20:09 sgallagh: the test mailing list is not for developers, and neither is bugzilla. 17:20:17 bodhi has rss feeds as well. 17:20:21 mjg59, that depends purely on the severity of the issue and the regresssion 17:20:26 nirik: we could probably publicize that better. 17:20:29 t8m: No it really doesn't 17:20:35 mjg59, no it really does 17:20:39 t8m: If security updates break things then users stop updating 17:20:40 adamw: Not everyone bothers to CC themselves on a bugzilla that matches their problem 17:20:49 yeah, I wonder if people don't realize how easy it is to be a proventester? and use fedora-easy-karma. 17:20:53 Because it's more important to them that their computer works than that their computer is secure 17:21:09 mjg59, sometimes you cannot fix a security issue without regressing a functionality 17:21:16 mjg59: if a security update is tested to resolve the security issue by someone other than the package submitter, then you already have 1 regression test right there 17:21:44 dledford: I see no reason at all for us to accept less testing for security updates than for normal updates 17:22:00 adamw: What I'd rather see is for the update UI to allow you to specify "I want to install testing versions of this package" instead of only stable versions 17:22:08 Without having to enable updates-testing globally 17:22:24 t8m: rarely is that true, and when it is, it probably needs broader discussion. 17:22:31 nirik: i am intending to mention it on the forums at some point. but again, cycles. 17:22:38 And when I have testing packages on the system, I should be prompted on a daily basis (disable-able) to provide karma 17:22:58 sgallagh: I don't think that will work 17:23:02 mmaslano: Why not? 17:23:08 sgallagh: yeah, I think we have wanted that for a while, but needs packagekit development cycles. ;( 17:23:16 sgallagh: I install updates-testing but I give only -1 17:23:23 sgallagh: I can't go through all packages 17:23:23 mjg59: it's a case by case issue in my mind...isolated patch for security issue, should go out quick with confirmation that security issue is resolved (said confirmation which provides at least some level of non-regression assurance), security fix by wholesale package update is a different matter. IMHO anway. 17:23:28 i can see a simpler version of sgallagh's proposal - just add a blurb about the karma process to packagekit's entry for -testing . 17:23:42 i think predicating changes on significant tooling work is a bad road to go down 17:24:20 dledford: We've seen cases where such isolated backports have resulted in unexpected consequences 17:24:25 notting: right, that's what i'm worried about: as i said earlier, every time this comes up, it seems like we do a bit of hand-waving about improving the amount of testing that gets done, and then nothing much happens, and maintainers continue to be frustrated. 17:24:39 dledford: Case in point: the change to xmlrpc-c to remove kerberos forwarding 17:24:45 It broke FreeIPA badly. 17:25:02 But I agree, it should be safer. But the point of it being safer is to make it more likely that testing is successful, not to avoid the need for testing. 17:25:05 this is bad example 17:25:20 adamw: which is not to say there aren't things we can do there 17:25:22 mmaslano: What is? 17:25:29 Maybe, just maybe, the big part of the problem is that Red Hat is paying pretty big part of maintainers of critpath packages, but it does not pay the equivalent part of QA for the same packages? :D 17:25:54 sgallagh: long story, never mind 17:26:00 nirik: 15 minutes? 17:26:02 t8m: that's not something fesco can solve, though 17:26:07 notting: sure. i'm just reaching the point, i guess, where i'm thinking it's not unreasonable to adjust our expectations of what level of testing should be easy to achieve, as they don't seem to be matching reality at present. if we get all awesome and kickass about getting more testing, we can always change them again. 17:26:10 #proposal Ask QA for recommendations on increasing participation in testing, come back to this topic 17:26:13 so it sounds like a) "get more testers" is the best solution if we could figure out how to do that. ;) But b) allowing updates after a timeout, might be a workaround until a exists? 17:26:28 notting, yeah, but fesco should make realistic decisions based on the situation 17:26:31 nirik: I think that'll just avoid us caring enough to follow through on (a) 17:26:36 adamw: but the threshold we're hoping for isn't "easy to achieve", it's "thorough enough to trust it" 17:26:56 mjg59, -1 that does not solve anything 17:27:03 adamw: if all we were concerned with were "easy to achieve", we wouldn't have an OS. 17:27:06 mjg59: see adamw's latest comment in the ticket.. .they've already discussed it 17:27:20 notting: we more discussed dledford's proposal than the issue of getting more testers. 17:27:21 I think mizmo's bodhi2 mockups will help us better facilitate testing http://lewk.org/img/bodhi-20-frontpage.png 17:27:55 notting: No, they appear to have discussed reducing testing requirements because there's not enough participation 17:27:59 notting: i'm not sure we'd have any amazing ideas beyond what's been discussed here. 17:28:07 Which I think is an amazingly bad idea 17:28:21 If QA's response is "QA is hard, let's have less" then what are we even doing at all 17:28:25 * nirik would be happy to beat a drum for more testers, but not sure how much luck I will have. 17:28:34 I believe broken updates will be marked as broken before they've got into stable 17:28:55 mjg59: not entirely true, condition #1 of my proposal isn't about less testing, it's about when you have successful testing for filed bugs, you need to be able to get those known good fixes out. That's not less testing, that's paying attention to the testing you've done. 17:29:13 mmaslano: and if so, the question is what can we do to make the people who would report those problems do so earlier? 17:29:16 mjg59: we have to work with the resources we have available. it's very easy to sit in an ivory tower and declare what seems like a minimal amount of required testing, but if we just don't have the people to do that testing, where does that leave us? not to throw stones, but i can sit here and draw up minimal levels of required functionality for the kernel all day, but it doesn't mean my suspend bugs get magically fixed. =) 17:29:18 Are the packages from updates-testing being downloaded? Do we have the statistics? 17:29:21 lmacken: Would it be possible to modify Bodhi to allow specifying the same package for N releases in one update? 17:29:30 right, there's also the case of 'package is 2 weeks in testing, but has no karma' but it's actually been tested by lots of people, they just didn't add karma. 17:29:34 dledford: No, because then you have people focusing on testing what you've fixed rather than testing that nothing else has been broken in the process 17:29:38 mmaslano: and "pay them" doesn't seem like it's the answer to that, since they're doing the work anyway. 17:29:41 nirik: i mentioned that earlier, under 'implied testing'. 17:29:42 If so, perhaps we can assume that they are being installed and thus regression tested 17:29:46 As in, if I push the exact same patch to F14, F15 and F16, and only F15 and F16 get tested, I can push all three? 17:30:05 So I can't see a problem with a timeout if a package does not have a negative karma 17:30:07 adamw: Then let's focus on getting you more resources 17:30:08 pjones: I didn't say test more. I'm saying that broken updates with big impact are found soon enough 17:30:12 adamw: yeah, just re-iterating it. ;) 17:30:13 sgallagh: I wouldn't want to attempt that in the current codebase, but it's definitely something I've considered for bodhi2. 17:30:15 adamw: Let's not just throw our hands up and say it's impossible 17:30:18 regardless if it is critpath or not 17:30:28 This is one place where Launchpad has us beat 17:30:28 mmaslano: no, they're found exactly not soon enough historically. 17:30:31 sgallagh: i am very leery of that, due to the underlying stacks being different 17:30:37 mmaslano: that's the whole reason we have critpath! 17:30:42 notting: Could you elaboratE? 17:30:48 mjg59: i didn't say that. i said it may be practically the better option to adjust the policy to the levels of testing we are capable of achieving at the present time. if we become capable of achieving more testing, we can tighten up the policy. 17:30:51 * nirik also doesn't like the multirelease thing. 17:31:09 right now it seems like there's something of a mismatch between the two, there has been for a while, and there's been lots of hand-waving about syncing them up but not a whole lot of action. 17:31:18 sgallagh: i can trivially propose a patch that builds, verifies on f15 and f16, and causes broken deps on f14 17:31:24 * nirik notes we are way over 15min... everyone ok to continue? 17:31:33 i know part of that is my fault personally and qa's fault as a team, sure. i'm not playing the blame game, just looking at what's the best choice from a practical standpoint. 17:31:37 +1 to continue 17:32:04 adamw: If we work around the bug then we never fix the bug 17:32:24 pjones, that's not true, the update policy was enforced at once with this incredibly unrealistic requirement for getting the karma 17:32:37 pjones, the problem before was there was no timeout 17:32:44 t8m: you're going to have to be a bit more specific about what you think isn't true. 17:32:48 pjones, the updates were being pushed immediately 17:32:52 to stable 17:32:59 or rather, the maintainer had the choice of pushing immediately 17:33:04 yes 17:33:06 (and this was often exercised) 17:33:09 yes 17:33:15 #info only 4 of the 21 items are f16. 17:33:51 Perhaps we need a stricter definition of critpath? 17:34:19 sgallagh: that merely defines around the problem. and, if you consider 'booting' critical path, mdadm would be in it anyway 17:34:29 I mean, glibc is obvious, but I've seen a few other pieces here and there that are only critpath because evolution has a dep on them (for example) 17:34:40 notting: Not saying it isn't. 17:34:41 adamw: ... and that went poorly. 17:35:25 so, we seem to have: change nothing, try and get more testers vs allow timeout after 2 weeks? 17:35:28 sgallagh: so... your point is what? we need to define fewer things as critical? 17:35:33 one question - is it worth having different policies for branched vs. released? i can certainly see having a timeout on branched be 'more ok' than having it on released 17:35:35 or are there any other proposals? 17:35:56 nirik, I don't need to vote for such proposal 17:36:04 that's basically do nothing 17:36:05 pjones: I'd argue that critpath should be only @core 17:36:07 nirik: I would hope that condition #1 in my proposal is still on the table 17:36:08 notting: possibly. I wouldn't be against making branched more lenient. 17:36:14 But I'm in the minority here, so I'll shut up 17:36:21 sgallagh: so, that sounds like another discussion entirely then 17:36:34 (not against discussing it, but it's a separate issue) 17:36:46 notting: yeah, we already do have different for prebeta branched 17:36:50 nirik, my proposal is for dledford's #3 probably with the disabling of the timeout if there is a negative karma 17:37:10 nirik: is that actually implemented? 17:37:18 I thought so. 17:37:23 yes, it is. 17:37:30 All critical path updates MUST get one +1 karma from a proventester before being moved to stable. 17:37:41 critpath still doesn't have a timeout in branched pre-beta, but it only requires +1 proventester. 17:37:46 non-critpath has a 3 day timeout instead of 7. 17:37:51 then after beta it goes to the normal: 'All critical path updates MUST have a sum of +2 karma, one of which must be from a proventester." 17:39:09 so, should we vote on mjg59's or t8m's proposals? 17:39:19 or would anyone care to venture some more? 17:39:23 So if I'm understanding correctly, the proposal is to allow critpath to go stable after 2 weeks if-and-only-if there is no -1 karma? 17:40:10 yeah, I think thats what t8m is proposing... 17:40:14 for the record, either way the vote goes, i will try and implement some of the ideas for getting more proventesters. 17:40:26 (if anyone would to submit some qa trac tickets for those, that'd be great.) 17:40:28 yeah 17:40:29 would like* 17:40:32 mjg59's was: "Ask QA for recommendations on increasing participation in testing, come back to this topic" 17:40:32 I should clarify that I'm not against migration after 2 weeks because I think it'll break things, but because I think it'll remove the incentive to actually fix things 17:40:39 nirik: And what is mjg59's proposal? 17:40:56 I don't think those are mutexed 17:40:57 adamw: I'd be willing to help on that. 17:41:08 they aren't. 17:41:17 For the record, I'm +1 to both 17:41:35 sgallagh: well, 'come back to this topic' implies do nothing else 17:41:58 but adamw say he'd already take that ball, so it looks like it's getting done whether we approve it or not 17:42:18 I'm definitely +1 on mjg59's proposal, and I consider dledford's to be rather extreme. perhaps justifiable, though I doubt it, but we won't know until we investigate other options in earnest. 17:42:28 If we get a sudden massive influx of testers, we can revisit this in the future 17:42:32 I think a timeout would be allowing for the implicit testing to have happened... 17:42:40 (so at least for now, -1 to the proposals in the ticket) 17:42:55 I am certainly +1 to my proposal 17:42:58 I'm +1 for t8m's (dlenfords) because we still don't have enough testers 17:43:07 * nirik is +1 to both too, but yes, revisit if we can gather enough testers. 17:43:43 We not only don't have enough testers, you can test until you are blue in the face, but without a test plan that identifies all the things to *be* tested, you are likely to miss critical interactions. 17:43:44 I'd rather be cautious and revisit *making the change* after we find out if we can get more testers or not. 17:44:07 expecting more time to magically find problems is just hubris. 17:44:09 dledford: the testing needed for critical path is: does my system boot and can I apply updates after that. 17:44:29 expecting stopping the testing requirement to do so is even worse 17:44:39 nirik: well, it's a bit more involved than that - you shouldn't +1 an mdadm update if you have no mdraid array, for e.g. 17:44:51 sure. 17:44:55 https://fedoraproject.org/wiki/Proven_tester 17:45:25 (honestly I'm more concerned with proventesters who say "not tested" in bodhi...) 17:45:37 yeah, thats anoying and useless. 17:46:01 nirik: but unless the proventesters know all the scenarios in which mdadm is used, and cover all those scenarios, then the testing is incomplete. I'm just trying to figure out if people on the other side of this issue are truly arguing that it needs to be complete. 17:46:28 In an ideal world we certainly wouldn't push changes without every single aspect of the change being tested in all environments 17:46:31 But that's clearly impossible 17:46:34 dledford: we know we're never going to get 100% acceptance testing - that's not the goal, either 17:46:39 so, thats +3 to t8m's proposal... needs at least 2 more for passing. ;) 17:46:40 pjones: the instructions say not to do that, if you catch anyone doing it, let me know and i'll bop 'em. i think i caught the last few cases. 17:47:20 OK, then if 100% isn't possible, then it's just a matter of where to draw the line. Right? 17:47:46 nirik, if I count right there were already +4 17:47:57 dledford: right. additionally if you have a machine using mdadm, and update to the version under testing, it should work at least as well as the old version, right? 17:48:13 nirik: one would hope, yes 17:48:15 t8m: ah, right you are. ;) 17:48:27 adamw: I saw one earlier today; I'll try to find it for you after the meeting. 17:48:34 nirik: certainly that's the goal, and if I get test feedback that it doesn't, a new build is the next thing the update gets 17:48:59 nirik: mjg59 and I are both clearly -1 ... how many are present today? 17:49:20 notting and ajax ? 17:50:04 i have extreme difficulty forming an opinion, let alone expressing one. 17:50:05 nirik: do we have stats on > 2-week critpath that i missed? 17:50:28 notting: not that I know of. I was looking at the old_critpath emails in my bodhi admin folder. ;) 17:50:41 was curious what the numbers are 17:50:42 So, if it's about where to draw the line, I would suggest that when users can't boot on what's in updates at the moment, and what's in updates-testing is a known solution, then the process has drawn the line at the wrong place. 17:51:00 dledford: the trick is getting people to say that. ;( 17:51:02 dledford, +1 17:51:26 and if i'm honest i think the whole discussion is badly framed and i don't really feel like condoning it by voting 17:51:45 nirik: it was in the bugs for my updates, but these people aren't proventesters. But in some cases, it's more important that you have the right hardware than it is to be a proventester. 17:51:46 dledford: I don't think "where to draw the line" is quite how I'd phrase it, but sure. 17:51:50 ajax: fair enough. Any alternate ideas/proposals? 17:51:53 but i also have vanishing interest in trying to reframe it because nobody can be civil about, so. 17:52:10 dledford: being a proventester is really easy. ;( 17:52:24 nirik: which makes you wonder if it's just a silly barrier to entry 17:52:27 * nirik sees ajax has been reading the ticket. 17:52:31 "it does not fix $unrelatedbugthatisnotaregression" -> -1 .... 17:52:36 i did my part and applied for the proventester merit badge. 17:52:39 pjones: or just a 'didn't know' much more likely. 17:52:42 Ok so how about we look at this differently 17:53:06 Drop the proventester requirement but allow maintainers to flag specific bodhi comments as to be ignored 17:53:07 * nirik listens to mjg59. differently is good here I think. 17:53:46 so, no more proventesters at all? 17:53:58 that helps with stray negative feedback. not so much with no feedback 17:54:03 nirik: it doesn't seem to accomplish much. 17:54:05 how would this help us with no feedback? 17:54:10 That would lower the barrier to getting testing but would allow maintainers to avoid the situation where a package is being -1ed by utter irrelevance 17:54:17 it wouldn't - it only helps the intermediate case 17:54:34 mjg59, currently the negative votes are no blocker 17:54:34 Or also, if the maintainer is paying attention, to ignore +1s that are clearly not real testing 17:54:41 I don't know that usless -1's are that much of an issue, are they? 17:54:56 nirik: The kernel gets a bunch. I don't think it matters in the current case. 17:55:05 nirik: they get the karma down so even more testers are needed 17:55:07 nirik: I've seen several on my packages 17:55:26 the kernel usually gets enough +s to cancel that out... 17:55:27 nirik: they are for me, but moreso in bugs that in bodhi, I have to sometimes clone a bug to separate out users that attach distinctly different issues to an existing, similar sounding bug 17:55:46 w.r.t. t8m's proposal, i'm moderately in favor of it as a means of getting things done, but it does appear to be solving the problem in the wrong place. i would be more interested if it were time-limited in some way that we could revisit it 17:55:48 * nirik fears we are drifting from the topic here. 17:56:18 ok, how about an alternate of t8m / dledford's: 17:56:18 Requiring a proventester to provide feedback means that there's less incentive for a non-proventester to bother 17:56:51 The question is really whether proventester feedback is more reliable than non-proventester feedback 17:56:57 if a critpath update has been in testing for more than 2 weeks, has no - karma, then a proventester may add a +1 to get it moving. 17:57:17 ie, no bodhi changes, we just allow proventesters to do this, and when revisited we just ask them not to anymore. 17:57:32 hopefully because things have gotten testing and it's moot 17:57:45 notting, well we can certainly revisit the policy any time 17:58:02 t8m: but without a timeout, we have no incentive/urgency to do so 17:58:29 nirik: a +1 w/o testing? 17:58:39 * nirik tries to think of a time bound that would make sense here. 17:58:41 yeah. 17:59:16 nirik, that seems like a ugly kludge 17:59:20 yeah 18:00:03 ok, proposal (we have been at this a while): try and drum up more testers, revisit next week, hope for inspiration on a way to solve it. 18:00:17 which is mjg59's I guess. 18:01:18 sure, why not. 18:01:26 I'm +1 to that 18:01:39 I'd give +1 to it only if in case we do not come up with any innovative way how to get more proventesters/testers to +1 updates then we automatically accept at least the timeout proposal :D 18:01:42 since we don't seem to have enough votes for the timeout. 18:02:19 Well, since the only other alternative is continuing to talk in circles for another hour, +1. 18:02:48 obviously I've said I'm +1 to mjg59's proposal several times now. 18:03:12 since we seem to not be making progress, sure, +1 18:03:18 #agreed will try and get more testers and revisit next week. 18:03:34 and more fun: 18:03:36 #topic #664 Older releases need a different approach for updates 18:03:41 .fesco 664 18:03:42 nirik: #664 (Older releases need a different approach for updates / karma) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/664 18:03:54 this also would be better with more testers. 18:04:32 as much as more meetings sounds unfun, I kinda like the idea of a proventester meetup... 18:04:44 would get more press for testing, help with test plans, etc. 18:05:48 a two week timeout is proposed here too. 18:05:51 What do we have in the way of machines available for automated testing? I know AutoQA exists, but that's about it, and I have no clue what it's functional capabilities are... 18:05:55 "automatic pushes after lets say 2 weeks if at least one proventester has tested this package and found no regressions. This should also apply to critical path updates. 18:05:55 " 18:06:06 dledford: Right now autoqa is pretty much just a dependency check 18:06:09 i don't get it - if it's not critical path, wouldn't it fall under the 'normal' timeout? 18:06:39 notting: it should 18:06:47 notting, yeah, this seems to change things only for critpath removing the need for regular tester +1 18:06:47 notting: yeah, sounds like somebody doesn't understand 18:06:49 autoqa isn't yet at the point where we could start doing widespread functional testing of packages. 18:07:04 another case of these updates... openchrome in f14 has been in updates-testing for 3 months. ;) 18:07:20 it's still a reasonable amount of work to add any given single test to the autoqa framework at present; obviously this should change, but we're not at that stage yet. it's also quite difficult for third parties to contribute tests to autoqa right now. 18:07:58 on the discussion as to whether proventesters are necessary: the proventester bar is set low on purpose, on the theory that it's better to accept lots of people and then kick out anyone who doesn't do the job right than it is to try and guess beforehand whether people are going to do the job right. 18:08:19 mjg59, adamw: my thought wasn't widespread functional testing, my thought was automated vm update/reboot cycles. The stated goal of CRITPATH is to verify that bootup/upgrade still works after a package update. Wouldn't that be fairly automatable with the right set of predefined virtual machines? 18:08:21 nirik, perl as well 18:08:26 in practice, we've not yet had to kick anyone out of being a proventester. there was one case where we didn't accept the application because it was, figuratively, written in bright red crayon and badly misspelled. 18:09:07 t8m: it got karma a few days ago 18:09:13 dledford: Yes, I agree. It's just not implemented 18:09:17 nirik, yeah, finally 18:09:23 dledford: it'd be an interesting target, sure. i suspect it may wind up being a bit more work than it seems, but it's probably achievable. (it's already a surprising amount of work to do an entirely automated test installation, as a frame of reference.) 18:09:31 dledford: Stronger autoqa would be a great way to avoid some of the problems we have here 18:10:01 if fesco has some specific targets they feel would be good for autoqa to go after, i can certainly take those to the autoqa team and see what they think. 18:10:23 mjg59, unless we would lower the requirements on karma then it would not fix the problem with updates stalled in testing 18:10:50 t8m: I know. I think we could consider lowering the requirements if autoqa was sufficiently strong 18:10:56 For some set of packages, at least 18:11:04 in general we're at a sticky point with autoqa right now: the team doesn't want it to be their job to develop and maintain specific tests, they want to maintain the framework. but at the same time, the framework's not quite yet at the point where it's easy for people to contribute tests. but i can try and short-circuit that somehow, for specific goals that are of major importance. 18:11:09 adamw: automated system testing would be nice 18:11:09 so, for this I think we are in the same boat as the last one... lets revisit next week? 18:11:24 adamw, what about the proventesters meeting? 18:11:36 adamw, are you fine with this proposal? 18:11:38 t8m: it's a nice idea: it would require someone with the motivation to do it and keep it going 18:11:54 t8m: the only worry i have is it may end up like bugzappers, which sorta comes and goes when someone decides to care about it for a few months 18:12:12 * pjones holds his tongue about bugzappers 18:12:14 i personally wouldn't commit to doing it because i know it would be the first thing out the window when a release crunch was on, for me 18:12:39 so, if someone wants to make a commitment to being The Person for proventesters and running a meeting every week and being really punctual about it, sure 18:13:01 hehe so we are again at the low on QA resources problem 18:13:04 yup. 18:13:08 Might I suggest that the problems with CRITPATH updates are at least in part due to CRITPATH being overly widely used to denote a variety of different problems, and with those different problems needing different testing to address, 18:13:24 dledford: yeah, that's becoming quite clear 18:14:06 So, then I would follow that up with "A better definition of CRITPATH would likely help lead to a better process/policy and help resolve these issues." 18:14:23 yeah, that's certainly an avenue worth looking down. 18:14:44 right now, we basically say 'these are the packages you need to boot and update a system, so all these packages plus their deps are critpath'. 18:14:48 dledford: I don't see how you're going to get mdadm off it though ;) 18:14:54 one obvious problem is where a critpath package depends on another package *for non-critpath functionality* 18:15:30 pjones: if critpath was narrower, it'd be more feasible to make a super-special effort to go out of our way to test critpath updates...right now critpath is so broad that's kinda tough. 18:15:32 pjones: I don't expect to, I expect mdadm to be CRITPATH - bootup/hardware dependent. However, automated testing can verify mdadm's requirements. 18:15:43 so tightening down critpath would help even the stuff that remained on critpath, indirectly. 18:17:02 so, what can we conclude here? revisit next week? or does someone have a proposal? 18:17:15 I'd offer to do the proventesters meetings, but I don't know if thats overcommiting my time. 18:17:30 I guess I could try and start it and see if I can pawn it off on someone after it's moving. 18:18:07 that's the spirit. 18:18:18 yeah. like i said, i'd love to say i'll do it, but i know i'd do it for two weeks and then drop it because i had twelve bugs to test for an RC. 18:18:58 fine, I'll try and start it. 18:19:24 #action nirik to try and start up proventesters meetings 18:19:38 if you could do #action items for me for anything that's on qa that comes out of this discussion that'd be great, save me reading through the whole scrollback 18:20:37 #action adamw to talk with QA about getting more testers involved 18:20:51 #action adamw to talk with QA about getting more proventesters/more press for proventesters. 18:20:55 I think thats it? 18:20:58 I propose we initiate a process to subdivide the CRITPATH umbrella into more meaningful groups. 18:21:00 yay non-specificity! 18:21:04 autoqa thing? 18:21:11 dledford: That does sound sensible 18:21:17 Yes 18:21:40 I think that's something we need to look in to doing, yes. 18:21:41 If you know why something is CRITPATH, then it's easier to design specific testing. Whether autoqa or instructions for proventesters. 18:22:32 right now critpath is defined from comps... 18:22:34 http://kojipkgs.fedoraproject.org/mash/branched-20110912/logs/critpath.log 18:23:42 q 18:23:51 grr, focus 18:23:51 dledford: would you be willing to work on a proposal for this? or should we discuss over the week on devel/etc and see if we can come up with something? 18:24:00 nirik: right now CRITPATH includes everything needed to "boot or update your system". I would suggest that these are two very different actions with different testing requirements. 18:24:16 dledford: Actually, it's more than that 18:24:18 nirik: yes, I'm willing to work on such a thing (aka, I'll volunteer) 18:24:23 It's boot or update a GRAPHICAL system 18:24:37 dledford: excellent. ;) If you need any info, happy to help... 18:24:42 We probably also want to differentiate headless critpath vs GNOME 18:24:46 sgallagh: fair enough, but boot and update are still too very different jobs ;-) 18:24:49 dledford: i think another area to look into is the one i highlighted: cases where a critpath package depends on some other package, but that dependency does not relate to critpath functionality 18:25:12 of course, that could be very hard to deal with without a lot of manual weeding. or optional dependencies, to bring up another unicorn. 18:25:55 nirik: I will most certainly need help because I don't know the ins and outs of how CRITPATH is pulled today, nor the discussions that have already been had over the process, so someone to work on this with me would be greatly appreciated. 18:26:12 dledford: I think the short version is: if it's in the default install, it's CRITPATH 18:26:33 dledford: https://fedoraproject.org/wiki/Critical_path_package should mostly document it. 18:26:59 adamw: yep, I think it likely is a weed garden right now, but it should be looked at 18:27:00 Right, I was wrong. 18:27:20 adamw: thanks for the link 18:27:50 #action dledford to work on a proposal to split up critpath into different functional groups. 18:28:16 so, revisit this next week? or close with the action of the proventesters meeting? 18:29:54 +1 revisit, same problems as previous ticket 18:30:46 nirik, I am +1 to revisit as well 18:30:55 +1 18:30:55 ok... 18:30:56 +1 18:30:57 bit late to this discussion.. fyi one problem with proven testers is that we dont have any testcases to follow 18:31:08 * notting is +1 to revisit 18:31:11 #action will revisit this ticket next week. 18:31:21 Viking-Ice: yeah, thats a related issue for sure. 18:31:28 so a lot of testing that gets done results in Hey I updated nothing obvious broke 18:31:34 karma +1 18:32:07 which might still result in borked system of someone that actually is running things properly 18:32:20 just something to bear in mind 18:32:41 Viking-Ice: yeah, another thing a proventesters meeting could look at: update foo has no karma, hey, lets look at making a test plan for it... 18:32:48 and test it in the bargin 18:33:28 #topic #666 On-demand loading of socket activated services? 18:33:32 .fesco 666 18:33:34 nirik: #666 (On-demand loading of socket activated services?) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/666 18:33:36 the ticket of the beast. 18:34:07 my answer to this is "yes". 18:34:14 I don't see why we wouldn't want to support this 18:34:20 So, yes 18:34:21 but there are probibly more questions around it. 18:34:47 don't see why we wouldn't, either 18:34:49 replacing xinetd with systemd socket activated services seems like a win to me. 18:34:51 uhum not so sure about that proposal I probably finish converting all the xinetd stuff tonight 18:34:54 yeah, seems okay 18:34:55 nirik: without knowing what those questions are...? 18:35:07 as a raw feature i don't see any reason to forbid socket-activated services 18:35:09 but, there's also set to start on install right? 18:35:17 The biggest concern I see is the one raised by Steve Grubb 18:35:33 xinetd doesn't enjoy the same kernel protections that init does 18:35:36 is it the same issue he always raises about everything that involves software? 18:35:57 I don't much care for having a network-facing init process 18:36:40 It can't be reasonably contained by SELinux or other MLS mechanisms. 18:37:00 The network facing code should be entirely auditable 18:37:19 nirik: set to start on socket is effectively set to start on install 18:37:28 mjg59: My problem is that PID 1 should NOT be network-facing 18:37:39 sgallagh: Well, it is 18:37:42 So 18:37:44 A vulnerability there is a network-enabled rootkit 18:37:45 from what I read (although I have not looked at code), the right security measures are taken. 18:37:52 sgallagh: it can't be labeled? it can't have a context? 18:38:06 notting: ok... so we allow anything with a socket start to start by default on install then? 18:38:07 sgallagh: A vulnerability in any network facing code that binds to low ports is probably a network-enabled rootkit 18:38:21 pjones: It can't be killed (short of bringing down the machine) in the event of an intrusion 18:38:37 sgallagh: That really doesn't seem like a relevant issue 18:38:41 sgallagh: so? 18:38:44 kill -STOP 1 18:38:53 you also don't want it to be killed; you want fork to return an error or whatnot. 18:39:18 (or exec, as the case may more often be) 18:39:55 but if we're really going down that level of separation, just make init fork. 18:40:07 .fas llaumgui 18:40:07 llaumgui: llaumgui 'Guillaume Kulakowski' 18:40:32 Am I really the only one here who prefers the classic UNIX "do one thing and do it well" approach to processes? 18:40:34 hum sorry, is not french meeting... 18:40:58 sgallagh, certainly not the only one 18:41:00 It is doing one thing. It's launching processes. 18:41:03 llaumgui: we may be running long. sorry. 18:41:05 I like a lot of what systemd does, but I'm not comfortable with it replacing xinetd. 18:41:12 sgallagh: no, you are not only one 18:41:45 i think that phrasing maxims that way isn't a constructive way to talk about things. 18:41:48 sgallagh: as I see it xinetd was really a gross hack to try and defer deciding what to start with init. 18:42:16 and if init can make that decision, that's less gross. 18:42:23 * mmaslano would give this question to security team 18:42:37 Yeah, I think we're off-topic a little. 18:42:47 The more general case is whether to support socket loading at all 18:42:59 well, we don't really have a security team. ;) I guess we could ask RH security folks... 18:43:07 I think that would be a bad idea 18:43:17 I disagree with notting's assertion that socket loading is the same as starting by default. 18:43:35 I think it should probably have to meet the same restrictions, but it does have different effects. 18:43:35 how isn't it? 18:43:48 sgallagh, +1 18:43:53 Not the least of which being that if the service is never requested, the system won't be wasting those resources. 18:43:59 (or using as much power, etc.) 18:44:14 sure, but that's completely unrelated to the point he was making, as I read it. 18:44:24 (French speaking fedora meeting on #fedora-meeting-2) 18:44:36 pingou_: sorry for overlapping your meeting. ;( 18:44:46 It sounded to me like he was saying "If this is allowed, then just start it and don't bother with socket activation", though I could have misinterpreted 18:45:05 nirik, no pb 18:45:20 if your package is setup to ship a socket file, it's enabled when you install, right? there's no way to ship it and also have it disabled? 18:45:31 Viking-Ice: ^^ 18:45:44 nirik: there is 18:45:53 it's the same as for other services 18:46:03 ok, then I am getting confused. 18:46:27 i merely meant that if you enable the socket-activated the service, it's the same 'start by default' vector (or attack surface) as an enabled 'normal' service 18:46:40 we are at 15minutes. Shall we vote on the question in the ticket and defer more things until FPC asks us? or continue? 18:46:53 ok, that makes sense then. 18:47:06 I'm +1 on allowing it. 18:47:41 * notting is +1, as well 18:47:41 I'm +1 for allowing it for any package that wants to implement it. 18:47:49 * cwickert is pretty late, sorry 18:47:52 +1 18:47:54 +1 18:47:55 welcome cwickert 18:48:09 #agreed socket activated services should be allowed. 18:48:11 But only on UNIX sockets or localhost addresses 18:48:37 I'm still opposed to externally-facing socket activation at this time. 18:49:14 If a formal security audit is performed and green-light's it, I will revise this stance. 18:49:41 #undo 18:49:41 Removing item from minutes: 18:50:01 thats 4 'in general' and 1 with stipulations? 18:50:07 looks that way 18:50:16 +1 in general 18:50:49 I agree with sgallagh 18:50:49 #agreed socket activated services should be allowed. 18:51:20 +1 for the record 18:51:29 * cwickert still catches up reading the backlog 18:52:04 so, I gather FPC would then be updating/creating guidelines on these? 18:52:17 so, they would be the ones to voice concerns about external listening? 18:52:27 I think so 18:52:58 yeah 18:53:13 looks like it would be easy to specify... 18:53:16 anyhow. 18:53:39 #topic #665 F16Feature: OpenStack https://fedoraproject.org/wiki/Features/OpenStack 18:53:39 .fesco 665 18:53:40 nirik: #665 (F16Feature: OpenStack - https://fedoraproject.org/wiki/Features/OpenStack) - FESCo - Trac - https://fedorahosted.org/fesco/ticket/665 18:53:53 very late feature, but it's done supposedly 18:54:16 heh I see 80% there 18:54:24 and also unlike java doesn't interact with other things 18:54:33 yeah, 'scope' doesn't say which parts are done and which aren't 18:54:47 The trac ticket says "complete" 18:55:30 rbergeron: any ideas on above ^ 18:55:35 AFAICS only the test day is not done 18:55:53 oh wait, 'scope' does. 18:55:54 * notting is blind 18:56:14 if it's only the test day, I think we should do it 18:56:24 cwickert: +1 18:56:27 it's test day, and... I think that's about it, tbh. a bit of documentation. 18:56:32 ok then +1 18:56:35 though most of it is pretty replete. 18:56:36 in fact I wanted to make the test day a requirement since they missed alpha 18:56:58 lets say +1 as long as they make the test day soon (tm) 18:57:05 Yeah ok +1 18:57:07 well we're grabbing the test day on 10-20, that's the only one left. 18:57:11 for all cloud stuff 18:57:13 +1 here. 18:57:22 +1 given the test day is cheduled 18:57:27 scheduled, too. 18:58:07 yeah, +1 18:58:09 * rbergeron will go add it now, but it's been fully discussed 18:58:18 +1 18:58:29 * rbergeron passes out bacon and cookies 18:58:30 #agreed late feature is approved with test day. 18:58:43 I'm not sure how I feel about bacon cookies 18:59:08 sgallagh: works with chocolate and sea salt quite well. 18:59:23 #topic Open Floor 18:59:24 * pjones is +1 as well 18:59:30 ok, any open floor items this week? 18:59:41 oh, and: who wants to chair next week? ;) 18:59:44 sorry, i have to step out now 18:59:52 Yeah, I had a talk about Chrome today with spot and others. 18:59:59 nirik: I can take it for next week 19:00:01 Strong recommendation is NOT to include it in Fedora. 19:00:06 mmaslano: thanks! 19:00:11 #action mmaslano will chair next week. 19:00:16 sgallagh: debian unbundled it 19:00:28 cwickert: no, they didn't. 19:00:31 but it's quite a lot of patches 19:00:34 spot: oh 19:00:36 cwickert: at least, not the last time i looked they didn't 19:00:48 spot: when did you look? 19:00:54 cwickert: about a month ago 19:01:04 ok, then it's not unbundled 19:01:18 Also, we had a thread about maintainers +1ing their own updates, which we might revisit with our thinking about updates process... 19:01:22 spot: but Debian has a no-bundled-libs policy, too 19:01:32 cwickert: so does ubuntu, but they have it as well 19:01:44 nirik, yeah we should discuss that next week as well 19:02:04 nirik: +1 for maintainers giving +1 to their own packages. they know best what/how to test 19:02:27 * cwickert is aware that this can be abused 19:02:35 cwickert: lets talk about it next week, I fear it would take us way over time to do it today. ;) 19:02:40 agreed 19:02:50 I don't think it really matters. A maintainer could just choose to reduce the karma to the point where it's already met anyway 19:03:04 (excepting critpath, of course) 19:03:17 even there. 19:03:38 s/reduce the karma/reduce the positive karma requirement/ 19:03:44 well, I guess they would need another sockpuppet account there. 19:03:50 anyhow, anything else for open floor? 19:04:29 * nirik will close out in a minute if nothing more shows up 19:05:20 thanks for coming everyone! 19:05:23 #endmeeting