16:00:17 #startmeeting Fedora QA meeting 16:00:17 Meeting started Mon Nov 2 16:00:17 2015 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:17 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:21 #meetingname fedora-qa 16:00:21 The meeting name has been set to 'fedora-qa' 16:00:29 #topic Roll call 16:00:39 ahoyhoy folks, who successfully navigated the clock change? 16:00:51 and didn't drink themselves to death testing F23? 16:01:07 * roshi is here! mostly adjusted to time and not drunk to death, yet 16:01:22 * pschindl is here 16:01:50 * kparal is here 16:03:03 .fire roshi clearly not trying hard enough 16:03:03 adamw fires roshi clearly not trying hard enough 16:03:21 * tflink is here 16:03:24 lol 16:05:04 alrighty 16:05:08 #topic Previous meeting follow-up 16:05:35 oh hey looks like there isn't any 16:05:40 #info no action items from previous meeting 16:05:52 so we can go right on to... 16:06:06 #topic Fedora 23 status and final tasks 16:06:28 #info RC10 was signed off as Fedora 23 at the Go/No-Go meeting on Friday 16:06:39 #info thanks to all for the huge amounts of testing 16:06:43 * kparal is working on documenting some of the remaining commonbugs 16:06:55 yeah, that's one of the things we need to do now 16:07:11 * roshi put a couple of the cloud ones on there last week 16:07:16 #action kparal and adamw to update Common Bugs 16:07:26 kparal: we need to switch it over to the 'release mode' too, i can do that if you like 16:07:38 huh? 16:07:49 kparal: the boilerplate text changes a bit at release time 16:07:53 there's a template for it, i can do it 16:07:58 I see, thanks 16:08:26 we also need to take care of updates 16:08:52 the dnf-plugin-system-upgrade packages for F21 and F22 ideally need to get into stable, but i found a dep issue with lorax last week 16:09:07 #action adamw to check in with bcl and wwoods on dnf-plugin-system-upgrade updates status and lorax 16:09:41 other than that, i need to check in on the kf5 updates and make sure we don't regress stuff there... 16:09:55 and there's the freeipa upgrade mess to document 16:10:10 that's fun 16:10:48 anyone think of any other final F23 tasks? 16:11:16 * roshi will being doing the heroes of fedora post this week 16:11:22 cool 16:11:29 you'll want to use relval git master, or wait for me to cut a release 16:11:49 otherwise relval will trip up somewhat on the double-digit compose issue. not sure how bad it'd affect hte stats commands, but just in case 16:12:05 sure, I can hold off 16:14:16 alrighty 16:14:42 there's some other housekeeping crap i'll do, mostly creating f25 blocker bugs, if anyone's interested see https://fedoraproject.org/wiki/BugZappers/HouseKeeping and https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers 16:15:16 #topic Fedora 23 retrospective 16:15:40 so, we didn't put the retrospective wiki pages up for f22, i can do that for f23 if there's interest 16:15:46 but i figured we could kick it around here too 16:15:58 the idea being, thoughts on how the f23 cycle went - what was good, what was bad, what can we improve next time 16:16:51 openqa was awesome 16:17:20 more coverage next time! :) 16:17:34 it would be better if everyone would get their stuff done and QA didnt have to go into hero mode every release 16:18:10 aka features 16:19:57 Southern_Gentlem: yeah, this is something that comes up frequently... 16:20:04 not sure what we can do about it in concrete terms though 16:20:35 yell louder so people know that they, too, have to test? 16:20:42 kparal: for f24 cycle we should have a lot more workers for openQA and hopefully we can also ditch i386 testing, which would be nice 16:21:11 the only thing I can think of (and I'm not fully suggesting this) would be to drop features (even major ones) if they're not ready 16:21:15 roshi: yeah, to pivot a bit, i'm worried in general that the project as a whole seems to be more or less assuming the release blocker / validation process is the Magic Thing That Tests Everything 16:21:41 i keep having conversations which are implicitly predicated on the idea that all important bugs *must* be release blockers or FEs and if a bug isn't one of those things it's just not going to get fixed 16:21:44 which isn't actually how it works 16:22:07 tflink: that's supposed to be a part of the Change process, i think so far we've dropped maybe one or two Changes 16:22:36 * danofsatx has arrived. the meeting can start now. 16:22:47 uh, okay, we'll, uh, start half way through the f23 retrospective 16:22:57 the only thing I can think of is to start talking more about testing in general in the hopes that some non-qa folks would read it and start understanding more 16:23:03 #info kparal says openQA was great, more coverage next time 16:23:25 #info southern_gentlem notes that as usual, we had significant changes landing late 16:23:50 yeah, that's true adamw 16:24:06 i've got one: we badly need to come up with a better way of handling serious bugs that the 'release blocker' process doesn't really cover 16:24:22 so i'm gonna write one! but if anyone else has great ideas for one, please let me know so i don't have to 16:24:25 adamw: we could start tracking "serious/significant" buts 16:24:32 adamw: Can/should we publicly shame the features that landed late? 16:24:34 bugs 16:24:44 I think 'release blockers' covers those issue, we just consider it to be a 'compose blocker', which is a different thing 16:24:46 tflink: that was more or less my idea, yeah, i'm a deeply unimaginative person: more trackers and more process! everyone loves process. 16:25:10 kparal: yeah, to some degree it's semantics, but the process as currently constructed is really about the compose process 16:25:12 adamw: it'd be nice to make it a more hands-off "process" that doesn't feel so much like a process 16:25:37 * danofsatx has a couple of ideas 16:25:40 fire awawy 16:25:47 something along the lines of "submit bug as a significant problem here", can be voted on by anyone, vetoed by a subset of folks 16:25:59 I'll let 'em percolate for a bit, though, and send to list. They're not ready for public consumption yet. 16:26:40 danofsatx, getting our hopes up... 16:26:50 tflink: that's more or less what we had years ago when there was no formal review of the blocker / 'Target' trackers; anyone could just throw a bug on there. what tended to happen was no-one really cared 16:26:54 it's called "antici...........pation' 16:26:57 tflink: especially about the 'Target' tracker 16:27:07 ooh, you tease, dan 16:27:12 i think that we'll hit that no matter what, honestly 16:27:23 it'll be an issue, rather 16:28:06 "yet another tracker and system that devs dislike" 16:28:08 tflink: to a degree, yeah, but i think - being very machiavellian - there's an effect where the more officious process nonsense is going on, the harder it is to ignore stuff 16:28:09 yeah, I think so too 16:28:34 partly the current blocker process works because we actually enforce it by slipping releases, but it also works just because we make a lot of noise about it and bug people and send them emails 16:28:50 yeah, I don't think it's an insurmountable thing - just that it'll take more work than just "hey, here's this list of bugs that think are significant" 16:28:58 if we have a system where we more or less just make a list of bugs but don't have a big gong-banging din-making process around it, it's easier to ignore 16:29:36 the other possible trap is giving submitters too much hope if it seems like a "vote here for whether or not bug X should be fixed" 16:29:40 software engineering as public theatre :P 16:29:46 tflink: yep, indeed 16:29:53 * danofsatx wonders if we can set an "Acceptable" number of known bugs 16:30:08 kinda like the terminology issue we had with NTH bugs 16:30:08 danofsatx: i think that'd get extremely susceptible to fuzzing 16:30:15 not just blocker-worthy, but total bugs 16:30:21 yeah 16:30:36 that strikes me as a terrible idea 16:30:51 it would make people take notice 16:31:02 when you start putting artificial limits on the process, the gaming starts and things get much worse 16:31:05 I didn't say it was a good idea, it was just a thought 16:32:42 perhaps if you want a feature in the next release, you have to commit to so many testcase runs per compose 16:32:57 danofsatx: sorry if I seem like I'm jumping down your throat - that's just an example case I've seen used to illistrate the destructive power of bad management 16:32:58 which we can track with relval - hold features ransom for testing 16:33:26 tflink: no, I understand - I was just thinking out loud. I expected to get shot down ;) 16:33:26 what's a testcase run? 16:33:34 yeah, i was wondering that 16:33:39 result on the matrices 16:33:44 it'd get gamed 16:33:47 is the simplest thing I could think of 16:33:52 roshi: what if the feature isn't really anything to do with release validation? 16:33:56 and it would dilute the usefullness of the results as a whole 16:34:16 I don't care about that adamw - they agree to doing release validation in addition to testing their feature :p 16:34:16 if it is a marketed new feature, it should be validated 16:34:28 sure, not arguing with that part 16:34:40 so anyway, yeah, for the 'special blockers' stuff - which is USB writing app issues, upgrade issues, anything that would get fixed somewhere outside the release media - I will draft something up (taking account of all these thoughts), unless anyone else wants to take it 16:34:44 metrics should be used to ask questions, not answer them 16:34:46 yeah, I'm not really sure there is anything *to* be done 16:34:53 roshi: haha. buy one feature, get one bunch of testing free 16:34:56 * linuxmodder late 16:35:12 does anyone else want it? 16:35:15 ahoy linuxmodder 16:35:36 adamw: i can help but I don't have the spare cycles to take it on right now 16:35:41 rgr 16:35:56 adamw, howdy saw your ipa post that is nasty 16:36:41 I picked up a clusterf of a webmaster gig or I'd be up for cycles 16:36:47 #action adamw to draft up proposal for better handling of 'special blockers' 16:37:13 so another thing i think bears mentioning, though i don't have any fixes: Test Days are kinda falling off lately 16:37:37 again it's something we've noticed for a while and even been trying to counter, but we seem to getting sort of sucked into release validation all day every day 16:37:41 * danofsatx noticed the distinct lack of Test Days for 23 16:38:22 adamw, next one ping me a few days ahead I'll add to my blog / G+ etc 16:38:23 between release validation and working on tooling, that's where all our cycles have been going recently 16:38:35 I think it'll get better as tools get closer to production 16:38:54 openQA coverage, taskotron, etc 16:38:57 well, we can hope ;) 16:39:11 someone insert xkcd graph here 16:39:13 yup 16:39:14 well, there are a few of us that don't have the skillset for the tools, so why don't we get off our arses and run the Test Days? 16:39:32 it would be great if more folks helped out with running test days, absolutely 16:39:33 and it ' 16:39:43 for sure 16:39:45 and it's something you basically need no special knowledge or permissions or anything to do 16:39:54 I admit, I'm guilty of it myself. 16:40:07 https://fedoraproject.org/wiki/QA/SOP_Test_Day_management is the guide to running a test day, for anyone who hasn't seen it 16:40:16 only excuse I have is "Hey, life happens." 16:40:30 #info general agreement that Test Days have been falling off lately and it'd be good to get them more alive again 16:41:00 i'd note that it *does* take a decent chunk of time to run a test day really well, though, and it's great if you can commit to it - i used to spend a decent chunk of my working hours on them 16:43:01 adamw, come new year my schedule for such **Should** clear up fall /winter si always shit for me 16:43:08 that's good 16:43:41 i guess the other interesting topic in the f23 cycle was: i386 keeps freaking breaking 16:43:52 there are general project-wide moves afoot to stop caring so much about i386 16:43:58 one thing to note, for F24 - if Atomic is going to be a blocking deliverable, we should have a couple people getting familiar with it and docker so it can be tested 16:44:14 dropping i386 will make testing easier 16:44:15 should we throw our weight behind them? if nothing else it reduces our combinatorial explosion nightmares a bit 16:44:31 adamw, well the problem is third party software that requires i686 16:44:33 as I pretty much *only* did 32 bit testing 16:44:57 skype steam etc 16:45:13 Southern_Gentlem: you don't need an i686 install for those. 16:45:16 only the relevant i686 packages. 16:45:20 roshi, I'm attempting to wrap around dockeer 16:45:34 we don't have to stop building i686 packages at all, just stop blocking releases on the 32-bit media. 16:45:38 3rd party...do we care? it's not Fedora. 16:45:52 ^^ lol 16:45:52 and yeah, someone has to make the first jump and fedora has a proud history of being it 16:46:02 that's true 16:46:09 i suspect that if we stop blocking on 32 bit, it'll stop working well 16:46:15 we aren't that important in ourselves, but when we do something it creates a precedent for other distros to do the same and it comes with a heaping side order of implication that RHEL will do it at some point 16:46:32 adamw, i agree i dont see the need for 32bit media at all 16:46:45 tflink: people who really care about 32-bit third party apps can always do the necessary work to make sure their dependencies work alright 16:46:51 CentOS 7 didn't have 32 bit until just a couple months ago (and I'm not sure its "officially supported") 16:47:07 adamw: what if the installer doesn't work on 32 bit because it hasn't been tested? 16:47:12 danofsatx: yeah, i may be behind the times in terms of RHEL actually, i have no idea what RHEL ships / blocks on. 16:47:19 tflink: I'm quite sure steam's deps will be kept in good shape :) 16:47:42 tflink: well, i mean, that's kinda the question. 16:47:55 Some of us still use i686 machines. (Though mine seem to be dying off this year.) 16:48:10 tflink: do we get behind the various proposals to stop blocking on 32-bit media or shipping them at all 16:48:25 I say we back them on it 16:48:34 +1 to dropping it to a secondary arch 16:48:54 roshi: it's not quite the same thing as secondary arch 16:48:55 when the cloud WG changed their "main" deliverable, there were assurances that the base cloud images would still work even if they weren't blocking. i don't think that the same could be said about non-blocking 32 bit install images 16:49:18 and if there are no working 32 bit install images, why build 32 bit packages at all? 16:49:39 tflink: the use case I could see is, install from older media, and upgrade 16:49:41 tflink: see above. 16:49:42 Not blocking seems to be pretty reasonable. I think there should still be some way to easily install Fedora on an i686 machine. 16:49:56 brunowolff: the thing is, if we don't block on it working, it's very likely not to. 16:50:06 or, we could just do i386, since it will run on x86 hardware 16:50:10 and then it'd be done 16:50:10 adamw: for which concern? 16:50:12 * roshi ducks 16:50:19 tflink: running third-party stuff that's 32-bit only 16:50:28 i.e. Steam. 16:50:29 oh, good point 16:50:45 not sure how i forgot about that 16:50:56 * tflink blames monday morning with only 1 cup of coffee 16:51:05 fwiw, i'd vote entirely in favour of any move to drop i686 media. yes, there's a cost to doing everything, but we definitely need to reduce the scope of releases somehow 16:51:16 roshi: so we'd eventually be saying "install f22 and good luck upgrading to f26" 16:51:18 make libs available but not the os tree maybe? 16:51:40 I still have two i686 machines I use that typically run branched. So I am likely to notice busted kernels or live images, install specific issues not so much. 16:51:53 the Fedora 23 release has *61* images 16:51:55 tflink: it was an answer to "if we don't have 32bit installer why have 32 bit packages" - but that's where it'll likely end up, yes 16:51:55 yeah, I think that not blocking will effectively be the same thing as not building it 16:51:56 that's ridiculous 16:52:09 roshi: adamw's answer was better :) 16:52:17 * tflink just wasn't thinking correctly 16:52:24 if we ditched i686 that would cut out 21 of them, so we'd have 40, which is still ridiculous, but somewhat less so. :) 16:52:31 sure - but that's also what I'd have expected :p 16:52:45 We may want to have just one supported media for installing i686. 16:53:05 in practice that's probably the path we'll wind up going down 16:53:06 would that work? only have server netinstall for 32 bit? 16:53:17 what's actually happening is that various WGs are proposing to drop their 32-bit media 16:53:23 tflink: i think Server may well be dropping 32-bit for F24. 16:53:49 so it sounds like we have a few caveats but we're generally in favour of proposals to drop 32-bit media/stop blocking on them? 16:53:56 or some netinstall to reduce the number of images 16:54:16 I'm pretty -1 on not blocking on 32 bit media 16:54:25 either block on them or don't build them as part of the release process 16:54:59 i really think this is a FESCO question 16:55:08 tflink: yeah, that's a reasonable point 16:55:17 Southern_Gentlem: this is just if we, as QA, want to support the idea 16:55:22 not make the call 16:55:29 the "build and don't block on" thing is going to cause confusion and wasted disk space 16:55:35 Southern_Gentlem: in some cases it's up to WGs, in others up to FESCo, i just wanted to see if we had a common PoV on it as a group 16:55:36 s/is going/would 16:55:38 yeah, that's true 16:55:47 Southern_Gentlem: so we can then agitate for that PoV in WG/FESCo/whatever discussions 16:55:48 Wouldn't this be more a council thing than FESCO? 16:56:03 brunowolff: doesn't really matter, point is it's not us :) 16:56:14 i just think it's handy if we can say 'QA thinks X' and have this discussion to document it 16:56:26 adamw, how are you seeing 61 images 16:56:40 Southern_Gentlem: count them in the table at the top of https://fedoraproject.org/wiki/Test_Results:Fedora_23_Final_RC10_Installation 16:57:08 ok, a few are boot.iso dupes, so the real count is probably 54 16:57:23 Right, but we should tell a more encompassing group that supporting QA for i686 is a lot of QA work, for probably little benefit. 16:58:04 brunowolff: i was figuring wherever the discussions come up, we can provide our input 16:58:08 ok, so we're nearly at time 16:59:00 #agreed there's a general consensus among QA that 32-bit install media produce a lot of testing/bug fixing work for little benefit and we're generally in favour of dropping most or all 32-bit install media 16:59:05 that sound alright? 16:59:38 ack 16:59:59 generall in favour/wouldn't fight - depends on desired strength 17:00:02 ack 17:00:25 tflink: i prefer 'generally in favour' if you guys will wear it ;) i really really want to kill them with fire. 17:00:38 * tflink doesn't have strong feelings about it 17:00:56 it -> that particular phrasing 17:00:58 ack 17:01:22 ack 17:01:54 do we want to add "not in favor of having non-blocking 32 bit release media built as part of the release process" 17:02:17 I'd be ok with that 17:02:29 ack 17:02:42 block or drop :) 17:02:49 if it's a choice between that and no change at all i'd prefer that, but i'd prefer dropping them entirely over having non-blocking ones, yeah 17:03:10 wait, is it favor or favour? 17:03:18 how about: #agreed QA also doesn't think that having non-blocking 32-bit install media is a great compromise, if it's proposed, and would prefer to drop them entirely 17:03:30 danofsatx: former is US, latter is UK 17:03:31 yes 17:03:34 i'd make it a bit stronger 17:03:47 okay... 17:03:58 adamw: I know, just trying to cause trouble. I'll shut up now ;) 17:04:06 then: #agreed QA thinks it would be a bad idea to have non-blocking 32-bit install media, and dropping them entirely would be preferred 17:04:13 ack 17:04:31 * roshi is still kinda sad to see 32bit go, since there's still so much 32bit hardware out there... 17:04:33 like: #agreed QA also thinks that having non-blocking 32-bit install media would be a bad move, leading to stale bits in the release. If non-blocking install media is proposed, we would prefer to drop them entirely 17:04:36 ack 17:04:38 ack 17:04:41 ack 17:04:44 that works, too 17:04:49 and is shorter :) 17:04:50 ack 17:05:08 #agreed QA thinks it would be a bad idea to have non-blocking 32-bit install media, and dropping them entirely would be preferred 17:05:10 alrighty, we're over time 17:05:14 so very quickly 17:05:16 #topic Open floor 17:05:23 anyone have anything burning that can't hold for next week? 17:05:39 * roshi has nothing 17:05:45 see ya next week 17:06:11 thanks dan! 17:07:02 I saw some wierd size changes in the games live live image composes, For F24 I hope to have time to figure out what is going on if I still see them. 17:07:17 dgilmore probably has some idea, and bcl 17:07:50 They didn't seem likely to be caused by package changes, but rather something else in the composes. 17:07:52 alrighty, thanks for coming everyone! 17:08:04 brunowolff: yes, i mean, they're the folks who know the compose process and the tools 17:08:29 see you on the lists 17:08:31 #endmeeting