16:00:15 #startmeeting Fedora QA meeting 16:00:16 Meeting started Mon Feb 16 16:00:15 2015 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:16 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:22 #meetingname fedora-qa 16:00:22 The meeting name has been set to 'fedora-qa' 16:00:25 #topic Roll call 16:00:29 ahoyhoy folks, who's around? 16:00:34 * roshi is here 16:00:48 * kparal just joined 16:00:48 * mkrizek is here 16:01:02 * satellit_e here for 3/4 hr 16:01:03 * nirik is lurking in the back by the bacon 16:01:15 * pschindl is here 16:01:31 * tflink is alwo here 16:02:02 * jskladan lurks 16:02:39 #chair stellit_e jskladan 16:02:39 Current chairs: adamw jskladan stellit_e 16:02:47 and now let's try that gain with real people 16:02:51 #chair satellit_e 16:02:51 Current chairs: adamw jskladan satellit_e stellit_e 16:02:57 * dextone listening 16:03:15 * jreznik is around 16:03:31 thanks for coming out, everyone 16:03:39 all having a good monday? 16:03:52 oh yeah, got a bunch of snow last night :) 16:04:03 damn yooooouuu 16:04:15 http://www.snow-forecast.com/resorts/Whistler-Blackcomb/6day/mid 16:04:23 rugby world series over the weekend, what's not to like? :) 16:04:36 almost tuesday here 16:04:50 so you can make a very accurate assessment! 16:04:53 where's here? 16:05:04 Indonesia :) 16:05:05 * adamw counts...er...india? 16:05:09 ah, not too far. 16:05:12 haha 16:05:18 (time zone speaking.) 16:05:26 it's nice winter-spring weather here :) 16:05:44 alrighty, gonna skip previous meeting follow-up if no-one minds as the only action item was for the test day and we have a test day topic anyway 16:05:57 sooo moving right along to: 16:06:02 #topic Fedora 22 Alpha status 16:06:21 well, obviously, about a dozen manual testers, anaconda CI, and coconut all tell us...it's broke. 16:06:51 in case anyone didn't see it yet, current f22 installer showstopper is https://bugzilla.redhat.com/show_bug.cgi?id=1192702 , will likely be resolved in the next day or two 16:07:20 * satellit_e installed with calamares but not in anaconda 22.19-1 16:07:30 aside from that, though, wanted to get a broader overview of where we are - anyone have any thoughts or notes? 16:08:43 .fas corey84 16:08:44 Corey84: corey84 'Corey84' 16:09:20 not really, though I think we got some good coverage of anaconda from the testday 16:09:29 the net install still broke ? havent had chance to check 16:09:41 yes no boot.iso today 16:09:46 that works 16:09:50 right 16:10:10 yeah, i think we're mostly in decent shape; there are various improvements we could make to the test case set but we can still do that as we go along i guess 16:10:18 release criteria revisions mostly got done i think 16:10:50 dgilmore: nirik: either of you around? any estimate on the difficulty of doing tc1 once we have a non-showstopped package set? 16:13:21 * Kinokoio is here 16:14:16 welp, guess not. 16:14:42 moooving right along 16:14:52 * Corey84 hopes to get this MBP i7 equipped to make this testing much easier 16:15:07 #info Fedora 22 status is that we're waiting for showstoppers to clear and 22 Alpha TC1 to land, we have reasonable coverage so far through nightlies and anaconda/DNF test day 16:15:16 #topic Project Coconut (OpenQA) 16:15:44 adamw: I would think it would be pretty fine to make one once we get the goahead. I dont know of any stoppers. 16:15:46 so i hope the folks working on this don't mind me blowing their cover :) but it seemed like some smart cookies had it figured already 16:15:51 nirik: rgr, thanks 16:16:10 nirik: guess i'm just a bit worried about the product-y netinst / generic netinst stuff but we'll burn that bridge when we come to it 16:16:55 yeah, not sure where that is 16:17:28 adamw, netinstall stuff ? 16:18:06 ok, switch topics back again... 16:18:06 quit 16:18:13 #topic Fedora 22 status part 2 16:18:18 just to keep things organized 16:18:31 Corey84: there's a planned change to the media for f22 16:18:52 server and workstation (at least, not sure about cloud) will have dedicated network install images that are actually different 16:19:10 ah will have to relook as was not terribly happy with the netinst script fails on 21 tbh 16:19:37 cloud will only have raw/qcow2 and AMIs - no need for a netinst, doesn't make any sense aiui 16:19:38 and there will also be a 'generic' network install iamge 16:20:02 this is the plan, anyhow, and releng apparently has an idea how to do it, but as with the f21 plan, we'll see what happens when we actually have to buiild it :) 16:20:53 Corey84: script fails? 16:20:57 so 2 purpose builts then one that is akin to the current one that is no usb/cd user friendly? 16:21:17 yeah, one sec 16:22:10 Corey84: basically. i suspect the differences between the three will end up just being the default package set and logos and maybe one or two other things, but that's the design. 16:23:16 allrightly, moving back along... 16:23:21 * nirik still thinks it's overkill, but hey, whatever folks want to do 16:23:46 nirik - giving the people what they want 16:23:59 One ISO to install them all is not better? 16:24:23 in case of netinstall, of course 16:24:27 its in the pyanaconda scripts that fail adamw 16:24:29 depends on what angle you're viewing the term "better" from 16:24:29 Kinokoio: well, if the products want to be significantly different from each other, it becomes difficult to do that with the way the installer's built, aiui 16:24:48 there's kind of a whole nexus there which is being worked out sort of...fuzzily... 16:25:30 * satellit_e DVD with generic/workstation/server first screen like multi DVD we used to do? 16:25:53 heh, rebirth of multi-image media 16:25:59 that's actually not such a bad idea 16:26:11 wonder if anyone had considered it yet 16:26:19 : ) 16:27:07 that would be a huge dvd tho would it not 16:27:25 like cOS7 large 16:27:35 not just for netinsts, no 16:28:42 ok, moving back along... 16:28:50 #topic Project Coconut 16:29:31 so folks had been noticing big piles of validation results from 'coconut' in the matrices 16:30:33 coconut is a sort of quick hack automated QA system a few of the rh folks have been working on, using openQA 16:30:48 https://os-autoinst.github.io/openQA/ 16:31:39 the openQA folks have been bugging us about it for years and got a demo running against fedora at christmas, so a couple of folks finally decided why not try and get something rough running until we have install testing in taskotron or whatever, and it came up rather quickly 16:32:10 big kudos to jskladan and garretraziel 16:32:31 because it's a quick hack project the instances that are up and running atm are behind the rh firewall, but all the code and scripts and tests and stuff is open and you could set up an instance elsewhere pretty easily i believe 16:32:36 has it found any breakage yet? 16:32:59 though we haven't bothered making openqa itself run on fedora yet, the coconut boxes run opensuse :) 16:33:00 I think lbrabec reported one bug today 16:33:11 nirik: it caught the same showstopper everyone else did 16:33:27 cool. 16:33:29 for the moment, openqa reports just passes into the wiki, and we manually inspect the failures 16:33:31 but so far it's actually been designed mostly to do box-ticking for passes 16:33:38 excellent work 16:33:39 but jskladan can probably tell you more 16:33:41 it's configured to only report passes to the public wiki 16:33:54 i'd quite like to get it to run through the bug submission process when it hits a crash and report a failure, but hey 16:34:16 nirik: in practice, the Coconut will not report failures/bugs on itself, though. We only want to have it reporting passes, so the ever-repeating work on testing all the "simple" testcases is a bit less boring... 16:34:27 so, i thought it'd be good to explain what's going on there and maybe see if folks had any questions or were interested in helping (if we can work that out) and maybe we could talk a bit about how much more we want to do with it in f22 timeframe 16:35:16 * satellit_e I assume sugar (soas) testing will remain manual? 16:35:28 satellit: coconut doesn't take anything over 16:35:37 all it ultimately 'does' is file wiki reports, like any other tester 16:35:57 everyone's free to run the same tests manually, of course, as well 16:36:03 k 16:36:05 right now it does 7 or 8 of the core Installation tests 16:36:51 another test we do not do is installation of 2nd DE and compatability 16:37:11 adamw: https://bitbucket.org/rajcze/openqa_fedora_tools/src/ee062fac7b729c1600af1664010241f62e76988d/PhaseSeparation.md?at=master these are the covered testcases ATM 16:38:08 jskladan: thanks 16:38:09 slick :) 16:38:14 nice 16:38:19 so yeah, if anyone wants to play with the bits, they're all there: 16:38:24 https://bitbucket.org/rajcze/openqa_fedora_tools/ 16:38:25 adamw: ad plans - the consensus (at the moment) is that garretraziel will provide tests/checks for the rest of the un-covered Beta testcases, and then we'll "let it live by itself", checking up on the failures, and (if caused by a bug outside of the openqa) reproducing the errors manually. This is something at least pschindl, garretraziel and lbrabec will be participating on (IIRC) 16:38:57 so eventually this might live in taskotron or beaker? (once there are disposable clients?) 16:39:00 there's even an 'installation guide' tim wrote when he set up a new box, which should more or less cover what you'd need to do to set up your own coconut - but obviously we don't want too many automation bots running around reporting results wildly, so use with care :) 16:39:44 oh, btw, any reports from 'colada' are from the instance of coconut i have to play around with. 16:40:32 nirik: there are some big deployment challenges atm. and it does not support distributed test clients 16:40:34 nirik: depends on what you mean by "this" 16:40:43 jskladan: cool. i had some thoughts/plans along the lines of integration with compose releases, but we're just coming to the end of nominated nightlies for f22 so it might be best to wait for f23 there 16:41:10 nirik: openqa is a pretty 'vertically integrated' system with its own unique test format and stuff 16:41:11 sure, lots to be figured. ;) 16:41:25 i suppose you could have mini-coconuts *within* taskotron, that'd be nice and insane! 16:41:30 can/will coconut go on with rawhide? 16:41:35 nirik: it can for sure. 16:41:36 everything runs on a single box, that's the only supported approach by openqa 16:41:43 nirik: this (as in OpenQA) is not really in any way tied to disposable clients, and we do not plan on running it in beaker/taskotron ATM. The thing is, that you need to have an OpenSuSe box to run it, and we are not really going to put the time into porting it all on Fedora, as it would immediately loose the bang-for-the-buck advantage it has now 16:41:58 ah, I see. ok 16:42:25 nirik: it can run on just about anything, at least with my patches to use fedfind 16:42:32 which jskladan is going to review real soon now right? :P 16:42:55 i'd like to hack it up a magic mode where it runs on everything it can find every day - rawhide, branched, tc/rc. 16:42:59 * jskladan just applied the patches in his git ;) 16:43:35 poor jskladan, it's really hard to keep up with adamw's rate of development :P 16:45:20 so my brief sketch plan of an interesting thing to do with it was invent a new type of result page that was for coconut's results only 16:45:42 Test Results:Automatic (compose version) or something 16:46:09 coconut could report results for all builds it could find every day, and then we could do stuff with that data 16:46:16 like not nominate nightlies that coconut found were DOA 16:46:38 oh, the other thing i'd like to play with is getting it to emit results via fedmsg 16:47:04 that's going to be tricky, because fedmsg is not packaged for opensuse, I believe 16:47:08 kparal: it's in pypi 16:47:22 i have it on the qa-01 box atm 16:47:29 ok 16:47:32 looks like it needs some config file or something but shouldn't be too hard to fix 16:48:53 those will probably be fun time projects though now we're into alpha/beta phase, as they don't achieve anything we can't do relatively easily semi-manually (it's easy enough just to fire off a coconut run manually on a tc/rc before announcing it) 16:49:19 soo, yeah, thanks a lot to the folks who got that up and running! hopefully it'll save us monkey work on some of the boring test cases for the 22 cycle 16:49:35 if anyone has q's outside the meeting, qa-devel@ list is probably the place 16:51:38 any other thoughts/notes on it before we move on? 16:51:53 * roshi is going to have to play with it :) 16:52:20 * satellit_e afk 16:52:24 roshi: it's pretty neat to be honest, i'm having fun 16:52:49 as long as you don't need to get your hands dirty with perl :) 16:52:58 it looked like it when I glanced at it before 16:53:01 lol 16:53:13 roshi: openqa_fedora_tools just wraps the whole thing in a bunch of python :? 16:53:45 kparal: jskladan: oh, i talked to richard brown about coconut a bit and he said some of the stuff you're doing in fedora_tools looks like stuff openqa can do in core, i'll pass his notes along in a bit... 16:54:04 adamw: great 16:54:22 alrighty, just quickly: 16:54:25 #topic Test Days 16:54:29 oh damn 16:54:33 #topic Project Coconut 16:54:51 adamw: real quick on coconut. for the results page, you could just copy the source for the releng-dash 16:54:58 and modify it to query datagrepper for coconut messages. 16:54:59 #info Coconut is a quick-and-dirty deployment of openSUSE's OpenQA running a limited set of tests on Fedora composes and reporting results to the wiki. 16:55:13 #info see: https://bitbucket.org/rajcze/openqa_fedora_tools and https://os-autoinst.github.io/openQA/ 16:55:56 threebean: oh, i have a whole thing that does results pages, i'm the crazy-haired mad genius there. ;) 16:56:31 #topic Test Days 16:56:45 so the Anaconda/DNF Test Day went off pretty well i thought? did everyone have a good time? 16:57:06 yeah, I thought it went well 16:57:16 and we have a few more proposed/scheduled as well 16:58:17 I'm late 16:58:24 i'll try and get results from the anaconda/DNF day transferred into the wiki and a follow-up email sent out soon 16:58:36 did anyone have any issues with the event or thoughts on how we could do it better or anything? 16:58:55 the only thing I thought would have been nice 16:59:16 since kickstarts were a focus, is a good noob guide to writing kickstarts 16:59:25 * roshi can write it 17:00:28 or there is a tool - system-config-kickstart 17:00:45 for making simple kickstarts 17:00:54 ah, didn't know about that 17:00:59 it is gui-tool btw 17:01:17 Me neither until I saw him one hour ago :) 17:01:34 it's pretty old, i'm never sure how up to date it is :/ 17:02:34 but yeah, that would've been a good idea 17:02:45 i wasn't quite sure where to pitch the whole thing between beginners and experienced testers 17:02:57 When will be scheluded the next tests for anaconda? 17:02:58 #info Anaconda/DNF Test Day went off well with good attendance and multiple bugs found 17:03:10 Kinokoio: we'll just have the regular validation testing from this point on 17:03:15 we'll have lots of other test days, though 17:03:29 roshi: what's coming up next? 17:04:08 * roshi looks 17:04:36 Cockpit Test day in March 17:04:46 2014-03-24 17:04:46 okey dokey 17:04:51 mark your calendars, folks 17:04:59 #info currently next planned Test Day is Cockpit Test Day on 2015-03-24 17:05:01 (FTFY) 17:05:30 really quickly: 17:05:33 #topic Open floor 17:05:41 any other thoughts / notes / philosophical ramblings? 17:06:56 Blocker meeting to commence after this meeting and a 5 minute coffee break sound good? 17:08:28 sure 17:08:58 ok 17:09:24 alrighty, setting the TinyFuse 17:11:18 thanks folks! 17:11:20 3.2.1 17:11:22 #endmeeting