15:00:23 #startmeeting Fedora QA meeting 15:00:23 Meeting started Mon Jul 15 15:00:23 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:23 Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:00:38 #meetingname fedora-qa 15:00:38 The meeting name has been set to 'fedora-qa' 15:00:40 #topic Roll call 15:00:49 any rolls? calling all rolls 15:00:59 * Martix is lurking here 15:01:38 ahoyhoy 15:01:42 any other rolls? 15:01:52 * kparal also rolling 15:01:58 * mkrizek is here 15:02:40 adamw: ahoj 15:02:51 * adamw wonders what happened to america 15:02:59 * brunowolff is here 15:03:07 oh good, it didn't sink 15:04:24 do we have tflink? 15:04:57 doesn't look like it 15:04:59 or viking-ice 15:05:02 * jsmith lurks 15:05:11 * adamw checks to see if anyone sent apologies 15:05:33 adamw: Does it count if I apologize for being here? 15:05:35 * handsome_pirate waves 15:05:43 doesn't seem like it 15:05:48 man the cannons! 15:05:51 lol 15:06:01 adamw: have you read my msgs to you about ARM desktop testing? 15:06:05 handsome_pirate: have jsmith walk the plank 15:06:12 Martix: no, have I been missing them? 15:06:23 adamw: Arrr 15:06:34 adamw: on Thursday 15:06:37 adamw: on fedora-qa 15:07:07 whoops, not paying attention :-/ 15:07:08 Martix: oh, sorry 15:07:14 alrighty, we'd better get going 15:07:20 Martix: can you send me an email or something? 15:07:26 ok 15:07:27 oh hi, tflink 15:07:39 #topic ARM as a primary arch 15:07:44 so this has been the big bikeshed of the week 15:08:08 Heh 15:08:14 for anyone who hasn't read it, there's a megathread on devel@ at https://lists.fedoraproject.org/pipermail/devel/2013-July/184962.html 15:08:25 just wanted to throw a topic on the agenda so we can air out any concerns that aren't already covered 15:08:26 I saw the thread and that's why I didn't read it 15:08:36 just skimmed it a bit 15:08:40 adamw: check you INBOX 15:08:58 *your 15:08:59 * tflink also has been skimming more than anything 15:09:56 adamw: Mind if I go with this just a wee bit, considering my involvement in ARM? 15:09:57 There seems to be a lot of arguing over whether being a primary arch means you need to have comparable deliverables to i686/x86_64. 15:10:01 i think the biggest issue raised so far is the question of what exactly qa takes responsibility for 15:10:06 handsome_pirate: shoot! 15:10:18 Okay 15:10:19 brunowolff: right, though i'm not sure that concerns us a lot 15:10:38 I've been trying to get ARM to mirror the QA process 15:10:38 #chair handsome_pirate kparal tflink 15:10:38 Current chairs: adamw handsome_pirate kparal tflink 15:10:58 #info handsome_pirate has been trying to get ARM to mirror the QA process 15:11:11 is there really enough time before branch to get everything coordinated? 15:11:14 They're getting better, but not quite there 15:11:37 handsome_pirate: oh, you're jdulaney. how sneaky :) 15:11:46 FOr instance, they don't seem to understand how blocker bugs are supposed to work 15:11:49 kparal: Arrr 15:12:01 (or, the process, I should say) 15:12:16 well they seemed to stop proposing stuff as primary blockers after we talked to them about it during alpha 15:12:20 or did you mean something else? 15:12:30 At the end of the F19 cycle they finally started doing testing matrices 15:12:52 adamw: The intention was to get them to file against the arm blockers 15:13:42 Very few bugs that were blockers for them were actually filed against the trackers 15:13:43 right 15:13:46 ah, i see 15:14:27 My evaluation from a QA perspective is that they're close, but not quite ready for F20 15:14:50 so yeah, i do feel like we're going to need a lot of co-operation from 'arm people', however we describe it exactly (the stuff in the thread about what is technically the responsibility of which group feels a bit trivial) 15:15:01 another question I have is whether we consider automated tests a requirement for primary arch 15:15:14 practically speaking, if we're going to sign off on a bunch of ARM images as 'primary deliverables', we at least need to test them all 15:15:24 * tflink doesn't remember if pwhalen was able to get autoqa-on-arm working 15:15:24 and 'QA' as currently stands doesn't have all that hardware 15:15:25 tflink: The two tests we have now will work on ARM 15:15:43 handsome_pirate: I thought it was more of an autotest/autoqa issue than the individual tests 15:15:47 tflink: that's an interesting one that hasn't been raised yet 15:15:47 adamw: I can test Gnome/KDE on ARM, see my mail 15:16:07 adamw: I'll get some h/w for QA 15:16:11 Martix: sure, so can I 15:16:18 adamw: I assume you and tflink would need some 15:16:30 but I still don't think we have *all* the hardware that ARM images are built for, and even if we do, there's the time question of testing them all 15:16:38 no, not really. i have more than I can practically use, 15:16:42 tflink: The last I tried, the autotest issue was the autotest on f18 issue 15:17:04 handsome_pirate: which I may run into soon, I should have f18 and f19 clients by the end of the week 15:17:05 * nirik notes we have some SOC's setup for qa in phx2 already. 15:17:19 https://fedoraproject.org/wiki/Architectures/ARM/qa-machines 15:17:23 handsome_pirate: do you have something with currently supported GPU in F19? 15:17:32 handsome_pirate: I mean 3D accel 15:17:57 nirik: Aye 15:18:17 * tflink has some arm hardware but doesn't use it much 15:18:22 Martix: Everything I have is headless save for my chromebook 15:18:25 tflink: so the detail here is that autotest does not run on ARM on f18+? 15:18:28 tflink, autoqa worked well when using earlier versions 15:18:40 adamw: yeah, need to dig into it 15:18:45 I have access to all supported hardware for testing 15:18:53 I have an idea of what the problem is, need to test it 15:19:05 adamw: autoqa needs to be updated to worked with newer autotest versions. but we didn't bother, since we use RHEL6 and old autotest version 15:19:07 * tflink has been rebuilding and scripting virthosts 15:19:34 I've tried to follow the release validation testing as best as possible for f19 15:19:38 #info there are issues with running autotest on ARM hosts on Fedora 18 15:19:43 pwhalen: do you still have stuff running against f17 on arm? 15:19:56 pwhalen: sure, that's what handsome_pirate was talking about 15:19:57 handsome_pirate: well, I have some hw with Mali (supported in F19) and Adreno (userspace support will be in F20) 15:19:58 tflink, no 15:20:15 Martix: Chromebook is Mali 15:20:54 I don't understand one thing - I don't see any ARM devices around me, anywhere. except for RaspPi, which is not going to be supported anyway. so I might not understand what primary architecture means, but why would we have PA for something that is so thinly available? 15:21:44 kparal: Only a few specific devices are supported, but they're fairly cheap and easily had 15:21:56 kparal: it's kind of a split issue: the vague idea we might run on zillions of ARM tablets or something, and a very specific market for ARM servers. 15:21:58 kparal: because it is expected to be avaliable later, i think 15:22:03 kparal: I'll bring up about getting you some h/w if you want 15:22:12 plus a hedge against ARM Taking Over The World. 15:22:33 easily had, but I don't see them around me or my colleagues, or my friends, or my friends' friends, ... 15:22:41 anyway, probably a topic for the mailing list 15:22:43 * handsome_pirate notes that the biggest issue with arm is proprietary gfx 15:23:17 kparal: HP Touchpad, Cubieboard... 15:23:32 kparal: still - yeah, that's not a QA topic in particular 15:23:45 OLPC machines also use arm, though they need a special kernel now. But I have done updates and installs of non-kernel arm packages from Fedora. 15:24:02 A number of Fedora people got OLPC machines last summer. 15:24:09 so in terms of specific concerns: handsome_pirate is worried about following procedure in terms of using blocker bugs, tflink is worried about autoqa 15:24:12 do we have anything else? 15:24:31 Documentation of the deliverables that we need to test. 15:24:34 * Viking-Ice joins in 15:24:54 time for adjustment 15:25:04 we've been using blocker bugs, though work could likely be done, absolutely open to all suggestions 15:25:08 adamw: the biggest concern is whether we need to do the release validation for every single subarch (or how it is called) of ARM 15:25:12 we have what, 3-4 weeks before F20 branch 15:25:12 as you already stated 15:25:21 tflink: Mayve a release cycle where QA treats arm as primary but it doesn't actually block? 15:25:39 * Viking-Ice playing arm catchup 15:26:07 You could have us only block on some of the stuff that we would block on for i686/x86_64. 15:26:17 kparal: that's my concern, yeah, which ties in with the 'following procedure' thing, as in practice we will need pwhalen and other ARMers to ensure we can do that 15:26:22 kparal: Actually, the arm team is trying to make it so that there are only a limited number of images to test 15:26:36 ie, make them work on as wide a variety of h/w as possible 15:26:44 handsome_pirate: that didn't look to be in the works for f20, though 15:26:51 handsome_pirate: how many? 15:27:11 kparal: right now, there are three images 15:27:42 kparal: One supported piece of h/w requires a vfat boot partition 15:27:43 because it's not 'ARM as a primary arch', but 'ARM as an additional three primary archs', which sounds much scarier :) 15:27:49 Is the arm team going to expect the Gnome 3 desktop to be a deliverable? 15:28:03 kparal: Otherwise, lpae and a standard image 15:28:10 ie, lpae kernel 15:28:11 with a late chime in as I see it basically the main problem with us regarding arm is for someone to actually own arm devices and being able to test on them and apply relevant critera where applicable 15:28:55 Viking-Ice: we do have several people with some ARM devices, but we're still trying to determine exactly how much coverage is expected 15:29:27 adamw: Most of the testing is the same 15:29:32 adamw, simple really if the plan is to release an Gnome arm image the relevant Gnome release criteria applies to that image, if the plan is to release only KDE image only the KDE criteria applies etc 15:29:35 well, a lot of it is 15:29:40 Viking-Ice, I do my best to do that now and will going forward. I tested all images for f19 15:29:47 ANd, x86 tests will be sufficient 15:32:02 we're trying to reduce this to a single image at some point, currently we have two image types due to partitioning for omap requiring vfat. 15:32:54 pwhalen: so for f20, what exactly are the expected ARM deliverables assuming it's a primary arch? 15:33:02 pwhalen: We have an lpae image as well, yes? 15:33:03 yeah basically what we need to know ( QA ) is which "spins/de" arm architecture will be released on and we need to test that against the criteria and block based on matching criteria 15:33:51 adamw, that to my knowledge is not yet decided. dgilmore? 15:33:53 maybe some arm images might not be considered release blocking, if they are served to just a small audience? 15:34:32 handsome_pirate, there's our official offerings https://fedoraproject.org/en/get-fedora-options#2nd_arches 15:34:45 kparal, define small audience ;) 15:35:02 Viking-Ice: well, an example is GNOME vs XFCE 15:35:20 I don't know how many arm devices are covered by that or that image 15:35:26 okay, so there's basically two 'arches' for ARM for f19 15:35:27 just throwing ideas 15:35:30 pwhalen: My mistake 15:35:32 handsome_pirate, lpae kernel is installed on the hfp image 15:35:52 pwhalen: Ah 15:36:10 kparal, there are two "release" blocking images there on the link that pwhalen posted KDE and Minimal 15:36:16 Gnome is not listed there so 15:36:16 kparal: Most Fedora Arm folks I know of tend to run xfce, or at least that used to be the case 15:36:18 so if things were the same for f20 i'd expect the kde and 'minimal' images of both 'arches' to be release blocking: four images 15:36:20 so, it's a single arch, and two install images, right? but the exactly same packages? 15:36:35 Viking-Ice: they didn't do a GNOME image on the basis nothing would actually have hw accel support on it 15:36:35 kparal: AYe 15:36:42 handsome_pirate: ok, I get it finally 15:37:04 kparal, right 15:37:05 there's a single arch. 15:37:16 nirik: i was using the term incorrectly on purpose, hence the scare quotes 15:37:19 adamw: Not quite. No hw support and the sw fallback was broken. 15:37:20 one tree/collection of packages. ;) 15:37:22 adamw, we only match what the do against our criteria and based on that link only two ( in current release criteria setup ) are blocking the minimal and KDE 15:37:22 i was hoping everyone could follow that =) 15:37:29 s/the/they 15:37:30 pwhalen: any idea why Gnome is missing on the list? 15:37:31 another question, is anaconda expected to work for ARM in F20? 15:37:39 Martix: see above. 15:37:41 omap devices require vfat, wheras every other device works with ext 15:37:55 pwhalen: is there a chance of the vfat-vs-non-vfat thing being reconciled for f20? 15:37:58 adamw: I can run Gnome Shell on Adreno GPU 15:38:04 or are we stuck with that? 15:38:09 Martix: llvmpipe is borked because llvm is borked (/me is working on that) 15:38:29 in a perfect pony world all images would be run through the critera and blocked upon but hey we have an default... 15:39:21 adamw, I believe so, I think it was very close, dgilmore was working on it 15:39:23 handsome_pirate: work on llvm-pipe is great, but there is some HW loke Chromebook which has open source 3D GPU driver 15:40:14 Martix: Chromebook sort of has one 15:40:28 okay. so given that picture of the deliverables, the work doesn't seem crazily difficult 15:40:34 is anaconda expected to work for ARM in F20? that would affect the QA release validation time a lot 15:40:37 Martix: Chromebook is complicated, and not likely to be officially supported due to the bloody firmware 15:40:43 kparal: No 15:40:51 pwhalen: do you confirm re anaconda? 15:40:52 good :) 15:41:03 No Anaconda 15:41:19 adamw: I talke about that during job interview with clumens 15:41:29 no anaconda means we need to adjust the release criteria to match whatever installation method they use 15:41:47 without anaconda support, it might be actually so much work 15:41:55 Viking-Ice: it doesn't, really 15:41:58 (as I expected) 15:42:06 Installation method: dd to sd card 15:42:12 Viking-Ice: the criteria are couched in terms of 'the installer must blah' and 'the installer must foo' 15:42:20 If device has internal storage, then dd image to that 15:42:24 adamw, ah excellent 15:42:25 they don't actually explicitly say 'all images must use the installer' or anything 15:42:31 Anaconda was a required part of the promotion criteria 15:42:35 i've covered that on the devel@ list thread; i'm not too worried about that angle 15:42:40 mjg59: OFF TOPIC FOR QA MEETING 15:42:40 anaconda works as far as I can see. 15:42:46 god, this has already gone on for 40 minutes. 15:42:52 at least the ones I have installed have used anaconda text mode just fine. 15:42:52 we use anaconda for headless server installs, and preoduce images for dev boards 15:43:08 adamw: I'm aware. I'm just pointing out that if there's no plans for Anaconda to work in F20, worrying about primary arch QA for F20 seems premature 15:43:31 so it might be arm+anaconda or nothing, right? 15:43:50 well, what pwhalen just said doesn't sound anything like 'anaconda doesn't work'. 15:44:11 and nirik 15:44:13 mjg59: My talking with the Anaconda team led me to believe they have no plans for anaconda on arm 15:44:22 anaconda works. kickstart works. 15:44:25 is there something that says PA have to be installable by Anaconda 15:44:33 * nirik thinks this is in the weeds. 15:44:46 Anaconda does work, it's just not used very much 15:44:56 let's assume anaconda support will be added, because we have it confirmed that it works at least partially 15:45:02 * nirik uses it all the time. well, kickstarts usually with text mode to watch 15:45:06 which means more installation matrices! 15:45:20 ooh, that sounds like fun :-/ 15:45:26 limiting PA to anaconda dont make much sense since someday someone might want to write and use another installer for their "spin" 15:45:29 Viking-Ice: there's nothing QAish that says that, but mjg59 is saying someone else said that. i assume FESCo. 15:45:36 handsome_pirate: Oh, right. I don't know that anyone on the Anaconda team is working on it - it'd be the ARM SIG doing it 15:45:44 adamw: mjg59 said that 15:45:52 adamw: He wrote the criteria 15:46:03 https://fedoraproject.org/wiki/Secondary_Architecture_Promotion_Requirements 15:46:15 * Martix brought some snack to watch the flame :-) 15:46:17 Drafted rather than wrote, but they were signed off on by fesco 15:46:34 Anyway 15:47:06 well, the question of whether fesco wants to hold to that requirement is not ours to answer 15:47:09 I would see the reasoning for requiring the use of Anaconda 15:47:11 Anyway, sorry for diverting it. nirik says he has personal experience of it working, so it's not relevant 15:47:30 i don't think we can really block on ARM ever becoming a primary arch because it gives us another giant pile of matrix to work through, however much we want to :) 15:47:31 s/would see/would like to see 15:47:44 that's just going to wind up being one of those things we need extra people for 15:48:06 adamw: Hence my trying to get the arm folks to follow the qa processs 15:48:13 That way, they can do the work 15:48:17 Viking-Ice: can you take that to fesco or something? it's a fesco requirement 15:48:18 adamw, I will be on the hook for filling said matrices 15:48:20 handsome_pirate: right. 15:48:46 actually it might spark new willingness to test thus interest in participating in the QA community 15:48:54 ( arm that is ) 15:49:00 so let me try and boil this down 15:49:26 we didn't really get anywhere on the autoqa thing; tflink, do you have a path forward there? 15:49:55 adamw: a little bit, it'll depend on the results of my experiments 15:50:01 mkrizek had some patches, they might be obsolete already 15:50:07 okay 15:50:13 kparal: IIRC, they didn't work 15:50:22 are you actually *worried* about autoqa not working on arm, practically speaking? 15:50:31 or are you just concerned someone else might consider it a box that needs checking? 15:50:34 they did, but I dont think they do anymore 15:50:55 i'm not sure anything autoqa currently achieves is vital enough that we need to make it an obstacle to primary arch status 15:50:58 tflink, let me know if there is anything I can do, I got a fair amount of it working, however had some bugs with the latest version (would need to check which) 15:51:00 we can have upgradepath working for arm updates. depcheck might be a problem. since depcheck is very broken, I don't see it as a major issue 15:51:00 adamw: those are two different questions - I'm not sure it's going to work, no 15:51:01 though obviously it'd be nice 15:51:15 adamw: Autoqa (or its replacement) on arm is on my to do list 15:51:40 kparal: I thought depcheck was still mostly working 15:51:42 I dont consider autoqa not working for arm do be somekind of blocker for it from a QA perspective 15:51:56 s/do be/to be/ 15:52:09 tflink: yes, but it is arch-specific, so we might find some issues with it. upgradepath can be run on x86 box to check arm updates 15:52:30 kparal: but that's different from it being completely broken, no? 15:52:36 more of a not-quite-known? 15:52:44 tflink: ok, sometimes broken :) 15:52:59 bad wording from my side 15:53:00 so overall it seems like we don't feel like we need to make a big fuss about the autoqa angle 15:53:11 adamw: I don't think automation-on-arm is a blocker for F20 as we're not gating updates yet 15:53:16 but we should probably bake it into taskbot 15:53:31 if we start gating updates - even in rawhide, I think it'll be an issue 15:53:31 adamw: I wouldn't consider autoqa to be a large concern, no 15:53:42 * handsome_pirate can confirm that as of the last time he tried, taskbot worked on arm 15:53:45 #info we agreed that automated testing on ARM should not blocker primary arch elevation as we are not enforcing the tests yet 15:53:54 okay. 15:54:09 * tflink wonders if we should be adding arm clients to taskbot from the get-go, but that's off topic for here 15:54:20 Mind, that was about a month ago, I don't know how much tflink has worked on it since then 15:54:30 the other major angle we covered is the intersection of 'what deliverables will there be for ARM as a primary arch' and 'what resources do we have for testing' 15:54:34 handsome_pirate: not nearly as much as I would like :'( 15:55:06 it seems like the deliverables load is not too bad, but we feel like we'll definitely need co-operation from pwhalen and ARM-focused volunteers to ensure we can cover all the testing 15:55:10 tflink: If you get some work done, ping me and I'll fire it up on arm 15:55:12 is that a fair summary? 15:55:24 adamw: I'd say so 15:55:44 adamw: yeah, I think so. that and making the processes work together 15:55:47 adamw, I can look into our planned deliverables. and provide testing coverage 15:56:10 adamw: let's add that it is unclear how many different devices we need to test? 15:56:11 adamw: it's ok 15:56:21 kparal: just like x86 =) 15:57:12 speaking of upgrades I dont think the upgrade criteria takes effect until $NEXT_RELEASE for new PA as in we do not "support" upgrading from f18/19 on to f20 in the case of arm but only "support" upgrading from F20 to F21 and onwards ( the release architecture became PA and onwards ) 15:57:20 kparal: Actually, arm has fewer devices than x86 15:57:30 In that regaurd, arm is easier to test 15:57:31 #info as described by handsome_pirate and pwhalen, the ARM deliverables set does not look unmanageable, but we feel like we'll definitely need co-operation from pwhalen and ARM-focused volunteers to ensure we can cover all the testing 15:57:45 Viking-Ice: that seems reasonable 15:57:56 so overall, let me punt a #agreed 15:58:08 ack 15:58:19 ack 15:58:23 ack 15:58:24 ack 15:58:34 er, that wasn't it =) 15:58:53 what exactly are we ack'ing 15:58:55 ? 15:59:13 awesomeness and bacon 15:59:18 ack 15:59:34 you should have said it earlier 15:59:40 proposed #agreed QA does not see any definite roadblocks to ARM becoming a primary arch for F20, but expects that minor changes will be needed to the release criteria to define the ARM 'release blocking images' and assistance will be needed from ARM-focused testers to complete ARM validation testing 15:59:51 ack 15:59:55 ack 16:00:00 ack 16:00:01 * handsome_pirate assumed that was what he was acking 16:00:09 wow, you're clairvoyant too?! 16:00:15 arrr 16:00:34 ack 16:00:53 coolbeans 16:00:54 ack 16:00:58 #agreed QA does not see any definite roadblocks to ARM becoming a primary arch for F20, but expects that minor changes will be needed to the release criteria to define the ARM 'release blocking images' and assistance will be needed from ARM-focused testers to complete ARM validation testing 16:01:13 and we're right on time, with no time for the Change review topic :( 16:01:27 before i close out, does anyone have any really major concerns with any of the F20 Changes that can't wait till next week? 16:01:45 pwhalen: Wednesday, you want to take this? 16:02:02 adamw: I have something for the end 16:02:08 not really, I need to start reading those threads more closely 16:02:17 adamw: the AppInstaller might be worth discussion, but it can wait I think 16:02:32 * tflink wonders if the get-rid-of-syslog is going to turn interesting 16:02:43 okay, let's file those for next week 16:02:45 a very quick: 16:02:48 #topic open floor 16:02:51 handsome_pirate: what did you have? 16:02:57 * handsome_pirate needs a room mate for FLOCK 16:03:09 My room mate for the past several fudcons is not going 16:03:15 EOF 16:03:27 heh, okay 16:03:42 #info handsome_pirate looking for a roommate for FLOCK, applications on postcards 16:03:47 anything else? 16:04:08 yikes, if i don't clear up mine/tflink's room type problem you might wind up rooming with one of us... 16:04:25 adamw: I thought that was cleared up 16:04:33 tflink, thanks to FESCO/FPC yes it's going to be a more pain in the ass for us then it would be. I tried work in advance for the inevitable for us in QA with this https://fedoraproject.org/wiki/Features/systemd-journal but notting said no... 16:05:01 adamw: SInce I don't have a room booked, maybe you could not clear it up? :) 16:05:32 tflink: i'll ping you 16:06:11 alrighty, thanks for coming everyone! 16:06:16 Change review next week 16:06:20 #endmeeting