15:04:26 <adamw> #startmeeting Fedora QA meeting
15:04:26 <zodbot> Meeting started Mon Jul  8 15:04:26 2013 UTC.  The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:04:26 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
15:04:30 <adamw> #meetingname fedora-qa
15:04:30 <zodbot> The meeting name has been set to 'fedora-qa'
15:04:32 <adamw> #topic Roll call
15:04:40 * kparal here
15:04:41 * mkrizek is here
15:04:44 * Cerlyn is here
15:04:56 <adamw> sorry for the late start, folks - who's around for funtimes?
15:05:11 * tflink is here
15:05:28 * nirik is lurking.
15:05:32 * brunowolff is here
15:06:48 <adamw> yay funtimes
15:07:02 * jsmith is lurking
15:07:17 <adamw> like a ninja
15:07:44 <jsmith> I'm sneaky like that :-)
15:07:54 <adamw> alrighty!
15:08:06 <adamw> well, all i had for the agenda was a couple of things left from last time
15:08:11 <adamw> #topic Fedora 20 Change validation
15:08:42 <adamw> #info last week, jreznik asked if it was possible for us to do some form of 'validation testing' for the systemwide Changes for Fedora 20
15:09:08 <kparal> which changes?
15:09:12 <adamw> i figured we'd better discuss it
15:09:23 <adamw> kparal: Changes as in 'what used to be called Features'
15:09:34 <kparal> ah
15:09:38 <adamw> one of the problems is that AFAIK we don't have a list of them all yet
15:09:59 <kparal> validation after they are deemed complete, or before?
15:10:02 <nirik> the deadline is the 16th.
15:10:04 <tflink> and "some form of validation testing" is very vague
15:10:57 <adamw> well, that's one of the things to discuss, i guess.
15:11:05 <tflink> nirik: the 16th of july?
15:11:07 * jreznik is here
15:11:12 <adamw> jreznik: is https://fedoraproject.org/wiki/Category:ChangeReadyForWrangler really all the Changes that are proposed right now?
15:11:21 <nirik> yes.
15:11:23 <nirik> https://fedoraproject.org/wiki/Releases/20/Schedule
15:11:28 * pschindl is here.
15:11:32 * adamw notes https://fedoraproject.org/wiki/Category:SystemWideChanges is empty
15:11:33 <nirik> so, they arent even all in yet.
15:11:45 <adamw> right, which makes it hard to eyeball a list and come up with an answer
15:11:55 <jreznik> https://fedoraproject.org/wiki/Category:ChangeAcceptedF20 for accepted
15:12:12 <adamw> jreznik: did you have a more specific idea in mind for exactly what you'd like us to be doing here?
15:12:19 * adamw brb, call of nature
15:12:26 <tflink> wow, I didn't realize that the downtime between f19 and f20 was so short
15:12:43 <adamw> #chair kparal brunowolff
15:12:43 <zodbot> Current chairs: adamw brunowolff kparal
15:12:44 <jreznik> https://fedoraproject.org/wiki/Category:SystemWideChange should not be empty (it's not...)
15:13:02 <adamw> oh, I missed the 's'
15:13:04 <adamw> er, added an s.
15:13:08 <jreznik> tflink: it's really short, that's probably the reason why not so many proposals were submitted so far
15:13:21 <jreznik> and also f18, f19 were big, f20 seems to be smaller
15:13:23 <kparal> I assumed we're branching in September?
15:13:42 <brunowolff> August 8th I think.
15:13:42 <tflink> kparal: nope, august 06
15:13:42 * adamw notes that https://fedoraproject.org/wiki/Releases/20/ChangeSet has the incorrect category names
15:13:48 <brunowolff> Alpha freeze is Augist 20th I think.
15:13:53 <kparal> wow, the world is speeding up
15:14:09 * tflink should realy start paying more attention to fesco, this is going to impact automation plans
15:14:14 <jreznik> adamw: for ChangeSet - should be ready this week, I need to tweak my old scripts :(
15:15:05 <jreznik> tflink: the main reason why fesco asked me for this schedule was to avoid christmas holidays (there's approx 1 month buffer now)
15:15:39 <jreznik> and it's not final schedule - I think FESCo would listen to QA if you would need more time (and I'd say - propose QA automation as Change...)
15:15:44 <tflink> jreznik: yeah, I understand that but it still feels a bit like QA is being punished for the scheduling issues stemming from F18
15:16:15 <jreznik> tflink: this time, not only QA - you can see it on proposed changes
15:16:24 <jreznik> but back to the question
15:16:41 <tflink> in this timeframe, I'm skeptical that any formal system for testing will happen
15:16:43 <tflink> will/could
15:18:06 <jreznik> we usually don't have any feedback/validation of features/changes that are marked as done, so sometimes it just does not work, fo self contained changes, not a big deal... but once it's in release notes, it should work... if it's in announcement and picked up by medias, we should be sure it works as mentined in proposal etc.
15:18:52 <tflink> jreznik: sure, I don't think that anyone's arguing with you on that part
15:18:54 <jreznik> currently, there's no way, once owner says done, we trust them... so at least for system wide, visible changes, we should have at least some smoke testing "it's there and it works"
15:19:08 <tflink> jreznik: how often and starting when?
15:19:42 <jreznik> tflink: now the question is, if we have people to do it, if anyone is even willing to do so - as it is in the worst time for qa - during release
15:20:20 <adamw> what we used to do for features, in a kind of semi-organized way, was look through the list and pick out ones that looked important and ask the feature owners if they wanted to run test days on them
15:20:29 <adamw> you can kinda see that in the test day list for each release if you look
15:20:37 <adamw> it's quite a long way from there to some kind of 'validation process'
15:21:01 <kparal> some changes might be hard to verify, like some SSSD and similar changes -  I wonder whether anyone really understands it
15:21:42 <jreznik> adamw: yeah, and it's great it's covered at least by test days
15:21:56 <adamw> yeah, that's a good point from kparal
15:22:05 <adamw> with the best will in the world we don't have expertise on every tech in fedora
15:22:38 <adamw> i think at least my inclination would be that we're unlikely to be able to commit to some kind of formal testing of all systemwide Changes for f20 whatsoever they may be
15:22:48 <jreznik> adamw: then maybe test days have even more sense to do it this way and based on results we can say - it's ok/it's not ok
15:22:50 <adamw> we might be able to try and come up with something more than a test day to check on any really important ones
15:23:14 <adamw> jreznik: that would be an obvious improvement on the old way, yeah - try and hook the Change process into the results of any test days we run
15:23:46 * jsmith likes that idea
15:24:32 <jreznik> and I'd be really ok with coverage of the really important ones (big system wide change, change that's going to be featured in the announcement) + for some other test days coverage
15:24:43 <adamw> OK
15:25:19 <adamw> so, how about this: once the list of Changes is approved by FESCo we can revisit this topic at a future meeting, with jreznik/fesco present, and come up with a specific plan for what we can try and do for f20 'change validation
15:25:25 <adamw> we can review how that goes post-f20
15:26:17 <tflink> with an emphasis on minimal coverage, I'd say - making it clear that we aren't planning to cover everything
15:26:19 <jreznik> adamw: I'd be more than happy with it this way for F20
15:27:18 <adamw> tflink: agreed
15:27:27 <adamw> everyone else on board with that?
15:27:51 <brunowolff> Some changes are more coordination than use observable changes, like the boost change.
15:28:07 <brunowolff> s/use/user/
15:28:34 <kparal> adamw: yes
15:29:07 <brunowolff> Yeah, start small and see how things go, seems like a good approach.
15:29:12 <adamw> OK
15:30:16 <adamw> #agreed we cannot commit to do any form of 'validation testing' for any and all systemwide Changes sight unseen: when FESCo approves the F20 ChangeSet we will re-consider this issue and come up with a realistic plan for testing what we can of the approved Changes
15:30:38 <adamw> #topic Test Days
15:30:41 <jreznik> thanks!
15:30:47 <adamw> mkrizek: ahoy
15:31:05 <adamw> so now it's been pointed out that the f20 schedule is very tight we probably want to send out a call for test days pretty soon
15:31:40 * nirik notes most of the f20 sechedule is still draft, the proposed stuff there is 'no earlier than'
15:31:47 <adamw> ah, OK
15:31:53 * nirik needs more coffee after that typo.
15:32:17 <tflink> but what are the odds that it'll get figured out on wednesday?
15:32:30 <adamw> still, that was just a side note
15:32:38 <tflink> assuming that's still when the fesco meetings are
15:33:01 <adamw> what this topic is really for is https://fedorahosted.org/fedora-qa/ticket/389
15:33:01 <brunowolff> I suspect we want to get back to the previous schedule (October and May), so that trying to get this one done in a short time is desireable.
15:33:07 <tflink> yeah, it seems prudent to start organizing test days sooner than later
15:33:47 <tflink> adamw: ok, we're getting off topic then :)
15:33:51 <adamw> we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates
15:33:59 <adamw> #info we were asked by dpal if we can have an 'out of cycle' test day for some planned FreeIPA enhancements to be added to F19 as updates
15:34:22 <tflink> freeipa strikes me as something which would be difficult to find testers for
15:34:26 <adamw> i don't see a big problem with that, we can run it just as usual with announcements etc and add it on to the F19 schedule page for future tracking
15:34:31 <adamw> we've done freeipa test days before
15:34:34 <tflink> especially AD integration
15:34:44 <adamw> they seem to like the setup or they wouldn't bother asking for another
15:34:57 <adamw> i think even if no 'external' testers show up they like to have an organized event for 'internal' testing
15:34:58 * tflink doesn't see any problems with the idea of an out-of-cycle test day
15:35:20 <adamw> anyone else see any issues with an out-of-cycle test day or should we go ahead and say 'sure'?
15:36:20 <mkrizek> Martix: test days ^^^
15:37:47 <kparal> I don't see a problem
15:38:00 <tflink> +1 to 'sure'
15:38:25 <adamw> goddamnit, too many people starting with M.
15:38:29 <adamw> meetings too early.
15:38:32 <adamw> adam too idiotic.
15:39:56 <adamw> well, let's call that good.
15:40:15 <adamw> #agreed no problems with an out of cycle test day, we'll run it as always and append it to the F19 test day schedule
15:40:29 <adamw> #info Fedora 20 schedule may be quite tight so we should send out call for F20 test days soon
15:40:36 <adamw> #topic open floor
15:40:39 <adamw> so, any other business?
15:41:46 <brunowolff> This is more for nirik, when are we going to switch over the nightly composes to f20?
15:41:52 <nirik> already done. ;)
15:41:58 <nirik> yesterday.
15:42:05 <nirik> some of them failed compose.
15:42:19 <brunowolff> Thanks, I'll see if my changes broke anything.
15:42:46 <nirik> I have one other thing...
15:42:47 <brunowolff> I also saw that we might be doing weekly install composes. Is that going to start soon?
15:43:23 <nirik> There's a bunch of fedup bugs coming in. wwoods is not opposed to the idea of people helping triage. If you would like to do so, ping me over on qa or devel and we can try and go thru them.
15:46:01 <adamw> #info wwoods looking for help triaging fedup bugs, contact nirik to sign up
15:48:45 <adamw> nirik: any info on the weekly install composes?
15:48:57 <nirik> dunno.
15:49:07 <nirik> I think we should really ask anaconda folks what they would like.
15:49:13 <Martix> mkrizek: hi
15:49:14 <nirik> daily would actually be easier.
15:49:25 <nirik> but if they want weekly we could do that I guess.
15:49:53 <Martix> adamw: out of cycle test day is ok
15:50:22 * nirik doesn't know specifically who to ask over there, but someone should. ;)
15:50:32 <adamw> Martix: thanks
15:50:43 <adamw> nirik: bcl is my goto anaconda folk
15:50:48 <adamw> i'm sure he hates me for that
15:51:01 <nirik> cool. You want to ask him? or have him drop by fedora-releng and we can hash it out?
15:52:21 <adamw> well i'd feel odd asking him since i'm not in any way involved in or capable of doing weekly builds...
15:52:28 <nirik> sure, or I can.
15:52:28 <adamw> seems a bit unnecessarily go-between-ish
15:52:32 <nirik> no worries.
15:52:41 <brunowolff> There is a trac ticket for the install compose stuff. I'll see if I can find it.
15:52:43 <adamw> right, that'd seem to make more sense. he only bites as much as your AVERAGE anaconda dev.
15:52:54 <adamw> (wear gloves.)
15:53:17 * nirik is happy to talk to most anyone, just thought you might already be talking for some other reason or something... no worries.
15:53:21 <brunowolff> https://fedorahosted.org/rel-eng/ticket/5653
15:53:57 <adamw> #info nirik will talk to anaconda team about doing regular installer composes (again?)
15:55:41 <brunowolff> I was insterested in installer composes because I made some changes to the installer ks file to make things better. In particular, you shouldn't have to add a bunch of excludes when overriding repos.
15:56:17 <brunowolff> I haven't tested that on RHEL 6 though, and this needs a not too old version of pykickstart to work.
15:57:38 <brunowolff> I was also figuring that this time around, anaconda will probably be a lot more testable throughout the development period.
15:57:39 <adamw> brunowolff: that'd need a DVD compose, wouldn't it?
15:57:49 <adamw> the kickstart isn't relevant to netinst?
15:57:59 <brunowolff> That is correct.
15:58:35 <adamw> okay.
15:58:43 <adamw> well, we can try and figure that out.
15:58:48 * adamw sets the Quantum Fuse
15:59:15 <adamw> any other extremely important business before we all go drink?
16:00:42 <adamw> alrighty, thanks for coming folks
16:00:42 <adamw> #endmeeting