16:00:38 #startmeeting Fedora QA meeting 16:00:38 Meeting started Mon Dec 8 16:00:38 2014 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:38 Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00:46 #meetingname fedora-qa 16:00:46 The meeting name has been set to 'fedora-qa' 16:00:52 #topic Roll call 16:00:53 * kparal is here 16:00:57 ahoyhoy, folks, who's around for QA fun 16:01:00 * roshi is here 16:02:04 * satellit listening 16:02:33 * pschindl is here 16:02:37 looks like maybe we have brunowolff and amita too 16:03:16 * jreznik is lurking 16:03:23 * pwhalen is here 16:03:50 alrighty, thanks for coming out 16:03:55 so most important topic #1: 16:04:08 #topic Fedora 21 - thanks! 16:04:27 thanks to everyone for all your work on F21! it was a long effort, but everyone did a great job, thanks a lot 16:04:48 especial thanks to satellit and pwhalen for all the tests you guys ran 16:05:18 #info thanks to everyone for Fedora 21 testing work, great job 16:05:23 * kparal applauses 16:05:30 yeah - way to rock the matrices guys :) 16:05:52 #topic Fedora 21 final check-in 16:06:18 so I think we're looking pretty good for F21 release day, except for this fedup bug 16:06:32 which one? 16:06:32 it shouldn't be too much trouble for releng to fix that up, though. 16:06:42 * satellit testing it now 16:06:45 * roshi thought fedup was doing fine 16:06:51 the one satellit and I both ran into, https://bugzilla.redhat.com/show_bug.cgi?id=1171473 16:06:53 worked for me during final testing 16:07:02 I hit it over the weekend as well 16:07:11 fedup's fine, the .treeinfo.signed files on stage are completely wrong 16:07:13 ah, that's the one reported to the list 16:07:21 they are in fact the 32-bit live Workstation checksum files, for some reason. 16:07:24 yeah, all the same bug. 16:07:44 just a releng snafu in signing i'm guessing, i figure they should have it sorted pretty fast 16:07:44 * jreznik is reading 16:08:00 #info temporary bug in fedup #1171473 is a releng issue, should be sorted soon 16:08:16 https://bugzilla.redhat.com/show_bug.cgi?id=1171787 also dup? 16:08:30 yeah, I duped it. 16:09:21 I wrote up most of the bugs nominated for CommonBugs yesterday - if there are bugs we should have documented that aren't up there yet, please do add the CommonBugs keyword to them (and if you're feeling proactive, write them into the page yourself :>) 16:09:53 #info adamw has done the initial final release version of https://fedoraproject.org/wiki/Common_F21_bugs , please nominate any further bugs for the page by adding CommonBugs keyword 16:10:11 adamw: thanks a lot 16:10:27 I've also put up the F21 Retrospective page at https://fedoraproject.org/wiki/Fedora_21_QA_Retrospective 16:10:38 There's another fedup bug that I haven't had a chance to report yet 16:10:42 (Cosmetic, not functional) 16:10:45 that's for feedback on F21 from a QA perspective, please do add your thoughts there 16:10:47 sgallagh: what's that? 16:10:55 I provided some initial feedback for the retrospective page 16:11:11 If you're watching the upgrade proceed in the update environment, it eventually stops reporting what it's doing unless you hit a key 16:11:25 a screensaver I guess 16:11:30 It completes successfully, but it's concerning to watch 16:11:34 kparal: That was a joke, right? 16:11:46 oh, yeah, i think that one's filed somewhere 16:11:47 well it happened to me in a VM 16:11:55 console has a screensaver as well - black screen 16:12:00 you actually see something similar during a cmdline installation iirc 16:12:02 It wasn't a black screen 16:12:04 seems memory as Boxes requires more alt keys than bare metal 16:12:10 yeah i think it may be some kind of bug in the inactive blanking 16:12:12 It was showing something, just nothing useful 16:12:14 in VM I saw some graphical glitches 16:12:25 I've seen that in a vm as well 16:12:26 Hitting a key caused it to reprint the whole progress and resume 16:12:33 yeah 16:12:37 it's still running whether you hit a key or not 16:12:46 seemed more like virt graphical quirks than anyting else 16:12:48 Correct 16:12:50 just a display issue, but yeah, if someone can find the bug and throw a CommonBugs at it that'd be helpful 16:12:54 haven't seen it on bare metal though 16:13:00 i guess check fedup and systemd 16:13:01 I did 16:13:03 roshi: I hit it on bare metal 16:13:12 ah 16:13:15 (An old, slow Dell Studio Hybrid) 16:13:45 info: fedup in boxes just finished sucessfully with --nogdpcheck 16:13:49 #info multiple people have seen the corrupted console output while fedup is running until you hit a key, we will make sure it's filed and Common Bugs'ed 16:14:39 anyone seen any problems with current or pending update packages? 16:14:50 just want to make sure 0-day updates don't cause any problems on release day 16:14:59 well, i've got something weird going on with copy/paste, hope that's just me. 16:16:13 adamw: I also noticed one other thing: there dropping of video drivers makes the fedup warnings a little concerning 16:16:25 .hello dmossor 16:16:27 danofsatx-work: dmossor 'Dan Mossor' 16:16:29 sgallagh: ah, interesting 16:16:30 ahoy day 16:16:33 also dan 16:16:39 I'm here, really..... 16:17:01 sgallagh: what does it say exactly? 16:17:52 I don't have the exact text handy, but something to the effect that xorg-x11-drv- requires xorg-x11-server-, which may cause issues in the upgraded system. 16:18:17 It was one of the ancient drivers that we took out 16:18:22 hum, were they not properly obsoleted? 16:18:24 (And not relevant to the system I was upgrading) 16:18:26 well, worth looking into a bit 16:18:30 Yeah, perhaps 16:19:00 alrighty, i had a pet topic coming up next 16:19:31 #topic Fedora 22 planning 16:20:08 * kparal cheers 16:20:09 so, I wanted to propose one idea i've been kicking around a bit with anaconda and releng folks to hopefully reduce the release validation time stresses a bit 16:20:17 kparal: yeah, the reward for lots of testing is...more testing :) 16:20:57 is the idea that we should not break it so much? :) 16:21:06 that'd always help ;) 16:21:06 basically, i'd like to try and see if we can get more installer testing done earlier in the cycle. 16:21:26 that goes along my feedback on the retrospective page 16:21:28 it hasn't been decided yet so we can't be too sure, but anaconda folks are talking about freezing anaconda at alpha and going into bugfix mode only after that 16:21:31 * adamw reads it 16:21:39 I think the rawhide monthly validation matrices were really helpful 16:21:48 * amita reading 16:22:03 but this is probably about something else 16:22:04 kparal: yup, that's where i'm working from, and indeed right in line with your feedback 16:22:09 I thought that was pretty useful as well 16:22:09 nope, it's just the same :) 16:22:13 adamw: Right, I think there maybe occasional "Beta" exceptions, but they can let us know when that happens 16:22:15 so let me pitch the specific idea 16:22:16 * pwhalen started testing rawhide today 16:22:46 releng told me, which I didn't know, that the daily composes of rawhide and branched are kept around for a couple of weeks 16:22:52 http://kojipkgs.fedoraproject.org/mash/ 16:22:59 * kparal knows about that 16:23:14 that helps a lot, because it gives us a reliable URL to point to for any given date's boot.iso, and it solves the problem where there's no boot.iso on the mirrors if the compose fails on a give day 16:23:36 so, i was trying to come up with something between the monthly matrices (which are fine, but get messy, and are hard to testcase-stats) and having a new one *every day* 16:23:40 * satellit looking at koji some rawhide builds worked today 16:23:54 so what i came up with was this: we have something that listens to fedmsg for successful composes 16:24:13 when it sees a successful compose, it checks if the versions of any of the anaconda packages (anaconda, blivet, pyparted etc) are different from the last 'tested' compose 16:24:32 if they are, it creates a new Installation matrix (and maybe the others if we want to do all the testing) and sends an 'announcement' mail 16:24:42 ah, clever. but therefore the matrices will be strictly anaconda-based 16:25:01 kparal: it's a kind of approximate proxy 16:25:11 I'm fine with that 16:25:13 sounds great 16:25:18 we're probably gonna be _most_ interested in testing when those packages change 16:25:25 And let's be honest: anaconda testing accounts for the majority of criteria 16:25:33 and i believe bcl is planning to do anaconda package builds about once a week, so i expect what it'd usually result in is weekly matrices 16:25:36 * roshi can help with the fedmsg listener 16:25:45 (For good reasons; it's the first experience people have with Fedora) 16:25:47 we could also link that day's live.iso in that announcement. they are also kept around for some time 16:25:47 roshi: aw, i was planning to write it. :p no seriously, that'd be great 16:26:09 kparal: yeah, if we can figure a way to get the URL reliably 16:26:10 it's something I've been wanting to learn more about :) 16:26:18 the nightly live URLs are less friendly than the mash ones 16:26:32 roshi: tflink suggests putting it in taskotron 16:26:36 maybe they are also exposed over fedmsg, not sure 16:26:41 neat idea 16:26:42 Do you want any rate limiting in case there is a burst of Anaconda activity? Would say a week of daily updates be a problem? 16:26:49 where it could become the basis for Future Development (we can plug automatic smoke tests into it, when we have some) 16:27:02 Guest26560: tflink said the same :) i'm not sure it'd be a *problem* exactly except for exhausting people 16:27:11 i'm figuring maybe limit it to every 2-3 days or smth 16:27:42 weekly at the max, daily if there are big changes or new dependencies (eg. needs new blivet, etc.) 16:27:54 in the worst case nobody will test it 16:28:07 for a short while 16:28:25 so basically this needs the fedmsg listener and some wiki magic and some relval work, i'm aiming to get the wiki magic and relval work done this week 16:28:30 kparal: right 16:28:38 and not everything needs to be tested every time. With smaller changes it should be easier to see what needs testing. 16:28:39 * satellit wish the flattened .ks files were havested also for remix testing 16:28:52 * tflink has a ticket filed for taskotron to listen for compose events 16:29:00 bcl: yeah 16:29:24 testcase-stats should help quite a lot with seeing the overall coverage 16:29:38 Can we tie the automatic email to the anaconda commit list? That might be helpful in figuring out which pieces should be retested. 16:29:46 tflink: can you point roshi at it? thanks 16:29:50 That might help avoid a full recertification several times a week 16:30:04 Were you planning on pulling in any git comments from either the package or upstream (or perhaps just links to commits) into the matrix page, to help people know what changed? 16:30:07 sgallagh: as in, have it note what's changed since the last image? 16:30:12 brunowolff: jinx 16:30:16 =) 16:30:18 adamw: Yes, exactly 16:30:33 let's put that in the stretch goals 16:30:35 Presumably, the anaconda git has tags that match the version in Fedora 16:30:42 So it should be pretty easy to retrieve. 16:30:44 get the essential bits done first, and once those are going we can start welding bits on 16:30:48 /me volunteers to help with that. 16:30:56 sgallagh: each package build is tagged. 16:31:03 in anaconda git, that is. 16:31:18 Yeah, I was pretty sure I remembered that being the case 16:32:02 so i think this is going to be kinda useful whatever else happens, but the ideal goal for us i think would be that the anaconda freeze plan happens too, and the effect should be that by Alpha we'd all be a lot closer to 'final' state 16:32:16 i guess the other thing that would be helpful here is earlier blocker review (/me expects cries and sadness) 16:32:42 but if the goal is to have more certainty ahead of time about what needs doing, it would be valuable to review beta and final blockers as soon as they're proposed 16:32:51 if anaconda goes into bugfix only at Alpha, how does that affect us as QA? 16:32:53 that makes sense 16:33:48 adamw: Yeah, it also means that the anaconda folks will have sufficient time to actually address some of the blockers, which would be a nice change of pace :) 16:34:05 kparal: what i *hope* it would do is mean we'd have fewer new issues showing up during beta and final 16:34:13 kparal: yeah, I was wondering the same thing - what is "bugfix only" and how does that affect blockers? 16:34:26 and yeah, we'd be working with the devs in bug reports to resolve beta and final blockers as they came up, i'd expect 16:34:32 I assume a criterion violation falls under bugfix-only 16:34:38 the shape of what we're doing would be broadly similar, though, just try and spread it out 16:34:47 that was one reason why we started earlier TCs and I think it made a big differences... earlier blocker reviews could add some more weight 16:34:49 tflink: Realistically, I expect it means that they won't fix anything other than Blockers and FEs that they deem worthy 16:35:02 Which, frankly, is probably exactly what SHOULD happen 16:35:07 sgallagh: yep 16:35:15 i think the shape of an anaconda freeze policy is kind of up to them, but i'd be expecting something along the lines of sgallagh's thought 16:35:20 the only downside is that longer and more frequent blocker reviews, which was something esp. anaconda folks were not happy about 16:35:23 how is that significantly different than what we've done in the past? 16:35:28 but we really need developer presence there 16:35:32 other than earlier testing 16:35:43 tflink: The earlier testing is the biggest piece. 16:35:45 what this means is that things like product split, etc. that need anaconda development have to be proposed early and finished by Alpha. 16:35:46 kparal: in theory the *overall* time spent in blocker review shouldn't change, again it'd be a case of it moving around a bit 16:36:08 we might want to start blocker review earlier and maybe adjust the number of meetings to the flow of proposals 16:36:10 Moving development into rawhide as much as possible. 16:36:16 well, a lot of those blockers disappear before we even get to them... 16:36:16 kparal: If we do things earlier, we have the option of gathering the set and giving the developers a few days or a week to triage them 16:36:24 Rather than the few hours right before a blocker meeting 16:36:40 but I don't oppose in general 16:36:41 i *did* have a few early thoughts about possible blocker process tweaks, but they're less well figured out than this one 16:37:31 one thought i had was something along the lines of dev being allowed to accept blockers 16:37:37 the overall idea is fine, we just need to come up with some optimizations, I think 16:37:49 say a blocker's proposed and anaconda look at it and go 'oh this is obviously a blocker', they can just throw AcceptedBlocker on it and go to work 16:38:10 adamw: that's a good idea, I like it 16:38:28 but yeah, let's say - proposals for streamlining/improving blocker review are very welcome :) 16:38:36 sounds good to me 16:39:05 we can also try and get more 'obvious' blocker approval/rejection done in bugs - maybe try and avoid discussing controversial ones in-bug to avoid the flood of comments, but slam dunks could be done with a few comments 16:39:18 sounds great 16:39:36 bcl: what do you think of the 'let dev accept blockers' idea? would it be useful? 16:39:45 Having more blocker review meetings that were shorter would help me attend. Three hours during the day is really too long for me. 16:39:46 sounds good to me, but I'm not a developer 16:39:49 yes. As well as reject. 16:40:06 bcl: i think reject needs a *tad* more handling, but i was trying to think down those lines 16:40:17 Also, I don't like to see the +1/-1 voting in the bugs but a summary of votes would be ok. 16:40:29 If It needs lots of discussion it should be done on IRC. 16:40:43 bcl: i was thinking something along the lines of a provisional reject - just some flow which at least comes to the attention of The Machine so if anyone objects to the rejection we get to have the big argument 16:40:44 timezones :/ 16:41:02 Bugs should, mostly, be for data needed to debug the issue. 16:41:04 the issue with the current rejection process is that once a blocker's rejected once it drops pretty much entirely out of the process 16:41:45 bcl: yeah, that's always been my concern with doing the review in-bug, but at present we pretty much only have the meetings and the bug report, so if we want to shorten the meetings, the bug is the other natural place. we could look at a mailing-list based process, perhaps 16:41:48 a blocker-review list? 16:41:52 adamw: right. a DeveloperReject or whathave you 16:42:13 bcl: yeah, sounds right 16:42:31 the effect would be 'vote on this, but with extreme prejudice' 16:42:33 hrm. I'm not sure how much use a 3rd stream of stuff would be. 16:42:46 adamw: and very often it goes out of radar for the next release and pops out too late as serious issue... would be nice to track rejected blockers and start with it next release time (especially that on the edge blockers/last minute) 16:42:50 a DeveloperReject would then be equivalent to a comment 16:43:00 the problem with the in-bug stuff is that it makes it *really* hard to figure out what the real problem is when everyone starts piling on. 16:43:23 kparal: well it's structured, let's us do other process bits with it - blockerbugs can't be interpreting free text comments :) 16:43:29 yeah. 16:43:41 ok, so it sounds like everyone's broadly on board with the earlier matrices 16:43:46 jreznik: it's not out of the question to add that to blockerbugs, the data is still there. we'd have to figure out how to display it, though 16:43:48 and we have some good ideas for blocker review that we should refine a bit 16:44:19 propose #agreed we will try to implement the plan of organized early (pre-Alpha TC1) installer validation testing for Fedora 22 16:44:22 acks/nacks? 16:44:29 ack 16:44:29 ack 16:44:44 ack 16:44:49 ack 16:45:06 #agreed we will try to implement the plan of organized early (pre-Alpha TC1) installer validation testing for Fedora 22 16:45:17 #action roshi to look at implementing a compose event listener in taskotron 16:45:29 #action adamw to work on wiki magic and relval 16:45:36 tflink: I remember we talked about it once, do you want ticket? /me is not sure if it was filled that time or not 16:46:06 #info we will continue to discuss blocker review process improvements and discuss more specific proposals as they are made 16:46:18 those should probably go out to the list 16:46:23 yup 16:46:23 the proposals, I mean 16:46:25 One option might be to go back and review rejected blockers that were still open from time to time. 16:46:35 if you want to 'formally' propose a blocker process idea, please send a mail to the list, and we can take it from there 16:46:39 jreznik: I don't think a ticket was ever filed. If you would do that, give details on what you're looking for and what granularity you'd want, that'd be great. 16:46:50 brunowolff: the issue with that is it's *even more review* :) but yeah, it's a good thing to try and keep an eye on 16:47:13 so we've got 15 minutes and i had a few more sub-topics... 16:47:21 I think we would try to do this during the off season for blocker reviews. 16:47:56 so the next sub-topic, has anyone followed the devel@ tick/tock discussion? 16:48:04 * adamw is still comically behind on devel@ 16:48:10 * roshi has been 16:48:14 A few weeks ahead of starting alpha blocker reviews might be a good time to look. 16:48:22 as well as firewall discussion that's come back up... 16:48:35 tflink: first I'll propose it to the list as adamw asked to get more details together but it would be nice to have this feature 16:48:45 * danofsatx-work can't get sub'd to devel@ - it doesn't work for some reason. 16:48:55 probably because I'm not a developer..... 16:49:18 danofsatx-work: it's strange, it should work 16:49:29 roshi: so, do you see any qa concerns arising? 16:49:31 * kparal hasn't seen that discussion yet 16:49:36 I don't think the formal tick tock proposal is a good idea. I'd rather have some sort of review of feature proposals that look for ones that might conflict and defer some in those cases. 16:49:41 I know it should, but there must me some coding trick to hit the "subscribe" button or something 16:49:47 we're going to have to set some devel priorities for F22 - there's way too much stuff that "would be nice for F22" on the list as it is :) 16:49:50 not off hand, but am going to be thinking about it and I'll bring them up 16:50:00 kparal: http://www.phoronix.com/scan.php?page=news_item&px=MTg1NDI (phoronix warning) 16:50:06 roshi: cool, thanks 16:50:11 I actually read the article :) 16:50:16 but not the discussion 16:50:30 I'm afraid to read the discussion 16:50:38 sgallagh: same here :) 16:50:39 It got scary long over the weekend 16:50:49 sgallagh: it's surprisingly non-flame-y! 16:51:05 I was out on Fri/Sat/Sun (EMEA Amb FAD) and ... 16:51:13 ok, so i guess we're in monitoring mode on tick/tock 16:51:14 Oh good 16:51:32 #info so far we see no specific QA concerns in the tick/tock discussion, we will continue to monitor 16:51:43 next up, test days 16:52:04 guess i just wanted to see who's interested in co-ordinating test days this cycle, and if we have any specific ideas 16:52:14 roshi: are you thinking of doing it again? 16:52:32 sure, unless there's someone else who has the itch to do so 16:52:34 * danofsatx-work dropped the ball on test days this cycle 16:52:39 or wants to learn and I can work with them on it 16:52:43 participating in, anyhow 16:52:50 * roshi will be more deliberate about it this time around... 16:52:57 roshi: can always have more than one person helping out 16:53:12 anyone interested in working on it with roshi/taking over from that bum roshi? :) 16:53:29 * roshi can't tell if "bum" is a promotion or not... 16:53:32 :p 16:53:40 junland was instrumental in organizing the server test day. Should we see if he's ready/willing/able to become more involved? 16:54:18 can't hurt to ask! 16:54:29 it's a longer time commitment to help out with co-ordinating the whole cycle, though 16:55:03 understood 16:55:09 #info roshi will work on co-ordinating test days again this cycle, he's happy to have any help if anyone else wants to join in on that 16:55:24 I think pschindl will be interested to help out 16:55:27 but he seems afk 16:55:32 sweet :) 16:55:42 that he might want to help out, not that he's afk 16:55:43 roshi: i'm sure you two can manage to send each other an email :P 16:55:55 nah, probably need an action item 16:55:55 so i had tooling as the next sub-topic but we probably can't cover it in five minutes 16:55:56 yes, indeed. I will help roshi with test days :) 16:56:01 awesome 16:56:05 and a taskforce to ensure proper implementation 16:56:06 #info pschindl will help roshi with test day co-ordination 16:56:12 :) 16:56:16 roshi: weekly meetings! i expect minutes! 16:56:44 tflink: anything urgent to talk about on tooling or is next week fine? 16:56:51 i know you have the qa-devel meeting anyhow 16:56:57 signed and in triplicate - you got it 16:56:58 next week should be fine 16:57:03 notorized, even, maybe 16:57:33 =) 16:57:38 ok, last sub-topic then is release criteria 16:57:53 though i was hoping cmurf would be around 16:58:23 we had a few criteria issues come up late in validation particularly relating to multiboot, and i was hoping we could come up with a sensible plan for revising those based on our new knowledge of the underlying code behaviour 16:58:31 maybe that can go on next week's too, though 16:59:00 for info i'm planning to come up with a Server WG proposal to move server role requirements into a separate document which the criteria can reference, which will clean the server section up a bit 16:59:16 makes sense 16:59:20 like the default app requirements? 16:59:25 yeah, exactly 16:59:56 agreed. Server is, um, special. Yeah, that works, special. 17:00:54 =) 17:01:10 danofsatx-work: it's just so that as we get more server roles we don't make the criteria longer and longer with detailed stuff about what each role needs to do 17:01:17 alllllrighty 17:01:24 looks like we made it through 17:01:26 #topic Open floor 17:01:33 anyone have any other business that's not yet covered? 17:01:49 bcl: thanks for chipping in, it's appreciated 17:01:55 one thing to put on peoples' radars is the idea of gating packages/updates based on automated check results 17:02:00 np 17:02:10 I just spotted sgallagh's proposal for adjusting release/blocker criteria on devel today 17:02:11 tflink: which tests? 17:02:31 setting time constraints and stuff 17:02:42 I just wonder whether we should discuss it 17:02:53 but doesn't have to be right here and now 17:02:56 it came up during F22 and I'm not sure if that'll come up again soon but if we want to start doing that, we're going to need some discussion in the nitty-gritty of whether there are gaps that need to be addressed first 17:03:05 adamw: upgradepath and depcheck were the two that came up 17:03:56 kparal: I think it becomes less interesting if we solve this the way adamw is proposing (which is so close to the proposal I was writing up on Friday that I'm now checking for cameras in my office) 17:04:36 lol 17:04:36 It will hard to cover every case that can occur during upgrades. Dealing with dropped packages can be tricky. 17:05:41 sgallagh: ok 17:06:08 But we'll see what happens as we get closer to the next round of freezes, I suppose 17:06:21 tflink: i'm still in favour in general, i'm not sure about reliability of the tests 17:07:05 tflink: i was actually kind of wondering today if we could look at depcheck from another angle - you say it's a repo level test, so could we look at ways to run it on each day's updates-testing / updates compose, rather than trying to tie it to individual updates? 17:07:08 but it was just an idle though. 17:07:09 t 17:07:20 we're a bit over time, anyhow 17:07:27 adamw: yeah, I'm in favor as well but if we go that route, I want to make sure that everyone understands what it'll mean if we do 17:07:35 when has that stopped us in the past? 17:07:52 brunowolff: yeah, we're certainly aware of specific scenarios depcheck doesn't cover, tflink has all the details on that 17:08:03 do we know how reliable upgradepath is? 17:08:23 of course, with enforcing upgradepath we run into that whole debate with me and misc on whether it's actually desirable 17:08:24 adamw: if we don't tie it back to individual updates, who's going to check over everything to make sure that there were no failures? 17:08:33 I can think of at least one example right now where upgradepath blocking would cause issues 17:08:49 tflink: well see i was thinking we tie it back to individual updates after running it - figure out which update each failure is associated with 17:08:54 I've got several Django packages which cannot be built on F22/Rawhide because they aren't compatible with the version there. 17:09:01 adamw: that's pretty much how it works now 17:09:04 oh, k 17:09:05 But I still need to do security updates on F21 17:09:17 tflink: guess i need to look into it more 17:09:21 depcheck runs against a koji tag and we process the output to tie it back to individual updates 17:10:02 ok, we can take this out of meeting i guess :) it seems to be background discussion not anything requiring #actions or votes 17:10:06 so i'll set the acme fuse 17:11:27 * adamw opens the box marked ACME Hilariously Inaccurate Meeting Fuses 17:11:38 thanks for coming, folks! 17:11:55 5....4....2.....7....pi.....63....*kablooey* 17:12:01 #endmeeting