19:04:18 #startmeeting 19:04:18 Meeting started Wed Mar 11 19:04:18 2015 UTC. The chair is roshi. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:04:18 Useful Commands: #action #agreed #halp #info #idea #link #topic. 19:04:23 #meetingname Cloud WG 19:04:23 The meeting name has been set to 'cloud_wg' 19:04:27 #topic Roll Call 19:04:32 .hello roshi 19:04:33 roshi: roshi 'Mike Ruckman' 19:04:37 .hellomynameis dustymabe 19:04:38 dustymabe: dustymabe 'Dusty Mabe' 19:04:39 .hellomynameis jzb 19:04:41 jzb: jzb 'Joe Brockmeier' 19:04:50 .hellomynameis oddshocks 19:04:51 oddshocks: oddshocks 'David Gay' 19:05:26 it's a decent turnout :) 19:05:40 #topic previous meeting followup 19:05:50 dustymabe will create a track ticket about systemd-networkd 19:05:55 .hello rtnpro 19:05:56 rtnpro: rtnpro 'Ratnadeep Debnath' 19:06:13 roshi: going to do that now 19:06:16 sgtm 19:06:19 will post link in here 19:06:25 that's the only action item we had from last meeting 19:06:57 thanks dustymabe :) 19:07:03 ..hello mmcgrath 19:07:11 .hello mmcgrath 19:07:12 sorry :) 19:07:12 mmcgrath: mmcgrath 'Mike McGrath' 19:07:22 np :) we're easy to get along with 19:07:28 * oddshocks learns you can just use .hello 19:07:44 zodbot and meetbot have some cool stuff 19:07:50 .moar stuff oddshocks 19:07:50 here oddshocks, have some more stuff 19:07:52 .hello 19:07:54 mattdm: (hello ) -- Alias for "hellomynameis $1". 19:08:02 hmmm. :) 19:08:05 welcome mattdm :) 19:08:08 .hello mattdm 19:08:09 mattdm: mattdm 'Matthew Miller' 19:08:22 mattdm: I had thought you had to type out "hellomynameis" 19:08:29 ready to move onto the pending agenda items? 19:08:33 * oddshocks nods 19:08:41 oddshocks: only if there's a name conflict 19:08:51 #topic Maintaining docker images 19:08:54 #link https://fedorahosted.org/cloud/ticket/97 19:09:16 it would for sure be good to get this tested 19:09:34 I hadn't looked at them at all since they aren't blocking and we had enough to test during that last run 19:10:38 roshi: yes, yes it would 19:10:43 we oughta be able to automate that check though 19:10:44 roshi: so - I wanted to discuss this 19:10:49 perhaps tunir? 19:11:03 right now the docker image is "owned" by the base working group? 19:11:11 I was just explaining why *I* didn't test it :P 19:11:23 I think there's some confusion over who needs to "own" it 19:11:29 aiui, yeah 19:11:29 and we all dropped the ball for the alpha 19:11:40 it's also not represented on any of the release pages. 19:11:47 (nor release announcement) 19:12:14 so - does anybody object to cloud "owning" testing and promoting the base image? 19:12:26 if not, we add it to our tasks and need to make sure it's being produced. 19:12:29 properly. 19:12:55 * jzb looks around 19:12:59 the last part is the sticking point for me taking point on it 19:13:07 I don't object, but I we should maybe check with the Base people who said they want to own it? 19:13:12 because I only have a passing knowledge of docker 19:13:29 I figured jzb was wanting to get ducks in a row to take to them 19:13:29 mattdm: yeah, but that's a bridge to burn once we decide we want to do it. 19:13:32 not just take it 19:13:33 roshi: +1 19:13:53 mattdm: otherwise we go and say "why didn't this happen?" :-) 19:14:11 I don't object to cloud owning the docker image 19:14:21 jzb +1 19:14:26 I agree it does need some love 19:14:33 dustymabe: yep 19:14:39 OK 19:14:49 #action jzb get with base working group on docker image. 19:15:11 roshi: EOF for me. 19:16:13 sounds good to me 19:16:18 you going to update the ticket? 19:16:40 #topic Dockerfile Husbandry 19:16:46 #link https://fedorahosted.org/cloud/ticket/84 19:16:47 roshi: done 19:17:04 sweet 19:17:05 * roshi reads 19:17:44 cool, that's good to have done :) 19:18:04 #topic Checksums 19:18:10 #link https://fedorahosted.org/cloud/ticket/93 19:18:33 iirc, kushal was going to chase this one down with the systemd folks 19:20:05 yes, I think he's literally off at a conference doing that right now 19:20:23 :D 19:20:29 then we can move on 19:20:33 oooh, right -yeah. 19:20:50 #topic Atomic as a spin 19:20:53 #link https://fedorahosted.org/cloud/ticket/96 19:21:13 mattdm: you want to weigh in on this 1st? 19:21:31 sure 19:21:36 +1 19:21:40 I definitely haven't _completely_ thought this through :) 19:21:56 Does everyone understand the basics of the possibilities I was suggesting? 19:22:01 Or should I explain myself better? 19:22:11 I think B could use some explanation 19:22:27 would that be a demotion of the Base image, or would we just equalize them? 19:22:46 and we'd probably have to take this to fesco or someone to get it considered for blocker status 19:22:49 B would basically be a demotion of the base imge, yes. 19:22:50 because right now, it's not 19:23:08 I don't fully understand the consequences (good or bad) at this point 19:23:32 do we have any idea which of the two is used more by people in production? 19:23:44 Right now, almost certainly the base image. 19:23:53 yeah atomic is brand new 19:23:58 But, we don't have good numbers at all. 19:24:12 Even worse than our ability to count anything else 19:24:20 a fair point 19:24:20 since so many cloud instances won't check for updates 19:24:37 and so many others are going to be potentially thousands of instances behind one nat 19:24:51 true 19:25:01 is our infra even using it for anything? 19:25:08 ooc 19:25:10 roshi: it == ? 19:25:11 If it doesn't feel like we're ready to decide, then "C" might be the best for now. 19:25:24 here's my thinking - if Atomic is a spin, it doesn't get as much love officially but it has more flexibility 19:25:24 mattdm: my gut feeling is "c" 19:25:29 +1 for C 19:25:49 we have a second proposal from mmcgrath around doing 2-week sprint/releases 19:25:55 I'd _eventually_ like to have the cloud story be more straightforward -- less of an if-this-than-that situation. 19:26:08 where we'd have a rapidly accelerated release cycle around the Atomic Host. 19:26:16 I think that fits better with "spin" 19:26:25 and all I'm asking for there right now, is that every 2 weeks, whatever rawhide is at gets pushed to a mirror so we can link to it somehow on projectatomic.io 19:26:48 It gives us flexability on what we want to do WRT the stable releases. 19:26:50 mmcgrath and presumably that that thing is basically functional? 19:26:54 * roshi needs to read that thread closer 19:27:01 jzb: I thought spin was a good option for atomic so it could get some "focused" attention and like you said flexibility 19:27:05 yes, though I know CI can't promise that today. 19:27:08 did you see boyer's comments about spins though? 19:27:27 my concern is if we have rawhide being targeted on PA something to point to PLUS the final release 19:27:43 Fundamentally, what *I* want is to have a simple, compelling elevator pitch when I'm talking about Fedora Cloud. Right now, I really have to have two separate ones. 19:27:45 we are going to have a hell of a time explaining which to use, where to start, and fragment efforts a bit. 19:27:48 I don't think the PA site will link to final releases of atomic on Fedora 19:28:03 at least not on the basic "getting started" type pages. 19:28:07 * mattdm wonders where pennsylvania came into it 19:28:16 ProjectAtomic.io :) 19:28:17 mattdm: daaaamn 19:28:27 :) 19:28:28 oh. I thought you were ... never mind. 19:28:32 #link http://www.projectatomic.io/ 19:28:35 for the record 19:28:48 is Atomic the Shining Path we ultimately want to walk down? 19:28:51 mmcgrath: what we point to from PA is undetermined atm. 19:29:00 * roshi isn't against it or anything - just curious 19:29:01 roshi: yeah,that. 19:29:15 jzb: you're the content owner for that page, right? 19:29:27 I want https://getfedora.org/cloud/ to show the shining path 19:29:30 roshi: while I'm all-in on Atomic, I'm not sure it's the right thing for Fedora to put all its attention on. 19:29:41 that was my question 19:29:42 there's still a huge use case for the base image 19:29:52 jzb: I agree 19:29:58 mattdm: agreed on that, it's what Major would want :-D 19:30:02 mmcgrath: yes, but we have a semblance of governace that makes that decision 19:30:03 does our Fedora User base run a ton of containers or do they run more standard app stacks? 19:30:08 heh 19:30:16 mmcgrath: but I can push the bits 19:30:25 I don't think we should _ditch_ the base image. 19:30:36 roshi: well, let's also consider the user base we _want_ 19:30:40 I don't think people would set out to do that 19:30:40 jzb: k, lets figure that part out after the mtg then (I'm assuming it's a slightly different community over the Fedora community) 19:30:48 mmcgrath: yes 19:30:55 because the cloud target is intended to be a growth area 19:30:56 though significant overlap 19:30:59 coming at this from a QA perspective 19:31:14 if we make Atomic blocker, that's more testing resources we'll need 19:31:21 criteria, testcases, etc etc 19:31:38 cloud wg would be supporting two products that can block the whole release 19:31:54 I can see it being pitched later to demote the base image from being blocker 19:32:05 mattdm: I also, tbh, am still uncomfortable with Atomic having multiple release streams that are supported (e.g. f21, f22, f23) 19:32:17 which, could lead to it being ditched, just by default 19:32:23 as a top-level product we have to fit into that, as a spin, I don't think we do? 19:32:26 So, putting my Red Hat hat on for a second, I think the Atomic effort is important enough to Red Hat that Red Hat _wants_ it to be a release blocker in Fedora 19:32:46 and if lack of QA resources is a blocker for _that_, help me make the case 19:32:49 mattdm: yeah, but we also have trouble fitting into the freezes. 19:33:03 which is fine, but will we have people to do the testing and production and owners of it? 19:33:27 we can help with the case for that, sure 19:33:33 just want to make sure it's considered 19:33:42 Well we better have them or it won't work. Yes. Definitely needs to be said. 19:34:52 jzb: that's a key point too. If the Atomic $thing is primiarily rawhide-focused and has trouble lining up with release freezes, that argues that it's not ready yet 19:34:56 the other option, if this is what's wanted by the WG and fedora as a whole, is we could just bite the bullet and ditch the base image as blocker and refocus stuff onto atomic 19:35:21 roshi: *super* all-in, that is :) 19:35:23 mattdm: I mean, we *can* make it ready, but I think that may detract a bit 19:35:31 if we're polishing old news. 19:36:09 mmcgrath: how would hitting release targets *and* rawhide fit in with resources we have currently? 19:36:09 exactly 19:36:24 jzb: it'd be almost identical to what it is today 19:36:36 I'm not totally clear how we do release targets today, but I'd imagine it doesn't have to change. 19:36:42 in theory, updates testing for atomic images should be easier than for the base image 19:36:57 we already regularly build rawhide images (when the process works). I just want to link to one of those rawhide images every 2 weeks. 19:37:15 since it basically can't change, if I understand it correctly 19:37:36 mmcgrath Since _rawhide_ never freezes, that shouldn't be a problem 19:37:52 providing we have a build working during that time 19:38:13 it's not outside the realm of possible that we'd have two weeks of non-working rawhide builds 19:38:36 the issue is whether we should worry about polishing that up to be what's on the getfedora page as a numbered release 19:38:49 roshi: we're in that situation now actually 19:38:54 well, and who's doing the polishing 19:39:07 I think the main thing here is the intention and goal. We'll probably miss our goal from time to time. Hopefully less and less over time. 19:39:09 and the person with the refrectometer testing the level of polish 19:39:18 and defining the word polish :) 19:39:27 So I'm actually leaning more and more away from "C". :) 19:39:35 and I'm also thinking fairly short term here, I know walters and dwalsh would like to focus more on a more stable fedora, we just don't have the tool chain to do so today. 19:39:41 mattdm: and towards A? 19:39:59 I'm forgetting my own letters :) 19:40:04 for F22, I don't think we can do anything but C 19:40:11 mmcgrath: some of this boils down to upstreams we don't really control. 19:40:14 roshi: agreed 19:40:16 mmcgrath: e.g. k8s 19:40:21 If we _can_ satisfactorily answer all of the polish questions, I'm for making Atomic the main event. 19:40:30 and that probably isn't going to settle down for some time. 19:40:41 If we can only do a halfhearted version of it, we're better off clearly separating it out 19:40:45 we don't control the upstream but we do have control on when those upstreams make it into fedora so there is some protection as long as we catch it early enough (CI will help there) 19:40:51 is atomic on any other distros atm? 19:40:57 we could link to it from get.fpo but not necessarily put it in the spotlight there. 19:41:10 roshi: yes, released in rhel 19:41:13 and centos 19:41:14 also, what happens if there's a security flaw in f21 k8s that's discovered in a few months? 19:41:26 and debian based distros? 19:41:29 do we have someone who will own backporting it to f21 k8s? 19:41:33 roshi: nope. 19:41:48 plans to? 19:41:52 or talks or anything? 19:41:56 roshi: none I'm aware of. 19:42:09 roshi: first likely victim, er distro, would be openSUSE 19:42:11 jzb: that's _already_ what the maintainer has signed up for in maintaining the package 19:42:18 mattdm: OK 19:42:36 I just worry that if we're the *only* distro that doesn't have a vanilla cloud image, that might get us losing mindshare on the "should I run fedora in the cloud?" 19:42:53 like Digital Ocean, who puts our stuff up on their service 19:42:55 I think we do need to continue with the vanilla cloud image. 19:43:11 +1 for vanilla cloud image for a while 19:43:33 I might even say "Until even workstation apps run all in containers" 19:43:36 and practically speaking, I don't think that there are very many cases where that image would be blocked but the atomic one isn't. 19:43:55 probably not - but I'd want to know the intent in case it did 19:44:04 So my preference for B would be to ask for the base image to be a release-blocking spin 19:44:22 so base image becomes a "spin" now? 19:44:29 can a spin, by definition, block? 19:44:33 KDE does. 19:44:35 just want to make sure I understand 19:44:56 I guess I still see KDE as the Workstation product with a new face 19:44:58 That's the suggestion, yeah. 19:45:07 roshi: yeah, except that dream was not realized 19:45:13 nor shared by everyone on both teams 19:45:21 leading to the lack of realization, I guess :) 19:45:22 So then is atomic to be a spin that can block a release? or did I misread that? 19:45:34 other way around 19:45:38 is the proposal 19:45:48 base becomes a spin, atomic becomes fedora cloud 19:45:52 aiui 19:45:54 So, the status Quo is that Fedora Cloud has two outputs, one, the release-blocking base image, and two, the non-release-blocking atomic 19:46:07 and docker, and vagrant 19:46:12 at least that's the plan 19:46:25 nod 19:46:36 was the plan to put vagrant on get.fpo cloud section? 19:46:47 (if so, that's cool, just missed that) 19:46:56 mattdm: it's there 19:47:22 mattdm: https://getfedora.org/en/cloud/prerelease/ 19:47:25 oh why so it is, in the prerelease 19:47:28 nifty 19:47:38 we've got plenty of stuff that we're "producing" atm 19:47:38 though somehow we don't have the base cloud image, just Atomic. 19:48:06 It occurs to me we might have wanted Adam to weigh in more thoroughly as well. 19:48:07 jzb: wait, so that vagrant image there is atomic only? 19:48:11 oh so confusing. 19:48:14 mattdm: correct 19:48:15 mattdm: yes 19:48:20 mattdm: a twisty maze of images... 19:48:35 that's for sure 19:48:36 mattdm: which is why I'm sort of in favor of focusing on rawhide and not six-month releases. 19:48:36 hmmm fundamentally vagrant is being treated as a download format, then, not really a special target 19:48:54 jzb: and moving atomic offa there _entirely_? 19:49:13 mattdm: and having its own spin page and being linked from ProjectAtomic.io, yeah. 19:49:27 we can bring adam in if you want - afaik, he hasn't been following much if anything about cloud 19:49:29 mattdm: sort of in favor. I realize there's tradeoffs 19:49:51 Which is what I'd labeled Option A. 19:50:03 mattdm: yes 19:50:11 roshi: he +1'ed the concept (A) 19:50:27 but that was a drive-by comment, I'd like to get more of his thinking there. 19:50:54 so maybe we move (back) to the mailing list and take a vote next week? 19:51:07 we've been pretty good with getting consensus, and I think we can do so with this. 19:51:12 on list? 19:51:17 * roshi didn't see it 19:51:20 (I think if we did A, we could put a shiny atomic logo in the right column in the get.fpo page and point to it there) 19:51:25 roshi: discuss on list, maybe take a vote during the meeting. 19:51:35 I mean adams input 19:51:35 so we are leaning towards A ? 19:51:38 roshi: yes 19:51:53 dustymabe: quick (non-binding) poll? 19:52:06 oh, you mean adam miller 19:52:10 not williamson 19:52:17 roshi: ahh, I see the confusion. Yes. :-) 19:52:21 * roshi was confused - thought you meant more QA input 19:52:26 nvm 19:52:27 roshi: pfft, who needs that? 19:52:29 * jzb ducks 19:52:48 :) 19:52:53 * roshi lets jzb wander in the untested darkness with nary a warning, perhaps we'll see him again :p 19:52:57 Is A atomic as its own spin? 19:53:04 * oddshocks can't pull up the list 19:53:09 oddshocks: yeah 19:53:12 yes, let me reiterrate them... 19:53:15 oddshocks: yes 19:53:16 https://fedorahosted.org/cloud/ticket/96 19:53:23 While I don't know much about the consequences of all this of course, I did like the sound of A 19:53:39 A) atomic as own spin, maybe linked from get.fpo but not presented as "Fedora Cloud" 19:53:48 the only thing I didn't like about A was what boyer said 19:53:56 B) Atomic as the main Fedora Cloud, with the base image as a (release-blocking?) spin 19:54:02 C) as is for now. 19:54:26 Right, spins basically only get the amount of love and attention that the people working on the spins put into it and out 19:54:41 they're _not_ the main Fedora focus 19:54:49 (although I try to talk them up wherever appropriate) 19:55:17 I strongly dislike B, but A sounds good 19:55:27 I don't think it's time to make Atomic the "main" release yet 19:55:30 if you move atomic to a spin, would that spin primarily be atomic host? 19:55:38 mattdm: in theory we would be putting quite a bit of promo into the spin via Project Atomic, though I ack that it's not as well-known as Fedora, of course. 19:55:42 with A, atomic keeps it's non-blocker status, right? 19:55:51 roshi: yes 19:55:56 Rad 19:55:58 also going to want to file tickets with the design group as well 19:56:08 jwb atomic host as opposed to? 19:56:11 +1 C for F22, +1 A for F23 19:56:21 mattdm, as opposed to the current atomic cloud image 19:56:25 roshi: yeah. I think it's too late for F22 19:56:29 agreed 19:56:30 agreed 19:56:32 ^ I don't see anything wrong with roshi's feelings either 19:56:37 jwb the current atomic cloud image _is_ atomic host 19:56:39 +1 C for f22, +1 A for F23 19:56:45 oh yeah at this point I suppose we're headed for F22 beta huh :P 19:56:53 I'm with roshi and jzb then 19:56:53 I would presume you'd have two images for the atomic spin: cloud based atomic and bare metal based atomic 19:57:05 mattdm, what roshi just said 19:57:10 roshi: also vagrant boxen. 19:57:22 mattdm, i guess i was equating "atomic host" with "bare metal iso" thing 19:57:35 atomic host != atomic image, to me 19:57:38 jwb: oh I see. no. :) 19:57:53 but again, with our labrynthian image set we currently have 19:57:57 Atomic Host is the thing that hosts containers, running in virt or bare metal 19:57:58 mattdm, bummer. i could get walters' f22 atomic host iso working. i couldn't get the f21 cloud image to work 19:58:14 they're both Atomic Host 19:58:21 s/f21 cloud image/f21 atomic cloud image 19:58:37 I don't think the line is clear out in the wild though mattdm :) 19:58:41 f21 atomic cloud image should work 19:58:59 at least it's hosting my website :) 19:58:59 I think the spin would consist of both an image _and_ the bare metal installer 19:59:08 dustymabe, it booted. i couldn't get the metadata iso thing to work, so i couldn't actually do anything with it 19:59:14 mattdm: +1 19:59:15 anyway, i don't want to derail your meeting 19:59:21 mattdm, yay 19:59:24 jwb, last I checked it works fine with testcloud 19:59:40 that makes the most sense, I think for the spin 20:00:02 roshi, no idea what that is. i don't really care about _cloud_ in this aspect. i wanted to try atomic, and the iso worked great for my purposes 20:00:26 so, not that my opinion counts for anything, but making it its own spin away from cloud seems nice 20:00:37 * mmcgrath notes time check 20:01:21 in the straw poll, I'm for either A or B, depending on what people are excited about actually doing. The decision should be based on that. 20:02:00 at least in my case, it has less to do with excitement and more to do with time and resources 20:02:19 I think A is about what we can reasonably accomplish 20:02:38 A) would give Atomic the ability to diverge a bit more too right? 20:02:43 aiui 20:02:51 mmcgrath: yes 20:02:59 I think B is a little too aggressive.. just an opinion 20:03:06 mmcgrath: the only question in my mind is how much 20:03:10 * dustymabe selfishly still likes the cloud image for some things 20:03:17 Also, people would be able to involve themselves in a specialized group for the spin, in case they don't care much about cloud in general but are particularly interested in atomic 20:03:22 which I think in some ways is what Dan and Colin want. I think they'd love to use the F22 spin, and put the rawhide versions of their packages on it. 20:03:22 dustymabe: yes we definitely need it for lots of users. 20:03:31 I have no idea how to do that yet, but I know they'd like to see it. 20:04:04 we're +1 hour now, shall we continue discussion on the list? 20:04:23 mmcgrath, not too hard id have half of rawhid eon 21 20:04:25 that seems best 20:04:34 we have one more ticket, and I had something for open floor 20:04:39 Corey84: good to hear 20:04:42 +1 to the list 20:05:11 roshi: sure, sure, excitement, practical reality, whichever :) 20:05:26 :p 20:05:48 if only I could find a way to trim down my "this is cool and I want to play with it" list :p 20:06:18 #agreed Discussion on ticket 96 to move to the mailing list 20:06:24 #topic Updated images 20:06:31 #link https://fedorahosted.org/cloud/ticket/94 20:07:09 oddshocks: got anything for this? 20:07:17 since you're kind point man for this bit :) 20:07:33 Mhm 20:07:53 Like my last comment says, Dennis says there's been some sort of progress on this 20:08:12 in the form of (1) him updating the images (2) writing a script of some sort to help facilitate that, I guess? 20:08:29 That's it :P 20:08:56 also, that ticket I link to is relevant, apparently 20:09:02 https://fedorahosted.org/rel-eng/ticket/6098#comment:6 20:09:13 looks like it, thanks for tracking this one 20:09:19 tl;dr I think releng still needs more time with this 20:09:20 anybody else have anything for it? 20:09:32 nope 20:09:32 they're pretty swamped I think 20:09:42 #topic Open Floor 20:10:13 so, tomorrow we have a test day for Ipsilon 20:10:17 https://fedoraproject.org/wiki/Test_Day:2015-03-12_Ipsilon 20:10:32 Ipsilon just recently merge with what we all know as FedOAuth 20:11:00 I think it would be good for cloud folks to help with the testing of that, since as cloud people we presumably launch run and maintain web applications 20:11:30 * roshi is going to write a blog post about it today 20:11:43 There's been significant progress on adding a Cloud Image section to the releng dash, with respect to AMIs and whatnot. Soon, you'll be able to look at the releng dash and see the latest Fedimg uploads, and other info about them, like service, region, or AMI ID 20:11:52 That's probably the coolest news from my end 20:12:09 thanks to rtnpro for working on the widgets for that :) 20:12:10 sweet 20:12:32 oddshocks, :) 20:12:41 Also, Fedimg should be more stable now, thanks to recent fixes to the fedimg01 machine and whatnot 20:12:44 roshi: I wish I could help out with testing but right now I'm overbooked 20:12:52 no you're not 20:12:57 the book being over is a lie 20:13:00 :p 20:13:14 :) 20:13:34 I'm hoping to be a bigger part of test days in the future though 20:13:43 sweet, that'd be great :) 20:14:03 well, that's all I have 20:14:07 * roshi sets the fuse... 20:14:09 3... 20:14:29 2 .... 20:14:42 * oddshocks all set 20:14:48 * dustymabe cuts black wire 20:14:53 1...... 20:14:53 BOOM 20:14:54 BOOM 20:15:02 it was the wrong wire 20:15:05 yep 20:15:08 somebody screwed up the fuses :) 20:15:08 always cut the GREEN wire 20:15:15 or was it the blue? 20:15:23 roshi: that's what they want you to think 20:15:26 haha 20:15:32 thanks for coming folks! 20:15:37 what if its from the anarchist cookbook and all black tape covered? 20:15:43 #endmeeting