19:01:32 <rbergeron> #startmeeting Cloud SIG
19:01:32 <zodbot> Meeting started Fri Mar  2 19:01:32 2012 UTC.  The chair is rbergeron. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:32 <zodbot> Useful Commands: #action #agreed #halp #info #idea #link #topic.
19:01:35 <rbergeron> #meetingname Cloud SIG
19:01:35 <zodbot> The meeting name has been set to 'cloud_sig'
19:01:38 <jforbes> damn, beat me to it :)
19:01:41 * dgilmore is hungry
19:01:43 <gholms> Hehe
19:01:46 <rbergeron> #chair gholms jforbes
19:01:46 <zodbot> Current chairs: gholms jforbes rbergeron
19:01:58 <rbergeron> #topic Who's here?
19:02:04 * dgilmore 
19:02:24 * tdawson_ raises his hand.  "Here"
19:02:32 <mull> here
19:02:34 * mdomsch 
19:02:43 * gholms hums a few bars
19:02:44 * kkeithley is here
19:02:50 * jforbes is hereish
19:02:58 <rbergeron> ;)
19:03:08 * nteon 
19:03:13 * obino lurking in the background
19:03:40 <nteon> (I'm mostly lurking, but I'd be interted in helping with ec2 image related things)
19:03:48 <rbergeron> ;)
19:03:50 <rbergeron> obino! hi :)
19:03:55 * rkukura is here
19:04:00 <obino> hello there :)
19:04:12 <rbergeron> okay, well, full house it would seem
19:04:51 <rbergeron> i suppose we'll get started :)
19:05:04 <rbergeron> #topic Check Your Packages, yo
19:05:38 <rbergeron> #info just a sweet and friendly reminder that features should be at 100% by 3/20.
19:05:46 <rbergeron> Is anyone freaking out about that happening?
19:05:50 <rbergeron> hey russellb
19:05:54 * russellb waves
19:06:10 <gholms> Freaking out? Not yet.
19:06:11 <ayoung> \O/
19:06:13 * nilsson is late to the meeting
19:06:14 <rbergeron> LOL
19:06:28 <rbergeron> hey nilsson! welcome :)
19:06:34 <nilsson> hello
19:06:42 <rbergeron> gholms: okay. well, let me know, so i can get my camera... err, well, you know. :)
19:07:11 <gholms> #help Reviewer needed for Apache Rampart/C: https://bugzilla.redhat.com/show_bug.cgi?id=796462
19:07:14 <rbergeron> okay, that's all the flogging there i'll do today, unless anyone has any specific packaging drama they want to talk about
19:07:17 <gholms> plzkthx
19:07:45 <rbergeron> gholms: word
19:08:18 * rbergeron notes that if anyone is trying to get sponsored that might be a good package to review in conjunction with your kind sponsor
19:08:34 <rbergeron> alrighty, moving onwards
19:08:57 <rbergeron> #topic Different delivery formats
19:09:00 <nilsson> sponsored for what?
19:09:00 <rbergeron> So:
19:09:21 <rbergeron> https://lists.fedoraproject.org/pipermail/cloud/2012-February/001229.html
19:09:47 <rbergeron> nilsson: if you want to become a packager, you need a packaging sponsor, who will normally make you review other folks' packages as part of the "becoming sponsored" process. :)
19:10:01 <nilsson> ok, thanks
19:10:33 <rbergeron> soo - dgilmore sent out the note above a few weeks ago - I'm not sure we got anywhere with it.
19:11:07 <rbergeron> So I thought perhaps we could gather some thoughts and maybe pick some direction and apply help towards dennis' way.
19:11:14 <rbergeron> Don't you all go speaking at once now.
19:11:17 * rbergeron looks at russellb
19:11:23 <dgilmore> rbergeron: im not sure what the exact value is
19:11:30 <jforbes> I think the xz compressed raw is good personally
19:11:52 <nilsson> is there any demand for vmdk images?
19:11:56 <ayoung> agreed.  smaller images == faster downloads
19:11:59 * mull thinks we should at least pay attention to what Ubuntu does:  http://cloud-images.ubuntu.com/releases/11.10/release/
19:12:00 <dgilmore> nilsson: no
19:12:07 <ayoung> and from raw you can convert to most other formats
19:12:10 <mull> we don't have to follow them, but maybe some good ideas there
19:12:18 <ayoung> dgilmore, yes there is,  but we can let VMWare do that....
19:12:29 <dgilmore> the biggest issue i see is how do you get onto the system
19:12:55 <dgilmore> ayoung: well ive not had a single request for vmdk images
19:13:17 <dgilmore> the only thing ive been actually requested was qcow2
19:13:38 <dgilmore> the images would work just fine in say eucalyptus
19:14:01 <dgilmore> but how do you log into the image if you bring it up on your desktop system?
19:14:04 <mull> right, we (euca) do conversion for vmware ourselves
19:14:28 <dgilmore> since the images are setup to use cloud-init
19:15:03 <dgilmore> while ive had requests for them. im not sure how useful they actually are
19:15:11 <dgilmore> or if we need to say make 2 images
19:15:26 <dgilmore> one with cloud-init and one that runs firstboot
19:16:03 <ayoung> dgilmore pointer to issues with cloud-init?
19:16:10 <nilsson> would it be feasible to ship locked-down images, and provide documentation for how a user could use libguestfs to inject their own ssh keys into the image?
19:16:12 <rbergeron> mull: what on that page do you see that is immediately useful and/or with what specific audiences?
19:16:19 <dgilmore> personally i think its easy enough to just use a kickstart and do a install
19:16:43 <dgilmore> ayoung: well the issue is cloud-init goes off and fetches your ssh key so that you can ssh in as the ec2-user
19:17:24 <ayoung> dgilmore, I'm googling as we speak.  where does it get them from?  Perhaps we can make it possible to work for the desktop case.
19:17:29 <mull> rbergeron, well, first off, the idea of delivering info about how the image was produced is good
19:17:33 <dgilmore> so putting the image into your private euca cloud they should work just fine
19:17:45 <mull> also, they provide an ovf... not sure who uses it, but maybe there's a good reason
19:17:50 <dgilmore> ayoung: an amazon/euca webservice
19:18:57 <dgilmore> ayoung: there is a specific url that it talks to
19:19:11 <dgilmore> we could provide something to mimic it
19:19:27 <dgilmore> but its an extra something someone would need to run
19:20:12 <mull> dgilmore, for the desktop use case, does root console login  with no password work?  or is that explicitly disabled ?
19:20:37 <dgilmore> mull: i believe thats explictly disabled
19:20:57 <dgilmore> it would be a bad idea to have enabled
19:21:18 <dgilmore> if you happened to break out of the hypervisor you could access any other guest on the host
19:21:20 <mull> well, in the ec2 case it doesn't matter because there's no console on which to log in
19:21:33 <mull> I understand your point
19:21:52 <mull> but I've seen images which allow root login if you can get to a console, for debugging purposes
19:22:00 <ayoung> dgilmore, is it possible to put something into the .xml config file for the virtual machine that would indicate "run first-boot instead of cloud-init"  ?
19:22:35 <jforbes> If you happen to break out of the hypervisor, you have serious issues
19:23:05 <gholms> There might be a way to make cloud-init suppress what it usually runs and do something else.
19:23:08 <dgilmore> ayoung: i dont know of anything
19:23:29 <dgilmore> jforbes: i dont disagree
19:23:37 <dgilmore> i may be too paranid
19:23:38 * mull wonders if you could just have cloud init take a metadata service url as a boot parameter... and that url could be file:/// to do something baked into the image
19:24:09 <mull> I may be talking crazy
19:24:12 <obino> mull: I think it can, doens't it?
19:24:19 <ayoung> I think just cloud-init to start is OK
19:24:35 <mull> obino, maybe it does... I have not used it much
19:24:37 * obino looking for gholms to say something about it
19:25:07 <ayoung> but  we can post an article saying :  to set up a private cloud-init for your desktop,  run apache with the following ....
19:25:11 <gholms> That rings a bell.  I can't recall if that is true, but...
19:25:32 <dgilmore> ayoung: we can.
19:25:56 <dgilmore> i guess it largely depends on how we expect the images to be consumed
19:26:08 <ayoung> what happens if you boot the VM and it can't reach the public internet,  or the cloud-init request fails?  Is the VM still usable?
19:26:39 <ayoung> like,  reboot init 1  and edit /etc/firstboot
19:27:19 * rbergeron wonders if we are aiming for decision-making in this meeting (not trying to hurry anyone up, just trying to ensure we get something constructive or actionable here)
19:27:56 <mull> ayoung, reaching the public internet is not required... the metadata service is typically on a zeroconf IP
19:28:00 <dgilmore> ayoung:  curl -f http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key > /tmp/aws-key
19:28:06 <ayoung> rbergeron, question is do we need one image or two.  I'm leaning toward one, but don't have the necessary info to propose.  Personally, I'd rather just have first-boot,  but I see the benefit of cloud-init
19:28:16 <dgilmore> thats what we used to run to get the key before cloud-init
19:28:42 <dgilmore> ayoung: first-boot cant be used in ec2
19:28:55 <dgilmore> ayoung: you dont have a interactive console
19:29:00 <mull> I think we find a way to only need one.  I'm _guessing_ that ubuntu has solved this problem, since they appear to provide an ovf for use with virtualbox
19:29:06 <dgilmore> ayoung: you have to use cloud-init in ec2
19:29:08 <ayoung> dgilmore, so long as there is a clear way to cloud-init in the desktop case,  I say we go with one image,  and document it on the download page
19:30:03 <dgilmore> ayoung: listen on 169.254.169.254 and provide a ssh key at /latest/meta-data/public-keys/0/openssh-key
19:30:34 <ayoung> dgilmore, by listen I assume you mean HTTP?
19:30:43 <dgilmore> ayoung: yes http
19:31:12 <dgilmore> ayoung: as long as you do that i think cloud-init should work and you could then log in as ec2-user with ssh
19:31:14 <nteon> dgilmore: could cloud-init start first-boot if it can't connect to 169.254.169.254?
19:31:27 <dgilmore> nteon: maybe gholms ^
19:32:04 <dgilmore> nteon: we dont actually install firstboot
19:32:09 <ayoung> nteon, or perhaps we modify first-boot to run cloud-init, and fall back to its default processing upon failure?
19:32:12 <nteon> hmm
19:32:13 <gholms> We can't rely on that because the metadata service will not always be up beforehand.
19:32:14 <mull> http://cloud-images.ubuntu.com/releases/11.10/release/ubuntu-11.10-server-cloudimg-i386.ovf  <--- please read this
19:32:22 * ke4qqq shows up late
19:32:53 <rbergeron> howdy :)
19:33:07 <mull> it might give people ideas and avoid reinventing some wheels, just sayin'
19:34:20 <ayoung> mull, so you suggest we wrap the VM in an ovf and tell people that they can edit the xml to vary how to deploy?
19:35:07 <dgilmore> mull: we dont install the firstboot bits
19:35:15 <dgilmore> i guess we could and disable it
19:35:26 <dgilmore> the people could do something to tweak it
19:35:34 <rbergeron> tweakers!
19:35:44 <mull> ayoung, possibly.  right now I'm reading this:  https://help.ubuntu.com/community/UEC/Images
19:35:54 <mull> they have a section on "Ubuntu Cloud Guest images on Local Hypervisor"
19:36:02 <mull> which seems to match what we're talking about
19:36:07 <dgilmore> so seems we are sidetracked
19:36:20 <dgilmore> we could find ways to make things useable i guess
19:36:43 <dgilmore> the question then is do we just compress the raw image and say here you go
19:36:55 <nteon> gholms: what do you mean the metadata service won't be up beforehand?
19:36:59 <dgilmore> or do we also convert it to a compressed qcow2 image
19:37:15 <nteon> gholms: is there any other way to reliably detect that we're running on an ec2-like cloud?
19:37:33 <ayoung> should we create a # action for figuring out how to chose between cloud-init and first boot and leave it at that>
19:37:35 <ayoung> ?
19:37:42 <gholms> nteon: 169.254.169.254 is not always immediately accessible when cloud-init runs.
19:37:55 <mull> ayoung, seems reasonable
19:38:15 <mull> although the decision may be to do something simpler than firstboot
19:38:31 <gholms> That sounds reasonable.  I'm pretty sure cloud-init has some way of setting stuff up via boot args or something.
19:38:38 <ayoung> #action figure out how to chose between cloud-init, firstboot or different init process for vm images
19:39:00 <rbergeron> #chair ayoung
19:39:00 <zodbot> Current chairs: ayoung gholms jforbes rbergeron
19:39:04 <rbergeron> ayoung: press up and do it again
19:39:05 <ayoung> #action figure out how to chose between cloud-init, firstboot or different init process for vm images
19:39:08 <rbergeron> you rock
19:39:21 <gholms> rbergeron: You don't need to be a chair to use #action. :)
19:39:22 <ayoung> not according to zodbot I don't
19:39:42 <nteon> hrmph
19:40:31 <rbergeron> okay, so. have we ... baked that cookie well enough for now?
19:40:41 <ayoung> #action ayoung, dgilmore:  figure out how to chose between cloud-init, firstboot or different init process for vm images
19:40:45 <ayoung> meh
19:41:09 <rbergeron> ayoung: zodbot doesn't give a response to actions, if that's what you're looking for
19:41:26 <rbergeron> oh, you were adding names ;)
19:41:28 <gholms> I should be around to answer stuff about cloud-init.  If all else fails, ask smoser in #ubuntu-cloud.
19:41:28 <dgilmore> the question remains
19:41:46 <dgilmore> do we ship just compressed raw, or also qcow2
19:41:48 <rbergeron> #chair dgilmore
19:41:48 <zodbot> Current chairs: ayoung dgilmore gholms jforbes rbergeron
19:42:25 <ayoung> dgilmore, was there a significant size difference?
19:42:30 * ke4qqq would argue for raw, qcow2 and some other formats as well
19:42:51 <ayoung> I'd say one (the smallest) format, with explanations how to convert
19:43:08 <dgilmore> ayoung: compressed raw is about 114mb
19:43:20 <dgilmore> the compressed qcow2 is about 220mb
19:43:46 <dgilmore> i would put the on the mirrors http://dl.fedoraproject.org/pub/fedora/linux/releases/test/17-Alpha/Cloud/ or http://dl.fedoraproject.org/pub/fedora/linux/releases/test/17-Alpha/Virt/
19:43:52 <mull> quite the paradox.  :)
19:43:54 <dgilmore> so it would all get mirrored out
19:43:57 <ayoung> converting raw to qcow is just a matter of running qcom-img,  correct?
19:44:03 <dgilmore> ayoung: yep
19:44:10 <gholms> Everything can use raw anyway, right?
19:44:12 <ayoung> lets go raw.
19:44:27 <mull> +1
19:44:34 <russellb> is size a problem?  if not, it's nice to just download what you need and go ...
19:45:31 <dgilmore> russellb: well it would be a compressed 10gb raw image
19:45:46 <dgilmore> russellb: xz compressed it came in at 114mb
19:45:53 <russellb> i mean, space on the dl site
19:46:02 <dgilmore> russellb: mirrors like us to use less
19:46:06 <russellb> since i saw mention of just having one format
19:46:18 <mdomsch> 114mb is no problem
19:46:25 <nteon> dgilmore: uncompressed, is the image sparse, or does it take a full 10 GiB on disk?
19:46:29 <mdomsch> GBs would be a problem
19:46:39 <dgilmore> nteon: its not sparse
19:47:03 <mull> hmm, sparse would be nice
19:47:14 <ayoung> mull, +1
19:47:16 <nteon> dgilmore: I know tar can do sparse, any reason we don't?
19:47:28 <dgilmore> mdomsch: right. if we added 4  formats for 2 arches we would be looking at probably a couple of gb
19:47:36 <mull> ayoung, that's really a +1 to nteon
19:47:36 <mull> :)
19:47:55 <dgilmore> nteon: we dont get sparse images out of koji
19:48:09 <nteon> dgilmore: aah!
19:48:12 <nteon> gotcha
19:48:27 <dgilmore> we would need to do something to make them sparse
19:48:38 <dgilmore> down the road we should look at it
19:48:48 <dgilmore> but today its what we have
19:49:02 <nteon> sure, makes sense
19:49:40 <mdomsch> +1 for pushing sooner w/o sparse, add sparse when we can
19:49:40 <dgilmore> ok so to start we will just do a compressed raw image
19:49:56 <mdomsch> 2GB is fine
19:50:06 <dgilmore> if there is demand for more formats we can look at it
19:50:39 <ke4qqq> how would you measure demand?
19:50:55 <dgilmore> #action dgilmore will make compressed raw images and put under /Cloud or /Virt  next to /Live and /Fedora for Beta
19:51:00 <ayoung> ke4qqq, if we all come back in a month and say "give us qcow"
19:51:05 <dgilmore> ke4qqq: people coming and asking
19:51:19 <mdomsch> we can always put images under /pub/alt also
19:51:25 <ke4qqq> dgilmore: consider me asking for vmdk, vhd and qcow2 :)
19:51:25 <dgilmore> mdomsch: sure
19:51:26 <mdomsch> fewer mirrors, but fewer requests
19:51:43 <mdomsch> mirrors should be used for content we know will get used regularly
19:51:50 <dgilmore> mdomsch: we only put spins there for final
19:52:09 <dgilmore> mdomsch: so at least for Beta ill go with twhat i said above
19:52:10 <ke4qqq> mdomsch: makes sense - elevate to mirrors if volume increases
19:52:32 <dgilmore> for Final we could put it next to /Spins and /Mega on alt
19:52:36 * mdomsch is protective of the mirrors
19:53:04 <dgilmore> mdomsch: im very considerate of them also
19:53:12 <mdomsch> dgilmore: indeed
19:54:21 <ayoung> Anything else?
19:54:25 <dgilmore> nope
19:54:30 <dgilmore> can i go find food now?
19:54:35 <rbergeron> yes :)
19:54:38 <rbergeron> thanks, dennis :)
19:54:52 <gholms> Heh
19:54:57 <rbergeron> #topic Other Things
19:55:06 <gholms> Other things, eh?
19:55:13 <mdomsch> s3cmd sync - I've got a bunch of patches
19:55:31 <gholms> Any idea what upstream thinks of them?
19:55:33 <rbergeron> #info ke4qqq is on the record asking for vmdk, vhd, and qcow2, with a smiley face at the end
19:55:33 <mdomsch> last one (mtime compare) I don't like - way too slow and expensive
19:55:37 <rbergeron> #chair mdomsch
19:55:37 <zodbot> Current chairs: ayoung dgilmore gholms jforbes mdomsch rbergeron
19:55:39 <mdomsch> upstream has been a little quiet
19:55:51 <mdomsch> he thought he had written the mtime comparison code already
19:56:13 <mdomsch> anyhow, what we have is functional, albeit suboptimal
19:56:42 <mdomsch> I need to package up my changes into a version deployed into FI only for now
19:57:01 <mdomsch> dgilmore, is there a way to get notified when you do a push?
19:57:10 <mdomsch> e.g. that whole magic message bus thing we talked about years ago?
19:57:25 <rbergeron> MAGIC BUS
19:57:26 <mdomsch> so I don't have to put it on a cronjob, but instead catch a notification?
19:57:39 <dgilmore> mdomsch: not today
19:57:44 <mdomsch> ok
19:57:50 <dgilmore> mdomsch: i want to work on a composedb for fedora
19:57:50 <rbergeron> mdomsch: is this related to the magic bus spot talked about at the engineering meeting perhaps :)
19:58:08 <mdomsch> rbergeron, I thought that was for fucdon travel!
19:58:10 * rbergeron apologizees to spot for using his name in vain, er, in a meeting
19:58:19 <dgilmore> mdomsch: the composedb could tell you when the last pushes were done
19:58:35 <mdomsch> hmmm
19:58:40 <mdomsch> easy to query, from, say, bapp1?
19:58:47 <dgilmore> but you would still need to ask it
19:58:53 <dgilmore> mdomsch: it would be
19:59:00 <mdomsch> hmmmmm
19:59:04 <dgilmore> mdomsch: but the composedb is in very early planning stages
19:59:18 <mdomsch> ah
19:59:30 <mdomsch> well, plan on throwing a message somewhere (db trigger?)
19:59:40 <mdomsch> until then, I'll just cronjob a few times a day
20:00:01 <mdomsch> and wait for upstream to take or reject my patches
20:00:09 <nirik> yes, a message bus could solve this issue.
20:00:15 <dgilmore> mdomsch: :) the first parts of it i hope to have in place for f18
20:00:27 <nirik> "I am doing a compose" "I have finished a compose" for anything that listens to it.
20:00:50 <mdomsch> nirik, I need "I have written to the netapp in the /foo directory"
20:00:59 <nirik> sure.
20:01:35 <rbergeron> i hope it's called the magic bus project. and when people say, "Who is workign on it," we nod and say, "Yes. Exactly."
20:01:44 <dgilmore> nirik: hopefully ill cater for consumers
20:02:05 <nirik> yeah, I think the early thought is 0mq...
20:02:09 <dgilmore> rbergeron: we all point at you
20:02:35 * rbergeron will assume nobody got her musical joke and pouts
20:02:39 <rbergeron> nirik: yeah
20:03:04 <nirik> that build compose script sure plays a mean pinball!
20:03:06 * rbergeron notes that magic buses is one of russellb's favorite topics too, lol
20:03:10 <spot> why does it sound like CSI in here?
20:03:10 <rbergeron> nirik: OH GOD
20:03:17 <russellb> \o/
20:03:23 * gholms gets dragged off to another meeting
20:03:26 <rbergeron> i thought only spot played a mean pinball
20:03:36 <rbergeron> gholms: gregdek is cruel!
20:03:45 <spot> rbergeron: i've got nothing on that deaf, dumb, and blind compose script.
20:03:56 <gholms> This one is $bossman's, not gregdek's. :/
20:04:08 <rbergeron> okay. we are descending into bad jokes and i'm not sure if mdomsch is done :)
20:04:25 <rbergeron> gholms: oh
20:04:33 * rbergeron doesn't want to interrupt good train of thought
20:06:06 <rbergeron> .... okay, i think i just killed us somehow
20:06:16 <rbergeron> does anyone else have anything?
20:06:59 <nilsson> rbergeron, yeah, I'm looking for stuff to do
20:07:10 <rbergeron> #topic introducing nilsson
20:07:20 <rbergeron> nilsson: welcome. i think a number of folks saw your intro mail :)
20:07:56 <rbergeron> of course you come to an interesting group looking for things to do - because I am pretty sure there are many things to do. and there are probably people heere who could throw them at you.
20:08:07 * rbergeron also notes we started a google summer of code page for fedora, hoping to get that rolling
20:08:19 <rbergeron> ...so if anyone has any ideas, that might be a good place to look (or put ideas) as well
20:08:22 * rbergeron puls out said link
20:08:22 <nilsson> I see
20:08:40 <rbergeron> http://fedoraproject.org/wiki/Summer_coding_ideas_for_2012
20:09:09 <rbergeron> nilsson: that might be an interesting place for you to look. I think everyone just vanished for eating, but there are several folks doing packaging for projects that could use help, etc.
20:09:33 <nilsson> I noticed there is a Draft cloud doc, who is the point of contact for that?
20:09:40 <nilsson> rbergeron, any packages in particular?
20:09:58 <rbergeron> ah. the folks in #fedora-docs, mostly. i would talk to sparks.
20:10:04 * rbergeron pokes at sparks
20:10:16 <nilsson> ok, thanks
20:10:24 <rbergeron> nilsson: i know that the eucalyptus and cloudstack folks are hard at work, as are the opennebula folks.
20:10:51 <rbergeron> but it may depend on your experience, your interests, etc. :)
20:11:01 <nilsson> true
20:11:16 <nilsson> ok, that's really all the questions I had
20:11:40 <rbergeron> okay. I'll drop you a mail on list with a few suggestions :)
20:11:46 <rbergeron> thanks for coming!
20:11:48 <rbergeron> :)
20:11:53 <rbergeron> #topic Closing down in 10 seconds
20:12:04 <rbergeron> Bye, folks. :) see y'all next week.
20:12:10 <rbergeron> #endmeeting