Wide Awake Developers


Update: Sun Cloud API Not the Same as Amazon

It looks like the early reports that Sun's cloud API would be compatible with AWS resulted from the reporters' exuberance (or mere confusion.)

It's actually nicer than Amazon's.

It is based on the REST architectural style, with representations in JSON. In fact, I might start using it as the best embodiment of REST principles. You start with an HTTP GET of "/". In this repsonse to this and every other request, it is the hyperlinks in the response that indicate what actions are allowed.

Sun has a wiki to describe the API, with a very nicely illustrated "Hello, Cloud" example.

Amazon as the new Intel

Update: Please read this update. The information underlying this post was based on early, somewhat garbled, reports.

A brief digression from the unpleasantness of reliability.

This morning, Sun announced their re-entry into the cloud computing market. After withdrawing Network.com from the marketplace a few months ago, we were all wondering what Sun's approach would be. No hardware vendor can afford to ignore the cloud computing trend... it's going to change how customers view their own data centers and hardware purchases.

One thing that really caught my interest was the description of Sun's cloud offering. It sounded really, really similar to AWS. Then I heard the E-word and it made perfect sense. Sun announced that they will use EUCALYPTUS as the control interface to their solution. EUCALYPTUS is an open-source implementation of the AWS APIs.

Last week at QCon London, we heard Simon Wardley give a brilliant talk, in which he described Canonical's plan to create a de facto open standard for cloud computing by seeding the market with open source implementations. Canonical's plan? Ubuntu and private clouds running EUCALYPTUS.

It looks like Amazon may be setting the standard for cloud computing, in the same way that Intel set the standard for desktop and server computing, by defining the programming interface.

I don't worry about this, for two reasons. One, it forestalls any premature efforts to force a de jure standard. This space is still young enough that an early standard can't help but be a drag on exploration of different business and technical models. Two, Amazon has done an excellent job as a technical leader. If their APIs "win" and become de facto standards, well, we could do a lot worse.

Licensing for Windows on EC2

One thing I noticed when I fired up my first Windows instances on EC2 was that Windows never asked me for a license key.  From examining the registry, it appears that a valid license key is installed at boot time.  On two instances of image ami-b53cd8dc (ec2-public-windows-images/Server2003r2-i386-anon-v1.01 for i386) I got exactly the same key.

Likewise, on two different instances of ami-7b2bcf12 (ec2-public-windows-images/Server2003r2-x86_64-anon-v1.00 or x64), I got the same license key--though not the same key as the i386 image.

This tells me that the license key is probably baked into the image. It's also possible that these particular license keys are unique to my account. If someone else wants to compare keys, it'd be an interesting experiment.

Either way, the extra 2.5 cents per hour on the small instance must go to Microsoft to pay for license rental.


Windows on EC2, from a Mac

It may be a bit perverse, but I wanted to hit a Windows EC2 instance from my Mac. After a little hitch getting started, I got it to work. There are a few quirks about accessing Windows instances, though.

First off, SSH is not enabled by default. You'll need to use remote desktop to access your instance. Remote desktop uses port 3389, so the first step is to create a new security group for Windows desktop access

$ ec2-add-group windows -d 'Windows remote desktop access'
GROUP    windows    Windows remote desktop access

Then, allow access to port 3389 from your desired origin. I'm allowing it from anywhere, which isn't a great idea, but I'm on the road a lot. I never know what the hotel's network origin will be.

$ ec2-authorize windows -p 3389 -P tcp
GROUP        windows    
PERMISSION        windows    ALLOWS    tcp    3389    3389    FROM    CIDR

Obviously, you could add that permission to any existing group that you already use.

There's a bit of a song and dance to log in. Where Linux instances typically use SSH with public-key authentication, Windows server requires a typed password. Amazon has come up with a reasonable, but slightly convoluted, way to extract a randomized password.

You will need to start your instance in the new security group and with a keypair. The docs could be a little clearer, in that here you're providing the name of the keypair as it was registered with EC2. The first few times I tried this, I was giving it the path of the file containing the keypair, which doesn't work.

$ ec2-describe-keypairs
KEYPAIR    devkeypair    02:10:65:9e:51:73:7e:93:bd:30:e2:5d:91:03:d5:e1:d4:0e:c0:f4
$ ec2-run-instances ami-782bcf11 -g windows -k devkeypair
RESERVATION    r-82429ceb    001356815600    windows
INSTANCE    i-f172db98    ami-782bcf11            pending    devkeypair    0        m1.small    2008-10-23T20:01:36+0000    us-east-1a            windows

After all that, and waiting through a Windows boot cycle, you can access the Windows desktop through RDP.

What's that? You don't have an RDP client, because you're a Mac user? I like CoRD for that. I also saw a lot of references to rdesktop, which is available through Darwin Ports. (For today, I wasn't prepared to install Ports just to try out the Windows EC2 instance!)

Extract the public IP address of your instance:

$ ec2-describe-instances
RESERVATION    r-82429ceb    001356815600    windows
INSTANCE    i-f172db98    ami-782bcf11    ec2-75-101-252-238.compute-1.amazonaws.com    domU-12-31-39-02-48-31.compute-1.internal    running    devkeypair    0        m1.small    2008-10-23T20:01:36+0000    us-east-1a        windows

Fire up CoRD and paste the IP address into "Quick Connect".

Well, now what? Obviously, you'll use "Administrator" as the username, but what's the password? There's a new command in the latest release of ec2-api-tools called "ec2-get-password".

$ ec2-get-password i-f172db98 -k keys/devkeypair.pem

Note that this time, I'm using the path of my keypair file. EC2 uses this to decrypt the password from the instance's console output. At boot time, Windows prints out the password, encrypted with the public key from the keypair you named when starting the instance.

Success at last: fully logged in to my virtual Windows server from my Mac desktop.

Don't Break My Heart, EC2!

I'm a huge booster of AWS and EC2. I have two talks about cloud computing, and one that's pretty specific to AWS, on the No Fluff, Just Stuff traveling symposium.

With today's announcement about EC2 coming out of beta, and about Windows support, I wanted to try out a Windows server on EC2.


ec2-describe-images -a | grep windows
IMAGE    ami-782bcf11    ec2-public-windows-images/Server2003r2-i386-anon-v1.00.manifest.xml    amazon    available    public        i386    machine        
IMAGE    ami-792bcf10    ec2-public-windows-images/Server2003r2-i386-EntAuth-v1.00.manifest.xml    amazon    available    public        i386    machine        
IMAGE    ami-7b2bcf12    ec2-public-windows-images/Server2003r2-x86_64-anon-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-7a2bcf13    ec2-public-windows-images/Server2003r2-x86_64-EntAuth-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-3934d050    ec2-public-windows-images/SqlSvrExp2003r2-i386-Anon-v1.00.manifest.xml    amazon    available    public        i386    machine        
IMAGE    ami-0f34d066    ec2-public-windows-images/SqlSvrExp2003r2-i386-EntAuth-v1.00.manifest.xml    amazon    available    public        i386    machine        
IMAGE    ami-8135d1e8    ec2-public-windows-images/SqlSvrExp2003r2-x86_64-Anon-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-9835d1f1    ec2-public-windows-images/SqlSvrExp2003r2-x86_64-EntAuth-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-6834d001    ec2-public-windows-images/SqlSvrStd2003r2-x86_64-Anon-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-6b34d002    ec2-public-windows-images/SqlSvrStd2003r2-x86_64-EntAuth-v1.00.manifest.xml    amazon    available    public        x86_64    machine        
IMAGE    ami-cd8b6ea4    khaz_windows2003srvEE/image.manifest.xml    602961847481    available    public        i386    machine        

mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10
Server.InsufficientInstanceCapacity: Insufficient capacity.
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10
Server.InsufficientInstanceCapacity: Insufficient capacity.
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10 -z us-east-1a
Server.InsufficientInstanceCapacity: Insufficient capacity.
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10 -z us-east-1b
Server.InsufficientInstanceCapacity: Insufficient capacity.
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10 -z us-east-1c
Server.InsufficientInstanceCapacity: Insufficient capacity.

Ack! Insufficient capacity?! That's not supposed to happen. Wait a second... let me try my own image

mtnygard@donk /var/tmp/nms $ ec2-describe-images
IMAGE    ami-8a0beee3    com.michaelnygard/nms-base-v1.manifest.xml    001356815600    available    private        i386    machine        
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-8a0beee3
RESERVATION    r-0c4a9465    001356815600    default
INSTANCE    i-8e79d0e7    ami-8a0beee3            pending        0        m1.small    2008-10-23T17:25:21+0000    us-east-1c        
mtnygard@donk /var/tmp/nms $ ec2-run-instances ami-792bcf10
Server.InsufficientInstanceCapacity: Insufficient capacity.

Very interesting. Looks like there's enough capacity to run all the Linux based images, but not enough for Windows?

Seems like there might be some contractual limit on how many Windows licenses Amazon is allowed to rent out. I would also infer some serious pent-up demand to eat them all up this quickly.

Or maybe it's just a glitch. We'll see.

Update [1:15 PM] I was just able to start five instances. Could be fluctuations in demand, or it could be clearing of a glitch. It's always hard to tell what's really happening inside the cloud.

Update [2:50 PM] My plaintive post in the AWS forums got a very quick response. The inscrutable wizard JeffW posted a "we're working on it" and "it's fixed" messages just 3 minutes apart. We'll probably never know quite what was going on.

S3 Outage Report and Perspective

Amazon has issued a more detailed statement explaining the S3 outage from June 20, 2008.  In my company, we'd call this a "Post Incident Report" or PIR. It has all the necessary sections:

  • Observed behavior
  • Root cause analysis
  • Followup actions: corrective and operational

This is exactly what I'd expect from any mature service provider.

There are a few interesting bits from the report. First, the condition seems to have arisen from an unexpected failure mode in the platform's self-management protocol. This shouldn't surprise anyone. It's a new way of doing business, and some of the most innovative software development, applied at the largest scales. Bugs will creep in.

In fact, I'd expect to find more cases of odd emergent behavior at large scale.

Second, the back of my envelope still shows S3 at 99.94% availability for the year. That's better than most data center providers. It's certainly better than most corporate IT departments do.

Third, Amazon rightly recognizes that transparency is a necessary condition for trust. Many service providers would fall into the "bunker mentality" of the embattled organization. That's a deteriorating spiral of distrust, coverups, and spin control. Transparency is most vital after an incident. If you cannot maintain transparency then, it won't matter at any other time.


Canadian Privacy Commissioner Highlights Cloud Privacy Concerns

A little while ago, I wrote a piece about the conflict between "clouds" and the hard boundaries of the political sphere. There's no physical place called "cyberspace", and any cloud computing infrastructure has to actually exist somewhere.

Like many U.S. citizens, I really hate the idea that facts about me become somebody else's copyrighted property just because they get stored in a database. Canada has a justifiably good reputation for protecting its citizens' privacy. Their legal framework takes the refreshing position of protecting individuals rather than protecting the ability of non-corporeal entities (a.k.a. "incorporated persons", a.k.a. "corporations") to collect any and all information.

I hadn't realized that there were such offices as the "Information and Privacy Commissioner of Ontario", however.

Better still, Ontario's IPC Commissioner, Dr. Ann Cavoukian, is very current. She's just released a white paper on the privacy implications of cloud computing. She's calling for open standards around digital identity management, and outlines some technological building blocks needed for controllable trust and identity verification.

Unlike the U.S. approach to identity verification, Dr. Cavoukian's approach has nothing to do with catching illegal aliens, welfare frauds, or terrorists. Instead, it's about creating open, trustworthy ways for humans to interact in all their various modalities from commerce, to entertainment, and even to romance. 

Quickie: GAE is GA

According to eWeek, Google will make GAE open to public use on May 28th.  Which would be today.

The original GAE site isn't updated at this point, but you can get started anyway.  I just set up my account and registered an app. (I predict tens of thousands of empty apps. Long-tail distribution here, just like SourceForge: an overwhelming majority of empty projects, with a vanishingly tiny minority that have 99% of the traffic.)

Now I just need to find time to learn Python and write something cool. 

Project Hydrazine

Part of Sun's push behind JavaFX will be called "Project Hydrazine".  (Hydrazine is a toxic and volatile rocket fuel.)  This is still a bit fuzzy, and they only left the boxes-and-arrows slide up for a few seconds, but here's what I was able to glean.

Hydrazine includes common federated services for discovery, personalization, deployment, location, and development. There's a "cloud" component to it, which wasn't entirely clear from their presentation. Overall, the goal appears to be an easier model for creating end-user applications based on a service component architecture. All tied together and presented with JavaFX, of course.

One very interesting extension---called "Project Insight"---that Rich Green and Jonathan Schwartz both discussed is the ability to instrument your applications to monitor end-user activity in your apps.

(This immediately reminded me of Valve's instrumentation of Half-Life 2, episode 2. The game itself reports back to Valve on player stats: time to complete levels, map locations where they died, play time and duration, and so on. Valve has previously talked about using these stats to improve their level design by finding out where players get frustrated, or quit, and redesigning those levels.)

I can see this being used well: making apps more usable, proactively analyzing what features users appreciate or don't understand, and targeting development effort at improving the overall experience.

Of course, it can also be used to target advertising and monitor impressions and clicks. Rich promoted this as the way to monetize apps built using Project Hydrazine. I can see the value in it, but I'm also ambivalent about creating even more channels for advertising.

In any event, users will be justifiably anxious about their TV watching them back. It's just a little too Max Headroom for a lot of people. Sun says that the data will only appear in the aggregate. This leads me to believe that the apps will report to a scalable, cloud-based aggregation service from which developers can get the aggregated data. Presumably, this will be run by Sun.

Unlike Apple's iron-fisted control over iPhone application delivery, Sun says they will not be exercising editorial control. According to Schwartz, Hydrazine will all be free: free in price, freely available, and free in philosophy.

Who Ordered That?

Yesterday, I let myself get optimistic about what Jonathan Schwartz coyly hinted about over the weekend.

The actual announcement came today.  OpenSolaris will be available on EC2. Honestly, I'm not sure how relevant that is. Are people actually demanding Solaris before they'll support EC2?

There is a message here for Microsoft, though. The only sensible license cost for a cloud-based platform is $0.00 per instance. 


I said that OpenSolaris would be available on EC2. Looks like I should have used the present tense, instead.

$ ec2-describe-images -a | grep -i solaris
IMAGE	ami-8946a3e0	opensolaris.thoughtworks.com/opensolaris-mingle-2_0_8540-64.manifest.xml	089603041495	available	public		x86_64	machine	aki-ab3cd9c2	ari-2838dd41

Yep, ThoughtWorks already has an OpenSolaris image configured as a Mingle server.

(I've said it before, but there's just no need to pay money for development infrastructure any more.  Conversely, there's no excuse for any development team to run without version control, automated builds, and continuous integration.)

Sun to Emerge from Behind in the Clouds?

Nobody can miss the dramatic proliferation of cloud computing platforms and initiatives over the last couple of years. All through the last year, Sun has remained oddly silent on the whole thing. There is a clear, natural synergy between Linux, commodity x86 hardware, and cloud computing. Sun is conspicuously absent from all of those markets.  Sun clearly needs to regain relevance in this space.

On the one hand, Project Caroline now has its own website. Anybody can create an account that allows forum reading, but don't count on getting your hands on hardware unless you've got an idea that Sun approves of.

Apart from that, Om Malik reports that we may see a joint announcement Monday morning from Sun and Amazon.

I suspect that the announcement will look something like this:

  • Based on AWS for accounts, billing, storage, and infrastructure
  • Java-based application deployment into a Sun grid container
  • AWS to handle load balancing, networking, etc.

In other words: it will look a lot like Project Caroline and the Google App Engine, running Java applications using Sun containers on top of AWS.

Amazon Blows Away Objections

Amazon must have been burning more midnight oil than usual lately.

Within the last two weeks, they've announced three new features that basically eliminate any remaining objections to their AWS computing platform.

Elastic IP Addresses 

Elastic IP addresses solve a major problem on the front end.  When an EC2 instance boots up, the "cloud" assigns it a random IP address. (Technically, it assigns two: one external and one internal.  For now, I'm only talking about the external IP.) With a random IP address, you're forced to use some kind of dynamic DNS service such as DynDNS. That lets you update your DNS entry to connect your long-lived domain name with the random IP address.

Dynamic DNS services work pretty well, but not universally well.  For one thing, there is a small amount of delay.  Dynamic DNS works by setting a very short time-to-live (TTL) on the DNS entries, which instructs intermediate DNS servers to cache the entry only for a few minutes.  When that works well, you still have a few minutes of downtime when you need to reassign your DNS name to a new IP address.  For some parts of the Net, dynamic DNS doesn't work well, usually when some ISP doesn't respect the TTL on DNS entries, but caches them for a longer time.

Elastic IP addresses solve this problem. You request an elastic IP address through a Web Services call.  The easiest way is with the command-line API:

$ ec2-allocate-address

Once the address is allocated, you own it until you release it. At this point, it's attached to your account, not to any running virtual machine. Still, this is good enough to go update your domain registrar with the new address. After you start up an instance, then you can attach the address to the machine. If the machine goes down, then the address is detached from that instance, but you still "own" it.

So, for a failover scenario, you can reassign the elastic IP address to another machine, leave your DNS settings alone, and all traffic will now come to the new machine.

Now that we've got elastic IPs, there's just one piece missing from a true HA architecture: load distribution. With just one IP address attached to one instance, you've got a single point of failure (SPOF). Right now, there are two viable options to solve that. First, you can allocate multiple elastic IPs and use round-robin DNS for load distribution. Second, you can attach a single elastic IP address to an instance that runs a software load balancer: pound, nginx, or Apache+mod_proxy_balancer. (It wouldn't surprise me to see Amazon announce an option for load-balancing-in-the-cloud soon.) You'd run two of these, with the elastic IP attached to one at any given time. Then, you need a third instance monitoring the other two, ready to flip the IP address over to the standby instance if the active one fails. (There are already some open-source and commercial products to make this easy, but that's the subject for another post.)

Availability Zones 

The second big gap that Amazon closed recently deals with geography.

In the first rev of EC2, there was absolutely no way to control where your instances were running. In fact, there wasn't any way inside the service to even tell where they were running. (You had to resort to pingtracing or geomapping of the IPs). This presents a problem if you need high availability, because you really want more than one location.

Availability Zones let you specify where your EC2 instances should run. You can get a list of them through the command-line (which, let's recall, is just a wrapper around the web services):

$ ec2-describe-availability-zones
AVAILABILITYZONE    us-east-1a    available
AVAILABILITYZONE    us-east-1b    available
AVAILABILITYZONE    us-east-1c    available

Amazon tells us that each availability zone is built independently of the others. That is, they might be in the same building or separate buildings, but they have their own network egress, power systems, cooling systems, and security. Beyond that, Amazon is pretty opaque about the availability zones. In fact, not every AWS user will see the same availability zones. They're mapped per account, so "us-east-1a" for me might map to a different hardware environment than it does for you.

How do they come into play? Pretty simply, as it turns out. When you start an instance, you can specify which availability zone you want to run it in.

Combine these two features, and you get a bunch of interesting deployment and management options.

Persistent Storage

Storage has been one of the most perplexing issues with EC2. Simply put, anything you stored to disk while your instance was running would be lost when you restart the instance. Instances always go back to the bundled disk image stored on S3.

Amazon has just announced that they will be supporting persistent storage in the near future. A few lucky users get to try it out now, in it's pre-beta incarnation.

With persistent storage, you can allocate space in chunks from 1 GB to 1 TB.  That's right, you can make one web service call to allocate a freaking terabyte! Like IP addresses, storage is owned by your account, not by an individual instance. Once you've started up an instance---say a MySQL server, for example---you attach the storage volume to it. To the virtual machine, the storage looks just like a device, so you can use it raw or format it with whatever filesystem you want.

Best of all, because this is basically a virtual SAN, you can do all kinds of SAN tricks, like snapshot copies for backups to S3.

Persistent storage done this way obviates some of the other dodgy efforts that have been going on, like  FUSE-over-S3, or the S3 storage engine for MySQL.

SimpleDB is still there, and it's still much more scalable than plain old MySQL data storage, but we've got scores of libraries for programming with relational databases, and very few that work with key-value stores. For most companies, and for the forseeable future, programming to a relational model will be the easiest thing to do. This announcement really lowers the barrier to entry even further.


With these announcements, Amazon has cemented AWS as a viable computing platform for real businesses.

Geography Imposes Itself On the Clouds

In a comment to my last post, gvwilson asks, "Are you aware that the PATRIOT Act means it's illegal for companies based in Ontario, BC, most European jurisdictions, and many other countries to use S3 and similar services?"

This is another interesting case of the non-local networked world intersecting with real geography. Not surprisingly, it quickly becomes complex. 

I have heard some of the discussion about S3 and the interaction between the U.S. PATRIOT act and the EU and Canadian privacy laws. I'm not a lawyer, but I'll relate the discussion for other readers who haven't been tracking it.

Canada and the European Union have privacy laws that lean toward their citizens, and are quite protective of them. In the U.S., where laws are written about privacy at all, they are heavily biased in favor of large data-collecting corporations, such as credit rating agencies.  A key provision of the privacy laws in Canada and the EU is that companies cannot transmit private data to any jurisdiction that lacks substantially similar protections. It's kind of like the "incorporation" clause in the GPL that way.

In the U.S., particularly with respect to the USA PATRIOT act, companies are required to turn over private customer data to a variety of government agencies. In some cases, they are required to do this even without a search warrant or court order. These are pretty much just fishing expeditions; casting a broad net to see if you catch anything. Therefore, the EU/Canadian privacy laws judge that the U.S. does not have substantially similar privacy protections, and companies in those covered nations are barred from exporting, transmitting, or storing customer data in any U.S. location where they might be subject to PATRIOT act search.

(Strictly speaking, this is not just a PATRIOT act problem. It also relates to RICO and a wide variety of other U.S. laws, mostly aimed at tracking down drug dealers by their banking transactions.)

Enter S3. S3 built to be a geographically-replicated distributed storage mechanism! There is no way even to figure out where the individual bits of your data are physically located. Nor is there any way to tell Amazon what legal jurisdictions your data can, or must, reside in. This is a big problem for personal customer data. It's also a problem that Amazon is aware they must solve. For EC2, they recently introduced Availability Zones that let you define what geographic location your virtual servers will exist in. I would expect to see something similar for S3.

This would also appear to be a problem for EU and Canadian companies using Google's AppEngine. It does not offer any way to confine data to specific geographies, either.

Does this mean it's illegal for Canadian companies to use S3? Not in general. Web pages, software downloads, media files... these would all be allowed.  Just stay away from the personal data.

Suggestions for a 90-minute app

Some of you know my obsession with Lean, Agile, and ToC.  Ideas are everywhere.  Idea is nothing. Execution is everything.

In that vein, one of my No Fluff, Just Stuff talks is called "The 90 Minute Startup".  In it, I build a real, live dotcom site during the session. You can't get a much shorter time-to-market than 90 minutes, and I really like that.

In case you're curious, I do it through the use of Amazon's EC2 and S3 services. 

The app I've used for the past couple of sessions is a quick and dirty GWT app that implements a Net Promoter Score survey about the show itself. It has a little bit of AJAX-y stuff to it, since GWT makes that really, really simple. On the other hand, it's not all that exciting as an application. It certainly doesn't make anyone sit up and go "Wow!"

So, anyone want to offer up a suggestion for a "Wow!" app they'd like to see built and deployed in 90 minutes or less?  Since this is for a talk, it should be about the size of one user story. I doubt I'll be taking live requests from the audience during the show, but I'm happy to take suggestions here in the comments.

(Please note: thanks to the pervasive evil of blog comment spam, I moderate all comments here. If you want to make a suggestion, but don't want it published, just make a note of that in the comment.) 

Google's AppEngine Appears, Disappoints

Google finally got into the cloud infrastructure game, announcing their Google AppEngine. As rumored, AppEngine opens parts of Google's legendary scalable infrastructure for hosted applications.

AppEngine is in beta, with only 10,000 accounts available. They're already long gone, but you can download the SDK and run a local container.

Here are some quick pros and cons:


  • Dynamically scalable
  • Good lifecycle management
  • Quota-based management for cost containment


  • Python apps only
  • You deploy code, not virtual machines
  • Web apps only

At this point, I'm a bit underwhelmed. Essentially, they're providing a virtual scalable app runtime, but not a generalized computing platform. (Similar to Sun's Project Caroline.) Access to the really cool Google features, like GFS, is through Python APIs that Google provides.

If you fit Google's profile of a Python-based Web application developer, this could be a very fast path to market with dynamic scalability.  Still, I think I'm going to stick with Amazon Web Services, instead. 

The Granularity Problem

I spend most of my time dealing with large sites. They're always hungry for more horsepower, especially if they can serve more visitors with the same power draw. Power goes up much faster with more chassis than with more CPU core. Not to mention, administrative overhead tends to scale with the number of hosts, not the number of cores. For them, multicore is a dream come true.

I ran into an interesting situation the other day, on the other end of the spectrum.

One of my team was working with a client that had relatively modest traffic levels. They're in a mature industry with a solid, but not rabid, customer base. Their web traffic needs could easily be served by one Apache server running one CPU and a couple of gigs of RAM.

The smallest configuration we could offer, and still maintain SLAs, was two hosts, with a total of 8 CPU cores running at 2 GHz, 32 gigs of RAM, and 4 fast Ethernet ports.

Of course that's oversized! Of course it's going to cost more than it should! But at this point in time, if we're talking about dedicated boxes, that's the smallest configuration we can offer! (Barring some creative engineering, like using fully depreciated "classics" hardware that's off its original lease, but still has a year or two before EOL.)

As CPUs get more cores, the minimum configuration is going to become more and more powerful. The quantum of computing is getting large.

Not every application will need it, and that's another reason I think private clouds make a lot of sense. Companies can buy big boxes, then allocate them to specific applications in fractions. Gains cost efficiency in adminstration, power, and space consumption (though not heat production!) while still letting business units optimize their capacity downward to meet their actual demand. 

Sun Joining the Cloud Crowd

As I was writing my last post, I somehow missed the news that Sun is building their own cloud platform, called Project Caroline.

There's a PDF about it. It appears to be a presentation for JavaOne.  It may be locked down at any minute, so the link might not work by the time you read this.

Caroline looks a lot like Amazon EC2, but with some very nice control over VLANs (I suppose they would be Virtual VLANs?), load balancing policies, and DNS... all things that EC2 lacks today. ZFS instead of S3, that will make for a more familiar storage model. No trickery needed to make data persist across restarts.

All in all, it looks very nice.

(Hmmm.  On second glance, this presentation is from JavaOne 2007!  Not much of a scoop there, Reg.)

Does anyone know what happened to this project? 



A Cloud For Everyone

The trajectory of many high-tech products looks like this:

  1. Very expensive. Only a few exist in the world. They are heavily time-shared, and usually oversubscribed.
  2. Within the reach of institutions and corporations, but not individuals. The organization wants to maximize utilization.
  3. Corporations own many, as productivity enhancers, some wealthy or forward-looking individuals own one. Families time share theirs.
  4. Virtually everyone has one. To lack one is to fall behind. No longer a competitive advantage, the lack of the technology puts one at a disadvantage.
  5. Invisibility. Most people have or use several, but are not aware of it.

Depending on your age, you might have been thinking "cell phones", "computers", or even "televisions".  I don't think I have any blog readers old enough to have been thinking "telephones", "telegraphs", or "electric motors", but they all went through the same stages, too.

I feel very comfortable putting "cloud computing" in that list, too. Cloud computing is at stage 1. It's expensive enough that there are a few in the world: Amazon AWS, Mosso, BungeeConnect, even Force.com. They're shared, multitenant, and soon to be oversubscribed.

One day, I suspect that we'll each have our own computing cloud attending us, formed out of the many computing devices that surround us every day, but I'm getting ahead of myself.

Before that, we'll see enterprises, first large then medium and small, building their own computing clouds.

"Wait a minute," you object. "That misses the whole point of cloud computing. The entire purpose is to not own the infrastructure."

That's true, today. It was also true, at one time, that farmers did not want to own their own steam engines. So, they outsourced the job. Farmers would own machines like threshers that had everything except the troublesome boiler and engine. Those required technical expertise to run, so the farmers left that job up to folks who would bring their steam engine around, hook it up to the thresher, and charge the farmer for the length of time he needed it. As steam engines got cheaper and safer, they eventually got built right into the thresher.

This next part may sound like FUD. It isn't. I like cloud computing. I like virtualization. In fact, I think it's about to revolutionize our industry.

I like it so much that I think every company should have one.

Why should a company build its own cloud, instead of going to one of the providers? Several reasons, some positive, some not so much.

On the positive side, an IT manager running a cloud can finally do real chargebacks to the business units that drive demand. Some do today, but on a larger-grained level... whole servers. With a private cloud, the IT manager could charge by the compute-hour, or by the megabit of bandwidth. He could charge for storage by the gigabyte, and with tiered rates for different avaialbility/continuity guarantees. Even better, he could allow the business units to do the kind of self-service that I can do today with a credit card and The Planet. (OK, The Planet isn't a cloud provider, but I bet they're thinking about it.  Plus, I like them.)

I actually think this kind of self-service and fine-grained chargeback could help curb the out-of-control growth in IT spending, but that's a different post.

This would seriously raise the level of discourse. Instead of fighting about server classes, rack space, power consumption, and rampant storage sprawl, IT could talk to the business about levels of service. Does this app need 24x7 performance management with automatic resource allocation to maintain a 2 second response time? Great, we can do that! This other one doesn't need to be fast, but it had better work every single time a transaction goes through? We can do that, too! This application needs user experience monitoring, that database only needs non-redundant storage, because it can be recreated from other sources... it's a better conversation to have than, "No, our corporate standard is WebSphere running on RedHat Enterprise Linux 4, with Dell PowerEdge servers.  You can have any server you want, as long as it's a Dell PowerEdge."

I also think that the gloss will come off of the cloud computing providers. (I know, most people still haven't heard of them yet, but the gloss will inevitably come off.)

Accidents happen. Networks still break, today, and they will in the future too. Power failures happen. How would you defend yourself in a shareholders' lawsuit after millions in losses thanks to a service provider failure? (Actually, that suggests there may be an insurance market developing here. Any time you've got quantifiable risk and someone willing to pay to defray that risk, sure as hell, you'll find insurance companies.)

Service providers get oversubscribed. What happens when your application is slow, and remains slow for months? Having an SLA only means you get some money back, it doesn't mean your problem will get fixed. It's a dirty secret that some service providers are quite happy paying out credits, if they can avoid bigger costs. What's your recourse? Transition costs. It costs a lot.

Latency matters. It might matter more today than ever before, since most internal applications have gone to web interfaces. Keeping your endpoints on your own network at least lets you control your own latency. 

Then there's security. Many of my clients are dealing with PCI audits and compliance. I have no idea what they'd say if I suggested moving their data into the cloud. I'm pretty certain I wouldn't still be in the room to hear what they said. I'd probably be standing outside in the rain, trying to catch a cab back to the airport.

Like I said, I'm not trying to FUD cloud computing. I think that it's so good that every company should have one.

There's one more reason I think it makes sense to build internal clouds. I'll talk about that in my next post.