Virtualization vs The Cloud: How do you decide?

This article is available in other languages:

Imagine you’re in charge of technology for a hot new web startup. You’ve progressed past your minimum viable product, had great success in your private beta, and now need to design the production environment that your application will be deployed to that will satisfy your millions of hypothetical customers.

My guess is that you’ve immediately thought “deploy that app to the cloud”. Commodity cloud hosting has received a lot of press over the past few years, so you’d be forgiven for thinking that it’s the only sensible option these days.

But I want to talk about deploying to your own “private cloud” by building a virtualized environment on top of bought or leased hardware in a plain old vanilla datacenter, and why it might be the right option for your app.

I don’t want to give the impression I’m against commodity cloud hosting in general. The startup I work for now, a social network for movie-goers,  is hosted on public cloud.

Before co-founding this company, I was in charge of technology for an online marketplace host. Our most popular product was a website template marketplace, but by the time we left we were hosting 9 different marketplaces out of the same codebase and hosting environment.

With the marketplace startup, we actually went against the trend, taking our application off the cloud and onto a (mostly) virtualized environment on physical hardware under our control. So I want to contrast these two decisions, and hopefully help you identify whether your app might be better suited to deployment on your own private cloud.

A virtualized environment is a must

Whether you go to the cloud, or roll your own virtualized environment, one way or another, you should be deploying onto virtual servers. For me there are two main benefits of virtualized hosting that I can’t live without.

The first is that it turns app server builds (or rebuilds) into a software problem. A common best practice in web operations is to rebuild rather than repair or modify your servers. A lot of the complexity in managing a server pool is in the accretion of small changes to server configuration over time, especially if they are applied by hand.

In the past, you needed a lot of budget to have the spare servers lying about so you could build your new database, plug it in, switch out the old one, rebuild that one. Now you can just create a new “server” in minutes, apply your configuration and go with a few clicks or shell scripts.
The second is that it lets you do “Just In Time” capacity planning. As long as the underlying physical capacity is present (something to keep an eye on whether you go public or private cloud), you can create new servers of whatever specs you need as you need them.

So lets assume you’re in agreement, virtual servers are a good idea. Now do you go to the cloud or roll your own?

The main differences between Cloud and Virtualized Environments

Public cloud hosting gives you immediate benefits in the areas of upfront costs and flexibility, at the cost of lower raw performance, high individual instance failure rates, and the risk that you’ve got a lousy neighbour or two in the shared environment.

Running your own virtualized environment involves a higher upfront cost, and the need to manage the underlying hardware (or find someone to do that for you). But in return you get more predictable performance, higher individual instance availability and total control over your environment.

How does your business grow?

The first thing I look at when making this decision is “how will this business grow?”.
Before any other technical consideration, you need to remember that your production setup is a system to turn dollars into web traffic. You’re looking for the best bang for buck over the short and long term, taking into account not just the money you spend on hardware, but the opportunity cost in engineering time working on that stack.

For the moviegoers network, we weren’t sure when we were going to start getting traction. To start leasing servers immediately would mean spending money unnecessarily early. Flexibility trumped all other concerns for us. In exchange, we’ve traded off some performance, and some availability. For this stage of the business, we’re happy with that.

For the marketplace startup, the situation was completely different. Our product had stabilised, yet a lot of our growth was yet to come. Growth in revenue was tied very predictably to growth in traffic, and after a couple of years of business we understood the way traffic changed seasonally.

With a dedicated virtualized environment, we got to keep the flexibility at the micro-level we had on a public cloud while being able to have a “master plan” at the macro level that gave us significant cost advantages.

What does performance and availability mean to your business?

Ask your CEO, and they’re probably going to tell you “I want five nines of availability and to have the fastest site in our market segment”. Tell them how much that will cost them, and you’re likely to get a more nuanced response.

As mentioned above, for our social network, flexibility trumps all. We’re getting perfectly acceptable performance and uptime compared to our nearest competitors, and we’re happy with that.

For the marketplace company, the situation was much more complex. The marketplaces are for digital content creators to sell their goods to whoever wants to buy them. This meant we had all the ordinary requirements of an E-commerce application, combined with the need for high IO performance (because of all the large digital goods we were dealing with – web themes, videos, photos, design templates, etc)

On the E-commerce front, end user performance is very important. As the competition in your market rises, so too does the importance of your front-end performance. Availability is also incredibly important, every minute the site is down represents money you’re not making.

Because of the way instances, and sometimes whole availability zones fail, you need to architect your way around those problems on the cloud. Netflix does an outstanding job of this. The company’s articles on architecting for the cloud are the gold standard on the subject.

The problem for our marketplace company was that we were severely constrained in engineering resources. Instead of changing the nature of our application to suit the cloud, we got to change the nature of our cloud to suit our application.

By having full control over what passed over our networks, we had much more predictable performance. We were able to customise our underlying file storage to give us the best IO performance for our specific workload. When our databases had higher performance needs than our virtual servers could handle, we moved them onto bare metal servers, and parked them right next to our virtual environment which kept latency low. Because we had full control of the underlying machines, we knew that the risk of instance failure was low, and maintaining availability was simplified.

All we had to do to get all that was spend a little more up front (which paid itself back over time), and make sure we kept adding physical boxes to the pool quickly enough so that we had the capacity to add new virtual servers whenever we liked.

Conclusion

One way or another, virtualisation should be a part of your next production stack, whether you are building a new application now or migrating an existing one.

If your future traffic is uncertain – it may go up and down dramatically, certainly look towards commodity cloud hosting.

If you have a more predictable load profile, have specific performance or availability needs, or are a bit of a control freak, definitely have a look at setting up your own private cloud.

Image: Thinkstock