The Database Administrator is dead

Flying somewhere over Utah in a cramped regional jet, I started thinking about the good old/bad old days of being an Oracle Database Administrator (DBA) 15 years ago.

The DBA has traditionally been the person in charge of strategies, optimization, capacity planning and security for databases that store and supply information to a given application. However, it occurred to me that the role of the DBA has changed dramatically in the last few years, with a lot more changes to come.

As a sign of that change, most of our ObjectRocket customers don’t have DBAs on staff.

This isn’t really a surprise. As developers have been burdened or perhaps graced with ever-faster development cycles, and as applications rush to market, the role of the DBA continues to slip away. Times to market are down, innovation is up, and the programming languages that enable this kind of agility have exploded in popularity—node.js, php, and python apps are the new commonplace. Everything is just moving faster.

This trend has led developers to seek and adopt technologies that fit the design attributes they’re already using. The traditional Ops department can’t keep up. Initially, cloud-enabled developers began to rise, quickly spooling up prototypes and proof of concepts.

However, this quickly morphed into deploying full-fledged production apps on the cloud. These days, the technology decision maker is the dude with Sublime Text open and a cloud control panel up in Chrome. Indeed, a whole new generation of developers doesn’t even know how to use anything but the cloud.

Why should they? The experience a developer gets from the cloud is often more useful than with an in-house solution. If they even have an in-house alternative anymore.

The experience in the cloud has changed as well. Developers have become accustomed to, say, spinning up some nodes via Boto, firing up Flask, and suddenly they’re off to the races. Or perhaps they just want to use an app engine and forget about infrastructure all together.

So this brings us to the data.

Increasingly, data is becoming a massive component of any app. It always has been, but more sensor devices, larger friend networks, and the upward spiral of tweets have lead to an N-squared explosion of bytes that need to be stuffed somewhere.

Apps are becoming user interfaces for shuffling data from its spot at rest to your IOS or other mobile device so you can find the exact beer that fits your mood and its location on tap. These apps rely on complex geo or social designs. They need to scale and be hyper-fast. They need to have easy-to-use APIs. They need to NOT break.

These new complex data types, core API constructs, and scalability requirements have lead to a whole new category of datastores. You might have heard of NOSQL? For the most part, datastores-as-a-service in the cloud have not yet seen as widespread adoption as the applications-as-a-service model. For good reason, as running a datastore is a lot harder than hosting an application stack.

There are also the thorny issues of data-at-rest and trusting a hosting provider with your data—but that’s changing, too. Rackspace has MySQL services, Redis to Go and ObjectRocket (our company). We see Amazon’s offering in RDS and Dynamo. There are some smaller niche players like Cloudant and Clustrix as well.

Unless a company is the size of Facebook or Apple, there is real value in using the datastore-as-a-service approach. For one thing, developers get a lot of critical functionality instantly. They can focus on core API interactions, and forget the rest.

Developers as DBAs and dedicated DBAs seem antiquated in this model. Developers don’t have to worry about things like defragmentation, index rebuilds, and data file space, let alone infrastructure components like disk, RAID, Ubuntu kernel versions, etc. Whole classes of problems are transitioned to the provider.

This is all great, but the provider must build your trust and keep it. After all, this is your data we’re talking about. It’s game over to have it compromised, corrupted, or outright lost. A successful provider will ensure the product offering makes good on the promise of a seamless developer experience, all the while embracing popular open source APIs.

To use a phrase from the telco era—it’s dial tone. It will just work.

So perhaps the role of the DBA isn’t necessarily dead, it’s just moved to its new home at the datastore-as-a-service provider. The successful DBA will understand that this new world means handling petabytes of data and billions of operations on thousands of logical databases. They will cope with less mature database technologies in increasingly difficult workload environments. They will automate or die.

Long live the DBA.