Data scientist DJ Patil on why big data must have a purpose

DJ Patil, Data Scientist in residence at Greylock Partners is planning his on-stage appearance at LeWeb2012 London. He says that he is going to do some ‘live science’. Which brings to mind explosive visions of crazy-haired men in white coats.

Patil himself is affable, chatty, friendly and does not look as though he’s about to set any test tubes on fire. Maybe.

We’ve looked at big data and the work of data scientists in the past. Hilary Mason, chief scientist at Bitly told us at The Next Web conference that data science is just a buzz-word. Indeed it seems that data for academic research only is not such a useful pursuit and Patil feels that his work should lead to practical applications.

“I focus on helping companies in our current portfolio which includes Pandora, Facebook, Groupon and LinkedIn. We look at ways of using that data, how to build products. I also work with some of our earlier stage companies like Instagram and others you’ll be hearing about soon, both on the consumer and enterprise side,” he says.

So what does that mean in practical terms and how does data mining sit with audiences today? “The biggest thing about data now is how do you take it and return it back to the user in ways that are acceptable for privacy and all the necessary things that make people feel comfortable with that product,” says Patil.

And with the increase and ease of current data gathering comes the requirement to be able to filter it for practical use. “Today working with data is easier, you don’t need something like Watson to do these things, you can use open source technology,” says Patil. “We have a tremendous amount of data vomit out there. What is really useful is if there are tools that can work with that data. To turn it from data to insight.”

Pinpoint balance of privacy and use

Though many of us are  now accustomed to choosing what data we make available and setting it to suit our comfort zone, Patil believes that companies should make a virtue of working with their data in sensitive ways.

“One of the most critical aspects of data is that companies need to be good stewards,” he says. “Systems like some of the Amazon services would not work without gathering data. No one is selling that data though, so people feel comfortable with it. The data needs to be returned to the user with added value.”

So if the data makes useful products and can direct the future moves of companies while the public is becoming used to contributing to a large pool of information; what exactly is meant when we say, ‘big data’?

“Big Data is an ambiguous term,” says Patil. “It sounds like a confusing thing. The way to think about it is, it’s just data. We have to move from data to action. It doesn’t matter how big or how valuable the data is, it needs to be turned into actions.”

So it seems there is a lot of merit in collecting data in sensitive ways, but not putting it to good use is a waste of effort, both for companies who gather it and the users who trust them with it.