Bitly’s Hilary Mason on their new Dragoneye product, “The Bitly Bias” and coming to The Next Web Conference

Hilary Mason is a Keynote speaker at the upcoming The Next Web Conference

Hilary Mason is Bitly‘s Chief Scientist, a position she describes as “someone who blends, math, algorithms, and an understanding of human behavior with the ability to hack systems together to get answers to interesting human questions from data.”

On a rainy Friday morning at the Betaworks office, Mason tells me that Bitly began as an accident; it was a feature in another product that eventually failed. At its core, Bitly is a URL shortener service that gained so much popularity on Twitter and other social networking sites that some people think it’s just a part of the Internet, like .com or http, she tells me. Since Bitly’s inception, Mason and her 40-person team have built a simple business around Bitly that offers premium analytics. “But now that we’ve had two years to look at this massive amount of data and figure out what we want to do with it, we’re building the next stage of our business,” she says.

In the coming months, Bitly will release a few tremendously cool consumer and enterprise products built around its data. The product Mason is most excited about is called “Dragoneye”, which picks up phrases that are “bursting” disproportionately on the Internet. Mason gave me a Dragoneye demo on Friday, March 16th and unsurprisingly, the phrases “iPad” and “St. Patrick’s Day” were bursting in New York City.

While Bitly’s current Search product is only available to enterprise users, Bitly plans to release a new version around realtime links that will be available to all users. In relation to this Search product, Bitly will offer useful data feed products built around desired topics and locations. For example, if you want to know all of the celebrity gossip going on in a specific region or what people in New York City are tweeting about pizza, what’s trending from Etsy on Twitter or which American regions read Fox News vs. The Onion, Bitly has an API for you.

One of the more interesting things to note about Bitly’s data is that it collects engagement around a link, i.e. if people are actually clicking on it vs. just @mentioning it and sharing it on Twitter. For each link that passes through Bitly’s system, Mason and her team can construct a few dimensions around the content like what language it’s in, where the link is being shared and when.

I asked if and how Mason takes into account the fact that only certain, savvier Internet users are likely to use Bitly — she calls this “The Bitly Bias.” But contrary to my assumption, The Bitly Bias doesn’t mean that Bitly users are searching for space exploration research and recycling tips. In fact, Mason tells me that every data expert at Bitly goes through an emotional cycle as a result of the data. First, everyone is happy by how much data there is to be studied. Then, they become “super depressed” when they realize that most of the world is sharing salacious links about Kim Kardashian and Rebecca Black.

“Eventually, you come out the other side and you sit back with a bowl of popcorn. Really, it’s the greatest theater in the world; the most amazing piece of performance art you’ll ever see,” quips Bitly Data Scientist Mike Dewar.

While Bitly’s URL shortener competes with Tinyurl, its new consumer products put it in a realtime search competition with Google. “Except while Google is looking at older relationships between content and page rank, we’re more orthogonal,” explains Mason. “Our search is in a totally different direction than Google’s.”

She continues, “Our job at Bitly and in the data science group is to understand what the world is paying attention to and in real time. We’ve developed new angles on the data set, some are mature and some are a long way from maturity.” In essence: “At Bitly, we don’t just shorten links anymore,” she concludes.

Watch our short video interview below in which Mason tells us the weirdest thing she’s learned from looking at Bitly’s data, as well as what she’s planning to talk about at our upcoming conference in Amsterdam.

Speaking of which, we’re heading to The Next Web Conference next month! Will you be there?