For your eyes only: How the personalized Web is transforming our relationship with technology

Social media already adds a personal layer to our online experience, but next-generation Web services are tapping the power of the cloud and massive data sets to bring us a new level of personalization in recommendations and behavioral predictions.

Search plus your personal life

Unsurprisingly, Google is heavily invested in the move to personalization. It announced a significant step towards personalized search last week with a plan to begin sourcing information from Gmail, Calendar and Google+ for search results. This isn’t the first time that Google has taken its search engine personal, but the company is clearly trying to adjust our expectations for it.

For years, we’ve been conditioned to think of Google results as objective. The SEO industry was founded on the assumption of consistent search results. That’s not to say that marketing professionals can’t adjust to the new world of personalization, but it’s going to take a paradigm shift.

Ironically enough, if you want to understand the future of Google, you need look no further than Google Now. For those who are embedded in the constellation of Google’s Web services, Google Now has a near-magical ability to anticipate the information you’ll need throughout a day. With it, Google has gone from simply organizing the world’s information to harnessing it.

The promise of Google Now is that its automated card interface gets out of the way for users, but it is emerging at a time when we are also concerned that Google may know too much about us. In recent weeks, Google has faced criticism over its privacy policies for a quote in a legal defense and allegations that it reported a user to the police for searching for “pressure cooker bombs”. Both of those situations turned out to be overblown, but the severity of the public response is indicative of the level of fear that is circulating at the moment.

Google’s grand experiment with Google Now has come to a head with the Moto X, which is designed to have deep personalization on both the external and internal levels. Writing about the device, Robert Scoble recently described the “freaky line” where an additional cost to privacy is required in order to provide deeper contextual and personalized features.

Scoble says he’s “in 100%” and living over the line. I, for one, am not quite ready to welcome our new erosion of privacy overlords.

Civilization of this content

Beyond search, the content space is seeing plenty of interest in personalized recommendations. Gravity is working with publishers and advertisers to build interest graphs of its readers. Its technology runs semantic analysis on the articles that users read across sites on its network in order to recommend content that fits with your interests.

Here’s an example of a sample Gravity interest graph for a “32 year old male from Los Angeles [who] likes surfing, travel and music” after he read several articles:

The visualization of an individual’s interests is a fascinating thing, but it’s also disconcerting.  Thankfully, Gravity CEO Amit Kapur reassured me in a recent interview that the company’s data, which is collected through cookies, is anonymized. Users can also opt out of the service if they choose.

Outbrain also provides personalized content recommendations, basing its suggestions on a combination of factors, such as popularity, contextual, behavioral and personal. The company recently became an unfortunate example for the importance of Web safety after it suffered a security breach at the hands of the Syrian Electronic Army. This breach exposed several high-profile clients, including The Washington Post, Time and CNN.

Big data

Providing deep personalization for users hasn’t been a particularly easy problem to solve. Sure, Amazon has been able to recommend products based on other shoppers’ browsing and buying patterns for years now, but newer recommendation engines are combining big data, cloud computing and neuroscience to achieve their goals.

Nara, for instance, has tapped PhDs from MIT to build its personal discovery engine.  Currently, Nara serves up restaurant recommendations for users through a neural network. It creates a “Digital DNA” for diners that can then be matched up with others to help groups pick a place to eat.

Expect Labs’ Mindmeld is another instance of high-level computing being applied to personalization. The iPad app, which showcases the company’ “anticipatory computing engine”,  hasn’t even been released to the public yet, but it’s attracted the attention of numerous industry leaders, including Samsung, Intel, Telefonica, Liberty Global and Google Ventures. When it arrives, Mindmeld will act as a VOIP client that actively listens to the conversation and then provides recommendations based on the predicted behavior and needs of its users.

Expect Labs founder Tim Tuttle has a PhD from MIT’s Artificial Intelligence lab. In his view, the AI field has shifted in recent years to the point where advances in machine-learning once thought impossible to achieve in this lifetime are now within reach.

“The most important thing that changed was for the first time, we now have these data sets that are Internet scale and capture nearly every concept, term, important topic that exists on the Web. These data sets can be searched in real-time using large computing clusters and it’s not prohibitively expensive to do that,” he told The Next Web. “The fact that we can now have searchable data sets that know the names of every person, place, company, politician, product, food item, this ends up being the foundation that these systems are built on.”

The glut of information has also been accompanied by a change in AI methodology. Previous systems utilized complex algorithms on small data sets, but the newer model employs simpler statistical methods on massive data sets. The results have been substantial, especially in speech recognition, natural language processing and imaging.

When I asked Nara founder Tom Copeman why the timing is right for the onset of advanced personalization, he cited three broad factors. First, he highlighted how the need for personalized search didn’t arise until recently. As the amount of information available on the Internet continues to rise, recommendation services will reduce the pain of trying to sift through the noise. Syntertainment’s Will Wright recently referred to the process of intelligently filtering out information through technology as “decimated reality“. While its opposite, augmented reality, attempts to add more information to reality, decimated reality seeks to pare it down.

Copeman also pointed to the scalability of the cloud as another factor that makes personalization possible. Thirdly, he mentioned the influx of online reviews and opinions.

According to Copeman, the big data approach to recommendations represents a “quantum leap” in quality. That’s not very easy to measure with simple restaurant suggestions, but Nara’s technology is at least good enough to have earned the respect of wireless giant SingTel. Nara’s enterprise division will provide the brainpower for SingTel to offer personalized recommendations to its 450 million or so customers.

“We are investing in emerging technologies and Nara’s cutting-edge platform will help us lead and create serendipitous discoveries for our customers across the globe. Together, we share a commitment to leading web 3.0 through personalization,” Loo Cheng Chuan, Head (Local Life) of SingTel’s Digital Life division, said in a statement announcing a partnership between the two.

Data mine

The Internet hosts endless amounts of public data, but the real gold rush for personalization comes when users actively volunteer their preferences. That’s where a company like Ranker comes in. The list-making site has built up a veritable empire of users expressing their opinions and is gearing up to put that data to use for recommendations.

Since users vote on everything from their favorite cereal brands to TV shows, Ranker can make valuable connections between products and services. CEO Clark Benson said in a recent interview that the company is adding elements to its API that should allow for one-to-one granularity for recommendations.

Pandora was one of the first to begin categorizing data and amassing users’ opinions. Its music recommendation service was released in 2005, making it one of the earliest intelligent personalization offerings on the Web. Based on the company’s own Music Genome Project, Pandora categorizes songs with up to 450 characteristics in an effort to understand why you like a certain type of music.

MIT Media Lab spin-off The Echo Nest also got its start in 2005 and has since grown to provide its music data service to some of Pandora’s biggest rivals. The Echo Nest has worked with Spotify, iHeartRadio, eMusic and Rdio. Most recently, Rdio rolled out new YouFM and FriendFM personalized stations based on The Echo Nest’s database. To generate its recommendations, The Echo Nest crawls the Web using semantic analysis to track the social and cultural dimensions of songs and artists.

I, Spy

If recent NSA leaks are to be believed, governments have assembled one of the most powerful personalization services. Only they’re using it to recommend extended airport detentions and monitor most of our Internet activity.

User-specific data is powerful, so it needs to be protected. The same information that allows wonderful experiences like Google Now can also be used for surveillance, bullying and identity theft. A service that collects and analyzes all the content we read and create in order to figure out who we are can as easily be surveillance as it can be a consumer-friendly Web tool.

The personalized Web is here, but the move to Web 3.0 must also be accompanied by innovations in online privacy. Otherwise, the casualties – Groklaw, Silent Circle, Lavabit – that have started to pile up will continue. President Obama has taken a few steps towards greater transparency, but more still needs to be done.

When I ask Web personalization companies about the role privacy plays in their business, all of them have expressed that it’s foundational. Expect Labs’ Tim Tuttle put it this way: “This is the agreement that we make with customers: what you share with us we’ll use in a way that helps them.”

I’m unequivocally excited about this new generation of personalized connectivity, but I’m going to need some convincing that it will help more than it will hurt.

Header image credit: iStockphoto