How augmented reality will change the way we live

Consider a future device for individual use, which is a sort of mechanized private file and library…A memex is a device in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility.

Vannevar Bush foreshadowed the impact that such technology would have on our contemporary lives. By doing so, Bush inspired critical aspects of current online tools, including the hyperlink and the World Wide Web. Bush’s vision directly influenced researchers to create digital technology we nowadays consider commonplace. Today’s equivalent of Bush’s breakthrough is just as radical, except it’s far beyond the conceptual stage and is already in extensive development. This technology is termed “Augmented Reality” (AR). AR has the potential to act as a harbinger of future hi-tech transformations whilst irrevocably altering the basic nature of everyday life.

Just what the heck is augmented reality?

If you have even a fleeting familiarity with sci-fi pop culture, you’ve probably been exposed to the concept of AR via two of the most hackneyed references in cinematic history. The first comes from the movie Minority Report, where imagery is nonchalantly flung onto translucent screens via gloved gestures, and personalised advertorials bombard consumers.

This depiction echoes the 1990s version of AR – “Virtual Reality” – complete with a light-enhanced version of the data glove (though not the standard clunky headset). Virtual Reality is widely considered a precursor to AR, in that it engineers environments or spaces devoid of anchors to the physical world. Users participate in these simulated spaces with the intent of becoming completely absorbed in a Virtual Reality scenario, minus distractions that would pull users out of such worlds.

Virtual Reality is described as tech that encourages users to engage in completely immersive computer-generated arenas (often including Mixed Reality setups), rather than AR experiences which present digital enhancement through overlaps with – or over – the physical world. Confusingly, both AR and Virtual Reality share key elements that allow users to experience enhanced interactions through digital and online input, and often the terms are used interchangeably: with the increasing advancements of gesture-based interfaces (think the Kinect), distinctions between Virtual Reality and AR are becoming increasingly irrelevant.

The second film-based reference that’s inevitably trotted out when describing AR is from the movie Terminator, where a character is embedded with a HUD (Heads-Up Display) unit that continually scrolls updated data through a visual overlay. To continue this slightly-abused sci-fi referencing theme in regards to AR, we may as well throw in William Gibson’s character of Molly Millions, with her “Mirrorshade” inlays giving her enhanced vision similar to the point-of-view display used in the Terminator.  Although HUD units are far from new technology – having originated in the military – they are also being put to good use in existing AR applications. Pioneer has designed a car navigation system that merges AR with a HUD, combining projected mapping and navigational data on a plastic sleeve mounted before the driver-side windshield. Such an application of HUD-based AR has multiple uses, especially in regards to practical learning.

Although pop cultural references may make AR look somehow effortless, existing AR has sprung instead from years of dedicated research and development. Interestingly, even today’s top AR businesses have been inspired by great works of fiction: Maarten Lens-FitzGerald (VP of Marketing at Layar) describes their version of AR as:

Inspired by the 2006 Sci-Fi novel Rainbows End by Vernor Vinge…the team began exploring the possibilities offered by content layered on top of reality, and the concept for Layar was born…Layar wants to connect print with the online world. In an age where online consumption is skyrocketing, print publications must find a way to communicate with their readers in new and interactive ways. Layar merges the online and offline worlds, bringing new levels of engagement to a previously one-way conversation.

Initial rumblings for the formation of Layar began in 2007, when Lens-FitzGerald, Raimo van der Klein and Claire Boonstra (all part of the founding team of Mobile Monday Amsterdam) decided to formalise their vision for the future of AR mobile tech.  Almost two years later, on June 16^th 2009, the team released their first official Layar application.

Subsequent products developed by the Layar team proved enormously popular. Lens-FitzGerald claims that over 23 million people have downloaded Layar – including 1 million active users per month – with end-users, developers, publishers, and device manufacturers all learning AR through products conceived by the company.

Although Layar is funded to the tune of US$15.6 million in venture funding (investors include the likes of Intel Capital, Prime Ventures and Sunstone Capital), they recently introduced new pricing structures for developers and users alike. Their pricing model is freemium based, with users being able to generate “free” – though ad-supported – augmented pages, or pay a once-off fee for an ad-scrubbed version which provides additional analytics. According to Lens-FitzGerald, Layar is a stable, versatile product that provides an easy way for publishers to create augmented content, with a simple and straightforward learning curve – no visual additions (such as QR Codes) are needed.

Lens-FitzGerald also stresses the importance of context in relation to AR formation, especially when considering products that utilise print-based material to foster engagement. He stresses that with standard apps, a user starts a game or interactive experience and stays directly in that app, whereas with AR based around interactive print, it’s the physical properties of the actual source material that invite users to weld their phones or devices as portals to “where the magic happens”.

Perhaps the easiest way to explain the process of actual (as opposed to fictionalised) AR is to refer to another of the most recognisable names in the AR marketplace: Metaio. Trak Lord, Metaio’s Director of Marketing and Media Relations, describes AR from a user’s perspective:

For a given [AR] experience we try to make it very simple for the user, even as an entry point. Generally the user just has to take an initial action – download a free app or go to a website or a physical location – at which point they interact with a camera (mobile, webcam, in-store kiosk) and something in the real world. Our technology visualizes something, whether its location-based data, a 3-D model, or digital animations, directly overlaid in real-life.

Metaio formed as a company in 2003, but has development roots – including computer vision research – going all the way back to 1999. Thomas Alt (CEO) and Peter Meier (CTO) founded Metaio as a result of a grant provided by Volkswagen, which permitted them to Bootstrap the company.  Just as Layar concentrates on print immersive AR, Metaio initially targeted its products for engineering and manufacturing firms, but has since spread its AR focus wider.

Lord goes on to mention the mechanics of Metaio’s AR in Junaio, the free mobile app that acts both as content vehicle and development platform. 1.7 million people have downloaded Junaio, with approximately 5000 developers using it to manufacture augmented content. This free app functions as an entrance point for users and developers alike.

http://www.youtube.com/watch?v=tkcHs_LIGvo&

In terms of funding, Lord says their company is one of the rare (and potentially only) privately held and independently funded AR businesses. Metaio has a fully-fledged Research and Development division in control of patented computer vision algorithms. These algorithms are applied to areas covering visual search, image recognition and 3D content overlays. Like Layar, Metaio doesn’t use any markers in their AR tech: instead, Lord states they strive to create a seamless and unbroken interaction for users. Metaio works “…very hard to ensure the digital is an overall natural experience”.

In a nutshell, AR is a way of amplifying actual/physical reality to enhance information – enhancement created through the provision of additional contexts and content. Ironically, it’s often referred to as a “disruptive” technology, as it overlays or modifies input – hopefully with the result of engaging, rather than alienating, users. AR offers methods of data enrichment through direct superimposition onto the physical world. As such, it presents an interactive framework that has the potential to constructively affect all aspects of routine life.

AR’s positives: Benefits and booms

The potential for AR to impact all elements of our lives is massive: from education to gaming to manufacturing, the world seems poised on the brink of substantial AR adoption – with all associated benefits. Indeed, a recent study by Semico Research predicted that by the end of 2016, revenue produced by the AR Industry will total more than $600 billion. This study also determined that in 2014, approximately 864 million mobile phones will be AR-ready, and in excess of 100 million vehicles will come equipped with AR tech.

According to Brian Selzer from Ogmento, AR will not only change the way we live, but will actively shape how our lives progress: “There is no question that AR is going to have a profound impact..over the next decade. ” Ogmento’s primary emphasis is constructing AR experiences for gaming environments: ”

Ogmento was founded in 2010 by a group of video game industry veterans and computer vision experts to create mobile, location-based, augmented reality games.  Our aim was to create “real world” games, where the world itself becomes the game board… we come from the video game industry and focus predominantly on creating amazing user experiences. To do that successfully with AR requires a deep understanding and tight control of the underlying computer vision technology. You simply can’t do what we do by using a third party SDK or some other black box.  By developing our own computer vision technology in house, we have full control over every aspect of the final user experience.

Ogmento has been concentrating on experimenting with game concepts and building games like NBA King of the Court (a sequel is out soon) and cheeky uses for AR like this X-Ray vision tank (displayed on unsuspecting Augmented Reality Event 2012 Panelists):

Although this instance of SLAM (Simultaneous Localisation and Mapping) falls heavily into the gimmick category, it does suggest potential uses that far outweigh the usefulness of novelty gags. Students engaged in learning about history or archaeology could manifest specific AR tools and “unearth” artefacts, or directly engage with extinct flora and fauna through high-end AR simulations (already partially realised at the Royal Ontario Museum, Toronto):

Another such application of SLAM may involve visualisation techniques for patients undergoing invasive surgery. In future versions of SLAM tech, mapping and visualising the trajectory of a patient’s disease could act as a desensitisation tool. A patient’s specific awareness of exact surgical procedures could result in a decrease of anxiety levels and an increased sense of control.

Other medical implementations of AR could include further development of experimental apps like DanKam’s Colourblind Fix for correcting colour-blindness, or capitalisation on the growing trend of gamification of health through AR extensions for apps like Jumiya. Additional uses of medically-oriented AR could come via field of self-diagnosis: Singularity University’s Graduate program has produced a prototype called Med Sensation which integrates ultrasound through an AR Glove Tricorder. This type of futuristic tech could be extremely helpful when diagnosing conditions via physical exams.

Other areas that will undoubtedly be affected by future AR use include:

• Tourism – where geolocative media provides navigational/historically-relevant data.
  
• Publishing – enhancing print media through targeted AR (like in IKEAS’s recently released catalogue) generates boundless educational value for school textbook, manuals, and How-To books.
• The Military – uses include unmanned drones with complex HUD systems, and army trainees directly interacting with simulated data overlaid on the physical.
• Manufacturing – where technicians will conveniently  follow AR manuals, making repair and construction vastly simpler and potentially cheaper:

• The Arts – creatives such as Mark Skwarek, Tamiko Thiel and Jeremy Hight are all trailblazers in terms of using AR artistically. Hight has been producing AR since 2002: 

My 2002 work “34 north 118 west” runs on a map in real time with augmentation points in sequence…the project collides past and present. I’ve published many essays and book chapters on AR and ways to make it malleable, layered, tied to rich archives and in future designs for interface. I’m currently developing a system of intuitive AR, where AR objects react to each other in real time and the observer learns of how the interactions will work or be problematic (main applications will be architecture, geology and scientific visualizations: the visualization will teach as you use it).

• Retail – AR shopping will result in sophisticated immersive experiences, with interfaces consisting of context-aware surfaces and applications:

More commonplace retail applications will include AR improvements concerning product information, especially during an actual shopping experience. Blippar, which is seed funded by Qualcomm Ventures, is one such company producing “blipps”, or AR that provides additional data layers over physical products. These blipps provides consumers with recipes and cooking instructions when examining products in a supermarket.

Blippar initially formed in the winter of 2010 when Ambarish Mitra (current CEO and co-founder) was playing with £20 bank note while joking with Omar Tayeb (co-founder and CTO ) about rendering the image of the Queen in AR 3D. 3 weeks later Tayeb succeeding in making the queen talk on the £20, using an early version of Blippar. Mitra continues:

We had a eureka moment realising if we can make a bank note interactive, we can make the whole world of 2d and 3d surfaces interactive. In summer of 2011 we made Blippar…along with co-founders Jessica Butcher and Steve Spencer.

AR’s downsides: Dystopias and busts

Despite the fact that AR is a burgeoning marketplace that’s capable of great growth and expansive scope, there’s been heavy industry criticism concerning the sluggishness of movement beyond a novelty phase into more robust, granular, less fragmented and more functional territory. Current hype likens the state of AR to a potential “Goldrush phase”, with a possible bubble/bust cycle impending.

Those not convinced of the positives of the AR industry view the tech as nothing more than gimmick-based platforms which are capitalising on entrepreneurial driven fads. They view AR as transitory, as being driven by slick marketeers intent on creating buzzworthy hype. These marketing teams are accused of promoting one-off novelty products in a business sector rife with software duplication and artificially inflated credibility channeled through over-engineered conferences.

Critics also stress the difficulty of establishing production standards in a fragmented industry which already seems saturated – just some of companies plying their AR wares include Layar, Wikitude, Microsoft, Seac02, Ogmento, Metaio, Aurasma, Total Immersion, Sony, Blippar, Google, and Vuforia/Qualcomm.  Blippar’s Mitra equates the AR space to a

…formula 1 grid where all the 20 cars on the grid look the same from outside. But when you actual drive them, the guys on the front with superior engineering perform significantly better.

Other criticisms levelled at AR as both an industry and a technology centre around more sinister forecasts rather than clumsy market machinations. In current forms, AR exists primarily in device-based gadgetry, such as mobile phone and tablets. Future trends also lean towards AR motion-based interfaces via wearable hardware: think Google Glass or Disney’s tactile modification system (called REVEL), or Headset units like the Oculus Rift which is garnering high praise in crowdsourcing circles. These types of AR can easily provoke a sense of disquiet with even the most hardcore of fans, leading to dystopian views such as:

• Enhanced surveillance via AR Monitoring – examples could be Panopticon-style monitoring systems for jails or mental health facilities, where users not only undergo camera surveillance, but also have their senses controlled and altered through AR tech.
• Disputed enforcement of business agendas – witnessed recently when the one of the inventors of bioimplant-based AR, Steve Mann (who pioneered similar technology to Google Glass), was forcibly thrown out of a French McDonald’s store over concerns regarding image logging via his embedded AR tech.
• Litigation nightmares arising from AR companies fighting over valuable locative sectors, with intellectual property rights causing additional problems for AR market fragmentation. For instance, if a user deliberately creates an AR avatar identical to another “real” person, are there relevant copyright or trademark issues at stake? If a business proclaims exclusivity of AR-derived output that’s been superimposed over traditional media coverage (of the Olympic Opening Ceremony, for instance) then who owns the rights regarding AR delivery into that space? Is there such a concept as AR companies claiming exclusivity – a type of locative “colonisation” – of popular geographic areas? What about questions of AR fair use?
• Lack of protected privacy/opt-out options (how do we switch off the unswitchable?). Do we really want businesses to have the ability to render our actual gestures patentable? Could our very senses be commandeered for content delivery where we are served up unsolicited advertisements or subpoenas via AR?

• AR devices shifting already dangerously busy headspaces towards data overload:

• Dangers of fractured attention – such as accidents caused through overt reliance on AR car navigation while the tech is still in its infancy, or spatial accidents caused by interactions like these:

• Medical complications – hazards of repetitive muscle injuries from repeated use of gesture controls and/or sensory strain.
• Replacing person-based interactions with AR-derived “people”, with similar problems arising as when companion robots for the elderly are in use. These AR avatars may create potential psychological dislocation, or alienation from actual care givers opposed to such artificially-induced entities.
• Unscrupulous individuals determined to use AR for the removal of personal liberty and associated questionable, if not downright nefarious, purposes:

Potential augmented futures

Just as the concept of the Memex eerily foreshadowed commonplace technology, AR offers a similarly historically-relevant leap towards the revolutionary. And although the Memex isn’t a direct conceptual throwback of AR, it is an example of a concept that has shaped the very building blocks of how digital devices used for communication, data interaction and information enhancement have changed the fabric of our day-to-day lives.  Augmented Reality – as both a concept and an emergent technological field – has the potential to impact our future just as much as the Memex. Let’s hope AR Industry Leaders act responsibly when recognising, and actualising, the capabilities of such game-changing tech.