Meet Airlock: Facebook’s mobile testing framework

It’s taken a while, but Facebook has started to move in the direction of “mobile-best”. With popular mobile apps on iOS and Android, how exactly does the company know what works and what doesn’t? With millions of people using its apps, it’s hard to continually assess what changes and tweaks are needed.

Enter Airlock, Facebook’s mobile A/B testing framework. Developed after the social networking company rewrote its mobile apps to run natively on iOS and Android, this new process is said to enable better control over “when and how items are downloaded, cached, and freed.”

Facebook engineers Ari Grant and Kang Zhang explain in a blog post that the purpose of Airlock is to help the company accomplish three objectives: understand how new features perform on mobile, assess what fixes are needed to improve performance and reliability, and know what to change in the user interface to enhance engagement. More traditional testing processes apparently weren’t cutting it for Facebook so it chose to create its own (is anyone surprised?).

The great deal of Grant and Zhang’s post goes into detail about the process, but the most interesting part is that it all started with a simple experiment. It took the A/B binning system used for Facebook’s Web-stack and modified it to display certain updates on its app for two separate user groups. After this and one other experiment, circumstances led the company to scale Airlock to handle the entire app:

The experiment that drove the evolution of Airlock was a project we started with the intent of evolving and simplifying the navigation model within our apps. Over the course of a few months we tested making the left-hand drawer narrower with only icons, putting a tab-bar at the bottom of the screen with your timeline in it, combining friend requests and notifications into one tab in a tab-bar, and eventually landing on the tab-bar design that is now the user interface for the Facebook for iPhone app. We built a bunch of different versions of the UI that didn’t make the cut, but that’s the nature of testing.

So far, Airlock supports between 10 to 15 variations of an experiment and helped Facebook better understand what users are doing on the social network’s various apps.  It complements Facebook’s alpha and beta testing programs on Android, which the company announced last year.

Photo credit: THOMAS COEX/AFP/Getty Images