Inside Shopify’s brave API gamble

A platform-as-a-service company is only as good as its APIs, and Shopify has dozens of them. These expose a specific part of Shopify’s product to third-party developers, who can then incorporate them into their own applications. And last week, at the annual Shopify Unite summit, the company announced it was moving away from traditional REST-based APIs to Facebook’s GraphQL technology.

While REST isn’t going away, much of the internal development interest will be focused on GraphQL, and the company is placing a great emphasis on ensuring its most popular external endpoints are available via GraphQL.

This isn’t the most perfect metaphor, but it’s a bit like going to work, and your manager announcing that instead of English, the lingua franca of the office was now Afrikaans or French. Sure, they’re still human-comprehendible languages. You could argue that they’re more beautiful or poetic than English. But still, they’re different, and come with an inherent learning curve.

Shopify’s decision to switch to GraphQL, therefore, is a radical one. I wanted to know why it took this step, so I flew to Toronto to meet with the company’s technical leadership.

What on earth is GraphQL?

Before we get into the meat of this piece, it’s probably useful to explain what GraphQL is, and how it differs from a traditional REST API. Helping me out is Avi Flombaum, Co-founder and Chief Innovation Officer at Flatiron School, a WeWork company.

“GraphQL is a standard for how applications can interact with each other over data,” Flombaum explained. “It’s a format for API requests. There have been many standards and formats for how applications should interact with each other over the years, SOAP, XMLRPC, and REST, to name a few, and now GraphQL is a new one.”

Expanding on that point is Jack Skinner, an Australian API consultant, who described GraphQL as a way to build HTTP APIs without the need for complex API design and development.

“At a very high level, GraphQL allows API providers like Shopify to make data available to developers using queries (to retrieve data) and mutations (to update it),” he explained.

This is particularly attractive in situations like Shopify’s Shopfront API ecosystem where developers have different data requirements for every unique store they design. The use of GraphQL allows Shopify to empower developers on their platform to make their own decisions about things like design and performance which are critical in highly competitive mobile retail environments.

Skinner hastened to add that there’s no such thing as a silver bullet with API design. “Despite its benefits, GraphQL itself presents a series of trade-offs,” he explained.

“It’s particularly useful when the ‘front-end’ of an application undergoes rapid iteration, or when there’s a wide ecosystem of applications and developers with different requirements on a particular data set. All in all, it does follow an industry trend of bring-your-own-frontend (BYOF) where platforms like Shopify and Contentful provide APIs and backend services, allowing developers to focus much more on the customer experience,” Skinner added

Coincidentally, on the day we’re publishing this, Honeypot.io published a 30-minute documentary about the history and development of GraphQL. We covered its previous short film about Ember.js, and would highly recommend you check this one out – especially if you’re a visual learner.

“Shopify is an API first company”

In a quiet corner of Toronto’s Beanfield Center, I sat down with Jean-Michel Lemieux, the Chief Technology Officer of Shopify. A wiry Quebecer with a Tom Cruise smile, Lemieux struck me as the type of CTO who feels more comfortable behind a terminal than in a boardroom.

I wanted to know why Shopify was taking such a radical step. The first thing Lemieux said to me was that Shopify was an “API first” company and has been for its entire lifespan.

“We spent 14 years building a company where we put APIs first. You know, back when [we started], we realized we couldn’t actually build everything ourselves,” he said.

Right, but that only underlines my main point: APIs are essential to Shopify. Why mess with something that works by replacing it with a new technology (GraphQL was first launched by Facebook in 2015, while REST APIs date back to 2000) that only a handful of developers properly understand?

According to Lemiuex, to answer that is based around the problems it faced during the middle of this decade, as it tried to scale its technical infrastructure to an ever-growing userbase.

“[In 2015] we were looking at redoing our mobile experience. We were looking at performance. We were looking at ease-of-use. And we really liked some of the things GraphQL brought to the table, particularly around performance and developer experience. You know, you get a graphical explorer with the language. It’s typed. The tooling is great,” explained Lemieux.

The initial push towards GraphQL came internally, from Shopify’s development team. That year, the company launched a couple of explorations with the tool and found it to be an adequate fit for its requirements. But it does have one major drawback: its origins.

As mentioned GraphQL was originally developed internally at Facebook. Given the social giant’s tarnished public image and repeated privacy missteps, were you wary of adopting its API technology, I asked Lemieux.

“To be frank, in 2015 I was a little concerned. I don’t think Facebook’s had a great track record in the open source community,” Lemieux said, undoubtedly referencing the licensing issues surrounding the React front-end framework.

But one thing that works in GraphQL’s favor is the fact that Facebook has – quite admirably – let go of the reins and handed them to the GraphQL Foundation. This nonprofit is dedicated to the preservation and the advancement of the technology, and is funded by a consortium of companies, with Shopify playing an important role.

Crucially, the development of GraphQL is left in the hands of the qualified technologists, and isn’t swayed by the commercial interests of any one party.

“Lee Byron, who was one of the co-creators, is a founding member of the GraphQL Foundation, which is a Linux Foundation managed group. The technical evolution of [GraphQL] is governed by Lee,” explained Lemieux

Turning a corner on REST

After lunch, I met up with Vanessa Lee, Director of Product at Shopify. Lee is unabashedly nerdy, and her eyes lit up when we started taking about APIs. I wanted to know exactly how the company plans to transition its users towards the new GraphQL endpoints.

“Nodes and edges aren’t fully natural when you’re switching from a table-based understanding,” admitted Lee.

As Lee explained, one of the best answers to this is to improve existing language libraries, in order to natively and intuitively support this relatively new paradigm.

“We’ve invested in updating all the libraries because we know that as a developer, you’re probably not making HTTP calls — like using the very base libraries. You’re probably have libraries that scaffold like above that, which you’re making method calls through,” she explained.

“One of the things that we did was we took a look at what are the most popular libraries that people are using across languages. So, I mentioned the Shopify API app gem in Ruby, but there was also the Node Shopify API. There’s also Python libraries and PHP libraries that we updated,” Lee said.

She added that the company had focused heavily on creating API documentation and training materials, in order to ease the transition. Also available is an online code editor, allowing Shopify’s community of developers to write and test queries against an online platform, in order to see how they work.

This is far less intimidating than testing against a live system and doesn’t involve the hassle of creating a local testbed, both of which are deeply attractive for developers.

The GraphQL Windfall

During Vanessa Lee’s keynote talk, she said that through GraphQL, Shopify has managed to reduce the number of total API requests made to its servers. In one common use-case, the company has gone from making 11 API requests to just one. During our interview, I probed her about this.

“If you want to render a page that says, like “here’s your order” and you want a little thumbnail for each item, that would require 11 API calls,” she said.

There are workarounds, of course, especially if you’re well versed in asynchronous coding, but Lee admits this isn’t particularly easy – or, for that matter, especially scalable. By moving to GraphQL, customers of a Shopify site should enjoy shorter load times. Developers will also find it easier to stay within their API limits, as they’re making fewer requests.

That being said, I can’t help but imagine Shopify will benefit massively from the switch to GraphQL. One of its inherent advantages is it allows developers to choose exactly what data they wish to extract from an endpoint. Compare that to a traditional REST API, which spits out a bit of JSON or XML, and forces developers to parse through it in order to find the information they actually care about.

This presumably means that Shopify will spend less on bandwidth, shuffling bits between its servers and its customers. And that’s before we factor in the fact that Shopify will be servicing fewer API requests overall.

Being the dogged and determined tech hack I am, I pressed Shopify for any figures they had on how this could reduce their bandwidth and computational burden, as well as if it foresaw any financial savings from the pivot towards GraphQL. Unfortunately, they couldn’t tell me anything concrete.

One thing is concrete though: GraphQL is a big deal, and it’s only getting bigger. Developers, if you fail to learn it, you do so at your own peril.