Save over 40% when you secure your tickets today to TNW Conference 💥 Prices will increase on November 22 →

This article was published on April 27, 2022

Open-sourcing Twitter’s algorithms is more complex than Elon Musk implies

The move isn't a transparency panacea — but it's a start


Open-sourcing Twitter’s algorithms is more complex than Elon Musk implies

Of all the grand ideas that Elon Musk has for Twitter, the one that he’s pitched most fervently is making the platform’s algorithms open source.

The Tesla tycoon proposed the plan before his buyout bid was disclosed, reiterated it the day his offer was revealed, and pitched it once again after Twitter agreed to the deal.

Musk outlined his proposal at a TED conference on April 14:

It’s just really important that people have both the reality and the perception that they’re able to speak freely within the bounds of the law. So one of the things that I believe Twitter should do is open-source the algorithm.

Musk argued that disclosing what amplifies or downranks tweets would reduce the risk of “behind the scenes manipulation.”

The approach has won support from some transparency advocates and critics of Twitter’s content moderation. They argue that the move will reveal how Twitter determines what you see on your timeline — and what you don’t.

“It has the potential to turn Twitter into a truly trusted platform, where users would understand why certain tweets show up on top of the list, and all concerns about behind-the-scenes secrecy or bias would be removed,” said Marc Linster, CTO at open source database firm EDB.

“These concerns have been rampant with Google and Facebook. This open-sourcing move could be game-changing for social media overall.”

Skeptics, however, have questioned the plan’s feasibility. They note that Twitter is comprised of various feeds, from the Trending section to your Home timeline, each of which is controlled by a complex mix of recommendation systems and human decisions.

These processes produce results that even their developers don’t fully comprehend. Some of them reportedly mocked Musk by adding a (now-removed) public repository on the company’s GitHub platform — with zero code.

Another issue is that algorithms alone offer limited insights.

There are various other factors behind a tweet’s ranking. They include content that enters the platform, each user’s profile, the algorithms’ training data, moderation rules, and the code that trained the models.

These constitute an enormous pool of data, which would be tough to trawl through and costly to disseminate.

“You can’t simply open-source an ML [machine learning] model like it’s some bubble sort implementation,” said Steve Teixeira, a Twitter vice president of product. 

 

Further complexities arise from the mutability of the system.

“Typically, recommender models get re-trained pretty often and will keep changing over time,” Bindu Reddy, CEO and co-founder of Abacus.AI, an artificial intelligence startup, told TNW.

“While it is possible to release all trained models as well on an ongoing basis, it won’t be very useful either unless you understand exactly what inputs and outputs go into the model for predictions.”

There are also potential dangers of the open-source proposal.

The information could be copied by competitors, provide a tempting target for cybercriminals, and violate user privacy. It could also hinder another of Musk’s ambitions: “defeat the spambots.”

On the other hand, open source offers new opportunities to find vulnerabilities and flaws.

 

Reddy is optimistic about the potential benefits. She argues that open-sourcing the ranking algorithm will be useful for research and evaluating any biases.

She also expects to find further insights from the infrastructure components that influence what’s flagged and filtered in feeds.

“Open sourcing those algorithms — and more importantly, those models — will go a long way in terms of transparency,” she said.

Another prominent proponent of the approach is Twitter cofounder Jack Dorsey.

The company’s ex-CEO has suggested letting users choose which — if any — algorithm they use.

Dorsey envisions creating an open marketplace of algorithms.

Users would pick the one that best serves their desires, from prioritizing nuanced conversations to surfacing a constant stream of thirst traps.

It sounds potentially idyllic — particularly if it can stop my feed from constantly showing obnoxious tweets by Elon Musk.

Get the TNW newsletter

Get the most important tech news in your inbox each week.

Also tagged with