Facebook has just revealed some details on an issue that occurred with some developer applications accidentally going offline. In the social networking company’s first post-mortem, it hoped to shed some light on what steps it was taking to prevent the occurrence.
The incident happened on August 13 and Facebook said that during routine monitoring to remove malicious apps from being used, its sweep instead inadvertantly targeted high-quality apps. Once this mistake was detected, Facebook stopped what it was doing and set about to repair the damage it caused.
Word first came out through a thread on Hacker News when a developer from Chute wondered why his startup’s applications were randomly disabled. A Facebook employee responded in the comments apologizing for the mistake, saying “We’re working on restoring the set of apps that were accidentally disabled. Many of them are back, some are still in process.”
Facebook’s developer advocate David Weekly chimed in as well, explaining:
We have systems that block spammy apps that are 99.9% of the time really incredibly sophisticated and get a ~0% false positive rate.
This is a case of the 0.1%. :( Folks here are scrambling to undo this. Very, very sorry. Things should now (17:47 PT) be all set.
But why did it take so long to restore? According to Facebook: “The process took longer than expected because of the number of apps affected and bus related to the restoration of app metadata.”
In order to prevent this from happening in the future, Facebook is implementing two new steps:
- We will create better tools to detect overly broad patterns and put in place better processes to verify that all apps matched are indeed malicious.
- We will address the bugs and bottlenecks that made the recovery process slower than expected.
It’s understandable that mistakes like this can happen, especially when you have a service as massive like Facebook and there are a plethora of applications that help businesses. The company needs to not only ensure access for all, but protect its users from malicious content and applications — unfortunately it appears that an overabundance of caution has led to a temporary denial of access for developers.
While it’s an inconvenience for developers, not only because they lose users, but also money in the process, it’s probably refreshing to receive a somewhat detailed explanation of what happened and hear what steps Facebook is doing to prevent further incidents like this from happening.
Photo credit: BRENDAN SMIALOWSKI/AFP/Getty Images