Martin SFP BryantFounder
Martin Bryant is founder of Big Revolution, where he helps tech companies refine their proposition and positioning, and develops high-qualit Martin Bryant is founder of Big Revolution, where he helps tech companies refine their proposition and positioning, and develops high-quality, compelling content for them. He previously served in several roles at TNW, including Editor-in-Chief. He left the company in April 2016 for pastures new.
Update: Affected websites are slowly coming back online, click here for more details.
The popularity of Amazon’s cheap, easily scalable hosting is showing its downside right now, with a number of popular websites and services throwing up errors or being down completely.
Foursquare, Quora, Reddit, Moby and Hootsuite are among those affected by technical troubles on Amazon’s servers. The company’s status dashboard currently shows problems with the company’s Elastic Compute Cloud and Relational Database Service operations, based in North Virginia, with connectivity issues confirmed.
We can confirm connectivity errors impacting EC2 instances and increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region. Increased error rates are affecting EBS CreateVolume API calls. We continue to work towards resolution
Quora pulls no punches on its error page, stating: “We’d point fingers, but we wouldn’t be where we are today without EC2.”
We’d ask you to leave a comment on this story but our commenting system, Livefyre, appears to have fallen victim to the problem too.
Update: In an update on its dashboard, Amazon says that “Delayed EC2 instance launches and EBS API error rates are recovering. Were (sic) continuing to work towards full resolution.”
Update 2: Over seven hours into this saga, service is yet to be restored. Amazon’s latest update notes:
“We’d like to provide additional color on what were working on right now (please note that we always know more and understand issues better after we fully recover and dive deep into the post mortem).
A networking event early this morning triggered a large amount of re-mirroring of EBS volumes in US-EAST-1. This re-mirroring created a shortage of capacity in one of the US-EAST-1 Availability Zones, which impacted new EBS volume creation as well as the pace with which we could re-mirror and recover affected EBS volumes. Additionally, one of our internal control planes for EBS has become inundated such that it’s difficult to create new EBS volumes and EBS backed instances.
We are working as quickly as possible to add capacity to that one Availability Zone to speed up the re-mirroring, and working to restore the control plane issue. We’re starting to see progress on these efforts, but are not there yet. We will continue to provide updates when we have them.
Get the TNW newsletter
Get the most important tech news in your inbox each week.