Foursquare, Quora, Reddit, Moby and Hootsuite are among those affected by technical troubles on Amazon’s servers. The company’s status dashboard currently shows problems with the company’s Elastic Compute Cloud and Relational Database Service operations, based in North Virginia, with connectivity issues confirmed.
We can confirm connectivity errors impacting EC2 instances and increased latencies impacting EBS volumes in multiple availability zones in the US-EAST-1 region. Increased error rates are affecting EBS CreateVolume API calls. We continue to work towards resolution
Quora pulls no punches on its error page, stating: “We’d point fingers, but we wouldn’t be where we are today without EC2.”
We’d ask you to leave a comment on this story but our commenting system, Livefyre, appears to have fallen victim to the problem too.
Update: In an update on its dashboard, Amazon says that “Delayed EC2 instance launches and EBS API error rates are recovering. Were (sic) continuing to work towards full resolution.”
Update 2: Over seven hours into this saga, service is yet to be restored. Amazon’s latest update notes:
“We’d like to provide additional color on what were working on right now (please note that we always know more and understand issues better after we fully recover and dive deep into the post mortem).
A networking event early this morning triggered a large amount of re-mirroring of EBS volumes in US-EAST-1. This re-mirroring created a shortage of capacity in one of the US-EAST-1 Availability Zones, which impacted new EBS volume creation as well as the pace with which we could re-mirror and recover affected EBS volumes. Additionally, one of our internal control planes for EBS has become inundated such that it’s difficult to create new EBS volumes and EBS backed instances.
We are working as quickly as possible to add capacity to that one Availability Zone to speed up the re-mirroring, and working to restore the control plane issue. We’re starting to see progress on these efforts, but are not there yet. We will continue to provide updates when we have them.