Personal credentials, proprietary source code, confidential employee contracts, even human genome sequences – these are just a few examples of sensitive data picked up by a security firm looking into a feature of Amazon’s Web Services.
“We were very surprised with both the amount and the sensitivity of the data that was shared publicly,” NVTEH CEO Ivo Ugrina told us.
Since launching AWS a little over ten years ago, Amazon has devoted commendable effort to provide quality training for businesses seeking to implement its cloud computing solutions into their already existing IT infrastructure.
But despite these attempts, it is still threateningly easy for uninformed users to make costly mistakes – mistakes that could not only set a company back millions of dollars, but also make invaluable data vulnerable to breaches.
As a consultant for cloud solutions, Croatian firm NVTEH is no stranger to the shocking magnitude of negligence behind internal data protection measures that affect both fledgling startups and seasoned Fortune 500 listers.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
“The issue of negligence is a serious problem that can lead to considerable losses,” said Ugrina.
“Using cloud services inherently means storing data on computers you don’t own. Mistakes made in the cloud are therefore more likely to become public and the impact can range from embarrassment to loss of customer data,” Ugrina continued.
One thing that particularly bugged the Croatian cloud expert was the sloppy manner in which certain companies relied on Elastic Block Store (EBS) snapshots to share sensitive information between employees and with partners.
For those unfamiliar with the matter, EBS provides highly available block level storage volumes for use with Elastic Compute Cloud (EC2) instances.
In a way, the relation between EC2 and EBS is akin to that between a server and a hard disks: Think of EC2 instances as virtual servers and EBS volumes as the virtual hard disks that power those servers.
Much like hard disks, EBS volumes have their own backup system, more commonly known within AWS as EBS snapshots. Snapshots are point-in-time backups of EBS volumes that practically function as copies of the data stored on these volumes.
“If you want to backup the data from your EC2 instance, then you probably want to create EBS snapshots of the EBS volumes attached to the instance,” NVTEH research lead Neven Vucinic explains. “Users with appropriate access can afterwards copy your snapshot and create their own EBS volumes based on your snapshot, while your original snapshot remains unaffected.”
In other words, if you get your hands on a EBS snapshot, you have access to all the data stored on that ‘hard disk.’
What makes EBS snapshots especially convenient for sharing data is how easy and fast they are to recover. This is why Amazon has included the functionality to share snapshots among users – and even the ability to do so publicly.
As NVTEH points out, while Amazon has good reasons to include the option to create easily accessible public snapshots, the internet giant appears to be well-aware of the dangers involved in this move. In fact, the company explicitly addresses this risk in its documentation:
When you share a snapshot (whether by sharing it with another AWS account or making it public to all), you are giving others access to all the data on the snapshot. Share snapshots only with people with whom you want to share all your snapshot data.
It seems not all AWS users take the cue though. “Unfortunately, there are dozens of publicly created EBS snapshots daily that contain some kind of data that should not be publicly accessible,” Ugrina tells us.
In an effort to warn users against the hazards involved in carelessly sharing unencrypted EBS snapshots publicly, the consulting firm came to the conclusion that the best way to raise awareness was to deliberately play the villain.
In preparation for this stint, NVTEH developed an AI-powered tracking system that would automatically monitor AWS and send notifications anytime it detects someone has shared a public EBS snapshot.
It additionally fine-tuned the algorithm to probe specifically for snapshots packed with potentially sensitive data.
As the experts expected, it didn’t take long before the notifications started rolling in.
Following a cumbersome vetting cycle, the company was startled by the sheer volume of sensitive information it was able to recover from its ethical ‘fishing’ operation.
Among other things, the NVTEH researchers were able to extract data containing detailed patient files (including genome sequences) owned by leading US universities, as well as AWS security credentials and API access keys frivolously shared by Fortune 100 companies.
The experts also came across tonnes of potentially damaging data like web server configurations, SSH keys and treasured source code for proprietary software.
To verify that this information was indeed meant to remain confidential, NVTEH traced the recovered EBS snapshots to the companies they belonged to and reached out to them to disclose the leaks.
While companies were quick to respond, most of them remained reserved about the possibility their data has indeed been breached… until Ugrina and Vucinic began to back up their claims with concrete evidence.
“You could see a rapid change in their facial expressions [at that moment],” the consultants remarked.
NVTEH says that although responses to its disclosure varied, it left more than a few companies in a state of absolutely dismay.
“We’ll have to call for a board of directors meeting to discuss it,” one company said, according to NVTEH, while another straight up said the leak “might ruin [them] completely.”
To thwart such disaster before it takes place, Ugrina believes that businesses and “users should be aware that confidential data may be compromised even if exposed for a blink of an eye.”
“Based on our experience, employees who believe that security and privacy are important are more likely to follow specific procedures,” he continued. “Employees should be made aware that data has become the lifeblood of many organizations and that exposed data can be used to hurt company’s brand.”
However, given the pace at which AWS and other cloud providers extend their line of services, keeping up with the latest trends in cloud computing is harder than it sounds.
Just last week, Amazon launched a training and certification portal specifically dedicated to luring technical talent to AWS. But while the initiative is certainly a step in the right direction, it will likely need more time until it starts producing qualified workforce at stable rates.
In the meanwhile, Ugrina advises companies to take matters into their own hands and set up their own cloud research and development teams.
One important thing to stress, is that sharing unencrypted EBS snapshots is merely one of the many bad habits AWS users have.
Just days ago, popular access management service OneLogin disclosed an threat actor had obtained a set of AWS keys to penetrate the AWS API to “perform reconnaissance.” The company did not reveal precisely how the attack took place, but it did caution it is possible the attacker could have also obtained the ability to decrypt the stolen data.
Curiously, the breach occurred on the same day NVTEH published its own post about the dangers of improper standard operating procedures – though there is nothing to suggest the two are related.
In another recent mishap, cybersecurity firm UpGuard found a “publicly exposed file repository containing highly sensitive US military data” stored in an unprotected AWS S3 bucket.
These are simply two isolated incidents though. What is truly scary is that, if businesses continue to blissfully ignore the importance of setting up appropriate guidelines for handling data in the cloud, the trend is likely here to stay – and the breaches bound to multiply.
Get the TNW newsletter
Get the most important tech news in your inbox each week.