This article was published on November 6, 2017

How you can detect hackers in your encrypted traffic

How you can detect hackers in your encrypted traffic
Carl Herberger
Story by

Carl Herberger

Carl Herberger is vice president of security at Radware. With over two decades of experience in the cybersecurity space, he now leads Radwar Carl Herberger is vice president of security at Radware. With over two decades of experience in the cybersecurity space, he now leads Radware's global practice.

More than half of web traffic is now encrypted, according to the Electronic Frontier Foundation (EFF). That’s a big win for businesses and all of us, since it guards against eavesdropping and tampering with content as it moves from device to server and back again.

The move from http to https has been driven in large part by Google, which highlighted http sites as unsecure and made encryption a ranking factor for its search results, pushing more and more businesses to adopt it.

Of course this rise in encryption comes with one big, obvious downside. Hackers too now use encryption for their attacks, making them harder to spot amidst a stream of encrypted traffic.

Attacks that weaponize two common encryption protocols, Secure Sockets Layer (SSL) and Transport Layer Security (TLS), are on the rise. Some 39 percent of organizations experienced an SSL or TLS attack in 2016, but only a quarter said they were confident they could detect and mitigate them. Beyond this, recent research found that there were twice as many encrypted malware payloads in the first six months of 2017 than the whole of 2016.

Stopping them is easier said than done. Many businesses are left in the dark, struggling to distinguish good traffic from bad when it’s all encrypted. But there are steps organizations can take to avoid becoming the next victim.

What encrypted attacks look like

Identifying encrypted attack traffic within encrypted traffic flows is worse than searching for a needle in a haystack; it’s like searching for a piece of hay in a haystack.

Decrypting and re-encrypting SSL traffic to find an attack only adds requirements for processing the traffic, which can ultimately clog up the network and application infrastructures. Tools that can decrypt tend to rely on limiting the rate of request, which can drop legitimate traffic, effectively making the attack a success.

Here are some of the most common forms these encrypted attacks take:

  • Encrypted SSL floods — These attacks seek to exhaust system resources by forcing use of SSL handshakes for illegitimate encrypted traffic.
  • SSL renegotiation — By initiating a regular SSL handshake and then immediately requesting the renegotiation of the encryption key, this attack repeats this renegotiation request to exhaust all server resources.
  • HTTPS floods — Floods of encrypted HTTP traffic overwhelm a server, often as part of multi-vector attack campaigns. While normal HTTP floods are a burden, HTTPS floods add the extra twist of tying up encryption and decryption mechanisms.
  • Encrypted web application attacks — By encrypting traffic, web application logic attacks often pass undetected through both DDoS and web application protections.

How to sift attack traffic from legitimate traffic

To make SSL more blessing than curse means having a strategy to effectively pluck malicious traffic from encrypted streams. Here are a few of the most effective.

Regain visibility — Decrypt and re-encrypt SSL sessions so that you can inspect both clear and encrypted traffic without compromising privacy when content is in motion from point A to point B.

Implement service chaining — Give your SSL inspection tool lines to one or more security solutions so it can selectively forward traffic as needed to quickly mitigate an attack.

Make traffic inspection flexible Your goal should always be to support the efficient flow of legitimate traffic, which can be tough when inspecting it all. But it’s possible by dynamically defining filters that intercept and open traffic for inspection.

Keep your SSL traffic inspection secure — Of course an SSL traffic inspection solution can itself become a target. It’ll remain safe if you block it from performing like a proxy and deny it an IP address.

Seek scale — As traffic continues to grow, you shouldn’t have to bother with forklift upgrades. Find an SSL traffic inspection solution that will seamlessly scale with your rise in traffic.

Maintain high availability — If the SSL traffic inspection solution always sends traffic to the fastest-responding available security servers, you can sidestep any out-of-service servers that might cause downtime.

Encrypted traffic is only going to grow

Encrypted traffic is already a majority of what passes back and forth on the internet, and there’s no going back. It secures sensitive information and maintains privacy when data is in motion. But the other edge of the sword is the blind spots it inevitably creates in any organization’s security.

It is possible, however, to have the best of both worlds. By culling the malicious traffic from the real traffic, legitimate users don’t notice a thing and attacks are stopped in their tracks.

Get the TNW newsletter

Get the most important tech news in your inbox each week.

Also tagged with

Back to top