Game One

by K. K. Mookhey, NII Consulting

e4
It’s late at night, and the phone rings. This had better be a world-changing revolution. But it’s something weirder. A client in East Asia informs us that his systems are behaving most abnormally. Before one can gather one’s senses, the information begins to flow:

“The primary trading systems, which offer web-based trading are down”

The panic in his voice is unmistakable. But this statement could mean many things, so we probe further.

Opening
Have the web servers or database servers crashed? No.

Are they simply not accessible over the network? Yes.

Are they not accessible from the Internet, or from the internal network or both? Inaccessible both ways.

Now, this client is one of the country’s largest Internet share trading portals. The stock markets have been going crazy, and transaction volumes are testing the very limits of infrastructure capacity.

So maybe it’s simply too many investors going for the same trades at the same time, or any other such market anomaly, which would go away on its own? No. There are no IPO’s being listed today, there is no special buzz on the bourses, and the other online trading systems are doing just fine. It’s just our systems.

Hmm… looks like a Denial of Service attack or a worm out there or something absolutely weird (other than the fact that it’s 3:30 AM). Time to call in the help of some friends.

Middlegame
We’ve just done a project for the primary Internet Service Provider in the country, and know the manager of their SOC (Security Operations Centre) well.

We call him up, and ask:
Any worm traffic you’ve been seeing lately hitting ports 80 and 443? Or anything specifically targeting IIS? There was a remotely exploitable vulnerability released recently in IIS, right? He doesn’t recollect seeing any seriously anomalous traffic for these ports, or any spikes in earlier worms targeting IIS. But he hints at something more sinister being at play: cyber-extortion.

We call back the client, and now are surer of the information we need to know:
Are the source IP’s local or largely overseas? The assumption is that if it were just a spike in trading, then it would be largely the local populace accessing the systems. We’re informed that the source IP’s are largely from Eastern Europe. Plus, they’re a very wide range of source IP addresses – the IDS is dropping packets at an alarming rate, and the firewalls are at 100% CPU utilization.

The panic levels are continuing to rise, as senior management is concerned about the loss in revenues due to the complete absence of trades being placed, not to mention the reputation loss that is likely to occur once the news hits the media.

We ask them to immediately inform their upstream ISP and block all source IP addresses, except those IP blocks allocated to the country. Our friend whom we called in earlier has also been informed to co-ordinate the activity on a war-footing.

Within 15 minutes the client calls back. The flood of packets has been blocked successfully at the upstream ISP, and trading is more or less back to normal. Obviously, people traveling outside the country are still not able to trade, but the DDoS attack is no longer the nightmare it could have been.

Endgame
So what really happened out here? The next day we looked around for more information about the symptoms we’d seen – Distributed Denial of Service attacks targeting web servers, specifically of systems where transaction volumes are so huge, that even a few minutes of downtime results in significant losses. The most likely answer, we learnt, is cyber-extortion. We checked with the client if they had received a fax or voice communication asking them to pay up or be subjected to huge losses, or were there any serious disputes with trading partners or any other indications that someone had a grudge. So far, we’d been dealing with the IT team, but now we’re told they’re off the case, and Internal Audit has taken it up.

Post-mortem
We were never really able to confirm one way or the other whether it was a case of cyber-extortion or not. It was most decidedly a Distributed Denial of Service attack targeted at the client. The fact that DdoS-based extortion threats are on the rise is becoming evident[1],[2]. We ruled out worm traffic, since that should normally have affected a larger number of web-based systems in that country, and none of the malware monitors showed any spikes in worm traffic in the region. Their servers were functioning perfectly all right, and normal trading resumed almost as soon as the upstream ISP filtered the traffic. The Internal Audit department handled the matter internally, and didn’t inform the IT team whether there really was an extortion threat, or we were simply being paranoid.

The key lesson here was that a well-planned incident response strategy can help prevent knee-jerk reactions when security emergencies arise. It can save millions of dollars of financial losses and control the loss to reputation that can occur even if the systems are only unavailable, not compromised. Also, the client eventually went in for TopLayer IPS.

References

1. http://www.networkworld.com/news/2005/051605-ddos-extortion.html

2. http://www.csoonline.com/read/050105/extortion.html