How Does the Adversarial Cycle Apply to Coding and Security?

The majority of application security problems stem from software bugs that leave the existing security controls, broken.

However, even if the code is perfect it doesn’t mean an attacker can’t exploit it.

The vulnerability can be hidden inside the business logic, not the code that powers it.

This type of vulnerability is called business logic vulnerability. It’s when an attacker abuses a legitimate flow of an application so that it results in negative consequences.

An example could be a contact form on a website that is used for sending out emails to the service owners. This form can be abused to send out spam messages instead of genuine support requests.

When dealing with business logic vulnerabilities we enter a branch of security called anti-abuse. Fighting abuse often requires thinking about these problems more creatively, because unlike patching the bugs in the code the business logic often can’t be changed to mitigate the risk.

We are stuck in a position where we can’t really fix the problem, we can only manage it. It requires active participation because an attacker will change the tactics as we put defense mechanisms in place.

That's what makes it an adversarial problem. To better understand it let's look at the following diagram.

So the defender directly controls stages 1 and 2, and the attacker directly controls 3 and 4. However, both of the adversaries control the entire cycle indirectly.

The objective of the game for the attacker is to make sure that the attack doesn’t get detected. If it does, they need to be able to change the plans fast enough to try a different approach.

The life of the defender is the same, but for different transitions. The defender wants to detect the attack and stage the defense as fast as possible.

Translating that to the diagram the attacker wants for transitions 1 and 2 to be as long as possible and transitions 3 and 4 as short as possible. Contrary the defender wants transitions 3 and 4 take a very short time, but stretch out transitions 1 and 2.

So, you may be wondering, how can it be applied in practice? 🤔

As a defender, it's easy to fall into the trap of spending too much time on intuitive solutions that don’t make the life of the attacker that much harder. I’ll give you an example.

Let’s say we need to protect the contact form mentioned earlier from sending spam. We see that the attackers are using a bash script and that is reflected in the User-Agent header.

Naturally you might think that we should block all attempts that come from this particular User-Agent. Since we don’t want the spam to be sent out we block the attempt right there on the page.

However, what just happened is we gave the adversaries a very tight feedback loop to figure out our defense.

To circumvent it they just need to spoof the User-Agent to look like a browser. It’s a simple change on their side and they are back in business. Not good for us, now it got harder to detect them.

The attackers get stronger as we fight them. Keeping in mind the adversarial cycle helps us avoid first-order-thinking and evaluate the consequences of an intuitive solution.

Co-published here.