Integrating Security Testing Into Your Development Cycle With Foundry and Diligence Fuzzing

Security continues to be one of the most important aspects of web3. As a developer, you should be securing your smart contracts by ensuring that testing—specifically a wide variety of testing—is integrated into your smart contract development lifecycle. If you aren’t embedding testing in your flow, you’re just biding your time until your smart contracts are hacked.

Let’s look at how fuzzing meets that need, and how you can quickly implement fuzzing by using the newly released Diligence Fuzzing support for your Foundry web3 projects.

Web3 security standards are still emerging

Web3 security standards are still being defined. We need more tools, more best practices, and more (I hate to say it) processes. Not only do we need to catch up to traditional development practices, we need to surpass them. After all, a lot of money is at stake in our smart contracts.

How do we contribute to growing and maturing security in web3? By building better tools, integrating more processes into our daily practice, and automating as much as we can. We have a long way to go to catch up with the automated CI/CD pipelines of traditional development … but we’re getting closer.

One step that’s a good start: integrating and automating fuzzing into your smart contract development cycle with Foundry and Diligence Fuzzing. Let’s look at what that means and how to actually do it.

What is Diligence Fuzzing (Fuzzing-as-a-Service)?

Diligence Fuzzing (aka Fuzzing-as-a-Service) is a web3 fuzzing tool for testing smart contracts by ConsenSys.

Using a spec of the expected behavior of your code, it creates a large number of inputs/transactions that might violate this expected behavior. It then runs the fuzzed inputs against your code, trying to cover as much code as possible, and trying to, well, make things fail! If your code violates the specification, the fuzzer raises a warning.

The specs are created using Scribble to annotate your smart contracts.

According to the Diligence site, Diligence Fuzzing is, “The most powerful fuzzer to find bugs and vulnerabilities before they are exploited by bad actors on mainnet.”

What is Foundry?

Foundry is an open source development framework, used for creating and deploying smart contracts on Ethereum/EVM-compatible blockchains. It has lots of components: a CLI, a testing framework, and a development framework.

From their own book: “Foundry manages your dependencies, compiles your project, runs tests, deploys, and lets you interact with the chain from the command-line and via Solidity scripts.”

Foundry has recently become very popular. I love it because it was built with security and testing in mind.

Interestingly, Foundry ships with a fuzzing tool that works pretty well. Foundry does a lot of things pretty well—that’s another reason I love it!

But Diligence Fuzzing was built just for fuzzing, and it takes your fuzzing game to the next level. A few quick reasons:

• It uses gray-box testing as opposed to black-box testing, essentially allowing the fuzzer to discover and test more code/paths. More code coverage, of course, allows you to find more bugs in your code. In a recent comparison, it found 25% more violations than Foundry.

• By default, it uses “stateful fuzzing” where it generates sequences of transactions instead of just one transaction. This can hit edge cases in the logic where multiple transactions are needed to trigger the bug.

• It’s faster!

  
  

And now existing Foundry projects can use Diligence Fuzzing with just a few steps.

How Fuzzing enhances your smart contract development cycle

Fuzzing is a type of software testing considered “property-based testing.” This type of testing checks that the output of your code matches expectations for a wide variety of inputs. A property-based tester automatically creates a whole lot of random inputs and runs them through your code, trying to find one that violates the expected output.

This kind of testing is usually a complement to your unit testing. Your unit tests check for known flows, simple inputs, and edge cases you’ve already caught and fixed … basically, the test cases you know you need to run.

Fuzzing, on the other hand, is a software testing technique that looks for defects in your source code that you don’t know about. It automatically runs through a lot of possibilities (more than you could ever test for manually), trying to find complex and subtle bugs that you wouldn't otherwise uncover.

Fuzzing in action

Let’s walk through the steps needed to start fuzzing a Foundry project with Diligence Fuzzing.

First, install the Fuzzing CLI by running the following on your command line (also be sure you’re on Python 3.6 or newer):

pip3 install diligence-fuzzing

You need a Diligence fuzzing account and an API key. Sign up here, then go to your dashboard to create the key.

Now run:

echo FUZZ_API_KEY='your api key here' > .env

This will add your API key to a .env file for the CLI.

Finally, run the fuzzing CLI and create a new fuzzing campaign.

fuzz forge test

That’s it!

You can target specific tests and contracts using the following options:

Usage: fuzz forge test [OPTIONS]

  Command to:  * Compile unit tests  * Automatically collect unit-test contracts  * Submit to fuzzing

Options:

  -k, --key TEXT         API key, can be created on the FaaS Dashboard.

  --dry-run              Outputs the data to be sent to the FaaS API without making the request.

  --match-contract TEXT  Only run tests in contracts matching the specified regex pattern

  --match-path TEXT      Only run tests in source files matching the specified glob pattern

  --build-args TEXT      Additional string of `forge compile` command arguments for custom build strategies (e.g. --build-args=--deny-warnings --build-args --use 0.8.1)

  --help                 Show this message and exit.

For more information, check out the documentation here.

Please note a couple things:

1. Everything above assumes you have installed Foundry and written some fuzzing tests. Since your existing Foundry tests probably contain several Solidity assertions and invariants, the fuzzing tool can just use those to start breaking things.
2. Only some of the Foundry cheat codes are currently supported: ‘warp’, ‘prank’, ‘deal’, ‘roll’, and a few more. Additional support is planned soon.

Keep security top of mind with Fuzzing and Foundry

Securing web3 smart contracts is critical, and it has been for a long time. But with the advent of new testing tools, new integrations, and more rigorous processes, we’re getting closer to a reliable, secure, and mature ecosystem. Fuzzing is one step towards that goal.