Using the Forward Trust Design Pattern to Make Scaling Easier

Introduction

There is a powerful pattern at the heart of the Yield Protocol. It allows us to instantly integrate smart contracts that do not know each other. It also allows us to create new business features on the blockchain without deploying new smart contracts. It allows us to be flexible and scalable, if careful.

I’ll call this pattern Forward Trust.

Forward Trust is a combination of two concepts:

- Multi call, in which a contract allows to batch several calls to itself in a single transaction.

- Balance tracking, in which instead of using `transferFrom` to pull tokens from somewhere, a contract records how much of a given token it holds. When a function in that contract is called, any surplus between the recorded and the actual balances is assumed to belong to the caller.

These two concepts can be combined so that basic token operations, complex functions, and external calls to other contracts become a higher-level language in which business features can be created off-chain and then immediately executed on-chain.

Forward Trust is a combination of two concepts: Multicall and Balance tracking.

The idea is not completely new. Furucombo, Instadapp, and many others allow users to compose features that integrate several DeFi protocols to build new features. You also have weiroll which is a more generalized version of the same idea.

I didn’t invent Multicall or Balance Tracking. It is quite possible that I wasn’t the first person either to put these two ideas together. I don’t think that matters either way, my intention is to make this pattern known and help others use it so that as an industry we build better and and faster. My intention also is to share the lessons learned from a year of using this pattern, so that others don’t repeat our mistakes.

In explaining Forward Trust as it is implemented for the Yield Protocol, I’ll start by giving an overview of Multicall and in general how to build a router smart contract for composable transactions. Following, I’ll explain how to use the Balance Tracking pattern. I’ll include notes alerting of the risks from an incorrect implementation or operation. Finally, I’ll briefly discuss the impact of Forward Trust with regard to core beliefs in smart contract development.**

Here we go, buckle up.

Building a Composable Router

In building a composable router, we are going to break up our business logic into pieces smaller than a business feature, and we are going to implement some tools to put them back together in any combination.

With “business feature”, I mean a feature that makes sense to your users. For the Yield Protocol, we have business features such as “add collateral”, “add collateral and borrow underlying”, “add liquidity to a pool”, and many others. These features often have variants, like “add collateral to a position using an off-chain signature”, “borrow using an ERC1155 as collateral”, or “remove my WETH collateral and send it to me as Ether”.

Building each of these features separately would require a lot of time, as you would need to code a smart contract and get it through the several stages of testing, auditing, and deployment. That’s __what we did for Yield v1__and after adding a handful of features we realized the enormous costs.

In a composable router, we break up our business logic into small pieces

Yield v2, on the other hand, has a composable router. We already have more than fifty features and can add more at a much faster pace. There aren’t as many constraints imposed by the smart contracts, because smart contracts don’t have to change as often.

The idea behind a composable router is that many business features share parts of their code. Some very basic actions that are repeated often are:

• Take tokens from a user wallet.
• Execute an off-chain signature for a token approval.
• Wrap Ether into WETH.
• Unwrap Ether into WETH.
  

Some other more complex actions that we still repeat often are things like:

• Build an empty debt position
• Manage collateral and debt balances
• Trade assets in YieldSpace
• Redeem mature fyTokens
  

Your protocol can surely be broken down in the same way. An example recipe for borrowing fyToken in the Yield Protocol would be built from the actions above as follows:

Tx1: [
    build(seriesId, ilkId, salt),                  // Create an empty position
    permit(ilk, ladle, posted, deadline, v, r, s), // Execute an off-chain approval
    transfer(ilk, ilkJoin, posted),                // Take tokens from the caller
    pour(vaultId, ignored, posted, 0),             // Add collateral and borrow
]

This recipe combines four of the actions available in the router to offer a finished business feature. There are variants to this that without a composable router would require extra smart contract code.

For example, some users cannot sign off-chain approvals. For those, the front end will build this instead.

Tx1: token.approve(spender, amount)    // Standalone transaction

Tx2: [
    build(seriesId, ilkId, salt),      // Create an empty position
    transfer(ilk, ilkJoin, posted),    // Take tokens from the caller
    pour(vaultId, ignored, posted, 0), // Add collateral and borrow
]

As another example, if the user wants to create a fixed-rate loan they need to sell their fyToken in YieldSpace as they borrow them. The recipe for that would be:

Tx1: [
    build(seriesId, ilkId, salt),
    permit(ilk, ladle, posted, deadline, v, r, s),
    transfer(ilk, ilkJoin, posted),
    serve(vaultId, receiver, 0, borrowed, maximumDebt),
]

The combinations grow exponentially as we match specific conditions and product offerings. Being able to express only the composing actions makes all the difference.

Multicall

The idea of batching several function calls in a transaction might have been first done by dYdX, but later ENS provided an incredibly elegant and succinct implementation for batching. By having a function that accepts an array of encoded function calls, and delegatecalls to itself with each one, we build a batching mechanism.

function multicall(bytes[] calldata data) external returns(bytes[] memory results) {
    results = new bytes[](data.length);
    for(uint i = 0; i < data.length; i++) {
        (bool success, bytes memory result) = address(this).delegatecall(data[i]);
        require(success);
        results[i] = result;
    }
    return results;
}

In this way, all the calls in the batch are executed in the same contract. The genius is in using `delegatecall` to preserve `msg.sender` and other transaction variables, as a regular `call` would overwrite `msg.sender` with the contract receiving the batch.**

A batching function accepts an array of encoded function calls and delegatecalls to itself with each one.

Coding the basic actions to use within multicall is fairly straightforward, they tend to be one or two lines of code each:

/// @dev Execute an ERC2612 permit for the selected token
function permit(
  IERC2612 token, address spender, uint256 amount,
  uint256 deadline, uint8 v, bytes32 r, bytes32 s
  ) external payable {
    token.permit(msg.sender, spender, amount, deadline, v, r, s);
}

/// @dev Take tokens from an user
function transfer(IERC20 token, address receiver, uint128 wad) external payable {
  token.safeTransferFrom(msg.sender, receiver, wad);
}

Our router has many more examples of actions that we compose into user features, and more can be added by upgrading the router. Calls to these actions are encoded in a calldata string, and then pushed into an array. The `batch` function reads all these actions and executes them sequentially.  A router built this way is very simple and therefore robust, because the actions are small and there is little nesting between them. Just be careful with known ways of misuse.

When we released Yield Protocol v2, we knew of a few features we wanted to build, but we also knew that we would regularly think of new features, and that deploying new smart contract code for each one would slow us down. Coding common actions as small individual functions, and using Multicall to put larger business features together from the frontend, we could build much faster.

Route

We soon realized that we could treat calls to other contracts as building blocks in this language. The Yield Protocol contains an automated market maker, YieldSpace, which acts as a permisionless unit on its own. By implementing a `route` building block in our router, which simply calls a function on any other contract using `call`, we could use a single function in the router to call any function on a YieldSpace pool. We soon extended that so that we could call any contract from the router.

The challenge we faced is that our router has permissions on some other contracts .If we allowed anyone to execute arbitrary calls from our router we would effectively allow them to impersonate our router, with catastrophic results.

We can call any function in any contract from the router

We saw several solutions for this problem:

- Make sure the contract implementing `route` never has permissions anywhere.

- Implement a blacklist of contracts that calls cannot be routed to.

- Install a relayer to `route` calls through, and make sure the relayer never has permissions anywhere.

We implemented both a relayer and a whitelist of contracts that we could route calls to, for added security. There is no simple best solution, but the lesson is that you can include simple calls to contracts as yet unknown as one of the actions in your router.

There were risks in allowing arbitrary calls from our router, but time has shown that the extra flexibility makes it worth it.

Token Transfers

Closer to the go-live, when we were busy creating feature after feature, I noticed that it was much easier to integrate with our YieldSpace pools than with other contracts.

That was because YieldSpace was modeled after Uniswap v2, and it tracked its own balances. When you wanted to sell Dai, you could send a transfer the Dai to the YieldSpace Pool contract, and then call the `sell` function there on the same transaction. The pool would check its Dai balance against the recorded reserves and would find that there was a surplus of Dai. Then it assumed the Dai was yours and executed the sale.

The extra storage required to track balances might seem like a waste, but in exchange, the caller doesn't need to approve the pool to take tokens.

The caller doesn’t need to know much about what will happen to the tokens, no more approvals needed.

You might not be impressed. The whole approve and transferFrom pattern is annoying, but it works, so what's the deal?

The deal is that with this pattern the caller doesn’t need to know much about what will happen to the tokens, only that they need to be dropped somewhere. If we are building transactions as a batch each function, perhaps in a different contract, follows a simple pattern:

• Check what the input was
• Do something with the input
• Leave the output somewhere
• Update recorded balances

No more approvals are needed. Each function does its thing and drops tokens in some other contract. The next contract to be called by the `batch` function finds out how many tokens he just received, and continues the execution. Just make sure that every token transfer inside a function has its own parameter for a receiver, you’ll thank me later.

Suddenly, you can scale. It doesn't matter anymore how many contracts you have in your protocol. Everyone is integrated with everyone else out of the box. Even with contracts that don't exist yet.

Here be dragons

Now that we have discussed the advantages of Forward Trust, we must discuss the risks. Everything always has its trade-offs, and the strength of the advantages implies the disadvantages are serious as well. You must tread lightly.

The Dark Forest

When you implement your contracts according to this pattern, you need to be very aware that you are entering the dark forest. Bots lurk in it and will take any tokens if they can.

When building the contracts, you need to be careful about how you track the balances, and how the tokens can leave the contract. If the surplus is not calculated correctly, anyone will take what they can.

When building recipes, ensure that you don’t leave tokens unattended when your transaction ends. If you send tokens somewhere and don't call the recipient so that it uses them or updates its balances, they will vanish.

Be aware that you are entering the Dark Forest

When communicating with your users, you need to stress that just calling functions on Etherscan is not as safe as they are used to. They can’t hurt your protocol, but they are likely to lose their funds.

You can use this pattern, and we have done so safely for a year at Yield, but be aware that you need to think differently now.

Tales from the front

The recipes to execute business features within the Yield Protocol are complex. They could be three, four, or maybe eight function calls batched with very precise parameters. Users only see an innocent-looking button, but a lot of work is required to make sure that a recipe is safe.

Using Forward Trust, the business experts build the recipes with help from smart contract developers. Then the frontend takes the enormous responsibility of implementing them correctly and testing them again to make sure they work as intended.

The frontend developers take an equal responsibility

There is a big shift in risk from the smart contract developers to the frontend developers, as only the frontend now protects the users against bots. You need to apply the same level of attention to your frontend and to your smart contracts.

Trust, but verify

There is a tradition of giving users complete visibility into how a smart contract works. Trust, but verify. By shifting development towards the frontend Forward Trust makes verification more difficult, unfortunately.

Frontend code is often private, as a moat protecting against protocol forks. Frontends could be updated at any time with new code that changes the recipes, maybe with bad consequences for the users. There are some calls to provide decentralized and verifiable frontends, and they are even more necessary using this pattern.

Shifting development towards the frontend makes verification more difficult

Transactions are not private, though, and they can be previewed and debugged. Tools like__Tenderly__ make it easy to see the effects of any transaction before it gets executed, and user communities could use it to verify frontend implementations and point out inconsistencies.

Conclusion

The Yield Protocol is built differently. Using Multicall and the idea of tracking token balances, we built a higher-level language to create business products with. The result was a very flexible and scalable protocol, but that came with its own set of challenges.

We didn't do what the auditors expected of us, and we had to figure out ways to use this pattern safely. Sometimes we made mistakes and learned from them. One year later, we are still alive, and pumping out features at a fierce rate. It's paying off.

I hope that you learned something useful from this article. Maybe you’ll get to code better applications on the blockchain, more powerful and flexible. Be careful of the risks while we come up with better ways of managing them. Good luck.

Thanks to whoever came up with Multicall first, I’m guessing that Nick Johnson. Thanks to Uniswap for coming up with Balance Tracking. Thanks to BoringSolidity who I based my own Multicall implementation on. Thanks to Allan Niemerg for giving me the opportunity to build the Yield Protocol. Thanks to transmissions11 and sorry I didn’t name the pattern as you suggested. Thanks to everyone who has helped me build and learn, there are lots of you.