Written by: William Entriken , Tadej Vengust This article introduces a technique that you can use to test in arbitrary ways, but . It works on Ethereum Mainnet, Wanchain, Ethereum Ropsten, Hyperledger Burrow, Proof of Authority Network, private chains, and every other network based on Ethereum Virtual Machine. any deployed contract without spending any gas This new technique was first demonstrated with the ERC-721 Validator , and it doesn’t work with Truffle. This technique can help you out in cases when: Your contract deployment is too complicated to use Truffle You are testing somebody else’s contract, possibly with no access to its source code You want to batch test deployed contracts, regardless of how they were deployed You are into unusual stuff and errors in the Ethereum Yellow Paper Note: we have written all Solidity examples in this article in a way they can be copy-pasted and compiled with . Page example uses Web3 and requires MetaMask. Remix Clean Room Testing vs. Live Testing The clean room testing methodology is preferred for the majority of professional smart contract developers. It is cheap (doesn’t spend gas), fast (doesn’t wait for transactions), and it provides reproducible results. Combined with Ganache (personal blockchain), Truffle is the best-known tool for this approach: Several situations could impede your ability to use this technique. For example, your contract may require live test data (not reproducible) to confirm that specific code paths work correctly. It could be that your deployment process is too complicated to use Truffle. Maybe, you want to test compatibility with a contract that has no source code available. Or, perhaps, your CEO had you deploy a smart contract before test cases were finished. is a new methodology that allows you to perform validation tests on smart contracts on Mainnet while having access to all Mainnet data. Live testing Of course, you can also deploy your stubs to Mainnet. However, this could turn out as unnecessarily expensive and slow. Also note that in this setup, the validation process has full access to all Mainnet contracts and data. Testing With a Deployed Contract The can be performed by deploying a smart contract written explicitly for testing to Mainnet and then calling that contract. live testing technique In our first example, we will test whether or not Su Squares (an ERC-721 contract) provides a total supply of 10,000 tokens. ( was deployed with, and is intended to always comprise, precisely 10,000 non-fungible tokens). Su Squares Please note that we picked the 10,000 tokens supply test because it allows for a simple demonstration of the technique. The same technique can be applied in much more complex tests. pragma solidity 0.5.6; import "https://github.com/0xcert/ethereum-erc721/src/contracts/tokens/erc721-enumerable.sol"; contract SuSquaresTests { ERC721Enumerable testSubject; /** * @notice Deploy test contract with a subject to be used for all tests. */ constructor( address _testSubject ) public { testSubject = ERC721Enumerable(_testSubject); } /** * @notice Test token supply invariant. */ function testIsTotalSupply10000() external view returns (bool testResult) { testResult = testSubject.totalSupply() == 10000; } } This contract has been deployed on Mainnet already, and you can play with it at . 0x37d3bffed6f784d2cb5542bb9d9007c16e5938df For some good reason, this test performs actions that require changing the Mainnet state. This makes the test expensive to deploy and costly to run. Also, the test takes time to execute (due to one-block confirmation). In the following chapter, we will remove one of these expenses. Tests With ESTIMATEGAS We can eliminate the cost of executing this test function by using a built-in feature of the standard Ethereum client JSON-RPC (and by extension Web3.js), namely the . ESTIMATEGAS In the diagram below, “Client” refers to a Geth/Parity console session, Truffle or a web browser using web3.js. The parameters to are equivalent to making a normal transaction except that you do not have to sign it and it delivers the amount of gas used. (It is also possible to have a false/failed test result if all gas is used). Simply put, runs everything the same way as when making a transaction, but instead of committing the transaction it provides the information on the cost of the transaction. estimateGas estimateGas You should also note that in the case of running on a transaction that reverts (fails), the returned gas amount will equal the total gas available in a block. estimateGas With this in mind, we can alter the above test case so that the test result is indicated by the amount of gas used: /** * @notice Test token supply invariant. */ function testIsTotalSupply10000() external { require(testSubject.totalSupply() == 10000); } In this new version, if the test fails, the gas spent will represent the total gas available. Now we have established a way to run arbitrary test cases without the need to pay each time we use them. In the next part, we will also eliminate the expense of deploying the contract. But before proceeding, we must violate a statement from the Ethereum Yellow Paper. The Singleton Transaction Pattern The current version of the (the specification which all Ethereum clients should follow) states: Ethereum Yellow Paper There are [only] two types of transactions: those which result in message calls and those which result in the creation of new accounts with associated code (known informally as “contract creation”). This is actually inaccurate since the Ethereum reference implementations we have checked — Geth, Parity and all other implementations we know — all allow for a third type of transaction. We will abuse this fact so that we can refactor the test contract, immediately perform a test case and return results — without the need to actually deploy the contract. How can we do that? By moving the test case into the constructor. pragma solidity 0.5.6; import "https://github.com/0xcert/ethereum-erc721/src/contracts/tokens/erc721-enumerable.sol"; contract SuSquaresTests { /** * @notice Deploy test contract with a subject to be used for all tests. */ constructor( ERC721Enumerable _testSubject ) public { testIsTotalSupply10000(_testSubject); } /** * @notice Test token supply invariant. */ function testIsTotalSupply10000( ERC721Enumerable _testSubject ) public { require(_testSubject.totalSupply() == 10000); } } This example has violated the Ethereum Yellow Paper specification since the transaction that created a contract has also performed a message call into that contract. One call can even create multiple contracts and perform multiple message calls to those contracts, as we will show in the stubs example later. You must compile this contract to bytecode, first. Here is how you can achieve this with Remix IDE: <html> <head> <title>Estimate gas tests</title> </head> <body> <script src="https://cdn.jsdelivr.net/gh/ethereum/web3.js@1.0.0-beta.34/dist/web3.min.js" type="text/javascript"></script> <script> if (typeof window.web3 !== "undefined" && typeof window.web3.currentProvider !== "undefined") { var web3 = new Web3(window.web3.currentProvider); } else { alert("No web3"); } const abi = [ { "constant": false, "inputs": [ { "name": "_testSubject", "type": "address" } ], "name": "testIsTotalSupply10000", "outputs": [], "payable": false, "stateMutability": "nonpayable", "type": "function" }, { "inputs": [ { "name": "_testSubject", "type": "address" } ], "payable": false, "stateMutability": "nonpayable", "type": "constructor" } ]; const bytecode = "0x608060405234801561001057600080fd5b5060405160208061021d8339810180604052602081101561003057600080fd5b81019080805190602001909291905050506100508161005660201b60201c565b506100e7565b6127108173ffffffffffffffffffffffffffffffffffffffff166318160ddd6040518163ffffffff1660e01b815260040160206040518083038186803b15801561009f57600080fd5b505afa1580156100b3573d6000803e3d6000fd5b505050506040513d60208110156100c957600080fd5b8101908080519060200190929190505050146100e457600080fd5b50565b610127806100f66000396000f3fe6080604052348015600f57600080fd5b506004361060285760003560e01c806347d54a4514602d575b600080fd5b606c60048036036020811015604157600080fd5b81019080803573ffffffffffffffffffffffffffffffffffffffff169060200190929190505050606e565b005b6127108173ffffffffffffffffffffffffffffffffffffffff166318160ddd6040518163ffffffff1660e01b815260040160206040518083038186803b15801560b657600080fd5b505afa15801560c9573d6000803e3d6000fd5b505050506040513d602081101560de57600080fd5b81019080805190602001909291905050501460f857600080fd5b5056fea165627a7a723058201eaa86ca0d93585836b0d06315ab78c83079efff596c1a0417c5195ad0fc9e2d0029"; async function test(){ const contract = new web3.eth.Contract(abi); const contractAddress = document.getElementById("contractAddress").value; // First we check max gas limit. let gasLimit = 8000029; await web3.eth.getBlock("latest", false, (error, result) => { gasLimit = result.gasLimit; }); contract.deploy({ data: bytecode, arguments: [contractAddress] }).estimateGas({ from: '0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2' }, (err, gas) => { // If there is no error and gas is bellow has limit the test succedded. if (err === null && gas < gasLimit) { document.getElementById("console").innerHTML = "Contract has 10000 tokens." } else { document.getElementById("console").innerHTML = "Contract does NOT have 10000 tokens." } }); } </script> <h3>Testing if ERC-721 enumerable contract has a total supply of 10.000.</h3> <h4>Quick copy contract addresses for tests:</h4> <b>Su Squares (passing test): </b>0xE9e3F9cfc1A64DFca53614a0182CFAD56c10624F <br/> <b>Axie Infinity (failing test): </b>0xf5b0a3efb8e8e4c201e2a935f110eaaf3ffecb8d <br/><br/> <input id="contractAddress" type="text" placeholder="Input contract address" /> <button onclick="test()">Test</button> <p id="console"></p> </body> </html> This is an arbitrary test case written in Solidity. It can be called against arbitrary contracts, it can use arbitrary data and make arbitrary calls to other contracts. Oh, and P.S.: This same test case can be run against any EVM network, such as Ethereum Mainnet, Wanchain, Ethereum Ropsten, Hyperledger Burrow, Proof of Authority Network, as well as private chains. Stubs and Artifacts A stub is a specific, additional contract that is required just to validate a test subject contract. For example, if you want to validate the ability of an ERC-721 contract to transfer tokens, you will need a contract that can receive tokens. You can check a fully developed . example at 0xcert Below is a basic example for an ERC-721 receiver stub. You should note that this example assumes that the contract owns a token. You can achieve this in several ways that we describe in the following section. The “giver” technique has also been implemented in the 0xcert example linked . here pragma solidity 0.5.6; import "https://github.com/0xcert/ethereum-erc721/src/contracts/tokens/erc721.sol"; import "https://github.com/0xcert/ethereum-erc721/src/contracts/tokens/erc721-token-receiver.sol"; contract StubsAndArtifacts { /** * @notice Deploy test contract with subject to be used for all tests. */ constructor( ERC721 _testSubject, uint256 _tokenId ) public { // This test assumes that this contract owns that particular _tokenId. You first // need to get a token. There are multiple ways to achieve this like "buying" it // before running test or using "the giver" technique. All of this is explained // below in the article. testDoesCorrectlyTransferData(_testSubject, _tokenId); } function testDoesCorrectlyTransferData( ERC721 _testSubject, uint256 _tokenId ) public { StubTokenReceiver stub = new StubTokenReceiver(); // we expect failure _testSubject.safeTransferFrom(address(this), address(stub), _tokenId, "ffff"); } } contract StubTokenReceiver is ERC721TokenReceiver { bytes4 constant FAKE_MAGIC_ON_ERC721_RECEIVED = 0xb5eb7a03; /** * @dev Receive token and map id to contract address (which is parsed from data). */ function onERC721Received( address _operator, address _from, uint256 _tokenId, bytes memory _data ) public returns(bytes4) { return FAKE_MAGIC_ON_ERC721_RECEIVED; } } In the same way you can deploy stubs employing the live testing technique, you can also test contract deployments. Limitations The shown technique provides a much-needed method of testing deployed contracts but does not come without limitations. We need to acknowledge the limitations and offer solutions to some. Smart contract are not accessible from within contracts, therefore, there is no way of testing them without actually performing a transaction. This means that testing whether an event is emitted is not possible with . events estimateGas What if you are testing something like token transfer? To be able to transfer tokens, you need to be the owner of tokens or an approved party. This is not a problem for since it does not need to be signed by a private key and therefore can be run from any address. This way, you can run the transaction as the address that actually owns the tokens even though you do not have its private key. estimateGas What about testing the method in an ERC-721 contract with no input from any of the token owners? In this case, you need a receiver similar to the one described in the section and with the ability to receive the token. Preferably, you would want to add multiple receivers with different types of passing/failing tests. You can solve this with minimum costs. All you have to do is create and deploy a few contracts that will serve as receiver tests. Once they are deployed, you can test any ERC-721 contracts , as long as you know an existing ID of that contract, and if the contract supports the enumerable extension, this method can produce immediate results. safeTransferFrom Stubs and Artifacts safeTransferFrom In practice, it would work like this: you find the token owner by token ID, and create an on a with the destination of the test contract you already deployed. To show you how simple it is, we’ve compiled a working example of this method that you can check . estimateGas safeTransferFrom here What about a case that involves multiple parties that need to provide approval/ownership? Well, this is a limitation that cannot always be solved completely gas-free. Still, here are two examples that show how you could do it: If you would like to test an ERC-721 asset that you don’t own or that has not yet been minted, but the asset is for sale on , your test case could run as the WETH account (with 2 million Ether available). After you purchase the token on OpenSea, you can use that token as you please. OpenSea Another way is the approach. This method is not completely free but makes the token completely safe. You can check the to see this approach in action. giver contract ERC-721 Validator Conclusion provides a complete such as Truffle. This new approach makes the entire state of the network available for your test cases. And since the way in which the test subjects were deployed is not important in your case, you can run a single test suite very easily against many deployed contracts. Live testing technique alternative to testing with clean room tools Reading circle questions Which test situations would work better in the clean room approach and which in the live test approach? Does the introduction of live testing limit the usefulness of Truffle and similar tools? What are the benefits of using a live test approach for research on security vulnerabilities? What changes could be made to the (Ethereum) network client to make live testing simpler? Was the Ethereum Yellow Paper literally wrong, or is this a contrived interpretation? The present article along with the method described and the code snippets is the work of William Entriken , lead author of ERC-721 standard, and Tadej Vengust , lead blockchain engineer at 0xcert. Check how ERC-721 Validator works Check the complete code of live testing with estimateGas on GitHub Check the validator example Visit and star our . GitHub repository <a href="https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href">https://medium.com/media/3c851dac986ab6dbb2d1aaa91205a8eb/href</a>
