Photo by Timo Wagner on Unsplash
If you are one of the strong readers who made it in part 2, welcome back! If you are not, I urge you to read part 1 and 2 with care, and get back when done.
Dip Dapp Doe — Anatomy of an Ethereum distributed fair game (part 2)_Today we are going to deploy the contracts and throw some colours on the screen as we design and a static web site to interact…_hackernoon.com
Today we are going to follow the Test Driven Development methodology on our frontend, along with Web3. We are also going to bundle the dapp and distribute it with IPFS.
Keep your seat belts fastened, because we are approaching our destination!
Picture by publicdomainphotos
As a reminder, the source code of the article can be found on the GitHub repo:
ledfusion/dip-dapp-doe_dip-dapp-doe — Distributed app featured in a Medium article_github.com
In the last article we dived into the architecture, design and the building blocks of our dapp’s frontend. For educational purposes, we even showed the integration of one of the blockchain transactions, but let’s not lose perspective. In TDD, we need to spec first and code later.
There are very nice tools which allow to automate UI tests and even record them visually among different browsers. However, in dapp testing we are limited by two important caveats:
**Only a few browsers support Web3**Browser support may extend with the release of new MetaMask plugins, but we are mainly pivoting around the Chrome engine and Gecko.
**We can’t get programmatic access to control MetaMask/Web3**Allowing Javascript code to accept Ethereum transactions would be a huge security flaw, because any web site could steal our funds at once. However, that is what we need to do in order to test our code.
The last issue would have been a major drawback for any serious project’s workflow. Until now.
Puppeteer is an official package from Google that allows to programatically control a Chromium instance from NodeJS on Linux, Windows and MacOS. However how do we add the MetaMask plugin and tell it to accept transactions if the plugin runs out of our window?
That’s where Dappeteer comes into play! It is another NPM package that features a embedded version of MetaMask, tells Puppeteer to run with the plugin enabled and provides some wrapper methods to import accounts, accept transactions and even switch to a different network.
In our web
folder:
$ npm i -D puppeteer dappeteer
If you recall, in part 1 we developed our smart contracts by deploying and testing them in a local blockchain. Test cases waiting for every public transaction to be mined would take ages to complete.
However, in part 2 we demonstrated the integration to the public blockchain from the browser. What happens, now? How can we use a local blockchain so that transactions can be mined as fast as when using Truffle?
The tool for this is Ganache CLI. It is another NPM package, which is part of the Truffle Framework and it is what we actually used under the hood in part 1.
$ npm i -D ganacle-cli
If you run it now, you should see something like this:
Ganache CLI output
As you see, it generates random wallets with 100 ether, but it can be fully customized. Now we can mine immediate transactions without polluting the public blockchain with junk.
I normal web projects, you may be used to working with Webpack started by a simple NPM script. However, in the current project we need to start combining different simultaneous components at the same time.
What needs to happen when we run our E2E tests?
You are free to use any task runner that you like, but to me this clearly becomes a job for a shell script. To get the best of both worlds, I’d suggest you to use [runner-cli](https://www.npmjs.com/package/runner-cli)
, along with a Taskfile. More on this.
$ [sudo] npm i -g runner-cli
Let’s create one:
$ run --new? What template do you want to use?Gulp fileNPM packageMakefile❯ Shell script
Now edit taskfile
and add a function called test
with the following set of commands (commented in-line)
function test {echo "Starting ganache"ganache-cli --mnemonic "$(cat ./dev/mnemonic.txt)" > /dev/null &ganache_pid=$!
Here we start the server in background (with the &
at the end) and retrieve the process PID by assigning $!
into ganache_pid
. Also note that "$(cat ./dev/mnemonic.txt)"
reads the contents of the mnemonic.txt
file and puts them as a Ganache parameter. With that, everyone can import the same account.
echo "Recompiling the contracts"cd ../blockchain./taskfile buildcd ../web
Here we go to the contracts
folder and run another script that launches Solc
to compile the contracts. Compilation can run concurrently with Ganache.
echo "Deploying to ganache"node ./dev/local-deploy.js
This script is quite similar to blockchain/deploy/lib.js
. Instead of deploying to the Ropsten network, it deploys them to Ganache. It also stores the instance address into .env.test.local
(will see it later).
echo "Bundling the web with NODE_ENV=test"NODE_ENV=test parcel build -d ./build --log-level 2 --no-source-maps src/index.html &parcel_pid=$!
Now that we know what address to attach to, we can tell Parcel to bundle from src
to build
with the appropriate environment variables in place. This can run in parallel with our next step:
echo "Starting local web server"serve build -p 1234 &serve_pid=$!
This will simply start an HTTP server, leave it on the background and get note of its PID. Run npm install -D serve
to add it to the project.
echo "Running the tests"wait $parcel_pidmocha ./test/frontend.spec.jsmocha_result=$?sleep 1
Here, we wait
for the Parcel process to complete, and when it does, we finally start our Mocha test cases. We keep the exit code of Mocha by reading $?
and a bit later we start to clean things up:
echo "Stopping the servers"kill $ganache_pidkill $serve_pidexit $mocha_result}
We kill the two background processes and finally exit with the status code returned by Mocha.
Ta da!
At the current point, if we run parcel -d ./build src/index.html
, we will start a dev server on port 1234 with a Web3 pointing to the Ropsten (test) network. But if we dorun test
, then we expect to have a web site that will connect to Ganache. How to achieve that without touching any code?
Parcel allows us to use .env
files and map the KEY=value
lines into process.env.*
variables. Let’s create a couple of files for our environments. In web/.env
:
CONTRACT_ADDRESS=0xf42F14d2cE796fec7Cd8a2D575dDCe402F2f3F8FWEBSOCKET_WEB3_PROVIDER=wss://ropsten.infura.io/wsEXPECTED_NETWORK_ID=ropsten
These are the environment variables that will be used by default. This is, when compiling the web, we will connect to the public Ropsten network, expect MetaMask to be on this network too and use the contract address where it is deployed.
However, when we are testing, we want those variables to look like below in web/.env.test.local
:
CONTRACT_ADDRESS="--- LOCAL CONTRACT ADDRESS GOES HERE ---"WEBSOCKET_WEB3_PROVIDER=ws://localhost:8545/wsEXPECTED_NETWORK_ID=private
When NODE_ENV
is set, Parcel will look for .env.$(NODE_ENV).local
and inject those values instead of the default ones. So process.env.EXPECTED_NETWORK
will evaluate to private
in testing and be ropsten
otherwise. More info here.
As we already mentioned, we need [web/dev/local-deploy.js](https://github.com/ledfusion/dip-dapp-doe/blob/master/web/dev/local-deploy.js)
to replace the CONTRACT_ADDRESS
placeholder by the contract’s local address. The main difference with the deployment script we already wrote in blockchain/deploy/lib.js
is the following function:
function setContractAddressToEnv(contractAddress) {if (!contractAddress) {throw new Error("Invalid contract address")}const filePath = path.resolve(__dirname, "..", ".env.test.local")
**let** data = fs.readFileSync(filePath).toString()
**const** line = /CONTRACT\_ADDRESS=\[^\\n\]+/
data = data.replace(line, \`CONTRACT\_ADDRESS=${contractAddress}\`)
fs.writeFileSync(filePath, data)
}
Every time we run test
, the .env.test.local
file is updated, and there is no code to modify.
What if I want to just develop on a version of the dapp using the local blockchain?
Two versions of the dev
task are available on the web folder’s taskfile on GitHub.
run dev
will provide an environment identical to the one used to run the tests, but leaving the browser open for yourun dev ropsten
will simply run Parcel’s dev server and rely on Chrome/Firefox’s MetaMask as any user would doCreate the web/test/frontend.spec.js
file and copy the following content into it:
Ready? Type run test
and see the magic happen :)
Everything we need is ready for us. To keep the article readable, we will not elaborate on every use case. Feel free the check the spec file on GitHub.
What happens next?
We could approach the specs by starting a game, switching to another account; accepting the game, switching account back again, etc. However, this could lead to overcomplex specs and check a behaviour that users will not experience like that. We’d rather focus in one player’s experience and make sure that all relevant use cases are checked.
To simulate the actions of the opponent, we will launch the corresponding transactions from the NodeJS testing script. So the approach we will follow looks like:
web/test/frontend.spec.js
to accept the game from accounts[1]
So how would such use-case test look like?
Writing UI specs like this can be slow at the beginning, but the effort pays off as soon as you have simulated 5 complete games in less than a minute.
A few things to note:
Given the following spec, we code the behaviour of the frontend accordingly.
Let’s watch the movie of our test case playing against itself:
Doesn’t it remind you to a well-known film?
After the first use case is tested, the slope doesn’t look steep anymore :)
What’s left for us is to spec the remaining use cases, code the frontend accordingly and bundle the static web site. Using the building blocks explained in part 2, the rest of the frontend functionality can be developed without major issues.
When our specs are ready and development is on the go, we see that it would be good to show the “Withdraw” button only when it hasn’t been done already. However, this means that we need to add a get operation to the smart contract.
What does it mean for us at this point?
blockchain/test/dipDappDoe.js
blockchain/contracts/DipDappDoe.sol
run test
on the blockchain
folderweb/test/frontend.spec.js
web/src/game.js
run test
Updates on the contract will immediately reflect on the frontend’s code, and automated testing will ensure that we broke nothing in about one minute.
Once we are happy with specs, results and UI performance, it’s time to think of distributing our dapp to the world. The first step is to use Parcel to bundle it with production settings:
function build {echo "Recompiling the contracts"cd ../blockchain./taskfile build > /dev/nullcd ../web
echo "Cleaning the build folder"rm ./build/*
echo "Bundling the web site"NODE_ENV=production parcel build -d ./build --log-level 2 --no-source-maps src/index.html}
Next, it is time to quick check that everything actually looks as expected, including the attachment to the Ropsten network:
function www {build
serve build -p 1234}
Navigate to http://localhost:1234/ and see that everything is Okay. These are the static files of our dapp:
At this point, we could simply upload these files to Netlify, Surge, S3 or whatever provider you like. Once our domain name pointed to the hosting IP address and the TLS certificate is ready, you should not worry about data integrity anymore, right? If nobody updates your git repo, your provider sticks to the SLA and corrupt governments don’t censor your web site, everything is fine.
However, it is a bit inconsistent that our dapp uses a smart contract that runs on a decentralized blockchain while it remains accessible through a centralized web site that a big fish could turn down.
This is one other main reasons why IPFS exists. IPFS stands for InterPlanetary FileSystem and it is conceived with the aim of making content freely and reliably accessible across the globe. It has many advantages and some drawback, but for educational purposes, we will go through one of the most popular decentralized filesystems.
In a similar way to a blockchain, the IPFS network is made a lot of nodes around the world that nobody controls. They act as a global peer to peer network, in which files are invoked by their hash. You can think of it like a Git + BitTorrent architecture that also provides an HTTP gateway.
Without further introduction, let’s jump into it. First, install the IPFS CLI:
$ curl -O https://dist.ipfs.io/go-ipfs/v0.4.17/go-ipfs_v0.4.17_darwin-amd64.tar.gz
$ tar xvfz go-ipfs_v0.4.17_darwin-amd64.tar.gz$ cd go-ipfs$ ./install.sh
Let’s init our local repository:
$ ipfs initinitializing IPFS node at /Users/jordi/.ipfsgenerating 2048-bit RSA keypair...donepeer identity: QmUMkM9Px3touHUaWjB5yKi1qRbVwA9zRk8gjAndzkAy9wto get started, enter:
ipfs cat /ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv/readme
What happens if we run the last line as a command?
Hello and Welcome to IPFS!
██╗██████╗ ███████╗███████╗██║██╔══██╗██╔════╝██╔════╝██║██████╔╝█████╗ ███████╗██║██╔═══╝ ██╔══╝ ╚════██║██║██║ ██║ ███████║╚═╝╚═╝ ╚═╝ ╚══════╝
If you're seeing this, you have successfully installedIPFS and are now interfacing with the ipfs merkledag!
-------------------------------------------------------| Warning: || This is alpha software. Use at your own discretion! || Much is missing or lacking polish. There are bugs. |
Not yet secure. Read the security notes for more. |
---|
Check out some of the other files in this directory:
./about./help./quick-start <-- usage examples./readme <-- this file./security-notes
Several things happened:
QmS4ustL54...
is the hash of an IPFS folder, and it depends on the contents of all of its filesQmS4ustL54.../readme
resolves the IPFS folder and retrieves the hash of the readme
fileAt the time, we are only using IPFS as a local client. Any content that is already in our repository will resolve immediately. But if we don’t have it, its hash will be requested to the network, transferred and eventually cached in our local repository. If nobody uses it for a while, it may be garbage collected.
How do we add our files and become content providers?
Let’s run the following command and see what happens:
IPFS has hashed our files, computed the hash of the root folder and added them to our local repository. If we cat
its content locally, this is what we get:
However, what happens if we add a simple space+line to index.html
?
You guessed it, the hash of index.html
is radically different, and the hash of the root folder too. Any attempt to alter data integrity will always generate new hashes.
But how do we access these files from a web browser?
IPFS provides an HTTP and HTTPS gateway. Any file or folder can be navigated to with a URL like: https://ipfs.io/ipfs/<hash>
. However, if we try to access this URL with the hash of the root folder, our browser will keep waiting forever because no node has such content yet.
Yes, we are not a reachable node yet. To join the network and provide our content, we need to start IPFS as a daemon. Open a new terminal tab and leave it running:
$ ipfs daemon
If we now visit https://ipfs.io/ipfs/<hash>
, it may take a few seconds but it will load. But not quite:
Everything has been fine when running on localhost
, but it turns out that ParcelJS is expecting the bundles to be available from the root folder of the server. But now we are under /ipfs/<hash>
.
A little change in web/taskfile > build
should make the difference:
# ...
NODE_ENV=production parcel build [...] --public-url ./ src/index.html
And then again, rebuild and add to IPFS:
Let’s copy the new hash and see what happens now:
DipDappDoe served from the IPFS gateway
After a bit of patience, our first request will finally complete and our dapp will be running! Subsequent requests will be much faster. What happens now?
If we run ipfs pin ls
we will get the following:
IPFS allows nodes to pin
files, so their content is never garbage collected on it. In our case, we have the two versions of our build
folder and the sample data created on ipfs init
.
Note that the indirect
entries are files contained in other IPFS folders. They are pinned, only because another pinned element contains them. recursive
entries correspond to the explicitly pinned folders.
Now, our daemon is running and our content is accessible, but what happens if we stop it? Any content that has not been accessed yet will remain unavailable. The files of the dapp we just visited will remain reachable for a few hours until the network nodes mark them as unused and clean them up.
Unused content will continue to be stored and available as long as an active node keeps them pinned.
Telling the world to connect to a different URL when the web site is updated, is not much convenient. Isn’t there any better?
IPFS provides the Inter-Planetary Name System (IPNS) mechanism. An IPNS hash acts as an alias to an IPFS hash, with the difference that it can be updated over time. An IPNS hash can only be updated from the account that created it, as it is signed with the user’s private key.
$ ipfs name publish QmbVfUBSHp42kYtDud9zr1pxedd4dgqDmAHuRqHRPKGywTPublished to QmUMkM9Px3touHUaWjB5yKi1qRbVwA9zRk8gjAndzkAy9w: /ipfs/QmbVfUBSHp42kYtDud9zr1pxedd4dgqDmAHuRqHRPKGywT
From now on, the IPNF hash QmUMkM9Px3...
will resolve to the file /ipfs/QmbVfUBSHp42...
In the browser, if we navigate to https://ipfs.io/ipns/QmUMkM9Px3...
will be the same as navigating to https://ipfs.io/ipfs/QmbVfUBSHp42...
If at a later time, we need to update the frontend and use another IPNS hash, we will need to repeat the steps above with the new one. Existing users will continue to use the same URL.
However the IPNS approach still presents a few issues.
A more desirable scenario could be to use our domain name instead of the hash on IPNS. To that end, IPFS allows using DNS TXT
records to indicate what IPFS resource should be served.
If our domain was dapp.game
, we would add a TXT
record that should contain a string like:
dnslink=/ipfs/<hash>
When changes propagate through the net, the IPFS gateway will be able to fetch the TXT record of the given domain and use the underlying hash. Then our dapp should be available via [https://ipfs.io/ipns/dapp.game/](https://ipfs.io/ipns/dapp.game/.)
. Easy to recognize, easy to check.
But yet, as we still use the ipfs.io
domain, the third issue above still remains.
To achieve the most user friendly approach, we would need the dapp to be accessible via dapp.game
, but then we will be in front of a tradeoff. TLS or IPFS.
dapp.game
can be CNAME
‘d to gateway.ipfs.io
but this will only work on HTTP.
**Own server**We could get a TLS certificate from LetsEncrypt, start a local IPFS node and use Nginx to proxy external requests to IPFS, but that defeats the advantages of using IPFS.
Workload, security and data integrity depend on our centralized server, which becomes the bottleneck. Netlify, Firebase or Amazon are much stronger candidates than your own server to host the static site.
It is true that the IPFS gateway could be considered as a central point as well, but it is backed by a decentralized network of nodes and has successfully overcome DDoS attacks and censorship attempts.
Hosting the static files on our domain would mitigate potential XSS vulnerabilities, but it would expose our server to threats that IPFS has already handled in the past. More info.
**IPFS HTTP gateway**On the other hand, DipDappDoe does not rely on external resources beyond the blockchain. XSS should not be an issue for DipDappDoe but communication through HTTP opens the door to DNS hijacking and Man in the Middle attacks.
The final decision will depend very much on the way the dapp is built and what kind of users will interact with it.
https://ipfs.io/ipns/dapp.game/
may be suitable if your dapp can not leak any information to any XSS attacker, does not load any dynamic content and your users don’t mind copying or typing slightly longer URLs.CNAME
-ing our domain to the IPFS gateway should be avoided as this will only run on HTTP.As you have seen, writing a simple dapp is far from simple. The list of technology involved is not short:
DipDappDoe is an effort to cover the entire process of building a fully functional dapp with the minimum viable technology.
In part 1 we learnt how to use the TDD methodology to develop the smart contracts of the dapp. In part 2 we saw how to deploy the contracts to a decentralized blockchain and we bootstrapped the architecture of the dapp’s frontend. In part 3 we have followed the TDD methodology again to develop the frontend of the dapp and have used a decentralized filesystem like IPFS to publish it to the world.
Now that our MVP is ready, what might be next?
Obviously, a blockchain version of the Tic-Tac-Toe will not be as exciting as a centralized real-time version. The core value of our version is provide a provably fair game powered by smart contracts that everybody can trust. Our main goal is to demonstrate the full stack of a distributed app and see how to use the building blocks at our disposal.
If DipDappDoe was a real project, there would be many, many details to improve and work on at this point.
The end.
Writing this series of articles has involved a big effort and countless hours of work. I’m honored to see that you made it to the end 🙂
If you found the series of articles interesting and useful, please stand up, clap your hands 👏, smile 😊 like a hero and share with the same 💚 that I’ve put into this piece of work. Thank you!
As said earlier, the project featured on the article can be found in the GitHub repo:
ledfusion/dip-dapp-doe_dip-dapp-doe — Distributed app featured in a Medium article_github.com
Stay tuned, because this is just the beginning of the above tech.
Photo by Timo Wagner on Unsplash