Oracles for Hyperledger Fabric with Convector Suite

How to gather data from the real-world (i.e.: APIs) and boost your smart contracts built with Convector Suite.

Hello all. Walter from WorldSibu here. Our open source community grows every day and as it matures along with the framework we can move to more complex topics.

This topic was recently discussed in the Discord chat so I decided to create an example of how to work with oracles in Convector Suite (not to be confused with “Oracle” the company 🧐).

This blog post is to help you understand what oracles are, how to integrate them, and get a real example up and running. I know how hard it is to find real examples of how to do blockchain in the enterprise — we try a lot to create repos like this one to guide developers in our Github.

I will be using Convector Smart Contracts, Hurley, NestJS, mockable.io, and of course, Hyperledger Fabric 1.4. Since oracles can be confusing at first I will try to explain and iterate in the data flow through multiple conceptual and technical diagrams.

No time to read? Go download the repo here!

When are oracles necessary

Oracles are the means by which a smart contract can access data from off the chain. A smart contract can’t rely on external data for computing — meaning that it needs deterministic data to compute and apply transactions to the ledger.

Consider the following scenario of what could happen if the smart contract could access an external resource like an API:

Bank A, Bank B, and Bank C get into a multi-party workflow.
A transaction depends on one value from the Central Bank.
Bank A sends the transaction to Smart Contract 1.
Bank A, Bank B, and Bank C run it. In a blockchain typically every member that will endorse a transaction need to emulate it locally on their peer.
Smart Contract 1 calls an external API from the Central Bank from every member of the network.
Bank 1 gets “10” as the response.
Bank 2 gets “10” as the response.
Bank 3 gets “11” as the response.
The transaction gets rejected. 🙅‍♂️

This may happen because external resources are not deterministic. They may change from time to time. Failures, changes, attacks, all can affect the computing of a smart contract.

Some people refer to this property by making this thought process: if I were to take all the historic data of the blockchain and re-compute it, the results should be the same.

However, if Central Bank was in the blockchain:

They add a record called “Value=10” to the ledger beforehand.
Smart Contract 1 references that value in the ledger.
Bank 1, Bank 2, and Bank 3 run the transaction.
Everybody gets the same result. ✅

This looks nice, but data needs to be put before its needed. What about on-demand off-chain data? That's where an oracle is helpful.

Solving the issue with oracles

An oracle allows for requesting external resources, save them, and compute based on that data. An oracle is more than just code — it’s a kind of trusted entity that everybody relies upon to “reflect” real-life data into the chain keeping the track of the requested resource and making the computing deterministic.

The transaction computes and requests the external resource through an event. The oracle computes the requests, gets the data, and “restarts” the transaction by calling the function __callback() inside of the original smart contract.

In a nutshell, it's like saying… “we start this transaction but need this value to finalize it. Hey Mr. oracle look and find the external resource for us and then store it in the blockchain and let us know to complete the transaction”.

We will look further into the flow later in the post.

Some common data that may be required from the outside includes:

Weather data.
IoT events.
Centralized database records.
News.
Payment proofs.

Example

A typical flow includes 2 main steps:

Step 1: Initiate the transaction and request the external resource.

Step 2: Get the response from the outside world and continue the transaction.

You would need these main components:

Your smart contract.
An oracle daemon (external web server listening to requests from the chain).
Your external resource (API for instance).

Step 1

Initiate the transaction and request the external resource from the smart contract.

Send the transaction that needs external data to complete.
The Smart Contract requests data from the outside through an event to the oracle daemon but it won't wait for the response.
The oracle daemon asynchronously requests for data (no more involvement for step 1).
The Smart Contract responds to the requesting user with a successful status if the consensus is reached.

Step 2

The oracle daemon gets the response from the outside world and continues the transaction.

External API responds with an external resource.
Sometimes oracles may handle retry and failure of requests.
The oracle daemon triggers a request to the function _callback() in Smart Contract 1.
The function triggers the smart contract to continue the transaction based on the input data.
The user gets an async notification of finality (optional).

Show me the code!

Download the source code here:

git clone https://github.com/worldsibu/convector-oracle

cd convector-oracle

There are a few key concepts:

The blockchain (Hyperledger Fabric network created with Hurley).
Your smart contract (carinsurance in Convector).
The oracle daemon (conv-oracle in Nestjs).
Your external resource (mockable.io).

The logic of the code is:

You call
```
create()
```
to create a car but the
```
.insuranceLevel
```
property will be set by the oracle by querying an external web service.
Once accepted the transaction, an event will be launched for the oracle daemon (nestjs) to fetch the data.
The oracle daemon is actively listening for events from the chaincode and once it gets a request to query data it goes to mockable.io to get a response and trigger
```
__callback()
```
.
The field
```
.insuranceLevel
```
will be set with the data returned by the API. Also, the value
```
dateOracleResponse
```
will be set to log when the oracle returned data.

Our model's structure is pretty simple:

And our controller looks like this:

A few things to notice:

In the function
```
create
```
we trigger an event
```
this.tx.stub.setEvent
```
.
That event is awaited by the oracle daemon (conv-oracle).The callback function will query for cars waiting for some identity to submit a transaction with the value for
```
insuranceLevel
```
.

Let’s explore the actual flow a bit more in detail:

To run the project first make sure to meet Hyperledger Fabric pre-requisites.

Then, you will need a mockable.io API running — https://www.mockable.io

In mockable.io you just need to follow the tutorial and configure it like this:

Notice the Verb in POST and the Path with bankapi.

Then just make sure to get it up:

Click on “stopped” — yeah the UX is weird, you click “stopped” to get it up.

Copy the url from the field

Path

including the

/bankapi

piece. This will emulate our external API that the oracle daemon will call.

Add a new file in

./packages/conv-oracle/src/.env

with the following content:

EXTERNAL_URL=<full-url>

This will tell the oracle daemon server where to fetch the data.

Ready to see it in action?

The first transaction will take a few seconds to instantiate the smart contract in the second organization.

npm install

# Start a blockchain network locally
npm run env:restart

# Install the smart contract
npm run cc:start

# Start the oracle daemon [ignore npx if you don't use npx]
npx lerna run start --scope conv-oracle --stream

# Make a call to see everything in action 
# YOU WILL NEED ANOTHER CONSOLE TAB 
hurl invoke carinsurance carinsurance_create "1" "volk" "1199"

Congratulations! You ran your first oracle with Hyperledger Fabric thanks to Convector! 🎊

Let’s explore the data — go in your browser to http://localhost:5084/_utils/#database/ch1_carinsurance/1

You will see something like:

Fields dateOracleResponse and insuranceLevel were set by the Oracle from the API.

Try some things to prove that it’s not cheating, turn off the oracle daemon (just type control+c) in the tab running the server.

Send another transaction:

hurl invoke carinsurance carinsurance_create "2" "volk" "1199"

And explore the results: http://localhost:5084/_utils/#database/ch1_carinsurance/2

Oracles are a great tool to have at your disposal — we make use of Convector’s ability to use Hyperledger Fabric’s events and with this bootstrapped nestjs oracle daemon you can start doing some neat things!

I hope this was useful and also to see you around in theDiscord communityHundreds of devs helping each other and learning together!

What to do next?

Just like blockchain infrastructure, oracle’s infrastructure is not trivial — in fact, that’s why we created Forma to solve the infrastructure pains without cutting out the decentralized benefits of a blockchain network. So here are my recommendations to what to focus on when creating oracles:

Create mechanisms for dynamic queries and dynamic models (responses).
Implement ways to encrypt queries and responses.
Secure the oracle identity and verify it in the blockchain (__callback() function) just like we advise in the identity-pattern here (this will protect you from hijacking since a request to get data is made through an event there’s no response so the only space for somebody to get in the middle is when the oracle queries the external API and then responds).
Create high availability for the oracle daemon, authentication and authorization, black and white-lists, network security, and queues to process requests and avoid duplication.
Log everything that happens and every oracle response.

Just to mention a few. Oracles are critical components of blockchain networks and as such they should be treated carefully.

Conclusion

What do you think? Real-life data is necessary for real-life blockchains. At WorldSibu we love exposing our experience and building along with the community.

If you want to learn more join the community!
Get to know about WorldSibu’s work in making blockchain easy for enterprises.