Why reputation systems don’t work — And how an old Hebrew saying can save the world of…

In this blog post I will analyze the two main approaches to Decentralized Oracle systems — Game theoretical and Mechanical approaches. Augur, TruthCoin and Gnosis will represent the first, while Oraclize and Chainlink will represent the second. My claim is both of the approaches don’t really work, and I will suggest a third alternative which I believe could be the future of Decentralized Oracles.

Game Theoretic Decentralized Oracles

This approach aims to disincentivize bad actors through a reputation system or other compensation mechanisms. You can see my case study of using the approach for a specific financial instrument here.

Augur — The first version

Whitepaper

Major points:

• Setting up a Quorum is by definition is very sybil attackable. Once you define a limit to reporters to an oracle feed, with any amount of reputation, one actor can immediately report from many different accounts, skew the result, and actually not be punished in any way by the mechanism because he’s achieved consensus with himself. Setting up these kinds of Quorum is like the opposite of PoW and comes from a very technical limitation of not wanting to run SVD on a large matrix.
• Nowhere is it mentioned that the participants in oracle reporting receive any incentive besides reputation. Well, you might say, what’s wrong with that ? REP tokens are tradable ! The thing is if the only value of reputation is to participate in the oracle reporting, and you gain no financial incentive from oracle reporting, then the only reason to earn REP would be to sell them afterwards. But who will be the buyer, unless this is a totally speculative market ? The only buyer that can actually achieve financial benefits by buying REP in this setup are fraudsters ! As by defrauding financial instruments they can receive cash-inflows out of their REP tokens.

Minor Points:

• “These fee payments are divided in half, and split into two Buy and Sell transaction outputs each: one sent to the market’s creator, and one sent to the market pool. “

This is way too much compensation for a market’s creator. Incentives in the oracle world are meant to make sure people will not turn to the dark fraudulent side, and you’re giving half the funds for doing that for someone who’s only part in the system is writing “What will be the USD/BTC price at timestamp X”.

• “To prevent collusion, the contents of Augur reports must be kept secret. To achieve this, after the user inputs his/her observations, the Report is encrypted by his/her local Augur software”

Security mechanisms implemented on the user’s software side are weak. If he really wants, the user can bypass the software and signal the network what he reported. What they should have emphasized is that by Sztorc consensus mechanism you earn from other people making mistakes, so you wouldn’t want to encourage them to be correct. Also, page 43 of the TruthCoin whitepaper deals with that issue much better through a cryptographic protocol.

Augur — The second version

Whitepaper

In this later version of the white paper that came up earlier this month they totally diverged from Truthcoin, and went for some kind of a hybrid approach. They first have the designated oracle, which is like a centralized oracle or could be even a mechanical oracle such as oraclize. I will refer to the process described afterwards though, as it is more relevant to the current discussion — the process of open reporting, dispute and forking.

Regarding the dispute mechanism, Consider the following strategies:

• Dispute or Treat attack -

Say a large financial entity is waiting for its funds — say 1M$ — to come back from a smart contract that depends on some oracled value — say the BTC-USD exchange rate. Now, as they have capital costs for any delay of delivery of that money, I threaten them that unless they will pay a 50$ commission, I will issue a dispute on the result of this oracle. They might just pay.

• Even worse — the Dispute or Treat 2.0

Same attacker as before, but now first he races to make the first public record on the chain from some account. Then he disputes it from another account, and is able to both collect the revenue for correctly disputing, and also delays everything in a week.

As for its general immunity to attacks, as Augur mention themselves,

״Since the maximum possible benefit to an attacker includes the unknowable quantity Ip, one can never be objectively certain that the oracle is secure against economically rational attackers ״

Because the oracle system is decoupled from any financial instruments that might be using it, it actually has no assurance of being secured.

They have one point which is excellent, and they go to great lengths to show it, and that is that REP market price will be determined by how beneficial it is to use the system for frauds. Assume that different attackers have different attack schemes with different profits and different reputation requirements. They will all be willing to pay these prices driving the REP token price up. Eventually, at some market price there won’t be any profitable attacks left. That’s the nice way of looking at it — The way Augur people look at it. The other way of looking at it is — yes, we will arrive at this REP price, but this means we literally went through ALL possible attacks using the system. This is like learning to drive by crashing into every possible obstacle on the way — Probably, if you survive you will be the best driver on Earth. But will you ?

For me this shows that a secondary token is an unnecessary utility. You could implement the same logics with just the native token, be it BTC or ETH or whatever platform you’re using. But more on that now when discussing TruthCoin.

TruthCoin

TruthCoin whitepaper

When explaining why you would need a secondary token, Sztorc says

“This fixed amount of “tradable reputation” has many benefits, the chief of which are [1] Sybil- attack immunity, and [2] the alignment of ownership and control (economic value added/destroyed translates directly to an economic reward/penalty). Third, it gives the system a way to penalize agents (by withdrawing their VTC) for laziness. Fourth, it eliminates the temporal dimension from all incentive calculations (payments can be compared with each other, regardless of when they take place); if not eliminated, this dimension would have presented catastrophic risks (namely, the “exit scam”6). “

All of these apply to just any other existing token, besides (3), which can also be implemented in a way that disincentivizes non-returning users.

Let’s discuss the Exit scam.

Exit scams are not prevented by having reputation be tradable. After all, people could have sold their reputable accounts, but still they decided to do the exit scam. Being tradable is not enough, it’s being tradable AND being more profitable than defrauding people. Actually, when you have tradable reputation, this is more like division of labor — I’m a respectable person, so I will earn reputation, and I will sell it to you, the fraud master, at a high price, because you are better than me at making scams.

But Exit scam is not really the right terminology for that — It’s the Always present scam. An actor may commit it at any point, given enough financial incentive.

Sztorc introduces a lot of rules that make life of REP owners hard. They need to constantly participate in all votes of a market, otherwise they lose their existing REP. If all people are honest, which should usually be the case, they actually don’t earn any more REP on making good votes.

This reminds me a little of the call for stricter Gun laws in the US. They will surely not prevent gangsters from owning guns, but they will prevent regular citizens from owning them. So you’re left with two communities that actually own guns — Gun enthusiasts, and Gangsters. This seems like the end-game for TruthCoin as well. But the healthy condition is actually to encourage as many people to hold TruthCoin, as the majority of honest oracles / wisdom of the crowd effect is achieved that way.

The essence of TruthCoin is the Sztorc consensus mechanism, and I don’t believe in it either. SVD is just a first year of College algebraic manipulation, not some higher entity from heaven that solves every problem in the world. The profile of a true fraudster is to report totally correct values all the time, besides making a one time hit and run. You have no assurance the VTC token prices are high enough to prevent that scenario. I think Paul Sztorc understands it, so by the end of the white paper you see him talking about a new concept, the ‘Miners as Voters’.

In the ‘Miners as Voters’, forget about all the reputation tokens. You just ask the miners to vote on all oracle decisions. This is:

• Not scalable. A quote from Truthcoin white paper:

“As all Miners must vote on everything, this concept does not scale very far. “

• Implicitly Centralized. The miners will still need some source of information to base their votes on. So, when they need to determine the BTC-USD exchange rate they will all probably go to Coinmarketcap because they’re lazy and they’re not very much compensated for it, also they want to do what the others are doing, and so centralization will return in the backdoor.

Gnosis

Gnosis used to be mentioned in the same breath with Augur and TruthCoin, but it seems like they gave up on the idea of Game Theoretic oracles. In their latest white paper there’s no real reference to Oracles, and this appears on their roadmap for 2018:

“We will look into oracle integrations with providers like Reality Keys or Oraclize. RealityKeys provides automated and human-verified data designed to enable a new generation of automated information services. Oraclize provides a way for smart contracts to break free of their constraints and gives them the ability to access all the data they need from the web without compromising their trustless nature.”

Mechanical Oracles

Good examples of what I call Mechanical Oracles are ChainLink and Oraclize. As they are not the main focus of my post I will group them together.

The idea of mechanical oracles is that instead of wasting a lot of efforts on incentivizing users to report to the blockchain real world events, you incorporate a built-in capability for the blockchain to ‘go outside of itself’ in a way and extract data safely from web pages, API endpoints, etc. I will not argue against the security of the process itself, which varies between the different implementations, but the idea of detaching incentive from data by parasitizing on existing data points is likely to hit back in the long run.

How hard should it be to hack cryptocompare.com once the incentive to change its displayed BTC-USD rate is high enough ? As we saw in the cryptocurrency mania scientists running bitcoin mining software in nuclear facilities, do you really believe no one in cryptocompare’s dev team will be tempted to ssh to a server, change the rate for a blip of a second and earn a lot of money in positions he opened on some related financial instrument ? That’s why these methods are OK technically but miss the point, in my opinion.

And now, for the promised Hebrew saying

I want to present a third approach as I don’t believe reputation systems are fixable. Any system that tries to prevents attacks in both a general way and a static predefined way is doomed to fail. As I described in a previous post, you might be able to tie specific financial tools with specific oracle feeds in a safe way. But I want to get back to one thing I wrote there -

Though Game-Theory-wise participants in repeating games behave much nicer than participants in a single round game, I believe the decentralized and anonymized nature of cryptocurrencies prevents effectively building reputation systems without over-centralization

Actually, the naming/branding point is maybe the most important here. Why not outsource reputation out of the system, so that it’s not following predefined rules whereby an actor can easily calculate gains and loss, and instead just make everything be based on building a brand ?

My offer is to have a marketplace where people can choose any domain to their liking for their oracle feed. Off-chain systems might track their reputation, their availability, and they may put stakes that make it easier for people to trust them — whether it is depositing money with a third party, publishing their details and names for people to believe a regulatory action can be called for in case of a fraud. Aggregate systems can also be built upon this — say coinmarketcap has a feed, cryptocompare has a feed, and so on, you could build an automatic on-chain aggregator that takes the median of these brands feeds and creates a new feed.

Now, anytime two parties plan to setup some financial instrument between them, e.g. The Volatility insurance I previously described, they agree upon an oracle and maybe some fallback oracles in case of unavailability. The oracle demands some fee for this service, but people would prefer using a trusted oracle than some parasitic price feed that steals its data, because they can’t trust they won’t tweak it. This is like the financial version of consumer products — most people would still prefer to pay for brands than go with cheap imitations. And in case there’s some parasitic copy of an oracle feed that is totally mechanic, the oracle feed owners should be able to ban it.

The reason an actor that wishes to make a bad move here is in a much worse condition than in explicit reputation systems is because he has no idea what repercussions he will face. He might lose all credibility and never earn anything again. The stakes are not totally known here.

And so this reminds me of the Hebrew saying, “Tov shem mishemen Tov” — Having a good name is better than having good oil. The oil in our case being reputation tokens.

There are two systems in development right now that seem to plug into this idea - Zap store and Aeternity. I’m not sure though how much the two are aligned they are with the line of thinking I presented here. Zap store are building their platform on Ethereum which might not currently have the prerequisites for an oracle feed marketplace as I described it — namely having the capability for oracle feed brands to ban mechanical copying of their data by parasitic entities. As for Aeternity, while their white paper is vague regarding what their vision is for oracles — at first I thought they were going for the ‘Miners as voters’ idea, but after pecking through their github I believe this best illustrates their approach.