Here’s How Libratus, a Poker-Playing AI, Bluffed Four Professional Texas Holdem Players

The world of technology is changing at a faster rate than we can possibly fathom. Long gone are the days when we were the sole trailblazers in a human-tech relationship when the incentive resided in our hands.

Now, the tables have turned and our creations have started to claim what’s rightfully theirs. No, I’m not talking about the robot apocalypse dystopian scenario; it’s about the increased involvement of machines in the process of creation. 

AI is coming...

You might have already guessed what type of technology I have in mind: artificial intelligence. In a digital revolution, AI has become probably the biggest, most active sector with never-ending innovations bombarding the entire industry. To name but one example, an AI-based software called Libratus took the initiative in its hands to defy all odds and devour four No-Limit Texas Hold’em pro players in a twenty-day competition.

Such pieces of AI software like Libratus are a clear milestone in our quest towards making the world as automated as it can get. With innovations like that, the notion of human-tech interconnectedness becomes ever-more realistic and attainable. Whether it’s a Wall Street financial speculation or professional gameplay of the video game, AI constantly proves its worth.

Why beating poker is extremely difficult for AI

Before this insanely difficult match between poker pros and AI, many artificial intelligence enthusiasts wouldn’t even dare dream about their software beating the poker gurus, not to mention in the most complicated version of the game.

Generally, even the basic video poker games are pretty difficult to beat because of the layout of the game: the players are always hiding their cards, not to mention their attempts to bluff and deceive their opponents. And when it comes to the No-Limit Texas Hold'em, the stakes are rising even higher.

According to Andrew Ng, contributing founder of Google’s AI lab, poker games are difficult for AI to beat because instead of the entire game, just a small portion of it is evident to the gamers. This creates an incredible challenge to the AI to study all the moves and possibilities of a hand which, in turn, makes predicting an optimal move impossible. 

So, to make use of its extraordinary computing capabilities, an AI randomizes its every action, making it difficult for its opponents to understand, when it is bluffing and when it is not. 

Libratus vs. four pro poker players

So, here’s where we come to this famous case when Libratus software managed to beat not one, not two, not even three, but four poker players in 2017. But not any part of this occurrence was ordinary by any stretch of the imagination - neither the players, nor the game, nor even the AI software itself.

Against Libratus at a casino in Pittsburgh sat the world’s best-known poker players such as Dong Kim, and three others. The game they played is no less than no-limit Texas Hold’em, which is rightfully considered the most complicated poker game for its complex betting strategies and hands.

As Kim himself elaborates, he felt like the program was literally seeing his cards, but not as if it was cheating or anything - “it was just that good,” says Kim. And as we’ve already mentioned, Libratus managed to beat Kim, as well as three other pro poker players, which is the first time in AI’s history.

But even the AI software wasn’t ordinary. In fact, it wasn’t one separate software that played against those four poker players. Noam Brown, a CMU student, and his professor, Tuomas Sandholm, created Libratus as a by-product of the three separate systems that worked seamlessly to ensure the ultimate result, which was basically to devour the pros. And there was even a human involvement in the process, but let’s not get carried away and discuss individual elements.

Learning the moves

The first important element responsible for discovering and testing all the possible hands in the game is called reinforcement learning. In today’s AI world, the most popular software that gets the biggest acclaim is called the deep neural network. The neural network allows the machine to mimic all the human actions and gestures and even surpass them at some point.

However, Libratus didn’t use deep neural networks for its operations. It was based on another type of AI called reinforcement learning. Basically, the software played the poker hands against itself, again and again, ultimately to perfect its knowledge base. 

But one difference from other similar platforms like, say, Google’s AlphaGo, is that Libratus didn’t play against humans to acquire the basic skills and then refine them on its own. Basically, it was given the rules of the game and then it had to learn everything from scratch. 

The most prominent factor in this process was playing random hands at an incredible frequency. And after trillions of played hands and months of vigorous training, Libratus reached the level of proficiency. And not only could it beat the pro players, but it could also play the most random hands that the pros couldn’t possibly guess.

Choosing the best moves

The second element in this process was a software called an “end-game solver”. When Libratus learned all the possible moves and hands on its own, it created a large database of hypothetical scenarios. However, AI didn’t have to go through every single one of them during the play and test the best suitable version.

With the help of the end-game solver, Libratus managed to focus its attention on the game and learn during the process too. This way, the number of possible strategies reduced exponentially as the game progressed, leaving only those scenarios that were suitable to the opponent’s hands. 

So, the two individual pieces of one AI software were quite sophisticated and capable of posing a challenge to the proficient poker gamers. However, they still weren’t effective enough to do the same with Kim and the likes of him. They could find the underlying patterns of every move Libratus made and use them to their advantage. 

Eliminating the patterns

To avoid that, the two creators behind Libratus devised the third platform, eliminating all the discernible patterns and similarities. Here’s how it worked: every evening, after the match, Brown and Sandholm would run their own algorithm that was created to detect those patterns and eliminate them. Usually, the process took the whole night, after which the patterns were no longer there.

So, as you can see, Libratus isn’t all about the artificial software and mechanization - it also involves a human touch, just like we noted earlier. And that’s the thing about AI: while many people believe it is a self-sufficient entity that can run for itself and be completely independent of human intervention, the reality is different. In real life, humans and AI work side-by-side, making certain adjustments to each other’s actions. In this case, humans are putting AI to the starting point, while AI reaches the finish line without the complications.

The future of AI is here

The future of the automated world is already making itself apparent. Humans and machines are breaking the ice between them, making the first steps towards contributing to each other’s activities. 

When the technological revolution started off in the early XIX century, the machinery slowly began to take over our work and be good at it as well. Now, AI is not only taking over what we’re doing, but it’s doing it on its own. And the Libratus case is the best example of this development.

When this AI software beat four professional poker players at no-limit Texas Hold’em, Libratus proved that it can bluff even more proficiently than any pro player, making its moves virtually impossible to predict. Who knows what the future has in store and which industry finds the next Libratus.