Scaling reliably during the World’s biggest sports events

Erlang Solutions works with some of the world’s leading online gambling & betting operators. See what we do.

“A busy day for an online betting shop is like Black Friday at Amazon, only prices change every second,” Martin Davies, Head of Technology at bet365 told us a year ago. During big sporting events, like football tournaments, that can go on for weeks with global interest and audience numbers, Amazon’s Black Friday pales in comparison.

Operators need to scale and innovate while serving massive numbers of punters on a daily and hourly basis. Here’s our tips on how to do that:

The innovation-scale dilemma:

Bookmakers must not only meet the demands of high frequency requests and traffic bursts, but also keep the tech-savvy, multi-channel customers engaged. This is where they come across their biggest challenge; the “innovation-scale dilemma.”

To meet user demand, the industry continuously creates and improves on new types of games and bets. This is when tech departments hit upon the dilemma. Over a certain number of users, the ability to innovate quickly is lost. Precious time is spent rewriting software so that it runs to scale, slowing release time.

To break this vicious cycle, the most innovative industry players — such as bet365 and William Hill — have chosen Erlang (or Elixir, a language based on the same underpinnings as Erlang) as their core technology. Erlang Solutions also supports global payment companies, like Klarna, messaging companies like ooVoo, and IoT companies such as Intel — all of whom face similar technical challenges to online bookmakers.

Read more on SBC News

Innovation at scale

“Innovate or die” is a reality of life for most consumer facing industries these days. As the pace of progress in the computing industry shows no signs of slowing down, disruption by innovative startups is an existential threat to established players in any market.

Online businesses are especially ripe for disruption as the tools and technologies required to start an online business are pretty much available for free with no strings attached. The online gambling and betting industry is no different. Given the recent surge in popularity of online gambling, especially during the World Cup, established players have to innovate just to stay where they are in the marketplace.

Every step of the way, user experience comes first. There is a relentless demand for it to be easier and smoother, regardless of backend challenges. This has left the gambling industry in search of the most efficient and effective ways of directing users to their objective, whether it’s registering, playing a game, placing a bet, cashing out that bet or making a deposit. Systems must be ultra-responsive and able to engage with customers when, where, and how they want.

Those who get this ‘right’ in comparison to their competitors enjoy higher customer engagement and retention, and hence profits. And for each operator, this journey to support customer-centricity through the tech stack is slightly different.

Some established operators have legacy systems that are hard to scale or modernise whilst maintaining uptime and quality of customer experience. Those operators would be wise to take a step back, review their technology choices and evaluate which of the bewildering range of tech, old and new, can best be deployed in helping them achieve these customer-centric goals. It can help to spend some time imagining what an ideal sportsbook architecture would look like, without the burden of legacy, and then mapping out an evolutionary roadmap to get there.

Read more on SBC News

Concurrency, scalability, and reliability

At any one time bet365’s systems are serving many hundreds of thousands of users live odds and results, while managing multiple data streams on the backend. In peak times, like the Super Bowl or the Grand National, the number of users swells by an order of magnitude. This is an awesome feat of concurrent engineering.

When it comes to concurrency, Erlang and Elixir, both built on the BEAM virtual machine, excel. They are both concurrency oriented functional languages, built to handle vast numbers of users at the same time. Users of Erlang across verticals as diverse as telecoms, adtech; financial payments; massive multiplayer online role playing gaming; and social media have all exploited its ability to provide impressive concurrency.

Modern betting infrastructures demand massive scalability, speed, and fault tolerance to run smoothly. Without scalability built into the system, operators can be left to rely on additional hardware to scale vertically, which is not a sustainable nor a cost effective approach.

Erlang and Elixir’s abilities around concurrency go hand-in-hand with their massive scalability. If Erlang isn’t the best tool for every job your system has, a distributed architecture that can plug into Erlang-based products can make your betting system quick to deploy and scale elastically. Tools built in Erlang, such as Riak KV, scale out, up and down predictably. Say goodbye to the headache of emergency hardware and unpredictable system performance under severe load.

Like the stock market, downtime for an online betting operator has immediate financial and reputational consequences. For online sports betting the provision of a ultra-reliable, real-time service is now a priority for bookmakers and punters alike. Just look at the booming in play market for sports events.

In this world, pauses of any kinds, for system failure, garbage collection, or queuing backlogs, are not acceptable. Online betting stacks must handle their constant torrents of data without impacting the system’s processes, or end users.

Erlang and Elixir’s concurrency, no-shared memory architecture and built-in ‘fail and recover’ approach make them behave extremely gracefully and predictably under highly variable stochastic load. In fact, Erlang can support data changing at four times other languages’ rate. This makes Erlang and Elixir ideal to build critical gambling and betting systems on.

Read more on SBC News

The best tool for the job

The software development community is going through a revolution of diversity and inclusion as more developers move away from the biggest programming languages in search of the right tool for the nuanced challenges of our new, technologically-literate world. The result: it’s becoming rarer for there to be de facto programming languages that organisations are forced to use because that’s what their developers community uses.

One-size-fits-all stacks will never scale indefinitely. The most important lesson we’ve learned working in the gaming industry has been realising that a silver bullet for solving every problem simply does not exist, and it’s important to pick a proper tool for the job. Erlang and Elixir are superb at scaling reliably and efficiently when betting operators need it to, but also complement other tools and languages which are better suited to other aspects of a system.

One needs to look no further than bet365, whose Systems Development Manager for Middleware, Andrew Deane, recently went on the record to say “When I joined bet365 4 years ago, the company was in its second wave of Erlang development.

Having proved the model with Erlang, the door was open to explore other languages. Continuing the mantra of the right tool for the right job, we’ve found use cases for Google’s Go in our notifications systems and testing tools and more recently for Elixir in our application tooling.”

Read more on SBC News

Erlang Solutions works with some of the world’s leading online gambling & betting operators. See what we do.

Originally published at www.erlang-solutions.com on June 22, 2018.