Into the Dark Forest: Engineering Working DeFi Arbitrage

1. Introduction — Into the Dark Forest 1. Introduction — Late one night, Dan Robinson, a researcher at the investment firm Paradigm, saw a distress call in the Uniswap Discord channel. Someone had accidentally sent tokens worth about $12,000 not to another user, but to the Uniswap contract itself, where they seemed irretrievably lost. to Robinson, however, saw a glimmer of hope. He realized that anyone could trigger the burn function on the contract, a public command that would force the contract to release the trapped tokens to the person who called it. burn Recognizing a chance to be a white-hat hero, Robinson prepared the rescue transaction. But he knew he wasn’t alone. The Ethereum mempool — the public waiting room for pending transactions — is a hunting ground for sophisticated bots known as “generalized frontrunners.” These predators continuously scan for any profitable action, instantly copy it, and replace the original address with their own to steal the reward. To outmaneuver them, Robinson devised a clever two-part transaction, hoping to get both parts mined simultaneously, leaving no window for interception. It didn’t work. The moment his first transaction appeared, a bot detected it, replicated the entire strategy, and snatched the $12,000 before Robinson’s second transaction could even be confirmed. His rescue attempt had been expertly ambushed and devoured. In a now-famous essay about the incident, Robinson gave this hostile, invisible ecosystem a name. He had discovered a fundamental truth of the blockchain: Ethereum is a Dark Forest. Ethereum is a Dark Forest. In this article, we’ll delve into the design of a system capable of reading real-time operations and data from blockchain transactions, with the goal of performing arbitrage. We’ll explore the mathematical models underlying AMM pricing, delve into the algorithms for opportunity detection and optimal price entry, define the architectural components of our bot, and discuss the critical strategies required to successfully and securely execute arbitrage in this high-stakes environment. mathematical models underlying AMM pricing algorithms for opportunity detection and optimal price entry architectural components critical strategies 2. The DeFi Landscape: AMMs, Liquidity, and Arbitrage Opportunities The “Dark Forest” described in our introduction isn’t just a hostile environment; it’s a vibrant ecosystem built upon a novel financial paradigm: Decentralized Finance (DeFi). At its core, DeFi aims to recreate traditional financial services on blockchain networks, eliminating the need for intermediaries through the use of self-executing smart contracts. Understanding the fundamental building blocks of this landscape is crucial to comprehending how arbitrage opportunities arise and why they are so fiercely contested. Automated Market Makers (AMMs): The Backbone of Decentralized Exchange Traditional exchanges rely on order books, where buyers and sellers place bids and asks, and a central matching engine facilitates trades. DeFi introduces a radically different model: the Automated Market Maker (AMM). Instead of matching buyers and sellers directly, AMMs leverage liquidity pools — smart contracts holding reserves of two or more tokens — to facilitate trades. Users, known as liquidity providers (LPs), deposit equivalent values of token pairs into these pools, earning a share of trading fees in return. The price of assets within an AMM pool is determined algorithmically by a constant product formula, pioneered by Uniswap: constant product formula Here, x and y represent the quantities of the two tokens in the liquidity pool, and k is a constant. When a user trades one token for another, the quantities of x and y in the pool change, but their product k must remain constant. This mechanism dynamically adjusts the price: buying more of token A will decrease its quantity in the pool, thus increasing its price relative to token B, and vice-versa. This relationship between the reserves and the price creates a bonding curve, which dictates the available price points for trades. bonding curve From this model, it’s possible to deterministically calculate the output amount (dy) from a swap, given the input amount (dx) and the pre-swap reserves of the two tokens (x and y): Key characteristics of AMMs: Key characteristics of AMMs: Always-on Liquidity: Unlike order books that can become thin, AMMs always offer liquidity as long as there are tokens in the pool. Permissionless: Anyone can become a liquidity provider or trade on an AMM without needing approval. Price Discovery: Prices are determined by the ratio of assets within the pool, adjusting with each trade. Slippage: Large trades can significantly move the price within a pool, leading to a phenomenon known as slippage, where the executed price is worse than the quoted price. This is a critical factor for arbitrage bots. Always-on Liquidity: Unlike order books that can become thin, AMMs always offer liquidity as long as there are tokens in the pool. Always-on Liquidity: Permissionless: Anyone can become a liquidity provider or trade on an AMM without needing approval. Permissionless: Price Discovery: Prices are determined by the ratio of assets within the pool, adjusting with each trade. Price Discovery: Slippage: Large trades can significantly move the price within a pool, leading to a phenomenon known as slippage, where the executed price is worse than the quoted price. This is a critical factor for arbitrage bots. Slippage: While the x⋅y=k model (often referred to as Uniswap V2) laid the groundwork, AMMs have evolved. Uniswap V3, for instance, introduced “concentrated liquidity” (CLAMM), allowing LPs to allocate their capital within specific price ranges. This significantly improved capital efficiency but also increased the complexity for LPs and, consequently, for arbitrageurs needing to track liquidity across various ranges. In this article’s implementation, we will primarily focus on AMMs utilizing the constant product formula (like Uniswap V2-style pools), as they provide a foundational understanding before tackling more complex models. we will primarily focus on AMMs The Essence of Arbitrage in DeFi The Essence of Arbitrage in DeFi Arbitrage, in its purest form, is the simultaneous purchase and sale of an asset in different markets to profit from a disparity in its price. In DeFi, this translates to exploiting price discrepancies between different AMM pools, or between an AMM and a Centralized Exchange (CEX), for the same token pair. The inherent permissionless nature of DeFi and the fragmented liquidity across various protocols create a fertile ground for these opportunities. The high volatility and lack of regulation in this nascent financial space often lead to significant price deviations, which are the lifeblood of arbitrageurs. Types of Arbitrage Opportunities in DeFi Simple Arbitrage (Two-Leg): This is the most straightforward form, involving two different venues. For example, if 1 ETH trades for 2000 DAI on Uniswap A, but 1 ETH trades for 2010 DAI on Uniswap B, an arbitrageur can buy ETH on Uniswap A with DAI, and immediately sell that ETH for DAI on Uniswap B, pocketing the 10 DAI difference (minus gas fees and slippage). Triangular Arbitrage (Multi-Leg): This type of arbitrage involves three or more assets within the same exchange (or across multiple exchanges) to form a profitable cycle. For instance, an arbitrageur might start with Token A, swap it for Token B, then Token B for Token C, and finally Token C back to Token A, ending up with more of Token A than they started with. A common example on Uniswap might be: WETH -> DAI -> USDC -> WETH. Our primary objective is to implement and operate Multi-Leg arbitrage between different DEX AMMs. Flash Loan Arbitrage: A powerful and unique aspect of DeFi, flash loans allow users to borrow uncollateralized assets, use them for a series of transactions (like an arbitrage), and repay the loan — all within a single blockchain transaction. If the entire sequence of operations (borrow, trade, repay) cannot be completed successfully within that single transaction, the entire transaction is reverted, as if it never happened. This eliminates the need for significant upfront capital and significantly lowers the barrier to entry for large-scale arbitrage, but it also means the entire arbitrage strategy must be carefully orchestrated within one atomic operation. We will incorporate a flash loan option into our bot, particularly for situations where the required arbitrage amounts are very high, enabling us to execute trades far exceeding our initial capital. Simple Arbitrage (Two-Leg): This is the most straightforward form, involving two different venues. For example, if 1 ETH trades for 2000 DAI on Uniswap A, but 1 ETH trades for 2010 DAI on Uniswap B, an arbitrageur can buy ETH on Uniswap A with DAI, and immediately sell that ETH for DAI on Uniswap B, pocketing the 10 DAI difference (minus gas fees and slippage). Simple Arbitrage (Two-Leg): Triangular Arbitrage (Multi-Leg): This type of arbitrage involves three or more assets within the same exchange (or across multiple exchanges) to form a profitable cycle. For instance, an arbitrageur might start with Token A, swap it for Token B, then Token B for Token C, and finally Token C back to Token A, ending up with more of Token A than they started with. A common example on Uniswap might be: WETH -> DAI -> USDC -> WETH. Our primary objective is to implement and operate Multi-Leg arbitrage between different DEX AMMs. Triangular Arbitrage (Multi-Leg): same Our primary objective is to implement and operate Multi-Leg arbitrage between different DEX AMMs. Flash Loan Arbitrage: A powerful and unique aspect of DeFi, flash loans allow users to borrow uncollateralized assets, use them for a series of transactions (like an arbitrage), and repay the loan — all within a single blockchain transaction. If the entire sequence of operations (borrow, trade, repay) cannot be completed successfully within that single transaction, the entire transaction is reverted, as if it never happened. This eliminates the need for significant upfront capital and significantly lowers the barrier to entry for large-scale arbitrage, but it also means the entire arbitrage strategy must be carefully orchestrated within one atomic operation. We will incorporate a flash loan option into our bot, particularly for situations where the required arbitrage amounts are very high, enabling us to execute trades far exceeding our initial capital. Flash Loan Arbitrage: single blockchain transaction We will incorporate a flash loan option into our bot The Race for Profit: Challenges and Competition The DeFi landscape is a highly efficient market. Price discrepancies are fleeting, often existing for mere milliseconds before being exploited by sophisticated bots. This intense competition presents several critical challenges for any aspiring arbitrageur: Gas Fees: Every interaction with a smart contract incurs a transaction fee (gas), which can vary significantly based on network congestion. A profitable arbitrage opportunity must yield enough profit to cover these costs. Slippage: The larger the trade relative to the pool’s liquidity, the greater the slippage, eroding potential profits. Accurately modeling slippage is crucial for calculating true profitability. Latency: The speed at which an arbitrage bot can detect an opportunity, calculate the optimal trade, construct a transaction, and submit it to the network is paramount. Even milliseconds can make the difference between profit and loss. Frontrunning and MEV: As discussed in the introduction, the “Dark Forest” is dominated by generalized frontrunners. These bots actively monitor the mempool for pending profitable transactions, replicate them, and submit their own transaction with a higher gas price to ensure their transaction is included in a block before the original one. This phenomenon falls under the umbrella of Maximal Extractable Value (MEV), representing the total value that can be extracted from block production in excess of the standard block reward and gas fees by arbitraging, liquidating, or reordering transactions within a block. Successfully navigating this environment often requires advanced strategies like leveraging MEV-Boost relays or private transaction pools. To mitigate the risk of being intercepted in public mempools, our implementation will primarily operate on Base, an EVM-compatible Layer 2 (L2) blockchain. Base’s architecture, which currently does not expose a public mempool in the same manner as Ethereum’s Layer 1, offers a different environment for transaction submission, potentially reducing traditional frontrunning risks. Complexity of AMMs: As AMMs evolve (e.g., Uniswap V3’s concentrated liquidity), the mathematical modeling and state tracking required for accurate arbitrage calculations become significantly more complex. Understanding these foundational elements of DeFi, from the mechanics of AMMs to the cut-throat nature of arbitrage competition, sets the stage for designing a robust and effective arbitrage bot. In the next chapter, we will begin to lay out the architectural blueprint for such a system. Gas Fees: Every interaction with a smart contract incurs a transaction fee (gas), which can vary significantly based on network congestion. A profitable arbitrage opportunity must yield enough profit to cover these costs. Gas Fees: Every interaction with a smart contract incurs a transaction fee (gas), which can vary significantly based on network congestion. A profitable arbitrage opportunity must yield enough profit to cover these costs. Gas Fees: Slippage: The larger the trade relative to the pool’s liquidity, the greater the slippage, eroding potential profits. Accurately modeling slippage is crucial for calculating true profitability. Slippage: The larger the trade relative to the pool’s liquidity, the greater the slippage, eroding potential profits. Accurately modeling slippage is crucial for calculating true profitability. Slippage: Latency: The speed at which an arbitrage bot can detect an opportunity, calculate the optimal trade, construct a transaction, and submit it to the network is paramount. Even milliseconds can make the difference between profit and loss. Latency: The speed at which an arbitrage bot can detect an opportunity, calculate the optimal trade, construct a transaction, and submit it to the network is paramount. Even milliseconds can make the difference between profit and loss. Latency: Frontrunning and MEV: As discussed in the introduction, the “Dark Forest” is dominated by generalized frontrunners. These bots actively monitor the mempool for pending profitable transactions, replicate them, and submit their own transaction with a higher gas price to ensure their transaction is included in a block before the original one. This phenomenon falls under the umbrella of Maximal Extractable Value (MEV), representing the total value that can be extracted from block production in excess of the standard block reward and gas fees by arbitraging, liquidating, or reordering transactions within a block. Successfully navigating this environment often requires advanced strategies like leveraging MEV-Boost relays or private transaction pools. To mitigate the risk of being intercepted in public mempools, our implementation will primarily operate on Base, an EVM-compatible Layer 2 (L2) blockchain. Base’s architecture, which currently does not expose a public mempool in the same manner as Ethereum’s Layer 1, offers a different environment for transaction submission, potentially reducing traditional frontrunning risks. Frontrunning and MEV: As discussed in the introduction, the “Dark Forest” is dominated by generalized frontrunners. These bots actively monitor the mempool for pending profitable transactions, replicate them, and submit their own transaction with a higher gas price to ensure their transaction is included in a block before the original one. This phenomenon falls under the umbrella of Maximal Extractable Value (MEV), representing the total value that can be extracted from block production in excess of the standard block reward and gas fees by arbitraging, liquidating, or reordering transactions within a block. Successfully navigating this environment often requires advanced strategies like leveraging MEV-Boost relays or private transaction pools. To mitigate the risk of being intercepted in public mempools, our implementation will primarily operate on Base, an EVM-compatible Layer 2 (L2) blockchain. Base’s architecture, which currently does not expose a public mempool in the same manner as Ethereum’s Layer 1, offers a different environment for transaction submission, potentially reducing traditional frontrunning risks. Frontrunning and MEV: our implementation will primarily operate on Base Complexity of AMMs: As AMMs evolve (e.g., Uniswap V3’s concentrated liquidity), the mathematical modeling and state tracking required for accurate arbitrage calculations become significantly more complex. Understanding these foundational elements of DeFi, from the mechanics of AMMs to the cut-throat nature of arbitrage competition, sets the stage for designing a robust and effective arbitrage bot. In the next chapter, we will begin to lay out the architectural blueprint for such a system. Complexity of AMMs: As AMMs evolve (e.g., Uniswap V3’s concentrated liquidity), the mathematical modeling and state tracking required for accurate arbitrage calculations become significantly more complex. Complexity of AMMs: Understanding these foundational elements of DeFi, from the mechanics of AMMs to the cut-throat nature of arbitrage competition, sets the stage for designing a robust and effective arbitrage bot. In the next chapter, we will begin to lay out the architectural blueprint for such a system. 3. Architectural Design: Building the Arbitrage Bot Infrastructure Building a profitable arbitrage bot in the “Dark Forest” of DeFi demands an architecture that prioritizes speed, reliability, and precision. Every millisecond counts, and the ability to process real-time data, identify opportunities, and execute trades swiftly is paramount. Our system is engineered with these imperatives at its core, leveraging the concurrency model of Go and a modular, event-driven design. Go was chosen as the primary development language due to its exceptional performance, robust concurrency primitives (goroutines and channels), and strong ecosystem for network programming and low-level system interactions. These features are critical for handling the high throughput of blockchain data and the need for parallel processing in a real-time arbitrage system. Furthermore, Go’s efficiency is evidenced by its adoption in core blockchain infrastructure, such as go-ethereum, the primary Ethereum client. go-ethereum The bot’s architecture is structured as an event-driven system composed of several independent services (modules), each running in parallel processes (goroutines). These services communicate asynchronously by sending messages through Go channels, ensuring a loosely coupled and highly responsive design. This approach allows for efficient resource utilization, simplifies fault isolation, and enables seamless scaling of individual components. event-driven system Overall System Architecture The arbitrage bot’s infrastructure can be visualized as a pipeline, where data flows from the blockchain, is processed and analyzed, and culminates in the execution of profitable trades. The core components, operating in parallel, are: Blockchain Data Reader Service: Responsible for real-time ingestion of blockchain events data. Market Graph Service: Maintains an in-memory representation of the DeFi market and identifies arbitrage paths. Arbitrage Strategy Service: Evaluates detected opportunities for profitability and prepares trade instructions. Transaction Builder Service: Constructs and dispatches blockchain transactions. Honeywall Service: A post-execution checker that enhances security and maintains market integrity by identifying and blacklisting malicious pools. This modularity allows each service to focus on a specific task, minimizing dependencies and optimizing performance for its particular workload. Communication between services is strictly asynchronous, leveraging Go’s channels for message passing, which naturally facilitates a non-blocking and highly concurrent operation. Blockchain Data Reader Service: Responsible for real-time ingestion of blockchain events data. Blockchain Data Reader Service: Responsible for real-time ingestion of blockchain events data. Blockchain Data Reader Service: Market Graph Service: Maintains an in-memory representation of the DeFi market and identifies arbitrage paths. Market Graph Service: Maintains an in-memory representation of the DeFi market and identifies arbitrage paths. Market Graph Service: Arbitrage Strategy Service: Evaluates detected opportunities for profitability and prepares trade instructions. Arbitrage Strategy Service: Evaluates detected opportunities for profitability and prepares trade instructions. Arbitrage Strategy Service: Transaction Builder Service: Constructs and dispatches blockchain transactions. Transaction Builder Service: Constructs and dispatches blockchain transactions. Transaction Builder Service: Honeywall Service: A post-execution checker that enhances security and maintains market integrity by identifying and blacklisting malicious pools. This modularity allows each service to focus on a specific task, minimizing dependencies and optimizing performance for its particular workload. Communication between services is strictly asynchronous, leveraging Go’s channels for message passing, which naturally facilitates a non-blocking and highly concurrent operation. Honeywall Service: A post-execution checker that enhances security and maintains market integrity by identifying and blacklisting malicious pools. Honeywall Service: This modularity allows each service to focus on a specific task, minimizing dependencies and optimizing performance for its particular workload. Communication between services is strictly asynchronous, leveraging Go’s channels for message passing, which naturally facilitates a non-blocking and highly concurrent operation. Blockchain Data Reader Service: The Eyes and Ears of Our Bot in the Data Stream This service acts as the bot’s primary interface with the raw, real-time data flowing through the blockchain. In the “Dark Forest,” information is currency, and our ability to quickly and accurately ingest it is paramount. We don’t just “read” the blockchain; we actively extract crucial financial data points that will feed our arbitrage decision-making engine. extract crucial financial data points Connection and Data Ingestion: The Reader connects to a blockchain node via WebSockets. This persistent, bi-directional connection allows for immediate reception of new block headers and, more importantly, event logs emitted by smart contracts. The service is configured to specifically listen for Swap, Mint, Burn, and Sync events from Decentralized Exchange (DEX) smart contracts. These events are crucial as they indicate changes in the reserves of liquidity pools, directly impacting token prices. New Block Headers: By subscribing to new block headers, we get immediate notification of state changes. Each new block represents a confirmed snapshot of the blockchain’s current reality, including new transactions, updated balances, and new liquidity pool states. This stream provides the foundational data for our market graph’s periodic updates and for confirming the outcome of our own transactions. Connection and Data Ingestion: The Reader connects to a blockchain node via WebSockets. This persistent, bi-directional connection allows for immediate reception of new block headers and, more importantly, event logs emitted by smart contracts. The service is configured to specifically listen for Swap, Mint, Burn, and Sync events from Decentralized Exchange (DEX) smart contracts. These events are crucial as they indicate changes in the reserves of liquidity pools, directly impacting token prices. Connection and Data Ingestion: WebSockets Swap Mint Burn Sync New Block Headers: By subscribing to new block headers, we get immediate notification of state changes. Each new block represents a confirmed snapshot of the blockchain’s current reality, including new transactions, updated balances, and new liquidity pool states. This stream provides the foundational data for our market graph’s periodic updates and for confirming the outcome of our own transactions. New Block Headers: func (er *EthereumReader) SubscribePairs() error { parsedABI := constants.PairAbi // Set up the filter query := ethereum.FilterQuery{ Topics: [][]common.Hash{ { parsedABI.Events["Swap"].ID, parsedABI.Events["Mint"].ID, parsedABI.Events["Burn"].ID, parsedABI.Events["Sync"].ID, }, }, } logs := make(chan types.Log) sub, err := er.ethClient.SubscribeFilterLogs(context.Background(), query, logs) if err != nil { return err } // Start Routine to read swaps events log.Println("[READING SWAPS...]") go func() { for { select { case err = <-sub.Err(): log.Println("[RESET CONNECTION...] Subscription error: ", err) pairInfo := GraphMessage{ Ok: false, } *er.pairChan er.currentBlockNumber { // New block detected, reset cache er.lastUpdatedBlock = nil er.lastUpdatedBlock = make(map[common.Address]uint64) er.currentBlockNumber = blockNumber } // Check if already updated for this pair in current block if _, exists := er.lastUpdatedBlock[pairAddress]; exists { continue } t0, t1, f, r0, r1, err := er.getPairDataFromHelper(pairAddress) if err != nil { continue } dex := f.String() router, err := constants.GetRouterAddressFromFactory(dex) if err != nil { continue } // Update cache er.lastUpdatedBlock[pairAddress] = blockNumber elapsed := time.Until(start) pairInfo := GraphMessage{ Ok: true, DexCheck: true, Pair: pairAddress.Hex(), Token0: Token{Address: t0.Hex()}, Token1: Token{Address: t1.Hex()}, Reserve0: r0, Reserve1: r1, Dex: router, GetTime: elapsed, } *er.pairChan <- pairInfo } } }() return nil } func (er *EthereumReader) SubscribePairs() error { parsedABI := constants.PairAbi // Set up the filter query := ethereum.FilterQuery{ Topics: [][]common.Hash{ { parsedABI.Events["Swap"].ID, parsedABI.Events["Mint"].ID, parsedABI.Events["Burn"].ID, parsedABI.Events["Sync"].ID, }, }, } logs := make(chan types.Log) sub, err := er.ethClient.SubscribeFilterLogs(context.Background(), query, logs) if err != nil { return err } // Start Routine to read swaps events log.Println("[READING SWAPS...]") go func() { for { select { case err = <-sub.Err(): log.Println("[RESET CONNECTION...] Subscription error: ", err) pairInfo := GraphMessage{ Ok: false, } *er.pairChan er.currentBlockNumber { // New block detected, reset cache er.lastUpdatedBlock = nil er.lastUpdatedBlock = make(map[common.Address]uint64) er.currentBlockNumber = blockNumber } // Check if already updated for this pair in current block if _, exists := er.lastUpdatedBlock[pairAddress]; exists { continue } t0, t1, f, r0, r1, err := er.getPairDataFromHelper(pairAddress) if err != nil { continue } dex := f.String() router, err := constants.GetRouterAddressFromFactory(dex) if err != nil { continue } // Update cache er.lastUpdatedBlock[pairAddress] = blockNumber elapsed := time.Until(start) pairInfo := GraphMessage{ Ok: true, DexCheck: true, Pair: pairAddress.Hex(), Token0: Token{Address: t0.Hex()}, Token1: Token{Address: t1.Hex()}, Reserve0: r0, Reserve1: r1, Dex: router, GetTime: elapsed, } *er.pairChan <- pairInfo } } }() return nil } Custom Smart Contract for Data Aggregation and Pre-filtering: To optimize efficiency and reduce redundant on-chain calls, the Reader utilizes a custom smart contract, specifically written for this purpose. This contract serves as an aggregator, providing a single, optimized call to retrieve the reserves and other aggregated information for multiple liquidity pairs. A key functionality of this custom contract is its built-in pre-check for common scam characteristics or excessive trading taxes within a pool before returning the data. This preliminary filtering significantly reduces the risk of interacting with malicious contracts downstream, acting as a first line of defense against potentially harmful or unprofitable pools. Below is the Solidity implementation of this helper contract. The core logic resides in the checkPair method, which assesses the safety of a token pair and returns aggregated data. Custom Smart Contract for Data Aggregation and Pre-filtering: To optimize efficiency and reduce redundant on-chain calls, the Reader utilizes a custom smart contract, specifically written for this purpose. This contract serves as an aggregator, providing a single, optimized call to retrieve the reserves and other aggregated information for multiple liquidity pairs. A key functionality of this custom contract is its built-in pre-check for common scam characteristics or excessive trading taxes within a pool before returning the data. This preliminary filtering significantly reduces the risk of interacting with malicious contracts downstream, acting as a first line of defense against potentially harmful or unprofitable pools. Custom Smart Contract for Data Aggregation and Pre-filtering: scam characteristics excessive trading taxes before Below is the Solidity implementation of this helper contract. The core logic resides in the checkPair method, which assesses the safety of a token pair and returns aggregated data. checkPair // SPDX-License-Identifier: MIT pragma solidity ^0.8.20; contract ArbHelperMap { mapping(address => address) public factoryToRouter; address public owner; modifier onlyOwner() { require(msg.sender == owner, "Not owner"); _; } constructor() { owner = msg.sender; // Pre-populate known mappings factoryToRouter[0x8909Dc15e40173Ff4699343b6eB8132c65e18eC6] = 0x4752ba5DBc23f44D87826276BF6Fd6b1C372aD24; factoryToRouter[0x02a84c1b3BBD7401a5f7fa98a384EBC70bB5749E] = 0x8cFe327CEc66d1C090Dd72bd0FF11d690C33a2Eb; factoryToRouter[0xFDa619b6d20975be80A10332cD39b9a4b0FAa8BB] = 0x327Df1E6de05895d2ab08513aaDD9313Fe505d86; factoryToRouter[0x71524B4f93c58fcbF659783284E38825f0622859] = 0x6BDED42c6DA8FBf0d2bA55B2fa120C5e0c8D7891; factoryToRouter[0x3E84D913803b02A4a7f027165E8cA42C14C0FdE7] = 0x8c1A3cF8f83074169FE5D7aD50B978e1cD6b37c7; factoryToRouter[0x9A9A171c69cC811dc6B59bB2f9990E34a22Fc971] = 0x1b7655aa64b7BD54077dE56B64a0f92BCba05b85; } function addFactoryRouter(address factory, address router) external onlyOwner { require(factory != address(0) && router != address(0), "Zero address"); factoryToRouter[factory] = router; } struct Result { bool success; address token0; address token1; address factory; uint112 reserve0; uint112 reserve1; } // Helper function to get pair data function _getPairData(address pairAddress) private view returns ( bool success, address token0, address token1, address factory, uint112 reserve0, uint112 reserve1, address router ) { success = false; try IPair(pairAddress).token0() returns (address _token0) { token0 = _token0; try IPair(pairAddress).token1() returns (address _token1) { token1 = _token1; try IPair(pairAddress).factory() returns (address _factory) { factory = _factory; try IPair(pairAddress).getReserves() returns (uint112 r0, uint112 r1, uint32) { reserve0 = r0; reserve1 = r1; router = factoryToRouter[factory]; if (router != address(0)) { success = true; } } catch {} } catch {} } catch {} } catch {} } // Helper function to check if pair passes tax limit function _checkTaxLimit( address router, address token0, address token1, uint amountIn, uint maxTaxPermille ) private view returns (bool) { address[] memory path = new address[](2); path[0] = token0; path[1] = token1; try IRouter(router).getAmountsOut(amountIn, path) returns (uint[] memory buyOuts) { if (buyOuts.length < 2) return false; address[] memory reversePath = new address[](2); reversePath[0] = token1; reversePath[1] = token0; try IRouter(router).getAmountsOut(buyOuts[1], reversePath) returns (uint[] memory sellOuts) { if (sellOuts.length = minReturn; } catch { return false; } } catch { return false; } } function checkPair(address pairAddress, uint amountIn, uint maxTaxPermille) external view returns (Result memory r) { // Initialize result with default values r.success = false; // Skip processing if pair address is zero if (pairAddress == address(0)) return r; // Get pair data bool success; address token0; address token1; address factory; uint112 reserve0; uint112 reserve1; address router; (success, token0, token1, factory, reserve0, reserve1, router) = _getPairData(pairAddress); // If we couldn't get pair data or there's no router, return early if (!success) return r; // Check tax limits bool passedTaxCheck = _checkTaxLimit(router, token0, token1, amountIn, maxTaxPermille); // Populate result if tax check passed if (passedTaxCheck) { r.success = true; r.token0 = token0; r.token1 = token1; r.factory = factory; r.reserve0 = reserve0; r.reserve1 = reserve1; } return r; } } // SPDX-License-Identifier: MIT pragma solidity ^0.8.20; contract ArbHelperMap { mapping(address => address) public factoryToRouter; address public owner; modifier onlyOwner() { require(msg.sender == owner, "Not owner"); _; } constructor() { owner = msg.sender; // Pre-populate known mappings factoryToRouter[0x8909Dc15e40173Ff4699343b6eB8132c65e18eC6] = 0x4752ba5DBc23f44D87826276BF6Fd6b1C372aD24; factoryToRouter[0x02a84c1b3BBD7401a5f7fa98a384EBC70bB5749E] = 0x8cFe327CEc66d1C090Dd72bd0FF11d690C33a2Eb; factoryToRouter[0xFDa619b6d20975be80A10332cD39b9a4b0FAa8BB] = 0x327Df1E6de05895d2ab08513aaDD9313Fe505d86; factoryToRouter[0x71524B4f93c58fcbF659783284E38825f0622859] = 0x6BDED42c6DA8FBf0d2bA55B2fa120C5e0c8D7891; factoryToRouter[0x3E84D913803b02A4a7f027165E8cA42C14C0FdE7] = 0x8c1A3cF8f83074169FE5D7aD50B978e1cD6b37c7; factoryToRouter[0x9A9A171c69cC811dc6B59bB2f9990E34a22Fc971] = 0x1b7655aa64b7BD54077dE56B64a0f92BCba05b85; } function addFactoryRouter(address factory, address router) external onlyOwner { require(factory != address(0) && router != address(0), "Zero address"); factoryToRouter[factory] = router; } struct Result { bool success; address token0; address token1; address factory; uint112 reserve0; uint112 reserve1; } // Helper function to get pair data function _getPairData(address pairAddress) private view returns ( bool success, address token0, address token1, address factory, uint112 reserve0, uint112 reserve1, address router ) { success = false; try IPair(pairAddress).token0() returns (address _token0) { token0 = _token0; try IPair(pairAddress).token1() returns (address _token1) { token1 = _token1; try IPair(pairAddress).factory() returns (address _factory) { factory = _factory; try IPair(pairAddress).getReserves() returns (uint112 r0, uint112 r1, uint32) { reserve0 = r0; reserve1 = r1; router = factoryToRouter[factory]; if (router != address(0)) { success = true; } } catch {} } catch {} } catch {} } catch {} } // Helper function to check if pair passes tax limit function _checkTaxLimit( address router, address token0, address token1, uint amountIn, uint maxTaxPermille ) private view returns (bool) { address[] memory path = new address[](2); path[0] = token0; path[1] = token1; try IRouter(router).getAmountsOut(amountIn, path) returns (uint[] memory buyOuts) { if (buyOuts.length < 2) return false; address[] memory reversePath = new address[](2); reversePath[0] = token1; reversePath[1] = token0; try IRouter(router).getAmountsOut(buyOuts[1], reversePath) returns (uint[] memory sellOuts) { if (sellOuts.length = minReturn; } catch { return false; } } catch { return false; } } function checkPair(address pairAddress, uint amountIn, uint maxTaxPermille) external view returns (Result memory r) { // Initialize result with default values r.success = false; // Skip processing if pair address is zero if (pairAddress == address(0)) return r; // Get pair data bool success; address token0; address token1; address factory; uint112 reserve0; uint112 reserve1; address router; (success, token0, token1, factory, reserve0, reserve1, router) = _getPairData(pairAddress); // If we couldn't get pair data or there's no router, return early if (!success) return r; // Check tax limits bool passedTaxCheck = _checkTaxLimit(router, token0, token1, amountIn, maxTaxPermille); // Populate result if tax check passed if (passedTaxCheck) { r.success = true; r.token0 = token0; r.token1 = token1; r.factory = factory; r.reserve0 = reserve0; r.reserve1 = reserve1; } return r; } } Event-driven Communication: Upon receiving and processing these events, the Reader normalizes the data and sends updates (e.g., new reserve values for a specific pool) as messages through a Go channel to the Market Graph Service. This ensures that the market’s in-memory representation is updated almost instantaneously. Event-driven Communication: Upon receiving and processing these events, the Reader normalizes the data and sends updates (e.g., new reserve values for a specific pool) as messages through a Go channel to the Market Graph Service. This ensures that the market’s in-memory representation is updated almost instantaneously. Event-driven Communication: Market Graph Service Market Graph Service: Mapping the DeFi Market The Market Graph Service is the central intelligence unit, maintaining a real-time, in-memory representation of the DeFi market. It models the market as a directed graph, where: directed graph Nodes: Represent individual cryptocurrencies (e.g., WETH, USDC, DAI). Edges: Represent liquidity pools on various DEXes (e.g., Uniswap V2 ETH/DAI pool, SushiSwap USDC/WETH pool). Each edge is associated with the current exchange rate (implied by the reserves) for the pair of tokens it connects. Data Structure and Updates: This service receives updates from the Blockchain Data Reader Service via channels. Upon receiving new reserve data for a pool, it updates the corresponding edge in the graph. It also handles the addition of new token pairs or DEXes as they are discovered. Precision with BigInt: All calculations involving token amounts and exchange rates utilize Go's math/big package (BigInt or BigFloat). This is crucial for maintaining arbitrary precision, preventing floating-point inaccuracies that could lead to missed opportunities or incorrect profit calculations. This is especially vital given the heterogeneous nature of amounts in DeFi, which can span from 8 to 18 (or more) significant digits, making standard floating-point arithmetic unsuitable. Arbitrage Path Detection: Bellman-Ford Algorithm: At the heart of this service is the FindArbitrage function, which employs a graph traversal algorithm, specifically Bellman-Ford. This algorithm is uniquely capable of finding negative cycles within a graph, which is precisely what corresponds to an arbitrage opportunity in our market model (where logarithmic exchange rates are used as edge weights). Unlike many other graph theory algorithms that focus on finding the most efficient path, Bellman-Ford's ability to detect negative cycles makes it exceptionally efficient for both DeFi and quantitative finance applications where profit is sought from cyclical discrepancies. Nodes: Represent individual cryptocurrencies (e.g., WETH, USDC, DAI). Nodes: Edges: Represent liquidity pools on various DEXes (e.g., Uniswap V2 ETH/DAI pool, SushiSwap USDC/WETH pool). Each edge is associated with the current exchange rate (implied by the reserves) for the pair of tokens it connects. Edges: Data Structure and Updates: This service receives updates from the Blockchain Data Reader Service via channels. Upon receiving new reserve data for a pool, it updates the corresponding edge in the graph. It also handles the addition of new token pairs or DEXes as they are discovered. Data Structure and Updates: Blockchain Data Reader Service Precision with BigInt: All calculations involving token amounts and exchange rates utilize Go's math/big package (BigInt or BigFloat). This is crucial for maintaining arbitrary precision, preventing floating-point inaccuracies that could lead to missed opportunities or incorrect profit calculations. This is especially vital given the heterogeneous nature of amounts in DeFi, which can span from 8 to 18 (or more) significant digits, making standard floating-point arithmetic unsuitable. Precision with BigInt : math/big BigInt BigFloat Arbitrage Path Detection: Bellman-Ford Algorithm: At the heart of this service is the FindArbitrage function, which employs a graph traversal algorithm, specifically Bellman-Ford. This algorithm is uniquely capable of finding negative cycles within a graph, which is precisely what corresponds to an arbitrage opportunity in our market model (where logarithmic exchange rates are used as edge weights). Unlike many other graph theory algorithms that focus on finding the most efficient path, Bellman-Ford's ability to detect negative cycles makes it exceptionally efficient for both DeFi and quantitative finance applications where profit is sought from cyclical discrepancies. Arbitrage Path Detection: Bellman-Ford Algorithm: FindArbitrage Bellman-Ford Arbitrage Strategy Service: Identifying and Optimizing Profit Subscribed to the update events from the Market Graph Service, the Arbitrage Strategy Service continuously monitors the market graph for newly detected arbitrage paths. Market Graph Service Opportunity Evaluation: Every time the graph is updated or a potential arbitrage path is identified by FindArbitrage, this service springs into action. It takes the negative cycle (arbitrage path) found by the Market Graph Service and initiates a comprehensive profitability calculation. Optimal Input Amount Calculation (Convex Optimization): A critical step is determining the optimal input amount (dx) for the arbitrage sequence. This is a non-trivial problem, as profitability is a non-linear function of the input amount, as demonstrated in the paper ‘An analysis of Uniswap markets’. It is affected by slippage and fees across multiple swaps. The service solves this as a convex optimization problem, utilizing Go’s gonum/optimize package. This ensures that the chosen input amount maximizes the net profit after accounting for all variables. Simulation of Swaps: Before committing to a transaction, the service performs a simulated execution of all swaps within the detected arbitrage path using the constant product formula and the calculated optimal input amount. During this simulation, minimum output amounts are also set for each intermediate swap step. This ensures that in the event of unexpectedly low actual outputs (e.g., due to sudden price swings or high slippage on-chain), the transaction will revert with minimal gas loss, rather than proceeding with an unprofitable or losing trade. This simulation meticulously accounts for: All Fees: Including DEX trading fees (e.g., 0.3% for Uniswap V2). Slippage: Accurately modeling the price impact of each trade within the sequence. Gas Costs: An estimate of the gas fees required for the entire transaction, considering the chain (Base) and current network conditions. Profit Thresholding: Only if the calculated net profit is at least 0.5% of the initial input amount (or a configurable threshold) is the opportunity considered viable. This threshold ensures that the effort and risk of sending a transaction are justified by a substantial return. Notification for Execution: If a profitable opportunity meets the criteria, the Arbitrage Strategy Service compiles all necessary details — the ordered sequence of swaps (edges), the optimal input amount, and any other relevant parameters — and sends a notification through a Go channel to the Transaction Builder Service. Opportunity Evaluation: Every time the graph is updated or a potential arbitrage path is identified by FindArbitrage, this service springs into action. It takes the negative cycle (arbitrage path) found by the Market Graph Service and initiates a comprehensive profitability calculation. Opportunity Evaluation: FindArbitrage Optimal Input Amount Calculation (Convex Optimization): A critical step is determining the optimal input amount (dx) for the arbitrage sequence. This is a non-trivial problem, as profitability is a non-linear function of the input amount, as demonstrated in the paper ‘An analysis of Uniswap markets’. It is affected by slippage and fees across multiple swaps. The service solves this as a convex optimization problem, utilizing Go’s gonum/optimize package. This ensures that the chosen input amount maximizes the net profit after accounting for all variables. Optimal Input Amount Calculation (Convex Optimization): optimal input amount convex optimization problem gonum/optimize Simulation of Swaps: Before committing to a transaction, the service performs a simulated execution of all swaps within the detected arbitrage path using the constant product formula and the calculated optimal input amount. During this simulation, minimum output amounts are also set for each intermediate swap step. This ensures that in the event of unexpectedly low actual outputs (e.g., due to sudden price swings or high slippage on-chain), the transaction will revert with minimal gas loss, rather than proceeding with an unprofitable or losing trade. This simulation meticulously accounts for: Simulation of Swaps: simulated execution All Fees: Including DEX trading fees (e.g., 0.3% for Uniswap V2). All Fees: Slippage: Accurately modeling the price impact of each trade within the sequence. Slippage: Gas Costs: An estimate of the gas fees required for the entire transaction, considering the chain (Base) and current network conditions. Gas Costs: Profit Thresholding: Only if the calculated net profit is at least 0.5% of the initial input amount (or a configurable threshold) is the opportunity considered viable. This threshold ensures that the effort and risk of sending a transaction are justified by a substantial return. Profit Thresholding: at least 0.5% of the initial input amount Notification for Execution: If a profitable opportunity meets the criteria, the Arbitrage Strategy Service compiles all necessary details — the ordered sequence of swaps (edges), the optimal input amount, and any other relevant parameters — and sends a notification through a Go channel to the Transaction Builder Service. Notification for Execution: Transaction Builder Service Transaction Builder Service: Swift Execution The Transaction Builder Service is the execution arm of the bot, tasked with rapidly constructing and submitting the arbitrage transaction to the blockchain. Speed is paramount here, as opportunities are extremely time-sensitive. Transaction Construction: Upon receiving an opportunity from the Arbitrage Strategy Service, this service immediately begins constructing the atomic blockchain transaction. This involves: Smart Contract Interaction (Atomic Swaps): This service interacts with a custom smart contract specifically designed to execute all arbitrage operations (multiple swaps) within a single, atomic transaction. This contract also handles token approvals within the same transaction flow. This logic is critical for preventing frontrunning or backrunning in the middle of the arbitrage sequence, as the entire operation succeeds or fails as one unit. Here is the Solidity function that handles an arbitrage execution without a flash loan, requiring the owner (the bot) to fund the initial amountIn: Transaction Construction: Upon receiving an opportunity from the Arbitrage Strategy Service, this service immediately begins constructing the atomic blockchain transaction. This involves: Transaction Construction: Smart Contract Interaction (Atomic Swaps): This service interacts with a custom smart contract specifically designed to execute all arbitrage operations (multiple swaps) within a single, atomic transaction. This contract also handles token approvals within the same transaction flow. This logic is critical for preventing frontrunning or backrunning in the middle of the arbitrage sequence, as the entire operation succeeds or fails as one unit. Smart Contract Interaction (Atomic Swaps): Here is the Solidity function that handles an arbitrage execution without a flash loan, requiring the owner (the bot) to fund the initial amountIn: without owner amountIn struct SwapStep { address router; address[] path; uint minOut; } function executeArb( address inputToken, uint amountIn, SwapStep[] calldata steps, uint minFinalOut ) external onlyOwner returns (uint finalAmountOut) { require(steps.length > 0, "No steps"); // Transfer tokens from msg.sender to contract require(IERC20(inputToken).transferFrom(msg.sender, address(this), amountIn), "Transfer in failed"); address currentToken = inputToken; uint currentAmount = amountIn; for (uint i = 0; i < steps.length; i++) { SwapStep calldata step = steps[i]; require(step.path[0] == currentToken, "Path mismatch"); address outputToken = step.path[step.path.length - 1]; // Save balance before swap uint balanceBefore = IERC20(outputToken).balanceOf(address(this)); // Safe approve require(IERC20(currentToken).approve(step.router, 0), "Reset approve failed"); require(IERC20(currentToken).approve(step.router, currentAmount), "Approve failed"); IUniswapV2Router(step.router).swapExactTokensForTokens( currentAmount, step.minOut, step.path, address(this), block.timestamp ); uint balanceAfter = IERC20(outputToken).balanceOf(address(this)); uint received = balanceAfter - balanceBefore; require(received >= step.minOut, "Slippage too high"); currentToken = outputToken; currentAmount = received; } require(currentAmount >= minFinalOut, "Final output too low"); require(IERC20(currentToken).transfer(owner, currentAmount), "Final transfer failed"); return currentAmount; } struct SwapStep { address router; address[] path; uint minOut; } function executeArb( address inputToken, uint amountIn, SwapStep[] calldata steps, uint minFinalOut ) external onlyOwner returns (uint finalAmountOut) { require(steps.length > 0, "No steps"); // Transfer tokens from msg.sender to contract require(IERC20(inputToken).transferFrom(msg.sender, address(this), amountIn), "Transfer in failed"); address currentToken = inputToken; uint currentAmount = amountIn; for (uint i = 0; i < steps.length; i++) { SwapStep calldata step = steps[i]; require(step.path[0] == currentToken, "Path mismatch"); address outputToken = step.path[step.path.length - 1]; // Save balance before swap uint balanceBefore = IERC20(outputToken).balanceOf(address(this)); // Safe approve require(IERC20(currentToken).approve(step.router, 0), "Reset approve failed"); require(IERC20(currentToken).approve(step.router, currentAmount), "Approve failed"); IUniswapV2Router(step.router).swapExactTokensForTokens( currentAmount, step.minOut, step.path, address(this), block.timestamp ); uint balanceAfter = IERC20(outputToken).balanceOf(address(this)); uint received = balanceAfter - balanceBefore; require(received >= step.minOut, "Slippage too high"); currentToken = outputToken; currentAmount = received; } require(currentAmount >= minFinalOut, "Final output too low"); require(IERC20(currentToken).transfer(owner, currentAmount), "Final transfer failed"); return currentAmount; } Flash Loan Integration: If the optimal amount for the arbitrage necessitates a flash loan, the builder integrates the flash loan logic (borrow → execute swaps → repay) into a single, indivisible transaction, utilizing a custom contract that facilitates this atomic operation via Aave’s FlashLoanSimple interface. This allows the bot to execute very large arbitrages without needing to hold substantial capital upfront. Here is the Solidity contract function executeOperation (part of a larger FlashLoanReceiver contract) that gets called by the Aave Pool and contains the arbitrage logic using the borrowed funds: Flash Loan Integration: If the optimal amount for the arbitrage necessitates a flash loan, the builder integrates the flash loan logic (borrow → execute swaps → repay) into a single, indivisible transaction, utilizing a custom contract that facilitates this atomic operation via Aave’s FlashLoanSimple interface. This allows the bot to execute very large arbitrages without needing to hold substantial capital upfront. Flash Loan Integration: FlashLoanSimple Here is the Solidity contract function executeOperation (part of a larger FlashLoanReceiver contract) that gets called by the Aave Pool and contains the arbitrage logic using the borrowed funds: executeOperation function startArbitrage( address token, uint256 amount, SwapStep[] calldata steps, uint256 minFinalOut ) external onlyOwner { bytes memory params = abi.encode(steps, minFinalOut); POOL.flashLoanSimple(address(this), token, amount, params, 0); } function executeOperation( address asset, uint256 amount, uint256 premium, address initiator, bytes calldata params ) external override returns (bool) { require(msg.sender == address(POOL), "Untrusted lender"); require(initiator == address(this), "Unauthorized initiator"); (SwapStep[] memory steps, uint256 minFinalOut) = abi.decode(params, (SwapStep[], uint256)); // Execute the arbitrage address currentToken = asset; uint currentAmount = amount; for (uint i = 0; i < steps.length; i++) { SwapStep memory step = steps[i]; require(step.path[0] == currentToken, "Path mismatch"); address outputToken = step.path[step.path.length - 1]; // Save balance before swap uint balanceBefore = IERC20(outputToken).balanceOf(address(this)); // Safe approve require(IERC20(currentToken).approve(step.router, 0), "Reset approve failed"); require(IERC20(currentToken).approve(step.router, currentAmount), "Approve failed"); IUniswapV2Router(step.router).swapExactTokensForTokens( currentAmount, step.minOut, step.path, address(this), block.timestamp ); uint balanceAfter = IERC20(outputToken).balanceOf(address(this)); uint received = balanceAfter - balanceBefore; require(received >= step.minOut, "Slippage too high"); currentToken = outputToken; currentAmount = received; } require(currentAmount >= amount + premium, "Insufficient profit"); require(currentAmount >= minFinalOut, "Final output too low"); // Repay the loan require(IERC20(asset).approve(address(POOL), amount + premium), "Approval failed"); // Transfer profits to owner uint profit = IERC20(asset).balanceOf(address(this)) - (amount + premium); if (profit > 0) { require(IERC20(asset).transfer(owner, profit), "Profit transfer failed"); } return true; } function startArbitrage( address token, uint256 amount, SwapStep[] calldata steps, uint256 minFinalOut ) external onlyOwner { bytes memory params = abi.encode(steps, minFinalOut); POOL.flashLoanSimple(address(this), token, amount, params, 0); } function executeOperation( address asset, uint256 amount, uint256 premium, address initiator, bytes calldata params ) external override returns (bool) { require(msg.sender == address(POOL), "Untrusted lender"); require(initiator == address(this), "Unauthorized initiator"); (SwapStep[] memory steps, uint256 minFinalOut) = abi.decode(params, (SwapStep[], uint256)); // Execute the arbitrage address currentToken = asset; uint currentAmount = amount; for (uint i = 0; i < steps.length; i++) { SwapStep memory step = steps[i]; require(step.path[0] == currentToken, "Path mismatch"); address outputToken = step.path[step.path.length - 1]; // Save balance before swap uint balanceBefore = IERC20(outputToken).balanceOf(address(this)); // Safe approve require(IERC20(currentToken).approve(step.router, 0), "Reset approve failed"); require(IERC20(currentToken).approve(step.router, currentAmount), "Approve failed"); IUniswapV2Router(step.router).swapExactTokensForTokens( currentAmount, step.minOut, step.path, address(this), block.timestamp ); uint balanceAfter = IERC20(outputToken).balanceOf(address(this)); uint received = balanceAfter - balanceBefore; require(received >= step.minOut, "Slippage too high"); currentToken = outputToken; currentAmount = received; } require(currentAmount >= amount + premium, "Insufficient profit"); require(currentAmount >= minFinalOut, "Final output too low"); // Repay the loan require(IERC20(asset).approve(address(POOL), amount + premium), "Approval failed"); // Transfer profits to owner uint profit = IERC20(asset).balanceOf(address(this)) - (amount + premium); if (profit > 0) { require(IERC20(asset).transfer(owner, profit), "Profit transfer failed"); } return true; } Gas Estimation and Price: Dynamically estimates the required gas for the transaction and sets an appropriate gas price (or priority fee on L2s like Base) to ensure timely inclusion in a block. Transaction Dispatch: Once constructed, the signed transaction is dispatched to the Base blockchain node. The choice of Base is strategic: unlike Ethereum L1, Base’s current architecture does not feature a publicly visible mempool in the traditional sense. This means that transactions are less susceptible to direct generalized frontrunning by bots scanning the mempool. While Maximal Extractable Value (MEV) still exists on L2s, the mechanisms for its extraction differ from L1, and direct mempool sniping is significantly reduced, offering a more predictable execution environment for arbitrageurs. Asynchronous Feedback: After sending the transaction, the service sends a notification to the Honeywall Service to signal that a transaction has been initiated and requires monitoring. Gas Estimation and Price: Dynamically estimates the required gas for the transaction and sets an appropriate gas price (or priority fee on L2s like Base) to ensure timely inclusion in a block. Gas Estimation and Price: Transaction Dispatch: Once constructed, the signed transaction is dispatched to the Base blockchain node. The choice of Base is strategic: unlike Ethereum L1, Base’s current architecture does not feature a publicly visible mempool in the traditional sense. This means that transactions are less susceptible to direct generalized frontrunning by bots scanning the mempool. While Maximal Extractable Value (MEV) still exists on L2s, the mechanisms for its extraction differ from L1, and direct mempool sniping is significantly reduced, offering a more predictable execution environment for arbitrageurs. Transaction Dispatch: Base Asynchronous Feedback: After sending the transaction, the service sends a notification to the Honeywall Service to signal that a transaction has been initiated and requires monitoring. Asynchronous Feedback: Honeywall Service Honeywall Service: Post-Execution Validation and Security The Honeywall Service acts as a critical post-execution checker and a robust security layer for the arbitrage bot. Its role is to validate the outcome of executed transactions and protect against malicious actors. Transaction Outcome Monitoring: After the Transaction Builder dispatches a transaction, the Honeywall Service monitors its inclusion in a block and its outcome. Profit Logging: If the transaction is successful and yields a profit (as expected from the simulation), the profit details are logged for performance tracking and analysis. Failure Analysis: In case of a transaction failure, the Honeywall analyzes the reason for the revert. Honeypot/Scam Detection and Blacklisting: A key security feature is its ability to identify “honeypot” tokens or pools that implement deceptive logic (e.g., allowing buys but preventing sells, or imposing exorbitant hidden taxes on sells). External Provider Integration: It integrates with an external provider or a database of known honeypot contracts to cross-reference the pairs used in failed transactions, thereby identifying potential scams. Dynamic Blacklisting: If a specific pair or pool is identified as an honeypot or problematic due to unexpected high taxes, it is immediately added to a database-backed blacklist. This ensures that the bot avoids these risky assets in future operations. Bloom Filter Integration: This blacklist is efficiently managed via a Bloom filter mechanism. This allows the Blockchain Data Reader Service to quickly check newly observed pairs against the blacklist before even retrieving their reserves or adding them to the market graph. This acts as a proactive defense, preventing the bot from wasting resources or attempting trades on known problematic pairs in the future. Transaction Outcome Monitoring: After the Transaction Builder dispatches a transaction, the Honeywall Service monitors its inclusion in a block and its outcome. Transaction Outcome Monitoring: Profit Logging: If the transaction is successful and yields a profit (as expected from the simulation), the profit details are logged for performance tracking and analysis. Profit Logging: Failure Analysis: In case of a transaction failure, the Honeywall analyzes the reason for the revert. Failure Analysis: Honeypot/Scam Detection and Blacklisting: A key security feature is its ability to identify “honeypot” tokens or pools that implement deceptive logic (e.g., allowing buys but preventing sells, or imposing exorbitant hidden taxes on sells). Honeypot/Scam Detection and Blacklisting: External Provider Integration: It integrates with an external provider or a database of known honeypot contracts to cross-reference the pairs used in failed transactions, thereby identifying potential scams. External Provider Integration: Dynamic Blacklisting: If a specific pair or pool is identified as an honeypot or problematic due to unexpected high taxes, it is immediately added to a database-backed blacklist. This ensures that the bot avoids these risky assets in future operations. Dynamic Blacklisting: database-backed blacklist Bloom Filter Integration: This blacklist is efficiently managed via a Bloom filter mechanism. This allows the Blockchain Data Reader Service to quickly check newly observed pairs against the blacklist before even retrieving their reserves or adding them to the market graph. This acts as a proactive defense, preventing the bot from wasting resources or attempting trades on known problematic pairs in the future. Bloom Filter Integration: Bloom filter Blockchain Data Reader Service before Conclusion of Architectural Design The modular, event-driven architecture implemented in Go, combined with specialized services for data ingestion, market modeling, opportunity optimization, rapid execution, and robust security, forms the backbone of our high-performance arbitrage bot. This design ensures that the system can react with unparalleled speed to fleeting market opportunities while also mitigating significant risks inherent to the DeFi “Dark Forest.” In the subsequent chapters, we will delve into the specific algorithms and implementation details of each of these services, starting with the intricate mathematics of opportunity detection. 4. Opportunity Detection and Optimal Execution: The Bot’s Brain The true intelligence of an arbitrage bot lies in its ability to quickly and accurately identify profitable opportunities within a constantly shifting market, and then to optimize the execution for maximum returns. This chapter delves into the core algorithms and mathematical models that power our bot’s decision-making process, from mapping the market as a graph to precisely calculating optimal trade sizes and simulating outcomes. Modeling the DeFi Market as a Graph As introduced in the architectural overview, our Market Graph Service represents the DeFi landscape as a directed graph. In this model, individual tokens (e.g., WETH, DAI, USDC) serve as nodes, while liquidity pools on various Decentralized Exchanges (DEXes) act as edges connecting these tokens. Each edge’s weight represents the cost of transacting through that pool. nodes edges To efficiently detect arbitrage opportunities, which manifest as profitable cycles, we transform the problem of finding a profitable sequence of trades into finding a negative cycle in our graph. This transformation is achieved by applying a logarithmic function to the exchange rates. negative cycle The Necessity of Logarithms for Cycle Detection The core idea behind arbitrage is to multiply a starting amount by a series of exchange rates to end up with more of the original asset. For example, if we start with A units of TokenX and trade it for TokenY, then TokenY for TokenZ, and finally TokenZ back to TokenX, our final amount would be: TokenX TokenY TokenY TokenZ TokenZ TokenX Working with products in graph algorithms is cumbersome. A common technique in computational finance to transform multiplicative problems into additive ones is to apply a logarithm. By taking the natural logarithm of each exchange rate, the product becomes a sum: Now, for a profitable cycle, we need ln(Afinal) > ln(A), which means ln(RateX→Y) + ln(RateY→Z) + ln(RateZ→X) > 0. However, typical shortest path algorithms (like Bellman-Ford, which we use) are designed to find paths with minimum sum of weights. To make a profitable cycle appear as a “negative cycle” in our graph, we simply negate the logarithmic rates: ln(Afinal) > ln(A) ln(RateX→Y) + ln(RateY→Z) + ln(RateZ→X) > 0 ln(RateX→Y) + ln(RateY→Z) + ln(RateZ→X) > 0 sum With this transformation, a sum of negative weights that results in a negative value (i.e., a negative cycle) directly indicates a profitable arbitrage opportunity. Handling Precision with BigInt BigInt The amounts of tokens in DeFi can vary wildly, from tiny fractions (e.g., for ERC-20 tokens with 18 decimal places) to very large numbers (e.g., stablecoins). This extreme heterogeneity in magnitude, spanning up to 18 significant figures, makes standard floating-point arithmetic highly susceptible to precision errors. Such errors, though seemingly small, can lead to misidentified opportunities or, worse, unprofitable trades. To overcome this, our Market Graph Service, and indeed all calculations involving token amounts and exchange rates within the bot, utilize Go’s math/big package, specifically BigInt for integer arithmetic and BigFloat for floating-point operations where necessary. While BigFloat offers arbitrary precision, applying log to BigInt or BigFloat values requires careful handling, as standard math.Log functions operate on native float64 types. Custom implementations or external libraries capable of arbitrary-precision logarithms are essential here. math/big BigInt BigFloat BigFloat log BigInt BigFloat math.Log float64 func getLogRate(reserve0, reserve1 *big.Int) *big.Float { const prec = 1024 resIn := new(big.Float).SetPrec(prec).SetInt(reserve0) resOut := new(big.Float).SetPrec(prec).SetInt(reserve1) // Effective Rate rate := new(big.Float).SetPrec(prec).Quo(resOut, resIn) logRate := bigfloat.Log(rate) return logRate.Neg(logRate) } func getLogRate(reserve0, reserve1 *big.Int) *big.Float { const prec = 1024 resIn := new(big.Float).SetPrec(prec).SetInt(reserve0) resOut := new(big.Float).SetPrec(prec).SetInt(reserve1) // Effective Rate rate := new(big.Float).SetPrec(prec).Quo(resOut, resIn) logRate := bigfloat.Log(rate) return logRate.Neg(logRate) } Arbitrage Path Detection: The Bellman-Ford Algorithm Once the DeFi market is accurately modeled as a graph with logarithmic negative edge weights, the task of finding arbitrage opportunities reduces to identifying negative cycles within this graph. For this, we employ the Bellman-Ford algorithm. negative cycles Bellman-Ford algorithm Named after Richard Bellman and Lester Ford Jr., Bellman-Ford is a versatile shortest path algorithm capable of handling graphs with negative edge weights. Unlike Dijkstra’s algorithm, which fails in the presence of negative cycles, Bellman-Ford is specifically designed to detect them. Its historical significance extends beyond theoretical computer science; it has found applications in diverse fields, including network routing (where it helps find the cheapest paths with varying costs) and, critically, in quantitative finance for identifying profitable trading opportunities in currency markets. The algorithm works by iteratively relaxing edges, progressively finding shorter paths to all nodes from a source. If, after ∣V∣−1 iterations (where ∣V∣ is the number of vertices), an additional N-th iteration finds a path that can still be “relaxed” (i.e., a shorter path can be found), it indicates the presence of a negative cycle. This property makes it perfect for our use case: a negative cycle implies a sequence of trades that results in a net gain, exactly what an arbitrage bot seeks. type Edge struct { Pair string From Token To Token LogRate *big.Float Reserve0 *big.Int Reserve1 *big.Int Dex string MinOut *big.Int } type Graph struct { nodes map[string]Token Edges map[string][]*Edge pairChan *chan GraphMessage dexCheckChan *chan DexDexMessage subscriptions []*chan time.Duration mu sync.RWMutex } // Bellman-Ford algorithm to find arbitrage cycles func (g *Graph) FindArbitrage(source Token) ([]*Edge, bool) { sourceKey := source.Address g.mu.RLock() defer g.mu.RUnlock() distance := make(map[string]*big.Float) predecessor := make(map[string]*Edge) // 1. Init for token := range g.nodes { distance[token] = new(big.Float).SetInf(false) } distance[sourceKey] = new(big.Float).SetFloat64(0) // 2. Relax edges V-1 times for i := 0; i < len(g.nodes)-1; i++ { for _, edgeList := range g.Edges { for _, e := range edgeList { from := e.From.Address to := e.To.Address if !distance[from].IsInf() && new(big.Float).Add(distance[from], e.LogRate).Cmp(distance[to]) < 0 { distance[to].Add(distance[from], e.LogRate) predecessor[to] = e } } } } // 3. Negative cycle detection var cycleStartToken string for _, edgeList := range g.Edges { for _, e := range edgeList { from := e.From.Address to := e.To.Address if !distance[from].IsInf() && new(big.Float).Add(distance[from], e.LogRate).Cmp(distance[to]) < 0 { cycleStartToken = to break } } if cycleStartToken != "" { break } } if cycleStartToken == "" { return nil, false // No Arbitrage } // 4. detect first cycle node visited := make(map[string]bool) current := cycleStartToken for !visited[current] { visited[current] = true edge := predecessor[current] if edge == nil { return nil, false // missing edge } current = edge.From.Address } // 5. Complete cycle cycleStart := current cycle := []*Edge{} for { edge := predecessor[current] if edge == nil { return nil, false // missing edge } cycle = append(cycle, edge) current = edge.From.Address if current == cycleStart { break } } // 6. Invert cycle for i, j := 0, len(cycle)-1; i < j; i, j = i+1, j-1 { cycle[i], cycle[j] = cycle[j], cycle[i] } return cycle, true } type Edge struct { Pair string From Token To Token LogRate *big.Float Reserve0 *big.Int Reserve1 *big.Int Dex string MinOut *big.Int } type Graph struct { nodes map[string]Token Edges map[string][]*Edge pairChan *chan GraphMessage dexCheckChan *chan DexDexMessage subscriptions []*chan time.Duration mu sync.RWMutex } // Bellman-Ford algorithm to find arbitrage cycles func (g *Graph) FindArbitrage(source Token) ([]*Edge, bool) { sourceKey := source.Address g.mu.RLock() defer g.mu.RUnlock() distance := make(map[string]*big.Float) predecessor := make(map[string]*Edge) // 1. Init for token := range g.nodes { distance[token] = new(big.Float).SetInf(false) } distance[sourceKey] = new(big.Float).SetFloat64(0) // 2. Relax edges V-1 times for i := 0; i < len(g.nodes)-1; i++ { for _, edgeList := range g.Edges { for _, e := range edgeList { from := e.From.Address to := e.To.Address if !distance[from].IsInf() && new(big.Float).Add(distance[from], e.LogRate).Cmp(distance[to]) < 0 { distance[to].Add(distance[from], e.LogRate) predecessor[to] = e } } } } // 3. Negative cycle detection var cycleStartToken string for _, edgeList := range g.Edges { for _, e := range edgeList { from := e.From.Address to := e.To.Address if !distance[from].IsInf() && new(big.Float).Add(distance[from], e.LogRate).Cmp(distance[to]) < 0 { cycleStartToken = to break } } if cycleStartToken != "" { break } } if cycleStartToken == "" { return nil, false // No Arbitrage } // 4. detect first cycle node visited := make(map[string]bool) current := cycleStartToken for !visited[current] { visited[current] = true edge := predecessor[current] if edge == nil { return nil, false // missing edge } current = edge.From.Address } // 5. Complete cycle cycleStart := current cycle := []*Edge{} for { edge := predecessor[current] if edge == nil { return nil, false // missing edge } cycle = append(cycle, edge) current = edge.From.Address if current == cycleStart { break } } // 6. Invert cycle for i, j := 0, len(cycle)-1; i < j; i, j = i+1, j-1 { cycle[i], cycle[j] = cycle[j], cycle[i] } return cycle, true } Optimal Input Amount Calculation: Maximizing Profit Once a negative cycle (arbitrage opportunity) is identified, the next critical step is to determine the optimal input amount (dx) for the initial trade in the sequence. This is not arbitrary; the profitability of an arbitrage opportunity is a non-linear function of the trade size due to the inherent slippage and fees associated with AMM swaps. optimal input amount As detailed in “An analysis of Uniswap markets”, the constant product formula inherently implies a convexity in the relationship between input and output amounts. Specifically, as the trade size increases, the effective exchange rate worsens due to the pool’s invariant. This means there’s a sweet spot: too small an amount might not cover gas fees, while too large an amount might incur excessive slippage, eroding profits. The problem of maximizing profit is a convex optimization problem. For a series of N swaps in an arbitrage path, the final output amount (and thus the profit) can be expressed as a function of the initial input amount (dx). While the exact analytical solution for multi-leg arbitrage can be complex, especially with varying fee structures and slippage curves across different AMMs, the function representing profit minus costs (including gas) is generally convex. This allows us to use numerical optimization techniques to find the global maximum. convex optimization problem Our Arbitrage Strategy Service addresses this by employing an optimization solver from Go’s gonum/optimize package. This solver takes a function representing the net profit (profit from swaps minus estimated gas fees and any flash loan premiums) and finds the input amount that maximizes this value. The objective function fed to the solver incorporates the amountOut formula dy = (x + dx) / (dx⋅ y) for each step in the arbitrage path, accounting for intermediate reserves, fees, and slippage at each stage. gonum/optimize amountOut dy = (x + dx) / (dx⋅ y) func getOptimalAmoutIn(edges []*Edge, decimals int) (*float64, error) { factor := math.Pow10(decimals) intMax, _ := constants.GetRouterReserveFromToken(edges[0].From.Address) maxCapital := new(big.Float).Mul(new(big.Float).SetInt64(intMax), big.NewFloat(factor)) fee := big.NewFloat(0.997) problem := optimize.Problem{ Func: func(x []float64) float64 { delta := big.NewFloat(x[0]) if delta.Cmp(big.NewFloat(0)) 0 { return math.Inf(1) } delta_i := new(big.Float).Set(delta) for _, edge := range edges { effectiveIn := new(big.Float).Mul(delta_i, fee) reserveIn := new(big.Float).SetInt(edge.Reserve0) reserveOut := new(big.Float).SetInt(edge.Reserve1) num := new(big.Float).Mul(reserveOut, effectiveIn) denom := new(big.Float).Add(reserveIn, effectiveIn) delta_i = new(big.Float).Quo(num, denom) } profit := new(big.Float).Sub(delta_i, delta) result, _ := profit.Float64() return -result }, } result, err := optimize.Minimize(problem, []float64{1.0}, nil, nil) if err != nil { return nil, err } return &result.X[0], nil } func getOptimalAmoutIn(edges []*Edge, decimals int) (*float64, error) { factor := math.Pow10(decimals) intMax, _ := constants.GetRouterReserveFromToken(edges[0].From.Address) maxCapital := new(big.Float).Mul(new(big.Float).SetInt64(intMax), big.NewFloat(factor)) fee := big.NewFloat(0.997) problem := optimize.Problem{ Func: func(x []float64) float64 { delta := big.NewFloat(x[0]) if delta.Cmp(big.NewFloat(0)) 0 { return math.Inf(1) } delta_i := new(big.Float).Set(delta) for _, edge := range edges { effectiveIn := new(big.Float).Mul(delta_i, fee) reserveIn := new(big.Float).SetInt(edge.Reserve0) reserveOut := new(big.Float).SetInt(edge.Reserve1) num := new(big.Float).Mul(reserveOut, effectiveIn) denom := new(big.Float).Add(reserveIn, effectiveIn) delta_i = new(big.Float).Quo(num, denom) } profit := new(big.Float).Sub(delta_i, delta) result, _ := profit.Float64() return -result }, } result, err := optimize.Minimize(problem, []float64{1.0}, nil, nil) if err != nil { return nil, err } return &result.X[0], nil } Simulation of Swaps and Profitability Assessment Before any transaction is dispatched, the Arbitrage Strategy Service performs a meticulous simulated execution of the entire arbitrage path. This step is crucial for verifying the actual profitability, given the real-time market conditions and the exact parameters of the proposed trade. simulated execution The simulation uses the current reserves of the involved liquidity pools and the calculated optimal input amount. For each step in the multi-leg path, it applies the specific AMM formula (e.g., the constant product formula for Uniswap V2-like pools) to calculate the expected output: func (ab *ArbitrageBuilderV2) calculateProfitabilityWithSlippage(edges []*Edge, decimals int) (*big.Float, *big.Float, error) { opt, err := getOptimalAmoutIn(edges, decimals) if err != nil { return nil, nil, err } optBig := new(big.Float).SetFloat64(*opt) amount := new(big.Float).Set(optBig) fee := big.NewFloat(0.997) for _, edge := range edges { if edge.Reserve0 == nil || edge.Reserve1 == nil || edge.Reserve0.Cmp(big.NewInt(0)) == 0 || edge.Reserve1.Cmp(big.NewInt(0)) == 0 { return nil, nil, errors.New("edge has invalid reserves") } reserveIn := new(big.Float).SetInt(edge.Reserve0) reserveOut := new(big.Float).SetInt(edge.Reserve1) amountInWithFee := new(big.Float).Mul(amount, fee) if amountInWithFee.Cmp(reserveIn) >= 0 { return big.NewFloat(-1.0), nil, errors.New("amount exceeds available reserves") } // "x * y = k" numerator := new(big.Float).Mul(reserveOut, amountInWithFee) denominator := new(big.Float).Add(reserveIn, amountInWithFee) amountOut := new(big.Float).Quo(numerator, denominator) amount = amountOut } profit := new(big.Float).Sub(amount, optBig) profit.Sub(profit, ab.EstimateGasCost(len(edges))) profit.Sub(profit, new(big.Float).Mul(optBig, big.NewFloat(0.005))) normalizedProfit := new(big.Float).Quo(profit, new(big.Float).SetFloat64(math.Pow10(decimals))) return normalizedProfit, optBig, nil } func (ab *ArbitrageBuilderV2) calculateProfitabilityWithSlippage(edges []*Edge, decimals int) (*big.Float, *big.Float, error) { opt, err := getOptimalAmoutIn(edges, decimals) if err != nil { return nil, nil, err } optBig := new(big.Float).SetFloat64(*opt) amount := new(big.Float).Set(optBig) fee := big.NewFloat(0.997) for _, edge := range edges { if edge.Reserve0 == nil || edge.Reserve1 == nil || edge.Reserve0.Cmp(big.NewInt(0)) == 0 || edge.Reserve1.Cmp(big.NewInt(0)) == 0 { return nil, nil, errors.New("edge has invalid reserves") } reserveIn := new(big.Float).SetInt(edge.Reserve0) reserveOut := new(big.Float).SetInt(edge.Reserve1) amountInWithFee := new(big.Float).Mul(amount, fee) if amountInWithFee.Cmp(reserveIn) >= 0 { return big.NewFloat(-1.0), nil, errors.New("amount exceeds available reserves") } // "x * y = k" numerator := new(big.Float).Mul(reserveOut, amountInWithFee) denominator := new(big.Float).Add(reserveIn, amountInWithFee) amountOut := new(big.Float).Quo(numerator, denominator) amount = amountOut } profit := new(big.Float).Sub(amount, optBig) profit.Sub(profit, ab.EstimateGasCost(len(edges))) profit.Sub(profit, new(big.Float).Mul(optBig, big.NewFloat(0.005))) normalizedProfit := new(big.Float).Quo(profit, new(big.Float).SetFloat64(math.Pow10(decimals))) return normalizedProfit, optBig, nil } Crucially, the simulation also incorporates minimum output amount (minOut) checks for each intermediate step. These minOut values are derived from the simulated expected outputs and are set as parameters in the actual on-chain transaction. If, due to network latency, frontrunning, or unexpected market conditions, an actual swap on-chain yields less than its specified minOut, the entire atomic transaction will gracefully revert. This mechanism is a vital safeguard, preventing the bot from completing an unprofitable sequence of trades and limiting losses to only the gas spent on the reverted transaction. minimum output amount (minOut) minOut minOut Only if the final net profit, after all fees, slippage, gas costs, and flash loan premiums, exceeds a predefined profit threshold (e.g., 0.5% of the initial input amount) is the opportunity deemed viable and passed to the Transaction Builder Service for execution. This threshold ensures that the bot only pursues opportunities with a significant enough margin to warrant the computational and on-chain costs. profit threshold 5. Transaction Engineering: Swift Execution in the Dark Forest Identifying a profitable arbitrage opportunity is only half the battle; the other, arguably more critical, half lies in the ability to execute the trade with unparalleled speed and reliability. In the hyper-competitive “Dark Forest” of DeFi, where opportunities are fleeting and sophisticated bots vie for every millisecond, transaction engineering becomes an art form. This chapter details the strategies and technical implementations within our Transaction Builder Service designed to ensure lightning-fast and secure execution. Transaction Builder Service The Imperative of Speed The profitability window for arbitrage opportunities on decentralized exchanges is often measured in milliseconds. Price discrepancies are rapidly detected and exploited by numerous automated systems, creating a fierce race to be the first to include a profitable transaction in a new block. Any delay, however minor, can result in the opportunity being seized by a competitor, leading to a failed transaction and wasted gas fees. Therefore, every design decision in the Transaction Builder Service is geared towards minimizing latency at every possible step, from transaction construction to network submission. In-Memory Optimization for Instantaneous Transaction Building To achieve the necessary velocity, our system prioritizes having all essential transaction components readily available in memory, eliminating costly I/O operations or on-chain calls during the critical transaction building phase. Pre-parsed and Packed ABIs: Smart contract Application Binary Interfaces (ABIs) define how to interact with contracts. Instead of parsing ABI definitions and encoding function calls on the fly for each transaction, our system pre-parses and packs the necessary ABI data structures and function selectors into raw bytes[] arrays. These pre-computed byte sequences for common contract interactions (e.g., swapExactTokensForTokens, flashLoanSimple, transferFrom) are stored in memory. When an arbitrage opportunity is identified, the Transaction Builder can quickly assemble the calldata by simply concatenating these pre-packed components with the specific trade parameters, significantly reducing processing time. Cached On-Chain Data for Transaction Fields: To avoid redundant on-chain calls for transaction metadata, a dedicated utility structure within the bot maintains critical, frequently updated values in memory: Account Nonce: The nonce (number of transactions sent from an address) is crucial for preventing replay attacks and ensuring transaction ordering. It is fetched once and then incrementally managed in memory, with robust error handling to re-sync if a transaction fails or is unexpectedly included out of order. Optimal Gas Parameters: Instead of querying the network for gas prices (or base fees/priority fees on EIP-1559 chains) for every transaction, the bot periodically fetches optimal gas parameters. These values are updated in memory and used for quick transaction construction, ensuring that the transaction is priced competitively for timely inclusion without overspending. Signer Information: The private key of the bot’s wallet and the associated signer object (used for cryptographically signing transactions) are loaded into memory upon initialization. This prevents any disk I/O or key derivation during the critical execution phase, ensuring that transactions can be signed almost instantaneously. Pre-parsed and Packed ABIs: Smart contract Application Binary Interfaces (ABIs) define how to interact with contracts. Instead of parsing ABI definitions and encoding function calls on the fly for each transaction, our system pre-parses and packs the necessary ABI data structures and function selectors into raw bytes[] arrays. These pre-computed byte sequences for common contract interactions (e.g., swapExactTokensForTokens, flashLoanSimple, transferFrom) are stored in memory. When an arbitrage opportunity is identified, the Transaction Builder can quickly assemble the calldata by simply concatenating these pre-packed components with the specific trade parameters, significantly reducing processing time. Pre-parsed and Packed ABIs: bytes[] swapExactTokensForTokens flashLoanSimple transferFrom calldata Cached On-Chain Data for Transaction Fields: To avoid redundant on-chain calls for transaction metadata, a dedicated utility structure within the bot maintains critical, frequently updated values in memory: Cached On-Chain Data for Transaction Fields: Account Nonce: The nonce (number of transactions sent from an address) is crucial for preventing replay attacks and ensuring transaction ordering. It is fetched once and then incrementally managed in memory, with robust error handling to re-sync if a transaction fails or is unexpectedly included out of order. Account Nonce: nonce Optimal Gas Parameters: Instead of querying the network for gas prices (or base fees/priority fees on EIP-1559 chains) for every transaction, the bot periodically fetches optimal gas parameters. These values are updated in memory and used for quick transaction construction, ensuring that the transaction is priced competitively for timely inclusion without overspending. Optimal Gas Parameters: Signer Information: The private key of the bot’s wallet and the associated signer object (used for cryptographically signing transactions) are loaded into memory upon initialization. This prevents any disk I/O or key derivation during the critical execution phase, ensuring that transactions can be signed almost instantaneously. Signer Information: signer By keeping these vital components in memory, the Transaction Builder Service can construct and sign a complete blockchain transaction in mere microseconds, ready for immediate dispatch. Dynamic Smart Contract Selection: Flash Loans vs. Direct Swaps The Arbitrage Strategy Service passes an optimized arbitrage path and the calculated optimal input amount to the Transaction Builder. Based on the magnitude of the amountIn and whether it exceeds a pre-defined capital threshold (or if the strategy explicitly calls for it), the Transaction Builder dynamically selects between two primary smart contracts for execution: amountIn Direct Swap Execution Contract: For opportunities that can be funded directly by the bot’s owned capital, the builder utilizes the executeArb function (or similar) on a custom multi-swap proxy contract. As shown in Chapter 3, this contract takes the input tokens from the bot's wallet and executes the entire sequence of swaps within a single, atomic transaction. This approach avoids the complexities and additional premium of flash loans when not necessary. Flash Loan Integrated Contract: When the calculated optimal amountIn for an arbitrage is significantly larger than the bot's available capital, the builder targets a separate, custom smart contract designed to initiate and manage flash loans. This contract's startArbitrage function (as detailed in Chapter 3) requests a flash loan from a protocol like Aave, which then calls back the contract's executeOperation function. Within executeOperation, the entire arbitrage sequence is performed using the borrowed funds, and the flash loan (plus a small premium) is repaid – all within that single, atomic blockchain transaction. This enables the bot to capitalize on very large, high-profit opportunities without requiring substantial upfront capital, democratizing access to large-scale arbitrage. Direct Swap Execution Contract: For opportunities that can be funded directly by the bot’s owned capital, the builder utilizes the executeArb function (or similar) on a custom multi-swap proxy contract. As shown in Chapter 3, this contract takes the input tokens from the bot's wallet and executes the entire sequence of swaps within a single, atomic transaction. This approach avoids the complexities and additional premium of flash loans when not necessary. Direct Swap Execution Contract: executeArb Flash Loan Integrated Contract: When the calculated optimal amountIn for an arbitrage is significantly larger than the bot's available capital, the builder targets a separate, custom smart contract designed to initiate and manage flash loans. This contract's startArbitrage function (as detailed in Chapter 3) requests a flash loan from a protocol like Aave, which then calls back the contract's executeOperation function. Within executeOperation, the entire arbitrage sequence is performed using the borrowed funds, and the flash loan (plus a small premium) is repaid – all within that single, atomic blockchain transaction. This enables the bot to capitalize on very large, high-profit opportunities without requiring substantial upfront capital, democratizing access to large-scale arbitrage. Flash Loan Integrated Contract: amountIn flash loans startArbitrage executeOperation executeOperation This dynamic selection ensures efficient capital allocation and optimal strategy execution based on the specifics of each detected opportunity. Mempool Dynamics: Navigating Ethereum L1 vs. Layer 2 Chains A critical aspect of arbitrage execution is understanding the blockchain’s transaction propagation mechanism, particularly the mempool. mempool Ethereum L1 Mempool: On Ethereum’s Layer 1, the mempool is a public, transparent waiting room for all pending transactions. Transactions broadcasted by users or bots are relayed to various nodes across the network, becoming visible to anyone monitoring the mempool. This transparency is the breeding ground for generalized frontrunning bots (often referred to as “searchers” or “MEV bots”). These sophisticated entities continuously scan the mempool for profitable transactions (e.g., large swaps that cause significant price impact, liquidations, or other arbitrage attempts). Upon detecting such a transaction, they quickly construct an identical or similar transaction, replace the original recipient address with their own, and submit it with a higher gas price (or higher priority fee in EIP-1559) to ensure their transaction is included in the block before the original, thereby stealing the profit. This competitive landscape makes direct arbitrage on L1 highly challenging without leveraging specialized MEV relays. Layer 2 (L2) Chains and Reduced Mempool Visibility (e.g., Base): Our bot strategically operates on Base, an EVM-compatible Layer 2 blockchain. The architecture of many L2s, including Base, fundamentally alters the traditional L1 mempool dynamic. Base does not currently expose a publicly visible mempool in the same manner as Ethereum Layer 1. Instead, transactions are typically sent directly to a centralized sequencer or a private mempool before being batched and committed to the L1. This architectural difference significantly reduces the direct threat of generalized frontrunning. While MEV still exists on L2s (e.g., through sequencer-controlled ordering or other means), the immediate, public visibility of pending transactions that enables L1 frontrunning is largely absent. This provides a more predictable and secure execution environment for our arbitrage transactions, as the bot’s crafted atomic operations are less likely to be “sniped” before they even reach a block producer. This improved execution predictability contributes directly to higher success rates for profitable arbitrages. Ethereum L1 Mempool: On Ethereum’s Layer 1, the mempool is a public, transparent waiting room for all pending transactions. Transactions broadcasted by users or bots are relayed to various nodes across the network, becoming visible to anyone monitoring the mempool. This transparency is the breeding ground for generalized frontrunning bots (often referred to as “searchers” or “MEV bots”). These sophisticated entities continuously scan the mempool for profitable transactions (e.g., large swaps that cause significant price impact, liquidations, or other arbitrage attempts). Upon detecting such a transaction, they quickly construct an identical or similar transaction, replace the original recipient address with their own, and submit it with a higher gas price (or higher priority fee in EIP-1559) to ensure their transaction is included in the block before the original, thereby stealing the profit. This competitive landscape makes direct arbitrage on L1 highly challenging without leveraging specialized MEV relays. Ethereum L1 Mempool: On Ethereum’s Layer 1, the mempool is a public, transparent waiting room for all pending transactions. Transactions broadcasted by users or bots are relayed to various nodes across the network, becoming visible to anyone monitoring the mempool. This transparency is the breeding ground for generalized frontrunning bots (often referred to as “searchers” or “MEV bots”). These sophisticated entities continuously scan the mempool for profitable transactions (e.g., large swaps that cause significant price impact, liquidations, or other arbitrage attempts). Upon detecting such a transaction, they quickly construct an identical or similar transaction, replace the original recipient address with their own, and submit it with a higher gas price (or higher priority fee in EIP-1559) to ensure their transaction is included in the block before the original, thereby stealing the profit. This competitive landscape makes direct arbitrage on L1 highly challenging without leveraging specialized MEV relays. Ethereum L1 Mempool: generalized frontrunning bots before Layer 2 (L2) Chains and Reduced Mempool Visibility (e.g., Base): Our bot strategically operates on Base, an EVM-compatible Layer 2 blockchain. The architecture of many L2s, including Base, fundamentally alters the traditional L1 mempool dynamic. Base does not currently expose a publicly visible mempool in the same manner as Ethereum Layer 1. Instead, transactions are typically sent directly to a centralized sequencer or a private mempool before being batched and committed to the L1. This architectural difference significantly reduces the direct threat of generalized frontrunning. While MEV still exists on L2s (e.g., through sequencer-controlled ordering or other means), the immediate, public visibility of pending transactions that enables L1 frontrunning is largely absent. This provides a more predictable and secure execution environment for our arbitrage transactions, as the bot’s crafted atomic operations are less likely to be “sniped” before they even reach a block producer. This improved execution predictability contributes directly to higher success rates for profitable arbitrages. Layer 2 (L2) Chains and Reduced Mempool Visibility (e.g., Base): Our bot strategically operates on Base, an EVM-compatible Layer 2 blockchain. The architecture of many L2s, including Base, fundamentally alters the traditional L1 mempool dynamic. Base does not currently expose a publicly visible mempool in the same manner as Ethereum Layer 1. Instead, transactions are typically sent directly to a centralized sequencer or a private mempool before being batched and committed to the L1. Layer 2 (L2) Chains and Reduced Mempool Visibility (e.g., Base): Base Base does not currently expose a publicly visible mempool in the same manner as Ethereum Layer 1. This architectural difference significantly reduces the direct threat of generalized frontrunning. While MEV still exists on L2s (e.g., through sequencer-controlled ordering or other means), the immediate, public visibility of pending transactions that enables L1 frontrunning is largely absent. This provides a more predictable and secure execution environment for our arbitrage transactions, as the bot’s crafted atomic operations are less likely to be “sniped” before they even reach a block producer. This improved execution predictability contributes directly to higher success rates for profitable arbitrages. Node Speed and Security: The Foundation of Reliable Execution The connection to the blockchain node is the single point of entry and exit for all data and transactions. Its speed and security are paramount: High-Performance Node Connection: The Transaction Builder Service connects to a dedicated, high-performance node provider (e.g., Alchemy, Infura, or a self-hosted node). A fast and low-latency connection is essential to minimize the time between signing a transaction and its propagation to the network. Any network delay here translates directly into lost arbitrage opportunities. Redundancy (connecting to multiple nodes) can also be considered for maximum reliability. Node Security and Integrity: The security of the connected node is equally vital. Using reputable node providers or ensuring a highly secured self-hosted node is crucial to prevent man-in-the-middle attacks or data tampering. A compromised node could potentially leak private keys, inject malicious transactions, or manipulate data, leading to catastrophic losses. Our system’s reliance on private RPC endpoints (if available from providers) and secure communication channels (https for HTTP, wss for WebSockets) reinforces this security posture. High-Performance Node Connection: The Transaction Builder Service connects to a dedicated, high-performance node provider (e.g., Alchemy, Infura, or a self-hosted node). A fast and low-latency connection is essential to minimize the time between signing a transaction and its propagation to the network. Any network delay here translates directly into lost arbitrage opportunities. Redundancy (connecting to multiple nodes) can also be considered for maximum reliability. High-Performance Node Connection: node provider Node Security and Integrity: The security of the connected node is equally vital. Using reputable node providers or ensuring a highly secured self-hosted node is crucial to prevent man-in-the-middle attacks or data tampering. A compromised node could potentially leak private keys, inject malicious transactions, or manipulate data, leading to catastrophic losses. Our system’s reliance on private RPC endpoints (if available from providers) and secure communication channels (https for HTTP, wss for WebSockets) reinforces this security posture. Node Security and Integrity: https wss By meticulously optimizing for speed at every layer, from in-memory data structures and pre-computation to strategic chain selection and robust node infrastructure, our arbitrage bot is designed to outmaneuver competitors and securely capitalize on the fleeting opportunities within the DeFi landscape. In the next chapter, we will discuss the critical operational strategies required to maintain and evolve such a high-stakes system. 6. Navigating the Dark Forest: Challenges, Ethics, and Future Prospects Building and operating an arbitrage bot in the DeFi “Dark Forest” is a testament to the power of decentralized technologies, but it also brings to light significant challenges and ethical considerations. While our system demonstrates the theoretical and practical viability of automated arbitrage, it’s crucial to acknowledge the adversarial landscape and its broader implications. The Constant Battle Against Malicious Actors: The Role of Bloom Filters The initial optimism surrounding DeFi’s permissionless nature has, unfortunately, been tempered by the proliferation of malicious actors. Our Honeywall Service serves as a vital last line of defense, but the ingenuity of these bad actors constantly demands evolving countermeasures. Honeywall Service A key component of this defense is the Bloom filter. A Bloom filter is a probabilistic data structure that can quickly and efficiently test whether an element is a member of a set. It is highly space-efficient but carries a small probability of “false positives” (indicating an element is in the set when it’s not), though never “false negatives.” In our context, the Bloom filter is used to pre-filter incoming event data from the Blockchain Data Reader Service. It contains hashes of known malicious or high-tax liquidity pair addresses. Before any detailed processing or reserve fetching, a quick check against the Bloom filter can immediately discard known problematic pairs, preventing wasted computational resources and potential risks. Bloom filter Despite the sophisticated pre-checks implemented in our custom ArbHelperMap smart contract (specifically the _checkTaxLimit logic that simulates a round-trip swap to assess taxes), some malicious pairs still manage to bypass these initial on-chain validations. They achieve this by manipulating the getAmountsOut function (used for price queries) to return seemingly normal, low-tax outputs. However, the true "honeypot" logic is embedded deeper within the actual swapExactTokensForTokens or underlying transfer functions. These functions might impose exorbitant hidden taxes (e.g., 99%) on sell operations, or even completely restrict selling, effectively trapping funds. ArbHelperMap _checkTaxLimit getAmountsOut swapExactTokensForTokens transfer During our testing phase, we encountered a significant number of such deceptive pairs. I have personally collected some pairs addresses that successfully passed initial getAmountsOut checks but revealed hidden taxes or sell restrictions only during an actual (simulated or reverted) transaction in a local database. This db file will be made publicly available on the project's GitHub repository, serving as a community resource to help others avoid these pitfalls. This ongoing cat-and-mouse game underscores the necessity of continuous monitoring, rapid blacklisting, and a multi-layered defense strategy. getAmountsOut Ethical Implications and the Dark Forest’s Shadow The “Dark Forest” analogy is apt not only for the cut-throat competition among bots but also for the broader ethical landscape of DeFi. The efficiency of arbitrage, while crucial for market health and price discovery, comes with a stark reality: profits generated by arbitrageurs often represent value extracted from less sophisticated market participants. The pervasive culture of FOMO (Fear Of Missing Out), coupled with a general lack of understanding of underlying blockchain mechanisms and financial instruments, makes many retail users easy prey in this environment. They enter highly volatile markets, interact with new protocols, and execute trades without full awareness of concepts like slippage, MEV, or hidden contract taxes. FOMO (Fear Of Missing Out) This dynamic, while economically logical for those with advanced tools, casts a shadow on the reputation of decentralized technologies. The narrative can quickly shift from “financial empowerment” to “predatory behavior,” eroding trust in a space that otherwise holds immense promise. DeFi, at its core, aims to democratize finance, offering permissionless access and transparency. However, the sophisticated nature of MEV and the prevalence of scams can inadvertently undermine these ideals, creating a two-tiered system where only the technologically adept can truly navigate safely. It is imperative that, as builders, we acknowledge these ethical dimensions and advocate for greater user education, robust security audits, and mechanisms that protect vulnerable participants. Conclusion: Still Navigating the Dark Forest Despite the inherent complexities and persistent challenges of the DeFi landscape, the journey of engineering this arbitrage bot has been a remarkable validation of theoretical principles meeting practical implementation. We successfully demonstrated the power of an event-driven, Go-based architecture, optimized for speed, precision, and data-driven insights, capable of detecting and executing multi-leg arbitrage opportunities. speed, precision, and data-driven insights Initially, a common expectation within the “Dark Forest” was that the vast majority of arbitrage value would be immediately intercepted by large, well-resourced players, leveraging self-hosted nodes and direct access to block producers. However, our testing and successful transactions have shown that it is indeed possible for smaller, well-engineered bots to consistently find and capitalize on these ephemeral opportunities: While profitable arbitrage on older AMM models like Uniswap V2 (which primarily rely on constant product pools) can be challenging to sustain long-term due due to escalating gas costs and heightened competition, the “Dark Forest” continues to evolve. Newer implementations, such as Uniswap V3’s Concentrated Liquidity AMMs (CLAMMs), introduce novel arbitrage vectors that require more sophisticated modeling but often yield higher returns due to increased capital efficiency. Furthermore, the burgeoning field of cross-chain arbitrage, leveraging bridges and inter-blockchain communication protocols, presents a vast frontier for extracting value from price discrepancies across different blockchain networks. These continuous evolutions suggest that this challenging environment will remain a fertile ground for sophisticated and adaptable bots. So, while I’m still poor, I can confidently say I’ve become an excellent forest navigator. My compass is sharper, my map more detailed, and I understand the whispers of the canopy. My compass is sharper, my map more detailed, and I understand the whispers of the canopy. Project Repository For those eager to dive deeper into the practical implementation and the very real data behind some of the “traps” we discussed, a sanitized version of our codebase and a database populated with known malicious token pairs are available in my GitHub repository. These 85 specific pairs, while numerically small, generate a disproportionately significant transaction volume as they continuously attempt to lure naive bots into unprofitable trades. It’s a stark reminder of the ever-present dangers in this ecosystem and underscores the critical need for robust security checks. GitHub repository GitHub repository References Dan Robinson, Georgios Konstantopoulos. “Ethereum is a Dark Forest”,* Paradigm. Guillermo Angeris, Hsien-Tang Kao, Rei Chiang, Charlie Noyes, and Tarun Chitra. “An analysis of Uniswap markets”, Cryptoeconomic Systems. Claudio Gebbia. “Analysis and Implementation of Arbitrage Bots in Centralized and Decentralized Finance”, University of Zurich. Y. Zhang, Z. Li, T. Yan, Q. Liu, N. Vallarano and C. J. Tessone, “Profit Maximization In Arbitrage Loops”. arxiv. Dan Robinson, Georgios Konstantopoulos. “Ethereum is a Dark Forest”,* Paradigm. Paradigm Guillermo Angeris, Hsien-Tang Kao, Rei Chiang, Charlie Noyes, and Tarun Chitra. “An analysis of Uniswap markets”, Cryptoeconomic Systems. An analysis of Uniswap markets”, Cryptoeconomic Systems Claudio Gebbia. “Analysis and Implementation of Arbitrage Bots in Centralized and Decentralized Finance”, University of Zurich. Analysis and Implementation of Arbitrage Bots in Centralized and Decentralized Finance”, University of Zurich Y. Zhang, Z. Li, T. Yan, Q. Liu, N. Vallarano and C. J. Tessone, “Profit Maximization In Arbitrage Loops”. arxiv. “Profit Maximization In Arbitrage Loops” arxiv