Cloud Companies Have Been Doing Load Balancing All Wrong—Here’s a Smarter Way

Table of Links

• Abstract and 1 Introduction
• 2 Background and 2.1 Programmable Multi-core SmartNICs
  • 2.2 Load balancers
• 3 SmartNIC-based Load Balancers and 3.1 Offloading to SmartNICs: Challenges and Opportunities
  • 3.2 Laconic Overview
  • 3.3 Lightweight Network Stack
  • 3.4 Lightweight synchronization for shared data
  • 3.5 Acceleration with Hardware Engine
• 4 Evaluation and 4.1 Experiment Setup

  • 4.2 End-to-end Throughput

  • 4.3 End-to-end Latency

  • 4.4 Evaluating the Benefits of Key Techniques

  • 4.5 Real-world Workload

• 5 Related Work
• 6 Conclusion and References
  • A Appendix

5 Related Work

Load Balancers. There are multiple works investigating high-performance load balancers. Katran [16] is an L4 load balancer implemented in eBPF. SilkRoad [38] and Hula [27] leverage programmable switches to build a high-performance L4 load balancer. Maglev [14] proposed by Google takes advantage of consistent hashing. Beamer [42] provides consistency without maintaining state. DRILL [19, 20] performs micro load balancing. Tiara [45] proposes a new hardware architecture for L4 load balancing. All of these works are based on connection-less load balancing and are unable to satisfy the need for connection-oriented L7 load balancing.

Network stack acceleration. TAS [28] provides a host TCP networking acceleration service. FlexTOE [44] and IOTCP [30] offload the host TCP stack onto the SmartNICs. Snap [36] is a high-performance networking stack proposed by Google. AccelTCP [39] offloads the control path of TCP onto the SmartNIC. Kim, D., Lee, S., and Park, K [29] explore offloading the TLS control plane onto the SmartNIC. Zilberman [47] offloads packet processing with XDP. Netchannel [11] disaggregates the network stack. They only offload a part of the control path of the networking protocols onto the SmartNIC and do not touch L7 protocols.

Measurements for SmartNIC. Multiple works provide valuable measurement for SmartNICs [26, 34]. Girondi et al. measured the performance of different high-speed concurrent hashtables [21]. We consulted these numbers during the design process.