Load balancing for heterogeneous serverless edge computing: A performance-driven and empirical approach

Serverless edge systems simplify the deployment of real-time AI-based Internet of Things (IoT) applications at the edge. However, the heterogeneity of edge computing nodes – in terms of both hardware and software – makes load balancing challenging in these systems. In this paper, we propose a performance-driven, empirical weight-tuning approach to achieve effective load balancing based on the characteristics and capabilities of the nodes. By extensively profiling the nodes, we gather knowledge on performance metrics such as throughput, energy efficiency, response time, AI accuracy, and cost. Using this acquired knowledge, we introduce a weighted round-robin strategy to optimize the performance metrics according to their observed significance. To address multiple objectives, we introduce a multi-objective method that aims to strike a balance between any arbitrary set of performance objectives simultaneously. Additionally, we explore a coordinated distributed approach to overcome the limitations of centralized load balancing. Next, we introduce Hedgi, a heterogeneous serverless edge architecture designed to efficiently configure and utilize the derived load balancing policies, validated empirically. To demonstrate the practicality of Hedgi, we containerize and serverlessize a real-time object detection application. Extensive empirical studies are conducted using Hedgi to evaluate the performance of the proposed load balancing approach. The results provide valuable insights into the design trade-offs of various load balancing policies and system designs in the heterogeneous serverless edge.