gRPC Conf 2025

When the volume of incoming requests greatly exceeds cluster capacity, some calls will fail. Instead of failing calls randomly, an overload protection mechanism can load shed low priority calls. But when should load shedding be engaged? Have the requests been correctly prioritized? By successfully answering these questions for each gRPC service, we can safely operate with fewer resources while minimizing the risk of an overload event.

At Netflix, we built a system to automatically tune, test, and deploy customized load shedding configurations to each cluster. We will give an overview of how we reason about the available buffer for a cluster, our automation to achieve this buffer, what did and did not work for system utilization signals, and how these techniques are allowing us to run both fewer clusters and smaller clusters.