gRPC Conf 2025

When you’re building AI-powered tools — especially ones that rely on fast feedback loops and multiple services — communication speed and clarity matter a lot. That’s the situation I found myself in while developing MergeWhisperer, an AI-based code review assistant.

In this talk, I’ll share how we evaluated gRPC for our architecture: the performance gains we hoped for, the developer experience concerns we ran into, and where it actually fits in a modern AI-heavy stack. This isn’t a deep technical dive — it’s a practical reflection from a builder trying to keep things fast, scalable, and realistic. If you’re curious whether gRPC makes sense for your own systems, this session will give you the questions (and red flags) worth considering.

Modern enterprises are embracing gRPC to modernize their most critical systems - systems that were once built on legacy stack. But as we scale, especially with bidirectional streaming, things get complex fast.

In this panel, seasoned engineers from one of the largest production deployments of gRPC will openly discuss:

- What worked well when scaling bidirectional gRPC in production
- What really hurt: hidden pitfalls, debugging nightmares, resource bottlenecks
- Design patterns and tradeoffs around stream multiplexing, client-side routing, observability, and backpressure
- Where the ecosystem can improve: tooling gaps, protocol evolution, and developer experience

This talk shares how users can securely expose their gRPC services to external consumers using a Kubernetes-native API management gateway built on Envoy Proxy and the Kubernetes Gateway API specification. While gRPC offers major performance advantages for internal communication, exposing it at the edge introduces a unique set of challenges around security, routing, and protocol translation.

We will walk through how the gateway architecture solves these challenges in a production environment, allowing users to publish gRPC APIs externally with confidence. In addition to native gRPC routing, we will discuss how our platform enables users to layer critical Quality-of-Service (QoS) features like authentication, rate limiting, retries, and policy enforcement on top of their gRPC APIs for external consumption.

This presentation demonstrates how gRPC enables scalable AI-powered quote generation systems processing documents 87.3% faster than traditional approaches. We'll explore distributed microservices architecture leveraging gRPC's bidirectional streaming for NLP pipelines, achieving 94.6% product identification accuracy across concurrent requests.
The technical deep-dive covers gRPC service mesh coordinating AI components: document parsing, NLP extraction, sentiment analysis, and text generation. We'll examine Protocol Buffers for efficient serialization and gRPC's load balancing for system resilience.
Key insights include streaming gRPC calls for real-time refinement, interceptors for auth/rate limiting, and backward-compatible protobuf schemas. Performance benchmarks show gRPC's HTTP/2 multiplexing reduces latency 43% vs REST when coordinating AI inference calls.
Includes Kubernetes deployment strategies, OpenTelemetry monitoring, error propagation handling

Are you passionate about Load Balancing? Join fellow Load Balancing enthusiasts in this interactive Birds of a Feather session! This isn't a typical presentation; instead, it's a unique opportunity to engage in open discussion, share your experiences, and get answers to your burning Load Balancing questions from the collective wisdom of the group. Come prepared to connect, learn, and contribute!

This session is for engineers who’ve deployed gRPC at scale and are ready to go deeper. We’ll dissect how deadlines propagate across service boundaries and impact retry behavior, why batching strategies (manual vs. automatic) are essential for CPU-bound and streaming workloads, and how keepalive pings affect long-lived connections in load-balanced environments—especially with proxies like Envoy in the mix. Expect insights from production tuning, including edge cases with connection churn, head-of-line blocking, and deadline expiration races. You'll walk away with a nuanced understanding of how to fine-tune your gRPC services for high-performance, low-latency systems under real-world conditions.

Reddit runs a multi-cluster, multi-region, multi-cloud k8s environment. We generally do not use proxies to connect our services together, except for cluster-wide ingress proxies. However as our infrastructure became more complex service owners were faced with choosing kube DNS or setting up application code to make the routing match our increasingly complicated infrastructure.
We solved this problem using grpc and XDS integration to provide standard routing that would enable clients to do client-local load balancing w/ orca data, and a control plane to replicate the routes across cluster boundaries . With multi-cluster service discovery, smart clients only specify a single name that works either locally (incluster) or picks the nearest cluster. This architecture eliminates all proxies and also gives Infra teams a control surface for efficient load balancing across all workloads.