← All talks
Talk
A Guide to AI API Gateways and Semantic Routers
Event: Cloud Native AI + Kubeflow Day, LLMs & Generative AI @ KubeCon + CloudNativeCon Europe Location: Amsterdam Delivered in: English Link to Conference Website
AI Engineering Cloud Native
Abstract
AI Engineering is taking over fast — the hype around GenAI and the cloud native landscape is adapting to this trend very fast. Through the complexity of providing product-ready LLMs, the options for managed services, and multiple other providers participating in this market, the demand and interest for routing requests and handling the traffic of AI-enhanced solutions has spiked.
Let's see what's out there: which solutions exist, which use cases they're good for, and when they might make no sense. We'll compare them side by side to unfold the differences between solutions like Envoy AI Gateway, LightLLM, and vLLM semantic routers — to name just the prominent ones. With this guide you'll be equipped to find your right communication handler.