LLM Inference at Scale: vLLM and llm-d

Wed, 26 Aug 2026 00:00:00 +0000

Running AI infrastructure breaks a lot of the traditional ways we’re used to dealing with systems. In this blog, I write about how you can serve AI models efficiently for inference on Kubernetes on hardware like GPUs (which are super expensive!!!), and how vLLM can help you manage memory efficiently while llm-d can help load balance the system smartly to your needs.

Vllm on Kashish Blog

LLM Inference at Scale: vLLM and llm-d