AI
llm-d
How production LLM inference is actually served - the engine (vLLM) that optimises one replica's execution, the control plane (llm-d) that decides where each request lands, and why prefill and decode are split onto different hardware.
Read the llm-d guideWhere llm-d sits
How this component relates to the rest of the stack. Items without a link are not covered here yet.