vLLM

An in-depth guide to vLLM's User API, exploring how PagedAttention solves GPU memory bottlenecks and how to implement high-throughput LLM inference for models like DeepSeek-V3 and Claude 3.5 Sonnet.

A deep dive into how vLLM uses PagedAttention to solve GPU memory fragmentation and boost LLM serving throughput.

Inferact, a new startup founded by the creators of the vLLM project, has raised $150 million in a seed round valuing the company at $800 million to accelerate high-throughput LLM inference solutions.

A deep dive into decoupling the AI stack using vLLM, LiteLLM, and the SOLV Stack to move from local experimentation to enterprise-grade AI infrastructure.

Deep dive into Speculative Decoding: the technique that boosts LLM inference speeds by 2-4x without compromising model quality or weights.

A comprehensive guide to deploying and optimizing vLLM, the industry-standard inference engine for high-throughput LLM serving using PagedAttention.