vLLM

vLLM is an open-source inference engine for LLMs with PagedAttention for efficient memory management. Serves models 2-4x faster than traditional approaches.