LMCache | LMCache blog website

LMCache supports gpt-oss (20B/120B) on Day 1

Complete integration guide and performance benchmarks

By Yihua, Kobe

Posted on August 5, 2025

LMCache now supports OpenAI’s newly released GPT-OSS models (20B and 120B parameters) from day one! This post provides a complete guide to setting up vLLM with LMCache for GPT-OSS models and demonstrates significant performance improvements through our CPU offloading capabilities. [Read More]

CacheGen: Store Your KV Cache on Disk or S3—Load Blazingly Fast!

By Kuntai Du, Kobe

Posted on July 31, 2025

TL;DR: 🚀 CacheGen lets you store KV caches on disk or AWS S3 and load them way faster than recomputing! It compresses your KV cache up to 3× smaller than quantization so that you can load your KV cache blazingly fast while keeping response quality high. Stop wasting compute —... [Read More]

Shortest Prefill First—Smarter Scheduling for Faster Prefill!

By Kuntai Du

Posted on July 29, 2025

TL;DR: ⚡ Shortest Prefill First (SPF) scheduling cuts LLM time-to-first-token by up to 18% in prefill-decode disaggregation—unlocking even greater gains when combined with LMCache! [Read More]

LMCache Lab: Only prefilling? We reduce decoding latency by 60%!

By Kuntai Du

Posted on July 23, 2025

TL;DR: 🚀 LMCache Lab cuts decoding latency for code/text editing by 60% with speculative decoding! ⚡ [Read More]

LMIgnite: Fastest LLM Inference for Conversational and Long-Document AI, Only One Click Away

By Junchen Jiang, Hanchen Li

Posted on July 22, 2025

TL;DR: LLMs are transforming every product and service—from chatbots and copilots to intelligent document search and enterprise workflows. But running LLMs in production is still painfully slow, prohibitively expensive, and complex to manage. That changes today. [Read More]