LMCache | LMCache blog website

Extending LMCache Backends: A Comprehensive Guide to Custom Backend Development

Learn how to build custom backends for LMCache using the external backend extension mechanism
By Baolong, Kobe
Posted on September 11, 2025

Abstract [Read More]
Tags:
- backend
- extension
- customization
- storage
- lmcache
🎉 LMCache Hits 5,000+ GitHub Stars — Thank You, Community!

A milestone that shows KV cache has become a first-class citizen in the LLM inference stack
By LMCache Team
Posted on August 28, 2025

We’re thrilled to share that LMCache has officially crossed 5,000 GitHub stars! 🚀 This milestone is not just a number — it’s a strong signal that KV cache technology has become a first-class citizen in the LLM inference stack, and that our community is leading the way. [Read More]
Tags:
- milestone
- community
- github
- stars
LMCache supports gpt-oss (20B/120B) on Day 1

Complete integration guide and performance benchmarks
By Yihua, Kobe
Posted on August 5, 2025

LMCache now supports OpenAI’s newly released GPT-OSS models (20B and 120B parameters) from day one! This post provides a complete guide to setting up vLLM with LMCache for GPT-OSS models and demonstrates significant performance improvements through our CPU offloading capabilities. [Read More]
CacheGen: Store Your KV Cache on Disk or S3—Load Blazingly Fast!
By Kuntai Du, Kobe
Posted on July 31, 2025

TL;DR: 🚀 CacheGen lets you store KV caches on disk or AWS S3 and load them way faster than recomputing! It compresses your KV cache up to 3× smaller than quantization so that you can load your KV cache blazingly fast while keeping response quality high. Stop wasting compute —... [Read More]
Shortest Prefill First—Smarter Scheduling for Faster Prefill!
By Kuntai Du
Posted on July 29, 2025

TL;DR: ⚡ Shortest Prefill First (SPF) scheduling cuts LLM time-to-first-token by up to 18% in prefill-decode disaggregation—unlocking even greater gains when combined with LMCache! [Read More]

Older Posts

Extending LMCache Backends: A Comprehensive Guide to Custom Backend Development

Learn how to build custom backends for LMCache using the external backend extension mechanism

🎉 LMCache Hits 5,000+ GitHub Stars — Thank You, Community!

A milestone that shows KV cache has become a first-class citizen in the LLM inference stack

LMCache supports gpt-oss (20B/120B) on Day 1

Complete integration guide and performance benchmarks

CacheGen: Store Your KV Cache on Disk or S3—Load Blazingly Fast!

Shortest Prefill First—Smarter Scheduling for Faster Prefill!