TL;DR: A key contributor to the LMCache community just secured a major investment. This will greatly accelerate our mission of building the best KV cache library for every developer.
Come join us in building the future AI-native data layer!
An Independent Layer for the KV Cache
KV cache is no longer just a byproduct of the LLM inference process. It stores important information that can persist beyond the lifecycle of a query, an inference cluster, and even an LLM.
As a result, both industry and academia have started to realize that KV caches should be managed, distributed, and optimized independently of inference engines, storage devices, accelerators, and inference orchestrators.
In short, a new independent layer for AI-native dataβthe KV cacheβis being born. This new reality calls for new open-source projects, a community of domain experts, startups, and eventually products.
LMCache
From the very beginning, LMCache has been proud to play a central role in this evolution.
LMCache has pioneered the first open-source project and created a community of KV cache experts focused on managing KV caches when they are not in GPU memory. It now has over 220 contributors, with contributions from over 30 industry partners.
LMCache is designed to be an independent layer, with integrations across a wide range of inference engines, storage vendors, accelerator vendors, and inference orchestrators.
Tensormesh
Yet pushing the frontier of KV cache and maintaining such a project is an expensive endeavor. Investment is needed.
Tensormesh was founded a year ago by researchers from UChicago who started the LMCache project. LMCache has since grown substantially, with a vibrant and diverse contributor community, but Tensormesh continues to help maintain and improve key parts of the project.
Recently, Tensormesh secured $20M in funding, on top of its original $4.5M seed round, with major strategic investments from AMD, NVIDIA, and CoreWeave.
These investments show not only a commitment from major industry partners, but more importantly, a broader industry consensus that improving KV cache layer performance is key to unlocking the best performance from accelerators, storage, and inference systems.
Tensormesh exists to boost the adoption and development of LMCache and deepen its connections with other ecosystem partners. Tensormesh is committed to improving the performance, compatibility, and reliability of LMCache.
LMCache is not changing its open-source commitment.
Join Us
The LMCache community includes some of the best experts in storage, inference optimization, and machine learning algorithms.
The future of AI infrastructure is open. Join the LMCache community and help build the open-source KV cache layer together!
ππππ² ππ¨π§π§πππππ π°π’ππ‘ ππππππ‘π:
- Follow us on X: https://x.com/lmcache
- Follow us on LinkedIn: https://www.linkedin.com/company/lmcache-lab
- Join our Slack community: https://join.slack.com/t/lmcacheworkspace/shared_invite/zt-3eck2v8ub-j2hPn0AdzcfyykatTBXSTg
- Follow our WeChat Official Account: https://drive.google.com/file/d/1a-Sxe917QISjsbqmVjqzgpmD5YB6bovM/view?usp=sharing
Leave a Reply