Tags: #kv-caching
LLM Inference Server & Desktop Utility
macOS
11.7k
jundot/omlx
An LLM inference server optimized for Apple Silicon, featuring continuous batching, tiered KV caching, and macOS menu bar management for efficient local AI.
An LLM inference server optimized for Apple Silicon, featuring continuous batching, tiered KV caching, and macOS menu bar management for efficient local AI.