Tags: #gguf
AI Inference Server
4.7k
Michael-A-Kuykendall/shimmy
Shimmy is a Python-free Rust inference server that provides a 100% OpenAI-compatible API for running local Large Language Models (LLMs) with zero dependencies.
Replaces:
Details Shimmy is a Python-free Rust inference server that provides a 100% OpenAI-compatible API for running local Large Language Models (LLMs) with zero dependencies.