Tags: #gguf
Local LLM Inference Server
Rust
4.0k
Michael-A-Kuykendall/shimmy
A Python-free Rust-based inference server providing an OpenAI-compatible API for local GGUF and SafeTensors LLM models.
Replaces:
Details A Python-free Rust-based inference server providing an OpenAI-compatible API for local GGUF and SafeTensors LLM models.