Ecosystem & Stack: llamacpp

Run open-source large language models locally on your machine with a simple CLI, REST API, and client libraries.

A terminal tool that detects your hardware and recommends optimal LLM models, providing performance benchmarks for local execution.

Operit AI is the first fully-featured AI assistant application for Android, offering powerful tool-calling, local model support, and a built-in Ubuntu 24 environment for advanced automation.

A production-ready toolkit enabling developers to integrate private, offline, and fast on-device AI capabilities like LLMs, speech-to-text, and text-to-speech into their applications across various platforms.

llama-swap enables seamless hot-swapping and management of multiple local generative AI models, acting as a unified API gateway compatible with OpenAI and Anthropic standards.

A free and open-source Android application for local and remote interaction with various large language models, offering features like on-device inference and API key integration.

An open-source project providing Chinese LLaMA-2 and Alpaca-2 large language models with expanded Chinese vocabulary, enhanced capabilities, and support for ultra-long contexts up to 64K.

An open-source AI model family for high-fidelity, expressive speech and sound generation across diverse real-world applications.

A Node.js binding for llama.cpp, enabling local execution of large language models with advanced features like JSON schema enforcement and function calling.

A versatile interface for OuteTTS models, providing flexible text-to-speech generation capabilities across various AI inference backends and hardware platforms.

A local AI API platform for running various AI models (vision, speech, language) on diverse hardware with an OpenAI-compatible API.

A native macOS app that allows users to chat with a local LLM, responding with information from files, folders, and websites on your Mac, ensuring privacy and offline functionality.