Ecosystem & Stack: vllm

A comprehensive framework from ModelScope for efficiently fine-tuning, evaluating, and deploying over 1000 large language models and multimodal large models using advanced techniques.

A unified, production-ready inference API for effortlessly deploying and serving open-source language, speech, and multimodal AI models across various environments.

A low-code development tool for building and iteratively optimizing multi-agent LLM applications with agility and efficiency.

LMCache is an LLM serving engine extension designed to significantly reduce Time-To-First-Token (TTFT) and boost throughput, especially for long-context scenarios, by intelligently reusing KV caches.

An easy-to-use, scalable, and high-performance open-source framework for Reinforcement Learning from Human Feedback (RLHF) based on Ray and vLLM.

TaxHacker is a self-hosted AI accounting app that automates expense and income tracking for freelancers and small businesses by analyzing receipts, invoices, and transactions with LLMs.

An ultra-compact 7B parameter AI agent designed by Microsoft to automate multi-step computer tasks through visual perception and direct interface interaction.

An end-to-end platform for fine-tuning, evaluating, and deploying open-source Large Language Models (LLMs) and Vision Language Models (VLMs).

A Python library for generating and curating high-quality synthetic data for AI model training and structured data extraction.

CosyVoice is an advanced multi-lingual large language model-based text-to-speech system offering state-of-the-art voice generation, cloning, and full-stack deployment capabilities.

A toolkit for quantizing (compressing) Large Language Models (LLMs) with hardware acceleration across various GPUs and CPUs, integrating with popular inference frameworks.

An efficient and user-friendly library for scaling Reinforcement Learning with Large Language Models on large-scale GPU resources.

A modular framework for aligning any-modality large models with human intentions and values using diverse fine-tuning and reinforcement learning methods.

An open-source project providing Chinese LLaMA-2 and Alpaca-2 large language models with enhanced Chinese capabilities and support for ultra-long contexts up to 64K.

A tokenizer-free, multilingual Text-to-Speech system offering advanced voice design, controllable cloning, and high-quality audio output.

A versatile interface for OuteTTS models, providing flexible text-to-speech generation capabilities across various AI inference backends and hardware platforms.

A state-of-the-art open-source text-to-speech system leveraging LLMs to generate human-like, emotional, and low-latency speech with zero-shot voice cloning capabilities.