Tags: #multimodal-ai

An open-source Python framework for building real-time, voice-first, and multimodal conversational AI agents with composable pipelines.

SGLang is a high-performance serving framework for large language models and multimodal models, optimizing inference throughput and latency.

Nexent is a zero-code platform for auto-generating production-grade AI agents, unifying tools, skills, memory, and orchestration with built-in controls.

An open-source framework for building real-time, multimodal conversational AI agents with advanced features like voice assistance, diarization, and lip-sync.

An open lakehouse format for multimodal AI, offering high-performance random access, vector indexing, and data versioning.

An advanced multimodal AI chatbot framework that enables conversational interaction and deep understanding of video and image content, integrating various large language models.

An open-source, developer-friendly embedded retrieval library and multimodal AI lakehouse for fast, scalable vector search and data management.

A GPU-accelerated, scalable toolkit for multimodal data preprocessing and curation, designed to train better AI models faster.

A streamlined tool to accelerate the fine-tuning of popular multimodal models like Florence-2, PaliGemma 2, and Qwen2.5-VL.

BELLE is an open-source project dedicated to fostering the development of Chinese conversational large language models, aiming to make LLMs accessible to everyone.

Provides a multimodal toolset for AI agents to generate, edit, and display professional-grade images, videos, and audio using a CLI-powered architecture.

A declarative, transactional Python library for building multimodal AI applications with incremental data storage, transformation, indexing, and orchestration.

A unified, reproducible, and efficient multimodal evaluation toolkit for large language models across text, image, video, and audio tasks.

Parlor is an on-device, real-time multimodal AI that enables natural voice and vision conversations, running entirely on your local machine.

A modular and scalable PyTorch-based framework for state-of-the-art vision and language multimodal research from Facebook AI Research.

A series of video foundation models and large-scale datasets designed for comprehensive multimodal video understanding and generation.

A curated directory of open-source alternatives to ChatGPT and GPT-4, encompassing text and multimodal large language models, designed to assist users in navigating the AI landscape.

A comprehensive research hub for large-scale self-supervised pre-training of foundation models across diverse tasks, languages, and modalities.

A comprehensive multimodal AI system specializing in long-term streaming video and audio interactions, offering advanced vision-language understanding and composition.

A cloud-native framework for building and deploying high-performance multimodal AI applications with built-in scaling and orchestration.

Janus-Series is a family of unified autoregressive multimodal AI models designed for both understanding and generating content across various modalities, featuring a novel decoupled visual encoding strategy.

Edit Banana transforms static, uneditable content like images of diagrams into fully manipulatable and editable assets using advanced AI.

MMPreTrain is an OpenMMLab project providing a comprehensive, open-source PyTorch-based toolbox for pre-training and benchmarking various computer vision and multi-modal models.

InternGPT is an open-source, pointing-language-driven visual interactive system that significantly enhances user communication with AI models like ChatGPT, improving efficiency and accuracy in complex vision-centric tasks.

An open-source large language and vision assistant (LLaVA) that achieves GPT-4V level multimodal capabilities through visual instruction tuning.