Reinforcement Learning Framework
9.4k 2026-04-26
OpenRLHF/OpenRLHF
An easy-to-use, scalable, and high-performance open-source framework for Reinforcement Learning from Human Feedback (RLHF), leveraging Ray and vLLM for distributed training of LLMs and VLMs.
Core Features
Scalable and high-performance RLHF with Ray + vLLM distributed architecture.
Unified agent-based design paradigm for extensible reinforcement learning.
Supports state-of-the-art RL algorithms including PPO, REINFORCE++, GRPO, RLOO.
Comprehensive RLHF pipeline capabilities: Supervised Fine-Tuning (SFT), Reward Model, RL Training.
Advanced support for Vision-Language Model (VLM) RLHF and multi-turn VLM interactions.
Detailed Introduction
OpenRLHF is a pioneering open-source framework designed for efficient and scalable Reinforcement Learning from Human Feedback (RLHF). It integrates Ray and vLLM for distributed training, enabling high-performance fine-tuning of Large Language Models (LLMs) and Vision-Language Models (VLMs). With its unified agent-based design, OpenRLHF offers a flexible and extensible pipeline for various RL algorithms, addressing the complexities of modern AI model alignment and interaction in both single and multi-turn scenarios.