OSS Alternative - Discover Top Open Source Alternatives to Popular Software

deepseek-ai/3FS

A high-performance distributed file system optimized for AI training and inference workloads, leveraging modern SSDs and RDMA networks for scalable and consistent storage.

Core Features

Disaggregated architecture for massive throughput.

Strong consistency using Chain Replication with Apportioned Queries (CRAQ).

Standard file interfaces, eliminating the need to learn new APIs.

Optimized for diverse AI workloads including data preparation, dataloaders, checkpointing, and KVCache.

Leverages modern SSDs and RDMA for superior performance.

Quick Start

git clone https://github.com/deepseek-ai/3fs

Detailed Introduction

The Fire-Flyer File System (3FS) is an advanced distributed storage solution meticulously engineered to meet the extreme demands of AI training and inference. It achieves unparalleled performance by disaggregating storage resources across thousands of SSDs and hundreds of storage nodes, interconnected via high-speed RDMA networks. 3FS ensures strong data consistency through its CRAQ implementation and offers familiar file interfaces for ease of use. It efficiently supports critical AI workflows such as data preparation, high-throughput checkpointing, and provides a cost-effective KVCache for large language model inference, establishing itself as a robust shared storage layer for AI infrastructure.