OSS Alternative - Discover Top Open Source Alternatives to Popular Software

kyegomez/OpenMythos

An open-source, theoretical reconstruction of the Claude Mythos LLM architecture, featuring a Recurrent-Depth Transformer and sparse Mixture of Experts for advanced reasoning.

Core Features

Implements Recurrent-Depth Transformer (RDT) with three stages (Prelude, Recurrent Block, Coda).

Supports switchable attention mechanisms (MLA and GQA).

Utilizes a sparse Mixture of Experts (MoE) for compute-adaptive, depth-variable reasoning.

Provides pre-configured model variants ranging from 1B to 1T parameters.

Built with PyTorch, offering a flexible framework for LLM research.

Quick Start

pip install open-mythos

Detailed Introduction

OpenMythos is an independent, community-driven open-source project that aims to theoretically reconstruct the Claude Mythos large language model architecture based on publicly available research. It introduces a novel Recurrent-Depth Transformer (RDT) design, incorporating distinct Prelude, Recurrent Block, and Coda stages. A key innovation is its sparse Mixture of Experts (MoE) implementation, enabling efficient, compute-adaptive, and depth-variable reasoning. Developed in PyTorch, OpenMythos provides researchers and developers with a flexible framework to explore and experiment with advanced LLM architectures, offering scalable model configurations from 1 billion to 1 trillion parameters.