AI Research Hub
22.1k 2026-04-18
microsoft/unilm
A comprehensive research initiative by Microsoft focusing on large-scale self-supervised pre-training to develop advanced foundation models across diverse tasks, languages, and modalities.
Core Features
Pioneering new foundation model architectures (e.g., DeepNet, RetNet, BitNet, LongNet).
Extensive collection of pre-trained models for NLP, computer vision, and speech processing.
Multimodal AI capabilities integrating language with vision, audio, and document layouts.
Emphasis on model scalability, efficiency, generality, and cross-lingual transferability.
Development of specialized models for tasks like text embeddings, document understanding, and text-to-speech.
Detailed Introduction
Microsoft's UniLM project serves as a central hub for cutting-edge research in large-scale self-supervised pre-training, aiming to build robust and versatile foundation models. It explores innovative architectures like DeepNet and RetNet to enhance model stability, generality, and efficiency. The project encompasses a broad spectrum of AI domains, including natural language processing, computer vision, speech, and multimodal understanding, providing a rich ecosystem of pre-trained models designed for diverse applications and supporting over 100 languages.