AI-powered UI Automation Framework
12.8k 2026-04-27
web-infra-dev/midscene
An AI-powered, vision-driven UI automation framework that enables natural language control and scripting across web, mobile, and custom interfaces.
Core Features
Natural Language Automation: Control UIs by describing goals and steps.
Cross-Platform Support: Automate tasks on Web, Android, iOS, and custom interfaces.
Comprehensive APIs: Offers Interaction, Data Extraction, and Utility APIs for developers.
Advanced Debugging: Provides visualized replay reports, a built-in playground, and a Chrome Extension.
Integration Capabilities: Seamlessly integrates with tools like Puppeteer and Playwright.
Quick Start
npm install @midscene/webDetailed Introduction
Midscene.js is a cutting-edge, AI-powered UI automation framework that revolutionizes how users interact with digital interfaces. By combining computer vision and natural language processing, it allows for intuitive automation of tasks across web browsers, Android, iOS applications, and even bespoke interfaces. It empowers both developers and non-technical users to create robust automation scripts using natural language descriptions or a JavaScript SDK, offering a more efficient and accessible approach to UI automation with powerful APIs and debugging tools.