Netflix/maestro

Maestro is Netflix's robust, general-purpose workflow orchestrator designed to manage complex data and machine learning pipelines at an immense scale. It operates as a fully managed workflow-as-a-service, serving a diverse user base including data scientists, engineers, and analysts. Capable of scheduling hundreds of thousands of workflows and millions of jobs daily with strict SLOs, Maestro emphasizes high scalability, extensibility, and enhanced usability. It integrates with various systems, including AWS and Kubernetes, and offers a Python SDK for programmatic workflow creation, streamlining the orchestration of critical business processes.