3. Orchestration and ML Pipelines 3.0 Introduction: ML pipelines and Mage Why do we need ML operations? How Mage helps MLOps Mage setup Example data pipeline 3.1 Data preparation: ETL and feature engineering Ingest raw data Prepare data for training Build training sets 3.2 Training: sklearn models and XGBoost Training pipeline for sklearn models Training pipeline for XGBoost Tracking training metrics with experiments 3.3 Observability: Monitoring and alerting Dashboard for sklearn training pipeline health Dashboard for XGBoost model explainability Dashboard for model training performance Alerts for successful pipeline runs and errors 3.4 Deploying: Infrastructure on AWS Containerization and Docker setup Deploy to AWS using Terraform Automate development workflow with CI/CD 3.5 Production: Running pipelines Automatic retraining pipeline Online inference pipeline for real-time predictions Code for entire project https://github.com/mage-ai/mlops.git Notes previous editions 2022 Prefect notes 2023 Prefect notes