Machine Learning Operations Engineer Associate AI-300

• Design and implement an MLOps infrastructure.
• Manage machine learning model lifecycle and operations.
• Design and implement a GenAIOps infrastructure.
• Implement generative AI quality assurance and observability.
• Optimize generative AI systems and model performance.
• Use Azure Machine Learning for model training, deployment and monitoring.
• Use Microsoft Foundry for generative AI applications and agents.
• Implement infrastructure as code using Bicep and Azure CLI.
• Automate workflows using GitHub Actions.
• Monitor and optimize production AI systems.

Module 1: Design and implement an MLOps infrastructure

• Create and manage resources in an Azure Machine Learning workspace.
• Create and manage machine learning assets.
• Implement infrastructure as code for Machine Learning.
• Configure GitHub integration.
• Deploy Machine Learning workspaces and resources using Bicep and Azure CLI.
• Automate resource provisioning with GitHub Actions.
• Restrict network access to Machine Learning workspaces.
• Manage source control for machine learning projects using Git.

Module 2: Implement machine learning model lifecycle and operations

• Orchestrate model training.
• Configure experiment tracking with MLflow.
• Use automated machine learning.
• Use notebooks for experimentation and exploration.
• Automate hyperparameter tuning.
• Run model training scripts.
• Implement distributed training.
• Implement training pipelines.
• Compare model performance across jobs.
• Register and version machine learning models.
• Evaluate models using responsible AI principles.
• Manage the model lifecycle.
• Deploy models as real-time or batch endpoints.
• Test and troubleshoot model endpoints.
• Implement progressive rollout and rollback strategies.
• Monitor machine learning models in production.
• Detect and analyze data drift.
• Configure retraining or alert triggers.

Module 3: Design and implement a GenAIOps infrastructure

• Implement Microsoft Foundry environments and platform configuration.
• Configure identity and access management.
• Implement network security and private networking.
• Deploy infrastructure using Bicep and Azure CLI.
• Deploy and manage foundation models for production workloads.
• Select appropriate models for specific use cases.
• Implement model versioning and production deployment strategies.
• Configure provisioned throughput for high-volume workloads.
• Implement prompt versioning and management using source control.
• Design and develop prompts.
• Create and compare prompt variants.
• Implement prompt version control using Git repositories.

Module 4: Implement generative AI quality assurance and observability

• Configure evaluation and validation for generative AI applications and agents.
• Create test datasets and data mappings.
• Implement AI quality metrics.
• Evaluate groundedness, relevance, coherence and fluency.
• Configure risk and safety evaluations.
• Automate evaluation workflows.
• Implement observability for generative AI applications and agents.
• Monitor performance metrics including latency, throughput and response times.
• Track token consumption and resource usage.
• Configure logging, tracing and debugging for production troubleshooting.

Module 5: Optimize generative AI systems and model performance

• Optimize retrieval-augmented generation (RAG) performance and accuracy.
• Tune similarity thresholds and chunk sizes.
• Optimize retrieval strategies.
• Select and fine-tune embedding models.
• Implement hybrid search.
• Evaluate RAG performance using relevance metrics and A/B testing.
• Implement advanced fine-tuning methods.
• Create and manage synthetic data for fine-tuning.
• Monitor and optimize fine-tuned model performance.
• Manage fine-tuned models from development through production deployment.

4 Days

Certification: Microsoft Certified: Machine Learning Operations Engineer Associate
Exam: AI-300: Operationalizing Machine Learning and Generative AI Solutions
Duration: 100 minutes of exam time; Microsoft role-based exams may have 120 minutes of seat time when no lab is included
Format: Computer-based, proctored Microsoft role-based certification examination

• Data scientists.
• Machine learning engineers.
• DevOps professionals.
• Professionals designing and operating production-grade AI solutions on Azure.
• Professionals working with MLOps and GenAIOps workflows.

• Experience with Python.
• Foundational knowledge of machine learning concepts.
• Basic familiarity with DevOps practices.
• Experience with source control, CI/CD and command-line tools.
• Experience with Azure Machine Learning.
• Experience with Microsoft Foundry.
• Familiarity with GitHub Actions.
• Knowledge of infrastructure as code using Bicep and Azure CLI.