AI-300T00: Operationalize Machine Learning and Generative AI Solutions

• Design and implement MLOps infrastructure on Azure.
• Manage the machine learning model lifecycle and operations.
• Implement machine learning training, deployment, monitoring, and maintenance workflows.
• Design and implement GenAIOps infrastructure.
• Deploy and manage foundation models for production workloads.
• Implement generative AI quality assurance and observability.
• Optimize RAG performance and generative AI model performance.
• Use infrastructure as code with Bicep and Azure CLI.
• Automate workflows using GitHub Actions.
• Apply monitoring, tracing, evaluation, and optimization to production AI systems.

Module 1: Design and Implement an MLOps Infrastructure

Create and Manage Resources in a Machine Learning Workspace

• Create and manage a workspace
• Create and manage datastores
• Create and manage compute targets
• Configure identity and access management for workspaces

Create and Manage Assets in a Machine Learning Workspace

• Create and manage data assets
• Create and manage environments
• Create and manage components
• Share assets across workspaces by using registries

Implement Infrastructure as Code for Machine Learning

• Configure GitHub integration with Machine Learning
• Deploy Machine Learning workspaces and resources by using Bicep and Azure CLI
• Automate resource provisioning by using GitHub Actions workflows
• Restrict network access to Machine Learning workspaces
• Manage source control for machine learning projects by using Git

Module 2: Implement Machine Learning Model Lifecycle and Operations

Orchestrate Model Training

• Configure experiment tracking with MLflow
• Use automated machine learning to explore optimal models
• Use notebooks for experimentation and exploration
• Automate hyperparameter tuning
• Run model training scripts
• Manage distributed training for large and deep learning models
• Implement training pipelines
• Compare model performance across jobs

Implement Model Registration and Versioning

• Package a feature retrieval specification with the model artifact
• Register an MLflow model
• Evaluate a model by using responsible AI principles
• Manage model lifecycle, including archiving models

Deploy Machine Learning Models for Production Environments

• Deploy models as real-time or batch endpoints with managed inference options
• Test and troubleshoot model endpoints
• Implement progressive rollout and safe rollback strategies

Monitor and Maintain Machine Learning Models in Production

• Detect and analyze data drift
• Monitor performance metrics of models deployed to production
• Configure retraining or alert triggers when thresholds are exceeded

Module 3: Design and Implement a GenAIOps Infrastructure

Implement Foundry Environments and Platform Configuration

• Create and configure Foundry resources and project environments
• Configure identity and access management with managed identities and RBAC
• Implement network security and private networking configurations
• Deploy infrastructure using Bicep templates and Azure CLI

Deploy and Manage Foundation Models for Production Workloads

• Deploy foundation models using serverless API endpoints and managed compute options
• Select appropriate models for specific use cases
• Implement model versioning and production deployment strategies
• Configure provisioned throughput units for high-volume workloads

Implement Prompt Versioning and Management with Source Control

• Design and develop prompts
• Create prompt variants and compare performance across different prompts
• Implement version control for prompts using Git repositories

Module 4: Implement Generative AI Quality Assurance and Observability

Configure Evaluation and Validation for Generative AI Applications and Agents

• Create test datasets and data mapping for comprehensive model evaluation
• Implement AI quality metrics, including groundedness, relevance, coherence, and fluency
• Configure risk and safety evaluations for harmful content detection
• Set up automated evaluation workflows using built-in and custom evaluation metrics

Implement Observability for Generative AI Applications and Agents

• Examine continuous monitoring in Foundry
• Monitor performance metrics, including latency, throughput, and response times
• Track and optimize cost metrics, including token consumption and resource usage
• Configure detailed logging, tracing, and debugging capabilities for production troubleshooting

Module 5: Optimize Generative AI Systems and Model Performance

Optimize Retrieval-Augmented Generation (RAG) Performance and Accuracy

• Optimize retrieval performance by tuning similarity thresholds, chunk sizes, and retrieval strategies
• Select and fine-tune embedding models for domain-specific use cases and accuracy improvements
• Implement and optimize hybrid search approaches combining semantic and keyword-based retrieval
• Evaluate and improve RAG system performance using relevance metrics and A/B testing frameworks

Implement Advanced Fine-Tuning and Model Customization

• Design and implement advanced fine-tuning methods
• Create and manage synthetic data for fine-tuning
• Monitor and optimize fine-tuned model performance
• Manage a fine-tuned model from development through production deployment

4 Days

Certification: Microsoft Certified: Machine Learning Operations Engineer Associate

Exam: AI-300: Operationalizing Machine Learning and Generative AI Solutions

Duration: 120 minutes

Format: Proctored Microsoft certification examination

• Create and configure an Azure Machine Learning workspace.
• Create and manage data, environments, components, compute, and datastores.
• Track machine learning experiments using MLflow.
• Perform automated machine learning and hyperparameter tuning.
• Build and execute machine learning pipelines.
• Automate ML workflows with GitHub Actions.
• Deploy machine learning models to real-time or batch endpoints.
• Test and troubleshoot deployed model endpoints.
• Implement progressive rollout and rollback strategies.
• Monitor model performance and data drift.
• Configure Microsoft Foundry environments and projects.
• Deploy foundation models for production workloads.
• Create and manage prompt versions using Git.
• Evaluate generative AI applications using quality and safety metrics.
• Configure observability, logging, tracing, and monitoring.
• Optimize RAG retrieval performance.
• Fine-tune and manage models through production deployment.

• Data Scientists
• Machine Learning Engineers
• DevOps Professionals
• AI Engineers
• Professionals designing and operating production-grade AI solutions on Azure

• Experience with Python programming.
• Foundational understanding of machine learning concepts.
• Basic familiarity with DevOps practices such as source control, CI/CD, and command-line tools.
• Experience or familiarity with Azure Machine Learning and Microsoft Foundry is relevant to the certification-level skills.
• Familiarity with GitHub Actions, Bicep, and Azure CLI is recommended.