Experiment Tracking

After this skill you can…

• Log experiments with MLflow or Weights & Biases
• Compare runs in the experiment UI
• Register the best model to a model registry

Prerequisites

Watch (10 videos)

Deploying ML Models & LLMs on GCP with MLOps: A Practical Demo

Analytics Vidhya · beginner

→ Log and compare model runs→ Use experiment tracking for MLOps

Grinder vs Grinder 😱 This Was NOT Supposed to Happen #technology

Factsivo · advanced hands-on

→ Design experiments to test power tool limitations→ Analyze the effects of extreme pressure on materials

Toyota Research Institute on Experiment Tracking with Weights & Biases

Weights & Biases · beginner hands-on

→ Track experiments→ Streamline workflow→ Collaborate on research

An Experiment Tracking Tutorial with Mlflow and Keras

Automata Learning Lab · beginner hands-on

→ Track Experiments→ Analyze Model Performance→ Compare Hyperparameters

Why Experiment Tracking is Crucial to OpenAI

Weights & Biases · advanced hands-on

→ Track experiments→ Compare results→ Optimize resource utilization

Track your machine learning experiments locally, with W&B Local - Chris Van Pelt

Weights & Biases · intermediate

→ Use W&B Local to track machine learning experiments→ Capture system level metrics and metadata→ Compare thousands of experiments

Troubleshooting and Iterating ML Models with Lee Redden (2019)

Weights & Biases · beginner

→ Track model experiments→ Analyze model results

Metaflow Tags: Basic Tagging

Outerbounds · beginner hands-on

→ Track experiments→ Organize ML workflows

MLOps Tutorial: Build a Full ML Pipeline with MLflow, DVC & Deploy on AWS

Analytics Vidhya · beginner hands-on

→ Use MLflow for experiment tracking→ Track model performance and hyperparameters

Model CI/CD Course: LLM Evaluation results

Weights & Biases · intermediate

→ Track model experiments→ Compare baseline and candidate models

Read (10 articles)