SLAs, SLOs & Error Budgets for ML

The Reliability Framework for Machine Learning

How do you measure whether a production machine learning service is functioning properly?

Site Reliability Engineering (SRE) defines a three level framework:

┌─────────────────────────────────────────────────────────────┐
│ 1. SLI (Service Level Indicator):  What we measure (P99 Latency = 45ms). │
│ 2. SLO (Service Level Objective):  Internal target (P99 Latency < 50ms). │
│ 3. SLA (Service Level Agreement):  External contract (P99 < 100ms or Pay!).│
└─────────────────────────────────────────────────────────────┘

                   Target Objective (SLO: 99.9% Uptime)
                                   │
                                   ▼
         [ ERROR BUDGET = 0.1% Allowable Downtime per Month ]
  (Use Error Budget to deploy new models and experiment safely!)

Infrastructure SLOs vs Model Quality SLOs

Traditional web services only monitor Infrastructure Metrics.

Machine learning services require both Infrastructure SLOs and Model Quality SLOs:

┌──────────────────────────┬──────────────────────────┐
│ INFRASTRUCTURE SLOs      │ MODEL QUALITY SLOs       │
├──────────────────────────┼──────────────────────────┤
│ - Availability: 99.9%    │ - Precision@10 > 0.80    │
│   uptime per month.      │ - Prediction Drift (PSI) │
│ - P99 Latency < 50ms.    │   < 0.10.                │
│ - API Error Rate < 0.1%. │ - Calibration Error (ECE)│
│                          │   < 0.05.                │
└──────────────────────────┴──────────────────────────┘

A model server can boast $100%$ infrastructure uptime while serving completely wrong predictions due to data drift. Model Quality SLOs prevent silent quality failures.

Managing Error Budgets

An Error Budget is calculated as:

$$\text{Error Budget} = 100% - \text{SLO}$$

• If your Availability SLO is $99.9%$, your monthly Error Budget is $0.1%$ (roughly $43$ minutes of allowed downtime per month).

How Teams Use Error Budgets

• Budget Remaining ($> 0$): Product teams have room to deploy new candidate models, run canary tests, and innovate quickly.
• Budget Exhausted ($0$): Deployments are frozen! All engineering efforts pivot to system stability, bug fixes, and infrastructure reliability until the budget resets.

Say this out loud

SLAs, SLOs, and Error Budgets govern machine learning reliability. Service Level Indicators measure operational metrics like latency and precision. Service Level Objectives set internal performance targets. Error budgets represent allowable downtime or quality drops, balancing fast model deployments against system stability.

Followups to expect

1. What happens when a Model Quality SLO is breached? Automated alerts notify MLOps teams, and traffic may automatically revert to a safe baseline model via an automated rollback mechanism.
2. What is Latency Tail Risk in ML APIs? Long response delays occurring in the 99th percentile (P99) of requests, often caused by dynamic batching timeouts or GPU memory swapping.