Your cloud fleet is bleeding -$5.9M every month.

Aroa Xinping

Data Scientist · FinOps Analytics

Context

Before we start: three concepts.

01

Virtual Machine

A server that runs in the cloud: Amazon, Google, or Microsoft's data centers. You pay by the hour, whether the machine is doing useful work or sitting idle.

02

CPU & Memory Utilisation

How busy the machine actually is. CPU = processing power. Memory = working space. A VM at 5% CPU is like a chef hired for a banquet who spends the whole shift washing one cup.

03

Over-provisioning

Requesting more resources than you will ever need, just in case. Engineers provision for peak load and nobody resizes later: the cost lands on a different team's budget.

Agenda

What we will cover today.

01Introductionwhat the system does

02The Problemwhy cloud waste happens

03The Data123,363 real VMs analysed

04The Insightwhy CPU alone is not enough

05The Modelthree predictions, not one

06Results$5.9M identified

07Recommendationswhat to do now

08Next Stepsroadmap for production

01 · Introduction

An ML system that scores every VM in your fleet.

→ Predicts CPU and memory per VM: p10, median, p95.
→ Maps each prediction to an action: terminate, downsize, or keep.
→ Asymmetric penalty: underestimating costs 3× more. Conservative by design.

Without a model, every sizing decision is a bet:

Terminate wrong outage SLA breach. Emergency re-provision. Trust lost. 3× penalty in the model

Keep wrong waste Idle VM billing 24/7. Nobody notices until audit. 1× penalty · a missed saving

Do nothing drift Waste compounds. Engineers move on. Fleet grows. 30% cloud spend wasted (Gartner)

The 3× coefficient means terminating a busy VM is penalised 3× harder than keeping an idle one. The model errs on the side of caution.

02 · The Problem

Any cloud. Same problem.

Amazon Web Services

Microsoft Azure

Google Cloud

No matter which provider you run on, every fleet accumulates idle machines over time. Engineers size for peak load, then move on. Nobody resizes later because the cost shows up on a different team's budget.

02 · The Problem

Over-provisioned and never revisited.

Cloud teams provision for peak load and forget. Each VM is a virtual server you pay by the hour, whether it works or not. Whoever requests the resources is not who pays the bill.

Gartner estimates 30% of cloud spend is wasted industry-wide. This fleet costs $11M/month, running at nearly double that waste rate.

-$5.9M/mo identified across 123,363 VMs · 54% of total spend

Keep (68%)

Downsize (16%)

Review (15%)

Terminate (1%)

VM Classification · SAP Cloud Infrastructure · 123,363 machines

03 · The Data

63% of all VMs run below 10% CPU.

CPU Utilisation Distribution · 123,363 VMs

The red bars are machines doing almost nothing, yet running 24/7 and billing by the hour. Two thirds of the fleet. That is the opportunity this model goes after.

SAP Cloud Infrastructure Dataset · CC BY 4.0 · Harvard Dataverse · Costs via AWS EC2 on-demand pricing us-east-1

04 · The Insight

Low CPU does not mean idle.

CPU at 7%, looks idle.
Memory at 72%, it's busy.
A single model terminates it. That's an outage.
CPU and memory measure different kinds of work.
One signal is not enough.
Two independent models, one per signal.
If either says keep → the VM stays.

avg 71.8% memory utilisation vs 7.9% CPU · same machines

Keep: low CPU, high memory

Downsize: low CPU, low memory

Hot (keep): high CPU

Zombie: terminate

Memory vs CPU Utilisation · representative sample

05 · The Model

Calibrated for risk, not just accuracy.

Three bounds per VM. Cautious when it matters. The model would rather miss a saving than cause an outage.

A range, not a single guess

Averages hide the extremes. We train three separate models, each targeting a different scenario, so every recommendation comes with a safety range:

p10Safe floor: usage almost never exceeds this p50Typical day: what normal usage looks like p95Worst case: peak load, fully covered

Errors are not symmetric

Switching off a busy server is catastrophic. Keeping an idle one wastes a few dollars. These are not the same, so the penalty isn't either:

too optimisticVM was busier than expected → outage risk · 3× penalty too cautiousVM was idler than expected → wasted money · 1× penalty

When in doubt, the model keeps the VM.

05 · The Model

Conservative by design.

If CPU says "terminate" but memory says "keep", the VM is kept. One cautious vote overrides.

Dual model, conservative by design

SPIKE DETECTED VM whose 30-day usage surges beyond its historical baseline

When usage spikes, the model can't predict reliably: it was trained on stable patterns, not anomalies.

Forcing a recommendation on a VM behaving abnormally risks a wrong action.

Spike detected → Both models bypassed

→ Flagged for human review → Manual decision

06 · Results

$5.9M/month identified.
54% of a $11M/month fleet · touching only 32% of VMs.

502Terminate

20,174Downsize

18,619Review

83,968Keep

Data quality

CPU

98.1%

MEM

87.6%

Monthly savings breakdown by action

07 · Recommendations

Four actions, ranked by confidence.

Week 1

Immediate

$240K/mo

502 VMs · terminate

No activity for 30 days. Shut them down · saving starts immediately.

Zero risk · act now

Weeks 1–10

Phased rollout

$4.1M/mo

20,174 VMs · downsize

Switch to a smaller size. Rolling 10% per week · 98% model confidence.

Low risk · phased

Weeks 2–3

Human review

$1.55M/mo potential

18,619 VMs · spike-flagged

Usage spikes detected. Rightsizing here risks outage. A human reviews each VM before any action is taken.

Uncertain · review first

Quarter 2

Architecture

18,619 VMs

spike-flagged · evaluate autoscaling

Spiky workloads are poor candidates for static rightsizing. Autoscaling policies absorb demand peaks without over-provisioning, eliminating the spike risk permanently.

Architecture change · high impact

08 · Next Steps

From analysis to production.
Four steps from one-off insight to a system that runs every night.

Automation

Nightly pipeline with Airflow + dbt

Planned · 10 weeks

Nightly refreshes · recommendations ≤24 h stale
dbt transforms versioned, tested, and audited
Airflow DAG with retry logic + Slack alerting

Data quality

Daily memory series

Data dependency

Daily snapshots replace 30-day rolling summaries
Detects seasonal memory spikes automatically
Pushes memory model coverage above 87.6%

Validation

Temporal cross-validation

Always ongoing

Walk-forward splits on unseen months only
Guards against overfitting on temporal data
Real performance ≠ in-sample accuracy

Data enrichment

Environment tags

Quick win · client-side

Tag VMs with env (prod / staging / dev) and owner
Unlocks automated triage · no human review needed for non-prod spikes
The model already has the signal; it just needs the context

Let's talk

Ready to find what's hiding in your fleet?

Potential savings $71M / year 54% of a $132M/year cloud bill · money you stop paying

Connect on LinkedIn View on GitHub