Continuous rightsizing: why cloud savings evaporate

Almost every organization running cloud workloads has done a rightsizing exercise at some point: review the instances, find the oversized ones, shrink them and watch the bill drop. The problem shows up a few months later, when the bill climbs back up without anyone making an explicit decision to spend more. The savings were not lost because of a single mistake. They were lost because rightsizing was treated as a project instead of an operating practice.

This article explains why savings decay, which FinOps principles are worth applying, which signals to base decisions on, and how an AI agent can sustain the work over time without taking control away from the people accountable for availability.

Why one-off savings don't last

The right size for a resource is a snapshot of a moment. Operations, on the other hand, never stop moving. At least four forces push spending back up:

• New workloads. Every new project deploys resources sized on estimates made before real usage is known. That estimate is rarely revisited after launch.
• Defaults. Infrastructure as code templates, shared modules, Helm charts and console wizards ship with predefined sizes. If the default is generous, every copy inherits the excess.
• Fear of under-provisioning. For an operations team, a capacity-related incident has a visible and personal cost. Excess capacity has a diffuse cost paid from someone else's budget. From where they sit, oversizing is the rational choice.
• Application changes. A new release can cut memory usage or double it. If nobody connects deployments to size reviews, the resource stays tuned to an application that no longer exists.

On top of that, providers release new instance families with a different price to performance ratio. A resource that was well chosen two years ago may not be today, even if its usage has not changed.

The practical conclusion is simple: if rightsizing has no owner, no cadence and no mechanism to detect drift, the savings evaporate.

FinOps principles worth applying

The FinOps Foundation, a Linux Foundation initiative, publishes the FinOps Framework, an open framework for managing the financial value of the cloud. You don't need to adopt all of it to use its core ideas. For rightsizing, these are the most useful:

• Teams collaborate. Finance, technology and business share the same information and decide together. Rightsizing is not something Finance imposes on Operations.
• Everyone owns their usage. Whoever deploys a resource is accountable for its cost. To do that, they need to see it, ideally allocated through consistent tagging.
• Data should be accessible and timely. A monthly report arrives too late to fix a workload deployed at the wrong size three weeks ago.
• Decisions are driven by business value. The goal is not to spend the minimum, but to spend what the committed service level requires.

The framework also describes a three-phase cycle: Inform, Optimize and Operate. A common mistake is to stay in Optimize as a one-time event. Continuous rightsizing lives in the Operate phase: a recurring process with owners and metrics, not a project with an end date.

Which signals to use: percentiles, not averages

Most bad rightsizing decisions come down to a bad metric. Average CPU is the most widely used and the most misleading. A server averaging 15% can hit 90% every day during the accounting close. If you shrink it based on the average, the next close will be an incident.

We recommend working with these signals:

• High utilization percentiles (p95 or p99) for CPU and memory, calculated over a window that covers the full business cycle. For many financial processes that means at least a month, so the close is included.
• Memory on its own. CPU can be shared and throttled; memory cannot. Running out of memory usually ends with a crashed process, not a slower one.
• Saturation signals, not just usage: run queues, CPU throttling in containers, disk wait, IOPS and network throughput against the limits of the instance type.
• Service indicators: latency and error rate of the transactions that depend on the resource. If a change degrades them, the savings don't matter.
• Known calendar events: closes, campaigns, quarterly batch jobs. Information that is not in the metrics cannot be inferred from them.

In Kubernetes the logic is the same, applied to each container's requests and limits. Tools such as the Vertical Pod Autoscaler or cloud provider recommendations (for example, AWS Compute Optimizer or Azure Advisor) produce useful suggestions. Treat them as one more input, not as a decision.

How an agent proposes changes

A rightsizing agent is not a script that shrinks instances. It is a process that observes, reasons over the evidence and prepares a proposal a person can evaluate in minutes. A reasonable flow has these steps:

1. Inventory and context. The agent reads the resource inventory, its tags (owner, service, environment, criticality) and the applicable pricing, including commitments or reservations that affect actual savings.
2. Usage analysis. It computes percentiles over the defined window, cross-checks them against service indicators and the event calendar, and discards resources with insufficient history.
3. Evidence-backed proposal. For each candidate it produces a record: current size, proposed size, observed percentiles, headroom left after the change, estimated savings and assumptions.
4. Rollback plan. The record states how to revert, how long it takes and which indicator triggers the rollback.
5. Change through the normal channel. If infrastructure is managed as code, the proposal arrives as a pull request against the repository, not as a direct change in the console.
6. Post-change verification. After the change, the agent watches the indicators for an agreed period and reports whether the outcome matches the forecast.

An illustrative example, with made-up numbers meant to show the format and not to suggest typical results: a virtual machine with 8 vCPU and 32 GB shows, over 45 days that include a month-end close, a CPU p95 of 22% and a memory p99 of 11 GB. The agent proposes moving to 4 vCPU and 16 GB, which would put the estimated CPU p95 near 44% and leave 5 GB of memory headroom above the p99. It states the monthly savings at current pricing, notes that the resource has no associated reservation and defines the rollback: return to the previous size if the service's p95 latency exceeds the SLO threshold for fifteen minutes.

What matters in the example is not the numbers, but that every claim can be verified and that the rollback criterion is written before the change.

Approval flow based on risk

Not every change deserves the same level of review. Classifying by reversibility and criticality avoids two opposite failures: approving everything without looking, or blocking everything out of caution.

Type of change	Reversibility	Suggested approval
Adjusting requests in development environments	Immediate	Automatic within limits, with a weekly report
Resizing a non-critical service with autoscaling	Minutes	Service owner
Resizing a critical service or one under a committed SLA	Minutes, within a window	Service owner and change management
Databases, storage or a change of instance family	Slow or risky	Change board with a test plan
Purchasing or modifying usage commitments	Not reversible in the short term	Finance and Operations; the agent only recommends

This gradation ties into a broader question: how much autonomy to give an agent and how to expand it over time. It is best defined with an explicit scale of autonomy levels.

Measuring savings that stick

A rightsizing program that only reports "identified savings" proves nothing. What matters is how much of those savings is still in place six months later. We recommend measuring:

• Realized versus identified savings. How many proposals were approved, applied and remain applied.
• Rollback rate. What share of changes had to be undone. A high rate points to aggressive thresholds; a zero rate may mean the agent is too conservative.
• Attributable incidents. Any incident caused by a resize is analyzed as a problem and feeds back into the rules.
• Drift per resource. Resources that grew again after being adjusted, and why.
• Unit cost. Where possible, cost per transaction, per customer or per process. Total spend can rise because of legitimate growth; unit cost shows whether efficiency holds.

For these metrics to be reliable you need consistent tagging and a baseline recorded before every change. Without a baseline, any savings figure is an opinion.

Finally, address the cause and not just the symptom. If the agent finds that most of the excess comes from the same infrastructure module or shared template, the best proposal is not to adjust a hundred resources, but to fix the default that produced them. That finding is documented and prioritized alongside the rest of the automation backlog, for example using the Value and Complexity framework we use to rank initiatives.

If you are just getting started with agents in operations, our article on AI agents in IT operations provides the broader context.