The Real Cost of Cloud Repatriation: When On-Prem Is a $10M Mistake

Cloud repatriation makes sense when a workload runs flat, predictable, and above 65% utilization with no dependency on burst capacity. For everything else, especially AI/ML training pipelines and spiky traffic, the on-prem business case usually collapses once you add the three costs that never make it into the deck: infrastructure talent, capex refresh cycles, and lost elasticity. Before modeling any repatriation, most enterprises can cut their cloud bill 30 to 50% through rightsizing and commitment discounts, which often makes the whole exercise moot.

I have watched a well-run engineering org build a repatriation business case that looked airtight. A single spreadsheet. A monthly AWS bill on the left, a three-year server lease on the right. The delta was seven figures a year. The CFO loved it. The CTO signed off. Eighteen months later they were re-hiring bare-metal operations engineers at a premium, discovering that their "predictable" ML workload actually needed 10x capacity three days a month, and quietly moving pieces back to the cloud.

This is not a pro-cloud or anti-cloud article. Repatriation is real and it works for the right workloads. But the way most decks are built, they justify a decision that costs more than staying put. Here is the framework I use to tell the difference.

The Spreadsheet That Justifies a $10M Mistake

The classic repatriation model is two columns. Column A is your current cloud bill, annualized. Column B is the cost of equivalent hardware, either purchased outright or leased over three years, plus colocation. Column A is almost always bigger. The savings look obvious, and that is exactly the problem.

The model treats compute as a commodity you can swap one-for-one. It ignores that a managed cloud service bundles operations, patching, redundancy, and elasticity into the price. When you move to owned hardware, you unbundle all of that and someone on your team has to do it. That someone costs money the spreadsheet never counts.

Three line items are almost always missing:

- Infrastructure talent. You need people who can rack hardware, manage firmware, run capacity planning, and be on-call for physical failures. After a decade of cloud-native hiring, most orgs no longer have these people, and the market for them is thin.

- Capex refresh cycles. Servers are not a one-time purchase. GPUs age out in three to four years. You are signing up for a recurring capital expense and the depreciation schedule that comes with it.

- Elasticity loss. On-prem, you provision for peak. If your peak is 10x your average, you buy 10x the hardware and let 80% sit idle. Cloud lets you rent that peak by the hour.

When you add these back, the seven-figure annual savings frequently shrinks to a rounding error, or flips negative. The framework below forces those numbers onto the page before anyone signs.

Why 37signals Worked and Your AI Pipeline Won't

37signals (the company behind Basecamp and HEY) is the poster child for repatriation. They publicly documented moving off the cloud and projected around $7 million in savings over five years, plus a roughly $2 million reduction in annual spend after buying their own hardware [1][2]. It worked. But it worked for reasons specific to their workload.

Basecamp and HEY are steady-state applications. Traffic is predictable. Utilization is high and consistent. They do not spin up 500 GPUs for a weekend training run and then shut them off. Their demand curve is close to a flat line, which is the single best predictor of repatriation success. When your utilization is high and stable, owned hardware wins because you are not paying the elasticity premium for capacity you rarely use.

Now contrast that with an AI/ML training pipeline. GPU demand during a training run can spike 10x, then sit idle 80% of the time between runs. If you provision on-prem for that peak, you buy a fleet of expensive accelerators that mostly gather dust. If you provision for the average, your training jobs queue for days. Neither is acceptable.

The elasticity premium you pay in the cloud is insurance. You are paying a markup on peak capacity in exchange for never having to buy it. For bursty workloads, that insurance is cheaper than the alternative of owning idle hardware. This is why teams running modern cloud modernization and migration work keep training and inference bursts in elastic environments even when they repatriate steady-state databases. The workload shape decides, not the vendor.

The Hidden Cost Ledger Nobody Puts in the Deck

Every repatriation deck I have reviewed underweights the operational tail. Buying hardware is the visible cost. Running it for three years is where the money actually goes, and most of that never makes it into Column B.

Here is the ledger that belongs in every business case:

The talent line is the one that surprises people most. For ten years the industry optimized hiring around cloud-native skills: Terraform, Kubernetes, managed services. The engineers who know how to diagnose a failing RAID controller at 2am are now rare and expensive. You are not just paying salaries, you are competing in a shallow talent pool and accepting longer time-to-hire.

Then there is opportunity cost, which never appears as a line item at all. Every hour your engineers spend racking servers, negotiating colo contracts, or planning capacity is an hour not spent on product. For a company whose competitive advantage is software, redirecting engineering time to infrastructure plumbing is a strategic tax you pay quietly, quarter after quarter.

The Workload Repatriation Scorecard

Repatriation is a per-workload decision, not a company-wide one. The mistake is treating it as all-or-nothing. Score each workload independently against five dimensions, and let the score tell you which tier it belongs in.

The five dimensions that matter:

- Utilization %. How much of provisioned capacity is actually used, averaged over 90 days. High and flat is good for on-prem.

- Burst dependency. Does the workload need to scale 5-10x on demand? Burst dependency is the strongest signal to stay in the cloud.

- Data gravity. How much data lives here, and what does egress cost? Large, stable datasets with low egress favor on-prem.

- Compliance driver. Is there a regulatory or data-residency requirement forcing physical control? This can override economics.

- Team capability. Do you actually have the operations talent, or are you assuming it?

Here is how the scoring maps to action. A workload lands in a tier based on where it falls across the dimensions above.

- Green (repatriate candidate): Utilization above 65%, no burst dependency, high data gravity, capable team. Examples: primary production databases, object storage with stable access patterns, steady internal services.

- Yellow (optimize first, revisit): Utilization 40-65%, occasional bursts, moderate data gravity. Examples: mixed application tiers, batch jobs with predictable windows. Rightsize in cloud before deciding.

- Red (keep in cloud): Utilization below 40% or heavy burst dependency, low data gravity, thin ops team. Examples: AI/ML training, dev/test environments, seasonal traffic, event-driven pipelines.

If you want to see this scoring applied to your actual footprint, an engineering intelligence and cost visibility layer can attribute utilization and spend per workload so the scorecard runs on data instead of guesses.

Cut the Bill 40% Before You Ever Model Repatriation

Here is the uncomfortable truth for most repatriation projects: the cloud bill they are trying to escape is inflated by waste they never addressed. Industry FinOps data consistently shows organizations waste roughly 30% of cloud spend, with many estimates running higher [3][4]. Fix that first, and the repatriation delta often disappears.

The optimization playbook is well understood and boring, which is why teams skip it in favor of the dramatic repatriation project. Work through it in order:

1. Kill zombie resources. Unattached volumes, idle load balancers, orphaned snapshots, dev instances left running over weekends. Free money.

2. Rightsize. Match instance types to actual utilization. Most instances are provisioned two sizes too large.

3. Commit to discounts. Savings Plans and Reserved Instances cut steady-state compute costs meaningfully in exchange for one- or three-year commitments.

4. Use spot for batch. Interruptible workloads like CI, batch processing, and non-urgent training runs can run on spot capacity at a large discount.

5. Tier storage. Move cold data to infrequent-access and archive tiers automatically.

A tagging and rightsizing policy is the foundation. Without accurate tags you cannot attribute cost, and without attribution you cannot rightsize. Here is a minimal tagging enforcement and Savings Plan strategy example:

# Cost allocation tagging policy (enforced at provisioning)
required_tags:
  - key: team
    allowed: [platform, ml, product, data]
  - key: workload
    allowed: [prod-db, training, dev-test, inference, batch]
  - key: utilization_tier   # feeds the repatriation scorecard
    allowed: [green, yellow, red]

# Commitment strategy driven by workload tag
savings_plan:
  # Only commit to the flat, predictable baseline
  target: workload in [prod-db, inference]
  commitment_coverage: 0.70   # cover 70% baseline, keep 30% on-demand for headroom
  term: 1yr
  payment: no_upfront

spot_eligible:
  target: workload in [batch, training, dev-test]
  max_interruption_tolerance: high

The pattern here matters: commit only to the flat baseline, run bursty and interruptible work on spot, and keep 30% on-demand so you never over-commit. Teams that run this discipline routinely take 30 to 50% off the bill. That is the number your repatriation deck should be competing against, not the un-optimized bill you started with.

A Decision Framework CTOs Can Defend to the Board

A defensible repatriation decision follows four steps, in order, with no skipping. Skip step one and everything downstream is built on a bad number.

Step 1: Baseline true cost. Pull 90 days of spend by service and by tag. Not the summary invoice, the itemized breakdown. You need cost attributed to workloads, not to accounts. If your tagging is incomplete, fix that before anything else.

Step 2: Optimize. Run the full playbook from the previous section. Rightsize, commit, spot, tier, and kill zombies. Re-baseline after optimization. This new, lower number is what repatriation must beat.

Step 3: Isolate candidates. Run every workload through the scorecard. Only green-tier workloads proceed. Yellow gets optimized further and revisited in two quarters. Red stays in the cloud, full stop.

Step 4: Model three-year TCO. For each green candidate, build the full ledger including the hidden costs: hardware plus refresh, power and cooling, talent, capacity headroom, downtime risk, and migration cost itself. Compare against the optimized cloud number over three years, not one.

The board will ask two questions: what does it save, and what could go wrong. This framework answers both. It shows the optimized baseline, the per-workload rationale, the three-year TCO with hidden costs included, and the reversal cost if the assumptions break. That is a decision you can defend.

Frequently Asked Questions

Is cloud repatriation a good idea in 2026?

For specific workloads, yes. Repatriation makes financial sense for flat, high-utilization workloads above 65% sustained usage with no burst dependency and a capable operations team. For bursty AI/ML, dev/test, and seasonal workloads, it usually costs more once hidden costs are included.

How much can I save by repatriating?

It depends entirely on workload shape. 37signals reported around $2M annually and roughly $7M over five years [1][2], but their workloads were ideal candidates. Most enterprises can capture larger, lower-risk savings by first cutting 30 to 50% of cloud waste through rightsizing and commitments.

What workloads should stay in the cloud?

AI/ML training with spiky GPU demand, dev/test environments, event-driven pipelines, and any workload with strong burst dependency or utilization below 40%. The elasticity of the cloud is worth its premium for these.

What is the biggest hidden cost of going on-prem?

Infrastructure talent. After a decade of cloud-native hiring, the engineers who can run bare-metal operations are scarce and expensive, and most repatriation decks assume the existing team can absorb the work for free.

Your Next 30 Minutes

In the next 30 minutes, pull your last 90 days of cloud spend broken down by service and by tag. Not the invoice summary. The itemized cost-and-usage export. If more than 20% of your spend is untagged or unattributed, you have found your first project, and it is not repatriation. It is visibility.

This week, start tracking one metric: cost per unit of business value. Cost per active user, per inference, per transaction, whatever maps to what you actually sell. A rising cloud bill is only a problem if it rises faster than the value it produces. That single metric reframes the entire repatriation conversation from "the bill is big" to "the bill is efficient or it is not."

Then go back to that two-column spreadsheet. Add the three missing lines: talent, refresh, elasticity. Run every workload through the scorecard. Optimize the bill by 40% first. If, after all that, a green-tier workload still shows a three-year TCO advantage north of 20%, repatriate it with confidence. Most workloads will not clear that bar, and the ones that do are exactly the ones 37signals moved. The $10M mistake is not choosing on-prem. It is choosing it for the wrong workload with the wrong numbers.

References

[1] David Heinemeier Hansson, "The Big Cloud Exit FAQ," 37signals, 2023. https://world.hey.com/dhh/the-big-cloud-exit-faq-20274010

[2] David Heinemeier Hansson, "We stand to save $7m over five years from our cloud exit," 37signals, 2023. https://world.hey.com/dhh/we-stand-to-save-7m-over-five-years-from-our-cloud-exit-53a01f6

[3] Flexera, "State of the Cloud Report 2024," 2024. https://www.flexera.com/about-us/press-center/flexera-releases-2024-state-of-the-cloud-report

[4] FinOps Foundation, "State of FinOps 2024," 2024. https://data.finops.org/

[5] Andreessen Horowitz, Sarah Wang and Martin Casado, "The Cost of Cloud, a Trillion Dollar Paradox," 2021 (still the most cited analysis of cloud repatriation economics). https://a16z.com/the-cost-of-cloud-a-trillion-dollar-paradox/

[6] Amazon Web Services, "Savings Plans User Guide," 2024. https://docs.aws.amazon.com/savingsplans/latest/userguide/

[7] Google Cloud, "Rightsizing recommendations documentation," 2024. https://cloud.google.com/compute/docs/instances/apply-machine-type-recommendations-for-instances