After failing to realize the savings that public cloud was supposed to deliver, there’s a growing trend of enterprises moving their applications back to on-premise infrastructure. The complaint—well-documented in tech circles—is the out-of-control spending in the cloud. My main observation has always been that enterprises tend to have applications that aren’t cloud-native and don’t scale up or down. For this reason, enterprise applications demand a fixed amount of CPU and RAM and rarely scale their needs down. Call this their baseline computing need. If there’s a really high baseline of compute required in an organization, each compute instance in the cloud is being billed at 730 hours per month. Let’s see if we can concisely answer the question, “How Much Cloud Can You Get Out of a Single Rack”.
Just a reminder: the promise of the cloud was unlimited scaling with the option to buy as much or as little as you’d like, with no long-term commitment or capital expenditure.
A quick tangent—bear with me. Early in my career, I used to drive to client locations almost all day long. Having a car with no factory-installed air conditioning made this pretty miserable during hot Virginia summers, so I decided it was time to upgrade. When I finally settled on the car I wanted, my monthly payment was about $30 more than I had initially budgeted. I had developed a really good rapport with the sales manager, but we hit an impasse on price. We were both stuck—until he asked if it would be worth $1 per day to show up at client sites without the back of my shirt soaked in sweat. This method of taking a cost and breaking it down to a daily rate is called “breaking it down to the ridiculous.” For the purposes of this model, I wanted to apply the same logic.
So, the goal in this article is to answer the question: What would a 10 kW setup cost on-premise, and what would the equivalent amount of compute cost in the public cloud? When we examine large environments—or talk in the tens of millions—it’s hard to visualize the impact. That’s why I decided to relate everything to a single cabinet drawing 10 kW in a standard data center.
Rather than clutter the beginning of this post with all my assumptions, I’ll list them at the end. I developed the tech stack I wanted to use and stuck with notable brands. I will mention that I had AI price out my model to save time. These costs included:
- Hardware
- Software
- Data center space
- AWS costs
I also let AI recommend the AWS instance sizes, and I used 3-year, all-upfront reserved instances to save as much money in the cloud as possible.
The On-Premise Solution
From my days as a Sales Engineer who priced out private cloud solutions, I think this is a decent representation of what an organization might build:
- Cabinet: Maxed out at 10,000 watts, with 80% sustained power load (~8,000W)
- 500TB of usable storage (with redundancy) using JBODs
- 16 high-density servers running VMware ESX as host servers
- Each server: dual AMD EPYC CPUs, 512GB RAM, NVMe cache
- Redundant switches, firewalls, and management switches
The Yield:
- 413 VMs across a 3-tier architecture: web, app, and database servers
- Web: 290 VMs (1–2 vCPU / 1–4GB RAM)
- App: 101 VMs (2–4 vCPU / 8–16GB RAM)
- DB: 22 VMs (4–10 vCPU / 16–32GB RAM, in a large, sharded cluster)
On-Premise Costs for 3 Years
CapEx (One-Time Investment)
- Servers, storage, and networking: $215,500
- VMware licensing (vSphere + vCenter): $78,000
- Total CapEx: ~$293,500
OpEx (Recurring, Over 3 Years)
- Colocation (space, power, bandwidth): $118,800
- Hardware maintenance contracts: $77,580
- VMware support: $58,500
- Total OpEx: ~$255,000
Monthly Cost (Over 3 Years)
- $14,000 per month
Note: I used a 3-year window in this model. It’s likely that modern hardware, in a climate-controlled data center, would last 5 years or more. For readability, I’ve included hardware specifications in the appendix.
AWS – Public Cloud Equivalent
While we’re paying for all of these instances upfront, the numbers below are shown monthly for easier comparison. It’s not a perfect mapping to our 10 kW on-premise solution, but we used the following instance types to best match:
3-Year Reserved Instance Pricing (All Upfront)
- 290
t3.mediuminstances for web servers - 152
m6i.large/xlargefor app servers - 33
r6i.xlargefor database VMs - 500TB of EBS
gp3storage with multi-AZ redundancy - 1 Gbps bandwidth and network services (NATs, load balancers)
AWS EBS Storage Costs
For 500TB (512,000 GB) of General Purpose SSD (gp3) storage:
- Storage Cost: $0.08 per GB per month
- Total: 512,000 × $0.08 = $40,960/month
Bandwidth Costs
While I specified unmetered 1 Gbps bandwidth in the colocation model, it wasn’t meant to be fully sustained. I’ll assume we only sustain 25% bandwidth usage in AWS:
- 81,000 GB/month @ $0.09 = $7,290/month
(A full 1 Gbps sustained would be ~324,000 GB/month)
AWS Cost Summary (Monthly, with 3-Year Upfront)
- EC2 Instances: $12,835
- EBS Storage: $40,000
- Data Transfer: $7,290
- Total: $60,125/month
- Total Over 3 Years: $2.16 Million
Final Cost Comparison
| Model | Monthly Cost | 3-Year Total Cost |
|---|---|---|
| Public Cloud (AWS) | $60,125 | $2.16 Million |
| On-Premise | $14,403 | $518,500 |
Issues with the Model
In our How Much Cloud Can You Get Out of a Single Rack analysis, It’s easy to poke holes in a model like this. Rather than let others do it, here are some critiques and clarifications of my own assumptions:
Model Changes That Could Favor AWS
Why didn’t you price out object storage?
You’re right—object storage is cheaper. But in this scenario, our enterprise applications can’t use object storage. They require storage mapped as actual block devices.
No elasticity?
Correct. Many enterprises I work with don’t have elasticity either. Cloud migrations are often “lift and shift” VM-to-VM operations with no dynamic scaling.
250 Mbps of sustained bandwidth seems excessive.
Fair point. Removing bandwidth saves $7,290/month or $262,440 over 3 years—about 12% of total cloud costs.
What about containers like EKS, ECS, or Fargate?
Sure, that’s a cheaper model—but we could also run Kubernetes on-premise for far less. Plus, many enterprises still run 3-tier legacy apps.
No staffing costs for on-prem?
Ongoing management can be outsourced to a data center or MSP. Talent pools are often similar either way.
Why not hybrid or Direct Connect?
True. But then we’d need to factor in port charges and outbound data fees ($0.02–$0.09 per GB). Inbound is typically free.
Model Changes That Could Favor On-Premise
Servers don’t need to be replaced after 3 years.
Agreed. Hardware can easily last 5+ years, reducing long-term costs.
Cloud model still has a huge upfront cost.
Yes. The AWS pricing assumed a 3-year upfront payment. Using on-demand pricing would raise the 3-year total to $2.68 million ($74,500/month).
VMware licensing is expensive.
Correct. Alternate hypervisors like Hyper-V, XCP-ng, or Nutanix could cut costs.
Appendix A – Assumptions
- No specific load balancing solution was included in either model
- Replicas/snapshots were available, but no offsite redundancy was considered
- On-prem storage used JBODs instead of brand-name SAN/NAS solutions
- vCPUs in VMware do not directly equate to AWS vCPUs
- Servers: Supermicro 1U twins or Dell R650s
- AMD EPYC 9354P (32 cores, 2.75GHz, 280W)
- 512GB RAM
- 2 × 1.92TB NVMe SSD
- Dual 10G or 25G NIC
- Switching: Redundant 48-port switches with 10/25G and 2× 40/100G uplinks
- Firewalls: Enterprise-grade (e.g., Fortinet or Palo Alto)