Economics · 7 min read
Dedicated GPU server vs GPU cloud: what the hourly price hides
The lines that do not appear on an hourly GPU price: storage that keeps billing, egress, idle hours, pre-emption, shared hosts. And when the cloud is still the right call.
In short
- A month is 730 hours: the median on-demand H100 costs about $2,460 a month running; our H100 SXM is $1,134 whether it runs or not.
- The hourly price hides storage that keeps billing, egress, idle hours, pre-emption and shared hosts.
- Cloud for bursts and uncertainty; dedicated for anything that runs more than half the hours in a month.
An hourly price looks small and a monthly price looks large, and the two are rarely compared on the same footing. This guide puts them on one: a card that runs for a month, with everything a month of running costs.
730 hours
A month is 730 hours. The median on-demand H100 in 2 September 2026 was $3.37 an hour, which is $2,460 a month if the instance runs continuously; the cheapest in-stock offer was $1.73, or $1,263. Our H100 SXM is $1,134 a month, whether it runs or not. The cloud is cheaper for any workload that runs less than 336 hours a month at the median rate, or 655 at the cheapest. Most serving workloads and most multi-week training runs are above both.
The lines under the hourly price
- Storage
- Cloud volumes bill whether the instance runs or not, at $0.08 to $0.20 per GB per month. A 2 TB dataset is $160 to $400 a month before compute. On a dedicated server, 1.5 TB of NVMe per H100 is in the price.
- Egress
- $0.05 to $0.09 per GB at the large clouds. Moving a 500 GB checkpoint out five times a month is $125 to $225. Ours is $0.
- Idle time
- An instance you forget to stop bills at full rate. A server billed by the month cannot surprise you.
- Pre-emption
- Spot capacity is cheap because it can vanish mid-run. A dedicated server is yours for the term; the only interruption we allow ourselves is announced maintenance under four hours a month.
- Shared hosts
- Many cloud GPU instances are virtual machines on hosts shared with other tenants: PCIe bandwidth, CPU and network are contended. A dedicated server has one tenant.
- Quotas and approval
- Eight H100s at a large cloud is a quota request and a sales call. Here it is a form.
Side by side
| DediGPU | Dedicated GPU hosts | GPU clouds | |
|---|---|---|---|
| 1× H100, per month | $1,134 — the cost sheet | $1,230 – $2,099 | $2,460 (median on-demand ×730 h) |
| 1× RTX 4090, per month | $138 | $274 – $323 | $307 (median on-demand ×730 h) |
| How the price is set | published cost, zero margin | cost plus margin, unpublished | market rate, changes weekly |
| Setup fee | $0 | $0 – $99 | $0 |
| Minimum term | one month, no notice | one to twelve months | none |
| GPU dedicated to you | yes, whole server | yes | often shared hosts |
| Egress | $0 | metered above a quota | $0.05 – $0.09 per GB |
| Identity check | none | ID + card, usually | full KYC, credit card |
| Payment | crypto balance | card, wire, PayPal | card |
Dedicated-host figures are the published monthly prices of Latitude.sh, GPU-Mart and HostKey for a single-card server on 2 September 2026; cloud figures are the median on-demand rate tracked by getdeploying.com the same day, times 730 hours.
When the cloud is right
- Bursts. A hyper-parameter sweep that needs 64 GPUs for six hours is a cloud job. Nobody rents 64 cards for a month for that.
- Uncertainty. If you do not know whether you need a GPU at all next week, pay by the hour until you do.
- Managed services. Notebooks, pipelines, model registries and the people who run them. We rent hardware.
- Compliance paperwork. If your customer needs a SOC 2 report from the GPU provider, a crypto-only host with no KYC is not it.
When dedicated is right
- Anything that runs more than about half the hours in a month.
- Serving, where latency and noisy neighbours matter more than elasticity.
- Large datasets on local NVMe, moved often.
- Teams that want one bill they can predict, paid up front, with no card on file.
The cheapest way to check for your own workload is the calculator: a card, a count, a term, and the number a month of the cloud would have to beat. For an RTX 4090 the number is $139; for an H100, $1,134.
Keep reading.
