Tim Furlong

What one copy of a frontier model costs

·
Tim FurlongTim Furlong
Cover image for What one copy of a frontier model costs

This is a companion to I wanted to know how AI data centers are built, where I worked through the one rack it would take to serve a single copy of a model like Opus 5.5. Once I had the parts list, I obviously wanted to know what it all costs (I spend my days looking at cloud bills, so this was the part I was most curious about).

Same caveat as before, only more so. Nothing about how Anthropic serves its models is public, and NVIDIA doesn't publish a price for this rack. So this is built on a representative machine (a rack in the style of NVIDIA's GB300 NVL72), other people's price estimates, and a couple of my own guesses where I couldn't find anything. I'm doing this to learn, and a rough number with the math shown was good enough for me.

Why everyone's asking right now

It's been hard to read the news this week without running into data center costs. Electric bills turned into a real issue in Tuesday's midterms. Fortune reports that PJM's market monitor blamed data centers for 63% of the price increase in one of its capacity auctions (about $9.3 billion that ended up on customers' bills), and candidates in both parties ran against them. At the same time, Wall Street is getting nervous about the other side of the ledger. Meta, Alphabet, Microsoft and Amazon spent close to $207 billion on capex last quarter and lost close to $900 billion in market value after reporting it, according to Bisnow, and economists quoted by CNBC say the buildout needs to slow to a more sustainable pace.

All of that is about enormous numbers: hundreds of billions of dollars, gigawatts, whole states. I have a hard time picturing any of it, so I wanted to look at the same question at the size of one rack.

Here's the rack again, if you want to poke at it while you read:

Fig 1. One rack, one frontier model · open it full size

What the rack costs

The best number I found comes from Data Gravity, which looked at actual purchase orders: just under $5.0 million for a GB300 NVL72 in August 2026, for small orders. Big cloud providers almost certainly pay less, and other reports range from about $4 million up to $6.5 million. Almost all of that is NVIDIA silicon:

What a ~$5.0M GB300 NVL72 is made of: GPU modules ~$3.4M, CPUs, NICs, trays and NVLink ~$1.2M, power, cooling, steel and integration ~$0.4M

The 72 GPU modules are about two thirds of the rack's price. The liquid cooling inside it is only about $50K.

Everything else in the picture

The rack isn't the whole bill, though. Here's the rest of what's in the figure:

  • The rack (GPUs, CPUs, NICs, switch trays, NVLink spine, power shelves, manifolds): ~$5.0M, from Data Gravity's purchase orders (via Wing VC)
  • Half of a leaf switch (144 ports, shared by 2 racks' worth of GPUs): ~$65K, from reseller listings at $107K to $150K
  • 800G optics and fiber for 72 links: ~$100K to $200K, my guess at roughly $700 to $1,200 a transceiver
  • The CDU: ~$32K, from Data Gravity, for a 250 kW unit
  • Busway, whips, cable trays and the network frame: ~$15K to $30K, my guess
  • Hardware in the figure: about $5.3M

The building and the power bill

Then there's the building. Data center construction runs about $11.3 million per megawatt according to JLL, and more like $15 to $25 million for AI-ready buildings. This rack draws about 0.135 MW, so its slice of the building is somewhere around $1.5 to $3.4 million. Most people rent that space instead of building it, and Data Gravity puts running costs (mostly power and colocation) at $240K to $410K a year.

As a sanity check on the power part: 135 kW, plus about 20% for cooling and losses in the building (my assumption), running all year is about 1.4 million kWh. That's about what 130 average American homes use in a year (the EIA puts a typical home at about 10,800 kWh). At around 9 cents per kWh, the US industrial average, it comes to roughly $128K a year in electricity. Not a huge share of this rack's yearly cost, as you'll see below, but multiply it across every rack being built right now and it's easy to see why it's showing up on people's electric bills.

One year of the rack

To turn that into a yearly number, I spread the hardware over five years (Amazon recently moved some of its servers to a five-year life, and other big clouds use six):

One year of the rack: hardware written off over 5 years ~$1.05M, power and space ~$0.33M

About $1.38M a year, or roughly $158 an hour for the whole rack ($2.19 per GPU-hour).

Down to a cost per token

In the first post I estimated this rack decodes about 22,600 tokens a second across 256 conversations at once. A rack that never rests would write about 712 billion tokens a year, so the cost per token depends a lot on how busy you keep it:

  • Busy 100% of the time: ~$1.94 per million output tokens
  • Busy 60% of the time: ~$3.23 per million
  • Busy 30% of the time: ~$6.46 per million

The one thing I took away from this: utilization matters as much as the hardware price. An idle rack costs nearly the same as a busy one.

What this leaves out

That's a lower bound on a narrow question. It leaves out reading prompts (prefill also takes GPU time), spare capacity for traffic spikes, storage, the staff who run it, the cost of training the model in the first place, and any margin. And it's not a claim about what Anthropic pays or charges. It's just what my one imaginary rack costs to run.

About the guesses

The rack price, its breakdown, the cooling cost, the CDU price, the running costs, construction costs and electricity prices are other people's published estimates (linked below). The optics and power distribution numbers are my own guesses. The 20% building overhead, the five-year write-off and the utilization levels are assumptions I picked, and the token rate comes from the model estimates in the first post. Change any of them and the answer moves, which was kind of the point of doing it this way. If you know these numbers firsthand and I'm way off, I'd love to hear about it.

Sources

In the news:

For the numbers:

The interactive figure is made with Anatomy, an open-source Claude skill. Thanks again to that project for making it possible.

More posts