Build Versus Buy

The comparison almost everyone runs first is a price per million tokens against a GPU price per hour. It is the wrong comparison, and the reason is a single missing variable. A hosted API charges you for tokens you consumed. A GPU charges you for time, whether or not you consumed it. Converting between the two requires knowing how many tokens the GPU produces per hour in your workload, and that number is a property of your traffic pattern, not of the hardware.

Get that variable wrong by a factor of five, which is entirely normal for a first estimate, and the break-even point moves by a factor of five. Everything else on this page follows from that.

All prices were observed on 22 August 2026 and each links to its source. This page is marked as analysis: the prices are facts, the framework for combining them is an interpretation.

The two cost shapes

Hosted pricing is variable cost with near-zero fixed cost. Self-hosting is fixed cost with near-zero marginal cost. That is the entire structural difference, and it means the two are not comparable at a single traffic level. They are two lines with different gradients. The interesting question is where they cross.

The break-even formula

Write your own numbers into this. It is deliberately arithmetic, not a rule of thumb.

# Hosted side: cost of serving one month of traffic through an API
hosted_monthly =
      (monthly_input_tokens  / 1e6) × input_rate
    + (monthly_output_tokens / 1e6) × output_rate

# Self-hosted side: cost of the capacity, regardless of use
self_hosted_monthly =
      gpus × gpu_hourly_rate × 730
    + engineering_cost_monthly
    + observability_and_egress_monthly

# Capacity you actually get, which is the term people omit
deliverable_tokens_monthly =
      gpus × sustained_output_tokens_per_second_per_gpu × 3600 × 730 × utilisation

break_even_when: self_hosted_monthly / deliverable_tokens_monthly
                 < hosted_monthly     / (monthly_input_tokens + monthly_output_tokens)

Four of those terms are cheap to obtain and two are not.

• input_rate and output_rate come straight off a vendor page. Observed 22 August 2026: gpt-5.1 at $1.25 per million input tokens and $10.00 per million output tokens (OpenAI pricing); Claude Sonnet 5 at $2 input and $10 output per million tokens (Claude pricing).
• gpu_hourly_rate comes off a cloud page. Observed 22 August 2026: $3.99 per GPU per hour for an 8×H100 SXM instance and $2.79 for 8×A100 SXM 80 GB (Lambda GPU cloud); an effective reserved rate of $41.528 per hour for p5.48xlarge (8×H100) in US East (N. Virginia), which is $5.191 per accelerator (EC2 Capacity Blocks pricing).
• monthly_input_tokens and monthly_output_tokens you measure. Do not estimate them; every vendor returns token counts in the response.
• sustained_output_tokens_per_second_per_gpu and utilisation are the hard terms. They are not published anywhere for your workload, because they depend on your model, your quantisation, your sequence lengths and your arrival pattern. You have to benchmark them. Until you have, the break-even is unknown, not favourable.

The honest way to use this formula before you have benchmarked is to invert it: solve for the throughput you would need to beat the API, then ask whether that throughput is plausible. If the answer requires the GPU to be busy nearly all the time at a batch size that violates your latency target, the API wins and no further modelling is needed.

Utilisation is the term that decides it

Self-hosted cost per token is inversely proportional to utilisation. Halve utilisation and you double cost per token, with no change to any price on any page. This makes the traffic shape more important than the traffic volume.

Consider two workloads with identical monthly token counts. One is a nightly batch job that runs for four hours; the other is an interactive assistant used during office hours in one time zone. The batch job can saturate a GPU for its window and then release it. The assistant has a peak-to-trough ratio that means capacity sized for peak sits mostly idle, and capacity sized for the mean fails at peak. The first is a plausible self-hosting candidate. The second is not, unless you have other work to backfill the troughs.

This is why the batch tier on hosted APIs is such a strong competitor to self-hosting: the vendor is doing exactly the backfilling you cannot do, and passing part of it back. Anthropic publishes a 50% discount on both input and output tokens for its Batch API (Claude pricing, observed 22 August 2026), and Google states that Gemini models are available in batch mode at a 50% discount (Vertex AI pricing, observed 22 August 2026). Your self-hosted cluster has to beat the discounted rate, not the standard one, for exactly the workload that is easiest to self-host.

Continuous batching, KV cache sizing and chunked prefill are the mechanics of raising utilisation. They are covered in GPU capacity and utilisation.

Latency SLOs cap your batch size, and batch size is your throughput

Throughput on a GPU comes from batching: running many sequences through the same forward pass so the weights are read from memory once and amortised across requests. Larger batches mean more tokens per second per GPU and a lower cost per token.

They also mean higher latency for each individual request, because each request waits for the batch. The vLLM tuning documentation exposes this as max_num_seqs and max_num_batched_tokens, and it documents the failure mode when you push it too far: insufficient KV cache space causes vLLM to preempt requests and recompute them, which “can adversely affect end-to-end latency”. Same page, same knob, both directions.

So a latency SLO is a throughput ceiling, and a throughput ceiling is a cost floor. A tight time-to-first-token target for an interactive product forces small batches and therefore a high cost per token. A background pipeline with a per-hour deadline can run large batches and reach a much lower one. Two teams in the same company, self-hosting the same model on the same hardware, can legitimately compute self-hosted costs per token that differ several-fold, and both be right.

Write the SLO down before you model the cost. A cost model without a stated latency target is not a cost model.

Reserved against on-demand capacity

If you self-host, the second decision is how you buy the hardware, and the published discount structure is steep in both directions.

The Vertex rows deserve attention because they are a hosted vendor selling you the self-hosted cost shape: a committed hourly rate for reserved throughput rather than a per-token charge. The gradient between the one-week and one-year commitment on the Vertex AI pricing page is the price of optionality, quoted explicitly. That is a useful reference point, because it tells you what a vendor charges to carry the utilisation risk you would otherwise carry yourself.

Note also that AWS states on its Capacity Blocks pricing page that reservation prices are updated regularly based on supply and demand trends, and that the current prices were scheduled to be updated next in October 2026. Accelerator capacity is not priced like a commodity VM. A twelve-month financial model built on today’s reservation rate is a projection, not a quote.

The costs that never appear per token

The per-token comparison omits everything below. In a small team these terms frequently exceed the compute.

Engineering time, continuously. Serving stack upgrades, kernel and driver compatibility, model version migrations, capacity planning, autoscaling, and being on call for a component whose failure mode is a GPU out-of-memory error at 3am. The vLLM tuning documentation is a fair sample of the surface area you take on: preemption modes, tensor and pipeline parallel sizing, chunked prefill budgets, attention backend selection, NUMA binding on multi-socket nodes. None of that is difficult in isolation. All of it is somebody’s job.

Evaluation and regression testing. You now own model version changes. When you upgrade weights or change quantisation, output quality moves and you need a harness to detect it. With a hosted API you also own this, but the vendor at least publishes deprecation timelines.

Redundancy. One GPU node is not a production deployment. A second is not free, and it is idle capacity by design, which drags utilisation down and cost per token up.

Observability and egress. Metrics, tracing and log retention for the serving tier, plus network egress if the model and the application are not co-located.

Against that, self-hosting buys things a per-token rate cannot:

Data exposure control. Prompts and completions do not leave infrastructure you control. This is the argument that most often decides the question for regulated workloads, and it is a compliance requirement rather than a cost saving. Note that hosted vendors have partially priced this: Anthropic documents a 1.1× multiplier on all token categories for US-only inference on Claude 4.6 and later on its pricing page, and OpenAI documents a 10% uplift on data residency endpoints for eligible models released on or after 5 March 2026 on its pricing page. Data locality now has a published price on both sides of the decision.

Freedom from rate limits and deprecation. Your capacity is your capacity, and a model you have downloaded cannot be retired underneath you. Anthropic’s pricing page lists several models as retired or retired-except-on-partner-platforms; that is a migration you do not schedule.

Marginal requests at zero cost. Once the cluster is paid for, an extra evaluation run costs nothing. This changes what your team is willing to try, which is a real benefit and an easy one to overstate.

A decision order that avoids the common error

1. State the latency SLO. Without it, throughput and therefore cost per token are undefined.
2. Measure token volumes and traffic shape from a hosted deployment. Buy first; you cannot model what you have not observed, and the API is the cheapest instrument available.
3. Check whether the workload is batchable. If it is, compare against the discounted batch rate, which both Anthropic and Google publish at a 50% discount on their pricing pages (observed 22 August 2026).
4. Benchmark sustained throughput for your model and sequence lengths at the batch size your SLO permits. This is the number the whole comparison rests on.
5. Add the operational terms before comparing: headcount, redundancy, observability.
6. Decide on the non-cost axes too. Data exposure and deprecation risk are frequently decisive and are not denominated in tokens.

The pattern this ordering produces is unsurprising and worth stating plainly: hosted APIs suit variable, spiky, low-to-moderate volume workloads and any team without a platform function; self-hosting suits high, steady, latency-tolerant volume, or a hard data-residency constraint, in an organisation that already runs GPUs for something else. The interesting cases sit in the middle. For those, fill in the formula above.

Related

• Inference cost structure - what hosted vendors meter
• GPU capacity and utilisation - the throughput side
• Context window economics - why prompt design changes both sides
• /ai - the wider section