How Much Does a Multi-Turn LLM Conversation Actually Cost?

§1 · Why a conversation is not "input + output, once"

An LLM API is stateless — it remembers nothing between calls. To continue a conversation, your client has to resend the entire history as input on every single turn: the system prompt and tool definitions (fixed, but resent each time), plus the full transcript of every previous user message and model response, plus the new message.

So the naive mental model — "price one prompt, that's the cost" — is wrong for anything conversational. A chatbot, a copilot, or an autonomous agent bills the same early tokens again and again. The single-prompt figure can be several times too low.

§2 · The math, made simple

Let the fixed per-turn pieces be: S = system prompt, T = tool definitions, U = a user message, A = an assistant reply. On turn t, the input you send is everything stable plus every prior exchange plus the new user message:

That per-turn input grows linearly with the turn number — each new turn adds one more (U+A) block that must be resent forever after. Now add it up across N turns. Summing a linearly growing quantity is the classic 1 + 2 + 3 + … + N pattern, which equals N(N+1)/2:

The history term carries an N². Double the turns and the history you resend roughly quadruples. Meanwhile output is different: you generate exactly one new reply per turn, so total output cost grows only linearly with N — and output is never cached by any provider. It is the resent input that quietly explodes; output just rides along.

§3 · A worked example — 10 turns on Claude Opus 4.8

Concrete beats abstract. Take a realistic assistant with a decent system prompt and a tool block, on Claude Opus 4.8 (July 2026 list price: $5 / 1M input, $25 / 1M output). Per-turn sizes:

• System (S) = 2,000 tokens · Tools (T) = 1,000 tokens
• User (U) = 200 tokens · Assistant (A) = 500 tokens

Turn 1 sends 2000 + 1000 + 200 = 3,200 input tokens. Turn 10 sends 2000 + 1000 + 9×700 + 200 = 9,500 — the same conversation, now three times heavier on input, entirely because of resent history.

Turn 10 costs 2.1× turn 1 — same task, all history resent. The "cached" columns assume every turn hits the cache within its TTL (see §4). Numbers computed with the exact model behind the conversation cost simulator.

The same result, laid out plainly

Without caching
  Input tokens resent, turn 1  = 2,000 + 1,000 + 200         =  3,200
  Input tokens resent, turn 10 = 2,000 + 1,000 + 9x700 + 200 =  9,500
  Total input over 10 turns    =  63,500 tok  -> $0.3175  (@ $5 / 1M in)
  Total output over 10 turns   =   5,000 tok  -> $0.1250  (@ $25 / 1M out)
  Total conversation cost      =  $0.4425

With prompt caching  (read 0.1x input, write 1.25x input)
  Cache reads over 10 turns    =  54,000 tok  -> $0.0270
  Cache writes over 10 turns   =   9,500 tok  -> $0.0594
  Total input with caching     =  $0.0864   (vs $0.3175 -> input down 73%)
  Total output (never cached)  =  $0.1250
  Total conversation cost      =  $0.2114

  Saved by caching  =  $0.2311   =  52% of total cost
                       (73% of input cost; output dilutes the total)

Two things to internalize from these numbers. First, the input side is where the money hides: input is 72% of the uncached bill even though each reply is longer than each question, purely because the prefix is resent ten times. Second, caching attacks exactly that side — it collapses $0.3175 of input to $0.0864, but can do nothing about the $0.1250 of output, which is why the total saving (52%) is smaller than the input saving (73%).

§4 · Prompt caching — the lever, and how it works

Prompt caching lets you mark the stable prefix — system prompt, tool definitions, and the already-seen transcript — so on repeat turns it is served from a cache at a fraction of the input price instead of full price. The mechanics that matter for cost:

• Reads are cheap. A cached prefix token costs about 0.1× the normal input rate on Anthropic and OpenAI's current models. That is the whole game — the huge resent prefix reprices to a tenth.
• Writes cost a small premium, once. New content entering the cache pays roughly 1.25× input. In a conversation, only the fresh (U+A) from the previous turn is written each time — tiny next to the prefix being read.
• Output is never cached — anywhere. Correct across all providers. Generation is always billed at the full output rate.
• Caching has a TTL. Anthropic's default is 5 minutes. Turns within that window hit the cache; an idle gap longer than the TTL forces a fresh (full-price) write on the next turn. Real interactive traffic usually stays warm; overnight agents may not.

Put it together and the payoff scales with the conversation: the longer it runs and the larger the fixed prefix relative to each new message, the more of your cost is repeated prefix — and the more caching claws back. On short exchanges with a tiny system prompt there is little to cache; on a 25-turn agent with a 4,000-token tool block it is the difference between a sane invoice and a shocking one.

§5 · Prompt caching rates by provider (verified July 2026)

Rates are expressed as a multiple of each model's normal input price. "Read" is a cached-prefix token; "write" is a token entering the cache.

Snapshot verified 2026-07-13 against provider pricing/caching docs. Caching rates and the exact placement rules change more often than base prices — confirm against your provider the week you ship, then plug current numbers into the simulator.

§6 · Four things that make it worse (or better)

• Big tool blocks are resent every turn too. Agents often carry 2,000–4,000 tokens of tool schemas in T. That block rides in the prefix on every single turn — a prime caching target, and a brutal cost if left uncached.
• Reasoning/thinking tokens bill as output. On reasoning models, the model's internal thinking is billed at the output rate and cannot be cached. A "cheap input" model with heavy hidden reasoning may not be cheap at all.
• Context editing and compaction cut the history term. Trimming or summarizing old turns shrinks the (t−1)·(U+A) that dominates late-conversation cost — the structural fix when caching alone is not enough.
• Model routing still applies. The blended per-token gap between a budget and a frontier model is large; classify or draft with a cheap model and reserve the frontier tier for turns that need it.

§7 · FAQ

§8 · Related tools