What Does One AI Support Conversation Cost in API Tokens? 2026 Math

If you are pricing a support chatbot built on a model API, the number you need is the price per conversation, not per million tokens. The two are linked by a token budget most estimates skip: how much text goes into the model every time it answers, and how much comes back.

This guide is for founders, developers and support leads weighing a do-it-yourself bot against a hosted tool. It breaks one grounded reply into its parts, measures them with OpenAI's own tokenizer, prices a three-reply conversation on ten current models from OpenAI, Anthropic and Google, and lists what the token bill leaves out. The arithmetic is shown so you can redo it with your own numbers.

PepoChat is our product; its prices, limits and prompt sizes below were checked in its code and documentation on 22 October 2026. Every model price comes from the vendor's own pricing page, opened the same day.

How many tokens does one AI support reply use?

One AI support reply grounded in your knowledge base uses about 3,800 to 4,300 input tokens and 200 output tokens in our measured example, and roughly 1,600 to 9,000 input tokens depending on how the bot is built.

A token is a chunk of text, often a short word or part of a longer one, that a model reads or writes. OpenAI's documented rule of thumb is about 4 characters, or 0.75 words, of English per token. Each reply is billed for everything the model reads to write it, including text it already read on the previous turn.

Token counts measured on 22 October 2026; the ranges are our estimates.

The measured column comes from a production agent: PepoChat's built-in instructions and four standard tool definitions, before a business adds its own, plus passages from its help documentation laid out as its search code sends them, eight passages and 12,000 characters at most. We counted with tiktoken, OpenAI's open-source tokenizer, using the o200k_base encoding it assigns to GPT-4o through GPT-5.6. It does not map the GPT-6 models yet, so treat the counts as close estimates. The help text came out at 0.73 to 0.74 words per token, in line with OpenAI's rule of thumb.

Retrieved passages are the largest part of most replies. Retrieval-augmented generation (RAG) is the pattern in which the bot searches your knowledge base for passages that match the question and gives them to the model to answer from; the RAG explainer walks through each step. Sending twice as many passages, or passages twice as long, moves the bill more than any rewrite of the instructions.

For the arithmetic below we round the measurements: 1,800 tokens of instructions and tool definitions, 2,000 tokens of passages (between the two measured contexts), a 30-token question and a 200-token answer of about 150 words.

How much do OpenAI, Anthropic and Google charge per million tokens?

On 22 October 2026, OpenAI, Anthropic and Google charged $0.10 to $0.30 per million input tokens for their small models and $2 to $10 for their largest. Output costs five times the input price on every model below except Gemini 3.5 Flash-Lite, where it is about eight times.

Prices per million tokens, checked 22 October 2026

US dollars per million tokens, standard tier. Prices from each vendor's pricing page, checked 22 October 2026. Claude Haiku 5.5 prices apply to prompts up to 100,000 tokens.

The rows come from OpenAI's pricing page, Anthropic's pricing documentation and Google's Gemini API pricing page; the tier labels are ours. OpenAI calls gpt-6-astra its flagship, Anthropic suggests Claude Opus 5.5 for most workloads and Fable 5.1 for demanding reasoning, and Google calls Gemini 4 Argon its flagship reasoning model.

Two of Google's prices are scheduled to double: Gemini 4 Argon goes from $2 input and $10 output to $4 and $20 on 1 December 2026, and Gemini 3.8 Flash from $0.75 and $3.75 to $1.50 and $7.50 on 1 January 2027. A cost model built on today's Google rates needs a date on it.

Per-token prices are not directly comparable between vendors either, because each one splits text into tokens its own way. Anthropic says the tokenizer in Claude 4.7 and later models produces about 30% more tokens for the same text than its earlier one, which fitted about 750,000 words in a million tokens, the same ratio as OpenAI's rule of thumb. The cost table below therefore adds 30% to the token counts on Claude rows.

Which LLM is cheapest for a support chatbot?

The cheapest current models for a support chatbot are OpenAI's gpt-6-luna and Anthropic's Claude Haiku 5.5, both listed at $0.10 per million input tokens and $0.50 per million output tokens on 22 October 2026.

Cheaper rows on the same lists belong to models on their way out. OpenAI's gpt-5-nano costs $0.05 input and $0.40 output, but its snapshot leaves the API on 11 December 2026, and gpt-4.1-nano was scheduled to leave on 23 October 2026. Google's Gemini 2.5 Flash-Lite, at $0.10 and $0.40, is limited to developers who already use it. A model you must replace within weeks is not the cheap option.

What does one three-reply support conversation cost?

One three-reply AI support conversation costs about $0.0015 on the cheapest current model and up to $0.20 on the most expensive, using our budget of 12,180 input and 600 output tokens at 22 October 2026 list prices.

Three replies is a busy but common conversation: a question, a follow-up and a final check. The conversation sizing guide starts most sites at two replies per conversation, so treat three as the cautious case. Here is the arithmetic:

1. Reply 1 input: 1,800 tokens of instructions and tools, 2,000 of passages and a 30-token question make 3,830 tokens.
2. Reply 2 input: the same 3,830 plus the first question and answer (230 tokens) make 4,060 tokens.
3. Reply 3 input: 3,830 plus two earlier turns (460 tokens) make 4,290 tokens.
4. Conversation total: 12,180 input tokens and 3 × 200 = 600 output tokens.
5. Price it: on gpt-6-luna, 12,180 × $0.10 ÷ 1,000,000 = $0.00122 for input, plus 600 × $0.50 ÷ 1,000,000 = $0.00030 for output, so $0.0015 per conversation or $1.52 per 1,000.

Cost per conversation at 22 October 2026 prices

Prices from each vendor's pricing page, checked 22 October 2026; token counts are our estimates, with 30% added on Claude rows for Anthropic's newer tokenizer. The caching column assumes the 1,800-token start of every prompt is read from a warm cache. Gemini caching is not modelled because Google's automatic caching starts at 4,096 tokens on Gemini 3.8 Flash and 4 Argon.

Per reply, divide by three: about 0.05 cents on gpt-6-luna, 1 cent on gpt-6.1-sol and 5 cents on gpt-6-astra. The dearest row costs 130 times the cheapest; whether the cheap rows answer your questions well, only your own test questions can show.

Why do real API bills come out higher than the estimate?

Real API bills for support bots often come out higher than a per-reply estimate because of four things a simple budget leaves out: hidden reasoning tokens, extra model calls for tools, old passages kept in the conversation, and oversized messages.

Hidden reasoning tokens are billed as output

Reasoning tokens are text a model writes to itself before it answers; you never see them, but all three vendors bill them. OpenAI's reasoning guide says they "are billed as output tokens", Anthropic says the same of Claude's thinking, and Google labels its output prices "including thinking tokens". Defaults matter: gpt-6-luna reasons at medium effort by default and can be set to none, gpt-6.1-sol and gpt-6-astra cannot go below low, and Claude Haiku 5.5 defaults to medium effort.

OpenAI says a model may generate anywhere from a few hundred to tens of thousands of reasoning tokens, depending on the problem. Every 500 reasoning tokens per reply adds 1,500 output tokens to a three-reply conversation, which raises the gpt-6-luna and gpt-6.1-sol figures above by about half. Read the reasoning count in each response's usage data, and use the lowest effort that still passes your test questions.

A bot that calls a search tool pays for its instructions twice

When the model decides for itself when to search, each reply takes at least two model calls, one that requests the search and one that writes the answer, and both carry the instructions and tool definitions. In our budget that adds roughly 2,000 input tokens per reply, lifting the conversation from 12,180 to 18,435 input tokens and its cost by about 44%.

It gets worse when the framework keeps every tool result in the history, because the first question's passages are then re-sent with the second and third. That turns 18,435 input tokens into 30,585, about 2.2 times the simple estimate's cost. Keep a short summary of each search result in the history instead. Many support agents make two or three model calls for one product question: one to decide to search, one to read the passages and one to write the reply.

Long messages and scripted abuse

A public chat box accepts whatever a visitor pastes. One 100,000-token message costs $0.01 to read on gpt-6-luna, $0.20 on gpt-6.1-sol and $1 on gpt-6-astra, and a script can send thousands of them. Cap message length and rate-limit sessions before launch: PepoChat caps messages at 4,000 characters and, as of 22 October 2026, allows 20 messages a minute per visitor session with a burst of 5.

Do prompt caching, batching and smaller models cut the bill?

Prompt caching cut our example conversation's cost by about a third, batch processing halves prices only for work that can wait up to 24 hours, and a small model costs a twentieth of a mid-tier one at a quality price you have to test.

Prompt caching

Prompt caching is a discount for repeated input: when a request starts with the same text as a recent one, that start is billed at a fraction of the input price. On the models in our table a cache read costs 2.5% to 10% of the input price, and writing to the cache costs 1.25 times the input price on OpenAI's newer models and on Anthropic's five-minute cache.

Only the unchanging start of each prompt qualifies. In a support bot that is the instructions and tool definitions, 1,800 of about 4,000 tokens, because the retrieved passages change with every question. Minimums apply as well: 1,024 tokens on OpenAI's GPT-5.6 and later models, 512 on Anthropic's current models, and 4,096 for Google's automatic caching on Gemini 3.8 Flash and 4 Argon.

Caches also expire, after at least 30 minutes by default on OpenAI's newer models and 5 minutes on Anthropic, refreshed each time they are read. A site with a handful of chats a day pays for cache writes more often than it collects cheap reads. OpenAI's prompt caching guide works through the break-even.

Batch processing

A batch API runs requests asynchronously at half price. OpenAI completes batches within 24 hours, Google targets the same turnaround, and Anthropic offers the same 50% discount. A visitor will not wait a day for an answer, so batching does nothing for live replies; it suits the offline work around a bot, such as nightly topic labels, conversation summaries and evaluation runs.

Smaller models

Moving from a mid-tier to a small model cut our example from $30.36 to $1.52 per 1,000 conversations on OpenAI. Grounded replies often suit small models, since the answer is in the passages and the model mostly reads and rephrases. The risk sits at the edges: multi-step troubleshooting, answers that combine two passages, and deciding when to hand over to a person. Run the same 50 or more real questions through both models before you choose.

What does a DIY support bot cost beyond tokens?

Beyond tokens, a DIY support bot pays for embeddings, a vector store, hosting, evaluation runs and engineering time, and only embeddings are as cheap as the model calls.

OpenAI lists text-embedding-3-small at $0.02 per million tokens and Google lists Gemini Embedding 2 at $0.20. Embedding a 200-page site of about 1,500 tokens a page, 300,000 tokens in all, costs $0.006 or $0.06, and re-embedding it every week costs a few cents a month. Embedding visitors' questions costs even less.

Evaluation runs are where token costs come back. Sending 200 test questions through the bot costs about $0.10 on gpt-6-luna, $1.93 on gpt-6.1-sol and $9.66 on gpt-6-astra per run, before any model grades the answers. Run that after every prompt change and, at low volume, the monthly figure can exceed the bot's own token bill, so put evaluations in a batch at half price.

The vector store, hosting, monitoring and the engineer who keeps ingestion current are priced line by line in our build vs buy breakdown, which estimates two to four developer-days of upkeep a month. Building on the API also makes the vendor's data terms your job; the guide to OpenAI API data policies covers them.

What happens to your costs when a model is deprecated?

When a model is deprecated, your cost per conversation changes because the replacement has different prices, a different tokenizer or different defaults, and the bot has to be re-tested before the shutdown date.

A deprecation is a vendor's notice that a model will be removed from its API on a set date. OpenAI's deprecations page shows how one plays out: on 14 October 2026, OpenAI notified developers that GPT-4o mini (gpt-4o-mini-2024-07-18) will be removed from the API on 14 January 2027, with gpt-6-luna as the recommended replacement. That is three months to migrate and re-test.

On paper the move saves money. GPT-4o mini is listed at $0.15 input and $0.60 output against $0.10 and $0.50 for gpt-6-luna, so our example conversation drops from $0.0022 to $0.0015. But gpt-6-luna reasons at medium effort by default, and on the Chat Completions API it supports function calling only with reasoning effort set to none. At 500 reasoning tokens per reply, the conversation would cost $0.0023, more than before. The new model stays cheaper only if someone sets the effort, checks the tool calls and re-runs the test questions.

The same page lists more shutdowns, through gpt-5.4-nano and gpt-5.1 on 1 April 2027. Anthropic also publishes a retirement commitment for each current model: Claude Haiku 5.5 will not be retired before 7 October 2027. Google lists Gemini 3.1 Flash-Lite for shutdown on 7 May 2027. On a hosted tool the vendor does the migration, and on PepoChat model costs are part of the published price per reply.

When is building on the API cheaper than a flat-priced support tool?

Building on the API is cheaper than a flat-priced support tool on tokens alone at any volume with a small model, but with two to four developer-days of upkeep a month it only catches up at about 110,000 to 220,000 replies a month.

PepoChat's Starter plan costs $49 a month for 2,000 AI replies, about 2.5 cents a reply, and further replies cost $10 per 1,000, or 1 cent each, with model costs included. Growth costs $149 for 10,000 replies, and every feature is on the free plan, which includes 200 replies a month.

PepoChat prices from its billing documentation and token prices from OpenAI's pricing page, checked 22 October 2026. DIY costs use our average of 4,060 input and 200 output tokens per reply. The last column subtracts the gpt-6-luna tokens and $25 a month of hosting from the PepoChat price, then divides by $65 an hour, about the US median developer wage the build vs buy breakdown uses.

On a mid-tier model the token bill alone, about 1 cent a reply, matches PepoChat's overage price, so building never wins on running cost. On a small model the gap is real but small in hours: at 10,000 replies it buys an hour and a half of engineering a month. Two developer-days of upkeep fit inside it from about 110,000 replies and four from about 220,000, before the 60 to 110 days of building.

Per-resolution and per-seat pricing change the comparison. The AI customer support pricing index lists the unit each of 30 tools bills, and the AI chatbot cost guide works those bills at three volumes.

What to do next

• Count your own budget: run your system prompt, tool definitions and five real retrieved passages through a tokenizer, then redo the five steps above with your model's prices.
• Compare the flat plans on the pricing page: Starter is $49 a month for 2,000 AI replies, with model costs included.
• To test answers before you spend engineering time, create a free workspace and ask it your 20 most common questions; every feature is on the free plan, with 200 AI replies a month.