AI Tokens: What They Are, Costs and Application Limits

AI tokens are the units in which a language model processes text and accounts for its input and output. A token can correspond to a word, part of a word or a combination of characters. For a business, this unit connects the apparently simple question in an interface to the cost, capacity and behaviour of the application behind it.

A short conversation can send the model instructions, earlier messages and documents that the user does not see in the writing field. Budgets therefore depend on the complete request and the entire process, not the length of the question. This guide explains the mechanism, calculations and useful controls for a product used every day.

For an explanation of the model using these units, start with what an LLM is and how it works. Here, we focus on tokenisation, API usage, limits and the design decisions that make consumption measurable.

What are AI tokens, and what do they not represent?

Processing units, not pages or commercial credits

The model does not receive text as a page viewed by a person. A tokenizer converts it into a sequence of numerical identifiers. Associated segments can include spaces, punctuation, word fragments and characters. The tokenizer's vocabulary and rules determine the segmentation, and the model processes the resulting representations.

A token used by an LLM is different from an authentication token or a crypto asset. A platform's commercial “credit” is not automatically a token either: the product provider can package usage according to its own catalogue. When selecting a solution, we explicitly identify the billed unit and the operations it covers.

Token count does not measure the value of an answer

A long response consumes generated text, but length does not establish correctness. For a stock check, a short output containing the product, quantity and source can be more useful than an extensive explanation. We define the result the process requires before deciding how much text the application should be allowed to produce.

How are words converted into tokens?

Language, diacritics and formatting matter

A common word can have a compact representation, while a rare one can be split into several fragments. Technical abbreviations, product codes, addresses and imperfectly extracted document text change segmentation. The same meaning expressed in Romanian and English can require sequences of different lengths.

A useful comparison uses actual project text: “Verifică disponibilitatea produsului” and its English equivalent, “Check product availability”, for example. We measure them with the tokenizer associated with the selected model, preserving the exact spaces, diacritics and punctuation. We do not assign these sentences a universal token count or remove diacritics for an assumed saving.

Local estimates and complete-request counts

A local tokenizer is useful for exploring plain text. To size an integration, we also use the provider's counting mechanism where available: message structure, tool definitions and multimodal content can add elements missing from a text string analysed on its own. We record the model and configuration alongside the measurement.

Input, output and reasoning tokens: what is counted?

Input is larger than the user's question

An application request can contain system instructions, selected history, document passages, tool definitions and tool results. A “Summarise the contract” button can trigger the submission of substantial material. An inventory of these components is part of designing custom applications with AI integration, rather than simply choosing a model.

Reported output is more than the displayed text

In OpenAI's Responses API, usage is reported through fields such as input_tokens and output_tokens. For reasoning models, intermediate reasoning tokens are included in the accounted output; invisible formatting tokens can also occur. The length of an answer copied from the interface does not automatically reproduce the total reported by the API.

Usage details are subcategories, not extra consumption to add indiscriminately. If part of the input is read from cache, we do not add it again on top of total input. Similarly, reasoning tokens reported in output details are not added to total output a second time. We reconcile the fields according to the provider's schema.

How do tokens affect the cost of an AI application?

The formula starts with separate categories and rates

For a volume-priced service, text cost is calculated by multiplying each category's token count by its rate and dividing by the catalogue's billing unit. If the price is per million tokens, the divisor is one million. Ordinary input, output and cache categories can have different rates.

A commercial estimate records the exact model, currency, processing mode and pricing date. An integration can have separate costs for search, storage, embeddings, tools or infrastructure. A chat-interface subscription and API usage are handled according to the terms of those products, rather than treated as interchangeable resources.

The company's budget also includes development, support and operation. The guide to AI application costs and maintenance explains those components. This article isolates token usage so it can be calculated and checked without confusing it with the price of the entire project.

Worked example: usage by an internal assistant

This example is for teaching purposes only. We use hypothetical rates of 2 currency units per million input tokens and 8 per million output tokens. These are not a WebHat quotation or prices attributed to a provider. We exclude caching and all other service categories, and output represents the total billable usage in that category.

Assume 1000 requests of identical size, each with 4000 input tokens and 1000 output tokens. This produces 4 000 000 input tokens and 1 000 000 output tokens. Input costs 8 units, output costs another 8, and the batch total is 16. Average cost per request is obtained by dividing the total by the number of requests.

If the process requires regeneration or another model call after using a tool, we add that stage's usage. An action in the interface is not necessarily a single API call. For planning, we distinguish user actions from the actual number of calls that make up each action.

For a monthly budget, the uniform batch is only a starting point. In a support assistant, questions about opening hours can be short, while analysing a situation involving documents requires a different volume. We estimate each task type separately, multiply it by expected frequency and retain a reserve for variation. After the pilot, we replace assumptions with the measured distribution and check the difference between forecast and consumption.

Four separate limits: context, output, throughput and budget

Request capacity and room for the response

The context window determines request capacity, while the output limit controls permitted generation. In a hypothetical example with a capacity of 16 000 tokens, allocating 10 000 to input and reserving 4000 for generation leaves a margin of 2000. These values illustrate planning; they do not describe any particular commercial model.

For reasoning models, the reservation also includes intermediate work under the model's rules. Fitting the document into input does not mean enough room remains for the required output. The application checks this condition before submission and explicitly handles a response stopped at its generation limit, rather than saving it as a completed result.

Request throughput and spending are separate controls

Requests-per-minute and tokens-per-minute limits describe processing capacity over an interval. An account can have budget available yet reach its throughput ceiling. Work queues, controlled concurrency and bounded retries manage load; the financial ceiling determines the consumption the business accepts. A budget alert should not be confused with a verified technical stop.

Why can long conversations consume more tokens?

History retained for the user and context sent to the model are different things. If an application resubmits the entire conversation, earlier messages reappear in the input of later requests. Server-side conversation storage or a conversation identifier does not turn that context into a free resource.

A more disciplined design retains the current task, relevant decisions and the information needed for the answer. For large archives, an AI chatbot connected to company documents through RAG can select relevant passages instead of submitting the entire library. Access permissions and document versions are still checked separately.

Summarising history has its own consumption and can lose details. We specifically test product names, accepted conditions and values entered by the user. Savings are useful when the process remains correct; if a summary omits a contractual exception and causes rework, the initial token reduction does not describe the final cost of the task.

What does prompt caching save, and what must be checked?

Prompt caching reuses computation for a repeated beginning of a request. Stable instructions and shared reference material can benefit when the model's conditions are met. Prefix reuse differs from saving a complete response in an application cache: the model can still generate a new answer.

Caching does not shrink the text in context. It changes how part of the input is processed and priced. Retention duration, eligibility thresholds and any write rates are checked for the model in use. In an integration with billed writes, we divide input into mutually exclusive categories before calculating cost, without charging the same portion twice.

The decision is based on reuse observed in the application. A document submitted once and common instructions used repeatedly have different profiles. We compare total cost and response time before and after configuration, using the same set of tasks and the same quality requirements.

Documents, images and audio: why is file size insufficient?

A file that is small in megabytes is not automatically a small model input. Processing depends on the format and endpoint. In OpenAI Responses, a PDF used with a vision-capable model can introduce both extracted text and page images. A plain-text tokenizer alone does not measure that complete request.

For AI document processing and data extraction, we decide what information must be read: text, tables, layout or images. A document where signature position matters has different requirements from a text list. Reducing resolution or extracting only text is validated against the required result.

Audio and images can have their own billing units and rules. We do not apply text-token rates to every modality. The technical estimate separates recognition, analysis, generation and storage so that estimated volume can be reconciled against actual usage after launch.

How do we reduce usage without degrading the application?

We start with clear instructions, without contradictory repetition, and information selected for the current task. We define an appropriate response format: a label and short reason for classification, or the structure needed for review in an analysis. Length follows the usefulness of the result, rather than arbitrary removal of important explanations.

We set a call ceiling for each automated process and prevent the same operation from being duplicated on retry. Requests exceeding allocated resources follow a clear path: better-selected documents, batch processing or approval for extended analysis. The user sees the operation's state and which result has been completed.

Context reduction respects data protection in an AI-integrated application. We retain authorisation instructions, provenance and conditions necessary for the process. After each optimisation, we rerun tests on representative tasks: lower cost is useful only when the application continues to deliver the defined result.

What do we measure before and after launch?

During the pilot, we select short and long requests, clean and difficult documents, new conversations and extended conversations. For each category, we record the model, instruction version, input, output, cache details, duration, cost and review result. Technical measurements can be retained without copying all sensitive content into logs.

In business dashboards and automated reports, the central indicator is the cost of a correctly completed task. Alongside it, we track average usage, large cases, retries, incomplete responses and usage by team or workflow. A healthy average can hide a few very expensive operations that need a dedicated control.

For an application development project, we document during analysis what enters a request, who can initiate the process and how ceilings are enforced. Tokens then become a managed resource: we size, measure, check and adjust based on usage, preserving the connection between consumption and operational value.

During functional acceptance, we also test the user experience at these boundaries: what message appears when a document is too large, how work already entered is preserved and who can approve continuation. An incomplete result must be clearly marked, and a retry linked to the original operation. These details distinguish a usage counter from an application that a team can use predictably.

Frequently asked questions about AI tokens

How many tokens are in a Romanian word?

There is no fixed correspondence. A tokenizer can represent a word as a single item or split it into fragments. A useful estimate measures representative Romanian text with the project's model and configuration, preserving diacritics and the actual formatting.

Does a short answer automatically mean low usage?

No. Input can include a substantial document and a long conversation, while certain models use intermediate reasoning tokens. Visible consumption is considered alongside the components reported by the API under its schema, without double-counting subcategories already included in totals.

Do cached tokens still occupy the context window?

Yes. Reusing computation does not remove information from the request. Context capacity must still be checked, while the cost benefit is calculated using the read or write rates and conditions applicable to the model being used.

Does a large context limit allow an equally large answer?

Total capacity and the output ceiling are separate limits. Before submission, the application checks input and the generation reservation, then applies the model's output limit. Room for reasoning is included where the model uses it.

Does available budget eliminate API limit errors?

No. Request throughput and token volume per interval can have separate ceilings. A properly sized application controls concurrency, uses queues and handles errors according to their reported cause, separately from checking accepted expenditure.

How do we determine the actual cost of a business AI assistant?

Through a pilot using representative tasks and reporting all calls within each process. Measured API usage is matched to the configuration's current rates, then auxiliary services, infrastructure and operation are added. The useful indicator remains the cost of a completed and checked task, not the isolated price of a token.