Token
A token is the smallest unit of text a language model processes: a word, part of a word or a character. How many characters fit in a token depends on the model and the language.
In practice (illustrative): a long customer conversation, plus the agent's instructions and catalog excerpts, can reach thousands of tokens.
Watch out: many model providers charge for input and output tokens. Instructions and documents sent along with the messages also count, so the cost of an agent depends on how it is designed, not only on the number of conversations. See how much AI costs a company.
Context window
The amount of text the model can consider at once when generating a reply: its working memory at that moment. It is different from what the model learned in training.
In practice (illustrative): to answer an exchange request, the window needs the relevant parts: the exchange rule, what the customer has already said about the order and the order details.
Watch out: what is not in the window, the model does not see. In long conversations, older parts may drop out. Remembering a customer from one week to the next requires storing information outside the model, with clear rules on what to keep. See the risks of AI memory.
Prompt
A prompt is the text sent to the model: the question, the context and the instructions. In an agent, the instructions on role, tone of voice, rules and limits (the so-called system prompt) are sent to the model on each run, according to how the agent is designed.
In practice (illustrative): "You serve store X. Only share delivery dates that are in the system. Discount requests above 10% go to the manager."
Watch out: written instructions guide the model but do not guarantee it always complies. That is why important rules also live in permissions and systems, and the prompt is tuned against a success criterion and tests with real conversations. See system prompts for AI agents.
Embedding
An embedding represents a piece of text as a list of numbers (a vector). The distance between two vectors suggests how related the texts are: texts with similar meaning tend to sit close together.
In practice (illustrative): "I want to return the product" and "how do I exchange the product" tend to sit close, because they are about the same subject, although the intent is not identical: a return means money back, an exchange means another item.
Watch out: closeness suggests a relationship; it does not guarantee that the text found answers the question. See AI knowledge management.
Vector search
Search that uses embeddings to bring back passages potentially relevant to the question, even without the exact word. It is one of the ways RAG, described above, chooses what to hand to the model.
In practice (illustrative): a customer asks "can I return it after using it?" and the search may bring back the policy paragraph about used products, even though it never says "return".
Watch out: the search may bring back a passage that is only similar, not the right one. An outdated document can contaminate the answer, and duplicate or contradictory bases raise the risk of error. Maintaining the base and checking what was used is part of the job.
Tool calling
Also called function calling. The model decides, based on the request and a tool's description, that it needs an external action and returns a structured request. The software responsible for the tool executes it, which may be the company's application or a service run by the model provider, under the configured permissions; the result goes back to the model.
In practice (illustrative): a customer asks where their order is; the agent calls the "look up order" tool with the order number, the system returns the status and the agent writes the reply.
Watch out: if a required piece of data is missing, the model may try to infer a value. Actions that change something (cancel, charge, update a record) need confirmed data and well-defined permissions. See system integration.
Hallucination
When the model confidently states something wrong or made up. NIST calls it confabulation.
In practice (illustrative): without access to the delivery date, the model answers "it arrives in three days" because it sounds plausible.
Watch out: letting the agent say "I don't know", answering from company documents (RAG, above) and checking information at the source reduce the problem but do not eliminate it. Critical information (price, deadline, contract) must come from a system or document, and quality should be measured by verified resolution. See AI customer service metrics.
Human in the loop
A design in which a person reviews, approves or takes over defined steps of an AI agent's work, with clear roles and responsibilities.
In practice (illustrative): a discount above the limit goes to the manager for approval; a complaint goes to a person with the full history.
Watch out: review that turns into a rubber stamp does not protect anyone. NIST points to risks such as automation bias and over-reliance on the system; whoever approves needs the time, criteria and authority to say no. See human intervention in AI customer service.