What Is an LLM? Large Language Models Explained in Plain English
An LLM (large language model) is an AI model trained on vast text to predict the next token. How it works, key terms, per-token costs and the main makers.
Short answer
An LLM (large language model) is an AI model trained on vast amounts of text to predict the next token, a word or word piece, so it can write, summarize, translate and answer questions. Most LLMs use the transformer from the 2017 paper "Attention Is All You Need", and each knows only its training data, up to a cutoff date. PepoChat's AI support agent uses OpenAI LLMs to answer from your knowledge base, free for 100 AI replies a month.
People look up "what is an LLM" for two reasons: the letters turn up on AI product pages everywhere, and they also name a law degree. If you run a business, lead a support team or study AI, you need the first meaning, with enough detail to judge what a tool built on one can do for you.
This guide defines a large language model in plain words, shows how one turns a question into an answer, explains the terms you will meet, and compares LLMs with AI, chatbots and AI agents. Every dated fact was checked on the maker's or researcher's own page on 12 November 2026.
What is an LLM?
An LLM, or large language model, is a computer program that has learned from an enormous amount of text how language continues. Give it a question, an instruction or half a sentence, and it predicts, one small piece at a time, the words most likely to come next. That skill lets it draft emails, summarize documents and answer questions.
| Term | Stands for | In one sentence | Everyday example | Not the same as |
|---|---|---|---|---|
| LLM | Large language model | An AI model trained on vast text to predict the next token | The model behind ChatGPT, Claude or Gemini | The chat app you type into |
| LL.M. | Master of Laws (Latin Legum Magister) | A postgraduate law degree for people who already hold a law degree | A one-year LL.M. in tax law | Anything to do with AI |
| GPT | Generative pre-trained transformer | OpenAI's name for its family of LLMs | The GPT models in OpenAI's API | ChatGPT, the app |
| SLM | Small language model | A compact language model built to run cheaply or on a device | Microsoft's Phi models | A worse LLM in every case |
What does LLM stand for?
LLM stands for large language model. "Large" refers to scale: billions of internal settings, called parameters, learned from a collection of text far bigger than any person could read. "Language model" is the older idea underneath it: a statistical model that estimates which piece of text is likely to come next.
The other LLM meaning belongs to universities. An LL.M. is a Master of Laws, from the Latin Legum Magister; Georgetown Law describes it as "a degree in law for individuals who have already received a first degree in law", taken full-time over two semesters. The rest of this page is about the AI meaning.
Where did large language models come from?
The design most LLMs use is the transformer, a type of neural network introduced in Attention Is All You Need, a paper posted on 12 June 2017 by eight researchers, most of them at Google Brain and Google Research. It proposed an architecture "based solely on attention mechanisms" and tested it on machine translation, not on chat.
Scale came next. OpenAI's 2020 paper "Language Models are Few-Shot Learners" described GPT-3 as "an autoregressive language model with 175 billion parameters" and showed it handling new tasks from a few examples written into the prompt, without retraining. OpenAI's own documentation still explains its product name the same way: its text models are "often referred to as generative pre-trained transformers or 'GPT' models for short".
How does an LLM work?
An LLM works by splitting your text into tokens, weighing how the tokens relate to each other, and predicting the most likely next token, over and over, until the answer is complete. Follow one example through: a shopper types "Can I return shoes after 30 days?"
- Your text becomes tokens. A token is a chunk of text, often a short word or part of a longer one. OpenAI's rule of thumb is about 4 characters, or 0.75 words, of English per token; its documentation shows "tokenization" split into "token" and "ization". The shopper's 33-character question is about eight or nine tokens.
- Tokens become numbers. Each token is mapped to a list of numbers that places it on a map of meaning, where "return" sits nearer "refund" than "tennis".
- Attention connects the words. Attention is the step in which the model weighs how much each token should influence every other one; the 2017 paper describes self-attention as relating "different positions of a single sequence". Here it ties "after 30 days" to "return", so the model reads a question about a returns window.
- The model predicts the next token. It scores every token in its vocabulary, picks one, perhaps "Most", adds it to the text and runs again. A setting called temperature decides how adventurous each pick is.
- It stops. The loop ends at an end marker or a length limit. The reply might be "Most stores accept returns within 30 days of delivery."
Notice what the model never did: it never checked this shop's returns policy. Unless the policy was in its training data or the prompt, the reply is a plausible guess. That is why business tools hand the model your own documents first; PepoChat, for example, searches your knowledge base and puts the matching passages into the prompt for every reply.
What is the difference between training and inference?
Training is the slow, expensive process of adjusting a model's parameters on a huge text collection, and inference is every later use of the finished model to produce an answer. Pretraining teaches next-word prediction; Anthropic's glossary of LLM terms notes that a pretrained model is "not inherently good at answering questions or following instructions". Further training on good answers and RLHF (reinforcement learning from human feedback, in which people rank sample replies) turn it into a useful assistant; Anthropic says Claude was refined this way.
When you chat with an LLM you are only running inference: nothing you type changes its parameters. A provider may train a later version on collected data under its own terms. OpenAI's API documentation says it does not train its models on API inputs and outputs; our guide to where a chatbot's data goes has the details.
What are tokens, context windows and parameters in an LLM?
Tokens, context windows and parameters are the three measures that describe how much an LLM can take in, how big it is and what it costs to run. A few more terms complete the vocabulary of any product page or contract.
Context window
A context window is the maximum amount of text, counted in tokens, that a model can consider in one request, including its own reply; Anthropic's glossary calls it the model's "working memory". On 12 November 2026, Anthropic's current Claude models listed 1 million tokens, Google's documentation gave Gemini 4 models 1,048,576, and OpenAI's flagship GPT models listed 1.05 million.
Bigger is not automatically better. Anthropic's context-window documentation warns that as the token count grows, "accuracy and recall degrade", which it calls context rot.
Parameters
Parameters are the internal numbers, also called weights, that a model adjusts during training and uses for every prediction. The count is a rough measure of size. GPT-3 had 175 billion in 2020; OpenAI's open-weight gpt-oss-120b has 117 billion, of which 5.1 billion are active, meaning only that part of the network works on any one token. On 12 November 2026, OpenAI's and Anthropic's pages for their hosted models listed context windows and prices, not parameter counts.
Temperature
Temperature is a setting that controls how random the model's word choices are. OpenAI's API reference, checked 12 November 2026, accepts 0 to 2, with a default of 1, and says lower values such as 0.2 make output "more focused and deterministic". Anthropic adds that even at 0, results are not fully deterministic. Low settings suit factual answers; higher ones suit brainstorming.
Prompt and system prompt
A prompt is the text you send the model: a question, instructions, examples or pasted documents. OpenAI puts it plainly: "Designing a prompt is essentially how you 'program' a model".
A system prompt is the set of standing instructions an application sends ahead of the user's message to set the model's role, tone and rules; OpenAI calls these developer messages and ranks them above user messages. In PepoChat, what you write under "How to sound and behave" and "Never do this" is added to the agent's system prompt after its base rules.
Fine-tuning
Fine-tuning is further training of an already trained model on your own examples, so it adopts a format, a style or a narrow task. It is less available than many people assume. On 12 November 2026, OpenAI's model optimization guide said "OpenAI is winding down the fine-tuning platform" and that it is closed to new users, and Anthropic's glossary said the Claude API "does not currently offer fine-tuning".
| Term | What it means | A real number | Why it matters to you |
|---|---|---|---|
| Token | A word or piece of a word | About 4 characters of English (OpenAI) | You pay, and hit limits, per token |
| Context window | Most tokens one request can hold | 1M on current Claude models | Caps how much you can paste in |
| Parameters | Weights learned in training | 117 billion in gpt-oss-120b | A rough guide to size and running cost |
| Temperature | Randomness of word choice | 0 to 2 on OpenAI's API | Keep it low for factual answers |
| Training cutoff | Last date of training data | June 2026 on current Claude models | The model knows nothing newer |
Figures from OpenAI's and Anthropic's documentation, checked 12 November 2026.
What can LLMs do, and what can't they do?
LLMs are strong at working with text you give them and unreliable at facts you don't. They draft and rewrite, summarize long documents, translate, classify and route messages, pull fields out of messy text, write code, and answer questions from passages placed in the prompt.
No built-in knowledge of your business
An LLM has never seen your price list, your returns policy or yesterday's order, so a question about them gets a confident guess unless the application supplies the facts.
A training cutoff
Every model's knowledge stops at a date. On 12 November 2026, OpenAI's models page listed a knowledge cutoff of 18 May 2026 for gpt-6-luna, and Anthropic listed June 2026 for its current Claude models; OpenAI's open-weight gpt-oss-120b stops at 1 June 2024. Ask about anything later and the model either says it does not know or guesses.
Hallucination
A hallucination is a fluent, confident answer that is false or made up. A September 2025 paper, Why Language Models Hallucinate, argues that training and evaluation reward guessing over admitting uncertainty, so models become "optimized to be good test-takers". Giving the model the right passages and permission to say "I don't know" reduces it; our guide to stopping chatbot hallucinations lists the fixes in order.
No memory between conversations
An LLM remembers nothing between requests. OpenAI's documentation says "each text generation request is independent and stateless", so a chat app looks continuous only because it resends the earlier messages every time. Products that "remember you" store notes and feed them back into the prompt.
Uneven skills
LLM ability is lumpy. Stanford HAI's 2026 AI Index reports that a Gemini model earned a gold medal at the International Mathematical Olympiad, yet the top model read analog clocks correctly just 50.1% of the time. Test a model on your own tasks rather than trusting a headline score.
What is the difference between an LLM, AI, machine learning, generative AI, a chatbot and an AI agent?
AI is the whole field, machine learning is the part where systems learn from data, generative AI is machine learning that creates new content, an LLM is a generative model for text, a chatbot is a program you converse with, and an AI agent is software that uses a model to take actions.
| Term | What it is | How it relates to an LLM | Use this word when |
|---|---|---|---|
| Artificial intelligence (AI) | Computers doing tasks that need judgment | The broad field LLMs belong to | You mean the field as a whole |
| Machine learning (ML) | Systems that learn patterns from data | LLMs are one kind of ML model | The system learns from examples, not rules |
| Generative AI | Models that create text, images, audio or video | LLMs are the text branch | The output is new content |
| Large language model | A model trained on text to predict the next token | The term itself | You mean the model, not the product |
| Chatbot | A program that holds a conversation | Modern ones often run on an LLM; older ones follow scripts | You mean the interface people talk to |
| AI agent | Software that uses an LLM to decide and act with tools | Uses an LLM to reason and write | It checks orders, books meetings or opens tickets |
Most confusion sits in the last three rows. ChatGPT is a chatbot app and OpenAI's GPT models are the LLMs inside it; our post on using ChatGPT for customer service explains why the app cannot sit on your website. An agent adds tools and rules on top of the model, as what an AI support agent is spells out.
Who makes the main LLMs?
The best-known LLM families are OpenAI's GPT, Anthropic's Claude, Google's Gemini, Meta's Llama and Mistral AI's models, and several of those makers also publish open-weight models you can download. Version names change every few months, so the table names families rather than versions.
| Maker | Family | How you use it | Open-weight models |
|---|---|---|---|
| OpenAI | GPT | OpenAI's API and ChatGPT | gpt-oss, Apache 2.0 licence |
| Anthropic | Claude (Haiku, Sonnet, Opus and others) | Claude API and major cloud platforms | None listed |
| Gemini | Gemini API | Gemma, which Google calls its "open models" | |
| Meta | Llama, plus newer Muse models | Download Llama; Muse through Meta's model API | Llama under Meta's own licence; Muse Spark Open |
| Mistral AI | Mistral and Ministral | Mistral's API or download | Several under Apache 2.0 |
| Microsoft | Phi | Download or Azure | Phi, described as an open model family |
Each family checked on its maker's own site, 12 November 2026.
Most frontier models come from companies, not universities: the 2026 AI Index reports that industry produced over 90% of notable frontier models in 2025, and that U.S. and Chinese models have traded the lead since early 2025, when DeepSeek-R1 briefly matched the top U.S. model.
What is the difference between open-weight and hosted LLMs?
An open-weight LLM is one whose trained parameters you can download and run on your own hardware, while a hosted LLM runs only on its maker's servers and you reach it through a paid API or app. Open-weight is not open source: you usually get the weights without the training data, under licences that differ. Mistral's documentation lists some models under Apache 2.0 and Mistral Large 4 under its own "Mistral Ecosystem Open Weight License".
Hardware matters too. OpenAI says gpt-oss-120b "fits into a single H100 GPU", a data-centre chip, while Google says Gemma runs "from cloud servers to laptops and even phones".
| Question | Hosted LLM | Open-weight LLM |
|---|---|---|
| Where does it run? | The maker's servers | Your servers or devices |
| How do you pay? | Per token used | Hardware, power and staff time |
| Who updates it? | The maker, on its schedule | You, when you choose |
| Where does your data go? | To the maker, under its API terms | Nowhere outside your setup |
| Use this when | You want the strongest models with no infrastructure | Data must stay in-house or volume is very high |
How do businesses use an LLM without training one?
Most businesses use an LLM in one of three ways, and none of them involves training a model from scratch: better prompts, retrieval-augmented generation, or, less often, fine-tuning an existing model.
Retrieval-augmented generation (RAG) is the method in which software searches your own documents for passages that match a question and gives them to the model to answer from. It suits support, because when a price changes you edit one page instead of retraining anything.
| Method | What you do | Cost and effort | When your facts change | Use this when |
|---|---|---|---|---|
| Prompting | Write instructions and examples | Lowest: hours | Edit the prompt | You need a tone, a format or a few rules |
| RAG | Index your documents; the system retrieves passages per question | Low with a hosted tool, weeks if you build it | Edit or re-index the page | Answers depend on your products, prices or policies |
| Fine-tuning | Train an existing model on many example pairs | Medium to high, and limited availability | Retrain | You need a fixed format or narrow task at volume |
| Training from scratch | Pretrain your own model | Very high | Retrain | Almost never, for a business |
PepoChat is a RAG tool of this kind: you add knowledge sources and the agent searches them before every reply.
How much does it cost to use an LLM?
Hosted LLMs are billed per token, with one price per million input tokens (what you send) and another per million output tokens (what the model writes), and output usually costs several times more. On 12 November 2026, OpenAI's pricing page listed gpt-6-luna at $0.10 per million input tokens and $0.50 per million output tokens, and gpt-6.1-sol at $2.00 and $10.00.
Say a support bot you build yourself sends 4,000 tokens per reply (instructions, retrieved passages and the chat so far) and gets 200 tokens back, roughly the budget in our token cost breakdown:
gpt-6-luna: 4,000 x $0.10 / 1,000,000 = $0.0004 input
200 x $0.50 / 1,000,000 = $0.0001 output
total $0.0005 a reply, $0.50 per 1,000 replies
gpt-6.1-sol: 4,000 x $2.00 / 1,000,000 = $0.008 input
200 x $10.00 / 1,000,000 = $0.002 output
total $0.010 a reply, $10 per 1,000 replies
Prices from OpenAI's pricing page, checked 12 November 2026.
Two things push real bills higher. Reasoning models write hidden working text first, and OpenAI's reasoning guide says those tokens "are billed as output tokens", from "a few hundred to tens of thousands" per request. And in a do-it-yourself build, hosting, search and engineering time usually cost more than the tokens. Hosted support tools charge per reply or conversation instead; PepoChat counts one AI reply as one unit, however many tokens it took.
What is a small language model?
A small language model (SLM) is a language model with far fewer parameters than a frontier LLM, built to run cheaply and fast on one server, a laptop or a phone. Microsoft says "Phi is the Microsoft family of small language models (SLMs)"; Google's Gemma family includes a 270-million-parameter model; Mistral's Ministral models come in 3, 8 and 14 billion parameter sizes under Apache 2.0.
There is no official line between small and large. Small models trade general knowledge and multi-step reasoning for speed, low cost and the option of keeping data on the device. Hosted makers have small tiers too: Anthropic pitches Haiku at "high-volume, latency-sensitive tasks such as classification, extraction, and routing". For support answers, where the facts arrive in the prompt, a small model is often enough; test it on your own questions.
What are common misconceptions about LLMs?
The most common misconceptions about LLMs come from treating the model as a database or a person.
"It looks the answer up." A plain LLM writes from patterns; only the application around it can search or read your documents.
"It learns from my chats as we go." Parameters stay fixed during use; earlier messages are simply sent again with each new one.
"A bigger context window means it reads everything carefully." Long inputs fit, but recall drops as they grow, so ten relevant passages beat a whole manual.
"LLM means ChatGPT." ChatGPT is one app on one maker's models; Claude, Gemini, Llama and Mistral's models are LLMs too.
"Open-weight means open source." You get the weights under a licence, usually without the training data.
How PepoChat uses LLMs
PepoChat is an AI support agent for websites, and an LLM is the part that writes each reply. It runs on OpenAI models under OpenAI's standard API terms. There is no model picker: PepoChat chooses the model, so versions and retirements are its job, not yours.
You do not train or fine-tune anything either. You add knowledge sources: a website crawl of up to 200 pages, a sitemap, PDFs and Word files, Q&A pairs, or pages synced from Notion and Google Drive. For every visitor message, the agent searches that private knowledge base, puts the best passages in front of the model with your instructions, and answers from them, treating retrieved web content as untrusted data. When nothing relevant turns up, or the visitor asks for a person, it says so and offers to bring in your team instead of guessing, and the conversation is marked escalated in the team inbox.
Billing never mentions tokens. One AI reply is one unit on every plan: 100 a month on Free, 1,000 on Starter at $39 a month, then $39 per 1,000 extra on paid plans.
The limits are real. You cannot pick Claude or Gemini, bring an open-weight model or fine-tune, and PepoChat publishes no compliance certifications, so keep health and payment details out of chat. If you need those, building on an API may suit you better. To watch an LLM answer from your own content, create a free PepoChat workspace; every chat feature is on the free plan.
What to do next
- See how an LLM answers from your own documents in RAG for customer support, explained.
- Read how PepoChat's agent answers, from retrieval to handoff, in the docs.
- Compare plans on the pricing page: Starter is $39 a month for 1,000 AI replies, Growth $149 for 5,000.
- Start a free workspace, add your help pages and ask your 20 most common customer questions in the free Test the agent drawer.
- Related explainers: What is RAG?, What is an AI hallucination? and What is an AI agent?.
Frequently asked questions
- What is LLM in AI, in simple terms?
- In AI, LLM stands for large language model: a program trained on a huge amount of text to predict the next word or word piece. Repeating that prediction lets it answer questions, write drafts, summarize and translate. It does not look facts up or understand the world the way a person does; it produces the most likely text given what it learned and what you give it.
- Is ChatGPT an LLM?
- Not exactly. ChatGPT is OpenAI's chat app, and the LLMs inside it are OpenAI's GPT models. The app adds features around the model, such as saved conversations, file uploads and account settings, while the model itself only turns text into more text. Businesses that build their own tools call the same kind of model through OpenAI's API instead of using the app.
- What are LLMs used for in business?
- Businesses use LLMs to answer customer questions, draft emails and replies, summarize calls and documents, translate, tag and route support messages, pull fields out of forms and write code. The strongest uses give the model your own text to work from. PepoChat, for example, uses OpenAI LLMs to answer website visitors from a business's own knowledge base and offers a person when it cannot.
- How many parameters does a large language model have?
- There is no fixed number. GPT-3, described by OpenAI in 2020, had 175 billion parameters; OpenAI's open-weight gpt-oss-120b has 117 billion, and small models such as Google's Gemma 3 270M have a few hundred million. OpenAI's and Anthropic's pages for hosted models, checked 12 November 2026, list context windows and prices rather than parameter counts, so compare models on your own tasks instead.
- Can an LLM learn from my customers' conversations?
- Not on its own. A deployed LLM's parameters stay fixed while it answers, and it only sees the conversation sent with each request. A provider can train a later version on data only under its own terms, and OpenAI's API documentation says it does not train on API inputs and outputs. PepoChat retrains nothing on your chats: to change an answer, you edit or add a knowledge source.
- Which LLM is best for customer support?
- No single LLM is best for every support team. Prices, speed and quality change every few months, and a small, cheap model often answers well when the right help-centre passages are in its prompt. What matters more is retrieval from your own documents, a clear handoff to a person, and testing with 30 to 50 of your real customer questions before you choose.
Try this on your own site in ten minutes
PepoChat includes every chat feature on the free plan — 100 AI replies and 5 knowledge sources a month, no credit card.
Keep reading
Chatbase vs PepoChat: Which AI Support Agent Fits Your Team in 2026?
An honest side-by-side of Chatbase and PepoChat on pricing, free plan or trial, sources, handoff, booking, actions, channels and voice, and a pick by team type.
Intercom Fin Pricing Explained, and What a Flat-Plan Alternative Costs
What Intercom Fin's $0.99 per outcome adds up to once you count resolutions and seats, with a worked 2,000-conversation example and when a flat plan wins.