What is an LLM?

A Large Language Model (LLM) is an AI system trained on massive amounts of text data to understand and generate human language. It can write code, answer questions, summarize documents, translate languages, and hold conversations — all by predicting what text should come next given a prompt.

Think of it as autocomplete taken to an extreme: trained on hundreds of billions of words, it learns not just grammar but facts, reasoning patterns, and even how to follow instructions.

---

The Core Idea: Predicting the Next Token

LLMs are trained on a deceptively simple task — given a sequence of words, predict the next word (technically, the next token).

Input:  "The capital of France is"
Output: "Paris"

Do this billions of times across trillions of words, and the model is forced to learn facts, grammar, logic, and context — not because anyone programmed those things in, but because they're necessary to predict text accurately.

---

Why "Large"?

The word "large" refers to two things:

More parameters + more data = better generalization. The model can handle tasks it was never explicitly trained on, which is called emergent capability.

---

How Does It Actually Work?

At a high level, an LLM is a transformer neural network — a stack of mathematical operations that:

1. Breaks your input into tokens — chunks of roughly 3-4 characters each
2. Converts tokens to vectors — numbers that represent meaning
3. Runs attention layers — each token "looks at" all other tokens to understand context
4. Outputs a probability distribution — over every possible next token
5. Samples from that distribution — to produce the response

You type:    "Explain gravity in simple terms"
     ↓
Tokenizer:   ["Explain", " gravity", " in", " simple", " terms"]
     ↓
Transformer: 96 layers of attention + feedforward math
     ↓
Output:      Probability over ~100k tokens → picks "Gravity"
     ↓
Repeats:     "Gravity", " is", " the", " force", " that"...

This happens token by token until the model decides to stop.

---

LLMs vs. Traditional Software

Traditional software follows explicit rules you write. LLMs learn patterns from data.

LLMs are not replacing traditional software — they're best used alongside it.

---

What Can LLMs Do?

LLMs are remarkably general-purpose. Core capabilities include:

Text tasks

• Summarization, translation, classification, extraction
• Writing: emails, reports, code, creative content

Reasoning tasks

• Multi-step math and logic (with limitations)
• Question answering over provided documents

Code tasks

• Write, debug, explain, and refactor code across dozens of languages

Conversation

• Follow instructions across a multi-turn dialogue
• Adapt tone and style to context

With external tools (via function calling / agents)

• Search the web, query databases, run code, call APIs

---

Key Concepts You'll Encounter

Context Window

The maximum amount of text an LLM can "see" at once — both your prompt and its own response count toward this limit. GPT-4o supports 128k tokens; Gemini 1.5 Pro supports 1M tokens.

Tokens

LLMs don't read words — they read tokens. A token is roughly 3-4 characters in English. "ChatGPT" = 2 tokens. "tokenization" = 3 tokens. This matters because API pricing is per token and context windows are measured in tokens.

Temperature

A setting (0 to 1+) that controls randomness in the output. Low temperature (0.1) = precise and deterministic. High temperature (0.9) = creative and varied.

Hallucination

LLMs sometimes generate confident-sounding text that is factually wrong. This happens because the model is optimizing for plausible text, not verified facts. Always validate LLM output for high-stakes tasks.

System Prompt

An instruction provided before the user's message that shapes the model's behavior — its persona, rules, and task focus.

Fine-tuning

Training an existing LLM further on a smaller, domain-specific dataset to make it better at a specific task or style.

---

Popular LLMs Today

The landscape moves fast. Model rankings shift every few months — benchmark on your specific task.

---

LLMs in Production: The Reality

Using LLMs in a product is different from using them in a demo:

• Latency: Generating 500 tokens takes 2–10 seconds depending on model size
• Cost: At scale, API costs add up quickly — caching and prompt optimization matter
• Reliability: Models can refuse, hallucinate, or behave inconsistently
• Privacy: Sending data to external APIs has compliance implications
• Evaluation: "Does it work?" is hard to define and measure

These are exactly the engineering challenges this guide covers — from prompt design to production monitoring.

---

The Mental Model

An LLM is not a search engine (it doesn't retrieve facts), not a database (it doesn't store structured data), and not a calculator (it approximates math). It's a very sophisticated text completion engine that has internalized patterns from human knowledge — and it's most powerful when you design systems that account for both its capabilities and its limits.

---

What's Next

Now that you understand what an LLM is, the following lessons explore the internals:

• Transformer Intuition — the architecture behind every major LLM
• Tokenization — how text becomes numbers
• Attention Mechanisms — why context understanding is possible
• Context Windows — the working memory of a language model