How AI Models Understand Language (Tokens, Context & Attention)

AI doesn't read English. It reads numbers. Understanding the simple journey your words take — from text, to tokens, to predictions — is the single biggest mental upgrade you can make as a prompt writer. You don't need any maths. Just five minutes.

1. Introduction

Have you ever wondered why an AI sometimes invents a word, misspells a name, or "forgets" something you mentioned earlier in the chat? The answer lies in how it actually processes language. Once you understand the three core ideas — tokens, the context window, and attention — almost every quirk of AI behaviour starts making sense.

2. The Concept Explained

Tokens — the units AI actually reads

When you type "The quick brown fox", the AI does not see four words. It first breaks the sentence into smaller pieces called tokens. A token is roughly a chunk of letters that often appears together. Common short words are one token. Longer or rarer words get split into multiple tokens. As a rough rule of thumb, 100 tokens ≈ 75 English words.

Each token gets converted into a number, and that number is what the model actually works with. The AI is, in a sense, a very sophisticated number-prediction machine — it looks at the numbers so far and predicts which number comes next, then turns that number back into a token, and that token back into text on your screen.

The context window — the AI's working memory

The AI can only look at a limited number of tokens at once. That limit is called the context window. Modern models support windows from a few thousand tokens to over a million. Everything inside the window is "visible" to the AI right now: your current prompt, the system instructions, plus the recent back-and-forth of your conversation. Anything that falls outside the window is effectively gone — the AI can no longer see it.

Attention — how the AI decides what matters

Inside the context window, not every token is equally important. The AI uses a mechanism called attention to figure out which earlier tokens are most relevant when predicting the next one. If you ask "What did I say about the budget?", the model "pays attention" to the earlier mention of "budget" and uses that as the anchor for its answer. Attention is the reason a well-written prompt with clear anchors (key terms, headings, bullet points) gets better results than a wall of unstructured text.

3. Why This Matters Without It

Most beginners assume AI works like a human reader — that it grasps meaning the way we do. That assumption causes three common problems:

• Forgotten context. You mention something in message 3, but by message 30 the model seems to have forgotten it. Why? It may have fallen outside the context window.
• Garbled rare words. Brand names, less common technical terms, or made-up project names get split across many tokens and the model handles them less reliably.
• Lost instructions. If your instructions are buried inside a giant block of text with no structure, attention spreads thin and the AI misses key constraints.

4. The Better Approach

so I'm working on this new app idea it's
basically a marketplace and I need help
thinking through the pricing structure
also can you check the grammar in my pitch
deck which I'll paste below and also write
me a tagline thanks…

I'm working on a marketplace app.

I need help with three things:

1. Pricing structure — suggest 2 models.
2. Grammar check on the pitch deck below.
3. One tagline (under 8 words).

Pitch deck text:
"""
… paste here …
"""

The second version uses numbered tasks, headings, and triple-quote delimiters. Each anchor tells attention exactly what to focus on, and the model is far more likely to address all three asks instead of just the first.

5. Step-by-Step: Writing for How AI Actually Reads

1. Put the most important instruction at the top or bottom. Tokens near the edges of a prompt often get more attention than tokens in the middle.
2. Use clear delimiters. Triple quotes """, XML tags, or markdown headings separate "instructions" from "content the model should work on".
3. Break long prompts into numbered tasks. Lists are easier for attention to track than paragraphs.
4. Repeat critical constraints at the end. A short reminder like "Remember: respond in JSON only." right before the AI generates helps it stick to the rule.
5. Keep relevant context inside the window. If you are working with a long document, paste only the relevant section, not the whole thing.

6. Practice Exercises

How many tokens are in the sentence 'Prompt engineering is genuinely useful'? Show me how you would split it.

7. Key Takeaways