How Context Windows Work and Why They Matter

The context window is the AI's working memory. It is the single most important constraint to understand once you start working on real projects — long documents, long conversations, large amounts of data.

1. Introduction

You may have noticed that AI sometimes "forgets" what you said earlier, or refuses to read a giant document. That is the context window at work. This tutorial shows what it is, how big it gets in modern models, and the practical habits that let you make the most of it.

2. The Concept Explained

Every AI model has a fixed limit on how many tokens it can pay attention to at once. That limit — measured in tokens — is the context window. Everything you send (the system prompt, the conversation history, your latest message, plus any attached documents) shares that same window. The reply the AI generates also counts towards it.

When the conversation gets too long, the oldest messages start falling out of the window. The AI does not "remember" them anymore. This is why long chats can feel forgetful.

Common context window sizes (rough)

• Small (4K – 16K tokens): Older or budget models. Roughly 3K – 12K words. Fine for chats, short documents.
• Medium (32K – 128K tokens): Most modern chat models. Can handle a long article, a chapter of a book, or thousands of lines of code.
• Large (200K – 1M+ tokens): Long-context models. Can ingest entire books, whole codebases, or huge transcripts in one shot.

The exact numbers move every few months. The principle does not.

3. What Goes Wrong Without Awareness

• Long chats start "forgetting" early instructions or facts.
• Pasting a huge document leaves no room for the reply — output gets cut off.
• Important context gets buried in the middle of the window where attention is weakest.
• Costs balloon: in APIs, you pay per token in and out, so unnecessarily long contexts cost real money.

4. Practical Habits for Working With Context Windows

1 — Paste only what is relevant

If your document is 50 pages but only chapter 3 matters, paste only chapter 3. Don't dump everything and hope the AI sorts it out.

2 — Summarise older history when chats get long

Ask the AI itself:

Summarise everything important from our chat so far into a single message I can reuse as context.

Then start a fresh chat with that summary at the top.

3 — Put critical info at the start AND the end

Attention is strongest at the edges. Repeat your most important constraint near the bottom of a long prompt.

4 — Reserve room for the reply

If you ask for a 2,000-word essay, you need around 2,600 tokens of free space. Leave it.

5 — Use file attachments wisely

Many tools turn attached PDFs into tokens behind the scenes. A 100-page PDF can eat tens of thousands of tokens.

Here's a 200-page PDF — please find the
section about pricing strategy and rewrite
it in our company tone.

Below is the pricing strategy section
(pages 47–52) from our internal handbook.

Rewrite it in our company tone:
- friendly, plain English
- short paragraphs
- bullet lists where helpful
- no jargon

Section:
"""
… paste only pages 47–52 here …
"""

5. Step-by-Step: Long-Conversation Survival Kit

1. Notice when the chat feels "forgetful". That is the window getting full.
2. Ask the AI to summarise the chat so far. One short summary captures the essentials.
3. Open a fresh chat and paste the summary first. You just compressed 30 messages into one.
4. Continue working with full context restored. Repeat as needed.
5. For huge documents, split them. Process chunk by chunk and stitch results together.

6. Practice Exercises

Summarise everything important from this conversation into a single message I can paste into a new chat.

7. Key Takeaways