Temperature, Top-P and Other AI Parameters Explained Simply

Most AI tools have a few knobs you can tweak: temperature, top-p, max tokens, frequency penalty, presence penalty. They sound intimidating but they boil down to "how creative do you want the AI to be?". This tutorial demystifies each one in plain English.

1. Introduction

If you only ever use ChatGPT in a browser, you may not see these controls. But every time you use an AI API, a custom app, or a playground, you will. Knowing what they do means you can dial the AI's behaviour exactly where you need it.

2. The Concepts Explained

Temperature — how predictable the AI should be

Temperature ranges from 0 to about 2. At 0, the AI always picks the single most likely next token — answers become consistent, factual, sometimes a bit dry. At 1, it picks from a wider mix — answers become more varied and creative. Above 1.2 you often get strange, surprising, or unreliable output.

Top-P (nucleus sampling) — limiting the pool of candidates

Instead of looking at every possible next token, top-p tells the AI: "only consider the smallest group of tokens whose combined probability adds up to P". A top-p of 0.9 means the AI ignores the long tail of unlikely tokens. Lower top-p = more focused. Higher = more variety.

Most of the time you set either temperature or top-p — not both. Pick one knob and leave the other at default.

Max tokens — the maximum length of the reply

This caps how long the AI's answer can be. If you set max tokens too low, the reply gets cut off mid-sentence. Set it high enough for the format you asked for. (Roughly 100 tokens ≈ 75 words.)

Frequency & Presence penalties — discouraging repetition

Both nudge the AI away from repeating itself.

• Frequency penalty reduces the chance of using the same exact words repeatedly. Useful when the AI keeps saying "however" or "moreover" in every paragraph.
• Presence penalty encourages introducing new topics. Useful when the AI loops on the same idea.

Values are usually between 0 and 2. Start at 0 and increase gradually if you see repetition.

Stop sequences — telling the AI when to stop

You can specify a string (or several) that causes the AI to stop generating as soon as it produces them. Useful when you want output in a precise format (e.g. stop after the first "---" line).

System prompt — the AI's standing instructions

Not really a "parameter", but worth listing here. The system prompt is a hidden instruction the AI receives before the conversation begins. It sets the role, the tone, and the rules. (Full deep-dive in Topic 11.)

3. Recommended Settings by Task

4. Before and After

Extract every product name and price from the invoice text below.
Return JSON only: [{"product":"","price":""}]

Invoice text:
[paste invoice here]

(Settings: temperature 1.2 — too high for extraction)

Extract every product name and price from the invoice text below.
Return JSON only: [{"product":"","price":""}]

Invoice text:
[paste invoice here]

(Settings: temperature 0 — best for factual extraction)

5. Step-by-Step: Choosing Parameters

1. Start with the task type. Factual? Creative? Structured?
2. Look up the recommended range from the table above as a starting point.
3. Adjust temperature first. Leave top-p at its default unless you have a specific reason to change it.
4. Set max tokens generously. Better to leave room than to be cut off.
5. Only use frequency/presence penalties if you notice repetition. Don't tune them preemptively.
6. Save the combo that works for repeat tasks — they become "presets" in your workflow.

6. Practice Exercises

7. Key Takeaways