Stable Diffusion Prompting: From Basic to Advanced

Stable Diffusion gives you the most control of any image model — and the most things to learn. Positive prompts, negative prompts, attention weights, samplers, CFG, LoRAs, ControlNet — they are all knobs you turn. This guide takes you from a workable beginner prompt to the techniques pro users rely on for production work.

Note: Stable Diffusion is an ecosystem with many UIs (Automatic1111, ComfyUI, Forge, etc.) and many base models (SD 1.5, SDXL, SD 3.x, Flux). Specific syntax varies; concepts are universal.

1. Introduction

Stable Diffusion is an open-source image generation model that you can run locally, fine-tune, and extend. Compared to Midjourney's curated aesthetic, Stable Diffusion is raw clay — capable of almost anything, but it expects you to bring the artistic direction. The reward for that effort is total control: you can fix a hand, regenerate only a small region, force a specific pose, or load a style trained on your own work.

This tutorial assumes you have access to a Stable Diffusion UI (any popular one will do). We'll cover the prompt structure, the parameters that move the needle, and the advanced add-ons — LoRAs and ControlNet — that separate hobby use from professional output.

2. Positive and Negative Prompts

Every Stable Diffusion generation takes two prompts: a positive prompt describing what you want, and a negative prompt describing what to avoid. The negative prompt is not optional — it is half the steering wheel.

POSITIVE:
portrait of a Sikkimese tea farmer in her 50s, deep
laugh lines, wearing a traditional bakhu, soft window
light from the left, shallow depth of field,
shot on Fujifilm X-T5, 56mm f/1.2, photorealistic,
high detail, natural skin texture

NEGATIVE:
extra fingers, extra limbs, deformed hands, plastic skin,
oversaturated, low resolution, blurry, watermark, signature,
cartoon, 3d render

Boilerplate negative prompts are normal — keep a tested one in your notes app and adjust per project.

3. Attention Weights and Prompt Editing

Stable Diffusion lets you increase or decrease the influence of any token using parentheses. Syntax varies by UI but the common conventions:

• (token) ≈ 1.1× weight, ((token)) ≈ 1.21×, etc.
• (token:1.4) sets an explicit weight (1.4× influence).
• [token] reduces weight (≈ 0.9×).
• [from:to:when] — prompt editing: switch from one phrase to another partway through generation. Powerful for composition vs detail trade-offs.

Use weights surgically. (soft window light:1.3) can rescue a flat-looking portrait; bumping a dozen tokens at once usually wrecks coherence.

4. Sampler, Steps, and CFG

Three parameters shape the generation process itself.

• Sampler — the algorithm that walks noise into an image. Common solid defaults: DPM++ 2M Karras and Euler a. Try two or three for your use case; choose whichever produces the cleanest output at low step counts.
• Steps — how many denoising iterations. 20–30 is enough for most modern samplers. Beyond ~50, diminishing returns.
• CFG (Classifier-Free Guidance) — how strictly the model follows the prompt. Low (3–5) = creative, soft. Medium (6–8) = balanced; the default sweet spot. High (10+) = literal but often "burnt" and over-saturated.

5. LoRAs and Embeddings: Specialised Styles and Subjects

A LoRA (Low-Rank Adaptation) is a small add-on file that teaches a base model a new style, character, or concept without retraining the whole model. Loading a LoRA into your prompt looks like this:

POSITIVE:
<lora:vintage_botanical_v2:0.8>
vintage botanical illustration of a mango tree in fruit,
pencil-line detail, paper grain, muted earth tones

NEGATIVE:
photo, 3d, glossy, modern, neon

The number after the colon is the weight. Combine multiple LoRAs at moderate weights (0.5–0.8 each) to stack styles. Textual inversion embeddings work similarly but are much smaller — useful for invoking a specific look with a single token.

6. ControlNet: Composition Without Guessing

ControlNet lets you constrain a generation with an additional reference — a pose skeleton, a depth map, a sketch outline, an edge map. You keep the freedom of text prompting and get the composition you wanted.

• OpenPose — match a specific human pose from a reference image.
• Depth / Normal — preserve the 3D structure of a scene.
• Canny / Lineart — keep the outlines of a sketch while restyling everything else.
• Tile / Inpaint — upscale or repaint specific regions cleanly.

A typical pro workflow: rough composition in any tool → ControlNet Canny + a styled prompt → output that matches the layout exactly with the new style applied. This is how studios produce dozens of consistent images quickly.

7. Practice Exercises

8. Key Takeaways