Diffusion Models Explained: Making Images From Noise

Many image generators are built on a technique called diffusion. The name comes from physics, but the intuition is easy to follow if you imagine cleaning a very grainy photograph.

Step one: ruining pictures on purpose

During training, the system takes real images and adds random noise in small increments until nothing remains but static. This forward process is simple and needs no learning, because adding noise is easy.

Step two: learning to reverse it

The difficult part is the reverse. A neural network is trained to look at a noisy image and predict what noise was added, so that it can be removed. Repeated across enormous numbers of images and noise levels, the network learns what real pictures tend to look like: edges, textures, shapes, plausible colors.

Step three: generating

To create a new image, the system starts with pure random noise and applies the learned denoising step many times. Gradually structure emerges, first coarse layout, then finer details. Because the starting noise is random, each run can differ.

How a text prompt steers it

A separate component converts the prompt into numerical representations that capture meaning. During denoising, the network is guided so that the emerging image agrees with those representations. A setting often called guidance scale controls how strongly the prompt is enforced versus how much freedom the model keeps. Too strong can produce stiff or distorted results; too weak may drift from your request.

Working in a compressed space

Handling full-resolution pixels is costly, so many systems operate in a compressed internal representation and convert to pixels at the end. This makes generation faster and less resource-hungry.

Why diffusion became popular

• Training is generally stable compared with some earlier approaches.
• Outputs can be highly detailed and varied.
• The step-by-step process allows editing tricks, such as repainting a masked area.

Typical weaknesses

• Fine details such as hands, text inside images and counting objects can go wrong.
• Multiple steps cost computation, though methods exist to shorten them.
• Results inherit biases and gaps from training pictures.

Same idea beyond pictures

The denoising principle has been applied to audio and video, and researchers have explored it for other data types too. The common thread is learning to recover structure from randomness.

A kitchen analogy

Controlling the result

Users influence outputs through several levers: the wording of the prompt, a negative prompt describing what to avoid, a seed value that makes a run repeatable, and the number of denoising steps. Some tools also accept a reference image, so the model starts from a sketch instead of pure noise. Experimenting with one lever at a time shows what each does and builds intuition far faster than rewriting everything between attempts.

Editing with the same machinery

Because generation is a gradual process, it can be interrupted and guided. Inpainting regenerates a selected region while keeping the rest, outpainting extends a picture beyond its borders, and image-to-image modes begin from an existing picture partly covered in noise. These features show why diffusion is useful beyond one-shot creation: the denoising routine is a general-purpose engine for turning a rough guess into a coherent picture.

Imagine tasting a soup and trying to work out which ingredients were stirred in. Train on thousands of soups and you eventually become skilled at recognizing and removing what does not belong. Now start with a random mixture and keep removing what does not belong to a good soup; a recipe appears. That is the spirit of diffusion, minus the kitchen.