Classifier-Free Guidance

The problem, from first principles

A text-to-image model should follow its prompt, but a plainly conditioned diffusion model often produces images that only loosely match it. Classifier-free guidance sharpens the effect of the condition with no extra classifier. During training the condition (say, the text prompt) is randomly replaced by a special null condition some fraction of the time, so one network learns both the conditional and the unconditional noise prediction. At sampling time you evaluate the network twice at each step, with the prompt and without it, and extrapolate from the unconditional prediction past the conditional one: eps = eps_uncond + w * (eps_cond - eps_uncond). A guidance scale w = 1 is the plain conditional model and larger w trades diversity for prompt adherence.

From theory to code

Implement cfg_combine and drop_condition.

Constraints

• cfg_combine(eps_uncond, eps_cond, w) returns eps_uncond + w * (eps_cond - eps_uncond).
• drop_condition(cond_ids, p, null_id, rng) returns a copy of the integer array cond_ids in which each entry is replaced by null_id with probability p. Draw exactly one array u = rng.random_sample(len(cond_ids)) and replace entries where u < p. rng is a np.random.RandomState.
• Do not modify the inputs.

Hints