Text Conditioning for Image Generation

Text conditioning is the part of a text-to-image model that turns a prompt into something the image generator can use, and the way that something enters the generator. A text encoder, a contrastive one such as CLIP’s, a language model such as T5, or several of them, maps the tokenized prompt to a sequence of embeddings, often with one pooled vector besides. The denoiser of a diffusion model then reads the sequence through cross-attention or a joint attention over text and image tokens, and receives the pooled vector the way it receives the noise level. These choices, the captions the model was trained on, and the way adherence to the prompt is measured decide much of whether “a red cube on a blue cube” comes out with the colours on the right cubes. The text-to-image article surveys the task and which encoders well-known systems use; this article covers the mechanisms, the evidence for each choice, and the benchmarks in detail.

Definition

Let a caption $y$ be tokenized into $M$ tokens. A text encoder $\tau$ produces a sequence of contextual token embeddings $C = \tau(y) \in \mathbb{R}^{M \times d_\tau}$ and, optionally, a pooled embedding $\bar c \in \mathbb{R}^{d_p}$ summarizing the whole caption. Text conditioning is the set of operations by which $C$ and $\bar c$ change the output of a conditional generator, here the noise or velocity prediction $\epsilon_\theta(x_t, t, y)$ of a diffusion or flow model. Four operations recur:

• Pooled-vector conditioning: $\bar c$ is projected and added to the timestep embedding, and the sum sets per-channel scales and shifts after normalization layers, as a class label does in a class-conditional model.
• Cross-attention: image features are the queries and the token embeddings the keys and values, in layers added to the denoiser.
• Key–value concatenation: the projected token embeddings are appended to the keys and values of the denoiser’s own self-attention.
• Joint attention: text and image tokens form one sequence that is attended over jointly, in Stable Diffusion 3’s MM-DiT with each modality keeping its own weights, so that the text tokens are updated too.

Spatially aligned conditions such as low-resolution images or semantic maps are usually concatenated to the input channels instead (Rombach et al., 2022, §4.3.2); text has no spatial layout, so it takes one of the routes above.

Intuition

A pooled vector is a single summary of the prompt that every layer sees in the same way. It can say “this image contains a red thing and a blue thing”, and in principle more, but it cannot point at one region of the image and say “this part is the red thing”: a scale and shift per channel act alike at every position. Cross-attention gives each image location its own query, so each location can draw on the words that describe it. A useful picture is a painter with a one-line brief pinned to the wall (pooled) against one who can reread any word of a long brief while working on each part of the canvas (attention).

The two are complementary rather than rival. Several large systems feed the pooled vector and the token sequence together: Imagen adds an attention-pooled embedding to the timestep embedding and cross-attends over the sequence (Saharia et al., 2022, §2.5, Appendix F.1), GLIDE uses the final token’s embedding in place of a class embedding and appends the token sequence to every attention layer’s context (Nichol et al., 2022, §4.1), and Stable Diffusion 3 modulates its transformer with pooled CLIP vectors while attending jointly over the token sequence (Esser et al., 2024, §4).

The encoder matters as much as the route. Imagen’s authors note that language models are trained on text-only corpora far larger than paired image–text data and are generally much larger than the text encoders of image–text models, and they compared both families (§2.1). In our reading, a contrastive encoder keeps what helped match whole captions to whole images, which need not include which word modifies which.

Formal Definition

The papers describe most of these operations in words or code; the equations below put them in one notation of ours, with tokens as rows. Write the denoiser’s timestep embedding as $e_t = \operatorname{MLP}(\phi(t))$, with $\phi$ a sinusoidal embedding, and let $X \in \mathbb{R}^{N \times d}$ be the $N$ image tokens or flattened feature-map positions of one layer.

Pooled vector and scale–shift normalization. The pooled caption is projected and added:

$$e = e_t + W_p\, \bar c, \qquad \operatorname{block}(h) = \operatorname{Norm}(h) \odot \big(1 + \gamma(e)\big) + \beta(e),$$

with $\gamma, \beta$ linear maps giving one scale and one shift per channel. Imagen adds its pooled text embedding to the timestep embedding as class-conditional diffusion models add a class embedding (§2.5), and its configuration enables this scale–shift normalization (use_scale_shift_norm, Appendix F.1); GLIDE puts the final token’s embedding in place of the class embedding of the ADM U-Net (§4.1). The pooled vector can be the encoder’s own summary token or a pooling of $C$. CLIP’s text feature is the top-layer activation at the end-of-sequence token, layer-normalized and linearly projected into the joint embedding space (Radford et al., 2021, §2.3), and SDXL’s code takes the pooled OpenCLIP vector from that token in the same way (pool in sgm/modules/encoders/modules.py). Imagen compares mean pooling, $\bar c = \frac{1}{M}\sum_j C_j$, with attention pooling, a learned query attending over $C$ (Appendix D.3.1).

In SDXL the vector added to the timestep embedding is wider than the caption. Its base configuration sets adm_in_channels: 2816, the 1,280-dimensional pooled OpenCLIP ViT-bigG embedding concatenated with Fourier embeddings of three size and crop pairs, 256 dimensions per number, and the U-Net adds label_emb(y) to the timestep embedding (sd_xl_base.yaml; openaimodel.py, as of November 2026). The Stable Diffusion article describes these size and crop “micro-conditionings”.

In a diffusion transformer the same idea is adaptive normalization. Stable Diffusion 3 follows DiT’s modulation mechanism and feeds it the timestep embedding and the pooled vector $c_{\text{vec}}$ (Esser et al., 2024, §4); in the reference code the two embeddings are summed, $c = e_t + \operatorname{MLP}(c_{\text{vec}})$, and each block computes from $c$ a shift, a scale and a gate for its attention and for its MLP, with $\operatorname{modulate}(x) = x \odot (1 + \text{scale}) + \text{shift}$ (DismantledBlock in mmditx.py, which also has a scale-only variant without shifts). The same $c$ modulates the text stream’s blocks. The pooled vector thus reaches every block, but only as a handful of vectors per block that are the same at every token position.

Cross-attention. With projections $W_Q \in \mathbb{R}^{d \times d_a}$ and $W_K, W_V \in \mathbb{R}^{d_\tau \times d_a}$,

$$\operatorname{CrossAttn}(X, C) = \operatorname{softmax}\!\Big(\frac{(X W_Q)(C W_K)^\top}{\sqrt{d_a}}\Big)\, C W_V \in \mathbb{R}^{N \times d_a},$$

added back to $X$ through a residual connection. Row $i$ of the $N \times M$ attention matrix is position $i$’s distribution over the caption’s tokens. Rombach et al. (2022, §3.3) introduced this layer for latent diffusion with a domain-specific encoder $\tau_\theta$, writing the projections on the left, $Q = W^{(i)}_Q \cdot \varphi_i(z_t)$; the latent diffusion model article gives the layer in their notation, and the cross-attention article gives its cost, linear in $N$ and in $M$.

Key–value concatenation. GLIDE projects the final layer of token embeddings to each attention layer’s width and concatenates it to the layer’s attention context (§4.1), and Imagen implements its cross-attention by concatenating the text embedding sequence to the key–value pairs of each self-attention layer of its 64-pixel base model and first super-resolution model (Appendix D.3.1):

$$K = \begin{bmatrix} X W_K \\ C W_K' \end{bmatrix}, \qquad V = \begin{bmatrix} X W_V \\ C W_V' \end{bmatrix}, \qquad \operatorname{Attn}(X) = \operatorname{softmax}\!\Big(\frac{X W_Q K^\top}{\sqrt{d_a}}\Big) V \in \mathbb{R}^{N \times d_a}.$$

One softmax now covers image and text keys, so a position trades attention to other positions against attention to words; the output is still image-sized and the text is not updated. Imagen’s $1024^2$ super-resolution model has no self-attention layers, so it gets explicit cross-attention layers instead (Appendix D.3.1).

Joint attention (MM-DiT). Stable Diffusion 3 embeds text and image patches to a common width and concatenates the two sequences, but uses two separate sets of weights, which its authors describe as two independent transformers joined for the attention operation (Esser et al., 2024, §4, Fig. 2b). With text tokens $C'$ and image tokens $X$ at layer $\ell$,

$$[\,Q;K;V\,] = \big[\, C' W^{c}_{QKV} \,;\, X W^{x}_{QKV} \,\big], \qquad [\,C'_{\text{new}} ;\, X_{\text{new}}\,] = \operatorname{softmax}\!\Big(\frac{Q K^\top}{\sqrt{d_a}}\Big) V,$$

after which each part goes through its own output projection and MLP, both modulated by $c$ (block_mixing in mmditx.py). Unlike the two previous operations, both sets are rewritten at every layer, so the text representation that later layers attend to depends on the image; the cost is $(N + M)^2$ scores per head. Stable Diffusion 3’s context is $C \in \mathbb{R}^{154 \times 4096}$: the penultimate hidden states of CLIP ViT-L/14 and OpenCLIP ViT-bigG/14, concatenated along channels (77 tokens of width 2,048) and zero-padded to 4,096, then concatenated along the sequence with the 77 final hidden states of a T5-v1.1-XXL encoder; the pooled outputs, of sizes 768 and 1,280, form $c_{\text{vec}} \in \mathbb{R}^{2048}$ (Esser et al., 2024, Appendix B.2).

The null caption. Classifier-free guidance needs a prediction without the text, so training replaces the condition some of the time: Imagen zeroes the text embeddings with 10% probability (§4.1), GLIDE fine-tuned with 20% of token sequences replaced by the empty sequence (§4.2), and Stable Diffusion 3, in the set-up given for its formulation experiments, zeroes each of its three encoders’ outputs independently with probability 46.4%, so that all three are dropped together in about 10% of steps (Appendix B.3). The paper’s later paragraph on flexible text encoders refers back to B.3 for the final model but gives the rate as 46.3% (Appendix C.4); $0.464^3 \approx 0.0999$, while $0.463^3 \approx 0.0993$.

Properties

Sequence beats pooled, in Imagen’s controlled comparison. Imagen’s authors trained the 64-pixel base model with a frozen encoder conditioned by mean pooling, attention pooling or cross-attention over the sequence. Over CLIP-score-against-FID curves swept over guidance weights, either pooled configuration was noticeably worse than attending over the contextual embeddings, in fidelity and in alignment (Appendix D.3.1, Fig. A.13a; summarized in §4.4). The sub-caption of Fig. A.13a reads “Comparison between different text encoders”, although its legend lists the three conditioning schemes. The paper does not say whether the cross-attention arm also had the pooled vector, as the final model does.

A pooled vector is coarse by construction. Stable Diffusion 3’s authors give exactly this reason, citing SDXL, for adding the sequence to their pooled modulation (§4). The SDXL authors’ own “concept bleeding” discussion is summarized in the text-to-image article. In our reading, the limit is not the vector’s size: a pooled vector could encode which colour goes with which object, but scale and shift per channel apply the same change everywhere, so a convolutional network must work out from the vector where each object goes, whereas an attention layer can look the answer up per position.

Joint attention against cross-attention. On CC12M, with the training set-up of their formulation study, Stable Diffusion 3’s authors compared DiT with text and image tokens concatenated and one set of weights, CrossDiT (cross-attending to the text tokens), UViT and MM-DiT. MM-DiT did best on validation loss, CLIP score and FID; CrossDiT beat UViT, although UViT learned faster at first; plain DiT was worst; a third set of weights, separating CLIP and T5 tokens, gained little (§5.2.3, Fig. 4).

Which layer of the encoder. Systems rarely take an encoder’s final output. Imagen uses the encoder part of T5 and, for CLIP, the penultimate layer of the text encoder (Appendix D.1). Stable Diffusion v1 cross-attends to the last hidden state of CLIP ViT-L/14 (FrozenCLIPEmbedder in the CompVis code, which pads or truncates prompts to 77 tokens). SDXL’s base configuration takes hidden state 11 of the 12-layer CLIP ViT-L (layer: hidden, layer_idx: 11) and the penultimate layer of OpenCLIP ViT-bigG and concatenates them along channels to a 2,048-wide context (context_dim: 2048), and Stable Diffusion 3 likewise uses penultimate CLIP layers but T5’s final layer (Appendix B.2). The sources state the choices without ablating them.

Normalizing the text embeddings. Imagen’s authors found layer normalization of the text embeddings in the attention and pooling layers to “help considerably” (§2.5).

Frozen or trained. The latent diffusion paper trains its text transformer, over BERT-tokenized prompts, jointly with the denoiser (§3.3, §4.3.1). GLIDE’s text transformer, 24 residual blocks of width 2,048, accounts for about 1.2 of its 3.5 billion parameters (Appendix B.1). Imagen freezes its encoders, which lets the embeddings be computed offline at negligible cost during training (§2.1), and Stable Diffusion 3 encodes text with pretrained, frozen models as well (§4). Freezing ties the generator to whatever the encoder represents; the T2I-CompBench authors’ own fine-tuning method, GORS, adapts both the CLIP text encoder and the U-Net of Stable Diffusion v2 with LoRA and found that better than adapting either alone (Huang et al., 2023, §6.5, Table 6).

The Encoder and Its Size

Imagen’s ablations are still the most direct evidence on encoders (Saharia et al., 2022, Appendix D.1). With a 300-million-parameter 64-pixel model, the authors compared frozen BERT-base and BERT-large, T5-Small to T5-XXL and CLIP ViT-L/14. Larger T5 encoders gave better image–text alignment, measured by CLIP score during training (Fig. A.6), and better CLIP–FID trade-offs (Fig. A.5a), with T5-XXL best. The comparison with CLIP that the text-to-image article reports used two model sizes: COCO scores came from the 300M models, but because these did badly on DrawBench’s harder prompts, the DrawBench comparison used 1-billion-parameter models, and the raters’ preference for T5-XXL was strongest for image–text alignment (Fig. A.5b, A.7). Scaling the frozen encoder improved quality more than scaling the U-Net from 300 million to 2 billion parameters (§4.4, Appendix D.3, Fig. A.13b).

The encoders differ in training data as well as objective: Imagen’s authors describe BERT as trained on about 20 GB of Wikipedia and BooksCorpus, T5 on the C4 corpus of about 800 GB with a denoising objective, and CLIP on image–text pairs (Appendix D.1). T5’s own paper describes C4 as about 750 GB and its largest model as about 11 billion parameters (Raffel et al., 2020, §2.2, §3.7); Imagen’s T5-XXL encoder has 4.6 billion (§4.4), and Stable Diffusion 3’s authors count 4.7 billion for theirs (Appendix C.4).

Later evidence is less controlled. GenEval’s authors saw no gain in overall score across Stable Diffusion v1.1 to v1.5 checkpoints after v1.2 but a clear gain for v2, and suggest that the change from CLIP ViT-L/14 to the larger OpenCLIP ViT-H/14, trained on different data, may explain it (Ghosh et al., 2023, §5.2). Stable Diffusion 3’s encoders can be dropped at inference because of the dropout above; its authors observed limited losses without T5 except for long, detailed prompts and written text, as the text-to-image article quantifies (Appendix C.4). DALL·E 3’s authors suspect that their T5 encoder makes text rendering unreliable, because the model sees tokens for whole words and must map them to letters, and they propose character-level language models as future work (Betker et al., 2023, §5.2).

Caption Quality and Recaptioning

The text a model is trained on bounds what its conditioning can learn. Betker et al. (2023), the DALL·E 3 report, argue that web captions describe the main subject and omit background objects, positions, counts, colours and visible text, and that alt-text is often simply wrong (§2). They trained an image captioner, a language model conditioned on a CLIP image embedding and pretrained jointly with a CLIP-style and a language-modelling objective following CoCa, then fine-tuned it twice: once to produce short synthetic captions describing the main subject, and once to produce descriptive synthetic captions covering surroundings, background, text, style and colour (§2.1).

Their ablations trained identical T5-conditioned latent diffusion models at 256 pixels for 500,000 steps at batch size 2,048 on the same images (§3.2, Appendix A):

• Caption type. Models trained on 95% short or 95% descriptive synthetic captions scored slightly higher CLIP scores than the model trained on original captions when scored against original captions, and markedly higher when scored against descriptive captions (§3.3, Fig. 4).
• Blending. Training only on synthetic captions would let the model learn the captioner’s habits (casing, punctuation, length, opening words), so original captions are mixed in at sampling time. Of blends of 65%, 80%, 90% and 95% synthetic, 65% fell far behind and was dropped, and higher blends always scored higher (§3.1, §3.4, Fig. 5).
• Inference-time “upsampling”. A model trained on long captions expects long prompts, so short user prompts are rewritten by GPT-4 into detailed descriptions (§3.5, Appendix C). The text points to “Table 7” and “Figure 7” for examples; the upsampled DrawBench examples are in Figure 6, and Figure 7 shows failure cases.

DALL·E 3 itself was trained on 95% synthetic captions, but the report says little about the model beyond its being a scaled-up version of the ablation model, and warns that its evaluations do not isolate the effect of the captions (§4, footnote 5). Its listed limitations trace back to the captioner: object placement (“to the left of”, “underneath”) is unreliable because the captioner states placement unreliably, and captions hallucinate details such as plant genus or bird species, so the model is unreliable on such specific terms (§5.1, §5.3).

Stable Diffusion 3’s authors repeated the experiment with an off-the-shelf vision–language model, CogVLM, using 50% original and 50% synthetic captions so that the generator would not forget concepts the captioner lacks (Esser et al., 2024, §5.2.2). Two depth-15 MM-DiT models trained for 250,000 steps scored as follows on GenEval (Table 4):

GenEval task	Original captions	50/50 mix
Colour attribution (attribute binding)	11.75	24.75
Colours	71.54	68.09
Position	6.50	18.00
Counting	33.44	41.56
Single object	95.00	93.75
Two objects	41.41	52.53
Overall	43.27	49.78

The gains are in the tasks with two objects or a number, largest for binding and position; single objects and single colours scored slightly lower.

Measuring Prompt Adherence

Prompt adherence, or prompt following, image–text alignment and prompt fidelity in other papers, is whether an image depicts what its prompt states. Betker et al. use “prompt following” for models that “overlook the words, word ordering, or meaning” of a caption (§1). The text-to-image article compares the measures and their agreement with people; the definitions follow.

CLIPScore and CLIP similarity

For an image with CLIP ViT-B/32 embedding $\mathbf v$ and a candidate caption with text embedding $\mathbf c$, computed with the prefix “A photo depicts”,

$$\text{CLIP-S}(\mathbf c, \mathbf v) = w \cdot \max\big(\cos(\mathbf c, \mathbf v),\, 0\big), \qquad w = 2.5,$$

averaged over pairs for a corpus score (Hessel et al., 2021, §3). The weight stretches the observed range of cosines, roughly 0 to 0.4, toward $[0, 1]$; RefCLIPScore, the harmonic mean of CLIP-S and the best cosine to reference captions, needs reference captions, which a text-to-image prompt does not have. Text-to-image papers use the name loosely: DALL·E 3’s report computes the plain cosine with ViT-B/32, averages it over 50,000 pairs and multiplies by 100, citing Hessel et al. (§3.2); and GLIDE multiplies the CLIP ViT-B/16 similarity by CLIP’s logit scale, 100 (Appendix A.2). Scores computed by different recipes are not comparable. Since the score compares one pooled text vector with one pooled image vector, it shares the coarseness of pooled conditioning.

T2I-CompBench

Huang et al. (2023) built 6,000 compositional prompts, 1,000 in each of six subcategories, split 700 for training and 300 for testing (§3): attribute binding for colour, shape and texture (800 template prompts “a {adj} {noun} and a {adj} {noun}” and 200 natural ones per subcategory, with test sets of 200 seen and 100 unseen adjective–noun pairs); spatial relations, with contrastive pairs made by swapping the two nouns; non-spatial relations such as “hold” or “look at”; and complex compositions. Ten images are generated per prompt (§6.2). Each family has its own metric (§4):

• Disentangled BLIP-VQA for attribute binding. The prompt is split into one question per noun phrase (“a green bench?”, “a red car?”), BLIP’s visual question answering model gives the probability of “yes” for each, and the score is their product. The released code forms the questions from spaCy noun chunks and multiplies the per-question probabilities (BLIP_vqa.py).
• UniDet for spatial relations. The paper’s rule: object 1 is left of object 2 if $x_1 < x_2$, $|x_1 - x_2| > |y_1 - y_2|$ and the boxes’ IoU is below 0.1, and likewise for right, top and bottom; “next to”, “near” and “on the side of” compare the centres’ distance with a threshold. The released code is softer than the paper’s rule: an overlapping pair earns $0.1/\text{IoU}$ instead of 0 for its position, and the image score is $0.25\,p_1 + 0.25\,p_2 + 0.5\,\text{position}$ with $p_1, p_2$ the detection confidences, set to 0 when it falls below 0.5; pairs that should be “near” but are far apart likewise earn a fraction (determine_position and the scoring loop in 2D_spatial_eval.py, at the commit read).
• CLIPScore for non-spatial relations, and the 3-in-1 average of CLIPScore, disentangled BLIP-VQA and UniDet for complex prompts.

The authors validated the metrics by rank correlation with human ratings of 150 prompts per model: for colour binding, Kendall’s $\tau$ was 0.6297 for disentangled BLIP-VQA against 0.1938 for CLIPScore; for spatial relations, 0.4756 for UniDet against 0.2741; for shape binding every metric correlated weakly, 0.2707 at best (Table 5). DALL·E 3’s report uses the three attribute-binding subsets with disentangled BLIP-VQA (Betker et al., 2023, §4.1.3). The repository also links an extended version, T2I-CompBench++, published in an IEEE journal, which was not read.

GenEval

Ghosh et al. (2023) generate 553 prompts from templates over the 80 COCO class names: single object (80), two objects (99), counting with 2, 3 or 4 (80), colours (94), position with above, below, left or right (100) and attribute binding, two objects with different colours (100) (Table 1). The evaluation runs a COCO-trained Mask2Former instance segmenter and, from the paper and the released evaluate_images.py:

• Objects count as detected above confidence 0.3, or 0.9 for the counting task, where low-confidence duplicates otherwise inflate counts (§3.2).
• Colour is classified by CLIP ViT-L/14 zero-shot over “a photo of a [COLOR] [OBJECT]”, on the object’s box crop with the background outside its mask replaced by gray (§3.2); the released code averages that template with two variants (“a photo of a [COLOR]-colored [OBJECT]”, “a photo of a [COLOR] object”). Section 3.1 speaks of 11 basic colour terms after Berlin and Kay; Appendix C.1 and the code list 10, because gray, the background fill, is excluded from prompts and from classification.
• Position compares box centres: the centre offset is shrunk on each axis by 0.1 times the sum of the two boxes’ sizes, divided by the length of the original offset, and a relation holds when its component exceeds 0.5 (relative_position; the paper describes a minimum distance proportional to the box dimensions).
• An image is correct only if every element of the prompt is right; the task score is the share of correct images, four per prompt, and the overall score is the mean of the six task scores (§3.2, §5.1). Overall scores varied by 0.01 to 0.02 across random seeds (Table 2 caption).

The two compositional tasks need not move together. In Stable Diffusion 3’s Table 5, DALL·E 3 scored 0.45 on attribute binding and 0.43 on position, and Stable Diffusion 3’s depth-38 model at $1024^2$ after preference tuning 0.60 and 0.33 (Esser et al., 2024).

Model judges

DALL·E 3’s report also judged DrawBench prompts with a vision-enabled GPT-4, asking for “correct” or “incorrect” for each of four images per prompt (§4.1.2, Appendix D), and found that judge no better than chance at counting, so it added human ratings (§4.2.1). In T2I-CompBench’s correlation study, MiniGPT-4 with chain-of-thought correlated with human ratings less well than the specialised metrics on every subcategory (Huang et al., 2023, Table 5).

Common Misconceptions

• “A larger image model fixes prompt adherence.” In Imagen’s ablations, scaling the frozen text encoder helped more than scaling the U-Net (Saharia et al., 2022, §4.4).
• “The pooled embedding is obsolete.” Imagen, GLIDE, SDXL and Stable Diffusion 3 all use one alongside the token sequence; what Imagen found worse was a pooled vector instead of the sequence.
• “Guidance adds what the conditioning lacks.” Guidance extrapolates from the unconditional toward the conditional prediction; in our reading it can sharpen only what the conditional prediction already distinguishes. The classifier-free guidance article shows its effect on a toy distribution.
• “CLIP score measures compositional correctness.” It compares one caption vector with one image vector. On T2I-CompBench its rank correlation with human ratings of colour binding was 0.19, against 0.63 for the question-based metric (Huang et al., 2023, Table 5).
• “Synthetic captions are free supervision.” They carry the captioner’s errors and habits: DALL·E 3 inherited its captioner’s weakness with spatial words and its hallucinated species names, and needed original captions mixed in and rewritten prompts at inference (Betker et al., 2023, §3.1, §3.5, §5).

Where It Is Used

Every conditional generator steered by language uses one of these mechanisms: Stable Diffusion and other latent diffusion models, pixel-space cascades such as Imagen, and text-guided image editing and inpainting, where cross-attention maps are themselves edited (see the cross-attention article). The same encoders and recaptioning are used in vision–language systems in the other direction: the captioners that rewrite training sets are image captioning models.