Stable Diffusion

Stable Diffusion (SD) is a family of text-to-image models whose weights are published for anyone to download. Given a prompt, it generates an image; related checkpoints edit and upscale images. Versions 1.1 to 1.4 were released for download through the CompVis group’s GitHub and Hugging Face pages on or around 22 August 2022, as the High Court judgment cited below records (¶34). The CompVis repository describes the model as a latent diffusion model made possible by a collaboration with Stability AI and Runway, trained on a subset of the LAION-5B image–text collection. Later versions came from Stability AI. Versions 1.x, 2.x and SDXL use a convolutional U-Net denoiser conditioned on frozen CLIP-type text encoders; Stable Diffusion 3 and 3.5 replace it with a multimodal transformer trained as a rectified flow. This article covers the family and its ecosystem; the latent diffusion article explains the two-stage design, and the classifier-free guidance article the guidance scale.

Problem Addressed

Stable Diffusion applied latent diffusion, which runs a diffusion model’s denoiser in an autoencoder’s latent space to cut its cost, to text-to-image generation at web scale. According to its README, the v1 model, with an 860M-parameter U-Net and a 123M-parameter text encoder, is relatively lightweight and runs on a GPU with at least 10 GB of memory.

Architecture

Every version has an autoencoder, a denoiser that generates latents, and frozen text encoders. The table summarizes the official descriptions; parameter counts are for the denoiser.

Version	Released	Denoiser	Text encoder(s)	Native resolution	Training target
1.1–1.5	2022 (v1.1–1.4 about 22 Aug)	U-Net, 860M	CLIP ViT-L/14 (123M)	$512^2$	noise
2.0, 2.1	24 Nov and 7 Dec 2022	U-Net, 865M	OpenCLIP ViT-H/14	$768^2$ (v), $512^2$ (base)	v (768), noise (base)
SDXL 1.0	26 Jul 2023 (0.9: 22 Jun)	U-Net, 2.6B	CLIP ViT-L and OpenCLIP ViT-bigG (817M together)	about $1024^2$, several aspect ratios	noise
SD 3 Medium	not dated on the card; supported by the sd3.5 code from 22 Oct 2024	MM-DiT	CLIP ViT-L, OpenCLIP ViT-bigG, T5-XXL	not stated on the card	rectified flow (SD 3 paper)
SD 3.5 Large, Large Turbo, Medium	inference code 22 Oct 2024 (Large, Large Turbo) and 29 Oct 2024 (Medium)	MM-DiT; MMDiT-X (Medium)	same three	up to 1440 px in Medium’s training	not stated; the cards cite the SD 3 paper

Sources: CompVis README and model cards, and the Getty judgment for the v1 release date (v1); the archived Stability AI README (v2); Podell et al., Table 1, and the generative-models README (SDXL); the SD 3 and 3.5 model cards and the sd3.5 README, whose dates are those of inference-code releases, as of October 2026. SD 3 Medium’s card states no parameter count; Esser et al. (2024) trained models up to 8 billion parameters.

Autoencoder

All U-Net versions use an $f = 8$ autoencoder with 4 latent channels (factor, shapes and scale factors: see the latent diffusion article). Podell et al. note that SD 2.x fine-tuned v1’s decoder with a reduced weight on the perceptual loss, and that SDXL retrained the same architecture from scratch at batch size 256 instead of 9, with an exponential moving average of the weights. On $256 \times 256$ COCO 2017 validation images the reconstruction Fréchet Inception Distance (FID) fell from 5.0 (v1) and 4.7 (v2) to 4.4 (SDXL), and PSNR rose from 23.4 and 24.5 to 24.7 dB (arXiv v1, Table 3).

Denoiser

The v1 and v2 U-Nets share one layout, with one transformer block at each of four resolution levels. SDXL drops the lowest level and redistributes the blocks as 0, 2 and 10, which with a wider cross-attention context (2048 against 768 or 1024) triples the size (Podell et al., §2.1, Table 1). It also conditions on the pooled OpenCLIP embedding and on two micro-conditionings, embedded with Fourier features and added to the timestep embedding: the original image size, so that small training images need not be discarded (39% of their pretraining data had a side below 256 pixels), and the crop coordinates, so that random cropping does not teach the model to cut objects off (§2.2). The optional refiner and SD 3’s MM-DiT transformer are described in the latent diffusion article. The SD 3.5 cards add QK-normalization, and the Medium card describes MMDiT-X with extra self-attention in the first transformer layers; the card says both “first 12” and “first 13” layers in different places.

Text encoders

Version 1 cross-attends to the non-pooled token embeddings of CLIP ViT-L/14, v2 to the penultimate-layer embeddings of OpenCLIP ViT-H/14, and SDXL to the concatenated penultimate outputs of both CLIP ViT-L and OpenCLIP ViT-bigG. SD 3 adds T5-XXL, a 4.7-billion-parameter language model, and drops each of its three encoders with probability 46.3% in training, so that any subset can be used at inference. Removing T5 had no effect on aesthetic ratings (50% win rate) and a small one on prompt adherence (46%), but mattered for rendering written text (38%) (Esser et al., Appendix B.3 and the paragraph on flexible text encoders).

Training

Data

The v1 model card lists LAION-2B(en) and subsets of it, all from LAION-5B. Each checkpoint resumed from an earlier one:

• v1.1: 237,000 steps at $256 \times 256$ on laion2B-en, then 194,000 steps at $512 \times 512$ on laion-high-resolution (170M images of at least $1024 \times 1024$).
• v1.2: 515,000 steps from v1.1 on images of at least $512 \times 512$ with an estimated aesthetics score above 5.0 and an estimated watermark probability below 0.5.
• v1.3 and v1.4: 195,000 and 225,000 steps from v1.2 on that subset, with the text conditioning dropped 10% of the time for classifier-free guidance.
• v1.5: 595,000 steps from v1.2 on the same data, released by Runway (archived model card).

The official sources disagree on the name of the v1.2 subset: the CompVis README calls it “laion-aesthetics v2 5+”, scored with the LAION-Aesthetics Predictor V2, while the v1.4 model card calls it “laion-improved-aesthetics” and uses the other name only for v1.4. The v1 data were not deduplicated, and the card notes some memorization of duplicated images. Training used 256 A100 GPUs, AdamW and batch size 2,048; the card estimates 150,000 A100 hours and 11,250 kg of CO₂-equivalent emissions.

For SD 2.0, the base model trained 550,000 steps at $256 \times 256$ and 850,000 at $512 \times 512$ on a LAION-5B subset filtered by LAION’s NSFW classifier at punsafe 0.1 (conservative, the card says) and by an aesthetic score of at least 4.5. SD 2.1 continued from the 768-pixel 2.0 model for 55,000 steps with the same filter and 155,000 more with punsafe 0.98 (archived v2 and v2.1 cards), a filtering the archived README calls less restrictive. SDXL trained on an internal dataset: 600,000 steps at $256 \times 256$ with batch size 2,048, 200,000 at $512 \times 512$, then multi-aspect training at about $1024^2$ pixels with offset noise of 0.05 (Podell et al., §2.5). For SD 3, Esser et al. filtered sexual content and low-aesthetic images, removed near-duplicates with SSCD image embeddings (Appendices C.1 and D), and replaced half the captions with synthetic ones from the CogVLM vision–language model, which raised a smaller model’s GenEval score from 43.27 to 49.78 (Table 4). The SD 3 Medium card states pretraining on 1 billion images and fine-tuning on 30 million aesthetic images and 3 million preference images.

Objectives

The v1 and SDXL U-Nets use the latent noise-prediction loss of the latent diffusion article, with the text embedding $\tau(y)$ as the cross-attention condition. SD 2.0-v, fine-tuned from the noise-predicting 512-pixel base for 150,000 steps and then 140,000 steps at $768 \times 768$, uses the v-objective of Salimans and Ho (2022), whose target $v$ is defined in the diffusion model article:

$$L_v = \mathbb{E}_{z_0,\, y,\, \epsilon,\, t}\Big[\big\lVert \sqrt{\bar\alpha_t}\,\epsilon - \sqrt{1-\bar\alpha_t}\, z_0 - v_\theta(z_t, t, \tau(y)) \big\rVert^2\Big] = \mathbb{E}\Big[\big(1 + \mathrm{SNR}(t)\big)\, \lVert z_0 - \hat z_0 \rVert^2\Big],$$

with $\hat z_0 = \sqrt{\bar\alpha_t}\, z_t - \sqrt{1 - \bar\alpha_t}\, v_\theta$. The second form is our derivation: substituting $z_t$ gives $z_0 - \hat z_0 = \sqrt{1 - \bar\alpha_t}\,(v_\theta - v)$, and $1/(1 - \bar\alpha_t) = 1 + \mathrm{SNR}(t)$. Unlike the noise loss, whose weight $\mathrm{SNR}(t)$ vanishes at the noisiest steps, it keeps a weight of at least 1 there. The README notes that v-models can be sampled with higher guidance scales. SD 3’s rectified-flow objective is summarized in the latent diffusion article; Esser et al. also fine-tuned their 2-billion- and 8-billion-parameter models with direct preference optimization (DPO; Appendix C.3).

Inference

The inputs are a prompt, optionally a negative prompt, a seed for the initial latent, the number of sampler steps and the guidance scale $s$ (1 means no guidance). The CompVis reference script defaults to the PLMS sampler with 50 steps, scale 7.5 and $512 \times 512$ output; the v2 script uses 50 DDIM steps at $768 \times 768$; the SDXL card’s base-plus-refiner example runs 40 steps, 80% of them on the base model; and the SD 3 Medium card’s example uses 28 steps at scale 7.0 (as of October 2026). The diffusion sampling article explains the samplers. The v1 reference script also runs a safety checker and adds an invisible watermark identifying the image as machine-generated; in diffusers, the v1.4 card explains, the checker compares the CLIP embedding of each output with hidden NSFW concepts.

The CompVis README also describes image-to-image editing by partially noising an input image and denoising it with a new prompt (SDEdit); a strength between 0 and 1 sets how much noise is added. Version 2 added a depth-conditioned model, fine-tuned from the base model for 200,000 steps with an extra zero-initialized input channel for a MiDaS depth estimate, and a text-guided $\times 4$ latent upscaler conditioned on a noise level added to the low-resolution input. Dedicated inpainting checkpoints (v1.5 and v2.0) add zero-initialized input channels for the latent of the masked image and for the mask; the one resumed from v1.5 trained 440,000 steps with synthetic masks, 25% of which masked the whole image (archived v1.5 card). Image inpainting has its own article.

Example: guidance scale, step count and diversity

Every reference script exposes the guidance scale and the step count. The toy below varies both, with exact scores instead of a trained network. Images are $32 \times 32$ disks, squares and triangles of radius 3, 5 or 7 pixels at one of four places, with pixel noise of standard deviation 0.3; invented captions name the shape, and 20% of them are wrong, as web captions can be. A linear $f = 4$ first stage (average pooling, decoded by upsampling) keeps the latents a Gaussian mixture, so the noise predictors are exact. Deterministic DDIM with Stable Diffusion v1’s noise schedule samples 200 images per caption for each setting from the same initial latents. The script reports prompt adherence (the nearest mode has the prompted shape), diversity (shares of small and large shapes, layout entropy) and saturation (the largest decoded pixel magnitude; the data lie near $[-1, 1]$).

import sys
import numpy as np

# Toy text-to-image data: 32x32 grayscale images in [-1, 1] of a disk, square or
# triangle (radius 3, 5 or 7 pixels, at one of 4 places) plus pixel noise of std 0.3.
# Invented captions name the shape, and 20% of captions are wrong (web-style noise).
H = W = 32
yy, xx = np.mgrid[:H, :W] + 0.5
def shape(kind, cy, cx, r):
    if kind == "disk":   return (yy - cy) ** 2 + (xx - cx) ** 2 < r ** 2
    if kind == "square": return (abs(yy - cy) < 0.85 * r) & (abs(xx - cx) < 0.85 * r)
    return (yy - cy < r) & (abs(xx - cx) < (yy - cy + r) / 2)          # triangle
kinds, radii, places = ["disk", "square", "triangle"], [3, 5, 7], [(9, 9), (9, 23), (23, 9), (23, 23)]
modes = [(k, r, p) for k in range(3) for r in range(3) for p in range(4)]   # 36 image modes
tmpl = np.array([np.where(shape(kinds[k], *places[p], radii[r]), 1.0, -1.0).ravel()
                 for k, r, p in modes])
shape_of, size_of, place_of = np.array(modes).T
caption_noise, pixel_std = 0.2, 0.3
Wc = np.where(shape_of[None] == np.arange(3)[:, None],
              (1 - caption_noise) / 12, caption_noise / 24)   # p(mode | caption), (3, 36)
Wu = Wc.mean(0)                                                # p(mode), captions equally likely

# First stage: f = 4 average pooling to an 8x8 latent (decoder: nearest upsampling),
# centred and scaled to unit variance. A linear encoder keeps the data a Gaussian
# mixture, so the exact conditional and unconditional noise predictors are known.
f = 4
enc = lambda x: x.reshape(-1, H // f, f, W // f, f).mean((2, 4)).reshape(len(x), -1)
dec = lambda z: np.repeat(np.repeat(z.reshape(-1, H // f, W // f), f, 1), f, 2).reshape(len(z), -1)
M0 = enc(tmpl)
shift, scale = M0.mean(), 1 / np.sqrt(M0.var() + pixel_std ** 2 / f ** 2)
M, v0 = (M0 - shift) * scale, (pixel_std / f * scale) ** 2   # latent modes, within-mode variance

T = 1000
beta = np.linspace(0.00085 ** 0.5, 0.012 ** 0.5, T) ** 2      # Stable Diffusion v1 schedule
ab = np.cumprod(1 - beta)

def eps(z, a, w):
    """Exact noise prediction for the mixture sum_k w_k N(sqrt(a) M_k, (a v0 + 1 - a) I)."""
    v = a * v0 + 1 - a
    lg = np.log(w) - ((z[:, None, :] - np.sqrt(a) * M[None]) ** 2).sum(-1) / (2 * v)
    r = np.exp(lg - lg.max(1, keepdims=True)); r /= r.sum(1, keepdims=True)
    return np.sqrt(1 - a) * (z - np.sqrt(a) * r @ M) / v

def ddim(c, s, steps, zT):
    """Deterministic DDIM with classifier-free guidance at scale s (s = 1: no guidance)."""
    ts = np.linspace(T - 1, 0, steps).round().astype(int)
    z = zT.copy()
    for i, t in enumerate(ts):
        e_u, e_c = eps(z, ab[t], Wu[None]), eps(z, ab[t], Wc[c])
        e = e_u + s * (e_c - e_u)
        x0 = (z - np.sqrt(1 - ab[t]) * e) / np.sqrt(ab[t])
        a_next = ab[ts[i + 1]] if i + 1 < steps else 1.0
        z = np.sqrt(a_next) * x0 + np.sqrt(1 - a_next) * e
    return z

seed = int(sys.argv[1]) if len(sys.argv) > 1 else 0
n = 200                                            # samples per caption
c = np.repeat(np.arange(3), n)
zT = np.random.default_rng(seed).standard_normal((3 * n, M.shape[1]))   # same z_T for every cell
R = dec(M0)                                        # what the first stage keeps of each mode
print(f"seed {seed}; data: right shape 0.800, small 0.333, large 0.333, layout 2.000 bits")
print("steps     s  right shape  small  large  layout bits  max |pixel|")
for steps in [5, 10, 25, 50, 200]:
    for s in [1, 2, 4, 7.5, 15]:
        x = dec(ddim(c, s, steps, zT) / scale + shift)
        k = ((x[:, None] - R[None]) ** 2).sum(-1).argmin(1)         # nearest mode
        layout = np.mean([np.log2(4) - sum(p * np.log2(4 * p) for p in
                          np.bincount(place_of[k[c == j]], minlength=4) / n if p > 0) for j in range(3)])
        print(f"{steps:5d} {s:5.1f}  {np.mean(shape_of[k] == c):10.3f}  {np.mean(size_of[k] == 0):5.3f}"
              f"  {np.mean(size_of[k] == 2):5.3f}  {layout:10.3f}  {np.abs(x).max():10.2f}")

# Why guidance favours large shapes: the implicit classifier p_t(caption | z_t) of a
# noisy latent, for images whose caption is right, by object radius (t = 300 of 1000).
t = 300
a, rng = ab[t], np.random.default_rng(seed)
def log_mix(z, w):
    v = a * v0 + 1 - a
    lg = np.log(w) - ((z[:, None, :] - np.sqrt(a) * M[None]) ** 2).sum(-1) / (2 * v)
    return np.logaddexp.reduce(lg, 1)
for r in range(3):
    ks = np.repeat(np.where((size_of == r))[0], 200)
    z = np.sqrt(a) * M[ks] + np.sqrt(a * v0 + 1 - a) * rng.standard_normal((len(ks), M.shape[1]))
    L = np.stack([log_mix(z, Wc[j][None]) for j in range(3)], 1)    # log p_t(z | caption j)
    post = np.exp(L - np.logaddexp.reduce(L, 1, keepdims=True))
    print(f"radius {radii[r]}: mean p_t(true caption | z_t) = {post[np.arange(len(ks)), shape_of[ks]].mean():.3f}")

Output (seed 0, about 4 s):

seed 0; data: right shape 0.800, small 0.333, large 0.333, layout 2.000 bits
steps     s  right shape  small  large  layout bits  max |pixel|
    5   1.0       0.723  0.498  0.165       1.981        1.13
    5   2.0       0.912  0.437  0.188       1.984        1.17
    5   4.0       0.960  0.333  0.258       1.989        2.36
    5   7.5       0.975  0.245  0.388       1.983        3.50
    5  15.0       0.975  0.167  0.570       1.984        6.11
   10   1.0       0.768  0.427  0.257       1.982        1.18
   10   2.0       0.937  0.357  0.288       1.985        1.18
   10   4.0       0.998  0.273  0.363       1.988        1.18
   10   7.5       1.000  0.167  0.498       1.981        1.18
   10  15.0       0.997  0.087  0.670       1.984        1.69
   25   1.0       0.795  0.382  0.300       1.981        1.24
   25   2.0       0.958  0.322  0.332       1.983        1.24
   25   4.0       1.000  0.247  0.415       1.988        1.24
   25   7.5       1.000  0.157  0.542       1.981        1.24
   25  15.0       1.000  0.072  0.722       1.981        1.24
   50   1.0       0.795  0.368  0.312       1.981        1.28
   50   2.0       0.963  0.313  0.353       1.983        1.28
   50   4.0       1.000  0.235  0.442       1.988        1.28
   50   7.5       1.000  0.152  0.558       1.980        1.28
   50  15.0       1.000  0.072  0.727       1.981        1.28
  200   1.0       0.792  0.352  0.325       1.981        1.31
  200   2.0       0.967  0.312  0.358       1.983        1.31
  200   4.0       1.000  0.230  0.448       1.989        1.31
  200   7.5       1.000  0.150  0.567       1.981        1.31
  200  15.0       1.000  0.070  0.742       1.981        1.31
radius 3: mean p_t(true caption | z_t) = 0.539
radius 5: mean p_t(true caption | z_t) = 0.678
radius 7: mean p_t(true caption | z_t) = 0.753

Without guidance, 50 steps reproduce the noisy captions: 79.5% of samples show the prompted shape, against 80% in the data. A scale of 2 raises this to 96.3%, and from 4 upward nearly every sample obeys the prompt. The price is a narrower distribution, but only along the attribute the caption is ambiguous about. At 50 steps the share of small shapes falls from 0.368 without guidance to 0.152 at $s = 7.5$, the default of the reference script, and to 0.072 at $s = 15$, while the layout entropy stays between 1.980 and 1.989 bits in every cell of the table. The last lines show why: pooled into a few latent cells, small shapes are the hardest to tell apart, so the implicit classifier that guidance amplifies is least sure of them (0.539 against 0.753 for the largest). In our reading, as in the guidance article’s toy, guidance removes the samples the condition explains worst, here small objects, and leaves alone what the condition does not mention.

The step count matters most at its low end. Five unguided steps distort the distribution (49.8% small shapes), and five steps at $s = 15$ overshoot to a pixel magnitude of 6.11, in our reading a toy version of the saturated colours of strongly guided samples; from 25 steps on, the table barely changes. The script takes the seed as its argument; seeds 1 and 2 give the same pattern: at 50 steps, small-shape shares of 0.397 and 0.340 without guidance, 0.175 and 0.165 at $s = 7.5$, and 0.098 and 0.080 at $s = 15$; layout entropies between 1.984 and 1.995 bits over all cells; and maximum magnitudes of 6.00 and 2.98 for five steps at $s = 15$.

Performance

The v1 and v2 model cards plot FID against CLIP score for guidance scales 1.5 to 8 on 10,000 COCO 2017 validation prompts, noting that the models were not optimized for FID. Podell et al. found that such metrics did not reflect SDXL’s improvements; raters choosing among four models preferred SDXL with the refiner 48.44% of the time, SDXL base 36.93%, SD 1.5 7.91% and SD 2.1 6.71% (arXiv v1, §2.5); in 17,153 comparisons on PartiPrompts, the preview model SDXL 0.9 was preferred to Midjourney v5.1 for prompt adherence 54.9% of the time (Fig. 9). Esser et al. report GenEval overall scores of 0.43 for SD 1.5, 0.50 for SD 2.1, 0.55 for SDXL and for SDXL Turbo, 0.67 for DALL·E 3, and 0.74 for their 8-billion-parameter model at $1024^2$ with DPO (Table 5).

Strengths

• Open weights and modest hardware. Users can run, inspect, fine-tune and extend the released versions locally, from 10 GB of GPU memory for v1.
• Modularity. The frozen autoencoder and text encoders let adapters and fine-tunes replace or condition single parts.
• Breadth. One model family serves text-to-image, image-to-image, inpainting, depth-conditioned generation and upscaling.

Limitations

• Failure modes stated by the developers. The v1 and v2 cards list imperfect photorealism, illegible text, poor compositionality (their example is a red cube on top of a blue sphere), badly generated faces and people, weaker results for non-English prompts, and a lossy autoencoder. Podell et al. add, for SDXL, malformed hands and “concept bleeding”, in which attributes of one object leak onto another (arXiv v1, Appendix B).
• Bias. The v1 card says white and Western cultures are often the default because the captions are mostly English, and the v2 card warns that the model amplifies biases so much that viewers should be cautious whatever the prompt. Bianchi et al. (2023) prompted SD v1.4 with “A photo of the face of [occupation]”: 99% of software-developer images were classified as white by a CLIP-based method, against 56% of U.S. software developers (arXiv v2, the pre-publication version).
• Memorization. Carlini et al. (2023) extracted near-copies of training images from SD v1.4 by sampling the most duplicated training captions; the image generation article gives the counts. SD 3’s training data were deduplicated for this reason (Esser et al., Appendix D).
• Unsafe training data. The v1 card notes that LAION-5B contains adult material and describes no NSFW filtering, which v2 added; the safety checker acts only after generation.
• Legal disputes (as of October 2026). In Getty Images v Stability AI, Getty abandoned its training claim, acknowledging no evidence that training took place in the UK, and its outputs claim after Stability blocked the prompts concerned ([2025] EWHC 2863 (Ch), ¶9). On 4 November 2025 the High Court of England and Wales dismissed the secondary copyright claim, holding that a model which does not store or reproduce the works is not an “infringing copy”, and found trade mark infringement in some watermarks generated by v1.x and v2.x models, findings it called “both historic and extremely limited in scope” (¶¶757–758). Permission to appeal the copyright ruling was granted to Getty in December 2025, and Stability was refused permission to appeal the trade mark findings ([2025] EWHC 3343 (Ch)).

Historical Importance

Stable Diffusion put a large text-to-image model’s weights in public hands in 2022. Bianchi et al. described it and DALL·E as the most popular such models, and Podell et al. (2024, §1) list work built on it from image editing to music generation and the reconstruction of images from fMRI scans. Its versions trace the field’s direction: a larger U-Net in SDXL, transformers and rectified flow in SD 3, and few-step distillation in the Turbo models.

Variants

Turbo models. Adversarial diffusion distillation (ADD; Sauer et al., 2024) fine-tunes a pretrained diffusion model into a 1-to-4-step generator. The student, initialized from the pretrained model, combines an adversarial loss, from a discriminator on frozen vision transformer features (DINOv2 ViT-S worked best, Table 1a), with a distillation loss toward a frozen teacher’s denoised outputs, and samples without classifier-free guidance. With SD 1.5 as the base, one step of ADD-M scored zero-shot COCO FID-5k 19.7 and CLIP score 0.326 in 0.09 s, against 20.1 and 0.318 for 25 DPM-Solver steps in 0.88 s (arXiv v1, Table 2). In their user studies, one step of ADD-XL was preferred over four steps of LCM-XL, and four steps of ADD-XL over 50 of SDXL in most comparisons, although they note lower sample diversity (§4.2–4.3). SDXL-Turbo, released on 28 November 2023 (generative-models README), and SD-Turbo are the ADD distillations of SDXL 1.0 and SD 2.1, and SD 3.5 Large Turbo is an ADD-distilled model for 4-step sampling (model cards).

Fine-tuning. DreamBooth (Ruiz et al., 2023) fine-tunes all layers of the model on typically 3 to 5 photos of a subject bound to a rare-token identifier, with a prior-preservation loss on generated images of the subject’s class; about 1,000 iterations took around 5 minutes on one A100 for Stable Diffusion. LoRA (Hu et al., 2022), proposed for language models, freezes the pretrained weights and trains small low-rank update matrices. Textual inversion (Gal et al., 2023) leaves the model frozen and learns one new word embedding from 3 to 5 images; the paper used the LAION-400M latent diffusion model, which has a BERT text encoder. As of October 2026, the diffusers library ships example training scripts that adapt DreamBooth, DreamBooth with LoRA, and textual inversion to Stable Diffusion.

ControlNet (Zhang, Rao & Agrawala, 2023) adds spatial conditions such as edges, depth, segmentation or human pose. It locks the pretrained model, trains a copy of its U-Net encoder on the condition, and joins the two through zero-initialized convolutions, so that training starts from the unchanged model; the paper’s figure uses SD 1.5, whose U-Net it says matches 2.1’s, and reports robust training on datasets from under 50,000 to over a million images. As of October 2026, Stability AI publishes ControlNets for SD 3.5 Large.

Implementations

• Reference code. The CompVis repository (v1); Stability AI’s generative-models repository (SDXL, Turbo; training and inference); and the sd3.5 repository, a minimal inference implementation for SD 3 and 3.5, under the MIT licence since 24 October 2024. As of October 2026, Stability AI’s v2 repository and Hugging Face pages and Runway’s v1.5 repository and pages have been removed; archived copies are cited here, and Hugging Face mirrors of v1.5 state that they are not affiliated with Runway.
• Diffusers. Hugging Face’s library provides pipelines, interchangeable schedulers and model components for every version, with the guidance scale in the $s$ convention.
• Licences, as of October 2026. The v1 weights carry the CreativeML OpenRAIL-M licence, which permits redistribution and commercial use but imposes use-based restrictions that must be passed on to downstream users (v1.4 card). SD 2.x and SDXL 1.0 use the CreativeML Open RAIL++-M licence; SDXL 0.9 was released under a research licence. SD 3 Medium and SD 3.5 use the Stability AI Community License, free for research, non-commercial use, and commercial use by organizations or individuals with less than US$1 million in annual revenue, with an enterprise licence above that. The licence files of SDXL-Turbo and SD-Turbo are also the Community License (last updated 5 July 2024), although SDXL-Turbo’s card metadata names a non-commercial community licence. Read the licence of the exact checkpoint before use.