Conditional GAN

A conditional GAN is a generative adversarial network in which both networks receive a condition $y$ alongside their usual inputs: the generator turns noise $z$ and $y$ into an image, and the discriminator judges whether an image is a real example of $y$. The condition can be a class label, a sentence embedding or a whole image, and the model then samples from the conditional distribution $p(x \mid y)$ instead of $p(x)$. Class-conditional GANs (often shortened to cGANs) were the standard GAN setting for ImageNet, and image-conditional GANs underlie paired image-to-image translation.

The architecture question is where the condition enters each network. The generator can receive $y$ as an extra input or through its normalization layers; the discriminator can receive it as an input, predict it with an auxiliary classifier, or combine it with its features by an inner product. These choices behave differently, and the auxiliary-classifier design in particular changes which distribution the generator learns.

Core Idea

An unconditional GAN matches the generator’s distribution $q(x)$ to the data distribution $p(x)$. A conditional GAN matches joint distributions instead: labels $y$ are drawn from the data’s label distribution, the generator draws $x \sim q(x \mid y)$, and the discriminator compares pairs $(x, y)$. Gong et al. (2019, Eq. 2) write the generic objective as

$$\min_G \max_D \; \mathbb{E}_{(x, y) \sim p_{xy}}\big[\log D(x, y)\big] + \mathbb{E}_{z \sim p_z,\, y \sim p_y}\big[\log\big(1 - D(G(z, y), y)\big)\big],$$

which, as they note, implicitly minimizes the Jensen–Shannon divergence between the joint distributions $p(x, y)$ and $q(x, y) = q(x \mid y)\,p(y)$. Because $p(y)$ is shared, matching the joints means matching every class-conditional distribution.

Odena, Olah and Shlens (2017, §2) describe the approach of Mirza and Osindero (2014) as supplying both the generator and the discriminator with class labels. The discriminator’s role is the key difference from an unconditional GAN. If it ignores $y$, it can only check that images are realistic, not that they belong to the requested class. In pix2pix, an unconditional discriminator let the generator produce nearly the same output whatever the input (Isola et al., 2017, §4.2).

Building Blocks

Conditioning the generator

• Input concatenation. The one-hot label, or a learned embedding of it, is concatenated with $z$. The AC-GAN generators take a 110-dimensional input (appendix, Tables 1 and 2), in our reading a 100-dimensional noise vector and a one-hot label over the 10 classes each model was trained on.
• Conditional batch normalization. Each batch normalization layer normalizes as usual but takes its per-channel scale $\gamma$ and shift $\beta$ from the condition, so the label modulates every layer rather than only the first. The projection-discriminator code stores one $(\gamma, \beta)$ vector per class in an embedding table, initialized to $\gamma = 1$, $\beta = 0$, inside each residual block of the generator (Miyato & Koyama, official code). Self-Attention GAN used conditional batch normalization in the generator and projection in the discriminator, as the projection work had (Zhang et al., 2019, §5).

Conditioning the discriminator

• Concatenation. The label is concatenated with the image at the input or, as an embedding broadcast over the spatial grid, with an intermediate feature map. Gong et al. (2019, §1) note that most methods feed the condition this way. The official projection code includes a concatenation baseline that appends a 128-dimensional class embedding to the feature map after the third of six residual blocks.
• Auxiliary classifier (AC-GAN). The discriminator receives only the image and has two heads: one outputs the probability that the image is real, the other a distribution over classes. The class is not an input but a target (Odena et al., 2017, §3).
• Projection. The discriminator computes features $\phi(x)$, a scalar $\psi(\phi(x))$, and adds the inner product of $\phi(x)$ with a learned embedding $e_y$ of the label. In the official code the output is a linear layer of the globally pooled features plus the sum of the elementwise product of the class embedding and the same features.

How Data Flows

Each training step draws real pairs $(x, y)$ from the dataset and labels $y'$ for the fake images, usually from the label distribution of the data, and generates $G(z, y')$. The discriminator scores real pairs and fake pairs; for AC-GAN it also classifies both. The generator is then updated through the discriminator with fresh noise and labels. At sampling time the user chooses $y$, and samples from different classes can share the same $z$. Odena et al. (2017, §4.5, Fig. 9) fixed $z$ across eight bird classes and found that layout, position and background were preserved while the class changed, since AC-GAN learns a representation of $z$ that is independent of the label.

Mathematical Formulation

AC-GAN

With source $S \in \{\text{real}, \text{fake}\}$ and class $C$, Odena et al. (2017, Eqs. 2–3) define the log-likelihood of the correct source and of the correct class,

$$L_S = \mathbb{E}\big[\log P(S = \text{real} \mid X_{\text{real}})\big] + \mathbb{E}\big[\log P(S = \text{fake} \mid X_{\text{fake}})\big],$$ $$L_C = \mathbb{E}\big[\log P(C = c \mid X_{\text{real}})\big] + \mathbb{E}\big[\log P(C = c \mid X_{\text{fake}})\big].$$

The discriminator maximizes $L_S + L_C$ and the generator maximizes $L_C - L_S$: it tries to fool the source head and to be classified as the class it was asked for. Gong et al. (2019, Eq. 3) write the same objective with a weight $\lambda_c$ on the two classification terms.

Why the auxiliary classifier biases the generator

The fake-image classification term is a cross-entropy, $-\mathbb{E}_{(x, y) \sim q}[\log C(y \mid x)]$. Gong et al. (2019, §2.2) add and subtract the generator’s own conditional entropy $H_q(y \mid x)$:

$$-\mathbb{E}_{q}\big[\log C(y \mid x)\big] = \mathrm{KL}\big(q(y \mid x) \,\Vert\, C(y \mid x)\big) + H_q(y \mid x).$$

When the classifier is updated, $H_q$ is a constant and the term trains $C$ properly. When the generator is updated, $H_q$ depends on it, so the generator lowers the cross-entropy partly by lowering $H_q(y \mid x)$, that is, by making the class easy to read from the image. Their Theorem 1 states the consequence: even when the marginal $q(x)$ equals $p(x)$, the generator that minimizes this term makes $q(y = k \mid x) = 1$ for the class the classifier ranks highest at $x$ and 0 otherwise. Where real classes overlap, as the radius classes in the example below do, the generator confines each class to the region the classifier assigns to it and stops producing images near the decision boundaries. Their remedy, TAC-GAN, adds a second classifier trained only on generated images, against which the generator plays adversarially, to estimate the missing entropy term.

Projection discriminator

The projection form can be derived from the optimal discriminator; the derivation below is our own, consistent with Gong et al.’s remark that the projection cGAN decomposes the density ratio (§3). For a joint GAN the optimal logit is $\log p(x, y) - \log q(x, y)$, which splits into a marginal and a conditional part:

$$\log \frac{p(x, y)}{q(x, y)} = \log \frac{p(x)}{q(x)} + \log \frac{p(y \mid x)}{q(y \mid x)}.$$

If both class posteriors are softmax classifiers on shared features, $p(y \mid x) \propto \exp(v_y^{p\top} \phi(x))$ and $q(y \mid x) \propto \exp(v_y^{q\top} \phi(x))$, the second term is $(v_y^p - v_y^q)^\top \phi(x)$ plus the difference of the two log-normalizers, which depends on $x$ but not on $y$. Collecting every $y$-independent part into $\psi(\phi(x))$ gives

$$D(x, y) = e_y^\top \phi(x) + \psi\big(\phi(x)\big), \qquad e_y = v_y^p - v_y^q,$$

which is the output the official code computes. Unlike AC-GAN, the discriminator never has to classify; the label enters as a learned direction in feature space whose agreement with the image raises or lowers the score.

Conditional batch normalization

Batch normalization maps a channel $c$ of features $F$ to $\gamma_c (F_c - \mathbb{E}_B[F_c]) / \sqrt{\mathrm{Var}_B[F_c] + \epsilon} + \beta_c$ (de Vries et al., 2017, Eq. 2). De Vries et al. predict changes to the two parameters from a language embedding $e_q$ with a one-hidden-layer MLP and add them to the pretrained values:

$$\hat\gamma_c = \gamma_c + \Delta\gamma_c(e_q), \qquad \hat\beta_c = \beta_c + \Delta\beta_c(e_q)$$

(Eqs. 4–5), so that training starts from the unmodified network. They describe the method as inspired by style-transfer work of Dumoulin, Shlens and Kudlur (2017), and note that with two parameters per feature map the batch-normalization parameters make up less than 1% of a pretrained ResNet’s parameters. For a class label, the MLP becomes a table with one $(\gamma, \beta)$ pair per class, as in the projection code. The same operation, a per-channel scale and shift computed from a condition, is called FiLM in multimodal fusion, and normalization layers covers the instance-normalization variants used in style transfer and StyleGAN.

Why It Works

A label splits a hard problem into easier ones. Odena et al. (2017, §4.6) found that training a fixed AC-GAN on more ImageNet classes harmed its samples, the failure being generator collapse, and trained 100 models on 10 classes each; they could not train an unconditional GAN to converge reliably even on one class. In our reading, conditioning gives the discriminator a sharper criterion for each image (is this a convincing goldfish?) and gives the generator a coordinate that separates modes it would otherwise have to discover from $z$ alone.

The three discriminators differ in what they ask for. In our reading, concatenation and projection estimate the joint density ratio and so, at the optimum, recover $p(x \mid y)$ with its overlaps. The auxiliary classifier adds a classification objective that a generator can satisfy better than the data does, by producing images more typical of their class than real ones. TAC-GAN’s authors also note that the theoretical justification of projection holds for the cross-entropy loss and not for the hinge loss, which is often preferred in practice for real data (Gong et al., 2019, §3).

Training Considerations

• Losses and stabilizers. Large class-conditional GANs pair conditioning with the stabilizers of unconditional ones. Self-Attention GAN trained with the hinge loss (Eq. 4), spectral normalization in both networks, Adam with $\beta_1 = 0$, $\beta_2 = 0.9$, and learning rates 0.0004 for the discriminator and 0.0001 for the generator (§5). The projection code’s ImageNet configuration uses the hinge loss, five discriminator steps per generator step and Adam with learning rate 0.0002, $\beta_1 = 0$, $\beta_2 = 0.9$. The Wasserstein GAN and gradient penalties apply unchanged.
• AC-GAN with many classes. AC-GAN’s training becomes unstable as the number of classes grows (Kang et al., 2021, §2), and Odena et al. split ImageNet across 100 models for that reason. Kang et al. (2021, §3.1) traced an early-training collapse to the softmax classifier’s gradient, whose norm grows with the norm of the discriminator’s features while the classifier is still wrong. Their remedy normalizes features and class weights onto a unit hypersphere.
• Class weight. TAC-GAN’s CIFAR-100 experiments found AC-GAN sensitive to the classification weight $\lambda_c$, needing a very small value to score well, and still producing images of low within-class diversity (Gong et al., 2019, §4.3, Fig. 6).
• Measuring diversity within a class. Self-Attention GAN’s authors note that the Inception Score is meant to check that samples are confidently recognized as one class, not to assess diversity within a class (Zhang et al., 2019, §5). Odena et al. (2017, §4.2) measured within-class diversity as the mean multi-scale SSIM of 100 random sample pairs from the same class (lower means more diverse), and Self-Attention GAN reported intra-FID, the FID between generated and real images within each class, which it credits to Miyato and Koyama (§5).

Example: AC-GAN and Projection on Overlapping Classes

The data are $16 \times 16$ images of a filled disk, antialiased by $4 \times 4$ supersampling, with values in $[-1, 1]$ and no added noise. The class sets the radius: $r \sim \mathcal{N}(\mu_k, 0.5^2)$ with $\mu_k = 3, 4, 5$ pixels, clipped to $[1.5, 6.5]$, so neighbouring classes overlap and a disk of radius 3.5 could belong to class 0 or class 1. The disk’s centre is shifted by up to one pixel in each direction, independently of the class. Because the generating process is known, every sample can be measured: its radius is recovered from the disk’s area, $\hat r = \sqrt{A / \pi}$ with $A = \sum (x + 1)/2$, and its horizontal centre from the intensity centroid. A sample is valid if it differs from a clean disk re-rendered at $(\hat r, \text{centroid})$ by an RMS error below 0.2. On valid samples of each intended class the script reports:

• Bayes accuracy, the fraction whose radius the Bayes-optimal classifier (equal priors, the true Gaussians) assigns to the intended class. Real data score below 1 because the classes overlap; a faithful generator should match the real value, not exceed it.
• Radius and centre standard deviations within the class, against 0.5 and about 0.58 for real data.
• $W_1$, the one-dimensional Wasserstein-1 distance between generated and real radii of the class, estimated from 100 quantiles and averaged over the classes.

The example compares an AC-GAN discriminator with a projection discriminator. Both share the same generator, a small fully connected network that receives $[z, \text{one-hot}(y)]$, and the same discriminator trunk $\phi$; they differ only in how the label enters the discriminator. Both use the cross-entropy loss, the loss for which Gong et al. state their analysis, with the non-saturating form for the generator. The “real data” row compares a second set of real images with the reference set, so its $W_1$ is the sampling floor (PyTorch 2.14.1, CPU, one thread, about 2 minutes for the three seeds; two runs give identical output).

import math, sys
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.set_num_threads(1)

# Data: 16x16 antialiased disks in [-1, 1]. The class sets the radius distribution,
# N(mu_k, 0.5^2) with mu = 3, 4, 5 pixels, so neighbouring classes overlap.
# The centre is jittered by up to 1 pixel, independently of the class.
MU, SIG, K, S = torch.tensor([3.0, 4.0, 5.0]), 0.5, 3, 16
sub = (torch.arange(4 * S) + 0.5) / 4                      # 4x4 supersampling grid
YY, XX = torch.meshgrid(sub, sub, indexing="ij")

def render(r, cy, cx):
    inside = ((YY - cy[:, None, None]) ** 2 + (XX - cx[:, None, None]) ** 2) <= r[:, None, None] ** 2
    return 2 * inside.float().reshape(-1, S, 4, S, 4).mean((2, 4)) - 1

def sample_real(n, g):
    y = torch.randint(0, K, (n,), generator=g)
    r = (MU[y] + SIG * torch.randn(n, generator=g)).clamp(1.5, 6.5)
    c = 7.5 + 2 * torch.rand(n, 2, generator=g) - 1
    return render(r, c[:, 0], c[:, 1]).reshape(n, -1), y

pix = torch.arange(S) + 0.5
def measure(x):  # radius from the disk's area, centre from the intensity centroid
    m = (x.reshape(-1, S, S) + 1) / 2
    A = m.sum((1, 2)).clamp_min(1e-6)
    cy, cx = (m.sum(2) * pix).sum(1) / A, (m.sum(1) * pix).sum(1) / A
    r = (A / math.pi).sqrt()
    valid = ((x - render(r, cy, cx).reshape(len(x), -1)) ** 2).mean(1).sqrt() < 0.2
    return r, cx, valid

def mlp(n_in, n_out):
    return nn.Sequential(nn.Linear(n_in, 128), nn.LeakyReLU(0.2), nn.Linear(128, 128),
                         nn.LeakyReLU(0.2), nn.Linear(128, n_out))

class Disc(nn.Module):
    def __init__(self, kind):
        super().__init__()
        self.kind = kind
        self.phi = nn.Sequential(*list(mlp(256, 1))[:-1])      # trunk -> 128 features
        self.src = nn.Linear(128, 1)                       # real/fake logit
        if kind == "AC-GAN":
            self.cls = nn.Linear(128, K)                   # auxiliary classifier
        if kind == "projection":
            self.embed = nn.Embedding(K, 128)              # class embedding e_y
            nn.init.xavier_uniform_(self.embed.weight)     # Glorot, as in the official code

    def forward(self, x, y):
        h = self.phi(x)
        if self.kind == "AC-GAN":
            return self.src(h).squeeze(1), self.cls(h)
        return self.src(h).squeeze(1) + (self.embed(y) * h).sum(1), None   # projection

def train(kind, seed, steps=4000, batch=128, zdim=16):
    torch.manual_seed(seed)
    g = torch.Generator().manual_seed(seed + 1000)
    G, D = nn.Sequential(mlp(zdim + K, 256), nn.Tanh()), Disc(kind)   # G gets [z, one-hot(y)]
    gen = lambda z, y: G(torch.cat([z, F.one_hot(y, K).float()], 1))
    opt_g = torch.optim.Adam(G.parameters(), lr=2e-4, betas=(0.5, 0.999))
    opt_d = torch.optim.Adam(D.parameters(), lr=2e-4, betas=(0.5, 0.999))
    for step in range(steps):
        x, y = sample_real(batch, g)
        yf = torch.randint(0, K, (batch,), generator=g)
        fake = gen(torch.randn(batch, zdim, generator=g), yf).detach()
        (sr, cr), (sf, cf) = D(x, y), D(fake, yf)
        loss_d = F.softplus(-sr).mean() + F.softplus(sf).mean()   # -log D(x) - log(1 - D(G(z)))
        if kind == "AC-GAN":                                      # classify real and fake images
            loss_d = loss_d + F.cross_entropy(cr, y) + F.cross_entropy(cf, yf)
        opt_d.zero_grad(); loss_d.backward(); opt_d.step()

        yf = torch.randint(0, K, (batch,), generator=g)
        sf, cf = D(gen(torch.randn(batch, zdim, generator=g), yf), yf)
        loss_g = F.softplus(-sf).mean()                           # non-saturating loss
        if kind == "AC-GAN":
            loss_g = loss_g + F.cross_entropy(cf, yf)             # and be classified as yf
        opt_g.zero_grad(); loss_g.backward(); opt_g.step()
    with torch.no_grad():
        y = torch.arange(K).repeat_interleave(2000)
        return measure(gen(torch.randn(len(y), zdim, generator=g), y)) + (y,)

def report(name, r, cx, valid, y, r_ref, y_ref):
    acc, sd_r, sd_c, w1 = [], [], [], []
    for k in range(K):
        rk, ck = r[valid & (y == k)], cx[valid & (y == k)]
        acc.append(((-(rk[:, None] - MU) ** 2).argmax(1) == k).float().mean())  # Bayes-optimal class
        sd_r.append(rk.std()); sd_c.append(ck.std())
        q = torch.linspace(0.005, 0.995, 100)                    # 1-D Wasserstein-1 to real radii
        w1.append((rk.quantile(q) - r_ref[y_ref == k].quantile(q)).abs().mean())
    f = lambda v: " ".join(f"{float(a):.2f}" for a in v)
    print(f"{name:20s} {valid.float().mean():5.0%}   {f(acc)}    {f(sd_r)}   {f(sd_c)}   {sum(w1) / K:.3f}")

if __name__ == "__main__":
    seeds = [int(s) for s in sys.argv[1:]] or [0, 1, 2]
    xr, yr = sample_real(6000, torch.Generator().manual_seed(123))   # reference real images
    rr, _, _ = measure(xr)
    print(f"PyTorch {torch.__version__}")
    print(f"{'':20s} valid  Bayes acc/class  radius std/class  centre-x std/class  W1 radius")
    xh, yh = sample_real(6000, torch.Generator().manual_seed(456))   # held-out real images
    report("real data", *measure(xh), yh, rr, yr)
    for seed in seeds:
        for kind in ["AC-GAN", "projection"]:
            report(f"{kind} (seed {seed})", *train(kind, seed), rr, yr)

Output:

PyTorch 2.14.1
                     valid  Bayes acc/class  radius std/class  centre-x std/class  W1 radius
real data             100%   0.84 0.68 0.85    0.50 0.50 0.48   0.58 0.56 0.58   0.028
AC-GAN (seed 0)        93%   0.98 0.92 0.99    0.37 0.28 0.32   0.53 0.58 0.53   0.179
projection (seed 0)   100%   0.86 0.70 0.87    0.49 0.49 0.48   0.57 0.58 0.56   0.035
AC-GAN (seed 1)        94%   0.99 0.98 0.99    0.36 0.21 0.30   0.54 0.54 0.47   0.211
projection (seed 1)   100%   0.83 0.70 0.83    0.51 0.48 0.48   0.53 0.56 0.54   0.039
AC-GAN (seed 2)        96%   0.97 0.98 0.99    0.34 0.22 0.31   0.54 0.57 0.53   0.194
projection (seed 2)   100%   0.83 0.68 0.83    0.52 0.50 0.49   0.59 0.57 0.57   0.028

Projection reproduces the real class-conditional distributions in all three seeds: Bayes accuracy, radius spread and centre spread all lie within a few hundredths of the real values, and its $W_1$ distances are close to the sampling floor of the real-data row. AC-GAN also produces mostly valid disks (93% to 96%), but each class is narrower than the data. Its radius standard deviation is 0.21 to 0.37 against 0.48 to 0.50, and its samples are classified correctly 92% to 99% of the time, more often than real images of the same class (68% to 85%). The middle class, which overlaps both neighbours, shrinks most. This is the behaviour Gong et al.’s Theorem 1 predicts: the generator lowers its classification loss by keeping each class out of the radius ranges it shares with the others, which the real data do not do. The centre, which carries no class information, keeps most of its spread under AC-GAN (0.47 to 0.58 against 0.56 to 0.58), so the narrowing falls on the factor the classifier reads rather than on diversity in general. These are three seeds of a toy model with $\lambda_c = 1$; the size of the effect at other weights and scales is not measured here.

Variants

Classifier-based

• AC-GAN (Odena, Olah & Shlens, 2017): an auxiliary classifier head on the discriminator, described above.
• TAC-GAN (Gong et al., 2019): a second, “twin” classifier trained on generated images only, which the generator plays against so that the class-conditional distributions are not pushed apart. On a one-dimensional mixture of three overlapping Gaussians, AC-GAN with the cross-entropy loss separated the classes completely, while TAC-GAN and the projection cGAN recovered the true densities; with the hinge loss, only TAC-GAN did (§4.1, Fig. 2).
• ReACGAN (Kang et al., 2021): normalized features and a data-to-data cross-entropy loss, which also compares each image with other images in the minibatch, to make the classifier-based approach stable at ImageNet scale.

Projection-based

Kang et al. (2021, §2) describe projection-based GANs, which followed the projection discriminator of Miyato and Koyama (2018), as the standard for conditional image generation. They (§1) name SNGAN, Self-Attention GAN and BigGAN as its representatives, and in their own comparisons give every model the same generator, with conditional batch normalization, so that only the discriminator’s conditioning differs (§4.2).

Image conditions

When $y$ is an image, the generator is usually an encoder–decoder that maps $y$ to the output, and the discriminator sees the input and output together. pix2pix (Isola et al., 2017) shows the discriminator the input image together with the real or generated output (Eq. 1), adds an $L^1$ loss to the target (Eq. 4), and classifies image patches rather than whole images. On Cityscapes labels to photos, an FCN segmenter recovered the label maps from its outputs with per-pixel accuracy 0.57 for the conditional adversarial loss alone, 0.22 for an unconditional discriminator and 0.66 with the added $L^1$ term (Table 1). SPADE (Park et al., 2019) argues that normalization layers tend to wash away a semantic layout fed only at the network’s input, and instead uses the layout to modulate the activations in the normalization layers through a spatially varying learned transformation. With a class label in place of the layout and the same parameters at every position, the layer reduces to conditional batch normalization (Park et al., 2019, §3). Image-to-image translation describes both methods and their benchmarks.

Use in Computer Vision

Class-conditional generation is the usual protocol for image generation on ImageNet, and the class-conditional ImageNet results below trace the architecture’s development. Image-conditional GANs serve translation tasks such as labels to photos, maps to aerial views and colorization, and adversarial losses appear in super-resolution and restoration networks.

Class-conditional ImageNet results as published

All rows are class-conditional ImageNet and come from the cited tables, each with its own implementation and evaluation; Zhang et al., Gong et al. and Kang et al. train their own models at $128 \times 128$; numbers from different tables are not directly comparable.

Model	Source	IS	FID	Intra-FID
AC-GAN	Zhang et al. (2019), Table 2	28.5	—	260.0
SNGAN-projection	Zhang et al. (2019), Table 2	36.8	27.62	92.4
Self-Attention GAN	Zhang et al. (2019), Table 2	52.52	18.65	83.7
AC-GAN	Gong et al. (2019), Table 1	7.26	184.41	—
TAC-GAN	Gong et al. (2019), Table 1	28.86	23.75	—
Projection cGAN	Gong et al. (2019), Table 1	38.05	22.77	—
AC-GAN (StudioGAN, batch 256)	Kang et al. (2021), Table 2	62.99	26.35	—
BigGAN (StudioGAN, batch 256)	Kang et al. (2021), Table 2	43.97	16.36	—
ReACGAN (batch 256)	Kang et al. (2021), Table 2	68.27	13.98	—
BigGAN (StudioGAN, batch 2,048)	Kang et al. (2021), Table 2	99.71	7.89	—
ReACGAN (batch 2,048)	Kang et al. (2021), Table 2	92.74	8.23	—

Self-Attention GAN’s table notes that the SNGAN-projection FID was computed from the officially released weights. Its row for its own model matches the model with self-attention on the $64 \times 64$ feature maps in its Table 1, where the model with self-attention on the $32 \times 32$ maps has the lower FID, 18.28, with IS 51.43; Kang et al.’s reprint of Self-Attention GAN pairs that 18.28 with IS 52.52. The AC-GAN paper itself reports no ImageNet Inception Score or FID: its main measure is the Inception accuracy, the fraction of samples that a pretrained Inception network assigns to the intended class, which was $10.1\% \pm 2.0\%$ for its $128 \times 128$ models against 81% for real training images (§4.1, §5). Self-Attention GAN’s table attributes its AC-GAN row to Odena et al. but does not say how the numbers were obtained. Gong et al.’s and Kang et al.’s AC-GAN rows show how strongly such numbers depend on the implementation: Gong et al.’s AC-GAN reaches IS 7.26 and FID 184.41, while Kang et al.’s re-implementation, with the same generator conditioning as their other models, reaches IS 62.99 and FID 26.35, and Kang et al. report that models with AC-GAN’s classification loss collapsed early in training on Tiny-ImageNet and CUB200 (§4.3, Table 3). Gong et al.’s “Projection cGAN” column is the row Kang et al. reprint as BigGAN from Gong et al. Gong et al. hypothesize that TAC-GAN’s lower IS than projection’s comes from samples in the overlap regions that do not clearly belong to their class (§4.4), the trade the Inception Score penalizes. BigGAN, a projection-based model (Kang et al., 2021, §1), is the GAN in the comparisons with later class-conditional diffusion models that Image Generation tabulates.

Limitations

• Labels are needed. Class-conditional GANs need a labelled dataset, and image-conditional ones paired data, which CycleGAN-style unpaired methods avoid.
• AC-GAN distorts the conditional distribution where classes overlap, giving discriminable but less diverse samples (Gong et al., 2019, Theorem 1), and trains unstably with many classes.
• Scores reward class-typical samples. In our reading, the Inception Score and Inception accuracy favour exactly the bias AC-GAN introduces, so within-class diversity needs separate measures such as intra-FID or pairwise similarity.
• The usual GAN problems remain. Conditioning reduces but does not remove instability and collapse: in Odena et al.’s ImageNet models, only 84.7% of classes had samples more diverse, by MS-SSIM, than the least diverse class of real training images (§4.2).

Important Models

Model	Year	Conditioning
Conditional GAN	2014	Label fed to both networks (Mirza & Osindero, as Odena et al. describe it)
AC-GAN	2017	Auxiliary classifier in the discriminator; label concatenated to $z$ (Odena, Olah & Shlens)
Conditional batch normalization	2017	Per-channel scale and shift from the condition (de Vries et al., inspired by style-transfer work of Dumoulin et al.)
pix2pix	2017	Image condition concatenated at the discriminator input, $L^1$ loss (Isola et al.)
Projection discriminator	2018	Inner product of a class embedding with discriminator features (Miyato & Koyama, as Gong et al. and the official code describe it)
Self-Attention GAN	2019	Conditional batch normalization and projection with self-attention and spectral normalization (Zhang et al.)
SPADE	2019	Spatially varying normalization parameters from a label map (Park et al.)
TAC-GAN	2019	Twin classifier that removes AC-GAN’s bias (Gong et al.)
ReACGAN	2021	Normalized classifier with a data-to-data cross-entropy loss (Kang et al.)