Inception Score

The Inception Score (IS) rates a generative model from its samples alone. Each generated image is passed through Inception, an ImageNet classifier, and the score is high when every image is classified confidently and the predicted classes are varied across images. Salimans et al. (2016) proposed it to evaluate GANs, which have no likelihood to report. Borji (2019) calls it perhaps the most widely adopted score for GAN evaluation; Heusel et al. (2017) proposed the Fréchet Inception Distance (FID) to improve on it. It never looks at real images: it measures how recognizable and how varied the samples are to one particular classifier, not how close they are to the training data.

Formula

Let $p(y \mid x)$ be the classifier’s softmax output over $K$ classes for a generated image $x$, and $p(y) = \mathbb{E}_{x \sim p_g}\, p(y \mid x)$ its average over the generator’s distribution $p_g$. Then

$$\mathrm{IS} = \exp\Big(\mathbb{E}_{x \sim p_g}\, \mathrm{KL}\big(p(y \mid x) \,\Vert\, p(y)\big)\Big),$$

where $\mathrm{KL}$ is the Kullback–Leibler divergence. Salimans et al. (§4) exponentiate so that values are easier to compare. In practice both expectations are replaced by averages over the same $N$ generated images $x_1, \dots, x_N$:

$$\hat p(y) = \frac{1}{N} \sum_{i=1}^{N} p(y \mid x_i), \qquad \widehat{\mathrm{IS}} = \exp\Big(\frac{1}{N} \sum_{i=1}^{N} \sum_{y} p(y \mid x_i) \log \frac{p(y \mid x_i)}{\hat p(y)}\Big).$$

Mutual information form. Writing the joint distribution of an image and its predicted label as $p(x, y) = p_g(x)\, p(y \mid x)$, the expected KL divergence is the mutual information between $x$ and $y$, so

$$\log \mathrm{IS} = I(x; y) = H(y) - \mathbb{E}_{x}\, H(y \mid x),$$

where $H(y)$ is the entropy of the marginal and $H(y \mid x)$ the entropy of one image’s prediction. Barratt and Sharma (2018, Appendix A) give the derivation: expanding the KL divergence gives $\sum_{x, y} p(x, y) \log \frac{p(x, y)}{p(x)\, p(y)}$, the definition of $I(x; y)$. Splitting the logarithm into $-\log p(y)$ and $\log p(y \mid x)$ gives the entropy form, which is how Borji (2019) writes the score.

Range. Mutual information is nonnegative and at most $H(y) \le \log K$, so $1 \le \mathrm{IS} \le K$. Barratt and Sharma state the bound as 1,000 for the ImageNet classes; in our reading the bound is the number of classifier outputs, which for the network used by the original code is 1,008 (see When to Use It). The minimum 1 is reached when every image gets the same prediction, the maximum $K$ when every prediction is one-hot and the classes are used equally often.

Intuition

The two entropy terms encode the two properties Salimans et al. asked for. Images that contain meaningful objects should give a peaked label distribution, so $\mathbb{E}\, H(y \mid x)$ should be small; a generator should produce varied images, so the marginal should be spread out and $H(y)$ large. The score is high only when both hold: confident predictions that disagree with each other from image to image.

Each failure lowers a different term. A generator that collapses to one class keeps $H(y \mid x)$ small but drives $H(y)$ toward 0. A generator of blurry or ambiguous images keeps $H(y)$ large but raises $H(y \mid x)$. Diversity enters only through the class labels: two images that the classifier assigns to the same class with the same confidence contribute identically, however different they look.

Interpretation

Higher is better, and values are comparable only under the same classifier weights, image preprocessing, sample size and split protocol. Salimans et al. (Table 3) report, for CIFAR-10 and 50,000 images each, $11.24 \pm 0.12$ for real data, $8.09 \pm 0.07$ for their best GAN, and scores from 7.54 down to 3.87 for ablations that remove or replace parts of their method. The real data scored highest, and models whose samples even partially collapsed scored relatively low. Their human study supports the score: Mechanical Turk annotators separated real from generated CIFAR-10 images with 78.7% accuracy, and only 71.4% when the generated images were restricted to the top 1% by Inception Score (§6.2).

The real data’s score is a reference, not a ceiling. Barratt and Sharma (Appendix, arXiv v2) pushed random images toward one class at a time with signed gradient steps of the classifier’s output, cycling through the classes, and reached a score of 986.10 starting from images of uniform random pixels. Starting the same procedure from samples of a Wasserstein GAN trained on CIFAR-10 gave 900.10 (900.15 in their Figure 1 caption), against about 11 for real CIFAR-10 images. On ImageNet the typical magnitude is different again: with 10 splits of 5,000, the 50,000 ImageNet validation images scored about 63 to 66 depending on the network port, with a split-to-split standard deviation of about 8 (Barratt & Sharma, Table 1). For class-conditional ImageNet models, which reach values in the hundreds, lowering BigGAN’s truncation threshold trades variety for confidence and raises the score directly, as the GAN article describes.

When to Use It

Report IS mainly for comparison with older work on ImageNet or CIFAR-10, alongside FID, and only under the conventions those papers used:

• The network. Salimans et al. used the pretrained Inception model that TensorFlow distributed as inception-2015-12-05, citing the Inception-v3 paper (Szegedy et al., 2016). Its softmax has 1,008 outputs rather than 1,000, which Barratt and Sharma flag as an error in the commonly used code. Their Table 1 calls the TensorFlow network “Inception V2”; in our reading it is this graph, since its CIFAR-10 score (11.237) matches the port described below.
• The protocol. The paper asks for a large sample, 50,000 images, because part of the score measures diversity. The released code (OpenAI’s improved-gan, inception_score/model.py) takes images with values from 0 to 255, splits the predictions into 10 consecutive parts, computes the marginal and the score within each part, and reports the mean and standard deviation of the 10 scores. With 50,000 images each split holds 5,000. The code forms the logits by multiplying the pool-3 features by the softmax layer’s weight matrix, without a bias term.
• A maintained port. As of October 2026, torch-fidelity (version 0.4.0) loads weights converted from inception-2015-12-05, computes the score from the 1,008 logits without the bias (its logits_unbiased layer), shuffles the images with a fixed seed and uses 10 splits by default. Its README gives $11.23678 \pm 0.0951$ for the CIFAR-10 training set and states that its values match the reference implementations to floating-point precision.

The split standard deviation measures the spread across splits of one sample set, not the uncertainty of a comparison between models; statistical significance in evaluation covers that question.

Failure Cases

It ignores the real data. Nothing in the formula refers to the training distribution. Lucic et al. (2018) note that a model producing one image per class can score a perfect IS while FID would be poor, and that a “memory GAN” that stores the training set scores perfectly on both. Borji (2019) adds that mixing in natural images from an entirely different distribution could deceive the score. This is the gap FID was designed to close: Heusel et al. (2017) compare generated with real feature statistics.

It misses collapse within a class. Diversity is measured only across classes, so a generator that emits one image per class, or a few, loses nothing. Borji gives the example of a class-conditional model that memorizes one example per ImageNet class and achieves high IS values. The example below shows a within-class collapse scoring above the real distribution.

It assumes ImageNet classes. On other datasets the two assumptions behind the score weaken. Barratt and Sharma found that the most frequent predicted classes for CIFAR-10 images include moving van, sorrel, container ship and milk can (Table 3), and that the average entropy of $p(y \mid x)$ is 4.664 bits for CIFAR-10 training images, 6.512 bits for images of uniform random pixels and 1.97 bits for ImageNet validation images. CIFAR-10 predictions are thus closer to those for noise than to those for ImageNet. They recommend the original score only for ImageNet generators, with classifiers trained on the target data otherwise. For faces, bedrooms or medical images the ImageNet labels carry little meaning, and the score says little about quality.

It depends on the implementation. In Barratt and Sharma’s Table 1, the same 50,000 CIFAR-10 training images score 11.237 with the TensorFlow graph, 9.737 with the PyTorch port of Inception-v3 and 10.852 with the Keras port, although the three networks reach similar ImageNet top-1 accuracy (0.756, 0.772 and 0.777). Their text describes the 11.5% Keras–PyTorch gap as on CIFAR “validation images”, while the table caption says training images. Gaps of this size exceed many of the differences between published models.

It is biased by the sample size. The marginal $\hat p(y)$ is estimated from the same images that are scored. Chong and Forsyth (2020, §3.3) show that the estimate of $\log \mathrm{IS}$ is biased low, by an amount that falls as $1/N$ and depends on the generator, so scores computed with the same $N$ are still not comparable. Barratt and Sharma’s Table 2 shows the effect of the split count: for one set of 50,000 images scored with the PyTorch Inception-v3, the mean score falls from 9.9147 with 1 split to 9.8669 with 10 and 9.0884 with 200.

It can be optimized without improving images. Salimans et al. caution that the score is only a rough guide for models trained by some independent criterion, since optimizing it directly produces adversarial examples. Barratt and Sharma’s near-maximal scores above are such examples, and they argue that using IS for early stopping, hyperparameter tuning or architecture choice optimizes it indirectly.

Lucic et al. also list its insensitivity to the prior distribution over labels and that it is not a proper distance.

Variants

• Modified Inception Score (m-IS). Gurumurthy et al. (2017, §5.1) reward diversity within a class by comparing images the classifier assigns to the same class: $\exp\big(\mathbb{E}_{x_i}\, \mathbb{E}_{x_j}\, \mathrm{KL}(p(y \mid x_i) \,\Vert\, p(y \mid x_j))\big)$, with $x_j$ ranging over images of $x_i$’s predicted class. They report it per class and averaged over classes.
• Improved Inception Score. Barratt and Sharma (§4.1.2) drop the exponential and compute the marginal over the whole sample, reporting the mean KL divergence, $I(x; y)$ in nats, which no longer depends on how the images are split.
• IS∞. Chong and Forsyth compute the score at several sample sizes, fit a line in $1/N$ and extrapolate to $1/N = 0$, as they do for FID; drawing the generator’s latent vectors from a Sobol sequence lowers the variance of the estimates.
• Scores with the real label distribution. Borji (2019) surveys the Mode Score of Che et al. and the AM score of Zhou et al., which compare the marginal $p(y)$ with the label distribution of the training data; for the AM score the classifier is one trained on the target dataset.

Example

No pretrained network is used, so a template classifier stands in for Inception; it is not Inception and knows nothing of ImageNet. Each $16 \times 16$ image in $[0, 1]$ is shifted so that its intensity centroid is centered, then correlated with 25 mean templates (five shape classes at five sizes, built from 2,500 fixed training images); each class keeps its best correlation, and a softmax turns these into $p(y \mid x)$. The shapes (disk, square, ring, bar, cross) have random position, size, brightness and noise. Four generators produce 10,000 images each: the data distribution itself, a generator collapsed to disks, one whose images are 50/50 blends of two different shapes, and one that memorized a single image per class, the most confidently classified of 50 draws. The code checks the entropy form and, for a 50,000-image sample of the real distribution, varies the number of splits.

import numpy as np

H, K = 16, 5                       # 16x16 images, 5 shape classes
yy, xx = np.mgrid[:H, :H]

def draw(kind, rng, r=None):
    """One 16x16 shape image in [0, 1]: random position, size, brightness, noise."""
    cy, cx = 7.5 + rng.uniform(-2, 2, 2)
    r = rng.uniform(3, 5) if r is None else r
    dy, dx = yy - cy, xx - cx
    if kind == 0:   m = dy**2 + dx**2 <= r**2                                  # disk
    elif kind == 1: m = (abs(dy) <= 0.8 * r) & (abs(dx) <= 0.8 * r)            # square
    elif kind == 2: m = abs(np.hypot(dy, dx) - r) <= 1.0                        # ring
    elif kind == 3: m = (abs(dy) <= 1.2) & (abs(dx) <= r + 2)                   # bar
    else:           m = ((abs(dy) <= 1) | (abs(dx) <= 1)) & (abs(dy) <= r) & (abs(dx) <= r)  # cross
    img = rng.uniform(0.6, 1.0) * m + rng.normal(0, 0.05, (H, H))
    return np.clip(img, 0, 1)

def center(imgs):
    """Shift each image so its intensity centroid is at the middle (integer roll)."""
    out = np.empty_like(imgs)
    for i, im in enumerate(imgs):
        w = im / im.sum()
        cy, cx = (w * yy).sum(), (w * xx).sum()
        out[i] = np.roll(im, (int(round(7.5 - cy)), int(round(7.5 - cx))), axis=(0, 1))
    return out

# ---- STAND-IN classifier (NOT Inception-v3): normalized correlation with
# templates (5 sizes per class), best template per class, then a softmax.
# "Pretrained" once, on 100 images per class and size.
trng = np.random.default_rng(100)
T = np.stack([center(np.stack([draw(k, trng, r) for _ in range(100)])).mean(0)
              for k in range(K) for r in np.linspace(3.2, 4.8, 5)]).reshape(K * 5, -1)
T -= T.mean(1, keepdims=True); T /= np.linalg.norm(T, axis=1, keepdims=True)

def p_y_given_x(imgs, temp=0.02):
    f = center(imgs).reshape(len(imgs), -1)
    f = f - f.mean(1, keepdims=True); f /= np.linalg.norm(f, axis=1, keepdims=True)
    z = (f @ T.T).reshape(len(f), K, 5).max(2) / temp  # logits
    z -= z.max(1, keepdims=True)                        # stable softmax
    p = np.exp(z); return p / p.sum(1, keepdims=True)

# ---- Inception Score from a matrix of p(y|x), one row per image ---------------
def log_is(p):
    py = p.mean(0)                                      # marginal p(y), same samples
    return np.mean(np.sum(p * (np.log(p + 1e-12) - np.log(py + 1e-12)), axis=1))

def is_splits(p, splits=10):                            # protocol of the released code
    s = [np.exp(log_is(c)) for c in np.array_split(p, splits)]
    return np.mean(s), np.std(s)

def entropy(q):
    return -np.sum(q * np.log(q + 1e-12), axis=-1)

# ---- Generators ---------------------------------------------------------------
def gen_real(n, rng):            # the data distribution itself, all classes
    return np.stack([draw(k, rng) for k in rng.integers(0, K, n)])
def gen_one_class(n, rng):       # mode collapse to disks
    return np.stack([draw(0, rng) for _ in range(n)])
def gen_ambiguous(n, rng):       # every image is a 50/50 blend of two different shapes
    a = rng.integers(0, K, n); b = (a + rng.integers(1, K, n)) % K
    return np.stack([0.5 * draw(i, rng) + 0.5 * draw(j, rng) for i, j in zip(a, b)])
def gen_one_per_class(n, rng):   # collapse WITHIN classes: one memorized image per class,
    brng = np.random.default_rng(7)  # the most confidently classified of 50 draws
    bank = []
    for k in range(K):
        c = np.stack([draw(k, brng) for _ in range(50)])
        bank.append(c[p_y_given_x(c)[:, k].argmax()])
    return np.stack(bank)[rng.integers(0, K, n)]

gens = [("real distribution", gen_real), ("one class only", gen_one_class),
        ("ambiguous blends", gen_ambiguous), ("one image per class", gen_one_per_class)]

N = 10_000
print(f"{'generator':22s} {'IS (10 splits)':>15s} {'IS pooled':>9s} {'H(y)':>6s} "
      f"{'E H(y|x)':>8s} {'pooled, seeds 0-4':>18s}")
for name, g in gens:
    res = []
    for seed in range(5):
        p = p_y_given_x(g(N, np.random.default_rng(seed)))
        res.append(np.exp(log_is(p)))
        if seed == 0:
            m, s = is_splits(p)
            hy, hyx = entropy(p.mean(0)), entropy(p).mean()
            assert np.isclose(log_is(p), hy - hyx)       # log IS = H(y) - E H(y|x)
            line = f"{name:22s} {m:8.3f} ± {s:.3f} {np.exp(log_is(p)):9.3f} {hy:6.3f} {hyx:8.3f}"
    print(line + f"   {min(res):.3f} to {max(res):.3f}")
print(f"upper bound K = {K}; real-data accuracy of the stand-in classifier: ", end="")
rng = np.random.default_rng(1)
kinds = rng.integers(0, K, 5000)
acc = np.mean(p_y_given_x(np.stack([draw(k, rng) for k in kinds])).argmax(1) == kinds)
print(f"{acc:.3f}")

# ---- Split size: the per-split estimate is biased low for small splits ---------
p = p_y_given_x(gen_real(50_000, np.random.default_rng(2)))
print("\nreal distribution, 50,000 images, IS averaged over splits:")
for splits in [1, 10, 100, 500, 1000]:
    m, s = is_splits(p, splits)
    print(f"  {splits:5d} splits of {50_000 // splits:6d}:  {m:.4f} ± {s:.4f}")

Output (NumPy 2.3.5, about 7 seconds):

generator               IS (10 splits) IS pooled   H(y) E H(y|x)  pooled, seeds 0-4
real distribution         4.167 ± 0.049     4.176  1.609    0.180   4.147 to 4.180
one class only            1.066 ± 0.003     1.066  0.448    0.383   1.065 to 1.067
ambiguous blends          2.226 ± 0.044     2.228  1.311    0.510   2.200 to 2.229
one image per class       4.833 ± 0.008     4.843  1.609    0.031   4.838 to 4.843
upper bound K = 5; real-data accuracy of the stand-in classifier: 0.986

real distribution, 50,000 images, IS averaged over splits:
      1 splits of  50000:  4.1717 ± 0.0000
     10 splits of   5000:  4.1705 ± 0.0103
    100 splits of    500:  4.1565 ± 0.0528
    500 splits of    100:  4.0975 ± 0.1207
   1000 splits of     50:  4.0247 ± 0.1773

The entropy columns show why each generator scores as it does. The real distribution uses all five classes ($H(y)$ is $\log 5 \approx 1.609$ nats), but some images are classified with doubt, so it stays below the maximum of 5. Collapsing to disks leaves only the marginal entropy that comes from the probability the classifier gives to squares for many disks; that doubt is also inside the individual predictions, so $H(y)$ and $\mathbb{E}\, H(y \mid x)$ nearly cancel and the score is barely above 1. The blends spread over the classes but each prediction is split, and the score loses about half its value. The generator with one image per class has no diversity within any class, yet it scores highest, above the data distribution itself, because its five memorized images are classified with near certainty, the case Lucic et al. and Borji describe. Over data seeds 0 to 4 every generator’s pooled score stays within 0.03 of the seed-0 value, so the ordering is not a sampling accident.

The split table reproduces the pattern of Barratt and Sharma’s Table 2 and Chong and Forsyth’s bias analysis: with 10 splits of 5,000 the mean is close to the pooled score, but the smaller the splits, the lower the mean and the larger the spread. With only five classes, 50 images per split already cost 3.5%; with 1,000 ImageNet classes, a split of 5,000 is a much smaller sample of the marginal.

Related Metrics

• Fréchet Inception Distance (Heusel et al., 2017) compares Gaussians fitted to Inception-v3 pool features of real and generated images, so it registers a mismatch with the training data, which IS cannot. Lucic et al. (2018) note that, unlike IS, it detects intra-class mode dropping. Its article also covers the Kernel Inception Distance, and precision and recall for generative models, which separate fidelity from coverage.