Datasets & Benchmarks

LSUN

The Large-scale Scene Understanding dataset from Princeton, about ten million web images in 10 scene categories and 59 million in 20 object categories labelled by a cascade of deep classifiers with crowd workers in the loop; its label precision, the LSUN challenges at CVPR 2015–2017, and its second life as a generative-modelling benchmark (bedrooms, churches, cats, cars), where FID protocols differ from paper to paper.

intermediate

LSUN, the Large-scale Scene Understanding dataset, is a collection of web photographs labelled with one scene or object category each: about ten million images in 10 scene categories, from bedroom and kitchen to bridge and church outdoor, and 59 million in the 20 object categories of PASCAL VOC. Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff and Jianxiong Xiao of Princeton University described it on arXiv in June 2015; the revision of June 2016, which added Thomas Funkhouser as an author, added the object categories. Its labels came from a cascade in which deep classifiers trained on a few crowd-labelled images decide the easy cases and send only the ambiguous ones back to people. LSUN supplied the scene classification task of the LSUN challenges at CVPR from 2015 to 2017. It is best known today as a benchmark for unconditional image generation: models are trained on one category, such as LSUN Bedroom or LSUN Church, and compared by Fréchet Inception Distance, computed under protocols that differ from paper to paper.

Purpose

The authors’ argument was about density, the number of images per category (v3, §1). ImageNet and Places had millions of images but spread over hundreds or thousands of categories, about a thousand per ImageNet class. The authors aimed instead for about 10610^6 images per category, which they describe as roughly 10 times denser than Places and 100 times denser than ImageNet. They noted that ImageNet and Places each took more than a year of Amazon Mechanical Turk (AMT) labour, so a dataset NN times bigger built the same way would take more than NN years. Their answer was to amplify human effort with a partly automated labelling pipeline, and LSUN was built both to test the pipeline and to provide denser training data.

The two arXiv versions frame this differently. Versions 1 and 2 (June 2015) describe a scene-centric database with millions of labelled images per scene category, and claim precision and recall guarantees that reach the quality of fully manual annotation. Version 3 (June 2016) adds the object categories, replaces the pipeline description and reports measured precision above 90% on average, which it calls slightly lower than manual annotation (§1).

Dataset Size

Part Images Source
10 scene categories, training 9,895,373 in total, from 126,227 (church outdoor) to 3,033,042 (bedroom) Challenge pages 2015–2017 (per-category table; the sum is ours)
10 scene categories, validation 300 per category Challenge pages 2015–2017
Scene test set 10,000 (1,000 per category), labels withheld Challenge pages; README
20 object categories 59 million, from about 0.4 million (cow) to 18.9 million (person) Yu et al., v3, §6 and Figure 3
Scene category Training images
Bedroom 3,033,042
Kitchen 2,212,277
Living room 1,315,802
Bridge 818,687
Tower 708,264
Dining room 657,571
Restaurant 626,331
Conference room 229,069
Classroom 168,103
Church outdoor 126,227

The paper gives the scene total as 10 million and the object total as 59 million (v3, §1 and §6); its abstract and conclusion speak of around a million labelled images per category, which the per-category counts show is an average, not a floor. Figure 3 prints only the person count; the other object counts are read from its bar chart, roughly 5.6 million for car and 4.7 million for dog, and are approximate. The paper adds that the team was still collecting at about 5 million images and 2 categories a week (v3, footnote 1), but no later counts were published. Versions 1 and 2 report the scene part alone: 10 million images in 10 categories for about $8,000 of crowd labour, with the bedroom category 43 times larger than its Places counterpart (v2, §5.1).

The README states that every category is stored as one LMDB database whose entries are the encoded images, resized so that the smaller side is 256 pixels and compressed as JPEG at quality 75 (commit 6617c24). This may not hold for every archive: StyleGAN’s script for 512×384512 \times 384 cars keeps only images at least 512 pixels wide and at least three quarters as high as wide, so the 893,000 Car images StyleGAN2 used have a shorter side of at least 384 pixels (create_lsun_wide; Karras et al., 2020, Table 1), which by our inference means the car archive was not resized to 256. The export script in the same repository has, since July 2016, written each entry to a file with a .webp extension, and the LSUN loader in OpenAI’s guided-diffusion repository names the bytes webp_data; we did not download the data to check which format the released files use. As of October 2026 the official server lists the scene archives (43 GB for the bedroom training set, 2 GB for church outdoor) and one archive per object category, from 15 GB (cow) to 477 GB (person), about 1.7 TB for the 20 object categories together (our sum of the listing).

Classes

The 10 scene categories were taken from the SUN database (v3, §6): bedroom, bridge, church outdoor, classroom, conference room, dining room, kitchen, living room, restaurant and tower, indexed 0 to 9 in that alphabetical order for challenge submissions (category_indices.txt). The 20 object categories are those of PASCAL VOC 2012 (v3, §6): airplane, bicycle, bird, boat, bottle, bus, car, cat, chair, cow, dining table, dog, horse, motorbike, person, potted plant, sheep, sofa, train and tv-monitor, the names used for the object archives. Each image carries one label, the category it was collected and verified for; LSUN does not say which other categories appear in it.

Annotation Types

Every image is a positive for its category, decided either by two agreeing crowd workers or by a classifier score that the pipeline judged confident enough. There are no boxes, masks or multiple labels. The construction is described here from version 3, with the earlier pipeline afterwards.

Candidates. Image URLs came from Google Images, queried with the category name combined with adjectives: 696 scene adjectives (messy, spare, sunny, desolate…) chosen from a list of popular English adjectives, and a different set for objects, each query restricted to a three-day window, over every such window since 2009. This yielded nearly 100 million URLs per category, more than 111 million unique URLs for “dog” (v3, §3). More than a billion images were downloaded; only those whose smaller side exceeds 256 pixels were kept, about 60%. Near-duplicates were deliberately not removed: the authors judged deduplication of tens of millions of images too expensive and too prone to removing non-duplicates, and argued that duplicates at different compression levels, sizes and crops augment the data naturally.

The cascade. Labelling one category is treated as a binary classification of its candidate pool (v3, §4). Each iteration samples 40,000 images uniformly from the unlabelled pool and has them labelled by people: 10,000 test the classifier’s confidence, 5,000 validate model selection, and the rest train it. The classifier then scores the whole pool, and two thresholds split it:

t+: the score above which 95% of labelled test images are positive,t−: the score below which lie only 1% of the positives.t_{+}:\ \text{the score above which 95\% of labelled test images are positive}, \qquad t_{-}:\ \text{the score below which lie only 1\% of the positives}.

Images above t+t_+ are labelled positive, images below t−t_- negative and discarded, and those in between go to the next iteration, together with the human labels of the ambiguous training images. The cascade stops when the remaining pool is small enough to label by hand. The authors compare this with active learning but stress the difference: the goal is correct labels for one fixed set of images, so each classifier may overfit to the images that are left. For the first iterations, a multi-layer perceptron with two hidden layers on the last fully connected layer of AlexNet removed more than half of the pool; after two iterations, that feature space stopped helping, and the pipeline switched to fine-tuning a pretrained GoogLeNet (learning rate 0.001, dropped to 0.0001 after 40,000 of 60,000 iterations, minibatches of 80), scoring the central crop at first and the average of 10 crops after ten labelling iterations. Model selection used two accuracies on the validation images, one with equal weights per image and one with positives and negatives weighted equally, and kept the model that removed more test images.

Crowd work. Each image shown to people was labelled by two AMT workers, and only labels both gave were kept (v3, §5). A task (HIT) contained 205 images: 15 tutorial images that must be answered correctly before continuing, 150 images to label, and 40 images with known answers, 20 checked before submission (at least 90% correct) and 20 hidden ones checked afterwards (at least 85%), so that a worker who reads the page’s code still cannot pass. Instructions defined each category with examples: trucks and buses are not cars; one visible wheel does not make a bicycle; computer-generated images, cartoons, photo collages, images with obtrusive text and objects shown on a printed page or screen are negatives. To find corner cases, the team had 30,000 images per category labelled by three workers each and inspected the disagreements.

Precision and effort. For 11 object categories, trained experts labelled samples of the final positives by hand, which the text describes as 2,000 images from 11 categories (v3, §6). Precision is normally around 90% and varies by category; in Figure 4a it ranges from about 86% for person to about 97% for dining table (our reading of the bars). The main errors were toys, computer-rendered images, collages and edited photos, with rendered images worst for person. The pipeline amplified human effort about 40 times on average over these categories, and for train people labelled less than 1/60 of the images (Figure 4b). Version 3 reports no precision measurement for the scene categories.

The 2015 pipeline. Versions 1 and 2 describe a different, three-stage procedure for the scene categories, aimed at keeping each split above 95% precision while losing less than 1% recall (v2, §3). Features came from AlexNet trained on Places (fc7 and fc8 concatenated); 200,000 sampled images were clustered by k-means into 2,000 clusters, and human labels on 100 images per cluster decided which clusters to keep or discard; global linear SVMs and per-cluster kernel or exemplar SVMs then split the rest; fine-tuned networks handled what remained, until fewer than about 300,000 images were left for exhaustive labelling. A test set was labelled by hand, at least 1,300 positives per category. For kitchen, Table 1 (v2) reports 2,215,244 positives by its fifth iteration (numbered 4) at an estimated precision of 95.9%, with human labels for 6.6% of them. Each HIT paid $0.10.

Splits

The scene part has the splits of the classification challenge: the training set above, 300 validation images per category, and a test set of 10,000 images whose labels the organizers kept (README; challenge pages). The object categories are released as one archive each, with no validation or test split (download server, October 2026).

Generative models use the training sets, usually of one category, and the subsets differ between papers. Progressive GAN trained on 100,000 images per category except bedroom and dog, which used all images (Karras et al., 2018, Appendix G). StyleGAN2’s LSUN Car had 893,000 images (Karras et al., 2020, Table 1), and StyleGAN’s script for the 512×384512 \times 384 version skips images too small for that crop (create_lsun_wide). Latent diffusion models used the authors’ own division of the bedroom, cat and church training images into training and validation parts, distributed as file lists (latent-diffusion README).

Tasks

  • Scene classification of the 10 categories, with one top-1 prediction per test image (image classification).
  • The LSUN challenge tasks, held with the dataset but on other data: saliency prediction on iSUN and SALICON, room layout estimation on 4,000 training, 394 validation and 1,000 test images, and caption generation hosted by COCO (2015).
  • Pretraining. The paper fine-tuned AlexNet pretrained on Places with Places alone or with Places and up to 200,000 LSUN images per category, and tested on the 10 categories in the Places test set: error fell from 28.6% to 22.2%, with improvements in 8 of the 10 categories (v3, §6, Figure 5). Pretraining on 300,000 images from each object category raised PASCAL VOC 2012 classification mAP on the validation set from 0.80 to 0.87 for AlexNet trained from scratch, and from 0.82 to 0.88 for VGG initialized from ImageNet, compared with ImageNet pretraining alone (Table 1).
  • Unconditional image generation, one category per model, at 64×6464 \times 64 in DCGAN and at 256×256256 \times 256 or more since Progressive GAN.

Evaluation

Scene classification

The README at commit 6617c24 defines the challenge submission: one line per test image with its key and the predicted category index, scored as the percentage of correctly predicted labels, that is top-1 accuracy over the 10,000 test images. The organizers reserved the right to re-check test labels and remove images with controversial labels from the final evaluation. The repository contains no scoring code (we read its four files line by line); submissions went to the organizers by email, at most once every five days, with each team’s best entry ranked (2016 page).

FID on LSUN in generative modelling

FID itself, its formula, its small-sample bias and its sensitivity to resizing, belongs to the Fréchet Inception Distance article. On LSUN the choices that change the value are the reference images (all training images, a 50,000-image subset, or a custom split), the number of generated samples, how images are cropped and resized, horizontal flips, and which snapshot is reported. The papers that defined the benchmark made these choices differently:

Paper LSUN data Preparation Reference statistics Generated samples Reported value
DCGAN (Radford et al., 2016) Bedrooms, a little over 3 million; about 275,000 near-duplicates removed 64×6464 \times 64 No FID — Samples only
Progressive GAN (Karras et al., 2018) All 30 categories at 256×256256 \times 256; 100,000 images per category except bedroom and dog Mirror augmentation except bedroom and dog Not stated “50K images”, not said whether generated, real or both One FID per category (Appendix G)
StyleGAN (Karras et al., 2019) Bedroom and Cat at 256×256256 \times 256, Car at 512×384512 \times 384 StyleGAN’s dataset_tool.py: centre crop to a square (for Car, to 4:3, skipping smaller images), resize with PIL’s antialiasing filter; no mirroring 50,000 training images, “drawn randomly” in the paper, the first 50,000 in stored order in the code 50,000 Lowest FID during training
StyleGAN2 (Karras et al., 2020) Car, Cat, Church, Horse As StyleGAN; no augmentation 50,000 training images (fid50k in the code) 50,000 Snapshot with lowest FID; each metric averaged over 10 evaluations with different seeds (Table 1)
DDPM (Ho et al., 2020) Bedroom, Church, Cat at 256×256256 \times 256 StyleGAN’s code; horizontal flips in training except Bedroom StyleGAN2’s code, whose fid50k uses 50,000 training images 50,000 Minimum FID over training
ADM (Dhariwal & Nichol, 2021) Bedroom, Horse, Cat at 256×256256 \times 256 Resize the short side with a box filter, then centre crop The entire training set; other models recomputed with the same code 50,000 1,000 sampling steps
LDM (Rombach et al., 2022) Bedrooms, Churches at 256×256256 \times 256 Centre crop, bicubic resize; random horizontal flips in training “The entire training set” (Appendix E.3.1); the code trains on the authors’ own split 50,000 torch-fidelity, cross-checked with ADM’s script

The code shows what “50,000 images from the training set” means. In the StyleGAN and StyleGAN2 TensorFlow code, fid50k fills an array with the Inception features of the first 50,000 images that the dataset loader returns, with shuffling disabled (shuffle_mb = 0), each flipped left–right with probability 0.5 if training used mirroring, and caches their mean and covariance; the 50,000 generated images are compared with that subset, not with the whole category (metrics/frechet_inception_distance.py, metrics/metric_base.py and run_metrics.py, commits 1e0d5c7 and bf0fe0b). For LSUN the stored order is the order in which create_lsun reads the LMDB database, so the StyleGAN paper’s “drawn randomly” is, in the code, a fixed subset. The later PyTorch code names the two protocols: fid50k keeps that 50,000-image reference as a legacy metric, while fid50k_full uses every training image, without flips (metrics/metric_main.py, commit d72cc7d). ADM’s evaluator reads precomputed reference statistics over the whole dataset, plus 10,000 reference images for precision and recall, which it computes with k=3k = 3 nearest neighbours in the same Inception pool features (evaluations/README.md and evaluator.py, commit 22e0df8). Preparation differs too: StyleGAN crops the centre square first and then resizes it with PIL’s antialiasing filter, whereas ADM resizes the short side with a box filter and then crops, and LDM crops, resizes with bicubic interpolation and, by default in training, flips half the images (ldm/data/lsun.py and the LSUN configurations, commit a506df5). We read these files; we did not run them, since they need the LSUN data and Inception weights.

Published LSUN FIDs are not always comparable

Recomputing the same models under one protocol moves their scores. ADM recomputed the baselines it had samples for against the full training set (Dhariwal & Nichol, 2021, Table 5 and §2.2), and explains that some earlier papers compared against arbitrary subsets of the training set, citing StyleGAN, StyleGAN2 and DDPM:

Model, category FID in the model’s paper FID recomputed by ADM
StyleGAN, Bedroom 2.65 (StyleGAN, Figure 10 caption) 2.35
DDPM, Bedroom 4.90 for the large model, 6.36 for the standard one (DDPM, Table 3) 4.89 (ADM does not say which model)
DDPM, Cat 19.75 (DDPM, Table 3) 17.1
StyleGAN2, Cat 6.93 (StyleGAN2, Table 3) 7.25
StyleGAN2, Horse 3.43 (StyleGAN2, Table 3) 3.84

The changes run in both directions, by up to about 13% for these entries; for comparison, StyleGAN2’s published Church FID is 8% below StyleGAN’s. Later tables mix the two sources: LDM’s Table 1 lists StyleGAN’s bedroom FID as 2.35, with ADM’s precision and recall, but its church FID as 4.21, the value StyleGAN2 published under the 50,000-image protocol. LDM’s authors computed their own scores with torch-fidelity and with ADM’s script and found them mostly equal; on LSUN only bedrooms differed, 2.95 against 3.0 (Appendix E.3.1). The church model’s sampling steps disagree between sources, all with FID 4.02: 200 DDIM steps in Table 1, 100 in the compute comparison of Table 18, and 400 (η = 0) in the repository’s model table; the caption of Figure 30 gives 200 steps with η = 0 and FID 4.48. Baselines also differ in training data: the Progressive GAN FIDs that DDPM’s Table 3 lists for Church (6.42) and Cat (37.52) come from models trained on 100,000 images with mirroring, while the StyleGAN and StyleGAN2 READMEs convert whole categories, with no image limit.

Storage is a further variable. Parmar et al. (2022, §4.3, Figure 8) trained StyleGAN2 on LSUN Church, whose images are stored as quality-75 JPEG, and found that compressing the generated images with JPEG lowered the FID from 4.00 to 3.48, best at quality 87, because the generator is then compared with images bearing the same compression artifacts. No held-out LSUN set is large enough for a 50,000-sample reference: the scene validation sets have 300 images per category, and the object categories have none. So every LSUN FID in the papers above is computed against training images, and a model that reproduced its training images would not be penalized; this is our reading of the protocols, which the example below illustrates. Benchmark overfitting and dataset bias discuss the general problem.

Reference set, sample count and preprocessing in a small example

No pretrained network is used: a fixed random projection of the pixels, 64 numbers per image, stands in for the 2048 Inception-v3 features, so the values are in arbitrary units and only their comparison matters. The images are rendered “bedrooms”, 32×4832 \times 48 pixels: a wall and floor of random colours, a bright window and a coloured bed, prepared like LSUN for training by a centre crop and a 2×22 \times 2 box downsampling to 16×1616 \times 16. Three stand-in generators are scored: an ideal one that renders fresh rooms, a memorizer that returns copies of training images, and a flawed one that draws every bed 5% of the image width too far right. The script compares a training-set reference with a held-out one, varies the number of generated and reference images, and prepares the reference images in three ways.

import numpy as np
from scipy import linalg

H, W = 32, 48                      # rendered at 2:3, like a landscape photo

def render_rooms(n, rng, bed_shift=0.0):
    """n synthetic 'bedrooms': wall, floor, a window and a bed, plus noise."""
    yy, xx = np.mgrid[:H, :W]
    wall = rng.uniform(0.4, 0.9, (n, 3))
    floor = rng.uniform(0.1, 0.5, (n, 3))
    horizon = rng.uniform(0.45, 0.65, n) * H                 # floor-wall line
    wx, wy = rng.uniform(0.1, 0.7, n) * W, rng.uniform(0.1, 0.25, n) * H
    bx = rng.uniform(0.15, 0.55, n) * W + bed_shift * W    # bed position
    bw, bh = rng.uniform(0.25, 0.4, n) * W, rng.uniform(0.15, 0.25, n) * H
    bed = rng.uniform(0.2, 1.0, (n, 3))
    img = np.where((yy < horizon[:, None, None])[..., None],
                   wall[:, None, None], floor[:, None, None])
    win = ((xx >= wx[:, None, None]) & (xx < wx[:, None, None] + 0.2 * W)
           & (yy >= wy[:, None, None]) & (yy < wy[:, None, None] + 0.2 * H))
    img[win] = 1.0
    bedm = ((xx >= bx[:, None, None]) & (xx < (bx + bw)[:, None, None])
            & (yy >= (horizon - 0.3 * bh)[:, None, None])
            & (yy < (horizon + bh)[:, None, None]))
    img = np.where(bedm[..., None], bed[:, None, None], img)
    img += rng.normal(0, 0.03, img.shape)
    return np.clip(img, 0, 1)                               # (n, 32, 48, 3)

# Three ways to turn a 32x48 render into a 16x16 training image.
def crop_box(x):        # centre crop 32x32, then 2x2 box average
    return x[:, :, 8:40].reshape(-1, 16, 2, 16, 2, 3).mean(axis=(2, 4))
def crop_point(x):      # centre crop 32x32, then keep every 2nd pixel
    return x[:, ::2, 8:40:2]
def squash(x):          # no crop: average 2x3 blocks, changing the aspect
    return x.reshape(-1, 16, 2, 16, 3, 3).mean(axis=(2, 4))

# Fixed random projection: a STAND-IN for Inception-v3 features (not a CNN).
P = np.random.default_rng(1234).normal(size=(16 * 16 * 3, 64))

def feats(n, rng, prep=crop_box, **kw):
    return prep(render_rooms(n, rng, **kw)).reshape(n, -1) @ P

def fd(f1, f2):         # Frechet distance between Gaussian fits, as in FID
    mu1, mu2 = f1.mean(0), f2.mean(0)
    s1, s2 = np.cov(f1, rowvar=False), np.cov(f2, rowvar=False)
    return float(((mu1 - mu2) ** 2).sum()
                 + np.trace(s1 + s2 - 2 * linalg.sqrtm(s1 @ s2).real))

rng = np.random.default_rng(0)
print("1. Reference set: 10,000 samples per generator, mean +/- sd over 5 seeds")
for n_train in (2000, 20000):
    train, heldout = feats(n_train, rng), feats(n_train, rng)
    gens = {"ideal (fresh rooms)": lambda r: feats(10000, r),
            "memorizer (copies)": lambda r: train[r.integers(0, n_train, 10000)],
            "beds shifted 5% right": lambda r: feats(10000, r, bed_shift=0.05)}
    print(f"  training set of {n_train:,}   vs training set   vs held-out set")
    for name, g in gens.items():
        a, b = [], []
        for seed in range(5):
            s = g(np.random.default_rng(100 + seed))
            a.append(fd(s, train)); b.append(fd(s, heldout))
        print(f"    {name:23s} {np.mean(a):6.2f} +/- {np.std(a):.2f}"
              f"   {np.mean(b):6.2f} +/- {np.std(b):.2f}")

print("\n2. Sample counts: ideal generator against a 20,000-image training set")
print("   N generated   ref = all 20,000   ref = random 2,000 of them")
for n in (1000, 5000, 20000):
    a, b = [], []
    for seed in range(5):
        r = np.random.default_rng(200 + seed)
        s = feats(n, r)
        a.append(fd(s, train)); b.append(fd(s, train[r.choice(20000, 2000, replace=False)]))
    print(f"   {n:11,d}   {np.mean(a):6.2f} +/- {np.std(a):.2f}"
          f"     {np.mean(b):6.2f} +/- {np.std(b):.2f}")

print("\n3. Preprocessing: ideal generator (crop + box) against 20,000 references")
s = feats(10000, np.random.default_rng(300))
for name, prep in [("crop + box, as the generator", crop_box),
                   ("crop + point sampling", crop_point),
                   ("squash, no crop", squash)]:
    print(f"   {name:30s} FD = {fd(s, feats(20000, np.random.default_rng(400), prep)):6.2f}")

Output (NumPy 2.3.5, SciPy 1.16.3, about 35 seconds):

1. Reference set: 10,000 samples per generator, mean +/- sd over 5 seeds
  training set of 2,000   vs training set   vs held-out set
    ideal (fresh rooms)       6.84 +/- 0.20     6.90 +/- 0.19
    memorizer (copies)        1.12 +/- 0.10    12.97 +/- 0.25
    beds shifted 5% right    16.10 +/- 0.29    20.87 +/- 0.26
  training set of 20,000   vs training set   vs held-out set
    ideal (fresh rooms)       1.73 +/- 0.12     1.68 +/- 0.08
    memorizer (copies)        1.12 +/- 0.07     2.31 +/- 0.15
    beds shifted 5% right    12.89 +/- 0.16    13.55 +/- 0.07

2. Sample counts: ideal generator against a 20,000-image training set
   N generated   ref = all 20,000   ref = random 2,000 of them
         1,000    11.48 +/- 0.49      16.70 +/- 1.22
         5,000     2.96 +/- 0.13       8.24 +/- 0.70
        20,000     1.08 +/- 0.11       6.41 +/- 0.51

3. Preprocessing: ideal generator (crop + box) against 20,000 references
   crop + box, as the generator   FD =   1.49
   crop + point sampling          FD =  26.61
   squash, no crop                FD = 110.11

Against the training set, copying wins: the memorizer scores below the ideal generator with both training-set sizes, by a factor of six when the training set has 2,000 images, because its samples come from the very images that define the reference. Against a held-out set the order reverses, and the memorizer scores worse than the ideal generator, since its 10,000 samples contain only as much variety as the training set. With 20,000 training images the memorizer’s advantage shrinks to about 0.6, still several times the spread across seeds. The 5% bed shift is a real defect and costs more than either effect.

The sample counts change the value of one and the same generator more than tenfold. With the full reference, the distance falls steeply as more samples are generated; with a 2,000-image reference subset it levels off near 6, the bias contributed by the small reference. A generator scored with 20,000 samples against all training images and the same generator scored with 5,000 samples against a subset differ by a factor of nearly eight. The preprocessing rows hold the generator fixed and change only how the reference images were prepared: point sampling adds aliasing that the box-filtered training images never had, and squashing without a crop changes the shapes, and the distance grows 18-fold and 74-fold. The proportions belong to this toy feature map, not to Inception, but the direction of each effect matches the protocol differences in the tables above.

License

As of October 2026 we found no licence or terms of use for LSUN. The archived dataset page (July 2023), the challenge pages of 2015 to 2017 and the README at commit 6617c24 describe the data and how to download it without stating any terms; the challenge pages add that requests for original images go to Fisher Yu. The address of the dataset page, www.yf.io/p/lsun, redirected in October 2026 to www.vis.xyz/p/lsun, which we could not reach, so the current page may say more. The paper’s conclusion promises to keep the dataset freely available to the community (v3, §7) but grants no licence.

The terms are unclear for the images in particular. They were found through Google Images search results (v3, §3), and the sources do not say who holds their copyright or what uses are allowed; in our reading, they remain the works of their original owners. The labels have no licence either. The download and export scripts declare __license__ = 'MIT' in their headers, which covers that code, not the data; the repository has no licence file.

Limitations and Bias

  • Label noise. About 90% precision on average, as low as about 86% for person, with toys, renderings, collages and edited photos among the positives (v3, §6). Generative models trained on a category learn its errors along with its images.
  • Duplicates. Near-duplicates were kept on purpose (v3, §3); DCGAN’s authors removed about 275,000 from the bedroom set before training, using hashes from a denoising autoencoder on 32×3232 \times 32 centre crops (Radford et al., 2016, §4.1.1).
  • Compression and size. The released images are small (shorter side 256 pixels, per the README, though apparently not in the car archive) and lossily compressed, which shapes what a generator learns and, as Parmar et al. found, what FID rewards.
  • Web sampling. Images come from search-engine results for adjective-plus-category queries since 2009, so each category reflects how such photos are posted and ranked online. A category label says nothing about the rest of the image.
  • People. Person is the largest category, 18.9 million images, and people appear in many scene images; the paper and the official pages say nothing about consent, privacy or removal requests.
  • Feature-space limits. On LSUN Cat and Car, StyleGAN2’s authors show pairs of models with the same FID (8.53 and 3.27) and nearly the same precision and recall, of which one produces cat- or car-shaped objects visibly more often; only perceptual path length separates them (Karras et al., 2020, Figures 13 and 14).

Common Uses

LSUN Bedroom, Church outdoor, Cat, Car and Horse at 256×256256 \times 256 (Car at 512×384512 \times 384) are standard unconditional benchmarks for GANs, StyleGAN, DDPM and latent diffusion models, usually reported with FID and sometimes with precision and recall for generative models. Bedrooms have also served to test training methods, as in the TTUR experiments of the FID paper. The scene categories were used for scene classification and, through the paper’s experiments, as extra pretraining data for scene and object recognition.

Historical Milestones

  • 2015: LSUN challenge. The first challenge, at the CVPR 2015 workshop on 12 June 2015, had four tasks; the challenge page lists Google as the winner of scene classification, UPC of saliency and UIUC of room layout.
  • 2015–2016: DCGAN bedrooms. Radford et al. trained on the 3 million bedrooms at 64×6464 \times 64, and showed samples after one epoch to argue that the model was not memorizing.
  • 2016 and 2017: later challenges. In 2016 the tasks were scene classification, saliency and room layout; in 2017, scene classification, segmentation of street images on Mapillary Vistas, and saliency on SALICON.
  • 2017–2018: all 30 categories. Progressive GAN generated every LSUN category at 256×256256 \times 256 and published an FID for each (Bedroom 8.34, Church outdoor 6.42), establishing the high-resolution benchmark.
  • 2019–2020: StyleGAN and StyleGAN2. Bedroom FID 2.65 for StyleGAN; StyleGAN2 reported 3.86 on Church and 2.32 on Car at 512×384512 \times 384.
  • 2020–2022: diffusion models. DDPM reached 4.90 on Bedroom with its large model, which the authors likened to Progressive GAN’s quality; ADM reported 1.90 on Bedroom against the full training set, below StyleGAN’s recomputed 2.35; LDM reported 2.95 on Bedroom, close to ADM with half its parameters and a quarter of its training resources by the authors’ account (§4.2), and 4.02 on Church. Each value is under the protocol of its paper, as above.
  • Places: millions of scene images in hundreds of categories, the baseline LSUN set out to be denser than; the LSUN paper’s scene experiments used Places-trained AlexNet and the Places test set.
  • SUN database: the source of LSUN’s scene categories and, with LSUN, of the room layout challenge images.
  • PASCAL VOC: the source of the 20 object categories.
  • ImageNet: the object-centric dataset whose low per-class density motivated LSUN.
  • CelebA (CelebA-HQ) and FFHQ: the face datasets that, with LSUN, form the standard unconditional generation benchmarks; CIFAR-10 is the low-resolution counterpart.

Resources

  • Official dataset page (archived July 2023): description, paper and links to the scene and object data.
  • Official repository fyu/lsun, commit 6617c24: README with the data format and submission format, download and export scripts, category indices.
  • Download server dl.yf.io/lsun/: scene and object archives with MD5 sums for the objects (as of October 2026).
  • Challenge pages for 2015, 2016 and 2017 (archived): data tables, tasks, schedules and organizers; the leaderboards were linked from them.
  • For generative evaluation: StyleGAN’s dataset_tool.py, the fid50k and fid50k_full metrics in NVIDIA’s StyleGAN repositories, ADM’s evaluator and reference batches in openai/guided-diffusion, and the LDM split files linked from CompVis/latent-diffusion.

Related

  • Fréchet Inception Distance

    The standard sample-based score for image generative models, the Fréchet (2-Wasserstein) distance between Gaussians fitted to Inception-v3 features of real and generated images; its formula, its small-sample bias, its sensitivity to resizing and compression, its ImageNet bias, and the related metrics IS, KID, precision and recall, and FD-DINOv2.

  • StyleGAN

    NVIDIA's family of style-based GAN generators, in which a mapping network turns the latent code into per-layer styles that modulate every convolution; the original StyleGAN (2019), StyleGAN2's weight demodulation and path length regularization, adaptive discriminator augmentation for small datasets, alias-free StyleGAN3, and StyleGAN-XL.

  • Generative Adversarial Network

    A generative model trained as a game between a generator that turns random noise into images and a discriminator that tries to tell them from real images; the minimax objective and its Jensen–Shannon interpretation, the non-saturating loss, DCGAN, mode collapse and training instability, Wasserstein and other remedies, conditional and large-scale GANs, and why diffusion models overtook them.

  • Latent Diffusion Model

    A diffusion model that runs in the compressed latent space of a pretrained autoencoder instead of in pixels; the two-stage design, the downsampling factor and its trade-off, the latent scaling factor, cross-attention and concatenation conditioning, the compute savings, and successors from SDXL to latent transformers and rectified flow.

  • PASCAL VOC

    The PASCAL Visual Object Classes challenge (2005–2012), whose 20-class datasets of Flickr photographs with bounding boxes and segmentations set the standard protocol for evaluating object detection.

  • Benchmark Overfitting

    How reusing one test set for years of model selection can inflate benchmark scores, what the theory of adaptive data analysis says about it, and what replication studies of CIFAR-10, ImageNet, MNIST and Kaggle found when they measured it.

References

  1. Yu, F., Seff, A., Zhang, Y., Song, S., Funkhouser, T. & Xiao, J. (2015). LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. arXiv:1506.03365v3 (June 2016; scene and object categories).
  2. Yu, F., Zhang, Y., Song, S., Seff, A. & Xiao, J. (2015). LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop. arXiv:1506.03365v2 (June 2015; scene categories only; v1 of 10 June 2015 has the same content without 'LSUN:' in the title).
  3. Yu, F. LSUN dataset page, with links to the scene and object data and the challenge (official page; archived copy of 13 July 2023).
  4. Yu, F. lsun: LSUN Dataset Documentation and Demo Code (README, download.py, data.py, category_indices.txt; official repository, commit 6617c24 of 19 November 2022, accessed October 2026).
  5. Yu, F. Index of /lsun/, /lsun/scenes/ and /lsun/objects/ (official download server listing, accessed October 2026).
  6. Princeton Vision Group. LSUN Challenge 2015: scene classification, saliency prediction, room layout estimation and caption generation (official challenge page; archived copy of 22 October 2018).
  7. Princeton Vision Group. LSUN 2016: Large-scale Scene Understanding Challenge (official challenge page; archived copy of 18 January 2019).
  8. Princeton Vision Group. LSUN 2017: Large-scale Scene Understanding Challenge (official challenge page; archived copy of 28 January 2019).
  9. Radford, A., Metz, L. & Chintala, S. (2016). Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. International Conference on Learning Representations (ICLR). arXiv:1511.06434.
  10. Karras, T., Aila, T., Laine, S. & Lehtinen, J. (2018). Progressive Growing of GANs for Improved Quality, Stability, and Variation. International Conference on Learning Representations (ICLR) (read in arXiv:1710.10196v3).
  11. Karras, T., Laine, S. & Aila, T. (2019). A Style-Based Generator Architecture for Generative Adversarial Networks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4396–4405 (read in arXiv:1812.04948v3).
  12. Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J. & Aila, T. (2020). Analyzing and Improving the Image Quality of StyleGAN. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8107–8116 (read in arXiv:1912.04958v2).
  13. Ho, J., Jain, A. & Abbeel, P. (2020). Denoising Diffusion Probabilistic Models. Advances in Neural Information Processing Systems 33 (NeurIPS), 6840–6851 (read in arXiv:2006.11239v2, with appendices).
  14. Dhariwal, P. & Nichol, A. (2021). Diffusion Models Beat GANs on Image Synthesis. Advances in Neural Information Processing Systems 34 (NeurIPS), 8780–8794 (read in arXiv:2105.05233v4, with appendices).
  15. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. (2022). High-Resolution Image Synthesis with Latent Diffusion Models. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10674–10685 (read in arXiv:2112.10752v2, with appendices).
  16. Parmar, G., Zhang, R. & Zhu, J.-Y. (2022). On Aliased Resizing and Surprising Subtleties in GAN Evaluation. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 11400–11410 (read in arXiv:2104.11222v3).
  17. NVIDIA. stylegan: dataset_tool.py (create_lsun, create_lsun_wide), run_metrics.py, metrics/metric_base.py and metrics/frechet_inception_distance.py (official repository, commit 1e0d5c7 of 28 October 2021, accessed October 2026).
  18. NVIDIA. stylegan2: run_metrics.py, metrics/metric_base.py, metrics/frechet_inception_distance.py and metrics/metric_defaults.py (official repository, commit bf0fe0b of 28 October 2021, accessed October 2026).
  19. NVIDIA. stylegan2-ada-pytorch: metrics/metric_main.py (fid50k_full and fid50k) (official repository, commit d72cc7d of 22 September 2023, accessed October 2026).
  20. OpenAI. guided-diffusion: evaluations/README.md, evaluations/evaluator.py and datasets/lsun_bedroom.py (official repository, commit 22e0df8 of 15 July 2022, accessed October 2026).
  21. CompVis. latent-diffusion: README (LSUN split and model table), ldm/data/lsun.py and the LSUN configurations in configs/latent-diffusion/ (official repository, commit a506df5 of 26 July 2022, accessed October 2026).