ImageNet

ImageNet is a database of photographs collected from the web and organized by the noun hierarchy of WordNet, built by Jia Deng, Li Fei-Fei, Kai Li and colleagues at Princeton University and presented at CVPR 2009; it is now run by Princeton and Stanford universities. Its best-known part is a 1,000-class subset used in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), held every year from 2010 to 2017. That subset, ILSVRC-2012, often called ImageNet-1k or simply “ImageNet”, became the standard benchmark for image classification and the usual dataset for pretraining backbone networks.

Purpose

Deng et al. (2009) set out to illustrate most of the roughly 80,000 noun synsets of WordNet with an average of 500–1,000 clean, full-resolution images each: tens of millions of labeled images. A synset (synonym set) is one concept that may be named by several words, such as “dog, domestic dog, Canis familiaris”. The project’s about page names two motivations: a clear object-categorization benchmark, and more data for methods that generalize.

Dataset Size

The 2009 paper reported 3.2 million images in 5,247 synsets from 12 subtrees, among them mammal, bird, vehicle, furniture and musical instrument. The full release grew to 14,197,122 images in 21,841 synsets (Yang et al., 2020); the website still displays these counts (as of October 2026), although they predate the 2021 removal of person synsets. This release is usually called ImageNet-21k (sometimes ImageNet-22k). The ILSVRC subset is much smaller:

ILSVRC classification data	Classes	Training	Validation	Test
2010	1,000	1,261,406	50,000	150,000
2011	1,000	1,229,413	50,000	100,000
2012–2017 (ImageNet-1k)	1,000	1,281,167	50,000	100,000

Training images per class range from 732 to 1,300 in ILSVRC-2012; the validation and test sets have exactly 50 and 100 per class (Russakovsky et al., 2015; the download page gives the same totals for 2012–2017). Images come in varying resolutions, so models resize and crop them, commonly to $224 \times 224$.

Classes

Every class is a WordNet noun synset, identified by a WordNet ID (wnid): the letter “n” (for noun) followed by an eight-digit number that identifies the synset in WordNet. For example, n01440764 is “tench, Tinca tinca”, the first class in sorted order, and n02084071 is “dog, domestic dog, Canis familiaris”. The parent–child links come from WordNet, so ImageNet is a hierarchy rather than a flat list: a pug is a dog, a canine, a carnivore and a mammal.

The 1,000 ILSVRC classes were chosen so that none is an ancestor of another, which makes them mutually exclusive labels. They include both leaves and internal nodes of the hierarchy. The class list changed in the early years and has been fixed since 2012, when 90 synsets were replaced by dog breeds to test fine-grained recognition. Only three ILSVRC classes are people categories: scuba diver, bridegroom and baseball player (Yang et al., 2020).

Annotation Types

• Image-level labels. Each image carries one synset label. Candidates came from image search engines, queried with a synset’s synonyms, with a parent synset’s word appended when it appears in the synset’s definition, and with translations. Several workers on Amazon Mechanical Turk then judged whether each image contains the synset; harder concepts (“Burmese cat” rather than “cat”) needed more agreeing votes. An independent check of 80 synsets found an average labeling precision of 99.7% (Deng et al., 2009).
• Bounding boxes. For single-object localization, every validation and test image and a subset of training images (523,966 in ILSVRC-2012) have axis-aligned bounding boxes around every instance of the labeled class.
• Detection annotations. The detection task (2013 onward) uses 200 basic-level classes, chosen mostly from objects people label easily, with every instance of every class boxed in the validation and test images. Its 2014 training set has 456,567 images.

Splits

Training images are a fixed subset of ImageNet, while validation and test images were collected separately in the same way (Russakovsky et al., 2015). Only the 2010 test labels were released, and the evaluation server accepts at most two submissions per week (as of October 2026). Papers therefore commonly report results on the 50,000-image validation set and treat it as the test set.

File layout. The ILSVRC-2012 training archive holds one tar file per synset, named by its wnid (n01440764.tar and so on), which loaders such as torchvision’s unpack into one folder per class. Validation images come in one flat folder; loaders sort them into the same folders using the development kit’s ground-truth file.

Tasks

• Image classification (2010–2014): return labels for an image. From 2015 classification error was scored within the localization task.
• Single-object localization (from 2011): return labels, each with one box locating an instance.
• Object detection (from 2013): find every instance of the 200 detection classes.
• Object detection from video (a taster in 2015, then a main task in 2016 and 2017): 30 classes, a subset of the detection classes.

Smaller tasks included fine-grained classification of 120 dog breeds as a taster in 2012, and scene classification with the MIT Places team in 2015 and 2016. In July 2017 the organizers handed the challenges over to Kaggle.

Evaluation

Each validation or test image has a single label, even when it contains several objects. To avoid penalizing a correct answer that names an unlabeled object, ILSVRC let algorithms return five ranked labels and scored an image as wrong only if none matches. With labels $c_{i1}, \dots, c_{i5}$ for image $i$ with true class $C_i$, and $d(c, C) = 1$ if $c \neq C$ and 0 otherwise, the top-5 error over $N$ images is

$$\text{error} = \frac{1}{N} \sum_{i=1}^{N} \min_{j} d(c_{ij}, C_i).$$

Top-1 error uses only the most confident label. A hierarchical error replaced $d$ with the height of the lowest common ancestor of the two classes in the hierarchy, so that confusing two dog breeds costs less than confusing a dog with a container ship. All three gave the same ranking of entries, and from 2012 the challenge used top-5 error alone (Russakovsky et al., 2015).

Localization counts a guess as correct only if the class matches and its box overlaps a ground-truth box with intersection over union above 0.5. Detection is scored by mean average precision, with the IoU threshold loosened for very small objects.

The code below scores an ILSVRC-style submission of five ranked wnids for each of four images, using eight real ILSVRC classes and their paths in ImageNet’s published WordNet hierarchy.

import numpy as np

# Paths below "whole" in ImageNet's WordNet hierarchy for eight ILSVRC-2012 classes
# (from image-net.org/data/wordnet.is_a.txt, taking the first parent where there are several).
A = "living thing/organism/animal/chordate/vertebrate/mammal/placental/carnivore/"
V = "artifact/instrumentality/conveyance/vehicle/craft/"
paths = {
    "n02110958": A + "canine/dog/pug",
    "n02085620": A + "canine/dog/toy dog/Chihuahua",
    "n02099601": A + "canine/dog/hunting dog/sporting dog/retriever/golden retriever",
    "n02123045": A + "feline/cat/domestic cat/tabby",
    "n02123394": A + "feline/cat/domestic cat/Persian cat",
    "n03095699": V + "vessel/ship/cargo ship/container ship",
    "n04273569": V + "vessel/boat/motorboat/speedboat",
    "n02690373": V + "aircraft/heavier-than-air craft/airplane/airliner",
}
chain = {w: ["whole"] + p.split("/") for w, p in paths.items()}   # root-to-leaf nodes
name = {w: c[-1] for w, c in chain.items()}

height = {}                     # longest path from a node down to a leaf; leaves have height 0
for c in chain.values():
    for depth, node in enumerate(c):
        height[node] = max(height.get(node, 0), len(c) - 1 - depth)

def lca(a, b):                  # lowest common ancestor of two classes
    return [x for x, y in zip(chain[a], chain[b]) if x == y][-1]

# An ILSVRC classification submission gives five ranked labels per image.
submissions = [                 # (ground-truth synset, five guesses, most confident first)
    ("n02085620", ["n02110958", "n02085620", "n02123394", "n02099601", "n02123045"]),
    ("n02123394", ["n02123394", "n02123045", "n02110958", "n02085620", "n02099601"]),
    ("n04273569", ["n03095699", "n02690373", "n02123045", "n02110958", "n02099601"]),
    ("n02099601", ["n02123045", "n02123394", "n03095699", "n02690373", "n04273569"]),
]

top1, top5, hier = [], [], []
for truth, guesses in submissions:
    d = np.array([g != truth for g in guesses])           # flat error of each guess
    h = np.array([height[lca(g, truth)] for g in guesses])  # hierarchical cost of each guess
    top1.append(d[0]); top5.append(d.min()); hier.append(h.min())
    best = guesses[h.argmin()]
    print(f"{name[truth]:>16}: top-1 {name[guesses[0]]!r}, correct in top 5: {not d.min()}, "
          f"hierarchical cost {h.min()} (via {name[best]!r}, common ancestor {lca(best, truth)!r})")
print(f"top-1 error {np.mean(top1):.2f}, top-5 error {np.mean(top5):.2f}, "
      f"mean hierarchical cost {np.mean(hier):.2f}")

Output:

       Chihuahua: top-1 'pug', correct in top 5: True, hierarchical cost 0 (via 'Chihuahua', common ancestor 'Chihuahua')
     Persian cat: top-1 'Persian cat', correct in top 5: True, hierarchical cost 0 (via 'Persian cat', common ancestor 'Persian cat')
       speedboat: top-1 'container ship', correct in top 5: False, hierarchical cost 3 (via 'container ship', common ancestor 'vessel')
golden retriever: top-1 'tabby', correct in top 5: False, hierarchical cost 6 (via 'tabby', common ancestor 'carnivore')
top-1 error 0.75, top-5 error 0.50, mean hierarchical cost 2.25

The Chihuahua called a pug counts against top-1 but not top-5. The two misses differ under the hierarchical measure: a speedboat called a container ship shares the ancestor “vessel”, a golden retriever called a tabby only “carnivore”. Heights are measured within this eight-class slice, not the full hierarchy.

License

ImageNet does not own the copyright of its images; it compiles lists of web images. Under the terms of access on the download page (as of October 2026), researchers may use the database “only for non-commercial research and educational purposes”, must accept responsibility for their use, and an employer that is a commercial entity is also bound by the terms. The ILSVRC-2012 classification and localization subset is also distributed through Kaggle.

Limitations and Bias

• One label for scenes with several objects. Beyer et al. (2020) re-annotated the validation set with multiple labels per image (“ReaL” labels) and found that about 29% of images contain several objects or a category matching several synonymous ImageNet classes, such as “sunglasses” and “sunglass”. Recent models’ gains were substantially smaller under the new labels.
• Label errors. Northcutt et al. (2021) flagged likely errors algorithmically and had crowd workers confirm 2,916 errors (about 6%) in the validation set; from an expert review of unflagged images they estimate the true rate is closer to 20%. They also note duplicated class names, such as two different classes both called “crane”.
• Overfitting to one test set. Recht et al. (2019) collected ImageNetV2, a new test set of 10,000 images (10 per class) gathered from Flickr and labeled on Mechanical Turk following the original procedure. Accuracy dropped by 11–14 percentage points across models, while their ranking was largely preserved.
• People, offensive categories and privacy. Yang et al. (2020) examined the 2,832 synsets in the person subtree of the full ImageNet, judged 1,593 potentially offensive, and found only 158 of the remaining 1,239 visual. The website reports removing 2,702 person synsets from the full release in 2021, without affecting the ILSVRC classes. Many ILSVRC images still show people incidentally: Yang et al. (2022) annotated 562,626 faces in 243,198 images (17%) and built a face-blurred version of ILSVRC; training and testing on it cost only 0.1–0.7 points of validation accuracy, and features transferred equally well.
• Web photographs. Accuracy on search-engine photographs need not carry over to sketches, paintings or other distributions (see Related Datasets).

Common Uses

• Benchmarking classifiers. ImageNet-1k top-1 accuracy is a standard comparison for new architectures and training methods.
• Pretraining backbones. ImageNet-trained networks supply initial weights or features for detection, segmentation and many other tasks. Kornblith et al. (2019) compared 16 networks on 12 classification datasets and found ImageNet top-1 accuracy strongly correlated with transfer accuracy, with $r = 0.99$ for fixed features and $r = 0.96$ for fine-tuning. On two fine-grained datasets, Stanford Cars and FGVC Aircraft, fine-tuning from ImageNet gave little benefit over training from scratch.
• Large-scale pretraining. Dosovitskiy et al. (2021) found that the Vision Transformer, a transformer applied to image patches, trailed convolutional networks when trained on ImageNet-1k alone but became competitive when pretrained on the public ImageNet-21k or larger datasets.

Historical Milestones

Top-5 test error of the winning classification entries (Russakovsky et al., 2015; for 2015, the official ILSVRC2015 results and He et al., 2016):

Year	Winner	Approach	Top-5 error
2010	NEC	SIFT and LBP features, nonlinear coding, SVM	28.2%
2011	XRCE	Compressed Fisher vectors, linear SVMs	25.8%
2012	SuperVision (AlexNet)	Convolutional neural network with 60 million parameters, trained on GPUs	16.4%; 15.3% with extra ImageNet training data
2013	Clarifai	Ensemble of CNNs	11.7%; 11.2% with extra data
2014	GoogLeNet	Deeper and wider CNN, using dimension-reduction layers	6.7%
2015	MSRA (ResNet)	Residual networks, six-model ensemble	3.57%

The second-best entry in 2012 still used Fisher vectors, at 26.2%; Russakovsky et al. call that year a turning point, after which the vast majority of entries used deep CNNs. They also measured human performance on 1,500 test images: a trained expert annotator reached 5.1% top-5 error, against 6.8% for GoogLeNet on the same images.

Related Datasets

Several test sets reuse the ImageNet classes to measure robustness:

• ImageNetV2 (Recht et al., 2019): a fresh replica of the validation set, described above.
• ImageNet-A (Hendrycks et al., 2021, CVPR): 7,500 real photographs from 200 classes, kept only if ResNet-50 misclassified them; DenseNet-121 reaches about 2% accuracy.
• ImageNet-R (Hendrycks et al., 2021, ICCV): 30,000 renditions, such as paintings, cartoons and embroidery, of 200 ImageNet classes. Hendrycks et al. note that ImageNet’s own annotators had been told to collect photographs only.
• ImageNet-Sketch (Wang et al., 2019): 50,000 black-and-white sketches, 50 for each of the 1,000 classes, collected with Google Image searches.

Resources

• ImageNet: the official site, with downloads of the full release and the ILSVRC subsets after accepting the terms of access.
• ILSVRC: pages for each challenge year from 2010 to 2017, and the evaluation server for the ILSVRC-2012 test set.
• Reassessed labels (Beyer et al., 2020) and face annotations (Yang et al., 2022) for the ILSVRC images.