Features & Representation

Histograms of Oriented Gradients for Human Detection

Dalal and Triggs's CVPR 2005 paper, which showed that dense grids of locally normalized gradient-orientation histograms with a linear SVM detect people far better than earlier feature sets, measured each design choice, and introduced the INRIA person dataset.

intermediate

“Histograms of Oriented Gradients for Human Detection” is a paper by Navneet Dalal and Bill Triggs of INRIA Rhône-Alpes, presented at the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) in 2005. It asked which feature set lets a simple window classifier find people in cluttered photographs, and answered with dense grids of locally contrast-normalized histograms of gradient orientations, scored by a linear support vector machine (SVM). The paper then varied each stage of the descriptor in turn and measured the effect, which made it as much a design study as a method paper, and it introduced the INRIA person dataset because its detector had saturated the older MIT pedestrian set. In 2015 it received the Longuet-Higgins Prize. The descriptor as it is understood and implemented today is covered in the histogram of oriented gradients article; this article covers the publication, its experiments and results, and its extended versions.

Problem

Dalal and Triggs framed person detection as a feature-set problem. People vary in pose, clothing and background, so the first need was a representation in which the human form stays separable from clutter under difficult lighting. To compare feature sets on equal terms, they fixed the rest of the pipeline: a linear SVM on a 64×12864 \times 128 window, scanned over the image, as described in sliding window detection. Their test case was pedestrian detection in the narrow sense they defined: mostly visible people in more or less upright poses.

The paper summarizes the human detectors it set out to beat as follows:

  • Wavelets with an SVM. Papageorgiou and Poggio fed rectified Haar wavelets to a polynomial SVM, and Mohan, Papageorgiou and Poggio built a parts-based (subwindow) variant. These MIT detectors were the paper’s main baseline.
  • Edge templates. Gavrila and Philomin matched edge images to learned exemplars by chamfer distance, an approach used in a real-time pedestrian system.
  • Boosted cascades. Viola, Jones and Snow detected moving people with an AdaBoost-trained chain of rejection rules over Haar-like features and space-time differences, the cascade design of Viola–Jones extended to motion.
  • Part assemblies. Ronfard, Schmid and Triggs combined SVM limb classifiers in a dynamic-programming framework, and Mikolajczyk, Schmid and Zisserman assembled detectors for faces, heads and upper and lower body parts built on orientation-position histograms of thresholded gradients. Dalal and Triggs contrasted their own single-window architecture with these, which they found simpler and, on pedestrian images, apparently much more accurate.

Orientation histograms themselves were not new. The paper traces them through Freeman and Roth’s hand-gesture histograms to their maturity in Lowe’s SIFT descriptor, which adds spatial pooling and normalization, and to Belongie, Malik and Puzicha’s shape contexts, which used log-polar cells but initially counted edge pixels without orientations. What it challenged was the prevailing use of such descriptors at sparse keypoints. Its authors reported informal experiments suggesting that the best keypoint-based approaches would have false positive rates at least one to two orders of magnitude higher than a dense grid, mainly because no keypoint detector they knew fired reliably on human body structures. The feature descriptors article places SIFT and its relatives.

The evaluation situation also motivated the paper. On the MIT pedestrian database, front and back views of people in city scenes with a limited range of poses, the new detectors gave essentially perfect results, so the existing benchmark could no longer separate good methods from better ones.

Contribution

The abstract and the summary make four claims:

  • Dense HOG for detection. Grids of histogram-of-oriented-gradient descriptors, computed on a dense grid of cells and normalized over overlapping blocks, significantly outperform the existing feature sets for human detection, including wavelets. The paper presents them as a dense counterpart of SIFT-style descriptors whose power, in its authors’ view, had been overshadowed by the success of sparse keypoint methods.
  • A stage-by-stage study. Each step of the computation was varied and measured. The conclusion, repeated in the abstract and the summary, is that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping blocks all matter.
  • A new dataset. The INRIA person dataset, with over 1,800 annotated person images and a much wider range of poses and backgrounds, was released for research.
  • A large gain over wavelets. The summary states that the approach reduces false positive rates by more than an order of magnitude relative to the best Haar wavelet detector of Mohan et al.

Method

The detector is described in the paper as a chain: optional gamma and colour normalization, gradient computation, magnitude-weighted orientation voting into cells, contrast normalization over overlapping blocks, concatenation of the block vectors over the window, and a linear SVM deciding person or non-person. The HOG article explains each step and the formulas; this section records what the paper fixed and what it left out.

All results in the paper’s parameter study are referred to one default detector: RGB colour with no gamma correction; the centred [−1,0,1][-1, 0, 1] gradient filter with no smoothing; linear voting into 9 orientation bins over 0°–180°; 16×1616 \times 16-pixel blocks of four 8×88 \times 8-pixel cells; a Gaussian spatial window with σ=8\sigma = 8 pixels; L2-Hys block normalization (the clipped L2 norm of SIFT); a block stride of 8 pixels, so each cell is covered four times; a 64×12864 \times 128 window; and a linear SVM. For colour images, gradients are computed per channel and each pixel keeps the one with the largest norm. Votes are interpolated bilinearly between neighbouring bin centres in both orientation and position. The paper tested two block geometries, rectangular (R-HOG) and circular log-polar (C-HOG), and reports both.

The classifier was SVMLight with a soft margin, C=0.01C = 0.01, slightly modified to use less memory on large dense vectors. Training followed a two-round recipe:

  1. Train a preliminary detector on the positive windows and a fixed set of random negative patches.
  2. Search the negative training photos exhaustively with it, and add every false positive, the “hard examples”, to the negatives.
  3. Retrain on the enlarged set. If the hard examples are too many, subsample them so that the final training descriptors fit into 1.7 GB of RAM.

Retraining improved the default detector by 5% at 10−410^{-4} false positives per window; further rounds made little difference, so the paper used one.

The paper concentrates on feature extraction and treats the rest of the detector briefly. Its Figure 1 caption says only that the window is scanned at all positions and scales and that conventional non-maximum suppression runs on the output pyramid. It prints no scan stride, no pyramid scale step and no merging rule. Those appear in the archived dataset page and in Dalal’s thesis (both below), and the sliding window detection and non-maximum suppression articles cover them.

The INRIA Person Dataset

The paper describes its new dataset in one paragraph. It contains 1,805 images of people at 64×12864 \times 128 pixels, cropped from a varied set of personal photos. The people are usually standing; the text says they appear in any orientation, the caption of Figure 2 that they are always upright; and they appear in a wide range of poses and against backgrounds that include crowds, and many are bystanders taken from the photos’ backgrounds, which the authors argue avoids a pose bias. Of the 1,805, the authors selected 1,239 as positive training examples, 2,478 with their left-right reflections; 1,218 photos without people supplied the negatives, starting with a fixed set of 12,180 randomly sampled patches. The paper does not state the size of the test set. The remainder of the 1,805 crops is 566.

Later documents give more detail, not all of it consistent with the paper:

  • Dalal’s thesis describes the collection: 498 images taken with a personal digital camera over eight months in many places and weathers, plus 497 images from the Graz-01 dataset with new annotations. Every upright person was annotated with a centre point and a bounding box; the crops were rescaled to 64×12864 \times 128 windows in which the subject is about 96 pixels tall, leaving a margin of about 16 pixels. The thesis gives 1,208 positive training windows and 566 test windows, both doubled by reflection, with 1,218 negative training images and 453 negative test images. The training count differs from the paper’s 1,239 while the implied test count agrees. The dataset table of Dollár et al.’s 2012 survey also lists 1,208 training and 566 test pedestrians.
  • The ECCV 2006 paper uses the same static training and test sets and gives them as 2,416 training and 1,132 test images, which are the thesis’s counts with reflections (1,208 and 566, each doubled).
  • The official dataset page, cited here from an archived copy of 2008 (as of October 2026, the archive’s captures of the page since late 2021 are all error responses), adds that a few images came from web searches, and that only upright people taller than 100 pixels are marked. It distributes the data in two forms: the original images with annotation files in the PASCAL challenge format, and the normalized crops “as used in the CVPR paper”. The normalized training positives are 96×16096 \times 160 pixels, a 16-pixel margin around the 64×12864 \times 128 window, and the test positives 70×13470 \times 134, a 3-pixel margin; the page explains the margins as a way to avoid boundary effects and tells users to take the centred 64×12864 \times 128 window. It also fixes the training protocol the paper left open: 10 random windows per negative training photo, hard examples defined as windows scoring above zero, and a hard-negative search over a pyramid that starts at the original size with a scale ratio of 1.2 between levels and a window stride of 8 pixels, the same sampling the page says is used to take negative windows from the test images. The whole set was a 970 MB download.

Evaluation Methodology

The paper evaluated window classifiers, not complete detectors. Its plots are detection error tradeoff (DET) curves: miss rate, 1−recall1 - \text{recall}, against false positives per window (FPPW), on log-log axes, a format that, the authors note, was used extensively in speech recognition and in NIST evaluations. They note that DET curves carry the same information as ROC curves but separate small probabilities more clearly. As a single reference point they used the miss rate at 10−410^{-4} FPPW, which they call arbitrary but no more so than the area under the ROC curve, and nearly every number in the paper is a difference in miss rate at that point.

Two conversions in the paper connect this measure to detection. First, in a multiscale detector 10−410^{-4} FPPW corresponds to a raw error rate of about 0.8 false positives per 640×480640 \times 480 image, before non-maximum suppression removes some of them. Second, the DET curves are shallow: for the default detector at 10−410^{-4} FPPW, the paper says, every 1% absolute (9% relative) reduction in miss rate is worth reducing FPPW at constant miss rate by a factor of 1.57. So small differences in miss rate correspond to large differences in false alarms.

The paper does not say how it counted windows for the first conversion. The script below counts the 64×12864 \times 128 windows a scan tests in images of the two sizes the paper mentions, with an 8-pixel stride and three pyramid scale steps, and converts each count to false positives at 10−410^{-4} FPPW.

import math

def window_count(width, height, scale_step, stride=8, win_w=64, win_h=128):
    """Number of 64 x 128 windows tested over an image pyramid (no upsampling).

    Level k has size floor(width / s) x floor(height / s) with s = scale_step**k;
    windows are placed every `stride` pixels on every level they fit in."""
    total, s = 0, 1.0
    while True:
        lw, lh = math.floor(width / s), math.floor(height / s)
        if lw < win_w or lh < win_h:
            return total
        total += ((lw - win_w) // stride + 1) * ((lh - win_h) // stride + 1)
        s *= scale_step

print("image    scale step  windows  false positives at 1e-4 FPPW")
for width, height in [(640, 480), (320, 240)]:
    for step in (1.05, 1.1, 1.2):
        n = window_count(width, height, step)
        print(f"{width}x{height}  {step:<10}  {n:7d}  {n * 1e-4:.2f}")

Output:

image    scale step  windows  false positives at 1e-4 FPPW
640x480  1.05          25132  2.51
640x480  1.1           13743  1.37
640x480  1.2            8077  0.81
320x240  1.05           2483  0.25
320x240  1.1            1404  0.14
320x240  1.2             887  0.09

The paper’s 0.8 false positives per 640×480640 \times 480 image matches a scan with a scale step of 1.2 and a stride of 8 pixels, the settings the dataset page gives for the hard-negative search and for the negative test windows; Dalal’s thesis gives a round figure of 10,000 windows for a 640×480640 \times 480 image. With the finer step of 1.05 that Dalal’s thesis later used for testing, the same per-window rate gives about three times as many false alarms per image. This is our reconstruction; the paper does not state the scan behind its figure. The paper’s other count, about 4,000 windows for a 320×240320 \times 240 image, matches none of these settings, so it implies a different scan, which the paper does not describe.

The two conversions are also consistent with each other on one point, by our arithmetic: a 1% absolute reduction is 9% relative only if the miss rate is about 11%, which is the default detector’s miss rate implied by the 89% recall the paper quotes in its gradient study.

Results

Comparison with other descriptors

The paper’s Figure 3 compares its final detectors with its own implementations of three other feature sets, all with the same SVM training and hard-example rounds:

  • Generalized Haar wavelets, an extended version of the MIT features: rectified responses of 9×99 \times 9 and 12×1212 \times 12 oriented first- and second-derivative box filters at 45° intervals, plus the second-derivative xyxy filter.
  • PCA-SIFT, after Ke and Sukthankar: gradient patches projected onto a principal-component basis learned from positive training images, computed on 16×1616 \times 16 blocks with the same derivative scale and overlap as the HOG blocks.
  • Shape contexts, simulated as C-HOG with a single orientation bin; the best layout had 16 angular and 3 radial intervals with radii from 2 to 8 pixels. Both gradient-weighted (G-ShapeC) and binary edge-presence (E-ShapeC) voting were tried, with the edge threshold chosen automatically (about 20–50 grey levels).

On the MIT test set the HOG detectors gave near-perfect separation, and on INRIA they reduced FPPW by at least an order of magnitude relative to the other feature sets. The paper’s own wavelets beat the MIT wavelets, which the authors attribute to the added second-order filters and to contrast-normalizing the output vector. The MIT panel also plots the best parts-based and monolithic detectors of Mohan et al., interpolated from their paper, with the authors’ warning that the comparison is not exact: they did not know how the MIT database had been split into training and test parts, and the MIT negative images were not available.

At 10−410^{-4} FPPW on INRIA, the paper reports these differences from linear R-HOG or C-HOG:

Variant Change in miss rate as printed
C-HOG instead of R-HOG Very similar; C-HOG slightly better
R2-HOG (R-HOG plus oriented second derivatives, twice the length) 2% better
Gaussian-kernel SVM instead of linear About 3% better, at much higher run time
EC-HOG (binary edge voting instead of gradient magnitude) 5% worse
Shape contexts, edges or gradients (no orientation bins) 33% worse, even with extra spatial or radial bins
PCA-SIFT Poor; no figure given in the text

For PCA-SIFT the authors offer an explanation: many more principal vectors (80 of 512) had to be kept than in Ke and Sukthankar’s keypoint setting to capture the same proportion of the variance, possibly because spatial registration is weaker without a keypoint detector. The kernel SVM was trained on the hard examples of the linear detector, because the kernel detector produced too few false positives of its own to help.

The parameter study

Section 6 of the paper varies one stage at a time from the default detector. Each row below is a result the paper prints, as a change in miss rate at 10−410^{-4} FPPW unless stated otherwise:

Stage (section) Comparison Effect as printed
Colour (6.1) Grayscale instead of RGB or LAB 1.5% worse (RGB and LAB comparable)
Gamma (6.1) Square-root compression per channel 1% better
Gamma (6.1) Log compression 2% worse
Gradient scale (6.2) Gaussian smoothing σ=2\sigma = 2 instead of none Recall falls from 89% to 80%
Gradient mask (6.2) Cubic-corrected [1,−8,0,8,−1][1, -8, 0, 8, -1] About 1% worse
Gradient mask (6.2) 2×22 \times 2 diagonal masks 1.5% worse
Gradient mask (6.2) Uncentred [−1,1][-1, 1] 1.5% worse
Vote weight (6.3) Square root of magnitude Slightly worse
Vote weight (6.3) Binary edge presence 5% worse
Orientation bins (6.3) Number of bins over 0°–180° Large gains up to about 9, little beyond
Orientation range (6.3) Signed (0°–360°), even with doubled bins Worse
Block overlap (6.4) Stride 4 (16-fold area coverage) instead of 16 (none) 4% better in the text; “around 5%” in the Figure 4(d) caption
Cell and block size (6.4, Fig. 5) 3×33 \times 3 blocks of 6×66 \times 6 cells Best, with a 10.4% miss rate
Block window (6.4) Gaussian, σ=0.5×\sigma = 0.5 \times block width 1% better
Multiple block types (6.4) Several cell and block sizes combined Around 3% better, at much greater length
Block shape (6.4) Vertical (2×12 \times 1) or vertical plus horizontal pairs 1% worse than 2×22 \times 2; horizontal pairs alone clearly worse
C-HOG angular bins (6.4) 12 instead of 4 1.3% worse
Normalization (6.4) L1-norm instead of L2-Hys, L2 or L1-sqrt (all equal) 5% worse
Normalization (6.4) No normalization 27% worse
Centre-surround (6.4) Per-cell normalization by Gaussian-pooled energy, σ=1\sigma = 1 cell 2% worse
Window margin (6.5) 8-pixel margin (48×11248 \times 112) instead of 16 6% worse in the text; “about 3%” in the Figure 4(e) caption
Classifier (6.6) Gaussian-kernel SVM About 3% better

Two pairs of numbers disagree within the paper, and the table keeps both. For block overlap, the text of Section 6.4 gives 4% for going from no overlap to a stride of 4, while the caption of Figure 4(d) says the overlap decreases the miss rate by around 5%; the discussion in Section 6.7 gives a third figure for a related comparison, that the four-fold redundant normalization of each cell in the standard detector raises recall from 84% to 89%. For the window margin, Section 6.5 says that shrinking the margin from 16 to 8 pixels costs 6%, and the caption of Figure 4(e) says about 3%. The figure plots a third window, 56×12056 \times 120, that the text does not mention. The paper also reports that keeping the 64×12864 \times 128 window but enlarging the person in it, which also shrinks the margin, costs about as much, even though the person is then seen at higher resolution.

Several findings came with interpretations that the paper offers as plausible rather than tested. Smoothing hurts, the authors argue, because much of the information is in abrupt edges at fine scales, so gradients should be taken at the finest scale and blurred only after orientation voting. Unsigned orientations win for people because clothing and background colours make the sign of a contrast uninformative, though the paper notes that the sign helps substantially for some other classes, such as cars and motorbikes. The best cells, 6–8 pixels wide whatever the block size, are about as wide as a limb in the training images, which the authors call an interesting coincidence. Blocks that are too large adapt poorly to local conditions, and 1×11 \times 1 blocks, which normalize over orientations alone, throw away spatial information. For C-HOG, at least two radial bins (centre and surround) and four angular bins were needed, a central radius of 4 pixels was best, and the expansion factor for later radii hardly mattered; the authors suggest thinking of C-HOG as advanced centre-surround coding rather than as a log-polar context descriptor. They report results only for C-HOG with an undivided central cell, similar to GLOH, and promise a technical report with further details. The regularizing constant ϵ\epsilon in the normalization was needed because descriptors are evaluated on empty patches too, but results were insensitive to its value over a large range.

What the detector looks at

The paper’s Figure 6 visualizes a trained linear R-HOG detector: the average gradient image over the training positives, the largest positive and largest negative SVM weight in the block centred on each pixel, and a test image’s descriptor weighted by the positive and the negative weights. The authors read it as showing that the detector cues mainly on silhouette contours, especially the head, shoulders and feet, and that the most important cells are those on the contour normalized with respect to blocks lying just outside it, on the background. In other words, they conclude, despite the cluttered backgrounds the detector relies on the contrast of the silhouette against the background, not on internal edges or on the silhouette’s contrast against the person. Gradients inside the person, especially vertical ones, count as negative evidence, which the authors explain as suppressing false positives where long vertical lines would trigger head and leg cells. The same analysis explains, in their account, why overlapping blocks help where a single centre-surround normalization does not: what matters is that each cell is normalized relative to several pooling regions at different offsets, not the pooling scale.

Speed

The paper’s only timing is in its conclusions: the linear detector processed a 320×240320 \times 240 scale-space image, about 4,000 detection windows, in under a second. The kernel SVM improved accuracy at much higher run time, which the paper does not quantify. The authors listed a coarse-to-fine or rejection-chain HOG detector as a way to speed it up.

Versions and Releases

The thesis. Dalal’s PhD thesis, “Finding People in Images and Videos”, defended at INP Grenoble on 17 July 2006 under the supervision of Bill Triggs and Cordelia Schmid, is the extended version of the CVPR paper and of its sequel. Its Chapter 4 repeats and extends the HOG study, including a bar-filter variant and a centre-surround variant, and gives the descriptor computation step by step, with trilinear voting in an appendix. It also contains much that the paper does not:

  • Evaluation of complete detectors. Besides DET curves for window classifiers, the thesis evaluates the full detector, scan and merging included, with recall-precision curves and average precision. It reports an average precision of 0.53 for the extended wavelets and 0.25 for edge-based shape contexts on INRIA, against about 0.74–0.755 for linear R-HOG; the R-HOG value is given as 0.755 in one sentence and as 0.74 in the next, where a Gaussian kernel raises it to 0.762.
  • Multiscale localization. Chapter 5 treats detections as points in position and log-scale and merges them by mean shift, and measures the effect of the scan’s scale step and stride. The sliding window detection article reports those tables.
  • Other classes. Section 4.6 tunes HOG for the ten classes of the PASCAL Visual Object Classes challenge 2006. Its Table 4.2 lists signed orientations (18 bins over 0°–360°) for cars, motorbikes, buses, bicycles, cows and sheep, and unsigned ones for people, horses, cats and dogs, with the window size and normalization chosen per class.
  • Motion and parts. Chapter 6 develops the motion descriptors of the ECCV 2006 paper, and Chapter 7 presents work in progress on combining head, torso and leg detectors. The appendix documents the INRIA static and moving person datasets and a person-part dataset.

The thesis also differs from the CVPR paper on some details: the INRIA training count (1,208 against 1,239, discussed above), the best shape-context layout (12 angular and 3 radial intervals with radii from 4 to 16 pixels, against 16, 3, and 2 to 8 in the paper), and its Table 4.2, which lists square-root-compressed RGB for the person class where the paper’s default detector used no gamma correction, although the thesis text reports that RGB and square-root RGB give similar results for people. For the MIT experiments, the thesis states that the negatives came from the INRIA negative images, since the MIT set has none; the paper does not say which negatives it used.

The ECCV 2006 paper. Dalal, Triggs and Schmid’s “Human Detection Using Oriented Histograms of Flow and Appearance” carried out the paper’s plan to add motion. It applies the HOG machinery to differences of dense optical flow, which cancel much of the camera and background motion, and tests several codings, some of motion boundaries and some of the relative motion of neighbouring regions. Combined with the static HOG descriptor, its best detector reduced the false alarm rate by a factor of 10 relative to the best appearance-only detector, for example to 1 per 20,000 windows at an 8% miss rate on its Test Set 1, unseen shots from the same DVDs as its training data; the abstract cites more than 4,400 human examples over its two test sets. The motion boundary histograms it introduced later became one of the descriptors of dense trajectories.

Code. Dalal released the detector as the INRIA Object Detection and Localization Toolkit, as Linux binaries only, together with the modified SVMLight, which he called “SVM dense”. The archived software page of June 2007 gives the latest release as 10 April 2006, describes the binaries as part of the larger toolkit written for his thesis, and permits research use only, prohibiting commercialization. The OpenCV implementation, whose source says its vectors are compatible with this toolkit, is described in the HOG article.

Impact

The IEEE PAMI Technical Committee lists the paper as one of two winners of the 2015 Longuet-Higgins Prize, which recognizes CVPR papers from ten years earlier with significant impact on computer vision research; the other was Buades, Coll and Morel’s “A non-local algorithm for image denoising”.

Gradient histograms became the standard ingredient of window-based detectors. Felzenszwalb, McAllester and Ramanan’s 2008 deformable part model describes its root filter as equivalent to a Dalal–Triggs model and adds part filters on HOG features at twice the resolution. Their paper also identifies a rigid HOG template from Dalal and Triggs as the winner of the 2006 PASCAL person detection challenge, with an average precision of 0.16, and reports roughly doubling that. The dataset had a parallel career. In Dollár, Wojek, Schiele and Perona’s 2012 survey of pedestrian detection, most of the sixteen detectors compared were trained on INRIA, and the authors observe that nearly all modern detectors use some form of gradient histogram. In their evaluation, which counts errors per image over full images, HOG remained somewhat competitive, with a mean rank of 8.2 among 14 detectors ranked on 28 subsets of four pedestrian datasets other than INRIA.

The survey also questioned the paper’s way of measuring. Evaluating classifiers on cropped windows, as Dalal and Triggs did, ignores the choices that turn a classifier into a detector, such as stride, scale step and non-maximum suppression, and tests different windows from those a full scan sees. In the authors’ copy of the survey, per-window and full-image results on INRIA were only weakly correlated, the ranking of methods changed substantially, and two detectors were found to have overfit window-boundary artefacts in cropped data, discovered only through full-image evaluation. The survey therefore advocates per-image metrics, such as the log-average miss rate over 0.01 to 1 false positives per image. Dalal’s thesis had already made a related point: a finer scan raised average precision while leaving the per-window error rate unchanged, so per-window rates were not a direct measure of detection performance.

When convolutional networks took over detection, the HOG-based deformable part model was the baseline they were measured against, as in the R-CNN paper. The descriptor kept other uses, which the HOG article surveys: cascades, tracking features and targets for self-supervised pretraining.

Limitations

The authors named three directions for further work, each pointing at a limit of the 2005 detector: speed, since the linear detector needed under a second per small image and kernel SVMs much more; motion, which the static detector ignores; and articulation, since a fixed template had proven hard to beat for fully visible pedestrians but a parts-based model with more local spatial invariance should do better on humans in general. The ECCV 2006 paper and deformable part models took up the last two.

Other limitations follow from the paper’s design:

  • Window-level evaluation. Every number in the paper is a window classifier’s miss rate at a fixed FPPW. The scan and the merging step, which decide how many false alarms a user sees, are neither described nor evaluated, and the paper’s conversion to false positives per image depends on an unstated scan.
  • Upright, mostly visible people. The dataset marks only upright people taller than 100 pixels, and the single 64×12864 \times 128 template assumes one aspect ratio and a roughly fixed layout. Occluded, seated, small or strongly articulated people are outside the test.
  • One dataset, one class. The parameter study was carried out on INRIA people only. The paper itself warns that signed gradients help for other classes, and the thesis’s per-class tuning showed that several defaults change with the object.
  • Inexact baselines. The comparison with the MIT detectors relies on points interpolated from another paper with unknown splits and unavailable negatives, and the comparison feature sets are the authors’ own reimplementations.
  • Inconsistent counts and figures. The training-set size differs between the paper and the thesis, and two of the paper’s effect sizes differ between text and captions.

What Came After

Within a year, faster HOG detectors appeared. Zhu, Avidan, Yeh and Cheng (2006) built the rejection-chain detector the paper had proposed, using integral histograms and boosted variable-size blocks; the HOG article describes it and Felzenszwalb et al.’s reduced HOG features. The motion descriptors of the ECCV 2006 paper fed into action recognition through dense trajectories. Deformable part models carried HOG to PASCAL VOC categories, and pedestrian detection moved to larger benchmarks with per-image evaluation, where the INRIA training set remained in use. The object detection article follows the line from hand-designed features to learned ones.

Related

  • Histogram of Oriented Gradients

    A dense descriptor that summarizes an image window by histograms of gradient orientations in small cells, contrast-normalized over overlapping blocks; how it is computed, why its parameters are what they are, and how its variants and implementations differ.

  • Sliding Window Detection

    Detecting objects by running a fixed-size window classifier at every position and scale of an image, how the stride, the pyramid's scale step and the window shape set what is found and what it costs, and how the idea survives inside modern detectors.

  • Deformable Part Models

    The HOG-based object detector of Felzenszwalb, Girshick, McAllester and Ramanan, which scores a coarse root filter plus higher-resolution part filters that may move at a quadratic cost, matched with generalized distance transforms and trained as a latent SVM; its model, training, releases, results and relation to convolutional networks.

  • Object Detection

    Finding every object of a set of categories in an image and returning, for each one, a bounding box, a class label and a confidence score.

  • Distinctive Image Features from Scale-Invariant Keypoints

    David Lowe's 1999 ICCV paper and 2004 IJCV journal paper that introduced SIFT, a scale- and rotation-invariant local feature, and used it to recognize objects in cluttered, partly occluded images.

  • Dense Trajectories

    A hand-crafted video representation that samples points densely, tracks them with median-filtered optical flow for 15 frames, and describes the space-time tube around each track; the improved version cancels camera motion with a homography.

References

  1. Dalal, N. & Triggs, B. (2005). Histograms of Oriented Gradients for Human Detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 1, 886–893.
  2. Dalal, N. (2006). Finding People in Images and Videos. PhD thesis, Institut National Polytechnique de Grenoble. HAL tel-00390303.
  3. Dalal, N., Triggs, B. & Schmid, C. (2006). Human Detection Using Oriented Histograms of Flow and Appearance. European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, 428–441.
  4. INRIA Person Dataset (official dataset page, pascal.inrialpes.fr/data/human; archived copy of 18 February 2008).
  5. Dalal, N. INRIA Object Detection and Localization Toolkit (official software page, pascal.inrialpes.fr/soft/olt; archived copy of 7 June 2007).
  6. Papageorgiou, C. & Poggio, T. (2000). A Trainable System for Object Detection. International Journal of Computer Vision, 38(1), 15–33.
  7. Mohan, A., Papageorgiou, C. & Poggio, T. (2001). Example-Based Object Detection in Images by Components. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(4), 349–361.
  8. Viola, P., Jones, M. J. & Snow, D. (2003). Detecting Pedestrians Using Patterns of Motion and Appearance. IEEE International Conference on Computer Vision (ICCV), 2, 734–741.
  9. Mikolajczyk, K., Schmid, C. & Zisserman, A. (2004). Human Detection Based on a Probabilistic Assembly of Robust Part Detectors. European Conference on Computer Vision (ECCV), Lecture Notes in Computer Science, 69–82.
  10. Lowe, D. G. (2004). Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision, 60(2), 91–110.
  11. Ke, Y. & Sukthankar, R. (2004). PCA-SIFT: A More Distinctive Representation for Local Image Descriptors. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2, 506–513.
  12. Belongie, S., Malik, J. & Puzicha, J. (2001). Matching Shapes. IEEE International Conference on Computer Vision (ICCV), 1, 454–461.
  13. Felzenszwalb, P., McAllester, D. & Ramanan, D. (2008). A Discriminatively Trained, Multiscale, Deformable Part Model. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1–8.
  14. Dollár, P., Wojek, C., Schiele, B. & Perona, P. (2012). Pedestrian Detection: An Evaluation of the State of the Art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(4), 743–761.
  15. IEEE Computer Society, Technical Committee on Pattern Analysis and Machine Intelligence. Longuet-Higgins Prize.