Eulerian Video Magnification for Revealing Subtle Changes in the World

“Eulerian Video Magnification for Revealing Subtle Changes in the World” by Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Frédo Durand and William Freeman, of MIT CSAIL and Quanta Research Cambridge, appeared in ACM Transactions on Graphics 31(4), the proceedings of SIGGRAPH 2012. It showed that two kinds of change too small for the eye to see in a video, the slight reddening of skin with each heartbeat and motions of a fraction of a pixel, can both be made visible by one simple operation: decompose each frame into spatial frequency bands, band-pass the time series of every pixel in every band, multiply the result by a factor $\alpha$ and add it back. No motion is estimated. The paper explained, with a first-order Taylor expansion, why amplifying temporal intensity changes at fixed pixels magnifies motion, and derived the bound that limits it. It belongs to computational photography in the broad sense of combining ordinary capture with computation, but it works along time: it exaggerates what changes in a video rather than improving a single picture. Its successor, which amplifies phase instead (Wadhwa et al., 2013), calls it linear Eulerian video magnification.

Problem

Invisible signals in ordinary video. The paper starts from the limited spatio-temporal sensitivity of human vision. Skin colour varies slightly with blood circulation, which earlier work had already exploited to measure the pulse from a camera: Verkruysse, Svaasand and Nelson (2008) recorded plethysmographic signals at more than a metre with a consumer camera in ambient light, strongest in the green channel, and Poh, McDuff and Picard (2010) automated pulse-rate measurement from webcam video of a face by face tracking and blind source separation of the colour channels. Poh et al., in Wu et al.’s account, focus on extracting a single number, the heart rate. Wu et al. wanted a picture instead: the pulse signal shown at every location of the face, so that, for example, an asymmetry in facial blood flow could be seen.

Lagrangian motion magnification. For motion, the reference was Liu, Torralba, Freeman, Durand and Adelson (2005), “Motion Magnification”, also from MIT. Their system registered the frames with an affine warp to remove camera shake, tracked feature points, clustered their trajectories into layers whose motions are correlated over time, interpolated a dense motion field per layer, segmented the pixels into those layers, multiplied the displacements of a layer chosen by the user, rendered the layers back to front, and filled the holes the larger motion uncovered by texture synthesis. Their magnification factors ranged from 4 to 40, and processing the swing-set sequence took 10 hours end to end. Wu et al. call this the Lagrangian approach, after the description of fluids that follows particles along their trajectories, and name its costs: accurate motion estimation is expensive and hard to make artifact-free at occlusion boundaries and with complicated motion, and motion segmentation and inpainting add further complexity.

Temporal filtering of video. Per-pixel temporal filters had been used before. Fuchs et al. (2010), in the paper’s account, used them to dampen temporal aliasing of motion and discussed high-pass filtering of motion, mostly for non-photorealistic effects and large motions. Wu et al. set out to make imperceptible motions visible and to explain when per-pixel filtering amounts to motion magnification.

Contribution

The introduction lists four contributions:

• Eulerian processing reveals invisible changes. Spatio-temporal processing of standard monocular video makes nearly invisible changes visible, and for the amplification values useful in practice it magnifies motion in natural videos without explicit motion estimation. The method is robust and runs in real time.
• An analysis linking temporal filtering to motion. Amplifying a temporal band-pass signal magnifies motion, and the approximation behind it is best suited to small displacements and low spatial frequencies.
• One framework for colour and motion. The same pipeline amplifies spatial motion and purely temporal changes such as the pulse, and it can be tuned to particular temporal frequencies, which, the authors note, Lagrangian methods do not support.
• Eulerian and Lagrangian compared under noise. An analytic and empirical comparison of the errors of the two approaches.

The name follows the fluid-dynamics analogy: in an Eulerian description, properties such as pressure and velocity are observed at fixed points in space as the fluid flows past. Here the fixed points are pixels, and what is observed and amplified is their colour over time.

Method

Pipeline. Each frame is decomposed into spatial frequency bands; every pixel of every band is passed through the same temporal band-pass filter; the filtered signal is multiplied by $\alpha$, possibly a different value per band, and added to the original; and the pyramid is collapsed into the output frame (Section 2, Figure 2). Bands are treated differently for two reasons: they may have different signal-to-noise ratios, and the linear approximation behind motion magnification fails for high spatial frequencies. When the aim is only to raise the temporal signal-to-noise ratio by pooling pixels, as for the pulse, the frames are low-pass filtered and downsampled (a Gaussian pyramid); in general a full Laplacian pyramid (Burt & Adelson, 1983) is used. The image pyramid article explains both. The pyramids are built with a separable five-tap binomial filter (Section 4). All processing is done in the YIQ colour space, so the chrominance channels I and Q can be attenuated before conversion back to RGB to reduce colour clipping.

Why amplifying intensity changes moves things. Section 3.1 considers a 1D signal translating by $\delta(t)$, so that $I(x,t) = f(x + \delta(t))$ and $I(x,0) = f(x)$. The goal is the magnified signal

$$\hat I(x,t) = f\big(x + (1+\alpha)\,\delta(t)\big).$$

A first-order Taylor expansion, the same linearization behind the brightness constancy equation of optical flow methods such as Lucas–Kanade and Horn–Schunck, gives

$$I(x,t) \approx f(x) + \delta(t)\,\frac{\partial f(x)}{\partial x}.$$

A temporal band-pass filter that removes the constant term $f(x)$ and passes $\delta(t)$ leaves $B(x,t) = \delta(t)\,\partial f / \partial x$. Adding it back with gain $\alpha$,

$$\tilde I(x,t) = I(x,t) + \alpha B(x,t) \approx f(x) + (1+\alpha)\,\delta(t)\,\frac{\partial f(x)}{\partial x} \approx f\big(x + (1+\alpha)\,\delta(t)\big),$$

where the last step assumes the first-order expansion still holds for the larger displacement $(1+\alpha)\delta(t)$. The intensity change at a fixed pixel is the image gradient times the displacement; scaling the change scales the displacement. If the filter attenuates the temporal frequency components $\delta_k(t)$ of the motion by factors $\gamma_k$, each component is magnified by its own factor $\alpha_k = \gamma_k \alpha$ (Eqs. 7–8), which is how the temporal filter selects which motions are amplified.

The bound. The approximation fails when the image changes quickly relative to the amplified displacement. Section 3.2 takes $f(x) = \cos(\omega x)$, writes $\beta = 1 + \alpha$, and requires

$$\cos(\omega x) - \beta\omega\delta(t)\sin(\omega x) \approx \cos\big(\omega x + \beta\omega\delta(t)\big),$$

which holds when $\cos(\beta\omega\delta(t)) \approx 1$ and $\sin(\beta\omega\delta(t)) \approx \beta\omega\delta(t)$ (Eqs. 12–13). The paper states that these small-angle approximations hold to within 10% for $\beta\omega\delta(t) \le \pi/4$, naming the sine term as the leading approximation, for which $\sin(\pi/4) \approx 0.9 \cdot \pi/4$. With spatial wavelength $\lambda = 2\pi/\omega$ this gives Eq. 14:

$$(1+\alpha)\,\delta(t) < \frac{\lambda}{8}.$$

At that point, however, the cosine term is $\cos(\pi/4) \approx 0.71$, 29% away from 1. In our reading the 10% figure holds for the sine term only, so the bound is a guide for the motion term rather than a 10% bound on the whole approximation; the code example below shows the contrast error it leaves. Figure 4 shows the effect on sinusoids, where exceeding the bound by factors of 2 and 4 gives mild, then severe, artifacts, and Figure 5 plots the error against wavelength with the bound marked on each curve. The paper calls Eq. 14 a guideline and leaves the cutoff as a user parameter, since violating it can be perceptually preferable in some videos.

Multiscale attenuation. Because the bound depends on wavelength, $\alpha$ is applied fully to bands whose wavelengths are above a cutoff $\lambda_c$ and reduced for finer bands (Section 3.3, Figure 6). The user makes four choices (Section 4): the temporal band-pass filter; $\alpha$; the cutoff $\lambda_c$; and the form of attenuation below it, either $\alpha = 0$ for all $\lambda < \lambda_c$ (suggested for the pulse) or a linear ramp of $\alpha$ down to zero (suggested for motion). Sharp edges are therefore amplified less than smooth structure, which the authors found still looked convincing, an effect Freeman, Adelson and Heeger (1991) had used to create the illusion of motion from still images.

The released MATLAB code (version 1.1, March 2013, whose change log lists only build and README changes since version 1.0 of September 2012) implements the ramp per pyramid level: it assigns the coarsest band a representative wavelength of one third of the frame diagonal, halves it at each finer level, sets the amplification of a level from the bound with an “exaggeration factor” of 2 that, in the code’s comment, boosts it above the bound for better visualization, and zeroes the finest and coarsest bands. These details are in the code, not the paper.

Temporal filters. The choice is application dependent (Section 4, Figure 9). A broad passband suits motion magnification; a narrow one gives a cleaner result for colour amplification of blood flow. Ideal band-pass filters, implemented with the discrete cosine transform, are used for colour amplification because of their sharp cutoffs and for videos where one frequency is selected, such as a single guitar string. Low-order IIR filters serve both purposes and suit real-time use; the text says the IIR band-pass was generally built from two first-order low-pass IIR filters with cutoffs $\omega_l$ and $\omega_h$, while the caption of Figure 9 says a Butterworth design converts a user-specified band into a second-order IIR filter for the real-time application, and labels its Butterworth panel with the subway band. The released code suggests the two descriptions are the same construction: its “Butterworth” variant subtracts two first-order Butterworth low-pass filters designed from the band edges $f_l$ and $f_h$, and its other IIR variant subtracts two exponential smoothers, $y_1[n] - y_2[n]$ with $y_i[n] = r_i x[n] + (1 - r_i) y_i[n-1]$, set by coefficients $r_1 > r_2$ rather than by frequencies. The difference of two first-order low-pass filters is, by our reading, a second-order band-pass, which reconciles text and caption. The code’s ideal filter uses the FFT rather than the DCT of Figure 9 and keeps only the positive-frequency bins of the band before taking the real part of the inverse transform; by our reading it therefore passes half the amplitude of the band, so the effective $\alpha$ of that variant is half the value passed in.

Noise and spatial pooling. The pulse changes a pixel by less than its noise, so amplifying single pixels amplifies noise. Section 5 assumes zero-mean white noise of variance $\sigma^2$, stationary in space, so a spatial low-pass filter of radius $r$ reduces the noise power to $\sigma'^2 = k\sigma^2 / r^2$, with $k$ set by the filter shape. Given a prior $S(r)$ on the signal power at the filter’s cutoff, the filter size needed to reveal the signal solves

$$S(r) = \frac{k\,\sigma^2}{r^2}.$$

In Figure 11, white Gaussian noise added to the face video hides the pulse after the binomial filter of size 20 used for the original but not after one of size 80 chosen with this equation. The image noise article covers the noise sources.

Eulerian against Lagrangian. Appendix A, with a longer derivation in the supplemental material, compares the errors of the two approaches in 1D, using a two-tap temporal filter for the Eulerian side and point-wise brightness constancy, $\tilde\delta = I_t / I_x$, for the Lagrangian side. Without noise, the paper gives the Lagrangian error as $\tfrac12(1+\alpha)\delta^2 I_{xx} I_x$, from the second-order term the motion estimate ignores, and the Eulerian error as $\big|\tfrac12(1+\alpha)^2\delta^2 I_{xx} - \tfrac12(1+\alpha)\delta^2 I_{xx} I_x\big|$, which grows quadratically with $\alpha$ and with the curvature $I_{xx}$, that is, at high spatial frequencies. The factor $I_x$ in the second-order terms appears to be a slip, present in the supplement too: dividing $I_t \approx \delta I_x + \tfrac12\delta^2 I_{xx}$ by $I_x$ gives $\delta + \tfrac12\delta^2 I_{xx}/I_x$, not $\delta + \tfrac12\delta^2 I_{xx}$ (Eq. 18). Carried through, our derivation gives $\tfrac12(1+\alpha)\delta^2 I_{xx}$ for the Lagrangian and $\tfrac12\alpha(1+\alpha)\delta^2 I_{xx}$ for the Eulerian error, which leaves the paper’s qualitative conclusions unchanged: the Eulerian error is quadratic in $\alpha$, the Lagrangian one linear. With additive noise both errors contain $(1+\alpha)n_t$, the amplified temporal noise, but only the Lagrangian error has terms in the spatial derivative of the noise $n_x$, which enters through the motion estimate. The authors conclude that Eulerian magnification is preferable for small amplifications and larger noise.

Results

Implementation. The results were produced with non-optimized MATLAB code on a six-core machine with 32 GB of RAM, taking a few minutes per video. A prototype written in C++, CPU only, processed 640 × 480 video at 45 frames per second on a standard laptop and let the user change the amplification and cutoffs on live video, which the authors describe as a microscope for temporal variations.

Parameters. Table 1 lists the settings for every video:

Video	$\alpha$	$\lambda_c$	$\omega_l$ (Hz)	$\omega_h$ (Hz)	Frame rate (Hz)
face	100	1000	0.83	1	30
face2 pulse	120	960	0.83	1	30
face2 motion	20	80	0.83	1	30
baby2	150	600	2.33	2.67	30
baby	10	16	0.4	3	30
wrist	10	80	0.4	3	30
camera	120	20	45	100	300
guitar, low E string	50	40	72	92	600
guitar, A string	100	40	100	120	600
subway	60	90	3.6	6.2	30
shadow	5	48	0.5	10	30

Colour. For face and face2, two adults with lighter and darker skin, the two finest Laplacian levels had $\alpha = 0$, which amounts to downsampling and low-pass filtering to suppress noise and quantization. An ideal band-pass filter of 0.83–1 Hz (50–60 beats per minute) and $\alpha \approx 100$ then showed periodic green-to-red variation at the heart rate and the way blood perfuses the face (Figure 1). baby2, a newborn filmed at the nursery of Winchester Hospital, came with readings from a hospital-grade monitor; the authors report that these confirmed their heart-rate estimate and that the extracted colour signal matched the monitor’s photoplethysmogram, without giving numbers.

Motion. In wrist, a band around the 0.88 Hz (53 bpm) heart rate with $\alpha = 10$ and a user-drawn mask made the movement of the radial and ulnar arteries visible in video from a point-and-shoot camera (Figure 7). In camera, a 300 Hz video of a digital SLR taking about one photo per second revealed the vibration caused by its flipping mirror; a laser pointer fixed to the camera, projecting about four metres away, oscillated in sync with the magnified motion, which confirmed the source. In the synthetic sim4 and in guitar, ideal filters tuned to one oscillation frequency amplified only one blob or one string (Figure 10 shows sim4). baby, face2 and subway exaggerated visible but subtle motion; subway was deliberately pushed beyond the bound to show the artifacts. Most examples are oscillatory, which the authors explain by such motion generally lasting longer and having smaller amplitudes, but shadow shows that non-periodic motion within the passband is also amplified: the sun’s shadow moving linearly over 15 seconds.

Noise comparison. On a synthetic 2D cosine oscillating at 2 Hz with an amplitude of 0.1 pixel and additive Gaussian noise, the measured errors followed the predicted dependence on noise and amplification (Figure 12). The Eulerian result was better for small amplification and high noise, the Lagrangian error grew with spatial noise while the Eulerian error hardly changed, and spatial regularization of the motion estimate did not change this significantly.

Discrepancies. For face2 the text describes motion amplification with a second-order IIR filter “with slow roll-off regions”, chosen as a broader passband, yet Table 1 lists the same 0.83–1 Hz cutoffs for face2 motion as for face2 pulse. A second-order filter with those cutoffs passes more than the ideal one (the Figure 9 caption says so of its second-order filters), so the two need not conflict, but the released code and video use a different band, 0.5–10 Hz. The result videos released on the project page carry parameters in their file names, which match the code’s script for reproducing the results, and several differ from Table 1: face and face2 pulse with $\alpha = 50$ on a Gaussian-pyramid level (4 and 6) rather than a Laplacian pyramid with $\alpha \approx 100$ or 120, camera with $\alpha = 150$, guitar with $\alpha$ of 75 and 150 and $\lambda_c = 10$, subway with $\lambda_c = 45$, and wrist with $\lambda_c = 16$ and the coefficient-based IIR filter ($r_1 = 0.4$, $r_2 = 0.05$). The code’s README acknowledges that in some cases the parameters in its script do not exactly match those reported in the paper, having been refined through experimentation; anyone reproducing a result should start from the released settings.

Example

The script below applies the method’s two halves to synthetic video with known ground truth. In part (a), a rendered disk on a grey background brightens by 0.2% at 1 Hz, half of one 8-bit grey level, under per-pixel noise of standard deviation 0.02 and 8-bit quantization; a four-level Gaussian pyramid pools the pixels, an ideal 0.8–1.2 Hz filter extracts the pulse and $\alpha = 100$ amplifies it. Part (b) translates a cosine grating of wavelength $\lambda$ by $\delta(t) = 0.25\sin(2\pi \cdot 2t)$ pixels and amplifies it with an ideal 1.5–2.5 Hz filter. With one $\alpha$ for every band a Laplacian pyramid sums to the identity, so the frames are filtered directly. The output displacement is read from the phase of the grating, and the error is measured against the truly shifted grating $f(x + (1+\alpha)\delta(t))$, relative to the true frame change. It needs NumPy and SciPy and runs in under a second.

import numpy as np
from scipy.ndimage import convolve1d

fs = 30.0                                    # frame rate (Hz)
k5 = np.array([1, 4, 6, 4, 1]) / 16.0        # 5-tap binomial filter, as in the paper

def blur_down(img):                          # one Gaussian-pyramid step
    img = convolve1d(convolve1d(img, k5, axis=-1, mode="reflect"), k5, axis=-2, mode="reflect")
    return img[..., ::2, ::2]

def ideal_bandpass(video, lo, hi):           # per-pixel ideal temporal filter (FFT along time)
    F = np.fft.rfft(video, axis=0)
    f = np.fft.rfftfreq(video.shape[0], d=1 / fs)
    F[(f < lo) | (f > hi)] = 0
    return np.fft.irfft(F, n=video.shape[0], axis=0)

def tone(signal, f0):                        # amplitude of the f0 component of a time series
    t = np.arange(signal.shape[0]) / fs
    return 2 * abs(np.mean(signal * np.exp(-2j * np.pi * f0 * t)))

# (a) Colour: a rendered disk whose brightness changes by 0.2% at 1 Hz (60 bpm),
# with sensor noise and 8-bit quantization. Gaussian pyramid, ideal 0.8-1.2 Hz band, alpha = 100.
def colour_demo(seed, levels=4, alpha=100.0):
    rng = np.random.default_rng(seed)
    T, N = 300, 64
    t = np.arange(T) / fs
    yy, xx = np.mgrid[:N, :N]
    disk = ((yy - 31.5) ** 2 + (xx - 31.5) ** 2 < 20 ** 2).astype(float)
    clean = 0.5 + 0.002 * np.sin(2 * np.pi * 1.0 * t)[:, None, None] * disk
    video = np.round(255 * (clean + rng.normal(0, 0.02, clean.shape))) / 255
    pooled = video
    for _ in range(levels):
        pooled = blur_down(pooled)                   # 64x64 -> 4x4
    band = alpha * ideal_bandpass(pooled, 0.8, 1.2)
    c = pooled.shape[-1] // 2
    floor = lambda s: np.mean([tone(s, f) for f in np.arange(2.0, 10.0, 0.1)])  # off-band noise
    return (tone(video[:, 32, 32], 1.0), floor(video[:, 32, 32]),
            tone(pooled[:, c, c], 1.0), floor(pooled[:, c, c]),
            tone(band[:, c, c], 1.0), band[:, 0, 0].std())

print("(a) colour: true change 0.0020, 8-bit step 0.0039, noise sigma 0.02")
a = colour_demo(seed=0)
print("                     amplitude at 1 Hz   noise floor")
print(f"single pixel         {a[0]:17.4f} {a[1]:13.4f}")
print(f"level 4 (4x4 frame)  {a[2]:17.4f} {a[3]:13.4f}")
print(f"amplified change (alpha=100)   {a[4]:.3f}")
print(f"amplified noise, corner std    {a[5]:.3f}")
amps = np.array([colour_demo(s)[2] for s in range(10)])
print(f"level-4 amplitude, seeds 0-9   {amps.min():.4f} to {amps.max():.4f}")

# (b) Motion: a grating of wavelength lam translated by delta(t) = 0.25 sin(2 pi 2 t) pixels.
# A single alpha for every band makes the pyramid sum to the identity, so frames are filtered directly.
def motion_demo(lam, alpha, d0=0.25, f0=2.0, T=90, W=256):
    t = np.arange(T) / fs
    x = np.arange(W)
    w = 2 * np.pi / lam
    d = d0 * np.sin(2 * np.pi * f0 * t)[:, None]
    video = 0.5 + 0.25 * np.cos(w * (x + d))                   # T x W (rows are identical)
    out = video + alpha * ideal_bandpass(video, 1.5, 2.5)
    truth = 0.5 + 0.25 * np.cos(w * (x + (1 + alpha) * d))
    c = (out - 0.5) @ np.exp(-1j * w * x) / (W / 2)            # complex grating coefficient per frame
    shift = np.angle(c) / w                                     # displacement read from the phase
    gain = np.abs(c).max() / 0.25                               # contrast of the output grating
    err = np.sqrt(np.mean((out - truth) ** 2) / np.mean((truth - video[0]) ** 2))
    return tone(shift, f0) / d0, gain, err

print("\n(b) motion: delta = 0.25 px at 2 Hz, ideal 1.5-2.5 Hz band")
print(" lam  alpha  (1+a)d/(lam/8)  shift gain  1+alpha  contrast  rel. error")
for lam in (16, 64):
    for alpha in (3, 7, 15, 31, 63):
        m, g, e = motion_demo(lam, alpha)
        r = (1 + alpha) * 0.25 / (lam / 8)
        print(f"{lam:4d} {alpha:6d} {r:15.2f} {m:11.2f} {1+alpha:8d} {g:9.2f} {e:11.2f}")

Output (NumPy 2.3.5, SciPy 1.16.3; identical on two runs):

(a) colour: true change 0.0020, 8-bit step 0.0039, noise sigma 0.02
                     amplitude at 1 Hz   noise floor
single pixel                    0.0033        0.0019
level 4 (4x4 frame)             0.0018        0.0001
amplified change (alpha=100)   0.179
amplified noise, corner std    0.019
level-4 amplitude, seeds 0-9   0.0018 to 0.0018

(b) motion: delta = 0.25 px at 2 Hz, ideal 1.5-2.5 Hz band
 lam  alpha  (1+a)d/(lam/8)  shift gain  1+alpha  contrast  rel. error
  16      3            0.50        3.87        4      1.07        0.16
  16      7            1.00        7.06        8      1.26        0.34
  16     15            2.00       11.19       16      1.85        0.69
  16     31            4.00       14.90       32      3.28        1.48
  16     63            8.00       17.39       64      6.32        3.92
  64      3            0.12        3.99        4      1.00        0.04
  64      7            0.25        7.93        8      1.02        0.08
  64     15            0.50       15.43       16      1.07        0.17
  64     31            1.00       28.18       32      1.27        0.34
  64     63            2.00       44.73       64      1.85        0.69

In part (a), a single pixel cannot show the pulse: its 1 Hz estimate is about as large as the estimates at frequencies where there is no signal, so it is mostly noise. Four pyramid levels average over roughly a 16 × 16 neighbourhood and cut the noise floor by more than an order of magnitude, after which the 1 Hz amplitude is measured consistently over ten noise seeds. It falls slightly short of the true 0.002 because the blur mixes some background into the centre of the disk. The change is far below one grey level, yet the noise dithers the quantization and pooling averages it out, which is what lets the pyramid recover it. Amplified 100 times, the change reaches a peak amplitude of about 0.18 in intensity, plainly visible. The amplified noise in the passband of the background reaches about 0.02, as large as the original per-pixel noise: amplification is linear in noise as well as in signal, which is why the pooling has to come first.

In part (b), the rows of the two wavelengths with the same ratio $(1+\alpha)\delta/(\lambda/8)$ have the same contrast and error, as the analysis predicts: the bound depends only on the amplified displacement relative to the wavelength. Below the bound the displacement is magnified almost exactly $1+\alpha$ times. At the bound the magnification is about 12% short and the grating’s contrast has risen by about a quarter, the cosine error noted above: the linear method adds $(1+\alpha)\delta\,\partial f/\partial x$ to the frame, which shifts the sinusoid but also makes it larger. Past the bound the displacement saturates, since the phase of $\cos\omega x - \beta\omega\delta\sin\omega x$ can never exceed a quarter period, while the contrast keeps growing; in a real 8-bit video that excess is what clips to black and white. A multiscale implementation avoids this by lowering $\alpha$ in the fine bands where the bound is exceeded, at the price of magnifying fine detail less.

Impact

The paper made video magnification a practical tool. Where the Lagrangian method needed hours of computation and careful segmentation, the Eulerian one is a pyramid, a per-pixel filter and a sum, fast enough to run live on a laptop in 2012. The project page released MATLAB code (version 1.0 in September 2012, 1.1 in March 2013) for non-commercial research use, executables that need no MATLAB licence, and the result videos with their parameters. The method received wide press coverage, which the project page collects.

The paper also shaped a research programme at MIT that the group’s Video Magnification page groups as magnifying motion and colour changes, and analysing small motions in video. Its linear model showed that a band-passed intensity change at a pixel is, to first order, a measurement of motion; in our reading, the follow-up work used that idea in two directions: better ways to magnify, and ways to measure physical quantities from tiny motions.

Limitations

The paper names its own limits:

• Small motion only. The analysis holds for smooth images and small motions, and Eq. 14 ties the usable amplification to the spatial wavelength. Lagrangian methods, the authors write, work better for motions of fine point features and support larger amplification factors.
• Noise is amplified with the signal. Amplifying intensity changes amplifies noise in the passband linearly in $\alpha$, so revealing a weak signal requires spatial pooling sized to the noise (Eq. 15), and the Eulerian error grows quadratically with $\alpha$.
• Clipping. Large amplifications push intensities past the displayable range; attenuating the chrominance in YIQ reduces colour clipping.
• Manual tuning. The filter, $\alpha$, $\lambda_c$ and attenuation are chosen per video, often by trial, and masks were drawn by hand for face and wrist to keep irrelevant regions from being amplified.

Later work added two further limits. Wadhwa et al. (2013) show the linear method’s artifacts and clipped intensities on a crane swaying in the wind, and note that denoising the linear result with video denoisers cost more time, removed little of the medium-frequency noise on noisy sequences, and in some cases, denoising before magnification, removed the small motion itself. The project’s capture tips add that codecs exploiting inter-frame redundancy, such as H.264, can introduce temporal signals of their own under some settings, so independently compressed or uncompressed video is preferable, and they ask for a tripod to minimize extraneous motion; it follows from the method that any camera motion within the passband is amplified along with the subject’s. Large motions in the same scene, such as a person walking while a small motion is of interest, also defeat a method that amplifies every change in the passband; the group’s Video Magnification page lists later work on magnification in the presence of large motions.

What Came After

Phase-based video motion processing (2013). Wadhwa, Rubinstein, Durand and Freeman replaced the Laplacian pyramid with a complex steerable pyramid, a decomposition by scale and orientation with quadrature pairs (the image pyramid article describes steerable pyramids), and amplified the temporal variation of the local phase rather than of the intensity. By analogy with the Fourier shift theorem, a phase change in a localized band corresponds to local motion. Their summary table compares the two methods: the linear method is exact for linear intensity ramps, its bound is $(1+\alpha)\delta(t) < \lambda/8$ and noise is magnified; the phase-based method is exact for sinusoids, its bound for a pyramid with $n$ filters per octave is $\alpha\delta(t) < \lambda n/4$ ($\lambda/2$ for a half-octave pyramid, which they describe as an improvement by a factor of 4) and noise is translated rather than amplified, since only phases are changed. The price is a more overcomplete representation, 33 times the frame size for their eight-orientation half-octave pyramid, against 4/3 for the Laplacian pyramid; in their timing, processing a 512 × 512 video of 300 frames took 56 seconds with a two-orientation octave pyramid and 280 seconds with the half-octave one, in MATLAB on a four-core laptop.

Riesz pyramids (2014). At ICCP 2014 the same authors made phase-based magnification fast. The Riesz pyramid splits the image into non-oriented bands with an efficient, invertible replacement for the Laplacian pyramid, then takes an approximate Riesz transform of each band with two three-tap finite-difference filters, which gives a local phase and dominant orientation per pixel. It is less overcomplete than even the smallest complex steerable pyramid, gave results of comparable quality in a quarter of the time, and a single-core C++ implementation processed live 640 × 400 video at 25 frames per second.

Measuring from small motion. Balakrishnan, Durand and Guttag (CVPR 2013), citing Eulerian magnification as an inspiration, measured heart rate from the subtle head motion caused by the influx of blood at each beat. They tracked features on the head, separated their trajectories into component motions with principal component analysis, chose the component whose temporal spectrum best matched a heartbeat, and on 18 subjects reported heart rates nearly identical to an electrocardiogram. The group’s Video Magnification page lists later analyses in the same vein, among them recovering sound from vibrating objects in video (2014) and estimating material properties from small motions (2015).

Learning-based motion magnification (2018). Oh, Jaroensri, Kim, Elgharib, Durand, Freeman and Matusik (ECCV 2018) replaced the hand-designed filters with a convolutional neural network made of an encoder that acts as the spatial decomposition, a manipulator that magnifies the difference between the shape representations of two frames, and a decoder that reconstructs the frame. Real magnified video pairs do not exist, so they trained on synthetic pairs: 7,000 objects segmented from PASCAL VOC pasted onto 200,000 COCO backgrounds and moved by small amounts, with sub-pixel motion and quantization modelled carefully. They report fewer ringing artifacts and better noise behaviour than previous methods on real videos, and that their representation can be combined with the usual temporal filters up to moderate magnifications. Their limitations section says the network, trained on two frames, degrades with temporal filters at high magnification and small motion, and can miss very small motions that the phase-based method reveals.

Colour amplification and the pulse. For colour, the linear method remained the recommended tool: as of October 2026 the project page advises the phase-based pipeline for amplifying motion and the linear pipeline of this paper for amplifying colour. Measuring the pulse from facial colour, remote photoplethysmography, predates the paper (Verkruysse et al., 2008; Poh et al., 2010) and grew into a field of its own; the paper’s contribution there is the visualization of the pulse signal at every pixel, not the measurement.