SlowFast

SlowFast is a video recognition network made of two 3D convolutional pathways that read the same clip at different frame rates: a Slow pathway with few frames and many channels for what is in the scene, and a Fast pathway with many frames and few channels for how it moves, joined by lateral connections. Feichtenhofer, Fan, Malik and He (Facebook AI Research) introduced it at ICCV 2019 for action recognition and for spatio-temporal action detection on AVA.

Problem Addressed

Most 3D networks treat time like a third spatial axis. The authors argue that this does not suit video: slow motions are more likely than fast ones, and the identity of what is seen (a hand, a person) changes more slowly than the motion it performs (waving, clapping), so each can get its own frame rate. Two-stream networks split by modality (RGB and optical flow) with the same backbone twice; SlowFast splits by frame rate, learns motion from raw frames and makes one pathway much lighter.

The authors cite studies of primate retinal ganglion cells as partial inspiration: by their summary, about 80% are parvocellular (fine spatial detail and color, slow responses) and about 15–20% magnocellular (fast temporal changes, no sensitivity to detail or color). The Fast pathway is meant as the M-cell analogue, an analogy they call “rough and premature”.

Architecture

In the notation of temporal modeling, the Slow pathway reads $T$ frames at stride $\tau$ , spanning $T\tau$ raw frames, and the Fast pathway reads $\alpha T$ frames of the same span at stride $\tau / \alpha$ , with $\beta$ times the Slow channels. Each pathway ends in global average pooling; the two vectors are concatenated and classified by a fully connected layer. Models are named $T \times \tau$ plus backbone, for example “SlowFast 4×16, R50”.

Slow Pathway

Any clip-level convolutional model can serve; the defining property is a large stride, typically $\tau = 16$ (about 2 frames per second of 30-fps video). In the paper’s example it reads $T = 4$ frames of a 64-frame clip, with no temporal downsampling. It uses temporal kernels larger than 1 only in res4 and res5, because earlier temporal convolutions lowered accuracy, which the authors attribute to fast-moving objects shifting too far between sparse frames for a small spatial receptive field.

Fast Pathway

High frame rate: $\alpha$ times more frames, typically $\alpha = 8$ .
High temporal resolution throughout: no temporal pooling or time-strided convolution before global pooling, so every feature map keeps $\alpha T$ frames, and temporal kernels in every block (a $5 \times 7^2$ conv1 and a $3 \times 1^2$ first convolution in each residual block).
Low channel capacity: typically $\beta = 1/8$ . Because a layer’s cost grows roughly quadratically with channel width, the Fast pathway takes “∼20% of the total computation” in the authors’ instantiations despite reading eight times as many frames.

Weaker Fast inputs (half resolution, gray-scale, frame differences, optical flow) all still beat Slow-only; gray-scale nearly matched RGB (75.5% against 75.6%, Table 5c).

Lateral Connections

One connection after each of pool1, res2, res3 and res4 fuses Fast features into the Slow pathway. A Slow feature map is $\{T, S^2, C\}$ and a Fast one $\{\alpha T, S^2, \beta C\}$ , so time must shrink by $\alpha$ . The paper tried:

Time-to-channel (TtoC): reshape to $\{T, S^2, \alpha\beta C\}$ , packing $\alpha$ frames into channels.
Time-strided sampling (T-sample): keep one frame in $\alpha$ , giving $\{T, S^2, \beta C\}$ .
Time-strided convolution (T-conv): a $5 \times 1^2$ 3D convolution with $2\beta C$ outputs and temporal stride $\alpha$ .

The output is summed with or concatenated to the Slow features. With the 4×16 R50 model on Kinetics-400 (Table 5a), every option beat Slow-only, and T-conv was best:

Fusion	Top-1	Top-5	GFLOPs
Slow-only	72.6	90.3	27.3
Fast-only	51.7	78.5	6.4
SlowFast, no lateral (late concatenation of pooled outputs)	73.5	90.3	34.2
TtoC, sum	74.5	91.3	34.2
TtoC, concatenation	74.3	91.0	39.8
T-sample	75.4	91.8	34.9
T-conv (default)	75.6	92.1	36.1

The Fast pathway alone reaches 51.7% yet adds 3.0 points to Slow-only. Among channel ratios (Table 5b), $\beta$ = 1/6 and 1/8 were best (75.8% and 75.6%); even 1/32 added 1.6 points for 1.3 GFLOPs.

Example Instantiation

Table 1 of the paper specifies SlowFast 4×16, R50 with $\alpha = 8$ , $\beta = 1/8$ (kernels $t \times h \times w$ , channels after the comma; outputs $T \times S^2$ at the $224^2$ training crop):

Stage	Slow pathway	Fast pathway	Output, Slow / Fast
data layer	stride 16 in time	stride 2 in time	$4 \times 224^2$ / $32 \times 224^2$
conv1	$1 \times 7^2$ , 64, spatial stride 2	$5 \times 7^2$ , 8, spatial stride 2	$4 \times 112^2$ / $32 \times 112^2$
pool1	$1 \times 3^2$ max, spatial stride 2	same	$4 \times 56^2$ / $32 \times 56^2$
res2 (×3)	$1 \times 1^2$ , 64; $1 \times 3^2$ , 64; $1 \times 1^2$ , 256	$3 \times 1^2$ , 8; $1 \times 3^2$ , 8; $1 \times 1^2$ , 32	$4 \times 56^2$ / $32 \times 56^2$
res3 (×4)	$1 \times 1^2$ , 128; $1 \times 3^2$ , 128; $1 \times 1^2$ , 512	$3 \times 1^2$ , 16; $1 \times 3^2$ , 16; $1 \times 1^2$ , 64	$4 \times 28^2$ / $32 \times 28^2$
res4 (×6)	$3 \times 1^2$ , 256; $1 \times 3^2$ , 256; $1 \times 1^2$ , 1024	$3 \times 1^2$ , 32; $1 \times 3^2$ , 32; $1 \times 1^2$ , 128	$4 \times 14^2$ / $32 \times 14^2$
res5 (×3)	$3 \times 1^2$ , 512; $1 \times 3^2$ , 512; $1 \times 1^2$ , 2048	$3 \times 1^2$ , 64; $1 \times 3^2$ , 64; $1 \times 1^2$ , 256	$4 \times 7^2$ / $32 \times 7^2$
head	global average pool of each pathway, concatenate, fully connected		number of classes

The table omits the laterals. In PySlowFast (commit 287ec00), the 4×16 R50 config uses $\alpha = 8$ and a 5-frame lateral kernel, while the 8×8 R50 config reads 32 Fast frames at stride 2 with $\alpha = 4$ and a 7-frame kernel; the paper does not print $\alpha$ for its 8×8 and 16×8 models.

Where the Computation Goes

The NumPy code counts the multiply-adds of Table 1’s convolutions at the $256^2$ view size of Table 5, including the T-conv laterals and the extra Slow input channels they add. Table 1 does not say where each stage’s spatial stride sits; the code puts it in the $1 \times 3^2$ convolution, as PySlowFast’s configs do.

import numpy as np

def conv(T, S, c_in, c_out, kt, ks):
    """Multiply-adds of a 3D convolution with T x S x S output positions."""
    return T * S * S * c_out * c_in * kt * ks * ks

def pathway(T, width, temporal_k, extra_in=(0, 0, 0, 0), S=256):
    """ResNet-50 pathway of Table 1: width = conv1 channels (64 Slow, 8 Fast).
    temporal_k: temporal size of the first 1x1 conv in res2..res5 (1 or 3).
    extra_in: channels concatenated from lateral connections into res2..res5.
    The spatial stride of res3..res5 sits in the 1x3^2 conv, as in PySlowFast,
    so each stage's first 1x1 conv still runs at the previous resolution."""
    S = S // 2                                    # conv1, spatial stride 2
    stem_kt = 5 if width < 64 else 1              # Fast conv1 is 5x7^2, Slow 1x7^2
    total = conv(T, S, 3, width, stem_kt, 7)
    S = S // 2                                    # pool1
    c_in = width
    for stage, (blocks, kt) in enumerate(zip([3, 4, 6, 3], temporal_k)):
        mid, out, S_in = width * 2 ** stage, 4 * width * 2 ** stage, S
        if stage > 0:
            S = S // 2                            # spatial stride 2 in res3..res5
        for b in range(blocks):
            cin_b = c_in + extra_in[stage] if b == 0 else out
            S_a = S_in if b == 0 else S
            total += conv(T, S_a, cin_b, mid, kt, 1) + conv(T, S, mid, mid, 1, 3) + conv(T, S, mid, out, 1, 1)
            if b == 0:
                total += conv(T, S, cin_b, out, 1, 1)  # projection shortcut
        c_in = out
    return total

alpha, beta, T = 8, 1 / 8, 4
slow_only = pathway(T, 64, [1, 1, 3, 3])
fast_only = pathway(alpha * T, int(64 * beta), [3, 3, 3, 3])
# time-strided conv laterals (5x1^2, 2*beta*C outputs, stride alpha) after pool1, res2, res3, res4
fast_ch = [8, 32, 64, 128]
S_lat = [64, 64, 32, 16]
lat = sum(conv(T, s, c, 2 * c, 5, 1) for c, s in zip(fast_ch, S_lat))
slow_fused = pathway(T, 64, [1, 1, 3, 3], extra_in=[2 * c for c in fast_ch])
total = slow_fused + fast_only + lat
print(f"Slow-only : {slow_only / 1e9:5.1f} GFLOPs   (Table 5a: 27.3)")
print(f"Fast-only : {fast_only / 1e9:5.1f} GFLOPs   (Table 5a:  6.4)")
print(f"SlowFast  : {total / 1e9:5.1f} GFLOPs   (Table 5a, T-conv: 36.1)")
print(f"Fast pathway share       : {fast_only / total:.1%}")
print(f"Fast + laterals + fusion : {(total - slow_only) / total:.1%}")

Slow-only :  27.3 GFLOPs   (Table 5a: 27.3)
Fast-only :   6.9 GFLOPs   (Table 5a:  6.4)
SlowFast  :  36.1 GFLOPs   (Table 5a, T-conv: 36.1)
Fast pathway share       : 19.1%
Fast + laterals + fusion : 24.5%

This reproduces the printed Slow-only and SlowFast costs; with the stride in the first $1 \times 1^2$ convolution instead, the count falls to 24.4 and 32.1. It overestimates Fast-only (6.9 against 6.4) for reasons we could not find. From the printed numbers, Fast-only is 6.4 / 36.1 ≈ 18% of SlowFast (our arithmetic).

Training

Kinetics models were trained from random initialization; with ImageNet pretraining they performed similarly (±0.3%), and the recipe trained the Non-local paper’s 3D ResNet-50 from scratch to 73.5% against 73.4% pretrained (Table 6). Each example is one clip at a random position (written as $\alpha T \times \tau$ frames, although Section 3.1 gives the clip as $T \times \tau$ frames, 64 in Table 1), with a random $224 \times 224$ crop or its flip from frames whose shorter side is drawn from [256, 320] ([256, 340] for R101). The appendix specifies:

synchronized SGD on 128 GPUs, 8 clips per GPU (mini-batch 1,024), batch normalization statistics within each GPU’s 8 clips;
a half-period cosine schedule, $\eta_n = \eta \cdot 0.5\,[\cos(n\pi / n_{\max}) + 1]$ , with $\eta = 1.6$ and linear warm-up over the first 8k iterations;
256 epochs (60k iterations) on Kinetics-400 when $T \le 4$ , 196 when $T > 4$ ; twice as long with $\eta = 0.8$ on Kinetics-600;
momentum 0.9, weight decay $10^{-4}$ , dropout 0.5;

Charades models are fine-tuned from Kinetics with per-class sigmoid outputs.

Example: Fast Motion That a Slow Pathway Cannot See

The PyTorch example (torch 2.14.1, CPU, about 20 s) builds a tiny SlowFast on $(N, C, T, H, W)$ tensors with $\tau = 8$ , $\alpha = 8$ , $\beta = 1/8$ and concatenated T-conv laterals, and trains it against a Slow-only model with the same Slow pathway. In 32-frame noisy clips, a square grows or shrinks (slow classes), or a dot moves 4 pixels per frame right or left on a 32-pixel ring, returning to its start every 8 frames, so at stride 8 the two fast classes look identical: a motionless dot.

import numpy as np
import torch
import torch.nn as nn

torch.set_num_threads(4)
RAW, SIZE, TAU, ALPHA = 32, 32, 8, 8              # raw frames, frame size, Slow stride, speed ratio
CLASSES = ["grow", "shrink", "fast right", "fast left"]

def make_clips(n_per_class, rng):
    """Slow classes: a square grows or shrinks over the clip.
    Fast classes: a dot moves 4 px/frame right or left on a 32-px ring, so it is
    back where it started every 8 frames and looks static to a stride-8 sampler."""
    X = np.zeros((4 * n_per_class, 1, RAW, SIZE, SIZE), np.float32)
    y = np.repeat(np.arange(4), n_per_class)
    for i, c in enumerate(y):
        cy, cx = rng.integers(8, 24, size=2)
        for t in range(RAW):
            if c < 2:                             # half-side grows 2 -> 7 or shrinks 7 -> 2
                h = round(2 + 5 * (t if c == 0 else RAW - 1 - t) / (RAW - 1))
                X[i, 0, t, cy - h:cy + h, cx - h:cx + h] = 1.0
            else:
                x = (cx + (4 if c == 2 else -4) * t) % SIZE
                X[i, 0, t, cy - 2:cy + 2, (np.arange(x - 2, x + 2)) % SIZE] = 1.0
    X += rng.normal(0, 0.3, X.shape).astype(np.float32)
    return torch.from_numpy(X), torch.from_numpy(y)

def c3(cin, cout, kt, stride_t=1, stride_s=2):
    return nn.Sequential(nn.Conv3d(cin, cout, (kt, 3, 3), (stride_t, stride_s, stride_s),
                                   (kt // 2, 1, 1), bias=False), nn.BatchNorm3d(cout), nn.ReLU())

class TinySlowFast(nn.Module):
    def __init__(self, width=16, beta=1 / 8, fast=True):
        super().__init__()
        self.fast = fast
        f = int(width * beta)                     # Fast channels: 2, 4, 8
        lat = 2 * f if fast else 0                # time-strided conv lateral: 2*beta*C channels
        self.slow = nn.ModuleList([c3(1, width, 1), c3(width + lat, 2 * width, 1),
                                   c3(2 * width + 2 * lat, 4 * width, 3)])
        if fast:
            self.fastp = nn.ModuleList([c3(1, f, 5), c3(f, 2 * f, 3), c3(2 * f, 4 * f, 3)])
            self.lat = nn.ModuleList([nn.Conv3d(c, 2 * c, (5, 1, 1), (ALPHA, 1, 1), (2, 0, 0))
                                      for c in (f, 2 * f)])
        self.fc = nn.Linear(4 * width + (4 * f if fast else 0), 4)

    def forward(self, clip, log=False):
        s = clip[:, :, ::TAU]                     # Slow: T = 4 frames at stride tau = 8
        x = clip[:, :, ::TAU // ALPHA]            # Fast: alpha*T = 32 frames at stride tau/alpha = 1
        for k in range(3):
            s = self.slow[k](s)
            if self.fast:
                x = self.fastp[k](x)
                if k < 2:                         # fuse Fast -> Slow by concatenation
                    s = torch.cat([s, self.lat[k](x)], dim=1)
            if log:
                print(f"stage {k + 1}: Slow {tuple(s.shape)}" +
                      (f"  Fast {tuple(x.shape)}" if self.fast else ""))
        feats = [s.mean((2, 3, 4))] + ([x.mean((2, 3, 4))] if self.fast else [])
        return self.fc(torch.cat(feats, 1))

def flops_by_part(model, clip):
    """Multiply-adds of every Conv3d, grouped by Slow, Fast and lateral modules."""
    counts = {}
    def hook(name):
        def f(m, inp, out):
            k = m.weight[0].numel()                # C_in * kt * kh * kw
            counts[name] = counts.get(name, 0) + out.numel() * k
        return f
    hs = [m.register_forward_hook(hook(n.split(".")[0])) for n, m in model.named_modules()
          if isinstance(m, nn.Conv3d)]
    model(clip)
    for h in hs:
        h.remove()
    return counts

torch.manual_seed(0)
probe = torch.zeros(1, 1, RAW, SIZE, SIZE)
sf, so = TinySlowFast().eval(), TinySlowFast(fast=False).eval()
with torch.no_grad():
    sf(probe, log=True)
    c = flops_by_part(sf, probe)
    total = sum(c.values())
    print({k: f"{v / 1e6:.2f}M" for k, v in c.items()},
          f"Fast share {c['fastp'] / total:.1%}, Fast + laterals {(c['fastp'] + c['lat']) / total:.1%}")
    print(f"Slow-only model: {sum(flops_by_part(so, probe).values()) / 1e6:.2f}M multiply-adds")

def train_eval(fast, seed):
    torch.manual_seed(seed)
    rng = np.random.default_rng(seed)
    Xtr, ytr = make_clips(150, rng)
    Xte, yte = make_clips(50, rng)
    model = TinySlowFast(fast=fast)
    opt = torch.optim.Adam(model.parameters(), 3e-3)
    for epoch in range(8):
        model.train()
        for idx in torch.randperm(len(ytr)).split(32):
            opt.zero_grad()
            nn.functional.cross_entropy(model(Xtr[idx]), ytr[idx]).backward()
            opt.step()
    model.eval()
    with torch.no_grad():
        pred = model(Xte).argmax(1)
    return [(pred[yte == k] == k).float().mean().item() for k in range(4)]

print("seed  model     " + "  ".join(f"{c:>10}" for c in CLASSES) + "     all")
for seed in range(3):
    for name, fast in (("Slow-only", False), ("SlowFast", True)):
        acc = train_eval(fast, seed)
        print(f"{seed:4d}  {name:9s} " + "  ".join(f"{a:10.2f}" for a in acc) + f"  {np.mean(acc):6.2f}")

stage 1: Slow (1, 20, 4, 16, 16)  Fast (1, 2, 32, 16, 16)
stage 2: Slow (1, 40, 4, 8, 8)  Fast (1, 4, 32, 8, 8)
stage 3: Slow (1, 64, 4, 4, 4)  Fast (1, 8, 32, 4, 4)
{'slow': '6.05M', 'fastp': '1.62M', 'lat': '0.08M'} Fast share 20.9%, Fast + laterals 22.0%
Slow-only model: 4.87M multiply-adds
seed  model           grow      shrink  fast right   fast left     all
   0  Slow-only       1.00        1.00        0.10        0.94    0.76
   0  SlowFast        1.00        1.00        1.00        1.00    1.00
   1  Slow-only       1.00        1.00        0.42        0.66    0.77
   1  SlowFast        1.00        1.00        1.00        1.00    1.00
   2  Slow-only       1.00        1.00        0.58        0.38    0.74
   2  SlowFast        1.00        1.00        1.00        1.00    1.00

The Fast pathway keeps all 32 frames at every stage; the Slow channel counts (20, 40) include the lateral channels. Both models learn grow and shrink. On the two fast classes, the Slow-only model’s inputs have the same distribution, so it guesses (0.10–0.94 per class over three seeds); SlowFast separates them in every seed, for about 60% more multiply-adds than Slow-only, a larger overhead than the paper’s because this toy’s Slow pathway is tiny.

Inference

For Kinetics, the authors averaged softmax scores over 10 uniformly spaced clips × 3 crops of $256 \times 256$ (shorter side 256), reporting cost as GFLOPs per view × views (action recognition explains why); for Charades they max-pooled scores over clips.

Action Detection on AVA

AVA labels each person in movie keyframes, one per second, with a box and possibly several actions; SlowFast was evaluated on 60 classes of AVA v2.1 by frame-mAP at IoU 0.5 (average precision). The detector is, in the authors’ description, similar to the two-stage Faster R-CNN with minimal changes for video, classifying actions rather than objects:

Backbone: SlowFast with res5 at spatial stride 1 and dilation 2.
RoI features: each keyframe box is replicated along time into a 3D RoI on the last res5 map, pooled by RoIAlign (from Mask R-CNN) in space and averaging in time, then max-pooled into a per-class sigmoid classifier.
Proposals: person boxes with confidence above 0.8 (91.1% recall, 90.7% precision) from a separately trained Faster R-CNN with a ResNeXt-101-FPN backbone, pretrained on ImageNet and COCO keypoint images and fine-tuned on AVA. Training uses ground-truth boxes.
Schedule: initialized from Kinetics classification models; 14k iterations (68 epochs), step decay, weight decay $10^{-7}$ ; inference on one clip centered on the keyframe.

Performance

Results as printed; Kinetics models have no pretraining, and Table 2 does not name its Kinetics-400 split:

Model	Dataset	Pretraining	Views	Result
SlowFast 4×16, R50	Kinetics-400	none	36.1 GFLOPs × 30	75.6% top-1, 92.1% top-5 (Table 2)
SlowFast 8×8, R50	Kinetics-400	none	65.7 GFLOPs × 30	77.0% top-1, 92.6% top-5 (Table 2)
SlowFast 16×8, R101+NL	Kinetics-400	none	234 GFLOPs × 30	79.8% top-1, 93.9% top-5 (Table 2)
SlowFast 16×8, R101+NL	Kinetics-600	none	234 GFLOPs × 30	81.8% top-1, 95.1% top-5 (Table 3)
Slow-only / SlowFast / SlowFast+NL, 16×8 R101	Charades	Kinetics-400	187 / 213 / 234 GFLOPs × 30	39.0 / 42.1 / 42.5 mAP (Table 4)
SlowFast+NL, 16×8 R101	Charades	Kinetics-600	234 GFLOPs × 30	45.2 mAP (Table 4)
SlowFast, 8×8 R101	AVA v2.1	Kinetics-400	one clip	26.3 val mAP (Table 7)
SlowFast+NL, 8×8 R101	AVA v2.1	Kinetics-600	one clip	27.3 val, 27.1 test mAP (Table 7)
SlowFast, 16×8 R101+NL	AVA v2.2	Kinetics-600	one clip	29.8 val mAP (Table 8)

Kinetics-600 models skip Kinetics-400 pretraining because the Kinetics-600 validation set overlaps the Kinetics-400 training set.

Accuracy against compute. Figure 2 plots Kinetics-400 top-1 (30 views) against GFLOPs per $256^2$ view for six Slow-only models (2×32 R50 to 16×8 R101) and their SlowFast counterparts. The Fast pathway raises each by the printed +1.7 to +3.4 points at small cost; SlowFast 8×8 R101 beats the Slow-only 16×8 R101 at lower cost.

Detection. On AVA (Table 9), adding the Fast pathway lifted a 4×16 R50 detector from 19.0 to 24.2 mAP, improving 57 of 60 categories, most of all “hand clap” (+27.7 AP), “swim” (+27.4) and “run/jog” (+18.8). Optical flow had added only +1.1 to +1.7 mAP to earlier AVA detectors, the authors note.

Strengths

Motion is learned end to end from raw frames, cheaply: no optical flow, and about a fifth of the computation.
The design is generic: other backbones, non-local blocks or a detection head can be attached.

Limitations

Compute. 234 GFLOPs × 30 views is about 7 TFLOPs per Kinetics video (our arithmetic), and training used 128 GPUs for up to 256 epochs.
Two fixed rates. $\tau$ and $\alpha$ are fixed per model, not adapted to content; in our reading, motion faster than the Fast pathway’s rate still aliases, and structure longer than a clip of a few seconds is left to score averaging.
External proposals. The AVA detector depends on a separately trained person detector and its threshold.

Historical Importance

SlowFast (2019) made frame rate an explicit design axis. In the authors’ experiments, a thin, high-frame-rate pathway learned from pixels outperformed the optical-flow approaches they compared, at about a fifth of the network’s computation.

Variants

Slow-only (“Slow” or “SlowOnly” in later code): the main baseline, a strong 3D ResNet itself (72.6% for 4×16 R50 at 27.3 GFLOPs, Table 5a).
Fast-only: 6.4 GFLOPs and 0.53M parameters, but 51.7%.
R101+NL: non-local blocks (Wang et al., 2018) on the fused Slow features of res4 only, giving the paper’s best results.
X3D (Feichtenhofer, 2020) expands a tiny 2D network axis by axis into an efficient single-pathway 3D network; see 3D convolutional networks.
MViT (Fan et al., 2021), from the same group, is a video transformer whose stages expand channels while reducing spatial resolution.

Implementations

The authors’ PySlowFast (Fan et al., 2020; Apache 2.0) implements SlowFast, Slow, C2D, I3D, Non-local networks, X3D and MViT, among others (as of October 2026, commit 287ec00 of 16 March 2026). Its model zoo lists, among others, Slow 4×16 R50 at 72.7% and SlowFast 8×8 R50 at 77.0% top-1 on Kinetics-400 (3 crops × 10 clips), the repository’s own figures.