Grad-CAM

Grad-CAM (Gradient-weighted Class Activation Mapping) produces a coarse heat map of where a convolutional neural network found evidence for a chosen output, such as the class “dog” in an image that also contains a cat. It takes the feature maps of one convolutional layer, weights each map by the spatial average of the gradient of the class score with respect to it, sums the weighted maps and keeps the positive part. Selvaraju, Cogswell, Das, Vedantam, Parikh and Batra presented it at ICCV 2017 as a generalization of class activation mapping (CAM), which Zhou, Khosla, Lapedriza, Oliva and Torralba had defined in 2016 for one particular architecture. Grad-CAM needs no change to the network and no retraining. It applies to classifiers with fully connected heads, such as VGG, and to captioning and visual question answering models, where the score can be a word or an answer.

Grad-CAM is a widely used member of a family of class activation mapping methods. Guided Grad-CAM sharpens it with a pixel-level map, Grad-CAM++ changes how the gradients are combined, Score-CAM replaces the gradients by the scores of masked inputs, and Chefer, Gur and Wolf adapted gradient weighting to vision transformers. The family’s place among explanation methods, and how explanations in general are tested, belong to neural network interpretability; this article covers the family in depth.

Problem

An image classifier returns a score per class but not the image regions behind it. Selvaraju et al. ask two things of a visual explanation for one class: it should be class-discriminative, localizing that class and not others in the same image, and high-resolution, showing fine detail. Pixel-space gradient methods such as guided backpropagation and the deconvnet of Zeiler and Fergus give fine detail, but in their Figure 1 the guided backpropagation maps for “tiger cat” and “boxer” on the same image are very similar. CAM is class-discriminative but works only for networks whose last convolutional maps go through global average pooling directly into the output layer, so applying it to another network means changing the architecture and retraining it. Grad-CAM keeps CAM’s class-discriminative map and drops the architectural requirement.

Inputs and Outputs

• Input: a trained network, an image, a target $y^c$ (any differentiable score: a class logit before the softmax, the log probability of a caption, an answer’s score), and a convolutional layer whose maps $A^k \in \mathbb{R}^{u \times v}$, $k = 1, \dots, K$, are used.
• Output: a nonnegative map $L^c \in \mathbb{R}^{u \times v}$ at the layer’s resolution, usually upsampled to the image and shown as a heat map. Selvaraju et al. note it is $14 \times 14$ for the last convolutional layer of VGG-16.
• Cost: one forward pass and one partial backward pass, down to the chosen layer.

Intuition

Each channel of a late convolutional layer responds to some visual pattern, such as a wheel, fur or a face, wherever it occurs. If the class score rises when a channel’s activations rise, that channel is evidence for the class, and the places where it fires are places where the evidence lies. Grad-CAM measures “rises when the activations rise” by the gradient of the score with respect to the channel, averaged over all positions, giving one importance weight per channel. The heat map is the importance-weighted sum of the channel maps, with negative evidence removed.

For a network that ends in global average pooling and one linear layer, the class score is already a weighted sum of channel averages, and the gradient of the score with respect to each activation is that channel’s weight divided by the number of positions. Grad-CAM then recovers CAM’s weights up to that constant. For any other network, the averaged gradient is a linear approximation of everything downstream of the layer, which Selvaraju et al. call a partial linearization.

Algorithm

1. Run the image forward and keep the maps $A^k$ of the chosen layer (by default, the last convolutional layer).
2. Set the gradient of the output to 1 for the target $y^c$ and 0 for every other output, and backpropagate to the chosen layer to get $\partial y^c / \partial A^k_{ij}$ (backpropagation stops there).
3. Average each channel’s gradient over positions to get its weight $\alpha^c_k$.
4. Form $\sum_k \alpha^c_k A^k$ and apply a ReLU.
5. Upsample bilinearly to the image size for display. For Guided Grad-CAM, multiply the upsampled map pointwise with the guided backpropagation map for the same class.
6. For localization, threshold the map and draw a box around the largest connected region (Selvaraju et al. use 15% of the maximum).

Mathematical Formulation

CAM. Let $f_k(x, y)$ be the activation of channel $k$ of the last convolutional layer at position $(x, y)$. Zhou et al. (§2) pool each channel to $F_k = \sum_{x,y} f_k(x, y)$ and compute the input to the softmax as $S_c = \sum_k w^c_k F_k$, with the bias set to zero. Exchanging the sums gives

$$S_c = \sum_{x,y} M_c(x, y), \qquad M_c(x, y) = \sum_k w^c_k f_k(x, y),$$

and $M_c$ is the class activation map: the class score before pooling, position by position. The derivation and its parameter count are worked through in fully connected layer. Zhou et al.’s text calls the pooling an average while their formula sums; the two differ by the number of positions.

Grad-CAM. For any network and a layer with $Z = u \cdot v$ positions, Selvaraju et al. define (Eq. 1 and 2)

$$\alpha^c_k = \frac{1}{Z} \sum_{i} \sum_{j} \frac{\partial y^c}{\partial A^k_{ij}}, \qquad L^c_{\text{Grad-CAM}} = \operatorname{ReLU}\Big( \sum_k \alpha^c_k A^k \Big).$$

The ReLU keeps features with a positive influence on $y^c$. The authors report that without it the maps sometimes highlight more than the class of interest and localize worse.

Reduction to CAM. Grad-CAM’s Eq. 3 writes CAM’s architecture with an averaging pool, $S_c = \sum_k w^c_k \frac{1}{Z} \sum_{i,j} A^k_{ij}$. Then $\partial S_c / \partial A^k_{ij} = w^c_k / Z$ at every position, so

$$\alpha^c_k = \frac{w^c_k}{Z}, \qquad L^c_{\text{Grad-CAM}} = \frac{1}{Z} \operatorname{ReLU}(M_c).$$

The paper’s text states $\alpha^c_k = w^c_k$ (referring to an appendix of its arXiv version for details); with its own averaged Eq. 3 the derivation gives the factor $1/Z$, and with Zhou et al.’s summed pooling it gives exactly $w^c_k$. The constant does not change the map’s shape, and it disappears when maps are normalized for display. Apart from the ReLU, which CAM does not apply, Grad-CAM is therefore CAM for this architecture.

Guided Grad-CAM. With $G^c$ the guided backpropagation map for class $c$ at pixel resolution and $\operatorname{Up}$ bilinear upsampling, Guided Grad-CAM is $G^c \odot \operatorname{Up}(L^c_{\text{Grad-CAM}})$. The authors report that the deconvnet in place of guided backpropagation gives similar maps with artifacts.

Score-CAM. Wang et al. (2020) weight each channel by how much the channel’s region alone raises the score. With $s(\cdot)$ min–max normalization to $[0, 1]$, a baseline image $X_b$ and $\circ$ the elementwise product (Definitions 4 and 5),

$$H^k = s\big(\operatorname{Up}(A^k)\big), \qquad C(A^k) = f(X \circ H^k) - f(X_b), \qquad L^c_{\text{Score-CAM}} = \operatorname{ReLU}\Big( \sum_k \alpha^c_k A^k \Big),$$

where Definition 5 sets $\alpha^c_k = C(A^k)$ and their Algorithm 1 takes $\alpha^c_k$ as a softmax of $C(A^k)$ over channels. With a softmax over channels, the baseline term is the same for every channel and cancels.

Parameters

• Layer. Selvaraju et al. use the last convolutional layer, arguing that it gives the best compromise between high-level semantics and spatial detail. They report that, going from deep to shallow layers, the discriminative ability of Grad-CAM drops markedly at layers with a different output dimensionality, and that in a ResNet visualizing deeper layers changes Grad-CAM little between adjacent layers and more across layers that reduce dimensionality. Zhou et al. found that a higher mapping resolution improved CAM’s localization, and removed layers from AlexNet, VGG and GoogLeNet to get $13 \times 13$ and $14 \times 14$ maps (§3.1).
• Target score. Grad-CAM uses the score before the softmax. Any differentiable quantity works: for captioning, Selvaraju et al. differentiate the log probability of the whole caption with respect to VGG-16’s conv5_3.
• Upsampling and display. Bilinear upsampling to the input size is the usual choice. Maps are normalized per image before display, so a heat map shows relative evidence within one image, not its amount.
• Threshold for boxes. 20% of the maximum in CAM’s localization experiments, 15% in Grad-CAM’s. Choe et al. (2020) argue that the threshold is itself a hyperparameter whose choice has been confounded with method improvements (see Uses).

Complexity

Grad-CAM costs one forward pass and a backward pass to the chosen layer, plus $O(KZ)$ operations to weight and sum the maps. Selvaraju et al. contrast this with occlusion-based localization, which classifies many perturbed copies of the image, and call Grad-CAM typically an order of magnitude more efficient. CAM needs only the forward pass and the output-layer weights. Score-CAM needs one forward pass per channel of the chosen layer, hundreds for a late layer of a typical CNN, though these batch well.

Implementation

The example below builds a small network with fixed, hand-made weights, so that what each unit detects is known. The images are $64 \times 64$ with crosses or rings on a noisy background. Layer 1 has four bar detectors (horizontal, vertical and two diagonals) followed by $4 \times 4$ average pooling. Layer 2 has four units on the $16 \times 16$ pooled maps: a cross detector and a ring detector, each a template of the pooled layer-1 responses to a clean prototype, and two generic units that respond to bars of any orientation (“edges”) and to horizontal bars. The layer-2 maps go through global average pooling into a logistic regression, the only fitted part, trained on 400 images that contain one to three shapes of a single class.

The class scores are $y_{\text{ring}} = z/2$ and $y_{\text{cross}} = -z/2$, where $z$ is the regression’s logit for “ring”, so that a softmax over the two scores reproduces the model’s probability. Gradients are computed by hand-written backpropagation and checked against finite differences. The script compares Grad-CAM with Score-CAM at both layers. The sources leave Score-CAM’s weighting open: Algorithm 1 applies a softmax over channels to scores its comment calls logits, §3.2 says the scores are post-softmax probabilities, and the official code weights each map by the probability of the masked image itself, with no softmax over channels. The script runs all three readings. Maps are upsampled bilinearly with half-pixel centres to $64 \times 64$ before anything is measured.

It measures:

1. the identity with CAM, with the head written on averaged and on summed maps;
2. on images with one, two and three instances of one class (20 each), the share of each map’s mass that falls within 8 pixels of each instance’s centre;
3. on 20 images with one cross and one ring, the share on the target instance, on the other instance, and the share of the target region’s mass that lies on the shape’s own pixels (a measure of resolution);
4. for one mixed image, each layer-2 unit’s total activation and the channel weights of each method;
5. the deletion and insertion areas under the curve of Petsiuk et al. (2018), described in Neural Network Interpretability, for the layer-2 maps of the 60 single-class images, in the form Wang et al. use for Score-CAM, without the blurred start image of that description’s insertion: pixels are set to 0 (the background level) in the map’s order, or restored into an all-zero image, 1% at a time, and the target-class probability is averaged over the curve. A random order and an “object pixels first” order give reference values.

The block that prints unit masses and channel weights (item 4) was added, with the editor’s approval, after a pilot run in which the three Score-CAM readings agreed to two decimals; it changes no other output. The script uses NumPy 2.3.5, SciPy 1.16.3 and scikit-learn 1.7.2 and takes about 25 seconds; two runs gave identical output.

import numpy as np
from scipy import ndimage
from scipy.signal import fftconvolve
from sklearn.linear_model import LogisticRegression

N_PER_SET = 20         # images per instance count and in the mixed set
H, POOL, NOISE, B1 = 64, 4, 0.10, 0.20
yy, xx = np.mgrid[:H, :H]
CROSS, RING = 0, 1

# ---------- synthetic images with known instances ----------
def stroke(kind, cy, cx):
    if kind == CROSS:
        return ((abs(yy - cy) <= 1) & (abs(xx - cx) <= 5)) | ((abs(xx - cx) <= 1) & (abs(yy - cy) <= 5))
    r = np.hypot(yy - cy, xx - cx)
    return (r >= 3.5) & (r <= 5.5)

def make_image(kinds, rng):
    """Shapes at random, well-separated centres on a noisy dark background.
    Returns the image and, per instance, (kind, stroke mask, region = disc of radius 8)."""
    centers = []
    while len(centers) < len(kinds):
        c = rng.integers(9, H - 9, size=2)
        if all(np.hypot(*(c - d)) >= 18 for d in centers):
            centers.append(c)
    img, inst = np.zeros((H, H)), []
    for k, (cy, cx) in zip(kinds, centers):
        m = stroke(k, cy, cx)
        img[m] = rng.uniform(0.6, 1.0)
        inst.append((k, m, np.hypot(yy - cy, xx - cx) <= 8))
    return img + rng.normal(0, NOISE, img.shape), inst

# ---------- a fixed, hand-made network ----------
def line_kernel(theta, size=7):
    r = size // 2
    v, u = np.mgrid[-r:r + 1, -r:r + 1]
    d = -np.sin(theta) * u + np.cos(theta) * v                 # distance across a line at angle theta
    k = np.exp(-d ** 2 / 2) - 0.5 * np.exp(-d ** 2 / 8)
    k -= k.mean()
    return k / np.linalg.norm(k)

K1 = np.array([line_kernel(t) for t in np.deg2rad([0, 45, 90, 135])])   # layer 1: four bar detectors

def layer1(X):                       # (N,64,64) -> pre-activations (N,4,64,64), threshold B1
    return np.stack([fftconvolve(X, k[::-1, ::-1][None], mode="same", axes=(1, 2)) for k in K1], 1) - B1

def pool(A):                         # 4x4 average pooling: (N,C,64,64) -> (N,C,16,16)
    n, c = A.shape[:2]
    return A.reshape(n, c, H // POOL, POOL, H // POOL, POOL).mean((3, 5))

def proto_feats(kind):               # pooled layer-1 maps of one clean, centred shape
    img = np.zeros((H, H)); img[stroke(kind, 32, 32)] = 0.8
    return pool(np.maximum(layer1(img[None]), 0))[0]

T = []                               # layer 2: matched templates in feature space, plus two generic units
for kind in (CROSS, RING):
    crop = proto_feats(kind)[:, 6:11, 6:11]
    crop = crop - crop.mean()
    T.append(crop / np.linalg.norm(crop))
edge = np.zeros((4, 5, 5)); edge[:, 1:4, 1:4] = 1 / 6.0      # edges of any orientation
horiz = np.zeros((4, 5, 5)); horiz[0, 1:4, 1:4] = 1 / 3.0    # horizontal bars only
T = np.array(T + [edge, horiz])

def layer2_pre(P):
    return np.stack([sum(ndimage.correlate(P[:, k], T[j, k][None], mode="constant") for k in range(4))
                     for j in range(len(T))], 1)
pc, pr = (layer2_pre(proto_feats(k)[None])[0] for k in (CROSS, RING))
B2 = np.array([pr[0].max(), pc[1].max(), 0.02, 0.02])       # cross/ring units silent on the other shape

def features(X):
    A1 = np.maximum(layer1(X), 0)                             # layer 1 maps, 64x64
    R2 = layer2_pre(pool(A1)) - B2[None, :, None, None]
    return A1, R2, np.maximum(R2, 0)                          # layer 2 maps, 16x16

# ---------- head: logistic regression on global-average-pooled layer-2 maps ----------
def dataset(n, seed):
    rng = np.random.default_rng(seed)
    X, y = [], []
    for i in range(n):
        img, _ = make_image([i % 2] * rng.integers(1, 4), rng)
        X.append(img); y.append(i % 2)
    return np.array(X), np.array(y)

Xtr, ytr = dataset(400, 0); Xte, yte = dataset(200, 1)
Ftr, Fte = features(Xtr)[2].mean((2, 3)), features(Xte)[2].mean((2, 3))
mu, sd = Ftr.mean(0), Ftr.std(0)
clf = LogisticRegression(C=1.0, max_iter=5000).fit((Ftr - mu) / sd, ytr)
w = clf.coef_[0] / sd                                         # logit of "ring": z = w . F + b
b = clf.intercept_[0] - mu @ w
SIGN = {CROSS: -0.5, RING: 0.5}                               # class scores y_c = -z/2, +z/2
Z = 16 * 16                                                   # positions averaged by the pooling

def score(X, c):
    return SIGN[c] * (features(X)[2].mean((2, 3)) @ w + b)

def prob(X, c):                                               # softmax over the two scores
    return 1 / (1 + np.exp(-2 * score(X, c)))

def grads(x, c):
    """d y_c / d A for both layers, by hand-written backpropagation."""
    A1, R2, A2 = features(x[None])
    G2 = np.broadcast_to(SIGN[c] * w[None, :, None, None] / Z, A2.shape)   # dy/dA2: constant over positions
    GR = G2 * (R2 > 0)                                        # through the ReLU
    GP = np.stack([sum(ndimage.convolve(GR[:, j], T[j, k][None], mode="constant") for j in range(len(T)))
                   for k in range(4)], 1)                     # adjoint of the layer-2 correlation
    G1 = np.repeat(np.repeat(GP, POOL, 2), POOL, 3) / POOL ** 2
    return {1: (A1[0], G1[0]), 2: (A2[0], G2[0])}

def upsample(a, out=H):                                       # bilinear, half-pixel centres
    n = a.shape[-1]
    if n == out:
        return a
    cc = np.clip((np.arange(out) + 0.5) * n / out - 0.5, 0, n - 1)
    i0 = np.floor(cc).astype(int); i1 = np.minimum(i0 + 1, n - 1); t = cc - i0
    a = a[..., i0, :] * (1 - t)[:, None] + a[..., i1, :] * t[:, None]
    return a[..., :, i0] * (1 - t) + a[..., :, i1] * t

# ---------- the explanation methods ----------
def grad_cam(x, c, layer):
    A, G = grads(x, c)[layer]
    alpha = G.mean((1, 2))                                    # Selvaraju et al., Eq. 1
    return upsample(np.maximum(np.tensordot(alpha, A, 1), 0)), alpha   # Eq. 2

def score_cam(x, c, layer, variant):
    A = grads(x, c)[layer][0]
    keep = [k for k in range(len(A)) if A[k].max() > A[k].min()]   # constant maps carry no mask
    M = upsample(A[keep])
    M = (M - M.min((1, 2), keepdims=True)) / (M.max((1, 2), keepdims=True) - M.min((1, 2), keepdims=True))
    masked = x[None] * M                                      # baseline X_b = 0 (cancels in a softmax)
    if variant == "code":                                     # official code: alpha_k = p_c(masked), maps upsampled
        return np.maximum(np.tensordot(prob(masked, c), upsample(A[keep]), 1), 0)
    s = score(masked, c) if variant == "logit" else prob(masked, c)
    alpha = np.exp(s - s.max()); alpha /= alpha.sum()         # Algorithm 1: softmax over channels
    return upsample(np.maximum(np.tensordot(alpha, A[keep], 1), 0))

METHODS = {"Grad-CAM": lambda x, c, l: grad_cam(x, c, l)[0],
           "Score-CAM, Alg. 1 (logit)": lambda x, c, l: score_cam(x, c, l, "logit"),
           "Score-CAM, Sec. 3.2 (prob.)": lambda x, c, l: score_cam(x, c, l, "prob"),
           "Score-CAM, official code": lambda x, c, l: score_cam(x, c, l, "code")}

def share(m, region):
    return (m * region).sum() / m.sum() if m.sum() > 0 else np.nan

def del_ins_auc(x, c, order, steps=100):
    """Petsiuk et al.'s deletion and insertion: p_c as pixels are set to 0 / restored, 1% per step."""
    ks = np.round(np.linspace(0, H * H, steps + 1)).astype(int)
    rank = np.empty(H * H, int); rank[order] = np.arange(H * H)
    gone = rank[None] < ks[:, None]                           # (steps+1, 4096)
    xs = x.ravel()[None]
    p_del = prob(np.where(gone, 0.0, xs).reshape(-1, H, H), c)
    p_ins = prob(np.where(gone, xs, 0.0).reshape(-1, H, H), c)
    trap = lambda p: ((p[1:] + p[:-1]) / 2).mean()
    return trap(p_del), trap(p_ins)

def order_by(m, tie):                                         # highest first; ties in a fixed random order
    return tie[np.argsort(-m.ravel()[tie], kind="stable")]

# ---------- 1. checks ----------
print(f"head: held-out accuracy {((Fte @ w + b > 0) == yte).mean():.3f}, weights on (cross, ring, edges, "
      f"horizontal) units {np.round(w, 1)}")
rng = np.random.default_rng(2)
x, inst = make_image([CROSS, RING], rng)
A2 = features(x[None])[2][0]
for c in (CROSS, RING):
    gc, alpha = grad_cam(x, c, 2)
    cam = upsample(np.maximum(np.tensordot(SIGN[c] * w, A2, 1), 0))          # CAM, mean-pooled head
    cam_sum = upsample(np.maximum(np.tensordot(SIGN[c] * w / Z, A2, 1), 0))  # CAM, head on summed maps
    print(f"class {'cross' if c == CROSS else 'ring '}: Z * Grad-CAM weights / CAM weights = "
          f"{np.round(Z * alpha / (SIGN[c] * w), 12)}; max|Grad-CAM*Z - CAM| / max CAM = {np.abs(gc * Z - cam).max() / cam.max():.1e};"
          f" summed pooling: max|Grad-CAM - CAM| / max CAM = {np.abs(gc - cam_sum).max() / cam_sum.max():.1e}")
A1, G1 = grads(x, CROSS)[1]
def y_of_A1(A):
    return SIGN[CROSS] * (np.maximum(layer2_pre(pool(A[None])) - B2[None, :, None, None], 0).mean((2, 3)) @ w + b)[0]
errs, rng_g = [], np.random.default_rng(3)
nz = np.argwhere(G1 != 0)
for k, i, j in nz[rng_g.choice(len(nz), 40, replace=False)]:
    fd = []
    for h in (1e-4, 1e-6):
        e = np.zeros_like(A1); e[k, i, j] = h
        fd.append((y_of_A1(A1 + e) - y_of_A1(A1 - e)) / (2 * h))
    if abs(fd[0] - fd[1]) <= 1e-6 * abs(fd[1]):               # skip entries whose step crosses a ReLU kink
        errs.append(abs(fd[1] - G1[k, i, j]) / abs(G1[k, i, j]))
print(f"layer-1 gradient vs central differences: max relative error {max(errs):.1e} ({len(errs)} entries)")

# ---------- 2. evaluation images ----------
rng = np.random.default_rng(4)
sets = {n: [(make_image([i % 2] * n, rng), i % 2) for i in range(N_PER_SET)] for n in (1, 2, 3)}
mixed = [make_image([CROSS, RING], rng) for _ in range(N_PER_SET)]
regs = [r for n in sets for (_, ins), _ in sets[n] for _, _, r in ins]
over = sum(int((np.array([r for _, _, r in ins]).sum(0) > 1).any())
           for ims in [[im for im, _ in s] for s in sets.values()] + [mixed] for _, ins in ims)
print(f"data: {sum(len(s) for s in sets.values())} single-class images, {len(mixed)} mixed; "
      f"{over} with overlapping instance regions; one region = {regs[0].mean():.3f} of the image")

# ---------- 3. several instances of one class ----------
for layer in (2, 1):
    print(f"\nlayer {layer} ({'16x16' if layer == 2 else '64x64'}): share of map mass per instance, largest first;"
          " in brackets: on all instances together")
    for name, f in METHODS.items():
        row, empty = [], 0
        for n in (1, 2, 3):
            sh = np.array([sorted([share(f(x, c, layer), r) for _, _, r in ins], reverse=True)
                           for (x, ins), c in sets[n]])
            empty += int(np.isnan(sh[:, 0]).sum())
            row.append(" ".join(f"{v:.2f}" for v in np.nanmean(sh, 0)) + f" ({np.nanmean(sh.sum(1)):.2f})")
        print(f"  {name:28s} " + " | ".join(row) + (f"   [{empty} all-zero maps]" if empty else ""))

# ---------- 4. one cross and one ring: class specificity and resolution ----------
print("\nmixed images: share on the target instance / on the other instance / on target-stroke pixels"
      " (of the mass in the target region)")
for layer in (2, 1):
    for name, f in METHODS.items():
        t, o, s = [], [], []
        for x, ins in mixed:
            for c in (CROSS, RING):
                m = f(x, c, layer)
                (_, st, rt), = [i for i in ins if i[0] == c]
                (_, _, ro), = [i for i in ins if i[0] != c]
                t.append(share(m, rt)); o.append(share(m, ro)); s.append(share(m * rt, st))
        print(f"  layer {layer}  {name:28s} {np.nanmean(t):.2f} / {np.nanmean(o):.2f} / {np.nanmean(s):.2f}")

# ---------- 4b. why: unit masses and channel weights on one mixed image (layer 2) ----------
x, ins = mixed[0]
A = features(x[None])[2][0]
print(f"\none mixed image, layer 2: activation mass of the (cross, ring, edges, horizontal) units "
      f"{np.round(A.sum((1, 2)), 2)}")
for c in (CROSS, RING):
    M = upsample(A)
    M = (M - M.min((1, 2), keepdims=True)) / (M.max((1, 2), keepdims=True) - M.min((1, 2), keepdims=True))
    s, p = score(x[None] * M, c), prob(x[None] * M, c)
    soft = lambda v: np.exp(v - v.max()) / np.exp(v - v.max()).sum()
    print(f"  {'cross' if c == CROSS else 'ring '}: Grad-CAM {np.round(grad_cam(x, c, 2)[1], 3)}  "
          f"Score-CAM Alg. 1 {np.round(soft(s), 2)}  Sec. 3.2 {np.round(soft(p), 2)}  code {np.round(p, 2)}")

# ---------- 5. deletion and insertion (layer 2) ----------
print("\ndeletion AUC (lower is better) and insertion AUC (higher is better), layer-2 maps, all single-class images")
tie = np.random.default_rng(99).permutation(H * H)
res = {k: [] for k in ["random order", "object pixels first"] + list(METHODS)}
for n in (1, 2, 3):
    for i, ((x, ins), c) in enumerate(sets[n]):
        res["random order"].append(del_ins_auc(x, c, np.random.default_rng(1000 * n + i).permutation(H * H)))
        obj = np.any([s for _, s, _ in ins], 0).astype(float)
        res["object pixels first"].append(del_ins_auc(x, c, order_by(obj, tie)))
        for name, f in METHODS.items():
            res[name].append(del_ins_auc(x, c, order_by(f(x, c, 2), tie)))
for name, v in res.items():
    v = np.array(v)
    print(f"  {name:28s} deletion {v[:, 0].mean():.3f}   insertion {v[:, 1].mean():.3f}")

Output:

head: held-out accuracy 1.000, weights on (cross, ring, edges, horizontal) units [-824.9 1009.4   15.6   -7.9]
class cross: Z * Grad-CAM weights / CAM weights = [1. 1. 1. 1.]; max|Grad-CAM*Z - CAM| / max CAM = 3.4e-16; summed pooling: max|Grad-CAM - CAM| / max CAM = 3.4e-16
class ring : Z * Grad-CAM weights / CAM weights = [1. 1. 1. 1.]; max|Grad-CAM*Z - CAM| / max CAM = 2.9e-16; summed pooling: max|Grad-CAM - CAM| / max CAM = 2.9e-16
layer-1 gradient vs central differences: max relative error 2.4e-07 (40 entries)
data: 60 single-class images, 20 mixed; 0 with overlapping instance regions; one region = 0.048 of the image

layer 2 (16x16): share of map mass per instance, largest first; in brackets: on all instances together
  Grad-CAM                     0.96 (0.96) | 0.59 0.37 (0.96) | 0.47 0.31 0.19 (0.96)
  Score-CAM, Alg. 1 (logit)    0.71 (0.71) | 0.39 0.32 (0.70) | 0.28 0.23 0.19 (0.71)
  Score-CAM, Sec. 3.2 (prob.)  0.71 (0.71) | 0.39 0.32 (0.71) | 0.29 0.23 0.19 (0.71)
  Score-CAM, official code     0.71 (0.71) | 0.39 0.32 (0.71) | 0.29 0.23 0.19 (0.71)

layer 1 (64x64): share of map mass per instance, largest first; in brackets: on all instances together
  Grad-CAM                     0.81 (0.81) | 0.46 0.38 (0.84) | 0.34 0.28 0.25 (0.87)   [13 all-zero maps]
  Score-CAM, Alg. 1 (logit)    0.89 (0.89) | 0.51 0.42 (0.93) | 0.37 0.31 0.26 (0.94)
  Score-CAM, Sec. 3.2 (prob.)  0.89 (0.89) | 0.51 0.42 (0.93) | 0.37 0.31 0.26 (0.94)
  Score-CAM, official code     0.89 (0.89) | 0.51 0.42 (0.93) | 0.37 0.31 0.26 (0.94)

mixed images: share on the target instance / on the other instance / on target-stroke pixels (of the mass in the target region)
  layer 2  Grad-CAM                     0.84 / 0.09 / 0.50
  layer 2  Score-CAM, Alg. 1 (logit)    0.36 / 0.36 / 0.37
  layer 2  Score-CAM, Sec. 3.2 (prob.)  0.36 / 0.36 / 0.37
  layer 2  Score-CAM, official code     0.36 / 0.36 / 0.37
  layer 1  Grad-CAM                     0.43 / 0.43 / 0.47
  layer 1  Score-CAM, Alg. 1 (logit)    0.47 / 0.46 / 0.70
  layer 1  Score-CAM, Sec. 3.2 (prob.)  0.47 / 0.46 / 0.70
  layer 1  Score-CAM, official code     0.47 / 0.46 / 0.70

one mixed image, layer 2: activation mass of the (cross, ring, edges, horizontal) units [ 1.86  1.62 32.41 16.32]
  cross: Grad-CAM [ 1.611 -1.972 -0.031  0.016]  Score-CAM Alg. 1 [0.38 0.38 0.09 0.14]  Sec. 3.2 [0.31 0.31 0.18 0.2 ]  code [0.65 0.65 0.1  0.21]
  ring : Grad-CAM [-1.611  1.972  0.031 -0.016]  Score-CAM Alg. 1 [0.11 0.12 0.47 0.3 ]  Sec. 3.2 [0.19 0.19 0.33 0.29]  code [0.35 0.35 0.9  0.79]

deletion AUC (lower is better) and insertion AUC (higher is better), layer-2 maps, all single-class images
  random order                 deletion 0.655   insertion 0.662
  object pixels first          deletion 0.505   insertion 0.948
  Grad-CAM                     deletion 0.497   insertion 0.929
  Score-CAM, Alg. 1 (logit)    deletion 0.510   insertion 0.945
  Score-CAM, Sec. 3.2 (prob.)  deletion 0.509   insertion 0.944
  Score-CAM, official code     deletion 0.509   insertion 0.945

Grad-CAM is CAM here, and the constant depends on how the head is written. Grad-CAM’s channel weights are exactly CAM’s weights divided by $Z = 256$ when the head acts on averaged maps, and exactly CAM’s weights when the same function is written with summed maps; the maps agree to rounding error. Grad-CAM’s map depends only on the function the network computes, while CAM’s weights depend on how its last layer is parameterized. In our reading, this accounts for the factor $1/Z$ between Grad-CAM’s text and its own Eq. 3: the text’s $\alpha^c_k = w^c_k$ holds for the head written on summed maps.

Several instances. At layer 2, Grad-CAM puts 96% of its mass on the instances for one, two and three objects and divides it among them (on average 0.47, 0.31 and 0.19 for three, sorted). Instances differ in contrast, and with a head that is linear in the pooled maps each instance contributes in proportion to its detector response. Score-CAM also covers every instance, with less of its mass on objects (0.71). Wang et al. report that Grad-CAM tends to capture only one of several objects of a class (their Fig. 7). That did not happen in this network, which has no nonlinearity between the layer and the score; in our reading, the behaviour they show depends on what lies downstream of the chosen layer.

Class specificity. On images with one cross and one ring, layer-2 Grad-CAM puts 0.84 of its mass on the target shape and 0.09 on the other. Every Score-CAM reading puts 0.36 on each: in this scene Score-CAM lost class specificity at layer 2. Before the run we had expected the §3.2 reading, whose softmax over probabilities between 0 and 1 keeps all channel weights within a factor of $e$ of each other, to be the least specific of the three. That expectation did not hold: the three readings agree to two decimals in every row. The printed block shows the cause in this scene. The generic edges and horizontal units carry about 32 and 16 units of activation, against about 2 for the cross and ring units, and all Score-CAM weights are positive and bounded, between 0.09 and 0.90 here. Whichever reading is used, the two shared units dominate the weighted sum, and they fire on both shapes. On this image, the masked images that keep only the cross unit’s region or only the ring unit’s region also receive equal or nearly equal weights (0.38 for “cross” under Algorithm 1, 0.65 in the code’s reading). Grad-CAM’s weights come through the logistic head, which gives the class units weights 50 to 130 times larger than the generic ones, compensating for their smaller activations, and gives the other class’s unit a negative weight that the ReLU then removes. This shows what can happen when channels differ widely in scale; it is not a measurement of how Score-CAM behaves on trained networks, where Wang et al. report class-discriminative maps (their Fig. 6).

Layer choice. At layer 1, whose maps have four times the resolution, Grad-CAM is no longer class-specific: 0.43 on the target and 0.43 on the other shape, since, by construction, crosses and rings both contain horizontal and vertical stretches that the same bar detectors pick up. Thirteen of its 60 maps on single-class images are entirely zero: at this layer, gradients are nonzero only where the layer-2 units are active, and averaging them over all 4,096 positions gave channel weights whose weighted sum is nowhere positive. We expected layer 1 to give finer maps too; for Grad-CAM that expectation did not hold (0.47 of the target region’s mass on the shape’s pixels, against 0.50 at layer 2), while Score-CAM’s layer-1 maps were much finer (0.70 against 0.37). Neither method at layer 1 tells the classes apart, which is the trade-off Selvaraju et al. describe for shallow layers.

Deletion and insertion. All layer-2 maps beat the random order clearly, by about 0.15 in deletion and 0.27–0.28 in insertion. Grad-CAM has the lowest deletion area (0.497), and Score-CAM the higher insertion area (0.944–0.945 against 0.929), close to the object-pixels-first reference. Deleting the whole image switches every unit off, so the probability at the end of deletion is set by the head’s intercept alone, not driven to 0, and deletion areas stay well above 0. These single-class images contain only shapes of the target class, so a map that spreads over all edges loses nothing: the measures did not register the failure on mixed images. In our reading, removal metrics on images with one class present do not test class discrimination, which has to be tested on images that contain more than one class.

Properties and Behavior

Localization. On the ILSVRC-15 validation set, Grad-CAM on the off-the-shelf VGG-16 had 56.51% top-1 and 46.41% top-5 localization error, against 57.20% and 45.14% for CAM on VGG-16-GAP, 61.12% and 51.46% for gradient saliency, and 70.92% and 63.04% for c-MWP (Selvaraju et al., Table 1). Grad-CAM was better on top-1 and CAM on top-5, but CAM’s modified network had higher classification error, 33.40% top-1 against 30.38% for VGG-16, and Grad-CAM leaves the classifier unchanged.

Class discrimination and trust. In a study with 43 Amazon Mechanical Turk workers on PASCAL VOC 2007 images containing exactly two categories, workers identified the visualized category in 61.23% of cases from Guided Grad-CAM, against 44.44% from guided backpropagation; adding Grad-CAM to the deconvnet raised it from 53.33% to 61.23%. In a second study, 54 workers compared explanations of VGG-16 and AlexNet on images both classified correctly. Asked which model was more reliable on a scale from −2 to +2, they gave VGG-16 an average of 1.00 from guided backpropagation and 1.27 from Guided Grad-CAM.

Faithfulness. Selvaraju et al. compared maps with occlusion maps, which measure the score change when patches are masked, on 2,510 PASCAL VOC 2007 validation images. Rank correlations were 0.254 for Grad-CAM and 0.261 for Guided Grad-CAM, against 0.168 for guided backpropagation, 0.220 for c-MWP and 0.208 for CAM. For a VQA model, Grad-CAM’s rank correlation with occlusion was 0.60 against 0.42 for guided backpropagation. Later tests were harder on part of the family. In the model-parameter randomization test of Adebayo et al. (2018), as summarized in Neural Network Interpretability, Grad-CAM’s maps changed when the network’s weights were randomized, while Guided Grad-CAM’s stayed nearly unchanged, as guided backpropagation’s did, behaving like an edge detector. Wang et al. report that Grad-CAM, Grad-CAM++ and Score-CAM all pass that test.

Localization of real findings. Saporta et al. (2022) compared seven saliency methods on chest X-ray classifiers with radiologists’ segmentations, as summarized in chest X-ray analysis. Grad-CAM on DenseNet-121 was generally best of the seven, yet its mean IoU was significantly below the human benchmark for five of ten pathologies and averaged 24.0% worse. The gap was largest for small pathologies with complex shapes, and localization was better when the model was more confident.

Beyond classification. For a captioning model without attention (neuraltalk2 with VGG-16), Grad-CAM’s mean activation inside boxes produced by a dense captioning model was 3.27 times that outside, where a uniform map scores 1.0; guided backpropagation reached 2.32 and Guided Grad-CAM 6.38. For a VQA model, Grad-CAM’s rank correlation with human attention maps on 1,374 question–image pairs was 0.136, statistically higher than chance. See image captioning and visual question answering.

Limitations

• Coarse resolution. The map has the chosen layer’s resolution, $14 \times 14$ for VGG-16’s last convolutional layer, and upsampling cannot add detail. Chefer et al. call the result coarse. Guided Grad-CAM adds detail but, like guided backpropagation, failed the randomization test.
• Linearization. The weights are a first-order approximation of everything downstream of the layer, averaged over positions. Gradients can be small where a unit saturates, which Wang et al. cite as a motivation for Score-CAM, and averaging can give a channel a small weight although its activation matters a lot at a few positions.
• Positive part only. The ReLU removes negative evidence, so a map for one class does not show what argued against it; a separate map for the competing class is needed.
• Layer dependence. Maps from shallow layers are finer but less class-discriminative, as in the example above, and no layer is correct by construction.
• Plausibility is not faithfulness. A map that covers the object does not show that the model used the object for the right reason. The general discussion is in Neural Network Interpretability.

Variants

• CAM (Zhou et al., 2016) is the special case for global average pooling followed by one linear layer; it needs that architecture.
• Guided Grad-CAM (Selvaraju et al., 2017) multiplies the upsampled Grad-CAM map with the guided backpropagation map, for high-resolution, class-discriminative maps. Gradient methods at pixel level are covered in saliency maps.
• Grad-CAM++ (Chattopadhay et al., 2018) combines gradients differently to form the channel weights; Wang et al. describe it only as a variation that differs in how gradients are combined, and its own paper could not be read for this article, so its formula is not given. In Wang et al.’s ImageNet tests with VGG-16, Grad-CAM++ had a 45.5% average drop and 18.9% average increase in confidence, against 47.8% and 19.6% for Grad-CAM (Table 1), and an insertion score of 0.346 and deletion score of 0.082 against 0.357 and 0.089 (Table 3).
• Score-CAM (Wang et al., 2020) replaces gradients by the target score of the image masked with each upsampled, normalized channel map. On 2,000 ILSVRC 2012 validation images with the PyTorch VGG-16, they report an average drop of 31.5% and average increase of 30.6% (Table 1), and insertion 0.386 and deletion 0.077 (Table 3), the best of the CAM methods they compared. On 500 validation images whose objects cover less than half of the image, 63.7% of Score-CAM’s map energy fell inside the ground-truth box, against 48.1% for Grad-CAM and 49.3% for Grad-CAM++ (Table 2). The paper and its code differ in how the weights are formed. Algorithm 1 applies a softmax over channels to the score increases, with a comment calling them logits; §3.2 says the scores are post-softmax probabilities; and the official code, at the commit read, multiplies each upsampled map by the target probability of its masked image, with no baseline and no softmax over channels, and skips constant maps. Wang et al. also state, in a footnote, that Grad-CAM normalizes its weights to sum to one; the ICCV Grad-CAM paper’s Eq. 1 does not.
• Transformers. Vision transformers have no convolutional maps, and only the [CLS] token reaches the classification head, so the gradient of the final layer’s output is zero for every other token. Chefer et al. (2021) found the best way to apply Grad-CAM to ViT-base was to treat the last attention layer’s [CLS] token as the feature map, without the [CLS] token itself. Their own method propagates relevance through all layers and multiplies each block’s attention relevance by the attention map’s gradient, keeping the positive part averaged over heads: $\bar{A}^{(b)} = I + \mathbb{E}_h\big[(\nabla A^{(b)} \odot R^{(n_b)})^+\big]$, with the blocks multiplied together. Attention rollout uses the same product with the raw attention, $I + \mathbb{E}_h A^{(b)}$, and is the same for every class. On ImageNet-Segmentation, Grad-CAM reached 64.44% pixel accuracy, 71.60% mAP and 40.82% mIoU against 79.70%, 86.03% and 61.95% for their method (Table 2). In their two-object examples, Grad-CAM was the only baseline whose maps differed between the two classes, though the authors judge its results unconvincing (Fig. 3).

Uses

Weakly supervised object localization. CAM was proposed for localizing objects from image labels alone. Zhou et al.’s GoogLeNet-GAP reached 37.1% top-5 localization error on the ILSVRC 2014 test set with a heuristic that takes boxes from the maps of the top three classes, and 42.9% without it, against 34.2% for a fully supervised AlexNet (Table 3). Global average pooling localized better than global max pooling (43.00% against 45.26% top-5 error on the validation set, Table 2), which the authors attribute to averaging rewarding the whole extent of an object. Many later methods built on CAM; CutMix, for example, reports its localization gains with class activation maps (see mixup and CutMix), and dilated convolution gives backbones finer maps. Choe et al. (2020) argue that localization from image labels alone is ill-posed: when background cues such as water go with a class more often than parts of the object, no scoring rule separates them. They also found that the methods had chosen thresholds and hyperparameters using localization labels. With a small, fixed set of fully labelled images for tuning, the operating threshold chosen per method, and the same splits, five methods published after CAM brought no major improvement over it. CAM’s mean over their three datasets and three backbones was 63.8%, and the best average gain was 0.2 points. A few-shot model trained directly on the labelled tuning images outperformed the weakly supervised methods.

Finding dataset bias. Selvaraju et al. fine-tuned VGG-16 to tell “doctor” from “nurse” on the top 250 image-search results per class. It generalized poorly (82%), and Grad-CAM showed it looking at faces and hairstyles. The search results were biased: 78% of the doctor images showed men and 93% of the nurse images women. After rebalancing the training set with the same number of images per class, accuracy on a balanced test set rose to 90%. The case is an instance of dataset bias; the map suggested the cue, and a changed training set confirmed it.

Diagnosing errors and models. Selvaraju et al. use Guided Grad-CAM to inspect VGG-16’s ImageNet mistakes, some of which come from ambiguities inherent in ImageNet classification, and to compare models, as in the trust study above. In medical imaging, CheXNet, CheXNeXt and CheXpert all show class activation or Grad-CAM maps as illustrations, with the limits described in Chest X-Ray Analysis.

Seeds for segmentation. Chefer et al. note that Grad-CAM, being class-specific and consistent, is used by downstream methods such as weakly supervised semantic segmentation.