Deepfake Detection

Deepfake detection is the task of deciding, from the content alone, whether an image or a video was generated or manipulated by a learned model. In its narrow sense it asks whether a face in a video has been swapped, reenacted or otherwise altered; research papers call this face forgery detection or facial manipulation detection. In a wider sense it asks whether any image came from a generator, which papers call synthetic image detection or the detection of CNN-generated or diffusion-generated images. The US National Institute of Standards and Technology (NIST, 2024, §3.2) groups both under synthetic content detection, and distinguishes detection from the content, which this article is about, from detection of provenance data such as watermarks and signed metadata, which this article covers at the end.

The phenomenon being detected, its methods, history, harms and legal treatment are described in Deepfakes. Here the subject is the detector: the benchmarks it is trained and tested on, the traces it looks for, why it fails on generators and compression it has not seen, how it is scored, and what marking content at the source adds. Methods are described as their papers report them; the article gives no guidance on making manipulated media or on avoiding detection.

Problem Definition

In its basic form the task is binary image classification: a detector $f$ maps an input $x$ (a face crop, a frame, a video clip or a whole image) to a score $s = f(x)$, higher meaning “more likely manipulated”, and a decision follows by comparing $s$ with a threshold $t$. Rössler et al. (2019, §4), who built the FaceForensics++ benchmark, cast face-forgery detection as per-frame binary classification of crops around a tracked face; Wang et al. (2020, §4.1) train a binary real-or-fake classifier on whole images from one generator and its training set.

What makes the task distinct from ordinary classification is where the test data come from. A detector is trained on a set of known manipulation methods $\mathcal{G}_{\text{train}}$ and the real images they were applied to, but it is used on content from generators, source videos and post-processing pipelines that may lie outside that set. The quantity that matters in practice is therefore not the in-distribution error but the error under these shifts:

• unseen manipulations or generators, $g \notin \mathcal{G}_{\text{train}}$ (cross-manipulation or cross-generator evaluation);
• unseen source data, such as different people, filming conditions or image categories (cross-dataset evaluation);
• unseen post-processing, such as resizing, blurring and lossy compression applied after generation, as happens when content is shared online.

Rössler et al. (§4.2) assume the manipulation affects the face region, which is why their detectors work on face crops. Wang et al. (§3.1) restrict their question to images output by CNN generators with an upsampling–convolution structure. Each assumption narrows what a detector can be expected to catch, and several of the open problems below follow from them.

Input and Output

• Input. Single images, face crops from a face detector or tracker, or short clips. FaceForensics++ detectors take a conservative crop, enlarged by a factor of 1.3 around the tracked face (Rössler et al., §4.2); a full-image XceptionNet was markedly less accurate (Table 1). Video methods such as LipForensics take sequences of grayscale mouth crops (Haliassos et al., 2021, §4.2).
• Output. A score per image or frame, often averaged over frames into a video-level score (Haliassos et al., §4.1; Shiohara & Yamasaki, 2022, §4.1), and a binary decision at a chosen threshold. Some detectors also output a map: the face X-ray marks where an image was blended (Li, Bao et al., 2020, §3.1). Watermark detectors output an extracted message and a hypothesis test on it (Fernandez et al., 2023, §3.1).

Variants

• Face-forgery detection, the narrow task, on face crops from videos manipulated by face swapping or reenactment.
• Synthetic image detection, on whole images from unconditional, conditional, text-to-image or image-to-image generators (Wang et al., 2020; Ojha et al., 2023; Wang et al., 2023).
• Video-level detection, which uses temporal evidence across frames; Shiohara and Yamasaki (§5) note that frame-level methods cannot see temporal inconsistencies.
• Localization, predicting which pixels were manipulated, as the face X-ray does for blended regions.
• Attribution, deciding which generator produced an image; NIST (2024, §3.2) lists it among the richer outputs some tools attempt, alongside the likely purpose of the content.
• Active detection, testing for a watermark the generator embedded on purpose, as opposed to the passive detection of unintended traces (NIST, 2024, §3.2, footnote 12).

Challenges

• Generalization. Detectors trained on some manipulations lose much of their accuracy on others. Li, Bao et al. (2020, Table 1) trained an Xception classifier on FaceForensics++ DeepFakes only: its AUC was 99.38% on held-out DeepFakes but 75.05% on Face2Face, 49.13% on FaceSwap and 80.39% on NeuralTextures. NIST (2024, §3.2) describes detection as a cat-and-mouse game in which detectors are often tied to the generators they were trained on.
• Compression and post-processing. Video compression removes the fine detail many detectors rely on. In FaceForensics++, the accuracy of an XceptionNet trained on all four methods fell from 99.26% on raw frames to 95.73% at light compression and 81.00% at strong compression (Table 1).
• A shrinking gap. Li, Yang et al. (2020, §1) built Celeb-DF because earlier datasets showed visible artefacts that circulating deepfakes did not; in Rössler et al.’s user study, people’s accuracy fell towards chance on strongly compressed frames, as the Deepfakes article describes.
• Shortcuts. Real and fake training images can differ in ways unrelated to manipulation: different source videos, resolutions or compression histories. A classifier rewarded only for separating them may learn those differences. Wang et al. (§3.2) process their real images with each generator’s own pipeline to keep real and fake distributions as close as possible.
• Partial manipulation and context. Much of a manipulated image may be authentic, and whether a manipulation is substantial can be subjective (NIST, 2024, §3.2). A genuine image shown out of context passes any content-based detector.

Approaches

The papers below describe a move from detectors trained on one benchmark’s fakes, which reached near-perfect in-distribution accuracy, to methods built for generalization (Li, Bao et al., 2020, §1; Shiohara & Yamasaki, 2022, §1): training on synthetic forgeries that carry only a generic trace, targeting cues every manipulation must leave, or using features that were never trained to separate real from fake.

Supervised classifiers on face crops

Rössler et al. (2019, §4.2) compared hand-crafted steganalysis features with a linear SVM, several small forensic CNNs and an XceptionNet pretrained on ImageNet, all on tracked face crops. On their test set, trained on all four manipulation methods, XceptionNet had the highest accuracy at each compression level (99.26%, 95.73% and 81.00% from raw to strongly compressed, Table 1), and the hand-crafted features fell from 97.63% on raw frames to 55.98% under strong compression. All detectors were less accurate on NeuralTextures, which the authors attribute to its training a separate model for every manipulation (§4.2). On their hidden benchmark of 1,000 frames that had been resized and compressed in unknown ways, the low-quality XceptionNet reached 70.10% total accuracy, and its precision on pristine images was 52.40% (§5, Table 2); the authors attribute the drop to the randomized quality level and possible tracking failures, and note that their detectors predict fake by default when tracking fails. Such classifiers, built on pretrained backbones and fine-tuned on a forgery dataset, remain the baselines that later papers compare against.

Blending boundaries

Li, Bao et al. (2020, §1) observe that most face-manipulation methods share a common step: blending the altered face into an existing background image. They write the blended image as

$$I_M = M \odot I_F + (1 - M) \odot I_B,$$

with a foreground face $I_F$, a background image $I_B$ and a soft mask $M$ with values in $[0, 1]$, and define the face X-ray as $B = 4\, M \odot (1 - M)$ (§3.1). $B$ is zero everywhere for a real image and peaks on the blending boundary. The cue is the boundary itself: images from different sources carry different acquisition traces, such as sensor noise and compression patterns, which change across it (§3).

Because the boundary is generic, training data can be made from real images alone: the authors blend two real faces with similar landmarks through a randomly deformed and blurred mask, and train an HRNet to predict $B$ and a real-or-blended label (§3.2, §4). Trained on these blended images only, without any output of a manipulation method, the detector reached an AUC of 99.17%, 98.57%, 98.21% and 98.13% on the four FaceForensics++ methods (§5.1, Table 1); on Celeb-DF it reached 74.76%, against 36.19% for an Xception trained on FaceForensics++ (Table 2). The authors list two limits: an entirely synthetic image has no blending step, and accuracy falls with resolution and compression, to an overall AUC of 87.35% on the lightly and 61.6% on the strongly compressed versions of FaceForensics++ (§6).

Self-blended images (Shiohara & Yamasaki, 2022) take the idea further: the source and target are two slightly transformed copies of the same real image, so the blend introduces boundary and colour inconsistencies without any identity change. An ImageNet-pretrained EfficientNet-B4 trained on them, with no manipulated images even for model selection (§4.1), reported a video-level AUC of 93.18% on Celeb-DF and 72.42% on DFDC after training on FaceForensics++ (Table 1). The authors note that their detector does not handle whole-image synthesis: on StyleGAN images against FFHQ its AUC was 69.11% (§5).

Frequency spectra

Generators build images by repeatedly upsampling feature maps, and the upsampling leaves traces in the image’s spectrum. Durall, Keuper and Keuper (2020, §2.3) analyse 2× upsampling in one dimension: inserting zeros between samples, the “bed of nails” behind transposed convolution, creates high-frequency copies of the signal’s spectrum, which a small kernel cannot remove, while bilinear interpolation suppresses them but leaves the spectrum too weak at high frequencies. In their experiments transposed convolutions added high-frequency content and interpolation-based upsampling lacked it (Fig. 5). The Fourier transform of a generated image therefore differs from that of a photograph, a form of the aliasing that the Transposed Convolution article explains for checkerboard artefacts.

They summarize each image by the azimuthal integral of its power spectrum, a 1D profile over spatial frequency, and classify it with an SVM (§2.1, §4.1.2). They report 100% test accuracy on their Faces-HQ set of 1024 × 1024 faces, even with 20 training samples, and 87% per frame and 90% per video on a FaceForensics++ subset (Table 1). Their description of that subset, 363 original sequences of 28 paid actors (§4.1.1), matches the counts that Li, Yang et al. (2020, §2.3, Table 1) give for the Google/Jigsaw DeepFake Detection dataset (DFD), not the 1,000 YouTube videos of FaceForensics++ itself (Rössler et al., §3).

Spectral cues are fragile in the way the Challenges above suggest: Wang et al. (2020, §4.6) note that FaceForensics++ deepfakes, which are resized, blended and compressed, show no obvious artefacts in their averaged spectra.

Cues from natural motion

Video detectors can test whether a face moves the way real faces do. LipForensics (Haliassos et al., 2021) pretrains a spatio-temporal network on lipreading, then freezes its feature extractor and fine-tunes only a temporal network on real and forged mouth sequences, so that it relies on high-level irregularities in mouth movement rather than low-level artefacts (§3). Trained on three FaceForensics++ methods (lightly compressed) and tested on the fourth, it reported an average video-level AUC of 97.1% against 94.9% for Face X-ray (Table 1). In a separate test, trained on uncompressed FaceForensics++ and exposed to corruptions not seen in training, it kept a high AUC under corruptions that affect high frequencies, where the other detectors deteriorated: averaged over severities, 96.1% under blur and 95.6% under video compression, against 60.2% and 62.1% for an Xception baseline (§4.3, Table 3). The authors note that it cannot judge single images and would probably miss a forgery that leaves the mouth unaltered or occluded (§4.6).

Detecting generated images in general

Wang et al. (2020) asked whether one classifier could detect images from many CNN generators. They trained an ImageNet-pretrained ResNet-50 on 720,000 images, real and generated, from ProGAN models for 20 LSUN categories, and tested it on 11 generators ranging from StyleGAN, BigGAN and CycleGAN to super-resolution and the FaceForensics++ DeepFakes (§3–4). Two findings shaped later work:

• Augmentation with blur and JPEG compression improves generalization, even when test images are not post-processed. With each applied with probability 0.5 during training, average precision on BigGAN rose from 72.2 to 88.2 and on GauGAN from 67.0 to 98.1 (§4.2, Table 2). It hurt on super-resolution and DeepFake images, and the authors’ compromise, applying both with probability 0.1, reached a mean AP of 92.6 over the 11 generators against 90.1 without augmentation (Table 2). The augmented models were also more robust to test-time blur and JPEG (Fig. 5).
• The detector transferred to a generator released after it was trained: the Blur+JPEG (0.1) model reached 99.1 AP on StyleGAN2 (§4.5, Table 3). On images edited with Photoshop’s face-aware liquify tool, by contrast, it performed at chance (§5).

The authors’ title adds “for now”, and their discussion says the situation need not persist (§5). Ojha, Li and Lee (2023) tested such a ProGAN-trained classifier on newer generators: on latent diffusion images it classified 99.61% of real images correctly but only 3.05% of generated ones (Table 1). They describe the real class as a “sink” that absorbs anything not resembling the training fakes (abstract, §3.2). Their remedy is not to learn the real-or-fake boundary at all: they map images with the frozen image encoder of CLIP ViT-L/14 and classify by nearest neighbour or a linear probe, still training only on ProGAN data. On unseen diffusion and autoregressive models the linear probe reached 95.00 mean AP against 75.51 for the best trained classifier (§6.1).

For diffusion models, DIRE (Wang et al., 2023) measures how well a pretrained diffusion model reconstructs an image: the image is inverted to noise with DDIM and regenerated, and $\mathrm{DIRE}(x_0) = |x_0 - R(I(x_0))|$ is the absolute error between the input and its reconstruction (§3.2, Eq. 9). The authors observe that generated images are reconstructed more closely than real ones, and train a binary classifier on the error image. On the bedroom images of their DiffusionForensics dataset, trained on images from one diffusion model (ADM), they report an average accuracy of 99.9% and AP of 100% across the diffusion generators tested (§4.2), with no drop under blur or JPEG at the levels tested (§4.4). The method needs a pretrained diffusion model at test time.

Datasets and Benchmarks

Face-forgery datasets are videos of real people, manipulated by the methods that Deepfakes surveys; they are described here from their papers.

Dataset	Real / fake videos	Manipulations	Notes	Source
FaceForensics++	1,000 / 4,000	DeepFakes, Face2Face, FaceSwap, NeuralTextures	YouTube videos; raw, HQ and LQ versions; hidden benchmark	Rössler et al. (2019)
DFD (Google/Jigsaw)	363 / 3,068	face swapping	28 consented individuals	Li, Yang et al. (2020, Table 1)
DFDC (first release)	1,131 / 4,113	face swapping	66 consented individuals	Li, Yang et al. (2020, Table 1)
Celeb-DF	590 / 5,639	improved face swapping	YouTube interviews of 59 celebrities	Li, Yang et al. (2020)
DeeperForensics-1.0	50,000 / 10,000	DF-VAE face swapping	100 consenting paid actors; 35 perturbations; hidden test set	Jiang et al. (2020)

• FaceForensics++ (Rössler et al., 2019, §3) applies four methods to 1,000 YouTube videos, giving more than 1.8 million manipulated images, with a fixed split of 720 training, 140 validation and 140 test videos (§4). Its three versions are uncompressed (raw), H.264 at constant quantization parameter 23 (HQ, visually nearly lossless) and 40 (LQ). Its hidden benchmark of 1,000 frames, processed with unknown resizing and compression and scored on the authors’ server, limits submissions to one every two weeks per submitter to prevent overfitting (§5).
• Celeb-DF (Li, Yang et al., 2020) was built to remove the visible artefacts of earlier sets; the ROC-AUC article describes how it scores detectors. Face X-ray’s paper (§5) describes it as 408 real and 795 synthesized videos, fewer than the 590 and 5,639 of the CVPR paper (Table 1); it cites the 2019 arXiv preprint, titled “Celeb-DF: A New Dataset for DeepFake Forensics”, so its Celeb-DF results may not be comparable with those on the CVPR release.
• DeeperForensics-1.0 (Jiang et al., 2020) swaps the faces of 100 consenting paid actors onto 1,000 FaceForensics++ target videos with an end-to-end face-swapping model, then applies seven types of distortion at five intensity levels, singly or mixed (§3). Its hidden test set contains manipulated videos ranked highly deceptive in user studies (abstract).
• Synthetic-image sets include Wang et al.’s ForenSynths, with images from 11 CNN generators (§3), and DIRE’s DiffusionForensics, with images from diffusion models on LSUN bedrooms, ImageNet and CelebA-HQ (Wang et al., 2023, §3.3).

Two protocols recur. Cross-manipulation training on three FaceForensics++ methods and testing on the fourth isolates the manipulation from the source videos (Haliassos et al., §4.2). Cross-dataset training on FaceForensics++ and testing on Celeb-DF, DFD, DFDC or others adds changes of people, filming conditions and pipelines; Shiohara and Yamasaki (§4.3) note that test sets vary from paper to paper and compiled results from the original papers into one table, which makes such comparisons approximate.

Evaluation Metrics

• Area under the ROC curve. Most face-forgery papers report ROC-AUC, at frame or video level; the ROC-AUC article defines it and its variants. It measures ranking and needs no threshold, which suits cross-dataset comparison but says nothing about whether a fixed threshold still works on new data.
• Accuracy at a threshold. FaceForensics++ reports accuracy (Rössler et al., §4), and Wang et al. (2020, §4.1) report it under the assumption that real and fake images are equally likely, in addition to AP. Ojha et al. (§5.3) set the threshold on held-out data from the training generator and also report an oracle accuracy whose threshold is calibrated on each test set, to gauge the best the classifier can do there.
• Average precision, used by Wang et al. (2020, §4.1) because it is threshold-free and insensitive to the base rate of fakes.
• Equal error rate, reported with AUC and AP by Face X-ray (Table 2).
• True positive rate at a fixed false positive rate, the natural measure when false alarms are costly. Stable Signature reports detection rates at false positive rates of $10^{-6}$ and below, derived from a binomial test on the extracted watermark bits rather than measured (Fernandez et al., 2023, §3.1, §5.1).
• Robustness curves: a metric plotted against blur strength, JPEG quality or other corruptions at several severities (Wang et al., 2020, Fig. 5; Haliassos et al., Fig. 4).

Frame-level and video-level numbers are not comparable, and neither are results on different releases or splits of the same dataset.

Example: A Spectral Detector and Its Threshold

The example below is a stand-in that uses no faces and no trained generator. “Real” images are 64 × 64 periodic textures whose power spectrum falls as $f^{-\alpha}$, with $\alpha$ drawn per image from $[1.5, 3]$. “Generated” images are 16 × 16 textures of the same kind passed through a decoder of two 2× upsampling stages. Each stage inserts zeros between pixels, applies the interpolation filter that defines the upsampler type, and then a 3 × 3 convolution:

• transposed: no interpolation filter, so the stage is a stride-2 transposed convolution with a 3 × 3 kernel;
• nearest: a 2 × 2 box filter, which turns zero insertion into nearest-neighbour repetition;
• bilinear: the separable filter $(\tfrac14, \tfrac34, \tfrac34, \tfrac14)$, which gives bilinear upsampling.

The 3 × 3 kernels are random, positive and normalized to sum to 1, fixed when the decoder is created and never trained. There are three decoders per type, with different random kernels. All convolutions wrap around the image borders, so no image has a border cue, and every image is standardized and stored as 8 bits.

A logistic regression classifier sees the logarithm of each image’s power spectrum, $\log |\hat{x}(u, v)|^2$ over the half-plane of the 2D discrete Fourier transform without the zero frequency (2,111 features), scaled to zero mean and unit variance on the training set. Each run leaves one upsampler type out. The classifier is trained on 1,200 real and 1,200 generated images from two decoders of each of the other two types. Two training variants are compared: clean images, and the “Blur+JPEG (0.5)” augmentation of Wang et al. (2020, §4.1), in which each training image is blurred with $\sigma \sim U[0, 3]$ and JPEG-compressed with quality drawn from $\{30, \dots, 100\}$, each with probability 0.5. Each test set has 400 real and 400 fresh generated images:

• seen: the training decoders;
• new weights: the third decoder of the training types;
• unseen type: the held-out upsampler;
• blur s=1: as seen, after a Gaussian blur with $\sigma = 1$ pixel;
• JPEG q=75: as seen, after JPEG compression at quality 75.

Generated images are the positive class. Each cell prints the AUC and, in parentheses, the accuracy at the classifier’s threshold of 0.5 on the predicted probability. The row ideal is the AUC of an oracle that knows the decoders that made the test fakes, and the blur: a decoder is linear, so its outputs span a 256-dimensional subspace of the 4,096 pixel values, and the oracle scores an image by its relative residual after projection onto that subspace plus a constant. Before training, the script checks the data: the fitted spectral slope of the real textures against $-\alpha$, the ratio of largest to smallest output of one upsampling stage on a constant input (above 1 means uneven overlap, the checkerboard), and the oracle’s residuals.

import io
import sys
import numpy as np
from scipy import ndimage
from PIL import Image
from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import roc_auc_score

SEED = int(sys.argv[1]) if len(sys.argv) > 1 else 0
N = 64                                         # image size; generator input is 16x16
TYPES = ["transposed", "nearest", "bilinear"]
b = np.array([0.25, 0.75, 0.75, 0.25])
INTERP = {"transposed": None,                  # zero insertion + 3x3 conv = transposed conv (k=3, s=2)
          "nearest": np.ones((2, 2)),          # zero insertion + 2x2 box = nearest-neighbour repeat
          "bilinear": np.outer(b, b)}          # zero insertion + this filter = bilinear 2x

def texture(n, alpha, rng):
    """Periodic Gaussian texture with power spectrum ~ f^-alpha (f in cycles per image)."""
    f = np.hypot(*np.meshgrid(np.fft.fftfreq(n) * n, np.fft.fftfreq(n) * n, indexing="ij"))
    amp = np.zeros_like(f)
    amp[f > 0] = f[f > 0] ** (-alpha / 2)
    return np.real(np.fft.ifft2(np.fft.fft2(rng.standard_normal((n, n))) * amp))

def up_stage(x, kind, k):
    """2x upsampling: zero insertion, the type's interpolation filter, then a 3x3 conv (all circular)."""
    z = np.zeros((2 * x.shape[0], 2 * x.shape[1]))
    z[::2, ::2] = x
    if INTERP[kind] is not None:
        z = ndimage.convolve(z, INTERP[kind], mode="wrap")
    return ndimage.convolve(z, k, mode="wrap")

def decode(x, dec):
    kind, k1, k2 = dec
    return up_stage(up_stage(x, kind, k1), kind, k2)

def to_uint8(x):
    z = (x - x.mean()) / x.std()
    return np.clip(np.round(128 + 32 * z), 0, 255).astype(np.uint8)

def blur(u8, sigma):
    if sigma <= 0:
        return u8
    y = ndimage.gaussian_filter(u8.astype(float), sigma, mode="wrap")
    return np.clip(np.round(y), 0, 255).astype(np.uint8)

def jpeg(u8, q):
    buf = io.BytesIO()
    Image.fromarray(u8, mode="L").save(buf, format="JPEG", quality=int(q))
    return np.array(Image.open(io.BytesIO(buf.getvalue())))

def real_images(n, rng):
    return [to_uint8(texture(N, rng.uniform(1.5, 3.0), rng)) for _ in range(n)]

def fake_images(n, decs, rng):
    return [to_uint8(decode(texture(16, rng.uniform(1.5, 3.0), rng), decs[i % len(decs)])) for i in range(n)]

def features(imgs):
    X = np.stack([im.astype(float) for im in imgs])
    X -= X.mean(axis=(1, 2), keepdims=True)
    P = np.abs(np.fft.rfft2(X)) ** 2
    return np.log(P + 1e-6).reshape(len(imgs), -1)[:, 1:]   # drop DC

def augment(imgs, rng):
    """Wang et al. (2020) Blur+JPEG (0.5): blur sigma~U[0,3] w.p. 0.5, JPEG q~U{30..100} w.p. 0.5."""
    out = []
    for im in imgs:
        if rng.random() < 0.5:
            im = blur(im, rng.uniform(0, 3))
        if rng.random() < 0.5:
            im = jpeg(im, rng.integers(30, 101))
        out.append(im)
    return out

# ---- fixed random decoders: 3 per type (0, 1 for training, 2 = "new weights") ----
drng = np.random.default_rng(1000)
def rand_kernel():
    k = drng.uniform(0, 1, (3, 3))
    return k / k.sum()
DECS = {t: [(t, rand_kernel(), rand_kernel()) for _ in range(3)] for t in TYPES}

# ---- oracle (ideal detector): knows the decoders (and the blur) behind the test fakes ----
def range_basis(dec, sigma=0.0):
    cols = []
    for i in range(256):
        e = np.zeros(256); e[i] = 1
        y = decode(e.reshape(16, 16), dec)
        if sigma > 0:
            y = ndimage.gaussian_filter(y, sigma, mode="wrap")
        cols.append(y.ravel())
    A = np.column_stack(cols + [np.ones(N * N)])
    Q, _ = np.linalg.qr(A)
    return Q

def oracle_scores(imgs, decs, sigma=0.0):
    Qs = [range_basis(d, sigma) for d in decs]
    X = np.stack([im.astype(float).ravel() for im in imgs])
    tot = ((X - X.mean(1, keepdims=True)) ** 2).sum(1)
    rel = np.min([((X - (X @ Q) @ Q.T) ** 2).sum(1) / tot for Q in Qs], axis=0)
    return -rel, rel

rng = np.random.default_rng(SEED)

# ---- data checks ----
alphas = rng.uniform(1.5, 3.0, 50)
f = np.hypot(*np.meshgrid(np.fft.fftfreq(N) * N, np.fft.fftfreq(N) * N, indexing="ij"))
errs = []
for a in alphas:
    P = np.abs(np.fft.fft2(texture(N, a, rng))) ** 2
    m = (f >= 2) & (f <= 30)
    slope = np.polyfit(np.log(f[m]), np.log(P[m]), 1)[0]
    errs.append(abs(slope + a))
print(f"real textures: |fitted spectral slope - (-alpha)| mean {np.mean(errs):.3f}")
for t in TYPES:
    out = up_stage(np.ones((16, 16)), t, DECS[t][0][1])
    print(f"{t:10s} stage on a constant input: max/min output = {out.max() / out.min():.3f}")

def test_set(kind, decs, rng, sigma=0.0, q=None):
    R = real_images(400, rng); F = fake_images(400, decs, rng)
    imgs = R + F
    if sigma > 0:
        imgs = [blur(im, sigma) for im in imgs]
    if q is not None:
        imgs = [jpeg(im, q) for im in imgs]
    return imgs, np.r_[np.zeros(400), np.ones(400)]

for held in TYPES:
    train_types = [t for t in TYPES if t != held]
    seen = [d for t in train_types for d in DECS[t][:2]]
    new = [DECS[t][2] for t in train_types]
    unseen = DECS[held]
    R = real_images(1200, rng)
    F = fake_images(1200, seen, rng)
    y_tr = np.r_[np.zeros(1200), np.ones(1200)]
    tests = {"seen": test_set("seen", seen, rng),
             "new weights": test_set("new", new, rng),
             "unseen type": test_set("unseen", unseen, rng),
             "blur s=1": test_set("blur", seen, rng, sigma=1.0),
             "JPEG q=75": test_set("jpeg", seen, rng, q=75)}
    print(f"\nheld-out upsampler: {held}  (trained on {', '.join(train_types)})")
    print(" " * 14 + "".join(f"{k:>16s}" for k in tests))
    # oracle row
    row = []
    for k, (imgs, y) in tests.items():
        decs = {"seen": seen, "new weights": new, "unseen type": unseen}.get(k, seen)
        s, rel = oracle_scores(imgs, decs, sigma=1.0 if k == "blur s=1" else 0.0)
        row.append(f"{roc_auc_score(y, s):.3f}")
        if k == "seen":
            print(f"  oracle residual (seen): fakes median {np.median(rel[y == 1]):.2e}, reals median {np.median(rel[y == 0]):.2e}")
    print(f"{'ideal (AUC)':14s}" + "".join(f"{v:>16s}" for v in row))
    for name, tr in [("clean", R + F), ("blur+JPEG aug", augment(R + F, rng))]:
        sc = StandardScaler().fit(features(tr))
        clf = LogisticRegression(C=1.0, max_iter=5000).fit(sc.transform(features(tr)), y_tr)
        cells = []
        for k, (imgs, y) in tests.items():
            p = clf.predict_proba(sc.transform(features(imgs)))[:, 1]
            cells.append(f"{roc_auc_score(y, p):.3f} ({np.mean((p > 0.5) == y):.2f})")
        print(f"{name:14s}" + "".join(f"{c:>16s}" for c in cells))

Output (NumPy 2.3.5, SciPy 1.16.3, scikit-learn 1.7.2, Pillow 12.0.0; the whole script took between 17 seconds and 2 minutes on a shared 10-core laptop):

real textures: |fitted spectral slope - (-alpha)| mean 0.049
transposed stage on a constant input: max/min output = 4.630
nearest    stage on a constant input: max/min output = 1.000
bilinear   stage on a constant input: max/min output = 1.000

held-out upsampler: transposed  (trained on nearest, bilinear)
                          seen     new weights     unseen type        blur s=1       JPEG q=75
  oracle residual (seen): fakes median 7.62e-05, reals median 2.33e-01
ideal (AUC)              1.000           1.000           1.000           1.000           1.000
clean             1.000 (1.00)    1.000 (1.00)    0.992 (0.55)    1.000 (0.50)    1.000 (1.00)
blur+JPEG aug     1.000 (1.00)    1.000 (0.94)    1.000 (0.96)    1.000 (0.99)    1.000 (1.00)

held-out upsampler: nearest  (trained on transposed, bilinear)
                          seen     new weights     unseen type        blur s=1       JPEG q=75
  oracle residual (seen): fakes median 7.78e-05, reals median 2.62e-01
ideal (AUC)              1.000           1.000           1.000           1.000           1.000
clean             1.000 (1.00)    1.000 (1.00)    1.000 (1.00)    1.000 (0.50)    0.995 (0.77)
blur+JPEG aug     1.000 (0.99)    1.000 (0.95)    1.000 (0.96)    1.000 (1.00)    1.000 (1.00)

held-out upsampler: bilinear  (trained on transposed, nearest)
                          seen     new weights     unseen type        blur s=1       JPEG q=75
  oracle residual (seen): fakes median 7.76e-05, reals median 2.74e-01
ideal (AUC)              1.000           1.000           1.000           1.000           0.998
clean             1.000 (1.00)    1.000 (1.00)    1.000 (1.00)    1.000 (0.50)    0.994 (0.83)
blur+JPEG aug     1.000 (1.00)    1.000 (0.98)    0.999 (0.76)    1.000 (1.00)    1.000 (1.00)

The checks confirm the construction. The real textures have the intended spectra, the transposed stage is uneven (its random kernel’s taps reach the four output phases with different total weight) while the box and bilinear stages are even, and generated images lie in their decoder’s subspace up to 8-bit rounding, with a median residual about 3,000 times smaller than that of real images. So the labels can be recovered from the pixels, and the oracle does so with an AUC of 1.000 in every cell but one JPEG test set (0.998).

The linear spectral classifier comes close to the oracle in ranking: its AUC is at least 0.992 in every cell, including new kernel weights and, against what we expected before the run, the held-out upsampler. Our hypothesis had been that a classifier trained on transposed and nearest-neighbour fakes would fail on bilinear ones, since Durall et al. describe the two kinds of upsampling as erring in opposite directions. In our reading, the three decoders share a stronger trace: by construction, every generated image is determined by a 16 × 16 input, so its spectrum above the input’s frequency band holds only copies or remnants of the lower band, while a real texture has independent content at every frequency. Real generators, with many trained layers and noise inputs, need not share their traces so neatly; this example shows the mechanism, not the size of the effect in practice.

The accuracy at the fixed threshold tells a different story from the AUC. After a blur with $\sigma = 1$, every clean-trained classifier still ranks the images perfectly (AUC 1.000) but has an accuracy of 0.50 on a balanced test set: with a perfect ranking, that is possible only if nearly every image falls on the same side of the threshold. The unseen upsampler, when transposed convolution is held out, and the JPEG test set for two held-out types show the same gap between a high AUC and a lower accuracy. A score shifted by post-processing that the training data never showed still orders images correctly, but its threshold no longer separates them, the distinction that Ojha et al. draw by reporting an accuracy with a threshold calibrated on each test set alongside the usual one. Augmented training restores the accuracy under blur and JPEG to 0.99–1.00. Its accuracy on new weights is lower than without augmentation in all three runs, and on the unseen type it is lower in two runs, down to 0.76, but higher (0.96 against 0.55) when transposed convolution is held out, while the AUC stays near 1 throughout: in this example augmentation moves the threshold more than the ranking. Wang et al. likewise found that augmentation helped on most of their generators and hurt on some.

Watermarking and Provenance

Content-based detection infers origin after the fact. The alternative is to record it at the source, so that later detection is a check rather than an inference; NIST (2024, §3.2) calls this provenance data detection and notes that it gives a strong positive signal about an item’s history, while detecting a watermark always carries some probability of error and metadata may be inaccurate.

Watermarks embed an imperceptible message in the content. HiDDeN (Zhu et al., 2018) jointly trains an encoder that hides a message in a cover image and a decoder that recovers it, with noise layers during training so that the message survives Gaussian blur, pixel dropout, cropping and JPEG compression (abstract). The Stable Signature (Fernandez et al., 2023) moves the watermark into the generator: it fine-tunes the latent decoder of a latent diffusion model so that every image it generates carries a fixed 48-bit signature, which a pretrained extractor reads back (§4.1–4.2). An image is flagged when enough bits match, and the false positive rate follows from a binomial model of the bits of unmarked images (§3.1, Eq. 2). On their own text-to-image model they report detecting 99% of unmodified generated images at a false positive rate of $10^{-9}$, 84% after a crop to 10% of the image, and 65% after a combined crop, brightness change and JPEG compression (§5.1). In §7 the authors evaluate deliberate removal; in their experiments, removing the mark without visibly degrading the image required knowledge of the generator’s own autoencoder or access to the watermark extractor, and they stress keeping the extractor private. NIST (2024, §3.1.1.3) reports that most current watermarking schemes have been found vulnerable to removal, and a watermark identifies output only from generators that embed it.

Signed provenance metadata. The C2PA technical specification for Content Credentials (version 2.5, October 2026) binds a manifest of signed assertions about an asset’s creation and edits to the asset by cryptographic hashes, a hard binding; a soft binding, a fingerprint or invisible watermark, can relink an asset whose manifest was stripped (§2.3.13–2.3.14, §2.4.1). An image made by a generative model records a c2pa.created action with the digital source type trainedAlgorithmicMedia (§16.4.1.2). The specification states that it makes no value judgement about whether provenance data is good or bad, only whether it can be validated (§1.2); assertions are trust signals for people to weigh (§1.3). Image Editing and Video Generation describe the specification in more detail. NIST (2024, §3.2) notes that, if provenance data became widespread, its absence where it would be expected could itself become a signal in some contexts.

The two approaches cover different cases. Provenance covers content from cooperating tools and platforms and says nothing about the rest; content-based detection applies to anything but is uncertain and tied to the generators it has learned.

Applications

• Platform moderation and verification, where detectors screen uploads. Rössler et al. (§4.1) designed their user study’s time limits of 2 to 6 seconds to mimic how long users look at an image on social media, and the hidden FaceForensics++ benchmark imitates the processing of uploaded videos (§5).
• Journalism and fact-checking, which combine detector scores with human judgement and provenance; NIST (2024, §3.2) describes human-assisted detection as labour-intensive and dependent on the reviewers’ expertise.
• Biometrics, where synthetic faces fed directly to a recognition system are injection attacks; Face Recognition describes how evaluations have treated them.
• Training data for detectors, made from generators themselves: FaceForensics++ released its manipulated videos, and Face X-ray and self-blended images make training forgeries from real images alone.
• Content labelling, for which Article 50(2) of the EU AI Act, applying from 2 August 2026 according to the European Commission’s guidelines on Article 50 and outlined in Deepfakes, requires providers of generative AI systems to mark their outputs in machine-readable form.

Open Problems

• New generators. Detectors fitted to one family of generators fail on the next, in the ways that Ojha et al. documented for diffusion images; whether any trace is shared by all generators, present and future, is open. Wang et al. (2020, §5) point out that a generator that matched the real distribution exactly would leave nothing to detect.
• Post-processing in the wild. Wang et al. (§5) call for more work on aggressive transformations of shared content, and the example above shows that even a detector that still ranks well can lose its threshold.
• Calibration. Cross-dataset results are mostly reported as AUC; the threshold a deployed detector needs is set on data that may not resemble what it will see.
• Shallow and partial edits. Wang et al.’s detector was at chance on Photoshop liquify edits (§5), Face X-ray assumes a blending step, and LipForensics assumes the mouth was changed.
• Adversarial robustness. Li, Bao et al. (§6) note that adversarial examples can be crafted against their detector and call the situation an arms race.
• Comparable benchmarks. Protocols, dataset releases and frame or video aggregation differ between papers (Shiohara & Yamasaki, §4.3), and face datasets built from online videos raise the consent questions that Deepfakes discusses; DFD, the first DFDC release and DeeperForensics-1.0 were recorded with people who consented (Li, Yang et al., §2.3; Jiang et al., §3.1).
• Provenance at scale. Watermarks and signed metadata cover only the generators and platforms that adopt them, and watermarks can be removed (NIST, 2024, §3.1.1.3).