ESRGAN

ESRGAN, the Enhanced Super-Resolution Generative Adversarial Network, is a convolutional model for single-image super-resolution at ×4. It was introduced by Wang, Yu, Wu, Gu, Liu, Dong, Qiao and Loy at the PIRM workshop of ECCV 2018. It keeps the overall design of SRGAN (Ledig et al., 2017), a residual generator trained with a feature loss and a generative adversarial network loss, and changes each of its three parts: the generator’s building block becomes the residual-in-residual dense block (RRDB) without batch normalization, the discriminator becomes a relativistic average discriminator, and the perceptual loss compares features before the activation rather than after. A variant won region 3 of the PIRM 2018 perceptual super-resolution challenge. Three years later, Real-ESRGAN (Wang, Xie, Dong and Shan, 2021) kept the same generator and retrained it on synthetic pairs from a second-order degradation process, with a U-Net discriminator, so that it could be applied to real photographs rather than to bicubically downscaled ones. This article covers both models and keeps apart what their papers state and what their released code does.

Problem Addressed

By 2017 two families of super-resolution networks existed. PSNR-oriented networks, among which the ESRGAN authors name EDSR (Lim et al., 2017), minimize a pixel loss and, in the ESRGAN authors’ words, tend to output over-smoothed results lacking high-frequency detail. SRGAN added an adversarial loss and a VGG feature loss and produced sharper textures; the SRGAN paper article describes it. Wang et al. (2018) still saw “a clear gap” between SRGAN’s outputs and the ground truth (§1), and found SRGAN’s hallucinated details often accompanied by unpleasant artifacts (abstract). They traced the problems to three components:

• The generator. SRGAN’s residual blocks contain batch normalization. The authors observed that batch normalization layers are more likely to bring artifacts when the network is deeper and trained adversarially, and that these artifacts appear occasionally across iterations and settings (§3.1).
• The adversarial loss. A standard discriminator estimates the probability that one image is real; the generator’s loss then depends only on generated images.
• The perceptual loss. Features after the ReLU are sparse in deep VGG layers, which the authors argue gives weak supervision and inconsistent brightness (§3.3; Perceptual Loss owns this argument).

Real-ESRGAN addresses a different gap. Most super-resolution networks, ESRGAN included, are trained on images downscaled with a bicubic kernel, which Wang et al. (2021, §1) call different from real degradations; a model trained that way is mismatched to photographs that carry camera blur, sensor noise and repeated compression.

Architecture

ESRGAN keeps SRGAN’s high-level layout (Fig. 3 of the paper): a first convolution, a trunk of basic blocks, a convolution, an upsampling stage and two final convolutions, so that most computation happens at low resolution. The paper makes two changes to the generator: it removes all batch normalization layers and replaces SRGAN’s residual block by the RRDB (§3.1). Several numbers that define the network appear only in the code; they are marked as such below.

Residual-in-Residual Dense Block

An RRDB nests residual learning at two levels (Fig. 4):

• Inner level: the dense block. Five 3 × 3 convolutions, the first four each followed by a leaky ReLU. As in a DenseNet, each convolution receives the concatenation of the block’s input and all earlier outputs in the block. The output of the fifth convolution is multiplied by a residual scaling constant $\beta$ and added to the block’s input.
• Outer level. Three dense blocks in sequence, whose result is again multiplied by $\beta$ and added to the RRDB’s input.

Writing $F_k$ for the $k$-th dense block’s residual branch, one RRDB computes

$$h_0 = x, \qquad h_k = h_{k-1} + \beta\, F_k(h_{k-1}) \;\; (k = 1, 2, 3), \qquad \mathrm{RRDB}(x) = x + \beta\, h_3,$$

which is the code’s out * 0.2 + x, with out the output of the third dense block. The paper describes residual scaling only as multiplying the residuals by a constant between 0 and 1 before adding them to the main path, citing Szegedy et al. and EDSR (§3.1). BasicSR v1.4.2 sets the constants: 64 feature channels, a growth of 32 channels per convolution, leaky ReLU slope 0.2, and $\beta = 0.2$ at both the dense-block and RRDB levels, where a comment calls 0.2 an empirical choice. Both levels are residual connections with a scaled branch.

The second training aid in the paper is “smaller initialization”: the authors found residual networks easier to train when the initial parameter variance is smaller, and refer to a supplementary file for details (§3.1), which could not be read for this article. In BasicSR, every convolution in a dense block is initialized with Kaiming normal initialization and then multiplied by 0.1 (default_init_weights(..., 0.1) in arch_util.py). The inference-only RRDBNet_arch.py in the original ESRGAN repository has the corresponding call, mutil.initialize_weights(..., 0.1), commented out.

Trunk and Upsampling

The paper gives no block count for the main model. The BasicSR configuration train_ESRGAN_x4.yml uses 23 RRDBs (num_block: 23). A convolution after the trunk is added to the output of the first convolution, a long skip over the whole trunk. Upsampling is not SRGAN’s sub-pixel convolution: the code doubles the resolution twice by nearest-neighbour interpolation, each time followed by a 3 × 3 convolution and a leaky ReLU, and ends with two more convolutions. Counting the layers’ weights and biases (Part C of the example below) gives 16,697,987 parameters for the ×4 network, 99.09% of them in the 23 RRDBs.

Discriminators

The ESRGAN paper does not describe its discriminator network. In BasicSR the ESRGAN configuration uses VGGStyleDiscriminator for 128 × 128 inputs: pairs of 3 × 3 and stride-2 4 × 4 convolutions with leaky ReLUs and, after the first convolution, batch normalization, halving the resolution five times, then two fully connected layers of 100 and 1 units. Removing batch normalization applied to the generator, not to this discriminator.

Relativistic Average Discriminator

ESRGAN borrows the relativistic average discriminator (RaD) from Jolicoeur-Martineau (2018), which could not be read for this article; the description here follows ESRGAN §3.2. A standard discriminator outputs $D(x) = \sigma(C(x))$, the sigmoid of a raw score $C(x)$. RaD instead estimates the probability that a real image $x_r$ is more realistic than the fake images on average:

$$D_{Ra}(x_r, x_f) = \sigma\big(C(x_r) - \mathbb{E}_{x_f}[C(x_f)]\big),$$

where the expectation is the average over the fake images in the mini-batch. The discriminator and generator losses are

$$L_D^{Ra} = -\mathbb{E}_{x_r}\big[\log D_{Ra}(x_r, x_f)\big] - \mathbb{E}_{x_f}\big[\log\big(1 - D_{Ra}(x_f, x_r)\big)\big], \tag{1}$$ $$L_G^{Ra} = -\mathbb{E}_{x_r}\big[\log\big(1 - D_{Ra}(x_r, x_f)\big)\big] - \mathbb{E}_{x_f}\big[\log D_{Ra}(x_f, x_r)\big], \tag{2}$$

with $x_f = G(x_i)$ for a low-resolution input $x_i$. The generator loss has the same form as the discriminator’s with the labels swapped. Because Eq. 2 contains both real and generated images, the authors write that their generator benefits from gradients from both, while in SRGAN only the generated part takes effect (§3.2). Two consequences can be read from the equations (our derivation; the example checks both):

• Real data reach the generator only through the fake scores. Real images do not depend on $G$, so every generator gradient passes through $C(x_f)$. The real term of Eq. 2 contributes to $\partial L_G^{Ra}/\partial C(x_f)$ through the batch mean $\mathbb{E}_{x_f}[C(x_f)]$, and its size depends on the real scores. SRGAN’s non-saturating loss $-\log \sigma(C(x_f))$, described in the GAN article, contains no real scores at all.
• Only score differences matter. Adding the same constant to every score leaves Eqs. 1 and 2 unchanged.

The ablation (§4.3, Fig. 8) reports that switching from the standard to the relativistic discriminator gave sharper edges and richer textures; the comparison is visual.

Training

The generator loss is

$$L_G = L_{\text{percep}} + \lambda L_G^{Ra} + \eta L_1, \qquad L_1 = \mathbb{E}_{x_i}\lVert G(x_i) - y \rVert_1, \tag{3}$$

with $y$ the ground truth, $\lambda = 5 \times 10^{-3}$ and $\eta = 10^{-2}$ (§3.3, §4.1). $L_{\text{percep}}$ compares VGG-19 features before activation; the paper does not name the layer used in the loss (its “VGG19-54”, the fourth convolution before the fifth max-pooling, appears in the sparsity argument). BasicSR’s configuration takes conv5_4 before its ReLU with an $L_1$ criterion.

Training follows SRGAN’s ×4 setting with low-resolution images made by MATLAB’s bicubic downsampling, mini-batches of 16 and 128 × 128 high-resolution crops, with random horizontal flips and 90° rotations, in RGB (§4.1). It has two stages:

1. A PSNR-oriented model is trained with the $L_1$ loss, learning rate $2 \times 10^{-4}$ halved every $2 \times 10^5$ iterations.
2. It initializes the generator, which is trained with Eq. 3 at learning rate $10^{-4}$, halved at 50k, 100k, 200k and 300k iterations, alternating generator and discriminator updates with Adam.

The authors report that pre-training with the pixel loss helps the GAN reach more visually pleasing results. The data are the 800 DIV2K training images, the 2,650 Flickr2K images and the OutdoorSceneTraining (OST) set; the authors found that the larger, more textured set gave more natural results (Fig. 8). They also report that the deeper network benefits from larger training patches, at more training cost.

Details found only in BasicSR v1.4.2 (train_ESRGAN_x4.yml, esrgan_model.py, gan_loss.py):

• Adam with betas (0.9, 0.99) for both networks, 400k iterations, and an exponential moving average of the generator weights with decay 0.999.
• The adversarial loss is binary cross-entropy on logits with mean reduction, and $\lambda$ (loss_weight: 5e-3) is applied to the generator only.
• Both relativistic losses are halved: the discriminator loss is $0.5\,\mathrm{BCE}(\cdot,1) + 0.5\,\mathrm{BCE}(\cdot,0)$ and the generator’s adversarial term is the mean of its two parts. The values equal $\tfrac12$Eq. 1 and $\tfrac12 \lambda$Eq. 2, so the effective adversarial weight is half the paper’s $\lambda$.
• In the discriminator update, each term uses the other set’s mean as a constant: the real term is computed from real_d_pred - torch.mean(fake_d_pred) with fake_d_pred detached, and the fake term from fake_d_pred - torch.mean(real_d_pred.detach()). A code comment gives a distributed-training error as the reason. The discriminator’s gradient therefore differs from the gradient of Eq. 1, which also flows through the two means; the example measures how much. The generator update keeps the gradient through the mean of the fake scores, so its gradient is exactly $\tfrac12\lambda$ times that of Eq. 2.

Network Interpolation

To trade perceptual quality against fidelity without retraining, ESRGAN interpolates the weights of the PSNR-oriented model and the GAN model fine-tuned from it (§3.4):

$$\theta_G^{\text{INTERP}} = (1 - \alpha)\,\theta_G^{\text{PSNR}} + \alpha\,\theta_G^{\text{GAN}}, \qquad \alpha \in [0, 1]. \tag{4}$$

The repository’s net_interp.py applies this to every tensor in the two state dictionaries. The authors compared it with interpolating the two models’ output images pixel by pixel, for $\alpha$ from 0 to 1 in steps of 0.2 (§4.4, Fig. 10): in their figures, network interpolation reduced artifacts while keeping texture, while image interpolation left the images either too blurry or noisy. They also list tuning $\lambda$ and $\eta$ as an alternative that would need retraining for every setting. Weight averaging is meaningful here because the GAN model starts from the PSNR model’s weights; this is our reading, since the paper does not discuss it. Network interpolation is unrelated to image interpolation in the resampling sense.

PIRM Variant

The challenge entry differed from the main model (§4.5): 16 residual blocks; a MINC loss, a perceptual loss on a VGG network fine-tuned for material recognition, which gave only a marginal gain in the perceptual index; the Pristine dataset added to training; $\eta$ raised to 10 because of the PSNR constraints; and back projection as post-processing. For regions 1 and 2, which require a higher PSNR, the authors interpolated between ESRGAN’s and RCAN’s output images rather than networks: image interpolation gave the lower perceptual index, although they found network interpolation more visually pleasing.

Inference

The generator maps a low-resolution RGB image to a ×4 output in one forward pass; $\alpha$ in Eq. 4 is chosen at inference by building an interpolated set of weights. The ESRGAN paper reports no timing. Real-ESRGAN’s code adds ×2 and ×1 models by pixel unshuffle, which rearranges each 2 × 2 or 4 × 4 neighbourhood into channels, so that the same ×4 trunk runs at the reduced resolution (Fig. 4 of the 2021 paper); the first convolution then takes 12 or 48 input channels. The ESRGAN README points users to the Real-ESRGAN repository, whose inference script it describes as supporting tiled inference, alpha channels, grey and 16-bit images (as of November 2026).

Performance

ESRGAN’s results are mainly visual. On the PIRM self-validation set (Fig. 2, read from the figure’s table), the submitted model had a perceptual index (PI, lower is better; see NIQE) of 2.040 at RMSE 15.15, against EnhanceNet’s 2.688 at 15.99, RCAN’s 4.831 at 10.87 and EDSR’s 5.243 at 11.16; two image-interpolated points between ESRGAN and RCAN reached 2.567 at 12.45 and 3.279 at 11.47. The figure’s region borders sit at RMSE 11.5, 12.5 and 16. In the challenge the method with the lowest PI in each RMSE region became that region’s champion (§1), and ESRGAN won region 3, the one with the largest RMSE allowed (abstract, §4.5).

Figure 7 gives PSNR on the luminance channel and PI for single test images at ×4. For baboon from Set14, ESRGAN scored 20.35 dB and PI 1.98, against SRGAN’s 21.15 dB and 2.62 and RCAN’s 23.12 dB and 4.20: the GAN models trade PSNR for perceptual index, as the perception–distortion trade-off in Super-Resolution predicts. The paper reports no PSNR table for the GAN model. The README of the official repository gives, for the PSNR-oriented RRDB model trained on DIV2K plus Flickr2K, PSNR/SSIM of 32.73/0.9011 on Set5 and 27.03/0.8153 on Urban100, the highest values in its table against EDSR and RCAN; the table does not state the scale or the evaluation protocol.

Real-ESRGAN’s paper is qualitative as well (Fig. 7), comparing ESRGAN, DAN, CDC, RealSR, BSRGAN, Real-ESRGAN and Real-ESRGAN+ on real images. Its supplementary Table 1 reports NIQE, lower being better, on real-world test sets. Real-ESRGAN+ had the lowest score on RealSR-Canon (4.5314, against 5.7489 for BSRGAN and 6.7715 for ESRGAN), RealSR-Nikon, DRealSR, DPED-iphone and OST300; on ImageNet validation ESRGAN scored best (3.6474, against 4.6448 for Real-ESRGAN+), and on ADE20K validation RealSR did (3.4102). The authors note that existing perceptual metrics do not reflect fine-grained human preferences, citing the PIRM report, and that Real-ESRGAN+ was not optimized for NIQE.

Strengths

• Each change is separable. The ablation in Fig. 8 adds batch-normalization removal, features before activation, the relativistic discriminator, the deeper RRDB network and more data one at a time, so each can be judged on its own, if only visually.
• Stable training of a deep generator without normalization. The authors contrast their easy training of a deeper model with SRGAN’s report that deeper models are increasingly difficult to train (§4.3).
• A control knob at inference. Network interpolation moves continuously between fidelity and texture with one parameter.
• One generator for two training regimes. Real-ESRGAN reuses the generator unchanged and moves the work to the data, so the same architecture serves bicubic and real-world settings.
• Released code and weights. Training code is in BasicSR and Real-ESRGAN, and the ESRGAN repository links pretrained models.

Limitations

• Hallucinated detail. Like every GAN-based super-resolution model, ESRGAN synthesizes textures that are plausible rather than correct; PSNR falls below that of PSNR-oriented models (Fig. 7).
• Evaluation by example. Neither paper reports a user study; quantitative evidence is a perceptual index on one validation set and NIQE, while the Real-ESRGAN authors themselves note that existing perceptual metrics cannot well reflect fine-grained human preferences.
• Bicubic training for ESRGAN. Trained on bicubic downscaling, ESRGAN inherits the degradation mismatch described in Super-Resolution; Real-ESRGAN’s Fig. 1 compares ESRGAN’s outputs on real-life images with its own.
• Real-ESRGAN’s own failures (§4.4, Fig. 11): twisted lines from aliasing, especially in buildings and indoor scenes; artifacts introduced by GAN training on some samples; and out-of-distribution degradations that it cannot remove and may amplify.
• Cost. At about 16.7 million parameters with every RRDB at the input resolution, the network is heavy; Real-ESRGAN calls ESRGAN a heavy network and introduces pixel unshuffle partly to reduce memory and computation.
• Settings that live in code. The block count, $\beta$, the initialization scale, the halved losses and the resize probabilities and ranges are given only by the code, so a reimplementation from the paper alone would differ.

Historical Importance

ESRGAN appeared in 2018, a year after SRGAN’s CVPR paper, and won region 3 of the PIRM 2018 challenge, the first super-resolution competition, in the authors’ description, to evaluate perceptual quality (§1). The SRGAN paper article describes it as revising all three of SRGAN’s parts. Its changes reached other articles separately: removing batch normalization from residual blocks followed EDSR, and the before-activation feature loss is part of the design discussion in Perceptual Loss. In 2021 Real-ESRGAN and BSRGAN (Zhang et al., 2021) turned the problem toward synthetic real-world degradations; Super-Resolution notes that both train ESRGAN-style networks on such pairs and target photographs without a high-resolution reference.

Variants

Real-ESRGAN (2021)

Real-ESRGAN extends the classical degradation model $x = [(y \circledast k)\downarrow_r + n]_{\text{JPEG}}$ (Eq. 1 of the 2021 paper) to a high-order one, $x = (\mathcal{D}_n \circ \cdots \circ \mathcal{D}_1)(y)$ (Eq. 5), in which each $\mathcal{D}_i$ is a full classical degradation with its own random parameters. “High order” differs from the order of a mathematical function and refers to how many times the same operation is applied (§3.2). The authors use $n = 2$ as a balance of simplicity and effectiveness, and replace the downsampling of each stage by a random resize. Their motivation is the life of a real photograph: taken by a phone with blur, noise and JPEG, then edited and uploaded, each step adding its own degradation (§1, §3.2). Its ablation (Fig. 8) shows a model trained with a first-order process failing to remove noise and blur that the second-order model handles.

Degradation settings in the paper (§4.1). Blur kernels are Gaussian, generalized Gaussian and plateau-shaped with probabilities {0.7, 0.15, 0.15}, of size 7 to 21; $\sigma$ in [0.2, 3] for the first stage and [0.2, 1.5] for the second; a sinc kernel replaces the blur with probability 0.1; the second blur is skipped with probability 0.2. Noise is Gaussian or Poisson with equal probability, gray (the same in all channels) with probability 0.4, with $\sigma$ in [1, 30] and [1, 25] and Poisson scales in [0.05, 3] and [0.05, 2.5]. JPEG quality is drawn from [30, 95]. A final sinc filter is applied with probability 0.8.

Sinc filter. To synthesize ringing and overshoot, the paper uses an ideal circular low-pass kernel

$$k(i, j) = \frac{\omega_c}{2\pi\sqrt{i^2 + j^2}}\, J_1\big(\omega_c \sqrt{i^2 + j^2}\big), \tag{6}$$

with cutoff $\omega_c$ and $J_1$ the first-order Bessel function of the first kind. It is applied in the blur stage and as the last step, and the order of the last sinc filter and the final JPEG is exchanged at random, because some images are over-sharpened and then compressed and others compressed and then sharpened (§3.3). Frequency Domain Filtering explains why such a kernel rings.

What the code adds (Real-ESRGAN v0.3.0, BasicSR v1.4.2). The kernel probabilities are split into six entries, [0.45, 0.25, 0.12, 0.03, 0.12, 0.03] for isotropic and anisotropic Gaussian, generalized Gaussian and plateau kernels, which sum to the paper’s {0.7, 0.15, 0.15}. circular_lowpass_kernel evaluates Eq. 6, sets the centre to its limit $\omega_c^2/4\pi$ and divides by the sum. The cutoff is drawn from $[\pi/3, \pi]$ for kernels smaller than 13 and from $[\pi/5, \pi]$ otherwise, and from $[\pi/3, \pi]$ for the final filter; without a final sinc, an identity pulse is used. Resizing is up, down or kept with probabilities [0.2, 0.7, 0.1] in the first stage (scale range [0.15, 1.5]) and [0.3, 0.4, 0.3] in the second ([0.3, 1.2]), by area, bilinear or bicubic F.interpolate. Filtering is a correlation with reflect padding. The random final order is between “[resize back + sinc] then JPEG” and “JPEG then [resize back + sinc]”, each with probability 0.5; a comment says other combinations introduced twisted lines. JPEG is DiffJPEG, a PyTorch reimplementation, which the supplement shows differs slightly from OpenCV’s encoder (§1.4 of the supplement). Every image is clipped to [0, 1] before compression and rounded to 8 bits at the end.

Training pair pool. Because degradations are synthesized on the GPU in batches, all images in a batch would share a resize factor; a pool of 180 pairs is shuffled to mix them (§4.1).

U-Net discriminator with spectral normalization. The VGG-style discriminator is replaced by a U-Net with skip connections that outputs a realness value per pixel, giving per-pixel feedback to the generator, and spectral normalization (Miyato et al., 2018) is added to stabilize training; the authors also observed that it reduces over-sharp artifacts (§3.4, Fig. 9). In the code, every convolution except the first and last is spectrally normalized, the encoder has three stride-2 levels and the decoder bilinear upsampling.

Losses. Real-ESRNet, the PSNR-oriented first stage, is trained with $L_1$ for 1,000k iterations at $2 \times 10^{-4}$, starting from ESRGAN; Real-ESRGAN then trains for 400k iterations at $10^{-4}$ with $L_1$, perceptual and GAN losses weighted {1, 1, 0.1}, using VGG-19 features conv1 to conv5 before activation with weights {0.1, 0.1, 1, 1, 1}, an exponential moving average and 256-pixel training crops in batches of 48 on four GPUs (§4.1). The paper names the adversarial loss only as a “GAN loss”. It is not relativistic: RealESRGANModel computes the generator loss as cri_gan(fake_g_pred, True, is_disc=False), a standard non-saturating binary cross-entropy on the U-Net’s per-pixel logits, and the discriminator loss as standard real and fake terms, with no batch means.

Real-ESRGAN+. Sharpening the ground truth with unsharp masking during training, rather than sharpening outputs afterwards, gave sharper results with fewer overshoot artifacts in the authors’ experiments (§4.1); Image Sharpening covers unsharp masking. In the released train_realesrgan_x4plus.yml, the sharpened ground truth is used for the $L_1$ and perceptual losses but not for the GAN loss (l1_gt_usm: True, percep_gt_usm: True, gan_gt_usm: False), and the first blur of the degradation is applied to the sharpened image. The paper does not say which losses use the sharpened targets.

Implementations

As of November 2026 the original repository xinntao/ESRGAN (Apache-2.0) holds test code, the network interpolation script and links to pretrained models, and states that it will not be modified; training code is in BasicSR (XPixelGroup/BasicSR, Apache-2.0) and Real-ESRGAN (xinntao/Real-ESRGAN, BSD-3-Clause), whose model zoo, according to the ESRGAN README, also serves the original ESRGAN model. The descriptions above are from BasicSR v1.4.2 and Real-ESRGAN v0.3.0.

The example below trains nothing and loads no weights. It checks the non-learned parts of the two models exactly: Real-ESRGAN’s second-order degradation as a known operator, the relativistic losses on fixed discriminator scores, and the generator’s size.

Set-up. NumPy 2.3.5, SciPy 1.16.3 and Pillow 12.0.0. The runtime is about one second.

• Part A, the degradation. A synthetic 512 × 512 grey image (a ramp, a disk, a rectangle and bars, all between 0.28 and 0.72, so that no noisy pixel clips) goes through one second-order degradation with parameters fixed inside the released ranges rather than drawn at random. Stage 1: an isotropic Gaussian blur ($\sigma = 2$, size 21), area resize by 0.5 to 256 × 256, gray Gaussian noise of $\sigma = 10/255$, and JPEG at quality 80. Stage 2: a Gaussian blur ($\sigma = 1$), no resize, noise of $\sigma = 8/255$. Final step, in both of the code’s orders: area resize by 0.5 to 128 × 128 plus a sinc filter of size 21, and JPEG at quality 60. The cutoff is a free choice within $[\pi/3, \pi]$, so two values are printed, $\pi/3$ and $2\pi/3$. Filters are correlations with reflect padding as in filter2D. Pillow’s libjpeg encoder on 8-bit greyscale images stands in for DiffJPEG and is not the same codec. The parameters were chosen at planning so that the effects show, so Part A is an illustration of the operator, not a test of Real-ESRGAN.
• Noise correlation. Because blur and resize are linear, stage-1 noise that passes through the stage-2 blur $k$ becomes the noise field $k * n_1$, whose lag-1 autocorrelation is $\hat\rho = \sum_{i,j} k(i,j)\,k(i,j+1) / \sum_{i,j} k(i,j)^2$ for white $n_1$. The script measures it on the noise propagated through the blur (16-pixel border excluded), for stage-2 noise, which is added after the last blur as in a first-order model, and for stage-1 noise passed through the stage-1 JPEG (pipeline with noise minus pipeline without), over noise seeds 0 to 19.
• Final order. For seed 0 it prints the largest difference between applying the stage-2 blur and the sinc in either order (circular boundaries), the RMS difference between the two final orders in 8-bit grey levels, and a blockiness ratio: mean absolute step across block edges over mean absolute step elsewhere. The JPEG grid is 8 pixels in the order that compresses last and 4 pixels in the other, which compresses at 256 × 256 before the resize; that order is compared with the same resize and sinc without JPEG.
• Part B, the relativistic losses. Sixteen real and sixteen fake scores $C(x)$ (the paper’s batch size) are drawn from normal distributions with standard deviation 1 under three conditions: the discriminator separates them (means +2 and −2), is unsure (0 and 0) or is fooled (−1 and +1). For seed 0 the script compares analytic gradients of Eqs. 1–2 with central finite differences, moves the real scores and measures how the generator’s gradient on the fake scores changes, adds 3.7 to every score, and evaluates BasicSR’s loss formulas. Over seeds 0 to 19 it prints the share of $\partial L_G^{Ra}/\partial C(x_f)$ coming from the real term of Eq. 2, the error of BasicSR’s generator gradient (by finite differences) against $\tfrac12\lambda\,\partial$Eq. 2, and the relative difference between BasicSR’s discriminator gradient on the real scores, with the other set’s mean held fixed as in the code, and $\tfrac12\,\partial$Eq. 1.
• Part C counts the parameters of BasicSR’s RRDBNet from its layer shapes.

import os
os.environ["OMP_NUM_THREADS"] = "1"
os.environ["OPENBLAS_NUM_THREADS"] = "1"
import io
import numpy as np
from scipy import ndimage, special
from PIL import Image

# ---------- Part A: Real-ESRGAN's second-order degradation, fixed parameters ----------
def scene(n=512):
    """Synthetic HR image, grey levels 0.28-0.72 so that no noisy pixel clips."""
    yy, xx = np.mgrid[0:n, 0:n] / n
    img = 0.35 + 0.2 * xx                                           # ramp
    img[(xx - 0.3) ** 2 + (yy - 0.35) ** 2 < 0.18 ** 2] = 0.72      # disk
    img[(yy > 0.6) & (yy < 0.9) & ((xx * n).astype(int) % 16 < 4) & (xx > 0.55)] = 0.28  # bars
    img[(yy > 0.65) & (yy < 0.85) & (xx > 0.1) & (xx < 0.45)] = 0.6  # rectangle
    return img

def gaussian_kernel(size, sigma):                  # isotropic case of Eq. 2
    r = np.arange(size) - (size - 1) / 2
    k = np.exp(-(r[:, None] ** 2 + r[None, :] ** 2) / (2 * sigma ** 2))
    return k / k.sum()

def sinc_kernel(wc, size):                         # Eq. 6, centre set to its limit, then normalized
    r = np.arange(size) - (size - 1) / 2
    rho = np.hypot(r[:, None], r[None, :])
    with np.errstate(invalid="ignore", divide="ignore"):
        k = wc * special.j1(wc * rho) / (2 * np.pi * rho)
    c = (size - 1) // 2
    k[c, c] = wc ** 2 / (4 * np.pi)
    return k / k.sum(), k.sum()

def filt(x, k, mode="mirror"):                     # correlation, reflect padding like filter2D
    return ndimage.correlate(x, k, mode=mode)

def area_down2(x):                                 # area resize by 0.5
    return 0.25 * (x[0::2, 0::2] + x[1::2, 0::2] + x[0::2, 1::2] + x[1::2, 1::2])

def jpeg(x, q):                                    # Pillow/libjpeg as a stand-in for DiffJPEG
    u8 = np.clip(np.round(x * 255), 0, 255).astype(np.uint8)
    buf = io.BytesIO()
    Image.fromarray(u8, mode="L").save(buf, format="JPEG", quality=q)
    return np.asarray(Image.open(io.BytesIO(buf.getvalue())), dtype=np.float64) / 255

def lag1(z, m=16):                                 # horizontal and vertical lag-1 autocorrelation
    z = z[m:-m, m:-m]
    z = z - z.mean()
    v = (z * z).mean()
    return (z[:, 1:] * z[:, :-1]).mean() / v, (z[1:, :] * z[:-1, :]).mean() / v

def blockiness(x, P):                              # |step| across P-pixel block edges / elsewhere
    d, e = np.abs(np.diff(x, axis=1)), np.abs(np.diff(x, axis=0))
    at = (np.arange(d.shape[1]) % P) == P - 1
    edge = np.concatenate([d[:, at].ravel(), e[at, :].ravel()]).mean()
    rest = np.concatenate([d[:, ~at].ravel(), e[~at, :].ravel()]).mean()
    return edge / rest

y = scene()
K1, K2 = gaussian_kernel(21, 2.0), gaussian_kernel(21, 1.0)
s1, s2 = 10 / 255, 8 / 255
pred = (K2[:, 1:] * K2[:, :-1]).sum() / (K2 ** 2).sum()
a = area_down2(filt(y, K1))                        # stage 1: blur, resize
print(f"A. Second-order degradation: HR {y.shape[0]}, stages 1-2 {a.shape[0]}, LR {area_down2(a).shape[0]}")

print(f"scene range [{y.min():.2f}, {y.max():.2f}]; Gaussian kernels sum-1: {abs(K1.sum()-1):.1e} {abs(K2.sum()-1):.1e}")
for wc, name in ((np.pi / 3, "pi/3"), (2 * np.pi / 3, "2pi/3")):
    k, raw = sinc_kernel(wc, 21)
    eps = 1e-6
    lim = abs(wc * special.j1(wc * eps) / (2 * np.pi * eps) - wc ** 2 / (4 * np.pi))
    print(f"sinc wc={name:5s}: |Eq.6 near 0 - wc^2/4pi| {lim:.2e}  raw sum {raw:.6f}  "
          f"|sum-1| {abs(k.sum()-1):.2e}  min {k.min():.6f}")

rows = {"stage-1 noise after stage-2 blur": [], "stage-2 noise (white)": [], "stage-1 noise through JPEG q=80, blur": []}
clipped = 0
for seed in range(20):
    rng = np.random.default_rng(seed)
    n1, n2 = rng.normal(0, s1, a.shape), rng.normal(0, s2, a.shape)
    clipped += int(((a + n1) < 0).sum() + ((a + n1) > 1).sum())
    rows["stage-1 noise after stage-2 blur"].append(lag1(filt(n1, K2)))
    rows["stage-2 noise (white)"].append(lag1(n2))
    d = filt(jpeg(np.clip(a + n1, 0, 1), 80), K2) - filt(jpeg(np.clip(a, 0, 1), 80), K2)
    rows["stage-1 noise through JPEG q=80, blur"].append(lag1(d))
print(f"clipped noisy pixels over 20 seeds: {clipped}")
print(f"predicted lag-1 correlation of blurred white noise: {pred:.5f}")
print("lag-1 autocorrelation over seeds 0-19     dir  mean      sd       min       max")
for name, v in rows.items():
    v = np.array(v)
    for j, dname in enumerate(("h", "v")):
        print(f"{name:40s}  {dname}  {v[:, j].mean():8.5f} {v[:, j].std():8.5f} {v[:, j].min():8.5f} {v[:, j].max():8.5f}")

rng = np.random.default_rng(0)                     # one full degradation, seed 0
n1, n2 = rng.normal(0, s1, a.shape), rng.normal(0, s2, a.shape)
x2 = np.clip(filt(jpeg(np.clip(a + n1, 0, 1), 80), K2) + n2, 0, 1)
print("final step  wc     linear commute   RMS(A-B)  blockiness A(8px)  B(4px)  no-JPEG(4px)")
for wc, name in ((np.pi / 3, "pi/3"), (2 * np.pi / 3, "2pi/3")):
    ks, _ = sinc_kernel(wc, 21)
    comm = np.abs(filt(filt(x2, K2, "wrap"), ks, "wrap") - filt(filt(x2, ks, "wrap"), K2, "wrap")).max()
    A = jpeg(np.clip(filt(area_down2(x2), ks), 0, 1), 60)       # [resize + sinc] then JPEG
    B = filt(area_down2(jpeg(x2, 60)), ks)                       # JPEG then [resize + sinc]
    ref = filt(area_down2(x2), ks)
    rms = np.sqrt(((A - B) ** 2).mean()) * 255
    print(f"            {name:5s}  {comm:.1e}          {rms:6.3f}     {blockiness(A, 8):.4f}          "
          f"{blockiness(B, 4):.4f}  {blockiness(ref, 4):.4f}")

# ---------- Part B: relativistic average discriminator on fixed logits ----------
def sig(u): return 1 / (1 + np.exp(-u))
def log_sig(u): return -np.logaddexp(0, -u)            # log sigma(u)
def log_1msig(u): return -np.logaddexp(0, u)           # log(1 - sigma(u))
def L_D(r, f): return -log_sig(r - f.mean()).mean() - log_1msig(f - r.mean()).mean()   # Eq. 1
def L_G(r, f): return -log_1msig(r - f.mean()).mean() - log_sig(f - r.mean()).mean()   # Eq. 2
def L_G_std(f): return -log_sig(f).mean()                                              # SRGAN's G

def grad_L_D(r, f):
    a, b = r - f.mean(), f - r.mean()
    t1, t2 = -(1 - sig(a)) / len(r), sig(b) / len(f)
    return t1 - t2.sum() / len(r), t2 - t1.sum() / len(f)

def grad_L_G(r, f):
    """Returns dL/dr, dL/df, and dL/df split into the real-data term and the fake-data term."""
    a, b = r - f.mean(), f - r.mean()
    t1, t2 = sig(a) / len(r), -(1 - sig(b)) / len(f)
    from_real = -t1.sum() / len(f) * np.ones(len(f))
    return t1 - t2.sum() / len(r), t2 + from_real, from_real, t2

def grad_L_G_std(f): return -(1 - sig(f)) / len(f)

def bce(x, t):                                   # BCEWithLogitsLoss, mean reduction
    return (np.maximum(x, 0) - x * t + np.log1p(np.exp(-np.abs(x)))).mean()

def num_grad(fun, z, h=1e-5):
    return np.array([(fun(z + h * e) - fun(z - h * e)) / (2 * h) for e in np.eye(len(z))])

lam, N = 5e-3, 16
print("\nB. Relativistic average discriminator, 16 real and 16 fake logits")
for cname, (mr, mf) in {"D separates": (2, -2), "D unsure": (0, 0), "D fooled": (-1, 1)}.items():
    shares, fd5, rel6 = [], [], []
    for seed in range(20):
        rng = np.random.default_rng(seed)
        r, f = rng.normal(mr, 1, N), rng.normal(mf, 1, N)
        _, g_f, from_real, from_fake = grad_L_G(r, f)
        shares.append(np.abs(from_real).sum() / (np.abs(from_real).sum() + np.abs(from_fake).sum()))
        bs_G = lambda ff: (bce(r - ff.mean(), 0) + bce(ff - r.mean(), 1)) / 2 * lam
        fd5.append(np.abs(num_grad(bs_G, f) - 0.5 * lam * g_f).max())
        m_f, m_r = f.mean(), r.mean()                       # detached means
        bs_D = lambda rr: 0.5 * bce(rr - m_f, 1) + 0.5 * bce(f - m_r, 0)
        full = 0.5 * grad_L_D(r, f)[0]
        rel6.append(np.linalg.norm(num_grad(bs_D, r) - full) / np.linalg.norm(full))
        if seed == 0:
            z = np.concatenate([r, f])
            err_D = np.abs(num_grad(lambda q: L_D(q[:N], q[N:]), z) - np.concatenate(grad_L_D(r, f))).max()
            err_G = np.abs(num_grad(lambda q: L_G(q[:N], q[N:]), z) - np.concatenate(grad_L_G(r, f)[:2])).max()
            r2 = r + rng.normal(0, 0.5, N)                      # move the real logits only
            d_std = np.abs(grad_L_G_std(f) - grad_L_G_std(f)).max()   # r does not enter
            d_ra = np.abs(grad_L_G(r2, f)[1] - g_f).max()
            c = 3.7
            sh_D, sh_G = abs(L_D(r + c, f + c) - L_D(r, f)), abs(L_G(r + c, f + c) - L_G(r, f))
            sh_std = abs(L_G_std(f + c) - L_G_std(f))
            bsD_val = 0.5 * bce(r - f.mean(), 1) + 0.5 * bce(f - r.mean(), 0)
            bsG_val = (bce(r - f.mean(), 0) + bce(f - r.mean(), 1)) / 2 * lam
            v_D, v_G = abs(bsD_val - 0.5 * L_D(r, f)), abs(bsG_val - 0.5 * lam * L_G(r, f))
    print(f"{cname} (seed 0): L_D {L_D(r, f):.4f}  L_G {L_G(r, f):.4f}  L_G_std {L_G_std(f):.4f}")
    print(f"  grad check vs finite differences: Eq.1 {err_D:.1e}  Eq.2 {err_G:.1e}")
    print(f"  dL_G/dC(x_f) change when real logits move: standard {d_std:.2e}  relativistic {d_ra:.2e}")
    print(f"  shift all logits by 3.7: |dL_D| {sh_D:.1e}  |dL_G| {sh_G:.1e}  |dL_G_std| {sh_std:.4f}")
    print(f"  BasicSR value - half of Eq.1: {v_D:.1e}   - half lambda Eq.2: {v_G:.1e}")
    print(f"  seeds 0-19: real-term share of dL_G/dC(x_f) {min(shares):.4f}-{max(shares):.4f};"
          f" BasicSR G grad err {max(fd5):.1e}; BasicSR D grad rel. diff {min(rel6):.4f}-{max(rel6):.4f}")

# ---------- Part C: RRDBNet parameter count from the layer shapes ----------
def conv(cin, cout, k=3): return k * k * cin * cout + cout
rdb = sum(conv(64 + 32 * j, 32) for j in range(4)) + conv(64 + 4 * 32, 64)
print(f"\nC. RRDBNet: RDB {rdb:,}  RRDB {3 * rdb:,}")
for scale, cin in ((4, 3), (2, 12), (1, 48)):
    total = conv(cin, 64) + 23 * 3 * rdb + 4 * conv(64, 64) + conv(64, 3)
    print(f"x{scale}: input channels {cin:2d}  parameters {total:,}  share in the 23 RRDBs {23 * 3 * rdb / total:.4f}")

Output:

A. Second-order degradation: HR 512, stages 1-2 256, LR 128
scene range [0.28, 0.72]; Gaussian kernels sum-1: 0.0e+00 0.0e+00
sinc wc=pi/3 : |Eq.6 near 0 - wc^2/4pi| 1.20e-14  raw sum 0.994398  |sum-1| 0.00e+00  min -0.011551
sinc wc=2pi/3: |Eq.6 near 0 - wc^2/4pi| 1.91e-13  raw sum 1.076250  |sum-1| 2.22e-16  min -0.038161
clipped noisy pixels over 20 seeds: 0
predicted lag-1 correlation of blurred white noise: 0.77864
lag-1 autocorrelation over seeds 0-19     dir  mean      sd       min       max
stage-1 noise after stage-2 blur          h   0.77732  0.00320  0.77232  0.78399
stage-1 noise after stage-2 blur          v   0.77703  0.00249  0.77103  0.78027
stage-2 noise (white)                     h  -0.00010  0.00318 -0.00420  0.00961
stage-2 noise (white)                     v   0.00036  0.00452 -0.00975  0.00906
stage-1 noise through JPEG q=80, blur     h   0.77148  0.00304  0.76636  0.77768
stage-1 noise through JPEG q=80, blur     v   0.77422  0.00287  0.76913  0.77905
final step  wc     linear commute   RMS(A-B)  blockiness A(8px)  B(4px)  no-JPEG(4px)
            pi/3   2.7e-15           1.466     1.7320          0.9978  0.9918
            2pi/3  2.7e-15           2.688     1.5869          1.0130  1.0050

B. Relativistic average discriminator, 16 real and 16 fake logits
D separates (seed 0): L_D 0.0262  L_G 9.3437  L_G_std 2.5785
  grad check vs finite differences: Eq.1 1.1e-12  Eq.2 8.8e-11
  dL_G/dC(x_f) change when real logits move: standard 0.00e+00  relativistic 1.47e-03
  shift all logits by 3.7: |dL_D| 0.0e+00  |dL_G| 0.0e+00  |dL_G_std| 1.9837
  BasicSR value - half of Eq.1: 0.0e+00   - half lambda Eq.2: 0.0e+00
  seeds 0-19: real-term share of dL_G/dC(x_f) 0.4964-0.5017; BasicSR G grad err 4.4e-13; BasicSR D grad rel. diff 0.2650-0.5618
D unsure (seed 0): L_D 0.9793  L_G 2.2967  L_G_std 1.0306
  grad check vs finite differences: Eq.1 1.9e-11  Eq.2 1.8e-11
  dL_G/dC(x_f) change when real logits move: standard 0.00e+00  relativistic 7.36e-03
  shift all logits by 3.7: |dL_D| 2.2e-16  |dL_G| 2.2e-16  |dL_G_std| 0.7386
  BasicSR value - half of Eq.1: 0.0e+00   - half lambda Eq.2: 0.0e+00
  seeds 0-19: real-term share of dL_G/dC(x_f) 0.4918-0.5114; BasicSR G grad err 7.6e-14; BasicSR D grad rel. diff 0.4718-0.5017
D fooled (seed 0): L_D 3.2546  L_G 0.5720  L_G_std 0.5348
  grad check vs finite differences: Eq.1 4.8e-11  Eq.2 3.2e-12
  dL_G/dC(x_f) change when real logits move: standard 0.00e+00  relativistic 5.02e-03
  shift all logits by 3.7: |dL_D| 0.0e+00  |dL_G| 0.0e+00  |dL_G_std| 0.3568
  BasicSR value - half of Eq.1: 0.0e+00   - half lambda Eq.2: 0.0e+00
  seeds 0-19: real-term share of dL_G/dC(x_f) 0.4601-0.5841; BasicSR G grad err 1.9e-14; BasicSR D grad rel. diff 0.4904-0.5084

C. RRDBNet: RDB 239,808  RRDB 719,424
x4: input channels  3  parameters 16,697,987  share in the 23 RRDBs 0.9909
x2: input channels 12  parameters 16,703,171  share in the 23 RRDBs 0.9906
x1: input channels 48  parameters 16,723,907  share in the 23 RRDBs 0.9894

Checks. The planned checks all held. Eq. 6 meets its centre limit and the normalized kernels sum to one; both kernels have negative lobes, deeper at the higher cutoff, which are what produce ringing; and the unnormalized sums (0.994398 and 1.076250) show why the code divides by the sum after truncating to 21 × 21. The data check held: the scene stays within its planned range, no noisy pixel clipped, so the noise passes through the blur and resize linearly. The analytic gradients agree with finite differences to $10^{-10}$ or better in every condition, within the planned $10^{-8}$; this check held.

Second order makes correlated noise. Stage-1 noise after the stage-2 blur has a mean lag-1 autocorrelation within 0.002 of the prediction 0.77864 in both directions, against a planned tolerance of 0.004, so this outcome held; single seeds range from 0.771 to 0.784. Stage-2 noise, added after the last blur as a first-order model adds it, is white: its means are within 0.0004 of zero and no seed exceeds 0.01 in magnitude, inside the planned 0.004 and 0.02, so this outcome held too. A single classical degradation adds white noise at the end; a second stage turns the first stage’s noise into spatially correlated grain whose correlation is set by the later blur. Passing the stage-1 noise through the stage-1 JPEG first lowers the correlation only slightly here (0.007 and 0.004 below the prediction at quality 80, within the planned 0.03), an outcome that held, so in this setting the effect survives the nonlinear step.

Order matters only at the nonlinear step. The stage-2 blur and the sinc filter commute to rounding error ($2.7 \times 10^{-15}$), as any two convolutions do, so their order needs no randomization; this outcome held. JPEG does not commute with [resize + sinc]: the two final orders differ by 1.5 and 2.7 grey levels RMS, more at the higher cutoff, above the planned 0.5, so this outcome held. The blockiness ratios show what the difference is. When JPEG comes last, its 8-pixel blocks remain (ratios 1.73 and 1.59, well above the 1.2 planned as the bar); when JPEG comes first, the resize and the low-pass sinc remove its blocks, and the ratio on the 4-pixel grid stays within 0.01 of the image that was never compressed (planned: within 0.05). Both parts of this outcome held at both cutoffs. This is the variety the random order buys: training inputs with compression artifacts on top of ringing, and others with ringing on top of smoothed compression.

The relativistic generator sees the real scores. In all three conditions, moving the real scores changes the relativistic gradient on the fake scores ($1.5 \times 10^{-3}$ to $7.4 \times 10^{-3}$), and the real term of Eq. 2 supplies between 0.46 and 0.58 of that gradient across seeds and conditions; this outcome held. The printed zero for the standard loss is not a finding: SRGAN’s non-saturating gradient $-(1 - \sigma(C(x_f)))/N$ takes no real scores, so its change is zero by construction. Together they make the paper’s “gradients from both generated data and real data” precise: the generator gradient still enters only through the fake images, but how strongly each fake is pushed depends on how the real images are scored. Shifting every score by 3.7 leaves Eqs. 1 and 2 unchanged to $2.2 \times 10^{-16}$ and changes the standard loss by 0.36 to 1.98; this outcome held, so the relativistic losses depend only on score differences, as the equations imply.

BasicSR against the paper. BasicSR’s loss values equal half of Eq. 1 and half of $\lambda$ times Eq. 2 exactly, and its generator gradient matches $\tfrac12\lambda$ times that of Eq. 2 to $4.4 \times 10^{-13}$; this outcome held. Its discriminator gradient on the real scores, with the other set’s mean detached as in esrgan_model.py, differs from half of Eq. 1’s full gradient by 27% to 56% of its norm in the separated case and by about half in the other two; this outcome held, so the released discriminator update is not the gradient of Eq. 1 but the gradient of each term with the other batch’s mean treated as a constant.

Size. Each dense block holds 239,808 parameters and an RRDB 719,424. The ×4 network has 16,697,987 parameters, 99.09% in the trunk; the ×2 and ×1 variants, whose first convolution takes 12 or 48 channels after pixel unshuffle, add a few thousand.