BigGAN

BigGAN is a class-conditional generative adversarial network for ImageNet, introduced by Andrew Brock, Jeff Donahue and Karen Simonyan in “Large Scale GAN Training for High Fidelity Natural Image Synthesis” (arXiv, September 2018; ICLR 2019). It takes a class label and a Gaussian latent vector and produces a $128 \times 128$, $256 \times 256$ or $512 \times 512$ image in one forward pass of a residual convolutional generator. It is a conditional GAN in the projection family: class information enters the generator through conditional batch normalization and the discriminator through a projection layer. Two sampling-time and training-time devices are associated with it: the truncation trick, which shrinks the latent distribution to trade variety for sample quality, and orthogonal regularization of the generator, which, according to the official README, made a model more likely to tolerate that shrinking. A deeper version, BigGAN-deep, uses bottleneck residual blocks. For several years BigGAN and BigGAN-deep were the GAN reference point for class-conditional ImageNet generation, against which VQ-VAE-2, MaskGIT, latent diffusion and the Diffusion Transformer reported their results.

Sources. The BigGAN paper is published on OpenReview, which this site does not use, so this article was not written from it. It is built on the authors’ official PyTorch code (Brock and Andonian, BigGAN-PyTorch, read at commit 9845943) and on readable later papers that describe or report BigGAN. Where a detail comes from the code rather than the paper, the article says so; the README lists where the code departs from the paper. The GAN article summarizes the paper’s scaling results and its Table 2, from an earlier reading of the arXiv version.

Problem Addressed

By 2018, class-conditional GANs on ImageNet had moved from auxiliary-classifier models to projection discriminators with spectral normalization, and then to Self-Attention GAN (SAGAN; Zhang, Goodfellow, Metaxas & Odena, 2019), which added self-attention, spectral normalization in the generator as well as the discriminator, and separate learning rates for the two networks. The conditional GAN article tabulates that progression: on ImageNet $128 \times 128$, SAGAN’s Inception Score was 52.52 and its FID 18.65. ImageNet remained hard for GANs: Zhang et al. note that convolutional GANs found some classes much harder to model than others on multi-class data such as ImageNet.

As its title says, BigGAN studied large-scale training. SAGAN’s own paper notes in a footnote that Brock et al. (2018), “which builds heavily on this work”, had since improved its results substantially. Razavi, van den Oord and Vinyals (2019, §4), writing a few months later, attributed BigGAN’s improvements mostly to incorporating architectural advances such as self-attention, better stabilization methods, scaling the model up on TPUs, and a mechanism to trade sample diversity for sample quality. The cost of that scale was stability: large models trained well for a time and then collapsed, as the official code’s README and later papers document.

Architecture

The generator and discriminator are residual networks. In the official code, the $128 \times 128$ generator starts from a dense layer that maps a chunk of the latent vector to a $4 \times 4 \times 16\,\mathrm{ch}$ tensor, then applies five upsampling residual blocks with output widths $16, 8, 4, 2, 1$ times the channel multiplier $\mathrm{ch}$, a self-attention block at $64 \times 64$, and a final batch normalization, ReLU, $3 \times 3$ convolution and $\tanh$ (BigGAN.py, G_arch). The discriminator mirrors it with downsampling residual blocks, self-attention at $64 \times 64$, a ReLU, global sum pooling and a projection head. The launch script for the full-size model sets $\mathrm{ch} = 96$ in both networks (scripts/launch_BigGAN_bs256x8.sh). Lucic et al. (2019, Appendix B) independently reimplemented the same architecture, matched it to the parameter shapes of Brock et al.’s released checkpoints, and list it layer by layer; their Tables 6–9 agree with the code.

Generator blocks and class conditioning

Each generator block applies batch normalization, ReLU, nearest-neighbour upsampling by 2 and a $3 \times 3$ convolution, then a second batch normalization, ReLU and $3 \times 3$ convolution, and adds a shortcut that is upsampled and passed through a $1 \times 1$ convolution (layers.GBlock). Every batch normalization in the blocks is conditional: it normalizes with batch statistics, without learned affine parameters of its own, and then applies a per-channel gain and bias predicted from a conditioning vector $c$:

$$\mathrm{ccbn}(h, c) = \big(1 + W_\gamma c\big) \odot \frac{h - \mu_B}{\sqrt{\sigma_B^2 + \epsilon}} + W_\beta c,$$

where $W_\gamma$ and $W_\beta$ are bias-free linear layers and the “1 +” makes the layer start close to plain normalization (layers.ccbn). The conditional GAN article explains conditional batch normalization and its origin.

Two choices in the conditioning are particular to BigGAN:

• Shared embedding. The official code of the projection discriminator stores a separate gain and bias per class in each generator block, as the conditional GAN article describes. BigGAN instead learns one class embedding $e(y)$, 128-dimensional in the launch script (--G_shared --shared_dim 128), shared by all layers, and each layer’s linear maps project it to that layer’s gains and biases. The code does not apply spectral normalization to this embedding; a comment says that doing so “seems to randomly cripple G”.
• Hierarchical latent. With --hier, the latent $z$ is split into equal chunks, one for the dense input layer and one for each residual block. At $128 \times 128$, $z$ has 120 dimensions split into six chunks of 20; each block’s conditioning vector is the concatenation $c_i = [e(y); z_i]$, so the conditional batch normalization layers receive both the class and a part of the noise (Generator.forward). Lucic et al.’s parameter listing shows the result: each gain and bias kernel of their first block has shape $(148, 1536)$, that is $128 + 20$ inputs. They also note (§3.2, footnote 2) that removing the labels from such a generator is not the same as using unconditional batch normalization, since the layers still take latent chunks.

Härkönen et al. (2020, §2.1), studying the trained models, call the latent inputs to intermediate layers “Skip-$z$” inputs; the model they studied, BigGAN-deep, feeds every layer the whole $z$ rather than a chunk (see BigGAN-deep below).

Self-attention

Both networks include one self-attention block of the SAGAN form (Zhang et al., 2019, Eqs. 1–3), described in the self-attention article. In the official code, the query and key maps reduce channels to $C/8$ and the value map to $C/2$, the key and value maps are max-pooled by 2 to cut the cost of the attention matrix, and the block’s output is scaled by a learned scalar $\gamma$ initialized at 0 and added to its input, so that the network starts without attention (layers.Attention). Its position depends on resolution: the TF Hub conversion script places it at $64 \times 64$ for the $128$ and $512$ models and at $128 \times 128$ for the $256$ model (TFHub/converter.py, get_config).

Discriminator and projection

Discriminator blocks apply ReLU (omitted at the input of the first block), a $3 \times 3$ convolution, ReLU, a second $3 \times 3$ convolution and $2 \times 2$ average pooling, with a $1 \times 1$ convolution on the shortcut; the last block neither downsamples nor convolves its shortcut, and the “wide” variant used by BigGAN and SAGAN sets the hidden width to the block’s output width (layers.DBlock). After the last block, the features are passed through a ReLU and summed over space to a vector $\phi(x)$, and the output is

$$D(x, y) = w^\top \phi(x) + b + e_D(y)^\top \phi(x),$$

a class-independent real/fake score plus the inner product of $\phi(x)$ with a learned class embedding $e_D(y)$: the projection discriminator of Miyato and Koyama (Discriminator.forward; Lucic et al., 2019, §3 and Figure 3, who note that this form of conditioning is used in BigGAN). The conditional GAN article derives the projection form.

Normalization and initialization

Every convolution and linear layer in both networks is spectrally normalized: its weight is divided by an estimate of its largest singular value, updated by one power iteration per step (layers.SN). The class embedding of the discriminator is spectrally normalized too. The launch scripts initialize all weights as orthogonal matrices (--G_init ortho --D_init ortho). The spectral norm layers also store their singular-value estimates for logging, and the code can track several singular values per layer (--num_G_SVs), which is how training spectra can be monitored.

Training

Data. The code and the reimplementations train on the ImageNet training set, which Lucic et al. (2019, §4) describe as 1.3 million images in 1,000 classes and call the largest and most diverse image dataset commonly used to evaluate GANs; they resize the images to $128 \times 128$, as Miyato and Koyama and SAGAN did.

Losses. BigGAN uses the hinge loss, which SAGAN also used (Zhang et al., 2019, Eq. 4). In the official code (losses.py),

$$\mathcal{L}_D = \mathbb{E}_{(x, y)}\big[\max(0, 1 - D(x, y))\big] + \mathbb{E}_{z, y}\big[\max(0, 1 + D(G(z, y), y))\big], \qquad \mathcal{L}_G = -\,\mathbb{E}_{z, y}\big[D(G(z, y), y)\big],$$

with $z \sim \mathcal{N}(0, I)$ and generated classes $y$ drawn uniformly.

Optimization. Lucic et al. (2019, §4) report using “exactly the same optimization hyper-parameters as Brock et al.”: Adam with learning rates $5 \cdot 10^{-5}$ for the generator and $2 \cdot 10^{-4}$ for the discriminator, $\beta_1 = 0$, $\beta_2 = 0.999$, two discriminator steps per generator step, batch size 2,048 and a 120-dimensional latent. The official README gives the same paper values, and says the PyTorch code instead uses SAGAN’s settings ($10^{-4}$ and $4 \cdot 10^{-4}$, one discriminator step) to shorten training, at slightly lower performance. The code assumes no access to a full TPU pod and accumulates gradients instead: the default script uses batches of 256 and takes an optimizer step every eight of them. On eight V100 GPUs it takes 15 days to reach 150,000 iterations (README).

Moving average and batch statistics. The code keeps an exponential moving average of the generator’s weights with decay 0.9999, starting at iteration 20,000, and evaluates the averaged copy (--ema --use_ema --ema_start 20000; utils.ema). Batch normalization at sampling time needs statistics. The README says that the paper computes “standing statistics”, averaging batch statistics over many forward passes after training, while the PyTorch code by default uses the running averages kept during training, which gradient accumulation keeps closer to standing values; standing statistics remain available as an option. The README also reports that cross-replica (synchronized) batch normalization, whose absence it lists among the differences from the original BigGAN, crippled training in both PyTorch variants the authors tried, so the code does without it. Lucic et al. trained on 128 cores of a TPU v3 pod with batch-normalization statistics synchronized across cores (§4).

Orthogonal regularization

Orthogonal regularization penalizes a weight matrix for having non-orthogonal rows. Liu et al. (2019, §2.1) cite Brock, Lim and Ritchie’s earlier work on introspective adversarial networks for “orthonormal regularization”. The official code has two forms: a default one that pushes $W W^\top$ toward the identity, and a modified one, the one the BigGAN option uses, that penalizes only the off-diagonal entries, so the rows’ norms are free and only their pairwise inner products are pushed toward zero. Both apply their gradient directly rather than differentiating a loss (utils.default_ortho and utils.ortho). For a weight $W$ reshaped to a matrix with one row per output channel, ortho adds

$$\beta \cdot 2\,\big[(W W^\top) \odot (\mathbf{1} - I)\big]\,W$$

to the gradient, which in our reading is the gradient of $\tfrac{\beta}{2}\,\big\lVert W W^\top \odot (\mathbf{1} - I) \big\rVert_F^2$, where $\mathbf{1}$ is the all-ones matrix. It applies to every parameter with at least two axes except the shared class embedding, for which, the code comments, it “makes no sense”. The option --G_ortho defaults to 0, with a code comment giving $\beta = 10^{-4}$ as BigGAN’s default; the launch scripts set it to 0.

Three accounts limit what the regularizer was for. Lucic et al. (2019, §4) left it out of their reimplementation because, citing Brock et al.’s Table 1, it “only marginally” improved sample quality, and they did not use truncation. The official README says that its released checkpoints were trained without orthogonal regularization, that training without it “seems to increase the probability” that a model will not be amenable to truncation, and that this particular model happened to tolerate truncation anyway. Karras, Laine and Aila (2019, Appendix B) summarize Brock et al. as observing that only a subset of networks is amenable to truncation even when orthogonal regularization is used. In our reading of these accounts, the regularizer’s role was to make a generator more likely to be amenable to truncation, not to improve untruncated samples.

Instability and collapse

Large BigGAN runs were stopped before they collapsed. The README describes the main released checkpoint as “taken just before collapse” and offers a second checkpoint from 100,000 generator iterations, “well before collapse”, as easier to fine-tune. Lucic et al. (2019, §5.4) read the paper’s reported FID and IS as those of the model just before collapse, “a form of early stopping”; their own reimplementation, with synchronized batch normalization on TPUs, trained stably for 250,000 generator iterations, and they could only speculate about the difference. Liu et al. (2019, §1), citing the BigGAN paper, write that BigGAN still suffers from training instability, especially as the batch size is scaled up, and that an $R_1$ gradient penalty in the discriminator improved stability at the cost of a 45% reduction in Inception Score. They then studied the singular values of spectrally normalized discriminators in smaller SN-GAN models and found that, in runs that collapsed, many singular values fell sharply after some point in training, which they called spectral collapse (§2.3). The mode collapse article covers collapse in general; the GAN article summarizes the BigGAN paper’s own analysis.

Inference

At sampling time BigGAN takes a class index and a latent vector and runs the generator once. Razavi et al. (2019, supplementary §F) measured about 100 ms for one $256 \times 256$ BigGAN-deep image on a GPU, against minutes for their autoregressive priors. The released models are the generator only: the official repository’s converter for the TF Hub generators sets latent sizes of 120, 140 and 128 for the $128$, $256$ and $512$ models (TFHub/converter.py). BigGAN has no encoder, so a real image cannot be mapped to its latent directly; Li et al. (2022, §3) note that no satisfactory encoder for BigGAN existed, and GAN inversion covers optimization-based alternatives.

The truncation trick

The truncation trick draws the latent from a narrower distribution at sampling time than during training. Karras, Laine and Aila (2019, Appendix B) give the reasoning: low-density regions of the training distribution are poorly represented and hard for a generator to learn, and drawing latents from a truncated or otherwise shrunk sampling space tends to improve average image quality at the cost of some variation. Naeem et al. (2020, §4.1.2) describe it as thresholding the latent noise with a parameter $\psi$ and state the expected effect: for larger $\psi$, fidelity decreases and diversity increases.

“Truncation” names more than one operation in BigGAN’s official sources:

• A truncated normal. Each coordinate is drawn from a standard normal restricted to $[-\tau, \tau]$, which is what redrawing every value with $|z_i| > \tau$ until it falls inside produces. Li et al. (2022, §3.2) describe BigGAN’s trick as sampling $z$ from a truncated normal.
• A scaled truncated normal. The sampling helper in the repository’s TF Hub porting code (deprecated, kept for reference) returns $\psi \cdot \mathrm{TN}(-2, 2)$, a standard normal truncated at $\pm 2$ and multiplied by the truncation value $\psi$, default 0.5 (TFHub/biggan_v1.py, truncated_z_sample). In this convention $\psi = 1$ is already truncated at two standard deviations.
• A scaled normal. The PyTorch code’s truncation curves instead sweep the spread of an untruncated normal (sample.py, --sample_trunc_curves). The value is called a variance, but the sampler passes it to PyTorch’s normal_ as the standard deviation (utils.Distribution). Its help text says this is not exactly identical to a truncated normal but should have approximately the same effect.

The difference matters most in high dimensions. With 120 latent coordinates and $\tau = 2$, each coordinate lies inside the threshold with probability $0.9545$, so a whole vector needs no redrawing with probability $0.9545^{120} \approx 0.004$: nearly every truncated latent differs from an ordinary Gaussian draw. A truncated normal also changes shape as $\tau$ falls, flattening toward a uniform distribution on $[-\tau, \tau]$, whereas scaling keeps the Gaussian shape. Both statements follow from the definition; the sources above do not compare the conventions.

Batch normalization interacts with truncation. The README of the TF Hub converter says the converted models run without truncation and that standing statistics must be accumulated at each truncation level to use it, since, in our reading, a narrower $z$ changes the activations whose statistics the normalization layers apply.

The trade-off in published comparisons. Several later papers chose a truncation setting explicitly:

• Razavi et al. (2019, Figure 5 and supplementary Figure 7) sampled BigGAN-deep at truncation 1.0, to yield its maximum diversity, and still found views and poses, such as top views of tench and close-ups of ostriches, missing from its samples. Their supplementary Figure 9 shows samples at truncation 0.02. In their precision–recall comparison over all 1,000 classes, VQ-VAE-2 had slightly lower precision and higher recall than BigGAN-deep, and for BigGAN-deep their classifier-based rejection sampling worked better than truncation (§5.2.2 and §5.3.1).
• Esser, Rombach and Ommer (2021, Table 4) evaluated the TF Hub BigGAN and BigGAN-deep “truncated at 1.0”.
• Li et al. (2022, §3.2) used truncation 0.9 when sampling a labelled dataset from BigGAN. Lower values would raise image fidelity and label quality, they note, but sample diversity also matters for downstream training, so they added classifier-based rejection sampling for finer control.

Truncation does not always help. Naeem et al. (2020, §4.1.2) truncated the latent of a convolutional VAE on MNIST at magnitude 1: coverage dropped from 0.411 to 0.291 while their density measure barely moved (0.208 to 0.205), so the truncation reduced diversity without improving fidelity. On StyleGAN images of FFHQ, by contrast, precision and density fell and recall and coverage rose as $\psi$ increased (Figure 7). StyleGAN’s own truncation, applied in its intermediate latent space rather than in $z$, is described in the StyleGAN article, and the latent space article covers the geometry of Gaussian latents.

Truncation interacts with evaluation too. The Inception Score rewards confident, class-typical images and does not penalize lost variety, so lowering the threshold raises it, as the GAN article reports from the BigGAN paper. The Fréchet Inception Distance compares generated with real images and so, in our reading, also registers the variety that truncation removes; precision and recall for generative models separate the two effects, as the FID article explains. A paper’s BigGAN number therefore depends on the truncation setting as well as on the model.

Performance

BigGAN’s own results, read from the arXiv version, are summarized in the GAN and image generation articles. The rows below come from readable papers that evaluated or reprinted BigGAN-deep on class-conditional ImageNet, each with its own protocol; values in different rows are not directly comparable.

Source	Resolution	FID	IS	Precision	Recall	Parameters
Peebles & Xie (2023), Table 2	256	6.95	171.4	0.87	0.28	—
Chang et al. (2022), Table 1	256	6.95	198.2	0.87	0.28	160M
Rombach et al. (2022), Table 3	256	6.95	203.6 ± 2.6	0.87	0.28	340M
Peebles & Xie (2023), Table 3	512	8.43	177.90	0.88	0.29	—
Chang et al. (2022), Table 1	512	8.43	232.5	0.88	0.29	—
Esser et al. (2021), Table 4, row “50K”, truncation 1.0	256	7.3	—	—	—	—

The three tables agree on FID, precision and recall but not on the Inception Score, and Chang et al. and Rombach et al. give parameter counts that differ by a factor of about two; none of them says where its BigGAN-deep IS comes from or how it was computed, so the discrepancy cannot be resolved from the papers. Peebles and Xie’s Table 3 notes that, following earlier work, precision and recall at $512 \times 512$ use 1,000 real samples, and also lists the sFID of BigGAN-deep: 7.36 at 256 and 8.13 at 512. Esser et al. list BigGAN at 7.1 in the same row, and 9.6 (BigGAN-deep 9.7) in a row labelled “18K” instead of “50K”; in our reading the labels give the number of samples, and the rows show how much the sample count moves FID.

At $128 \times 128$, reimplementations give a sense of what the architecture reaches without truncation. Lucic et al. (2019, §5) obtained FID 8.4 and IS 75.0 (median of three runs; FID from 50,000 samples against the 50,000 images they call the ImageNet test images), which they describe as reproducing Brock et al.’s FID. The official README reports a TensorFlow Inception Score of $97.35 \pm 1.79$ for its main PyTorch checkpoint, trained with batch 256 and eight accumulations and taken just before collapse. Kang et al.’s (2021) StudioGAN reimplementation, tabulated in the conditional GAN article, reached FID 7.89 with batch 2,048 and 16.36 with batch 256, in our reading an illustration of BigGAN’s dependence on batch size.

Strengths

• Fast sampling. One generator pass per image, about 100 ms for BigGAN-deep at $256 \times 256$ on a GPU in Razavi et al.’s measurement.
• High precision. Its precision of 0.87–0.88 on ImageNet in the tables above is at or above that of the guided diffusion models in the same tables.
• A control knob at sampling time. Truncation trades variety for fidelity without retraining, a role that guidance later played in diffusion models (see classifier-free guidance).
• Reusable design. Kang et al. (2021, §2) describe projection-based GANs, the family BigGAN belongs to, as the standard for conditional image generation, and the official code was written as a base for further research that can be modified in one or two places (README).

Limitations

• Low recall. Recall of 0.28–0.29, against 0.35–0.67 for the diffusion models in Peebles and Xie’s tables and 0.51 for MaskGIT in Chang et al.’s, means much of ImageNet’s variety is not generated, as Razavi et al.’s missing poses illustrate.
• Unstable at scale. Training eventually collapses and the reported models are early-stopped; Liu et al. describe the instability as worse at larger batch sizes.
• Cost. 15 days on eight V100 GPUs to reach 150,000 iterations for the PyTorch reimplementation at $128 \times 128$, according to its README.
• Labels required. BigGAN needs a labelled dataset; Lucic et al. found that training it on only 10% or 20% of the labelled ImageNet images (discarding the rest) collapsed after 60,000 to 120,000 iterations for every seed (§5.2).
• Truncation is not universal. Whether a trained generator tolerates truncation depends on the run, even with orthogonal regularization.
• No encoder. Real images cannot be mapped to the latent space directly, which limits editing and dataset uses.

Historical Importance

BigGAN showed in 2018–2019 that scaling an existing GAN design, with larger batches, wider networks and a few targeted changes, produced a large jump in sample quality on ImageNet. It also made the fidelity–variety trade-off an explicit sampling-time choice, a framing that later appears in evaluations of likelihood models and diffusion models. It became the standard GAN baseline: VQ-VAE-2, Taming Transformers, MaskGIT, latent diffusion and DiT all compared against it, often against BigGAN-deep sampled from its released TF Hub modules. Lucic et al. (2019) used it as the backbone for generation with few labels, matching its FID with 10% of the labels and surpassing it with 20% by combining self-supervised and semi-supervised learning (S³GAN). The diffusion model and generative model articles describe how guided diffusion models later exceeded it on FID while keeping higher recall.

Its components also outlived it. Rombach et al. (2022) built a larger inpainting latent diffusion model whose U-Net, following Dhariwal and Nichol’s diffusion architecture, uses the BigGAN residual block for up- and downsampling (§4.5). Its trained generators became research objects: GANSpace (Härkönen et al., 2020) computed principal components at the first linear layer of BigGAN-512-deep to find edit directions such as zoom and translation, found that they appeared class-independent in the classes tested, and fed different latents into different layers’ Skip-$z$ inputs to get StyleGAN-like style mixing without retraining (§2.2, §2.3, §3.1, Figures 5–7). BigDatasetGAN (Li et al., 2022) used BigGAN as a feature extractor: a single expert annotator labelled about five generated images per class with segmentation masks, a segmentation branch was trained on the generator’s internal features from three groups of resolutions, and the model then generated 100,000 labelled images for pretraining semantic segmentation networks (§3–§4, Table 1).

Variants

BigGAN-deep

BigGAN-deep replaces the residual blocks with deeper bottleneck blocks. In the official code (BigGANdeep.py), each generator block applies conditional batch normalization and ReLU before each of four convolutions: a $1 \times 1$ convolution reducing the channels by a factor of 4, two $3 \times 3$ convolutions and a $1 \times 1$ convolution restoring the width. The shortcut is not convolved; where the block narrows, the shortcut simply keeps the first channels, and it is upsampled with the main path. There are two blocks per resolution (--G_depth 2), only the second of which upsamples. The latent is not split into chunks: the whole $z$ is concatenated with the class embedding and fed to the first layer and to every conditional batch normalization layer. The discriminator uses matching bottleneck blocks, a stem convolution and a shortcut that concatenates new channels from a $1 \times 1$ convolution instead of replacing them. The launch script sets $\mathrm{ch} = 128$ and a 128-dimensional latent. The README warns that the authors had not fully trained BigGAN-deep with this code, so its scripts are untested; the later comparisons that name their source sampled BigGAN-deep from its TF Hub modules (Razavi et al., 2019; Esser et al., 2021).

Later reimplementations

• compare_gan BigGAN (Lucic et al., 2019): a TensorFlow reimplementation matched to the released checkpoints’ parameter shapes, trained without orthogonal regularization or truncation.
• StudioGAN BigGAN (Kang et al., 2021): a PyTorch reimplementation used as the projection baseline in comparisons of conditioning methods; its results are in the conditional GAN article.

StyleGAN, from the same period, moved truncation into a learned intermediate latent space; StyleGAN-XL later reached a lower ImageNet FID than BigGAN-deep (2.30 against 6.95 at $256 \times 256$ in Peebles and Xie’s Table 2).

Implementations

• BigGAN-PyTorch (Brock and Andonian), MIT licence: training and sampling code for BigGAN at $128 \times 128$ with gradient accumulation, scripts for BigGAN-deep, SA-GAN and SN-GAN, a converter for the TF Hub generators, and logs of the reference run. The last commit is from July 2019, as of November 2026, and the README lists PyTorch 1.0.1. Its README notes that its Inception metrics use PyTorch’s Inception network and are “only for monitoring purposes”; official scores need the TensorFlow Inception code.
• TF Hub modules: pretrained BigGAN generators at $128$, $256$ and $512$ (the resolutions the official converter handles) and BigGAN-deep, published under tfhub.dev/deepmind, which Razavi et al. (2019) and Esser et al. (2021) cite as their source of samples.
• compare_gan (Google) and StudioGAN (Kang et al.) include BigGAN reimplementations.