AlexNet

AlexNet is a convolutional neural network for image classification with five convolutional and three fully connected layers, introduced by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton in 2012; trained across two GPUs on ImageNet, it was the basis of the entry that won ILSVRC-2012. This article covers the network as an architecture: its layers and shapes, the two-GPU split read as grouped convolution, where its parameters sit, the single-GPU variant that libraries implement, and how it is still used. The publication itself, with its ablations, local response normalization, training recipe and results, is covered in the paper article.

Problem Addressed

The task is 1,000-way classification of natural photographs at ImageNet scale, for which convolutional networks had been too expensive to train (the paper article describes the systems it replaced). Several of AlexNet’s distinctive choices answer the constraints of 2012 hardware and data, as the authors present them: ReLUs train faster than saturating units, the split across two 3 GB GTX 580 GPUs makes the network fit in memory, and overlapping pooling and dropout reduce overfitting.

Architecture

The network is a plain feed-forward stack: no branches and no skip connections. Five convolutional layers turn a colour image into 256 feature maps of $6 \times 6$, which are flattened into a 9,216-dimensional vector and classified by a multilayer perceptron of two 4,096-unit hidden layers and a 1,000-way output. The kernel counts, sizes and first-layer stride below are from Krizhevsky et al. (2012, §3.5). The paper does not print the paddings of layers 2 to 5; the values shown are those of the Caffe reimplementation and of the author’s later cuda-convnet2 configuration, and they are the only symmetric paddings that keep the paper’s map sizes.

Layer	Kernels	Kernel, stride, padding	Groups	Output $(C, H, W)$	Followed by
conv1	96	$11 \times 11 \times 3$, 4, 0	1	$(96, 55, 55)$	ReLU, LRN, max pool $3 \times 3$/2 → $(96, 27, 27)$
conv2	256	$5 \times 5 \times 48$, 1, 2	2	$(256, 27, 27)$	ReLU, LRN, max pool → $(256, 13, 13)$
conv3	384	$3 \times 3 \times 256$, 1, 1	1	$(384, 13, 13)$	ReLU
conv4	384	$3 \times 3 \times 192$, 1, 1	2	$(384, 13, 13)$	ReLU
conv5	256	$3 \times 3 \times 192$, 1, 1	2	$(256, 13, 13)$	ReLU, max pool → $(256, 6, 6)$
fc6	4,096 units	—	—	4,096	ReLU, dropout 0.5
fc7	4,096 units	—	—	4,096	ReLU, dropout 0.5
fc8	1,000 units	—	—	1,000	softmax

Convolutional Layers

The stride-4 first layer reduces the image fourfold in each direction in one step, and the three pooling layers halve the maps again ($55 \to 27 \to 13 \to 6$) while the channels grow to 384; conv3 to conv5 are stacked $3 \times 3$ convolutions at $13 \times 13$ with no pooling or normalization between them. Every convolutional and hidden fully connected layer is followed by a ReLU, $\max(0, x)$ (see activation function). Local response normalization (LRN), which divides each activation by a power of a constant plus the scaled sum of squares of its neighbours across channels, follows the first two; the paper article works through it.

Overlapping Pooling

All three pooling layers take the maximum over a $3 \times 3$ window moved with stride 2, so neighbouring windows share a row or column. On the odd map sizes 55, 27 and 13 this gives $(n - 3)/2 + 1$ outputs, as many as non-overlapping $2 \times 2$ pooling would; Krizhevsky et al. compared the two at equal output size and reported slightly lower error with overlap (numbers in pooling).

The Input Size: 224 or 227

The paper gives the input as $224 \times 224 \times 3$, and its Figure 2 caption counts 150,528 input values, which is $224 \cdot 224 \cdot 3$. With the output-size formula $\lfloor (H + 2p - k)/s \rfloor + 1$ (see feature map), an unpadded $11 \times 11$ kernel at stride 4 then gives $\lfloor 213/4 \rfloor + 1 = 54$ positions, not the 55 the later layer sizes require. Reimplementations resolved this in two ways. Caffe’s bvlc_alexnet crops to $227 \times 227$, which gives exactly 55; the Stanford CS231n notes also use 227 and guess that three pixels of zero padding went unmentioned in the paper. TorchVision keeps 224 and pads the first layer by 2 on every side: $\lfloor 217/4 \rfloor + 1 = 55$.

The author’s own code suggests a third reading. The paper states that its network is specified precisely by the cuda-convnet code and parameter files. In cuda-convnet’s layer.py, the number of filter positions along each axis is $1 + \lceil (2p + H - k)/s \rceil$, rounded up rather than down, and the convolution kernel reads pixels outside the image as zeros. For $H = 224$, $k = 11$, $s = 4$, $p = 0$ this gives $1 + \lceil 53.25 \rceil = 55$: the last window starts at pixel 216 and overhangs the image by three pixels. cuda-convnet2, released in 2014, uses the same rule, and its ImageNet example trains a 1-GPU AlexNet with padding 0 on 224-pixel crops (--inner-size 224). Its two-GPU model-parallel configuration lays out the two-tower network itself, 48, 128, 192, 192 and 128 kernels per GPU with conv2, conv4 and conv5 reading only their own GPU’s maps, also with padding 0 in the first layer; it adds a second, within-map response normalization after conv2 that the paper does not describe. By our reading, the paper’s 224 and 55 are therefore consistent under its code’s convention, which amounts to three zero rows and columns on the bottom and right, close to the CS231n guess. The archived cuda-convnet source contains no ImageNet configuration, so this is an inference from the code, not a documented fact.

Two GPUs as Grouped Convolution

A GTX 580 has 3 GB of memory, too little for the network, so Krizhevsky et al. put half of each layer’s kernels on each of two GPUs and let the GPUs communicate only at certain layers. The kernels of conv2, conv4 and conv5 take input only from the maps on their own GPU; conv3 and the fully connected layers take input from both. This is why the paper prints conv2’s kernels as $5 \times 5 \times 48$ and conv4’s and conv5’s as $3 \times 3 \times 192$: each kernel sees half of the previous layer’s channels. The authors chose the connectivity pattern by cross-validation, to keep communication an acceptable fraction of computation.

In today’s terms, conv2, conv4 and conv5 are grouped convolutions with two groups: input and output channels are split into two halves, and each half of the outputs is computed from its half of the inputs only (the depthwise separable convolution article treats grouped convolution as a family). A grouped layer has $k^2 C_\text{in} C_\text{out} / g$ weights, so with $g = 2$ these three layers have half the weights of dense layers of the same width. The two halves, often called towers or columns, are not independent networks, since conv3 and fc6 mix them; Krizhevsky et al. compare the layout to the “columnar” CNN of Cireşan et al., whose columns were independent.

The restricted connectivity had a visible effect. The authors report (§6.1) that the 48 first-layer kernels on one GPU were largely colour-agnostic and those on the other largely colour-specific, in every run and independently of the random initialization, up to swapping the GPUs, and attribute this to the restricted connectivity. Our reading of the mechanism: conv1 itself is dense, but conv2 on each GPU reads only its own GPU’s conv1 maps, so each set of 48 conv1 kernels receives gradients only through its own tower and can specialize to what that tower uses.

Grouped convolution outlived the hardware reason. Xie et al. (2017) note that libraries such as Caffe and Torch supported grouped convolutions mainly for compatibility with AlexNet, with little evidence until then that they improved accuracy; ResNeXt supplied that evidence (see ResNet), and with one group per channel a grouped convolution becomes the depthwise half of a depthwise separable convolution.

Where the Parameters Are

The paper rounds the total to 60 million parameters. The count below, for the two-tower network as in the table above, gives 60,965,224, of which 96.2% are in the three fully connected layers and 61.9% in fc6 alone, the $9{,}216 \times 4{,}096$ matrix that connects the last pooled maps to the first hidden layer. The convolutional layers hold 3.8% of the parameters but about 92% of the multiply-adds, because each of their weights is applied at every position of a map. Krizhevsky (2014) gives the same split for convolutional networks in general, about 5% of the parameters and 90–95% of the computation in the convolutional layers, and uses it to parallelize the two kinds of layer differently.

The paper states that each convolutional layer holds no more than 1% of the model’s parameters. By our count conv3 holds 1.45% and conv4 1.09%; the remaining three are below 1%.

The script below (run with PyTorch 2.14.1) builds the full-size network with the two-GPU split written as groups=2 and passes a synthetic $227 \times 227$ image through it, diagonal red stripes of period 8 pixels and a green disk. It prints each layer’s output shape and parameter count, compares an ungrouped network of the same widths and two single-tower variants (see Variants), shows the first layer’s output width for the input sizes discussed above, and replaces the 48 conv1 maps of one tower with random values to check that only that tower’s 128 conv2 outputs change.

import torch
import torch.nn as nn
import torch.nn.functional as F

torch.manual_seed(0)
torch.set_num_threads(1)

def alexnet(widths=(96, 256, 384, 384, 256), groups=(1, 2, 1, 2, 2), classes=1000):
    """AlexNet-shaped network: 5 conv + 3 FC layers. The two-GPU layout is written as
    grouped convolutions (groups=2 in layers 2, 4, 5); LRN as in the 2012 paper
    (PyTorch divides alpha by size, so alpha = 5 * 1e-4)."""
    c1, c2, c3, c4, c5 = widths
    g = groups
    lrn = lambda: nn.LocalResponseNorm(5, alpha=5e-4, beta=0.75, k=2.0)
    pool = lambda: nn.MaxPool2d(3, stride=2)              # overlapping pooling
    return nn.Sequential(
        nn.Conv2d(3, c1, 11, stride=4, groups=g[0]), nn.ReLU(), lrn(), pool(),
        nn.Conv2d(c1, c2, 5, padding=2, groups=g[1]), nn.ReLU(), lrn(), pool(),
        nn.Conv2d(c2, c3, 3, padding=1, groups=g[2]), nn.ReLU(),
        nn.Conv2d(c3, c4, 3, padding=1, groups=g[3]), nn.ReLU(),
        nn.Conv2d(c4, c5, 3, padding=1, groups=g[4]), nn.ReLU(), pool(),
        nn.Flatten(),
        nn.Linear(c5 * 6 * 6, 4096), nn.ReLU(), nn.Dropout(0.5),
        nn.Linear(4096, 4096), nn.ReLU(), nn.Dropout(0.5),
        nn.Linear(4096, classes))

# Synthetic 227x227 RGB image: diagonal stripes (period 8 px) in red, a green disk.
y, x = torch.meshgrid(torch.arange(227.0), torch.arange(227.0), indexing="ij")
img = torch.zeros(1, 3, 227, 227)
img[0, 0] = 0.5 + 0.5 * torch.cos(2 * torch.pi * (x + y) / 8)
img[0, 1] = (((x - 150) ** 2 + (y - 80) ** 2) < 40 ** 2).float()

def macs_of(net, x):
    """Multiply-adds of the conv and linear layers for input x (biases not counted)."""
    total = 0
    for m in net:
        x = m(x)
        if isinstance(m, (nn.Conv2d, nn.Linear)):
            total += m.weight.numel() * (x.shape[-1] * x.shape[-2] if x.dim() == 4 else 1)
    return total

net = alexnet().eval()
print("Output shape after each layer with weights or pooling (two-tower AlexNet, 227x227 input):")
h, conv_macs, total = img, 0, 0
rows = []
for m in net:
    h = m(h)
    if isinstance(m, (nn.Conv2d, nn.Linear)):
        n = sum(p.numel() for p in m.parameters())
        total += n
        if isinstance(m, nn.Conv2d):
            conv_macs += m.weight.numel() * h.shape[-1] * h.shape[-2]
        name = f"conv{len(rows)+1}" if isinstance(m, nn.Conv2d) else f"fc{len(rows)+1}"
        rows.append((name, n))
        print(f"  {name:6s} {str(tuple(h.shape)):22s} params {n:>11,}")
    elif isinstance(m, nn.MaxPool2d):
        print(f"  {'pool':6s} {str(tuple(h.shape)):22s}")
conv = sum(n for name, n in rows if name.startswith("conv"))
macs = macs_of(net, img)
print(f"total {total:,} parameters; conv {conv:,} ({100*conv/total:.1f}%), "
      f"FC {total-conv:,} ({100*(total-conv)/total:.1f}%), fc6 alone {100*rows[5][1]/total:.1f}%")
print(f"largest conv layer share: {100*max(n for _, n in rows[:5])/total:.2f}%")
print(f"multiply-adds per image: {macs/1e6:.0f} million, {100*conv_macs/macs:.0f}% in conv layers")

ungrouped = alexnet(groups=(1, 1, 1, 1, 1))
nu = sum(p.numel() for p in ungrouped.parameters())
print(f"\nsame widths, no groups: {nu:,} parameters (+{nu-total:,} in conv2, conv4, conv5)")
for widths in [(64, 192, 384, 256, 256), (64, 192, 384, 384, 256)]:
    one = alexnet(widths, groups=(1, 1, 1, 1, 1)).eval()
    no = sum(p.numel() for p in one.parameters())
    print(f"single tower {widths}: {no:,} parameters ({100*(no/total-1):+.2f}% vs two-tower), "
          f"{macs_of(one, img)/1e6:.0f} million multiply-adds")

print("\nconv1 output width for the 224 vs 227 question:")
c1 = nn.Conv2d(3, 96, 11, stride=4)
for label, inp in [("224, no padding", torch.zeros(1, 3, 224, 224)),
                   ("227, no padding", torch.zeros(1, 3, 227, 227)),
                   ("224, padding 2 each side", F.pad(torch.zeros(1, 3, 224, 224), (2, 2, 2, 2))),
                   ("224, 3 extra on right/bottom", F.pad(torch.zeros(1, 3, 224, 224), (0, 3, 0, 3)))]:
    print(f"  {label:30s} -> {c1(inp).shape[-1]}")

# The two towers are independent in conv2: change only the maps of tower 1 (conv1
# channels 0-47) and only conv2 outputs 0-127 can change.
with torch.no_grad():
    a = net[:4](img)                      # conv1 -> ReLU -> LRN -> pool, (1, 96, 27, 27)
    b = a.clone(); b[:, :48] = torch.rand_like(b[:, :48])
    conv2 = net[4]
    d = (conv2(a) - conv2(b)).abs().amax(dim=(0, 2, 3))
print(f"\nconv2 weight shape {tuple(conv2.weight.shape)}; after changing tower-1 inputs, "
      f"max |change| in outputs 0-127: {d[:128].max():.3f}, in outputs 128-255: {d[128:].max():.3f}")

Output shape after each layer with weights or pooling (two-tower AlexNet, 227x227 input):
  conv1  (1, 96, 55, 55)        params      34,944
  pool   (1, 96, 27, 27)       
  conv2  (1, 256, 27, 27)       params     307,456
  pool   (1, 256, 13, 13)      
  conv3  (1, 384, 13, 13)       params     885,120
  conv4  (1, 384, 13, 13)       params     663,936
  conv5  (1, 256, 13, 13)       params     442,624
  pool   (1, 256, 6, 6)        
  fc6    (1, 4096)              params  37,752,832
  fc7    (1, 4096)              params  16,781,312
  fc8    (1, 1000)              params   4,097,000
total 60,965,224 parameters; conv 2,334,080 (3.8%), FC 58,631,144 (96.2%), fc6 alone 61.9%
largest conv layer share: 1.45%
multiply-adds per image: 724 million, 92% in conv layers

same widths, no groups: 62,378,344 parameters (+1,413,120 in conv2, conv4, conv5)
single tower (64, 192, 384, 256, 256): 61,100,840 parameters (+0.22% vs two-tower), 714 million multiply-adds
single tower (64, 192, 384, 384, 256): 61,838,248 parameters (+1.43% vs two-tower), 839 million multiply-adds

conv1 output width for the 224 vs 227 question:
  224, no padding                -> 54
  227, no padding                -> 55
  224, padding 2 each side       -> 55
  224, 3 extra on right/bottom   -> 55

conv2 weight shape (256, 48, 5, 5); after changing tower-1 inputs, max |change| in outputs 0-127: 1.139, in outputs 128-255: 0.000

Removing the groups would add 1,413,120 weights, about 60% more convolutional parameters, yet only 2.3% more in total. conv2’s weight tensor has shape $(256, 48, 5, 5)$, the paper’s $5 \times 5 \times 48$ kernels, and a change confined to one tower’s inputs leaves the other tower’s outputs exactly unchanged. The paper itself relies on fc6’s dominance in a control experiment: its one-GPU comparison network kept the full-size last convolutional layer, because most of the parameters are in the fully connected layer that reads it (footnote 2).

Training

The network outputs logits $\mathbf{z} \in \mathbb{R}^{1000}$ and is trained to minimize the average cross-entropy of the softmax over a batch $\mathcal{B}$,

$$\mathcal{L} = -\frac{1}{|\mathcal{B}|} \sum_{(x, y) \in \mathcal{B}} \log \frac{e^{z_y(x)}}{\sum_{j=1}^{1000} e^{z_j(x)}},$$

which the paper calls maximizing the multinomial logistic regression objective. Dropout with probability 0.5 acts on the outputs of fc6 and fc7; the data, augmentation, optimizer and schedule are in the paper article.

Two details matter when the architecture is reimplemented. The weights were drawn from a zero-mean Gaussian with standard deviation 0.01, and the biases of conv2, conv4, conv5 and the hidden fully connected layers were set to 1 to give the ReLUs positive inputs early in training (§5). Caffe’s replication instead set these biases to 0.1, which its readme says was necessary because initialization to 1 gave a flat loss. Second, the network has no batch normalization; TorchVision’s reference recipe trains it with an initial learning rate of 0.01 instead of its default of 0.1, which it calls too high for networks without batch normalization.

Inference

The input is a mean-subtracted $224 \times 224$ crop (or $227 \times 227$ in Caffe), and the output is a vector of 1,000 class scores turned into probabilities by a softmax. The paper averaged the softmax outputs of ten crops, the four corners and the centre of the $256 \times 256$ image and their mirror images, while single-crop evaluation is common in reimplementations. The forward pass costs about 724 million multiply-adds for the two-tower network and 714 million for the single-tower one, by the count above; TorchVision’s metadata for its single-tower weights lists 0.714 (its _ops field), which matches the latter.

Because fc6 expects exactly $256 \times 6 \times 6$ inputs, the original network is built for a single input size. The fully connected layers can be rewritten as convolutions whose kernels cover their whole input, which makes the network accept larger images and produce a map of class scores, as in fully convolutional networks. TorchVision instead inserts an adaptive average pooling to $6 \times 6$ before the classifier, a no-op at 224 pixels.

Performance

The paper article gives the results in full; in brief, as printed by Krizhevsky et al. (2012):

• ILSVRC-2010 test set: 37.5% top-1 and 17.0% top-5 error, averaging ten crops; 39.0% and 18.3% without the averaging (footnote 5).
• ILSVRC-2012: 40.7% top-1 and 18.2% top-5 validation error for one network; 15.3% top-5 test error for the winning ensemble of seven networks.

Reimplementations land near the single-network numbers. Zeiler and Fergus (2014) reported 40.5% top-1 and 18.1% top-5 validation error for their replication. Caffe’s bvlc_alexnet readme reports 57.1% top-1 and 80.2% top-5 validation accuracy from the centre crop alone, and TorchVision’s single-tower weights list 56.522% and 79.066% on ImageNet-1K (as of October 2026, metadata of AlexNet_Weights.IMAGENET1K_V1), from a centre $224 \times 224$ crop of an image resized to 256. All results are on ImageNet data; see accuracy for top-$k$ metrics.

Strengths

• Simple to reimplement. A sequential stack of eight weight layers, fully determined once the paddings are fixed.
• Cheap convolutional part. The convolutional layers hold only about 2.3 million parameters. Zhang et al. (2018) list AlexNet’s convolutional layers at 9.1 MB against 58.9 MB for the convolutional layers of VGG, and the LPIPS repository calls the AlexNet variant the fastest of its three networks.

Limitations

• The head is most of the model. 96% of the parameters are in the fully connected layers, the two hidden ones being where the paper applies dropout against overfitting. Zeiler and Fergus’s ablation (their Table 2) shows that the head is not where the accuracy comes from in proportion: removing fc7 changed their replication’s top-5 validation error from 18.1% to 18.4%, and removing both fc6 and fc7 raised it to 22.4%, while removing conv3 and conv4 raised it to 22.1% with far fewer parameters lost. Later classifiers replaced the head with global average pooling and one linear layer (see convolutional neural network).
• Aggressive first layer. Zeiler and Fergus found that the $11 \times 11$, stride-4 first layer learned a mix of very high and very low frequencies with little in between, and that the large stride caused aliasing artifacts in the second layer’s features (see sampling and aliasing).
• Components that did not last. Simonyan and Zisserman found that LRN did not improve their VGG networks (see the paper article); the two-GPU grouping was a memory workaround, and Zeiler and Fergus replaced it with dense connections.

Historical Importance

AlexNet set the template the next ImageNet classifiers refined: convolutions with ReLUs and max pooling that shrink the image and widen the channels, then a large fully connected head with dropout. VGG kept the head but replaced the large early kernels with stacks of $3 \times 3$ convolutions, and GoogLeNet replaced the head with average pooling and one linear layer (see convolutional neural network).

Its hidden-layer activations also became a common source of learned features (see transfer learning). Donahue et al. (2014) adopted the AlexNet architecture, trained on ImageNet, and used the activations of its hidden layers, which they called DeCAF, as generic features for other recognition tasks. R-CNN (Girshick et al., 2014) passed each object proposal, warped to $227 \times 227$, through a Caffe reimplementation of the network to obtain a 4,096-dimensional feature for detection.

It remains a reference point. SqueezeNet (Iandola et al., 2016) was designed to reach AlexNet-level ImageNet accuracy with 50 times fewer parameters, taking Caffe’s bvlc_alexnet as its baseline. In perceptual metrics, Zhang et al. (2018) used the five convolutional layers of the single-tower AlexNet, alongside SqueezeNet and VGG, to compare images by their deep features; with features from networks trained on classification, AlexNet’s scored 68.9% on their two-alternative forced-choice (2AFC) test of human similarity judgments, against 73.9% for human consistency and 63.2% for $\ell_2$ distance, averaged over their six test sets. The LPIPS metric built on that study, defined in the neural rendering article, uses AlexNet as its default network (as of October 2026, in the authors’ repository).

Variants

• Two-tower AlexNet (2012). The original network described above, with grouped conv2, conv4 and conv5. The paper’s ILSVRC-2012 entry also used a version with an extra sixth convolutional layer, pretrained on the full ImageNet release.
• Single-tower AlexNet (2014). For his study of parallel training, Krizhevsky (2014) used a minor variation of the 2012 model with one tower instead of two: no grouping, the same number of layers and the same map sizes, and 1,000 independent logistic output units instead of a softmax. He gives its filter counts as 64, 192, 384, 384 and 256 (footnote 1) and states that the model has 0.2% more parameters than the two-tower model. The released cuda-convnet2 configuration for the 1-GPU ImageNet network, and TorchVision, use 256 filters in the fourth layer instead of 384, and the script above shows that only that version matches the 0.2% (+0.22%, against +1.43% with 384), so the footnote’s fourth number appears to be a misprint (our inference). The note’s other figure, 2.4% fewer connections (undefined), also fits 256 better: that version has 1.4% fewer multiply-adds by our count, the 384 version more. With a batch of 128 on one GPU, the single-tower model reached 42.33% top-1 error on the ILSVRC-2012 validation set from the centre crop (his Table 1).
• ZFNet (2014). Zeiler and Fergus (2014) changed AlexNet’s first two layers after inspecting them with their deconvnet visualizations: a $7 \times 7$ first layer at stride 2 instead of $11 \times 11$ at stride 4, and stride 2 in the second layer. Their network also uses dense connections instead of the two-GPU grouping, and during training they renormalized any convolutional filter whose RMS value exceeded 0.1. Their text says the sparse connections replaced were in layers 3, 4 and 5; in AlexNet they are in layers 2, 4 and 5. Trained on one GTX 580, the single model reached 38.4% top-1 and 16.5% top-5 validation error against 40.7% and 18.2% for one AlexNet (their Table 1). The text calls this an improvement of 1.7% in test top-5 error, but the table lists only validation errors for single models. An ensemble of six models, five of that network and one with wider layers 3 to 5 (512, 1,024 and 512 maps), reached 14.8% top-5 test error.

Implementations

• TorchVision (torchvision.models.alexnet, version 0.29.0): its documentation says the model follows the “One weird trick” paper rather than the 2012 one. The layers are 64, 192, 384, 256 and 256 filters, with no grouping, padding 2 in the first layer and no LRN. Dropout is applied to the inputs of fc6 and fc7, the flattened pooled maps and fc6’s output, rather than to the outputs of fc6 and fc7 as in the paper and in cuda-convnet2’s configuration. The model has 61,100,840 parameters according to its weight metadata, matching the single-tower count above.
• Caffe (bvlc_alexnet): a replication of the two-tower network with group: 2 in conv2, conv4 and conv5 and $227 \times 227$ crops, trained without the colour augmentation and with biases of 0.1.
• cuda-convnet and cuda-convnet2: the author’s CUDA code. The paper points to cuda-convnet for the network’s precise specification. cuda-convnet2 includes ImageNet configurations of the single-tower model for one GPU and for data-parallel training on two or four GPUs, and a two-GPU model-parallel configuration with the two-tower layout.