HRNet

HRNet, the High-Resolution Network (also written High-Resolution Net), is a convolutional neural network that keeps a high-resolution representation from the start of the network to its output instead of shrinking the image to a coarse map and enlarging it again. Ke Sun, Bin Xiao, Dong Liu and Jingdong Wang of the University of Science and Technology of China and Microsoft Research Asia introduced it at CVPR 2019 for single-person human pose estimation, where it predicts one heatmap per body joint (Sun et al., 2019). Its two ideas are to run branches of different resolutions in parallel, adding a lower-resolution branch at each stage while keeping the higher ones, and to fuse the branches repeatedly, so that every branch receives information from all the others many times. A journal version renamed the pose network HRNetV1 and added HRNetV2, which concatenates all branches for semantic segmentation, and HRNetV2p, a feature pyramid built from it for object detection, presenting HRNet as a general backbone (Wang et al., 2021). HigherHRNet adapted it to bottom-up multi-person pose estimation with heatmaps at twice the resolution (Cheng et al., 2020). Later members of the family include Lite-HRNet, a lightweight version, and HRFormer, which keeps the multi-resolution parallel design with transformer blocks.

Problem Addressed

Heatmap-based pose networks need an output map fine enough to place a joint within a few pixels, and features at that resolution that also know what the joint is, which needs a large receptive field. Classification networks such as ResNet obtain the large receptive field by halving the resolution stage after stage, so their last features are rich but coarse. Sun et al. (2019) group the pose networks of 2016–2018 by how they then recover resolution: the stacked hourglass network mirrors its downsampling path with an upsampling path and skip connections, an encoder–decoder design; the cascaded pyramid network and SimpleBaseline attach a light upsampling process (bilinear upsampling, or a few transposed convolutions) to a heavy ResNet; other networks replace the last strides with dilated convolutions. SimpleBaseline, for example, ends a ResNet with three transposed convolutions and predicts heatmaps from the result (Xiao, Wu and Wei, 2018). In all of these the high-resolution features are reconstructed from low-resolution ones, connected in series.

HRNet’s authors argued that this reconstruction is the weak point: features recovered from a coarse map are less spatially precise, and the usual fusions combine low-level fine features with high-level coarse ones (Sun et al., 2019). Their alternative maintains high resolution throughout and fuses representations of “the same depth and similar level” across resolutions, which, they wrote, makes the predicted heatmap potentially more accurate and more precise. The paper places the design among earlier multi-resolution networks for classification and segmentation, convolutional neural fabrics, interlinked CNNs, GridNet and multi-scale DenseNets, which it judged not applicable to its problem because of their subnetwork design, their fusion schemes, or, in GridNet, because each of two fusion stages sent information in only one direction between resolutions; it also cites deep fusion networks as a partial inspiration for repeating the fusion.

Architecture

The network has a stem, a main body that outputs features at the stem’s resolution, and a head. The stem is two stride-2 $3 \times 3$ convolutions, which bring the input to 1/4 resolution; everything after that keeps a 1/4-resolution branch (Sun et al., 2019; Wang et al., 2021). The main body is the HRNet proper; HRNetV1, HRNetV2 and HRNetV2p share it and differ only in the head.

Parallel Multi-Resolution Branches

Write $\mathcal{N}_{sr}$ for the subnetwork in stage $s$ at resolution index $r$, whose resolution is $1/2^{r-1}$ of the first branch’s. A classification-style network with four stages connects them in series, one resolution per stage:

$$\mathcal{N}_{11} \rightarrow \mathcal{N}_{22} \rightarrow \mathcal{N}_{33} \rightarrow \mathcal{N}_{44}.$$

HRNet starts with a single high-resolution branch and, at each new stage, keeps every existing branch and adds one at half the lowest resolution so far:

$$\begin{array}{llll} \mathcal{N}_{11} \rightarrow & \mathcal{N}_{21} \rightarrow & \mathcal{N}_{31} \rightarrow & \mathcal{N}_{41} \\ & \searrow \mathcal{N}_{22} \rightarrow & \mathcal{N}_{32} \rightarrow & \mathcal{N}_{42} \\ & & \searrow \mathcal{N}_{33} \rightarrow & \mathcal{N}_{43} \\ & & & \searrow \mathcal{N}_{44} \end{array}$$

In the four-stage networks of both papers the branches run at 1/4, 1/8, 1/16 and 1/32 of the input, with $C$, $2C$, $4C$ and $8C$ channels, so the width doubles each time the resolution halves (Wang et al., 2021). In the official pose code, the new branch at each stage transition is computed from the previous stage’s lowest-resolution branch by a stride-2 $3 \times 3$ convolution with batch normalization and ReLU (pose code, pose_hrnet.py, _make_transition_layer).

Repeated Multi-Scale Fusion

Each stage is divided into blocks. In a block, every branch first applies its own convolutions (four residual units of two $3 \times 3$ convolutions each), and then an exchange unit, which the journal version calls a fusion module, mixes the branches. Its inputs are $s$ maps $X_1, \dots, X_s$ and its outputs $s$ maps of the same resolutions and widths:

$$Y_k = \sum_{i=1}^{s} a(X_i, k),$$

where $a(X_i, k)$ resamples $X_i$ from resolution $i$ to resolution $k$ (Sun et al., 2019). If $i = k$ it is the identity. Downsampling uses $(k - i)$ stride-2 $3 \times 3$ convolutions in sequence, one for a factor of two and two for a factor of four. Upsampling uses a $1 \times 1$ convolution to match the number of channels, followed by an upsampling of the map; the CVPR paper specifies nearest-neighbour upsampling, while the journal version’s text and figure specify bilinear upsampling (Wang et al., 2021). The official code follows each version: the pose code upsamples with nn.Upsample(mode='nearest'), and the segmentation code with bilinear F.interpolate (pose code, pose_hrnet.py; segmentation code, seg_hrnet.py). In both codes the transformed maps are summed and passed through a ReLU, and each strided convolution in a downsampling chain except the last is followed by a ReLU. An exchange unit between stages has one extra output, $Y_{s+1} = a(Y_s, s+1)$, which starts the next branch.

The point of the repetition is that context from the coarse branches reaches the fine branch many times during the network, and fine detail reaches the coarse branches, instead of once at the end. The journal version gives an interpretation: the parallel branches resemble a group convolution whose groups run at different resolutions, and the fusion resembles the fully connected multi-branch form of an ordinary convolution, in which every output group sums convolutions of every input group; the authors take this as an argument for using all four resolutions in the output rather than the highest only (Wang et al., 2021).

Instantiation and Widths

The first stage is four ResNet-50 bottleneck units of width 64 at 1/4 resolution, followed by a $3 \times 3$ convolution that changes the width to $C$. Stages 2, 3 and 4 contain 1, 4 and 3 blocks, so the network performs eight fusions in total (Sun et al., 2019). Networks are named by $C$, the width of the high-resolution branch: HRNet-W32 has branch widths 32, 64, 128 and 256, and HRNet-W48 has 48, 96, 192 and 384. The released COCO configuration for HRNet-W32 sets these numbers explicitly (one module in stage 2, four in stage 3, three in stage 4, four basic blocks per branch, widths 32 to 256) and fuses by summation (pose code, w32_256x192_adam_lr1e-3.yaml). The pose network is single-scale in its output: in the official code the last module of stage 4 returns only the high-resolution branch (multi_scale_output=False).

Representation Heads

The journal version names three heads on the same body (Wang et al., 2021):

• HRNetV1 reads only the high-resolution branch, as in the CVPR paper. For pose, a $1 \times 1$ convolution maps it to one heatmap per keypoint at 1/4 of the input resolution.
• HRNetV2 upsamples the three lower-resolution outputs bilinearly to 1/4 resolution without changing their channels, concatenates all four ($C + 2C + 4C + 8C = 15C$ channels) and mixes them with a $1 \times 1$ convolution. For segmentation, a linear classifier with softmax loss labels every position, and the result is upsampled four times bilinearly to the input size. In the official segmentation code the head is a $1 \times 1$ convolution keeping the $15C$ channels, batch normalization, ReLU and a final convolution to the class scores.
• HRNetV2p downsamples the HRNetV2 output to several levels, five as in the feature pyramid network, and plugs the resulting pyramid into detectors.

For pose estimation the authors observed that HRNetV1 and HRNetV2 performed almost the same and kept HRNetV1 because it costs slightly less. For ImageNet pretraining, an appendix of the journal version describes a classification head: each branch passes through a bottleneck that widens it to 128, 256, 512 or 1024 channels, the high-resolution output is repeatedly downsampled by a stride-2 convolution and added to the next branch, and the 1024 channels at the lowest resolution are widened to 2048 by a $1 \times 1$ convolution and average-pooled for the classifier (Wang et al., 2021, Appendix A).

HigherHRNet

Bottom-up methods detect all joints in the image at once and group them into people, so a person who occupies few pixels gets few heatmap cells. Earlier bottom-up methods predicted heatmaps at 1/4 of the input; PersonLab enlarged the input instead (Cheng et al., 2020). HigherHRNet keeps HRNet as its backbone and adds a deconvolution module on top of the 1/4-resolution output:

• The module’s input is the concatenation of the HRNet features and the heatmaps predicted from them. A $4 \times 4$ transposed convolution with batch normalization and ReLU doubles the resolution, four basic residual blocks refine the result, and a $1 \times 1$ convolution predicts heatmaps at 1/2 resolution. The two resolutions form what the authors call a high-resolution feature pyramid, which starts at 1/4 and goes up, unlike the feature pyramid network, which starts at 1/32 and comes up to 1/4. More modules can be stacked; one worked best on COCO.
• Grouping uses associative embedding: each keypoint gets a scalar tag, and keypoints whose tags are close are assigned to the same person (Newell, Huang and Deng, 2017). Tags are predicted only at 1/4 resolution, because at higher resolution the tag maps learned poorly and did not even converge.

The released configuration for HRNet-W32 with $512 \times 512$ inputs predicts heatmaps of $128 \times 128$ and $256 \times 256$ for 17 joints, with one tag map per joint (HigherHRNet code, w32_512_adam_lr1e-3.yaml).

Example: Keeping Resolution Against Recovering It

The example asks whether, in a small network trained from scratch, keeping a high-resolution branch and fusing it repeatedly with a low-resolution one localizes a keypoint more precisely than recovering resolution by upsampling. Each $64 \times 64$ synthetic image contains a thin ring, one small blob inside it (the keypoint, at a sub-pixel position) and three identical blobs outside it, plus noise. Which blob is the keypoint depends only on the ring, which is 24–32 pixels across, so the network needs a large receptive field; where it is needs pixel-level detail. Four networks of 0.34–0.38M parameters predict one heatmap at stride 2 ($32 \times 32$, Gaussian targets with $\sigma = 1$ cell at the unrounded position, mean squared error, decoded by argmax with HRNet’s quarter-cell shift):

• A1 follows SimpleBaseline’s pattern: strided convolutions down to $8 \times 8$, four residual blocks, two transposed convolutions back up, no skips.
• A2 adds encoder–decoder skips from the $16 \times 16$ and $32 \times 32$ encoder features, in the manner of the stacked hourglass network.
• B is HRNet-like: a $32 \times 32$ branch of 16 channels kept throughout and an $8 \times 8$ branch of 64 channels, with an exchange unit after each of four stages (nearest upsampling after a $1 \times 1$ convolution, strided $3 \times 3$ convolutions down, sum and ReLU, as in the official pose code).
• C is B with only the final exchange unit, the paper’s Table 6 variant (a).

How to upsample is not fixed by the sources (SimpleBaseline uses transposed convolutions without skips; the hourglass network uses skips), so both A1 and A2 are run. Two label-free baselines first check that the task is neither impossible nor trivial: predicting the image centre, and picking one of the four blobs at random with perfect precision. Each network is trained with Adam for 600 steps of 32 images on the same 4,000 training images, for seeds 0, 1 and 2, and tested on 1,000 images; the code prints the mean over seeds and the range. A failure is an error above 4 pixels (the wrong blob, or no peak near the keypoint); the hit error is the mean error of the rest. The example uses PyTorch 2.14.1 and NumPy 2.5.4 on the CPU with four threads and deterministic algorithms. It is slow: the run shown took about 20 minutes of wall-clock time, and a repeat with identical output about 12 minutes, on a shared ten-core machine. The plan, written before the first run, expected “under about a minute”; the script was not changed after its first run.

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F

torch.set_num_threads(4)
torch.use_deterministic_algorithms(True)
S = 64                      # image size; heatmaps are 32 x 32 (stride 2)

def make_set(n, rng):
    """Ring + one blob inside it (the keypoint) + three blobs outside, sub-pixel positions."""
    yy, xx = np.mgrid[0:S, 0:S].astype(np.float32)
    imgs, kps, blobs = np.zeros((n, S, S), np.float32), np.zeros((n, 2)), []
    for i in range(n):
        c, R = rng.uniform(22, 42, 2), rng.uniform(12, 16)
        d = np.hypot(xx - c[0], yy - c[1])
        img = 0.6 * np.exp(-0.5 * ((d - R) / 0.75) ** 2)            # thin ring
        while True:                                                   # blob inside the ring
            p = c + rng.uniform(-(R - 5), R - 5, 2)
            if np.hypot(*(p - c)) < R - 5:
                break
        pts = [p]
        while len(pts) < 4:                                           # distractors outside
            q = rng.uniform(3, 61, 2)
            if np.hypot(*(q - c)) > R + 5 and min(np.hypot(*(q - r)) for r in pts) > 6:
                pts.append(q)
        for q in pts:
            img += np.exp(-((xx - q[0]) ** 2 + (yy - q[1]) ** 2) / (2 * 1.2 ** 2))
        imgs[i] = img + rng.normal(0, 0.1, (S, S))
        kps[i] = p
        blobs.append(np.array(pts))
    return imgs, kps, blobs

def targets(kps, sigma=1.0):
    g = np.arange(S // 2, dtype=np.float32)
    k = kps / 2.0                                                     # unrounded, in cells
    return np.exp(-((g[None, None, :] - k[:, None, None, 0]) ** 2 +
                    (g[None, :, None] - k[:, None, None, 1]) ** 2) / (2 * sigma ** 2))

def cbr(i, o, k=3, s=1):
    return nn.Sequential(nn.Conv2d(i, o, k, s, k // 2, bias=False), nn.BatchNorm2d(o), nn.ReLU())

class Block(nn.Module):                                               # residual block, two 3x3 convs
    def __init__(self, c):
        super().__init__()
        self.f = nn.Sequential(cbr(c, c), nn.Conv2d(c, c, 3, 1, 1, bias=False), nn.BatchNorm2d(c))
    def forward(self, x):
        return F.relu(x + self.f(x))

class Upsample(nn.Module):                                            # A1 (skips=False), A2 (skips=True)
    def __init__(self, skips):
        super().__init__()
        self.skips = skips
        self.stem, self.d1, self.d2 = cbr(1, 16, 3, 2), cbr(16, 32, 3, 2), cbr(32, 64, 3, 2)
        self.body = nn.Sequential(*[Block(64) for _ in range(4)])
        self.u1 = nn.Sequential(nn.ConvTranspose2d(64, 32, 4, 2, 1, bias=False), nn.BatchNorm2d(32), nn.ReLU())
        self.u2 = nn.Sequential(nn.ConvTranspose2d(32, 16, 4, 2, 1, bias=False), nn.BatchNorm2d(16), nn.ReLU())
        self.head = nn.Conv2d(16, 1, 1)
    def forward(self, x):
        h32 = self.stem(x); h16 = self.d1(h32)
        y = self.u1(self.body(self.d2(h16)))
        y = self.u2(y + h16 if self.skips else y)
        return self.head(y + h32 if self.skips else y)[:, 0]

class Parallel(nn.Module):                                            # B (repeated=True), C (False)
    def __init__(self, repeated, stages=4):
        super().__init__()
        self.repeated = repeated
        self.stem, self.new_low = cbr(1, 16, 3, 2), nn.Sequential(cbr(16, 16, 3, 2), cbr(16, 64, 3, 2))
        self.hi = nn.ModuleList([Block(16) for _ in range(stages)])
        self.lo = nn.ModuleList([Block(64) for _ in range(stages)])
        n_fuse = stages if repeated else 1
        self.up = nn.ModuleList([nn.Sequential(nn.Conv2d(64, 16, 1, bias=False), nn.BatchNorm2d(16))
                                 for _ in range(n_fuse)])
        self.down = nn.ModuleList([nn.Sequential(cbr(16, 16, 3, 2), nn.Conv2d(16, 64, 3, 2, 1, bias=False),
                                                 nn.BatchNorm2d(64)) for _ in range(n_fuse)])
        self.head = nn.Conv2d(16, 1, 1)
    def fuse(self, j, h, l):                                          # exchange unit: sum, then ReLU
        h2 = h + F.interpolate(self.up[j](l), scale_factor=4, mode="nearest")
        return F.relu(h2), F.relu(l + self.down[j](h))
    def forward(self, x):
        h = self.stem(x); l = self.new_low(h)
        for s in range(len(self.hi)):
            h, l = self.hi[s](h), self.lo[s](l)
            if self.repeated or s == len(self.hi) - 1:
                h, l = self.fuse(s if self.repeated else 0, h, l)
        return self.head(h)[:, 0]

def decode(hm):
    """argmax + quarter-cell shift toward the higher neighbour on each axis, back to input pixels."""
    n, H, W = hm.shape
    idx = hm.reshape(n, -1).argmax(1)
    py, px = idx // W, idx % W
    x, y = px.astype(float), py.astype(float)
    for i in range(n):
        if 0 < px[i] < W - 1:
            x[i] += 0.25 * np.sign(hm[i, py[i], px[i] + 1] - hm[i, py[i], px[i] - 1])
        if 0 < py[i] < H - 1:
            y[i] += 0.25 * np.sign(hm[i, py[i] + 1, px[i]] - hm[i, py[i] - 1, px[i]])
    return np.stack([x, y], 1) * 2.0

def summary(err):
    hit = err <= 4.0
    return 100 * (~hit).mean(), err[hit].mean(), err.mean()

Xtr, Ktr, _ = make_set(4000, np.random.default_rng(1000))
Xte, Kte, Bte = make_set(1000, np.random.default_rng(2000))
Ttr = targets(Ktr)

rng = np.random.default_rng(0)                                        # label-free baselines
centre = np.hypot(*(Kte - S / 2).T)
pick = np.array([b[rng.integers(4)] for b in Bte])
print("baseline       fail%  hit err  mean err (px)")
print("image centre   %5.1f  %7.2f  %8.2f" % summary(centre))
print("random blob    %5.1f  %7.2f  %8.2f" % summary(np.hypot(*(pick - Kte).T)))

models = {"A1 upsample": lambda: Upsample(False), "A2 upsample+skips": lambda: Upsample(True),
          "B parallel, fused x4": lambda: Parallel(True), "C parallel, fused x1": lambda: Parallel(False)}
xtr, ttr = torch.from_numpy(Xtr[:, None]), torch.from_numpy(Ttr)
xte = torch.from_numpy(Xte[:, None])
print("\nmodel                  params  fail% (range)     hit err px (range)   mean err px")
for name, build in models.items():
    res = []
    for seed in (0, 1, 2):
        torch.manual_seed(seed)
        g = torch.Generator().manual_seed(seed)
        net = build()
        opt = torch.optim.Adam(net.parameters(), lr=1e-3)
        net.train()
        for step in range(600):
            b = torch.randint(0, len(xtr), (32,), generator=g)
            loss = F.mse_loss(net(xtr[b]), ttr[b])
            opt.zero_grad(); loss.backward(); opt.step()
        net.eval()
        with torch.no_grad():
            hm = torch.cat([net(xte[i:i + 250]) for i in range(0, len(xte), 250)]).numpy()
        res.append(summary(np.hypot(*(decode(hm) - Kte).T)))
    r = np.array(res)
    n_par = sum(p.numel() for p in net.parameters())
    print("%-21s %7d  %5.1f (%4.1f-%4.1f)  %5.2f (%4.2f-%4.2f)  %6.2f" % (
        name, n_par, r[:, 0].mean(), r[:, 0].min(), r[:, 0].max(),
        r[:, 1].mean(), r[:, 1].min(), r[:, 1].max(), r[:, 2].mean()))

baseline       fail%  hit err  mean err (px)
image centre    88.2     2.69      9.24
random blob     77.2     0.00     22.29

model                  params  fail% (range)     hit err px (range)   mean err px
A1 upsample            360417    0.0 ( 0.0- 0.0)   0.49 (0.48-0.50)    0.49
A2 upsample+skips      360417    0.0 ( 0.0- 0.0)   0.51 (0.50-0.52)    0.51
B parallel, fused x4   377441    0.1 ( 0.0- 0.2)   0.60 (0.55-0.64)    0.61
C parallel, fused x1   339233    0.1 ( 0.1- 0.1)   0.74 (0.66-0.91)    0.77

The baselines show that the task needs context: choosing a blob at random fails about three times in four, and the image centre is far from most keypoints. All four networks learned to use the ring, with failures at or below 0.2%, so the comparison is about precision. The plan’s hypothesis on precision, “B has the lowest hit error; A1 the highest”, was not borne out. The two upsampling networks were the most precise, about 0.5 pixel, and the parallel networks were less precise, B by about 0.1 pixel and C by about 0.25 pixel, with seed ranges that do not overlap A1’s and A2’s. Only the planned ordering of B and C held: fusing four times beat fusing once, with seed ranges that do not overlap, the direction of the paper’s Table 6. Even with a perfect heatmap, the quarter-cell shift on a 2-pixel cell leaves an error of up to a quarter cell, 0.5 pixel, per axis, and about 0.38 pixel on average in two dimensions for uniformly placed keypoints (our calculation), so the upsampling networks are within about 0.1 pixel of the floor set by decoding. In our reading this toy task, with one isolated blob as the target, gives the high-resolution branch little to add that the transposed convolutions could not recreate from an $8 \times 8$ map in 600 steps. The HRNet paper’s advantage was measured on real images, with 210 epochs of training from scratch or from ImageNet weights, at stride 4 on maps of up to $96 \times 72$; the example shows that its mechanism does not by itself guarantee a gain in a small setting like this one, while the effect of repeated fusion appears even here.

Training

Pose (HRNetV1). Sun et al. (2019) trained on COCO train2017 (57K images, 150K person instances). Each ground-truth person box is extended to a height-to-width ratio of 4:3, cropped and resized to $256 \times 192$ or $384 \times 288$, with random rotation in $[-45^\circ, 45^\circ]$, random scaling in $[0.65, 1.35]$, flipping and half-body augmentation. The loss is the mean squared error between predicted and target heatmaps,

$$\mathcal{L} = \frac{1}{K}\sum_{k=1}^{K} \frac{1}{H'W'} \sum_{\mathbf{p}} \bigl(\hat{H}_k(\mathbf{p}) - H_k(\mathbf{p})\bigr)^2,$$

with $K = 17$ keypoints and targets that are 2D Gaussians at the annotated joints, as described in heatmap regression. The released configuration sets $\sigma = 2$ on the $64 \times 48$ heatmap (pose code); the two versions of the paper disagree on the target’s spread, the journal version stating a standard deviation of 2 pixels and the CVPR paper 1 pixel (Wang et al., 2021; Sun et al., 2019). Optimization uses Adam with a learning rate of $10^{-3}$, lowered to $10^{-4}$ at epoch 170 and $10^{-5}$ at epoch 200, for 210 epochs, following SimpleBaseline’s schedule. The backbone can be trained from scratch or initialized from ImageNet; for ImageNet pretraining the journal version trained the classification network for 100 epochs with SGD (Nesterov momentum 0.9, batch 256, learning rate 0.1 reduced tenfold at epochs 30, 60 and 90). MPII models used the same augmentation and schedule with $256 \times 256$ crops.

Segmentation (HRNetV2). On Cityscapes, Wang et al. (2021) trained on random $512 \times 1024$ crops of the $1024 \times 2048$ images with random scaling in $[0.5, 2]$ and horizontal flipping, using SGD with learning rate 0.01, momentum 0.9 and weight decay 0.0005, a poly schedule of power 0.9, 120K iterations, batch 12 on four GPUs and synchronized batch normalization. PASCAL-Context models used $480 \times 480$ inputs, learning rate 0.004 and weight decay 0.0001 for 60K iterations at batch 16; LIP models used $473 \times 473$ inputs for 110K iterations at batch 40.

Detection (HRNetV2p). Faster R-CNN and Cascade R-CNN models were trained in the MMDetection toolbox with its standard settings and images resized to a shorter side of 800 pixels; FCOS and the CenterNet of Duan et al. (keypoint triplets) were trained with their authors’ implementations (Wang et al., 2021).

Bottom-up pose (HigherHRNet). Cheng et al. (2020) cropped $512 \times 512$ patches with random rotation in $[-30^\circ, 30^\circ]$, scaling in $[0.75, 1.5]$, translation in $[-40, 40]$ pixels and flipping, and generated ground-truth heatmaps at both resolutions with the same standard deviation of 2. Keeping $\sigma$ fixed in cells, rather than scaling it with the map, was deliberate: on the finer map the same $\sigma$ is half as wide in image pixels, which suits small people. The heatmap loss is the sum of the mean squared errors at the two resolutions, the grouping loss of associative embedding is added with weight $10^{-3}$, and Adam runs at $10^{-3}$, dropped to $10^{-4}$ and $10^{-5}$ at epochs 200 and 260, for 300 epochs. The authors report that simply lowering $\sigma$ at the original resolution made optimization harder and gave worse results.

Inference

The pose HRNet is a top-down method: a person detector finds each person, and each box is cropped and processed separately, as described for top-down methods in human pose estimation. Sun et al. (2019) used the person detections released with SimpleBaseline, which the repository describes as having a person AP of 56.4 on COCO val2017 and 60.9 on test-dev (pose code, README). The heatmaps of the crop and of its mirror image are averaged, and each joint is taken at the maximum adjusted by “a quarter offset” toward the second-highest response, in the paper’s words; the released code shifts it by a quarter of a heatmap cell along each axis toward the higher of the two neighbours, the decoding described in heatmap regression. On MPII the ground-truth person boxes were used with a six-scale image pyramid. HRNet-W32 at $256 \times 192$ costs 7.1 GFLOPs and has 28.5M parameters, without the detector (Sun et al., 2019, Table 1).

HigherHRNet resizes the short side of the image to 512 (models trained at larger sizes are tested at their training size), predicts heatmaps at both resolutions, upsamples all of them bilinearly to the input size and averages them, then groups peaks by their tags. Flip testing is used throughout; the best results also average over several image scales (Cheng et al., 2020).

Performance

The pose results below are COCO keypoint AP, which averages precision over OKS thresholds from 0.50 to 0.95 (see COCO and average precision); all top-down rows use the same person detector, and the HRNet rows use flip testing.

Set	Method	Backbone	Input	Params	GFLOPs	AP
val2017	SimpleBaseline	ResNet-50	$256 \times 192$	34.0M	8.9	70.4
val2017	SimpleBaseline	ResNet-152	$256 \times 192$	68.6M	15.7	72.0
val2017	HRNet (from scratch)	HRNet-W32	$256 \times 192$	28.5M	7.1	73.4
val2017	HRNet	HRNet-W32	$256 \times 192$	28.5M	7.1	74.4
val2017	HRNet	HRNet-W48	$256 \times 192$	63.6M	14.6	75.1
val2017	SimpleBaseline	ResNet-152	$384 \times 288$	68.6M	35.6	74.3
val2017	HRNet	HRNet-W32	$384 \times 288$	28.5M	16.0	75.8
val2017	HRNet	HRNet-W48	$384 \times 288$	63.6M	32.9	76.3
test-dev2017	SimpleBaseline	ResNet-152	$384 \times 288$	68.6M	35.6	73.7
test-dev2017	HRNet	HRNet-W32	$384 \times 288$	28.5M	16.0	74.9
test-dev2017	HRNet	HRNet-W48	$384 \times 288$	63.6M	32.9	75.5
test-dev2017	HRNet + AI Challenger data	HRNet-W48	$384 \times 288$	63.6M	32.9	77.0

Source: Sun et al. (2019), Tables 1 and 2; rows without a note use ImageNet pretraining. On the MPII test set, HRNet-W32 at $256 \times 256$ scored 92.3% PCKh@0.5, equal to the best previously published result on the leaderboard of 16 November 2018, and HRNet-W48 obtained the same 92.3, which the authors attributed to saturation of the benchmark (Table 3). The repository adds MPII validation results with flip testing: 90.3% PCKh@0.5 and 37.7% at the stricter PCKh@0.1 for HRNet-W32, against 89.6% and 35.0% for SimpleBaseline with ResNet-152 (pose code, README). On the PoseTrack 2017 test set, HRNet-W48 with SimpleBaseline’s flow-based tracking reached 74.9 mAP and 57.9 MOTA, against 74.6 and 57.8 for the same tracking with a ResNet-152 (Table 5).

The paper’s ablations, on COCO val2017 at $256 \times 192$, isolate its two ideas:

• Repeated fusion. Trained from scratch, HRNet-W32 with only the final exchange unit scored 70.8 AP, with exchanges across stages (three fusions) 71.9, and with exchanges within stages as well (eight fusions) 73.4 (Table 6). The journal version repeats the comparison on Cityscapes val with HRNetV2-W48: 74.8, 75.4 and 76.4 mIoU (Wang et al., 2021, Table 12).
• Maintaining resolution. A variant with all four branches added at the beginning, of equal depth and similar parameters and GFLOPs, scored 72.5 AP against 73.4, which the authors attribute to low-resolution branches learning less useful low-level features early. A network of similar cost with only the high-resolution branch did much worse; the paper gives no number.
• Which branch to read. Heatmaps predicted from the 1/4-, 1/8- and 1/16-resolution outputs of an ImageNet-initialized HRNet-W32 scored 74.4, 68.9 and 55.0 AP (HRNet-W48: 75.1, 69.6 and 55.4); the 1/32 output scored below 10 (Fig. 5).
• Input size. Against SimpleBaseline with ResNet-50, HRNet-W32’s advantage grew as the input shrank, from 4.0 AP at $256 \times 192$ to 6.3 AP at $128 \times 96$, and HRNet-W32 at $256 \times 192$ outperformed that SimpleBaseline at $384 \times 288$ (Fig. 6).

Segmentation. On Cityscapes val with single-scale testing and no flipping, HRNetV2-W48 reached 81.1 mIoU at 696.2 GFLOPs for a $1024 \times 2048$ input, against 79.7 for PSPNet and 79.6 for DeepLabv3+ at 2017.6 and 1444.6 GFLOPs (Wang et al., 2021, Table 3). With six scales and flipping on the Cityscapes test set it reached 80.4 mIoU trained on the train set and 81.6 trained on train+val, without the coarse annotations (Table 4). On PASCAL-Context it scored 54.0 mIoU over 59 classes and 48.3 over 60, below APCN’s 55.6 in the authors’ own table (Table 5), and on LIP 55.90 mIoU without pose or edge information (Table 6). The official segmentation repository reports 80.9 on Cityscapes val for HRNetV2-W48 trained on the train set and tested at a single scale without flipping, reproduced with its PyTorch 1.1 code (segmentation code, README).

Detection and instance segmentation. On COCO val, HRNetV2p backbones were compared with ResNet and ResNeXt FPN backbones of similar size in Faster R-CNN, Cascade R-CNN, FCOS, CenterNet, Mask R-CNN, Cascade Mask R-CNN and Hybrid Task Cascade (Wang et al., 2021, Tables 8–10). The gains were mixed and largest for small objects: with Faster R-CNN on a 2× schedule, HRNetV2p-W32 reached 40.9 AP against 39.8 for ResNet-101-FPN, and 24.4 against 22.9 AP on small objects, while on a 1× schedule HRNetV2p-W18 fell 0.5 AP below ResNet-50-FPN, which the authors attributed to too few training iterations (Table 8). On COCO test-dev, Faster R-CNN with HRNetV2p-W48 reached 42.4 AP against 41.1 for ResNeXt-101-64×4d-FPN on a 2× schedule (Table 11).

Classification. On ImageNet, HRNet-W18-C (21.3M parameters, 3.99 GFLOPs) had a top-1 error of 23.1% against 24.6% for a ResNet-38 of similar size, and the classification networks were “comparable to and slightly better than” ResNets at similar cost, in the authors’ words (Wang et al., 2021, Table 13).

HigherHRNet. On COCO test-dev2017, the authors’ bottom-up baseline with HRNet-W32 at 512 scored 64.1 AP; HigherHRNet with HRNet-W32 at 512 scored 66.4 AP, with 0.4% more parameters and 23.1% more GFLOPs, and HigherHRNet with HRNet-W48 at 640 scored 68.4 AP, or 70.5 with multi-scale testing, against 68.7 for PersonLab with multi-scale testing (Cheng et al., 2020, Table 1). Medium-sized people gained most: 66.6 against PersonLab’s 64.1 AP. The ablations on val2017 with HRNet-W32 at 512 show where the gains come from:

Variant (HRNet-W32, input 512, COCO val2017)	AP	AP medium	AP large
HRNet, heatmaps at 1/4	64.4	57.1	75.6
+ deconvolution module, predict at 1/2 only	66.0	60.7	74.2
+ heatmaps concatenated into the module	66.3	60.8	74.0
+ heatmap aggregation over both resolutions	66.9	61.0	75.7
+ four residual blocks	67.1	61.5	76.1

Source: Cheng et al. (2020), Table 4. Predicting at the finer resolution alone helped medium people and hurt large ones; averaging the two resolutions recovered the large people. A second deconvolution module, predicting at full input resolution, lowered AP from 66.9 to 66.5, with a 0.8-point drop for large people (Table 3). Training at 640 instead of 512 raised AP from 67.1 to 68.5, and 768 gave no further gain overall while large-person AP fell (Table 5). The paper’s text attributes the 1.7-point large-person gain of heatmap aggregation to a comparison of methods (d) and (e), but the gain appears between (c) and (d) of its Table 4 (74.0 to 75.7). Similarly, its Table 6 lists 47.8 GFLOPs for an HRNet-W32 model whose AP, 68.5, is that of the 640 training size in Table 5; the figure is close to the 47.9 GFLOPs of the 512 model in Table 1, whereas the official repository lists 74.8 GFLOPs for the 640 model with the same 68.5 AP (HigherHRNet code, README). On the CrowdPose test set, trained on its train and val sets with $640 \times 640$ crops, HigherHRNet-W48 reached 67.6 AP with multi-scale testing, above the 66.0 of the best top-down method with refinement in the authors’ comparison, and 65.9 without it (Table 7).

Strengths

• Spatial precision without a decoder. The high-resolution branch is computed throughout, so the output is not reconstructed from a coarse map, and the ablations show the high-resolution output to be the best one to read (Sun et al., 2019, Fig. 5).
• Context at every resolution. Repeated fusion gives the high-resolution branch information from the coarse branches many times; more fusions scored higher in both papers’ ablations, for pose and for segmentation.
• Accuracy per operation. At equal input size, HRNet-W32 matched or beat SimpleBaseline with ResNet-152 at about half the parameters and GFLOPs, and its advantage grew at small input sizes (Sun et al., 2019).
• One body for several tasks. The same body served pose, segmentation, detection and classification by changing only the head, and its released ImageNet-pretrained weights made it a drop-in backbone (Wang et al., 2021; classification code, README).

Limitations

• Top-down cost. The pose HRNet processes every detected person separately and depends on the detector’s boxes, the general limitation of top-down methods; HigherHRNet removes the detector but, on COCO test-dev, its best result (70.5 AP) stayed below the top-down HRNet-W48 (75.5 AP) (Cheng et al., 2020, Table 2).
• Operations against runtime. The papers compare parameters and GFLOPs, not measured speed or memory; that keeping many channels at 1/4 resolution costs more memory traffic than its GFLOPs suggest is our reasoning, not a finding of the papers. The authors listed aggregating multi-resolution representations “in a less light way” as future work (Sun et al., 2019).
• Mixed gains as a detection backbone. HRNetV2p beat ResNet backbones of similar size in most but not all of the authors’ detection comparisons; some short-schedule and ResNeXt comparisons went the other way (Wang et al., 2021, Tables 8, 10 and 11).
• Scale settings that depend on the data. In HigherHRNet, the number of deconvolution modules and the training size had to match the dataset’s person sizes: a second module or a larger input helped medium people at the expense of large ones (Cheng et al., 2020).
• Heatmap resolution still bounds precision. HRNet’s output is a heatmap at 1/4 resolution, so its localization still depends on the decoding and target choices of heatmap regression; DARK and UDP, evaluated with HRNet backbones, changed its COCO AP by about one to several points without changing the network (see heatmap regression).

Historical Importance

HRNet appeared in 2019, when pose networks had converged on two patterns: symmetric encoder–decoders such as the stacked hourglass network, and classification backbones with a light upsampling head such as SimpleBaseline. It showed that a network built around a high-resolution path from the start could beat both at lower cost, and HRNet-W32 and HRNet-W48 at $256 \times 192$ and $384 \times 288$ reappear as the backbones of later pose work: the decoding and target methods covered in heatmap regression (DARK, UDP, RLE, SimCC) report their results with HRNet backbones, and skeleton-based action recognition methods such as PoseConv3D take their 2D joints from it (see skeleton-based action recognition).

The journal version argued for the same design as a general alternative to classification backbones for “position-sensitive” tasks, and the authors released segmentation and classification code built on it alongside the pose code. Its later influence is visible in designs that keep the parallel multi-resolution structure with new blocks, such as Lite-HRNet and HRFormer, both from overlapping author groups.

Variants

• HRNetV1, HRNetV2 and HRNetV2p (2019–2021). The three heads described above (Wang et al., 2021). The journal version describes itself as an extension of the CVPR paper with material from an unpublished technical report on high-resolution representations for labeling pixels and regions, which the official segmentation and classification repositories link as their source. HRNetV2-W32 scored 73.6 AP on COCO val2017 pose estimation from scratch, against 73.4 for HRNetV1-W32, while HRNetV2 outperformed HRNetV1 clearly in segmentation and detection, except for a minor gain for the larger model on Cityscapes; a variant HRNetV1h, which only widens HRNetV1’s output by a $1 \times 1$ convolution to HRNetV2’s dimension, improved slightly, which the authors read as evidence that the gain comes from aggregating the low-resolution branches (Fig. 10).
• HigherHRNet (2020). Bottom-up pose with a high-resolution feature pyramid, multi-resolution supervision and heatmap aggregation (Cheng et al., 2020).
• HRNet with object-contextual representations. The segmentation repository combines HRNetV2 with object-contextual representations (OCR), reporting 81.6 mIoU on Cityscapes val and 82.3 on its test set trained on train+val with multi-scale testing and flipping (segmentation code, README, as of November 2026); the journal version reports 82.3% on Cityscapes test with coarse data for a combination with a variant of object context.
• Lite-HRNet (2021). Yu et al. (2021) applied ShuffleNet’s shuffle block to HRNet, found that its $1 \times 1$ convolutions dominated the cost, and replaced them with conditional channel weighting, which computes weights from all channels and all resolutions of the parallel branches. Lite-HRNet-30, with 1.8M parameters and 0.70 GFLOPs at $384 \times 288$, scored 69.7 AP on COCO test-dev (Table 4).
• HRFormer (2021). Yuan et al. (2021) kept HRNet’s multi-resolution parallel design and replaced its convolutional blocks with local-window self-attention and a feed-forward network containing a depth-wise convolution, which exchanges information across windows. HRFormer-B (43.2M parameters, 26.8 GFLOPs at $384 \times 288$) scored 77.2 AP on COCO val2017 and 76.2 on test-dev, against 76.3 and 75.5 for HRNet-W48 in the authors’ tables (Tables 3 and 4). The proceedings page titles the paper “HRFormer: High-Resolution Vision Transformer for Dense Predict[ion]”, while the paper itself prints “High-Resolution Transformer”; it is cited here by the printed title.
• Small HRNets. The classification repository releases two small models, HRNet-W18-C-Small-v1 and v2, with parameters and GFLOPs similar to ResNet-18 (classification code, README).

Implementations

As of November 2026, the official code is on GitHub under the MIT licence: the pose code in leoxiaobin/deep-high-resolution-net.pytorch, and the HigherHRNet, segmentation and classification code in the HRNet organization (HigherHRNet-Human-Pose-Estimation, HRNet-Semantic-Segmentation, HRNet-Image-Classification). GitHub’s metadata shows no recognized licence for the segmentation repository, but its LICENSE file is the MIT licence. The repositories link pretrained weights for the configurations in their tables, including ImageNet-pretrained backbones (HRNet-W18-C to HRNet-W64-C) and later checkpoints trained with longer schedules. The repositories were last updated in 2021–2022, and their READMEs name PyTorch versions from 0.4.1 to 1.4 for installation and testing.