PyTorch

PyTorch is an open-source Python library for numerical computing with multidimensional arrays (tensors) on CPUs and GPUs, with automatic differentiation and a library of neural-network building blocks. It was developed at Facebook (now Meta) from 2016 by people from the community around Torch, an older machine-learning library scripted in Lua, and since September 2022 it has been a project of the PyTorch Foundation, part of the Linux Foundation. In computer vision it is the framework in which many convolutional networks, vision transformers and detection and segmentation models are written, trained and released. Every API statement and example here was checked against PyTorch 2.14.1 and TorchVision 0.29.2, as of October 2026; the newest release at that date is PyTorch 2.15.0 (28 October 2026). The official documentation is the reference for details this article leaves out.

Purpose

Deep-learning frameworks of the mid-2010s, such as Caffe, CNTK, TensorFlow and Theano, built a static dataflow graph of the computation first and then executed it on batches of data. Paszke et al. (2019), the paper that describes PyTorch’s design, argue that this gives the framework a view of the whole computation for optimization but costs ease of use, ease of debugging and flexibility. PyTorch takes the opposite approach, which Chainer and DyNet had used before it: define-by-run or eager execution, in which each operation runs immediately when the Python line that calls it runs. A model is an ordinary Python program, so loops, conditionals, print statements and standard debuggers work inside it.

The paper names four design principles: be Pythonic, put researchers first, provide pragmatic performance, and “worse is better”, meaning a simple implementation is preferred to a complete but complex one. On performance it gives a concrete threshold: trading 10% of speed for a much simpler model is acceptable, trading 100% is not. The same eager style carries over to data loading, optimizers and parallel training, which are also plain Python objects that users can replace.

PyTorch is used for the whole life of a learned vision model: writing the architecture, training it, evaluating it, and converting it for inference elsewhere. The training method itself (splits, learning curves, early stopping, hyperparameter search) belongs to Training a Neural Network, and the gradient algorithm to Backpropagation; this article covers the framework that runs them.

Core Capabilities

Tensors

A torch.Tensor is an $n$-dimensional array with a data type (float32 by default for floating point, also float16, bfloat16, integer and Boolean types) and a device (CPU, a CUDA or ROCm GPU, an Apple GPU through the MPS backend, and others). About two thousand operators work on tensors (the PyTorch 2.0 release post puts the number at about 2,000 or more): arithmetic, reductions, linear algebra, indexing, and the convolution, pooling and normalization operators of deep learning. Images are stored channel-first; a batch of $N$ images with $C$ channels has shape $(N, C, H, W)$, the input layout that torch.nn.Conv2d documents.

Tensors exchange data with NumPy through torch.from_numpy() and .numpy() without copying: both objects describe the same block of memory, so the conversion takes constant time regardless of array size (Paszke et al., 2019). The DLPack format provides the same zero-copy exchange with other array libraries. Most of PyTorch is implemented in C++ (the libtorch core), with Python bindings generated from metadata; on GPUs, operators are queued asynchronously on CUDA streams, so Python can run ahead while the GPU works, and a caching allocator reuses GPU memory instead of calling CUDA’s memory routines on every operation, since cudaFree can block until all queued GPU work has finished (Paszke et al., 2019).

Automatic differentiation

torch.autograd is a reverse-mode automatic differentiation system. When a tensor with requires_grad=True takes part in an operation, PyTorch records the operation in a directed acyclic graph whose leaves are the inputs and whose roots are the outputs; each result’s grad_fn attribute points into this graph. Calling .backward() on a scalar output traverses the graph from the root and applies the chain rule, which is backpropagation in its general form. The graph is rebuilt from scratch on every forward pass, which is what lets the forward pass use arbitrary Python control flow: the documentation’s phrase is “what you run is what you differentiate” (Autograd mechanics, version 2.14).

Gradients accumulate: backward() adds into each leaf tensor’s .grad rather than overwriting it, so a training loop clears them every step, usually with optimizer.zero_grad(). Users can add their own differentiable operations by subclassing torch.autograd.Function with forward and backward methods. Paszke et al. (2019) describe a less common feature: autograd can differentiate code that modifies tensors in place, using a versioning system for tensors; the autograd notes explain that each tensor’s version counter detects when a value saved for the backward pass has been overwritten. The complicated cases are reported as errors rather than handled by silently copying data.

Modules and optimizers

A layer or a whole model is a subclass of torch.nn.Module. Its constructor creates parameters and submodules as attributes, and its forward method computes the output. Assigning an nn.Parameter or another module to an attribute registers it, so model.parameters() finds every trainable tensor and model.state_dict() returns them by name for saving and loading. model.train() and model.eval() switch layers such as dropout and batch normalization between their training and inference behaviour. torch.nn provides the standard layers and losses, among them Conv2d, Linear, MaxPool2d, BatchNorm2d and CrossEntropyLoss, and torch.nn.functional the same operations as functions without stored parameters.

Optimizers in torch.optim take the list of parameters and update them in place from their .grad fields; SGD, Adam and AdamW are among the documented ones, and torch.optim.lr_scheduler provides learning-rate schedules. PyTorch has no built-in fit method: the training loop is written by the user, typically as forward pass, loss, zero_grad(), backward() and step() for each mini-batch. Paszke et al. (2019) illustrate the benefit with a generative adversarial network, whose two models and two interdependent losses are awkward to express in a rigid training API but take a few lines in a hand-written loop.

Data loading

torch.utils.data separates what a dataset is from how it is read. A map-style dataset subclasses Dataset and implements __len__ and __getitem__, so it behaves like a list of samples that may be loaded lazily from disk; an iterable-style dataset subclasses IterableDataset and yields samples as a stream. A DataLoader wraps either kind and handles shuffling, batching (stacking samples into tensors through a collate_fn), loading in parallel worker processes (num_workers), and copying batches into page-locked memory (pin_memory=True) for faster transfer to a GPU. For images, the dataset’s __getitem__ usually decodes a file and applies transforms such as data augmentation; TorchVision provides the transforms and many standard datasets.

Eager mode and compilation

Eager execution sees one operator at a time, so it cannot fuse neighbouring operators or schedule across them, and Python adds overhead to each call. PyTorch has tried several ways of capturing a whole graph. TorchScript (introduced with torch.jit in PyTorch 1.0) offered torch.jit.trace, which records the operators run on example inputs, and torch.jit.script, which parses a subset of Python. Ansel et al. (2024) report their limits: tracing specializes on the path the example input takes through Python control flow, so other inputs can give incorrect results, and scripting worked on only 36 of the 80 TorchBench models they tested (45%).

torch.compile, introduced in PyTorch 2.0 (March 2023), is the current approach. Its front end, TorchDynamo, hooks into CPython’s frame evaluation API and rewrites Python bytecode just before it runs, extracting sequences of PyTorch operations into an FX graph. Where it meets something it cannot capture, such as a call into an unsupported library, it ends the graph (a graph break), lets Python run that part eagerly, and starts a new graph afterwards. Each compiled graph is protected by guards, checks on input shapes, types and other state; if a guard fails on a later call, the code is recompiled, up to a limit (8 by default in version 2.14) after which PyTorch falls back to eager execution. AOTAutograd traces the backward pass ahead of time, and the default back end, TorchInductor, generates code: Triton kernels for GPUs and C++ with OpenMP for CPUs. The mode argument selects trade-offs, for example "reduce-overhead", which uses CUDA graphs to cut Python overhead for small batches, or "max-autotune", which also tries Triton- or template-based matrix multiplications (and, on GPUs, convolutions).

Ansel et al. (2024) reported that TorchDynamo captured 93% of 80 TorchBench models, all 46 HuggingFace models and all 62 TIMM models in float32 inference on an NVIDIA A100 GPU; that its graph-capture overhead was 5% of eager inference time on TorchBench (measured on an NVIDIA V100); and, in their abstract’s summary, that TorchInductor gave geometric-mean speedups over eager PyTorch of 2.27× for inference and 1.41× for training on the A100 across more than 180 models from the three suites, using a PyTorch nightly build of 30 August 2023 with max-autotune, parameter freezing and CUDA graphs enabled. Speedups on other models, hardware and versions differ; the first call of a compiled function also pays the compilation time.

Distributed training

torch.distributed provides collective communication between processes through four built-in back ends in version 2.14: Gloo (CPU), NCCL (CUDA GPUs), XCCL (Intel XPU GPUs) and MPI. Two wrappers cover the common forms of data parallelism:

• DistributedDataParallel (DDP) keeps a full copy of the model in each process. At construction it broadcasts the parameters from rank 0 so that all replicas start equal; during the backward pass it groups gradients into buckets and starts an asynchronous all-reduce on each bucket as soon as its gradients are ready, so that communication overlaps with the rest of the backward pass and every replica ends the step with the same averaged gradients (DDP design note, version 2.14). Each process reads a different shard of the data, usually through a DistributedSampler.
• Fully sharded data parallelism (FSDP), in its second version fully_shard (FSDP2), splits each parameter across processes, gathers the full parameters just before a module’s forward or backward pass, and frees them afterwards, so that each device holds only its shard between uses. The documentation asks users of the first FSDP API to consider migrating to FSDP2.

Tensor parallelism and pipeline parallelism, which split a model’s layers or operations across devices, are built on the same communication layer. Automatic mixed-precision training (today torch.autocast with a gradient scaler; stable since PyTorch 1.6, then as torch.cuda.amp) is described in Training a Neural Network.

Minimal Example

The script renders 32×32 grayscale images of disks, squares and triangles at random sizes, positions and contrasts with additive noise, so the labels are exactly known. It shows the pieces above in turn: a map-style Dataset that renders image $i$ from a seed derived from $i$, a DataLoader with a seeded shuffle, autograd checked against a finite difference, a small CNN as an nn.Module, an explicit training loop with Adam, torch.compile checked against eager execution, and torch.export. It was run with PyTorch 2.14.1 on a CPU with four threads; it takes under 10 seconds (6.4 to 7.3 s in four runs on an Apple M5, including compilation) and printed identical output both times.

import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import Dataset, DataLoader

torch.manual_seed(0)
torch.set_num_threads(4)
print(f"torch {torch.__version__}")

# 1. A Dataset that renders a 32x32 grayscale shape (0 disk, 1 square, 2 triangle).
def render(kind, rng, size=32):
    yy, xx = np.mgrid[0:size, 0:size] + 0.5
    r = rng.uniform(5, 10)                        # half-size in pixels
    cx, cy = rng.uniform(r + 1, size - r - 1, 2)  # centre, fully inside the image
    if kind == 0:
        mask = (xx - cx) ** 2 + (yy - cy) ** 2 <= r ** 2
    elif kind == 1:
        mask = (abs(xx - cx) <= r) & (abs(yy - cy) <= r)
    else:                                         # upright isosceles triangle
        mask = (yy >= cy - r) & (yy <= cy + r) & (abs(xx - cx) <= (yy - (cy - r)) / 2)
    img = np.where(mask, rng.uniform(0.6, 1.0), rng.uniform(0.0, 0.3))
    img = img + rng.normal(0, 0.1, img.shape)     # sensor-like noise
    return img.astype(np.float32)

class Shapes(Dataset):
    def __init__(self, n, seed):
        self.n, self.seed = n, seed
    def __len__(self):
        return self.n
    def __getitem__(self, i):
        rng = np.random.default_rng([self.seed, i])  # image i is the same on every call
        label = i % 3
        x = torch.from_numpy(render(label, rng)).unsqueeze(0)  # (1, 32, 32)
        return x, label

train_set, test_set = Shapes(1800, seed=1), Shapes(600, seed=2)
loader = DataLoader(train_set, batch_size=32, shuffle=True,
                    generator=torch.Generator().manual_seed(0))
xb, yb = next(iter(loader))
print(f"batch {tuple(xb.shape)} {xb.dtype}, labels {yb[:8].tolist()}")

# 2. Autograd on one image: gradient of a loss with respect to a 3x3 kernel.
img = test_set[1][0].unsqueeze(0)                 # a square, shape (1, 1, 32, 32)
k = torch.randn(1, 1, 3, 3, requires_grad=True)
loss = F.conv2d(img, k).pow(2).mean()
loss.backward()                                   # reverse-mode autodiff
eps, e = 1e-3, torch.zeros_like(k); e[0, 0, 1, 2] = eps
with torch.no_grad():
    fd = ((F.conv2d(img, k + e).pow(2).mean() - F.conv2d(img, k - e).pow(2).mean())
          / (2 * eps)).item()
print(f"grad_fn {type(loss.grad_fn).__name__}; dL/dk[1,2] autograd "
      f"{k.grad[0, 0, 1, 2].item():.5f}, central difference {fd:.5f}")

# 3. A model is an nn.Module; its parameters are registered automatically.
class SmallCNN(nn.Module):
    def __init__(self, n_classes=3):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 8, 3, padding=1)
        self.conv2 = nn.Conv2d(8, 16, 3, padding=1)
        self.fc = nn.Linear(16 * 8 * 8, n_classes)
    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), 2)   # 8 x 16 x 16
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)   # 16 x 8 x 8
        return self.fc(x.flatten(1))

model = SmallCNN()
print(f"parameters: {sum(p.numel() for p in model.parameters())}")

# 4. The training loop: forward, loss, backward, step.
x_test, y_test = next(iter(DataLoader(test_set, batch_size=len(test_set))))
opt = torch.optim.Adam(model.parameters(), lr=3e-3)
for epoch in range(1, 9):
    model.train()
    total = 0.0
    for x, y in loader:
        loss = F.cross_entropy(model(x), y)
        opt.zero_grad()
        loss.backward()
        opt.step()
        total += loss.item() * len(y)
    model.eval()
    with torch.no_grad():
        acc = (model(x_test).argmax(1) == y_test).float().mean().item()
    print(f"epoch {epoch}  train loss {total / len(train_set):.4f}  test accuracy {acc:.3f}")

# 5. Compilation: same model, same weights, graph captured and compiled.
compiled = torch.compile(model)                   # default backend: inductor
with torch.no_grad():
    eager_out, comp_out = model(x_test), compiled(x_test)
print(f"compiled vs eager on {len(x_test)} test images: max |diff| "
      f"{(eager_out - comp_out).abs().max().item():.1e}, same predictions: "
      f"{bool((eager_out.argmax(1) == comp_out.argmax(1)).all())}")

# 6. Export: a whole-graph, Python-free representation for deployment tools.
ep = torch.export.export(model, (x_test[:1],))
ops = [str(n.target) for n in ep.graph.nodes if n.op == "call_function"]
print(f"exported graph: {len(ops)} operator calls: {', '.join(o.split('.')[1] for o in ops)}")

Output:

torch 2.14.1
batch (32, 1, 32, 32) torch.float32, labels [2, 1, 1, 0, 1, 1, 1, 2]
grad_fn MeanBackward0; dL/dk[1,2] autograd -2.03554, central difference -2.03586
parameters: 4323
epoch 1  train loss 1.0244  test accuracy 0.570
epoch 2  train loss 0.7534  test accuracy 0.675
epoch 3  train loss 0.5009  test accuracy 0.883
epoch 4  train loss 0.3194  test accuracy 0.878
epoch 5  train loss 0.2040  test accuracy 0.933
epoch 6  train loss 0.1478  test accuracy 0.978
epoch 7  train loss 0.1140  test accuracy 0.963
epoch 8  train loss 0.0898  test accuracy 0.977
compiled vs eager on 600 test images: max |diff| 0.0e+00, same predictions: True
exported graph: 8 operator calls: conv2d, relu, max_pool2d, conv2d, relu, max_pool2d, flatten, linear

The DataLoader turned single samples of shape $(1, 32, 32)$ into batches in the $(N, C, H, W)$ layout that Conv2d expects. The autograd gradient and the central difference agree to about one part in 6,000; in our reasoning the gap is the finite difference’s own error, since it divides a difference of two nearly equal float32 losses by $2 \times 10^{-3}$. The loss tensor’s grad_fn is the backward node of its last operation, the mean. The parameter count is the three layers’ weights and biases: $8 \cdot 9 + 8 = 80$, $16 \cdot 72 + 16 = 1{,}168$ and $1{,}024 \cdot 3 + 3 = 3{,}075$.

Training loss falls every epoch, while test accuracy on the 600 held-out images rises unevenly, with small drops at epochs 4 and 7, and ends at 97.7% for this single seeded run. Each epoch draws the same batches on every run because the shuffle has its own seeded generator and each image is rendered from its own seed.

The compiled model gives bitwise the same outputs as the eager one here. Printing the code Inductor generated (with the environment variable TORCH_LOGS=output_code) shows why: on this CPU it calls the same ATen convolution and matrix-multiplication operators as eager mode, and generates C++ (parallelized with OpenMP) only for two fused kernels, each applying a ReLU and the following max-pooling in one pass over the data; the first also copies the second convolution’s weights into another memory layout. Fusion removes intermediate tensors but, in this case, does not change the arithmetic; on other hardware or with other modes, compiled and eager results can differ in the last bits because operations are reordered. The exported program is the model as a flat list of eight ATen operator calls, with the Python class structure inlined: this is the form deployment tools consume.

Ecosystem and Integrations

TorchVision is the computer vision library of the PyTorch project: popular datasets, model architectures and image transformations (TorchVision documentation, version 0.29). Its transforms.v2 API transforms images together with bounding boxes, masks, keypoints and videos, using tensor subclasses (TVTensors) that tell each transform how to treat its input. torchvision.models provides classification, segmentation, detection, video and optical-flow architectures, with optional pretrained weights (ResNet and the vision transformer among them, used as starting points for transfer learning); torchvision.ops provides detection operators such as non-maximum suppression and box utilities. TorchVision 0.29.0 (September 2026) deprecated its image decoders and encoders in favour of the separate TorchCodec library, and 0.26 had already removed its video decoding and encoding; 0.29.2 is compatible with PyTorch 2.14 and later (TorchVision release notes). Other libraries of the project cover audio (TorchAudio), quantization (torchao, to which the PyTorch 2 export-based quantization flow moved in 2.8) and on-device inference (ExecuTorch).

Export and deployment have several paths, with different scopes:

• torch.export.export() traces a module ahead of time into an ExportedProgram, a graph of ATen operators with no Python semantics. Unlike torch.compile, it does not fall back to Python at a graph break; it raises an error, and the user rewrites the code or describes dynamic shapes. The documentation describes the result as sound, unlike torch.jit.trace: it records the conditions on input shapes under which the trace stays valid.
• ONNX export. torch.onnx.export() converts a model to the ONNX interchange format, which many runtimes, among them ONNX Runtime, can execute. Since PyTorch 2.9 it uses the torch.export path by default (dynamo=True); the older TorchScript-based exporter remains available with dynamo=False but has been deprecated since 2.8. The ONNX exporter goes back to PyTorch 0.3 (December 2017).
• AOTInductor, added in PyTorch 2.2, compiles an exported program ahead of time for server-side deployment without Python.
• ExecuTorch is the project’s solution for inference on edge devices, from mobile phones to microcontrollers, with models exported from PyTorch and lowered to the target hardware (ExecuTorch documentation, version 1.5).
• TorchScript (torch.jit.script, torch.jit.trace) is deprecated. PyTorch 2.10 added deprecation warnings to its user-facing API and stated that it is not guaranteed to work on Python 3.14; from 2.14 the warnings are visible FutureWarnings that point to torch.compile or torch.export. Models saved as TorchScript still load in 2.14.

Model hubs. torch.hub, introduced in PyTorch 1.0, loads a model definition and its weights from a GitHub repository that contains a hubconf.py file, as TorchVision’s own repository does. Pretrained models are also distributed through libraries built on PyTorch, such as the HuggingFace library of transformer models and TIMM, a library of vision models, which Ansel et al. (2024) describe as popular and use as two of their benchmark suites.

Hardware. Besides CPUs, the official builds of 2.15 support NVIDIA GPUs through CUDA, AMD GPUs through ROCm, Apple GPUs through MPS and Intel GPUs (2.15 release notes); the MPS device was added in PyTorch 1.12 (June 2022).

Alternatives

The alternatives are other general deep-learning frameworks, compared here only as their sources describe them.

• TensorFlow is one of the frameworks Paszke et al. (2019) describe as building a static dataflow graph, the design they contrast with PyTorch’s. Ansel et al. (2024) observe that users’ preference for eager programming has led graph-mode frameworks to adopt eager-mode programming models.
• JAX is grouped by Ansel et al. (2024) with PyTorch as an eager-mode framework.
• Chainer and DyNet pioneered define-by-run model authoring before PyTorch (Paszke et al., 2019). Theano, Caffe, CNTK and MXNet are earlier graph-based frameworks that the same papers compare against.

The only head-to-head speed comparison in these sources is dated: Paszke et al. (2019) measured training throughput of six models on one NVIDIA Quadro GP100 GPU in 32-bit floating point against Chainer, CNTK, MXNet, PaddlePaddle and TensorFlow, and found PyTorch within 17% of the fastest framework on every benchmark, which they attributed to all frameworks calling the same cuDNN and cuBLAS libraries. Such comparisons depend on versions and hardware and are not current.

Versions and Compatibility

PyTorch’s lineage starts with Torch, a modular machine-learning library described in a 2002 Idiap research report by Collobert, Bengio and Mariéthoz. Its later version, Torch7 (Collobert, Kavukcuoglu & Farabet, 2011), extended the Lua scripting language with an $n$-dimensional Tensor library, an nn package of neural-network modules, and OpenMP and CUDA implementations of the low-level routines. Torch7 relied on Lua’s garbage collector to free memory, which Paszke et al. (2019) cite as a reason PyTorch instead frees each tensor as soon as its reference count drops to zero. PyTorch’s licence file still lists Idiap Research Institute copyrights from 2001 to 2014 alongside Facebook’s.

The dates below are those of the GitHub releases of pytorch/pytorch, unless stated otherwise:

Date	Version or event	Main change
Sep 2016	v0.1.1	First alpha release on GitHub
Jan–Feb 2017	v0.1.6	“Beta is here”: the first public release
Dec 2017	v0.3.0	Model exporter to ONNX
Apr 2018	v0.4.0	Tensor and Variable merged into one class
May 2018	Blog post	PyTorch 1.0 plan: merge with Caffe2, torch.jit for production
Dec 2018	v1.0.0	TorchScript (torch.jit), redesigned torch.distributed, C++ front end, Torch Hub
Jul 2020	v1.6.0	Automatic mixed precision stable
Sep 2022	PyTorch Foundation	Project moved from Meta to the Linux Foundation
Mar 2023	v2.0.0	torch.compile (TorchDynamo, AOTAutograd, PrimTorch, TorchInductor)
Jan 2024	v2.2.0	AOTInductor
Oct 2025	v2.9.0	ONNX export defaults to the torch.export path
Jan 2026	v2.10.0	Deprecation warnings on TorchScript
Sep 2026	v2.14.0, v2.14.1	Version used in this article
Oct 2026	v2.15.0	Rewritten NCCL back end by default; binaries need Python 3.11 or newer

The sources date the first release differently. The v0.1.6 release notes say it was frozen and released on 21 January 2017, Paszke et al. (2019) give the initial release as January 2017, and Chintala (2022) marks the fifth anniversary in January 2022, while the Linux Foundation’s 2022 press release speaks of PyTorch’s “release in 2016”, which matches the alpha releases on GitHub from September 2016. Chintala (2022) writes that development started in 2016 with people from the Lua Torch community, much of the staff and funding from Meta, and contributors from NVIDIA, Twitter and others.

PyTorch 1.0 (December 2018) was the release aimed at production. The PyTorch team’s May 2018 post announced it as a merger of PyTorch with Caffe2, Facebook’s production framework, and introduced torch.jit. The PyTorch Foundation was announced on 12 September 2022, with a governing board of AMD, Amazon Web Services, Google Cloud, Meta, Microsoft Azure and NVIDIA; Chintala (2022) states that business decisions move to the foundation while technical decisions remain with the individual maintainers. PyTorch 2.0 (15 March 2023) kept the eager programming model and added torch.compile as an optional feature, which its release notes describe as fully additive and therefore fully backward compatible.

Compatibility points for users, as of October 2026:

• Release cadence. From 2.6 (January 2025) to 2.15 a feature release (2.x.0) appeared every 7 to 15 weeks (7 to 9 weeks apart between the 2026 releases), and four of them were followed by a bug-fix release; deprecated APIs usually warn for one or more releases before removal. This is our reading of the release dates and notes, not a stated policy.
• Python. PyTorch 2.15 dropped the Python 3.10 binaries; its wheels support Python 3.11 to 3.15. Each TorchVision release is built for a matching PyTorch release; TorchVision 0.29 is ABI-stable with respect to PyTorch 2.14, and since 0.29.1 its package requirement accepts any PyTorch from 2.14 on instead of one exact version.
• Licence. PyTorch is distributed under a BSD-style three-clause licence (the LICENSE file at v2.14.1), and TorchVision under the BSD 3-Clause licence.
• Saved models. For compatibility, the serialization notes recommend saving a module’s state_dict (its tensors by parameter name) rather than the module object. Since version 2.6, torch.load defaults to weights_only=True, which restricts unpickling to tensors and a few primitive types; the notes say this narrows, but does not remove, the risk of loading untrusted files. Serialized TorchScript is deprecated, and torch.export programs are the documented route to deployment formats.
• Reproducibility. The documentation states that fully reproducible results are not guaranteed across PyTorch releases, individual commits or platforms, or between CPU and GPU, even with identical seeds. Seeds and torch.use_deterministic_algorithms limit the sources of nondeterminism on one platform, device and version; the example also fixes the thread count, because in our understanding the order of floating-point sums depends on how work is split across threads.

Limitations

• Python overhead in eager mode. Each operator call passes through the Python interpreter, and the global interpreter lock lets only one thread run Python at a time (Paszke et al., 2019). Large tensor operations hide this cost; many small operations, small batches or per-pixel Python loops do not. torch.compile, CUDA graphs and vectorized code reduce it.
• Compilation has its own failure modes. Graph breaks split a model into several smaller graphs and reduce the benefit; Ansel et al. (2024) found 10 or more graph breaks in 22% of the TorchBench models that TorchDynamo handled. Changing input shapes can trigger recompilation until the recompile limit is reached, and the first call pays the compilation time.
• Export needs traceable code. torch.export and the default ONNX exporter require the whole model to be capturable as one graph, so data-dependent Python control flow must be rewritten, for example with the explicit control-flow operators the export documentation describes, and dynamic input sizes must be declared.
• A moving API. The deployment path has changed several times (TorchScript in 1.0, torch.export in 2.x, a new ONNX exporter default in 2.9), so older tutorials and released code can rely on APIs that are now deprecated. Version-pinned documentation, such as the 2.14 pages cited here, is the reliable reference.
• No built-in training loop or experiment management. Checkpointing, logging, early stopping and distributed launch scripts are written by the user or taken from libraries built on PyTorch.