ONNX

ONNX (Open Neural Network Exchange) is an open format for trained machine-learning models: a file, usually with the extension .onnx, that stores a model as a graph of standard operators together with its weights, plus the specification that defines what each operator computes. A model trained in one framework, typically PyTorch, is exported to ONNX and then run by another program, such as ONNX Runtime, NVIDIA TensorRT, Intel OpenVINO or the dnn module of OpenCV, without the training framework or its Python code. Facebook and Microsoft announced the format in September 2017; it is now a community project with open governance, hosted by the Linux Foundation’s LF AI & Data Foundation, and its specification and Python package live in the onnx/onnx repository under the Apache License 2.0. Every statement about the specification and the tools here was checked against ONNX 1.23.2, ONNX Script 0.7.2, ONNX Runtime 1.31.1, PyTorch 2.14.1 and TorchVision 0.29.2, as of October 2026. ONNX 1.23.2 (6 October 2026) was the newest ONNX release at that date; the specification documents linked in the references are the authoritative description of the format.

Purpose

Every deep-learning framework represents a model as a dataflow graph internally, but each in its own form. The ONNX overview argues that this makes each framework a siloed stack of API, graph and runtime, and that converting between them delays the move from research to production. Its answer is a common intermediate representation (IR): one portable description of the computation that frameworks can write and that runtimes and hardware vendors can optimize for.

The 2017 announcement by Facebook framed the problem concretely (Quiñonero Candela, 2017). Researchers wanted PyTorch’s flexibility; Facebook’s production pipelines ran Caffe2, which was built for performance and mobile devices. ONNX was to let models trained in PyTorch be deployed on Caffe2, and Caffe2, PyTorch and Microsoft’s Cognitive Toolkit would all support it.

In computer vision, ONNX is the hand-over point between training and inference: a convolutional network for classification, detection or segmentation is trained in PyTorch, exported once, checked, and then executed by whatever runtime suits the target device. The IR specification is neutral about how: an implementation may interpret the model, generate code from it or implement it in hardware.

The base format targets neural networks; a second variant, ONNX-ML, adds data types and operators for classical machine learning (tree ensembles, linear classifiers and similar models). Up to IR version 6, ONNX covered inference only. IR version 7 added an optional training extension, but the README of the main repository still describes inference as the project’s current focus.

Core Capabilities

The model file

An ONNX file is a serialized Protocol Buffers message of type ModelProto. The .proto files in the repository define its syntax, and docs/IR.md defines its semantics; where the two conflict, the IR document is normative. The model wraps one graph with metadata: ir_version (the version of the format the file follows), opset_import (the operator sets the graph uses), producer_name and producer_version (the tool that wrote it), an optional model_version, documentation strings, and free-form key–value pairs in metadata_props.

The graph (GraphProto) is the executable part. It contains:

• Nodes. Each node calls one operator, named by op_type and a domain, with a list of input names, a list of output names and named attributes. Inputs carry values computed at run time; attributes carry constants fixed when the graph is built, such as a convolution’s kernel shape, strides and padding. The node list must be in topological order and free of cycles, and every node output must have a unique name, so the graph is in single static assignment form. Edges are implicit: a node consumes a value by naming another node’s output.
• Initializers. Named constant tensors, which hold the trained weights. An initializer whose name also appears among the graph inputs is only a default value that a runtime may let the caller override; constants not meant to be overridden appear only in the initializer list.
• Inputs and outputs. The top-level graph must declare the element type and the rank of every input and output, though not necessarily each dimension’s size.

Tensor element types range from bool and integers of 2 to 64 bits to float16, bfloat16, float32 and float64, several 8-, 6- and 4-bit floating-point formats used for low-precision inference, complex numbers and strings (IR specification at 1.23.2). A static shape is a list of dimensions, each either a fixed integer (dim_value), a named symbol (dim_param) or unknown. A symbol such as "batch" declares a dimension whose size is fixed only at run time; dimension variables are not scoped, so the same symbol on two inputs means the two sizes must be equal.

Large models have a practical limit: a single protocol buffer message cannot exceed 2 GB, a limit the ONNX External Data document, the PyTorch exporter documentation and the OpenVINO documentation all mention. Tensors can therefore be stored in separate files that the model refers to by relative path, offset and length. The format also has functions, operators whose meaning is defined by a body of other operators, so that a runtime without a dedicated kernel can expand the function into its body and execute that.

Operators and operator sets

The ONNX operators are the shared vocabulary that makes the format portable. An operator is identified by a domain, a name and a since_version, and its documentation fixes its inputs, outputs, attributes, permitted types and semantics; docs/Operators.md lists all of them. Operators are grouped into operator sets (opsets). Each opset version is an immutable snapshot of a domain’s operators: when any operator is added, removed or changed in meaning, the domain’s opset version is increased, and the changed operator gets a new since_version (ONNX Versioning document). Adding an attribute, adding or reordering inputs, supporting new input types and even implicitly supporting broadcasting all count as breaking changes that need a new operator version.

A model lists the opsets it needs in opset_import, for example ("", 20) for version 20 of the default domain, written ai.onnx in documentation, and every node is bound to the newest version of its operator that is not newer than the imported opset. ReduceMean illustrates why the binding matters. Up to version 13 its reduction axes were an attribute; from version 18 they are an optional input tensor. A graph stamped with opset 20 must therefore supply the axes as a tensor, which is why the example below has an initializer that holds only two integers.

In ONNX 1.23.2 the default domain’s opset 28 lists 155 primitive operators and 48 functions; two of them, Scatter and Upsample, are marked deprecated, and the documentation sends users of Scatter to ScatterElements. The ai.onnx.ml domain holds the classical machine-learning operators. Other domains belong to vendors, who can define extension operators under their own reverse-DNS domain names; a model that uses them runs only where those operators are implemented, as with the “contrib operators” that ONNX Runtime provides for operations outside the standard.

Three version numbers

ONNX versions three things independently (Versioning document): the IR version of the file format, which changes rarely (14 in ONNX 1.23); the opset version of the operator semantics, which changes with almost every release (28 in ONNX 1.23); and the model version, the author’s own number, for which the specification only recommends conventions. The library release, 1.23.2, identifies the software rather than the format. A consumer must understand the IR version and every imported opset; the IR specification asks implementations to support all operators of an imported set or reject the model, the failure the pitfalls example below runs into.

Tooling in the onnx package

The onnx Python package reads and writes model files and provides utilities that work on the format alone:

• onnx.checker.check_model validates a model against the specification, from required fields such as ir_version and opset_import to the rules for the graph and its nodes, and raises an exception on the first violation. With full_check=True it also runs shape inference.
• Shape inference (onnx.shape_inference) propagates types and shapes through the graph, including symbolic dimensions since ONNX 1.10, and fills in value_info for intermediate values.
• The version converter (onnx.version_converter.convert_version) rewrites a model from one opset version to another by applying an adapter for each changed operator, and raises an error when a conversion is not supported.
• Helpers (onnx.helper) build nodes, graphs and models directly from Python.

The IR specification also suggests inspecting a file with protoc or a graphical viewer such as Netron, which the PyTorch exporter documentation recommends as well.

Exporting Models to ONNX

From PyTorch

torch.onnx.export converts a PyTorch module into an ONNX model. PyTorch has had two exporters, and the default changed recently:

• The torch.export-based exporter (dynamo=True) has been the default since PyTorch 2.9 (October 2025). It first captures the model with torch.export.export, trying a non-strict and then a strict capture, which produces a graph of functional ATen operators with Python control flow removed and the conditions on input shapes recorded. It then translates each ATen operator into ONNX operators, using translation functions written in ONNX Script, a subset of Python for authoring ONNX functions and models that lives in the separate onnxscript package (its torch_lib module in 0.7.2). Finally, by default (optimize=True), it runs ONNX Script’s optimizer, which folds constants and removes redundant nodes. The call returns a torch.onnx.ONNXProgram, whose save() method writes the file. The exporter needs the onnx and onnxscript packages.
• The legacy TorchScript-based exporter (dynamo=False) traces the model with TorchScript and maps each operator through per-opset “symbolic” functions. PyTorch 2.8 deprecated it, and in 2.14.1 calling it issues a DeprecationWarning. PyTorch 2.11 removed the fallback option, which had silently switched from the new exporter to the old one on failure; the release notes explain that the fallback hid errors and produced different models.

Several arguments matter for vision models. dynamic_shapes declares which input dimensions may vary, such as the batch size or the image height and width; without it, the tracer records the exact shapes of the example inputs. The older dynamic_axes argument was deprecated in PyTorch 2.10 in favour of it. opset_version selects the default-domain opset; in 2.14.1 both exporters default to opset 20 (2.9 release notes and source code). The legacy exporter supports opsets up to 20 and warns when asked for a higher one; the new exporter’s translations start at opset 18, a lower request is met by converting the exported model, and the 2.9 release notes suggest opset 23 for the newest operators. external_data (default True) stores weights in a separate file, which large models need because of the 2 GB limit. When an export fails, the documentation suggests running it again with report=True, which writes a Markdown report on the failure, and verify=True runs the exported model in ONNX Runtime as a check. Each exported node carries metadata naming the PyTorch module and source line it came from, which helps when a graph has to be traced back to the code.

PyTorch 0.3.0 (December 2017) introduced torch.onnx for “ConvNet-like and RNN-like (static graphs)” models, in the words of its release notes. The 2017 announcement explained why: a PyTorch model is a program rather than an explicit graph, so the exporter traced the operations executed for one example input, which fails for programs with dynamic control flow (Quiñonero Candela, 2017). The torch.export-based exporter instead refuses what it cannot capture soundly, as the pitfalls example below shows.

From other frameworks

• TensorFlow and Keras. tf2onnx, maintained in the ONNX organization, converts TensorFlow 2.x, Keras, TensorFlow.js and TensorFlow Lite models from the command line or Python. As of version 1.17.0 (March 2026), its README states that it is tested with TensorFlow 2.13 to 2.15 and opsets 14 to 18, uses opset 15 by default, and is looking for a new maintainer. It also warns that TensorFlow has many more operations than ONNX, so mapping a model sometimes fails. Keras 3 can write ONNX directly with model.export(path, format="onnx"), optionally with an opset_version.
• scikit-learn. sklearn-onnx converts scikit-learn models and pipelines into ONNX-ML graphs; its README gives opset 21 as the last supported opset.

Minimal Example

The script trains a small CNN briefly on rendered 32×32 images of disks, squares and triangles, so that the outputs mean something, then exports it with the default exporter and a symbolic batch dimension, inspects and checks the file, and runs it in ONNX Runtime on new synthetic images, comparing the outputs with PyTorch’s. The CNN has two convolution, batch normalization and pooling stages followed by global average pooling and a linear layer. The model is exported twice, with and without the exporter’s optimization pass, to show what that pass changes. It uses PyTorch 2.14.1, ONNX 1.23.2, ONNX Script 0.7.2 and ONNX Runtime 1.31.1 on a CPU with one thread, takes about 3 seconds on a laptop, and printed identical output on two runs.

import collections
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
import onnx
import onnxruntime as ort

torch.manual_seed(0)
torch.set_num_threads(1)
print(f"torch {torch.__version__}, onnx {onnx.__version__}, onnxruntime {ort.__version__}")

# Rendered 32x32 grayscale shapes with exact labels: 0 disk, 1 square, 2 triangle.
def render(kind, rng, size=32):
    yy, xx = np.mgrid[0:size, 0:size] + 0.5
    r = rng.uniform(5, 10)
    cx, cy = rng.uniform(r + 1, size - r - 1, 2)
    if kind == 0:
        mask = (xx - cx) ** 2 + (yy - cy) ** 2 <= r ** 2
    elif kind == 1:
        mask = (abs(xx - cx) <= r) & (abs(yy - cy) <= r)
    else:
        mask = (yy >= cy - r) & (yy <= cy + r) & (abs(xx - cx) <= (yy - (cy - r)) / 2)
    img = np.where(mask, rng.uniform(0.6, 1.0), rng.uniform(0.0, 0.3))
    return (img + rng.normal(0, 0.1, img.shape)).astype(np.float32)

def make_set(n, seed):
    rng = np.random.default_rng(seed)
    y = np.arange(n) % 3
    x = np.stack([render(k, rng) for k in y])[:, None]   # (N, 1, 32, 32)
    return torch.from_numpy(x), torch.from_numpy(y)

class ShapesCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1, self.bn1 = nn.Conv2d(1, 8, 3, padding=1), nn.BatchNorm2d(8)
        self.conv2, self.bn2 = nn.Conv2d(8, 16, 3, padding=1), nn.BatchNorm2d(16)
        self.fc = nn.Linear(16, 3)
    def forward(self, x):
        x = F.max_pool2d(F.relu(self.bn1(self.conv1(x))), 2)
        x = F.max_pool2d(F.relu(self.bn2(self.conv2(x))), 2)
        return self.fc(x.mean(dim=(2, 3)))             # global average pooling

# Train briefly so that the outputs mean something.
x_train, y_train = make_set(1200, seed=1)
x_test, y_test = make_set(300, seed=2)
model = ShapesCNN()
opt = torch.optim.Adam(model.parameters(), lr=1e-2)
perm = torch.randperm(len(x_train), generator=torch.Generator().manual_seed(0))
for epoch in range(4):
    model.train()
    for i in range(0, len(x_train), 50):
        idx = perm[i:i + 50]
        loss = F.cross_entropy(model(x_train[idx]), y_train[idx])
        opt.zero_grad(); loss.backward(); opt.step()
model.eval()
with torch.no_grad():
    acc = (model(x_test).argmax(1) == y_test).float().mean().item()
print(f"PyTorch test accuracy on {len(x_test)} rendered shapes: {acc:.3f}")

# Export with a dynamic batch axis; the default exporter (dynamo=True) is used.
batch = torch.export.Dim("batch")
for optimize, path in ((True, "shapes_cnn.onnx"), (False, "shapes_cnn_unopt.onnx")):
    prog = torch.onnx.export(model, (x_test[:2],), input_names=["image"],
                             output_names=["logits"], dynamic_shapes={"x": {0: batch}},
                             optimize=optimize, verbose=False)
    prog.save(path, external_data=False)           # weights inside the .onnx file
    ops = collections.Counter(n.op_type for n in prog.model_proto.graph.node)
    print(f"optimize={optimize}: {sum(ops.values())} nodes {dict(ops)}")

# Inspect and validate the file.
m = onnx.load("shapes_cnn.onnx")
onnx.checker.check_model(m, full_check=True)           # spec rules + shape inference
print(f"checker: passed; IR version {m.ir_version}; opsets "
      f"{[(o.domain or 'ai.onnx', o.version) for o in m.opset_import]}; "
      f"producer {m.producer_name} {m.producer_version}")
def shape(v):
    return [d.dim_param or d.dim_value for d in v.type.tensor_type.shape.dim]
for v in m.graph.input:
    print(f"input  {v.name}: {onnx.TensorProto.DataType.Name(v.type.tensor_type.elem_type)} {shape(v)}")
for v in m.graph.output:
    print(f"output {v.name}: {onnx.TensorProto.DataType.Name(v.type.tensor_type.elem_type)} {shape(v)}")
n_par = sum(int(np.prod(t.dims)) for t in m.graph.initializer)
print(f"{len(m.graph.initializer)} initializers, {n_par} values; "
      f"PyTorch parameters + buffers: "
      f"{sum(p.numel() for p in model.parameters())} + "
      f"{sum(b.numel() for n, b in model.named_buffers() if 'num_batches' not in n)}")

# Run both files in ONNX Runtime and compare with PyTorch on new synthetic images.
def session(path, level=ort.GraphOptimizationLevel.ORT_ENABLE_ALL):
    so = ort.SessionOptions()
    so.intra_op_num_threads = 1                    # fixed summation order
    so.graph_optimization_level = level            # ORT_ENABLE_ALL is the default
    return ort.InferenceSession(path, so, providers=["CPUExecutionProvider"])
sess = session("shapes_cnn.onnx")
sess_unopt = session("shapes_cnn_unopt.onnx", ort.GraphOptimizationLevel.ORT_DISABLE_ALL)
x_new, _ = make_set(64, seed=3)
for n in (1, 7, 64):
    x = x_new[:n]
    with torch.no_grad():
        ref = model(x).numpy()
    out = sess.run(["logits"], {"image": x.numpy()})[0]
    unopt = sess_unopt.run(["logits"], {"image": x.numpy()})[0]
    print(f"batch {n:2d}: output {out.shape}, max |ORT - PyTorch| {np.abs(out - ref).max():.1e}"
          f" (no graph rewrites {np.abs(unopt - ref).max():.1e}), "
          f"largest |logit| {np.abs(ref).max():.1f}, "
          f"same classes: {(out.argmax(1) == ref.argmax(1)).all()}")
    np.testing.assert_allclose(out, ref, rtol=1e-4, atol=1e-5)
print("parity check passed (rtol 1e-4, atol 1e-5)")

Output:

torch 2.14.1, onnx 1.23.2, onnxruntime 1.31.1
PyTorch test accuracy on 300 rendered shapes: 0.963
optimize=True: 8 nodes {'Conv': 2, 'Relu': 2, 'MaxPool': 2, 'ReduceMean': 1, 'Gemm': 1}
optimize=False: 13 nodes {'Conv': 2, 'BatchNormalization': 2, 'Relu': 2, 'MaxPool': 2, 'Constant': 2, 'Reshape': 1, 'ReduceMean': 1, 'Gemm': 1}
checker: passed; IR version 10; opsets [('ai.onnx', 20)]; producer pytorch 2.14.1
input  image: FLOAT ['batch', 1, 32, 32]
output logits: FLOAT ['batch', 3]
7 initializers, 1301 values; PyTorch parameters + buffers: 1347 + 48
batch  1: output (1, 3), max |ORT - PyTorch| 1.5e-07 (no graph rewrites 4.8e-07), largest |logit| 2.0, same classes: True
batch  7: output (7, 3), max |ORT - PyTorch| 1.4e-06 (no graph rewrites 9.5e-07), largest |logit| 4.8, same classes: True
batch 64: output (64, 3), max |ORT - PyTorch| 3.0e-06 (no graph rewrites 2.6e-06), largest |logit| 5.1, same classes: True
parity check passed (rtol 1e-4, atol 1e-5)

The exported graph is not a copy of the module’s structure. Without optimization the exporter produced 13 nodes, one per ATen operation plus helper Constant and Reshape nodes. The optimization pass folded each BatchNormalization into the weights and bias of the convolution before it, which is exact algebra for a model in evaluation mode, and removed the helpers, leaving 8 nodes. The parameter counts confirm the folding: PyTorch holds 1,347 parameters and 48 running statistics, while the file holds the 1,299 weights and biases of the three layers plus the two-element axes tensor that ReduceMean takes as an input from opset 18. The 48 batch-normalization parameters and 48 statistics have been absorbed into the convolutions. The PyTorch exporter writes opset 20 by default, the version released with ONNX 1.15, but stamps IR version 10. The two numbers are independent, and a runtime has to support both.

The batch dimension is the symbol batch on the input and propagates to the output, so one file serves batches of 1, 7 and 64 images. Across all three, ONNX Runtime agrees with PyTorch to within $3 \times 10^{-6}$ on logits as large as 5.1, about $6 \times 10^{-7}$ of the largest logit, and every predicted class matches. With all graph rewrites switched off, both in the exporter and in ONNX Runtime, the differences are of the same size. The folding is therefore not their source. In our reading they come from float32 rounding: float32 has a relative precision of about $1.2 \times 10^{-7}$, and the two libraries’ convolution and matrix kernels add the up to 72 products behind each output in different orders. The maximum grows with the batch, presumably because it is taken over more outputs. The tolerances in assert_allclose are our choice: the absolute term alone is more than three times the largest observed difference, and the relative term adds more room on large logits. In our view, a float16 or int8 model needs tolerances set from its own precision and is better compared on task accuracy, as the quantization article does.

Conversion Pitfalls

Most export problems fall into a few classes: control flow that depends on data, shapes fixed at export time, operators the exporter or the runtime does not support, and opset versions newer than the runtime accepts. The second script triggers three of them, in four parts; unsupported operators are discussed without code. It uses the same versions, also ran in about 2 seconds, and printed identical output twice. Besides what it prints, the exporter writes diagnostic logs to standard error, which are not shown.

import logging
import re
import warnings
import numpy as np
import torch
import onnx
import onnxruntime as ort
from onnx import version_converter

torch.set_num_threads(1)
logging.getLogger("torch").setLevel(logging.ERROR)   # fewer exporter log messages
cpu = ["CPUExecutionProvider"]
def lines(e):                              # error text without colour codes
    return [s.strip() for s in re.sub(r"\x1b\[[0-9;]*m", "", str(e)).splitlines() if s.strip()]

# A preprocessing step with data-dependent control flow: invert dark images.
class AutoInvert(torch.nn.Module):
    def forward(self, x):
        if x.mean() < 0.5:                 # Python branch on a tensor value
            return 1.0 - x
        return x

dark = torch.full((1, 1, 8, 8), 0.2)       # mean 0.2: inverted to 0.8
bright = torch.full((1, 1, 8, 8), 0.8)     # mean 0.8: passed through

# 1. Legacy TorchScript exporter: tracing records only the branch taken.
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    torch.onnx.export(AutoInvert().eval(), (dark,), "invert_legacy.onnx",
                      dynamo=False, input_names=["x"])
print("legacy exporter warnings:", sorted({w.category.__name__ for w in caught}))
s = ort.InferenceSession("invert_legacy.onnx", providers=cpu)
print(f"bright image: PyTorch {AutoInvert()(bright)[0, 0, 0, 0]:.1f}, "
      f"legacy ONNX {s.run(None, {'x': bright.numpy()})[0][0, 0, 0, 0]:.1f}")

# 2. Default exporter: torch.export refuses the data-dependent branch ...
try:
    torch.onnx.export(AutoInvert().eval(), (dark,), verbose=False)
except Exception as e:
    print(f"default exporter: {type(e).__name__}: {lines(e)[0].split('. ')[0]}; "
          f"cause: {type(e.__cause__).__name__}")

# ... and exports it once the branch is written with torch.cond.
class AutoInvertCond(torch.nn.Module):
    def forward(self, x):
        # a branch may not return its input itself (no aliasing), hence clone()
        return torch.cond(x.mean() < 0.5, lambda t: 1.0 - t, lambda t: t.clone(), (x,))

prog = torch.onnx.export(AutoInvertCond().eval(), (dark,), input_names=["x"], verbose=False)
prog.save("invert_cond.onnx", external_data=False)
m = onnx.load("invert_cond.onnx")
s = ort.InferenceSession("invert_cond.onnx", providers=cpu)
print("torch.cond export: nodes", [n.op_type for n in m.graph.node],
      "| dark ->", f"{s.run(None, {'x': dark.numpy()})[0][0, 0, 0, 0]:.1f}",
      "| bright ->", f"{s.run(None, {'x': bright.numpy()})[0][0, 0, 0, 0]:.1f}")

# 3. Static shapes: without dynamic_shapes the batch size is fixed at export.
conv = torch.nn.Conv2d(1, 4, 3).eval()
torch.onnx.export(conv, (torch.zeros(1, 1, 32, 32),), "conv_static.onnx",
                  input_names=["image"], verbose=False, external_data=False)
v = onnx.load("conv_static.onnx").graph.input[0]
print("static input shape:", [d.dim_value for d in v.type.tensor_type.shape.dim])
s = ort.InferenceSession("conv_static.onnx", providers=cpu)
try:
    s.run(None, {"image": np.zeros((4, 1, 32, 32), np.float32)})
except Exception as e:
    print(f"batch of 4: {type(e).__name__}: {lines(e)[1]}")

# 4. Opset mismatch: a valid model can use a newer opset than a runtime supports.
m20 = onnx.load("conv_static.onnx")
for target in (26, 27, 28):
    mt = version_converter.convert_version(m20, target)
    onnx.checker.check_model(mt)                       # valid for onnx 1.23.2
    try:
        ort.InferenceSession(mt.SerializeToString(), providers=cpu)
        print(f"opset {target}: loads in ONNX Runtime {ort.__version__}")
    except Exception as e:
        print(f"opset {target}: {type(e).__name__}: {str(e).strip().split('. ')[-1]}")

Output:

legacy exporter warnings: ['DeprecationWarning', 'TracerWarning']
bright image: PyTorch 0.8, legacy ONNX 0.2
default exporter: TorchExportError: Failed to export the model with torch.export; cause: GuardOnDataDependentSymNode
torch.cond export: nodes ['ReduceMean', 'Squeeze', 'Less', 'If'] | dark -> 0.8 | bright -> 0.8
static input shape: [1, 1, 32, 32]
batch of 4: InvalidArgument: index: 0 Got: 4 Expected: 1
opset 26: loads in ONNX Runtime 1.31.1
opset 27: Fail: Current official support for domain ai.onnx is till opset 26.
opset 28: Fail: Current official support for domain ai.onnx is till opset 26.

Data-dependent control flow. The legacy exporter traced the module with a dark example image, recorded only the inversion branch, and wrote a graph that inverts every image, so a bright image comes back as 0.2 where PyTorch returns 0.8. Apart from the deprecation notice, the TracerWarning it raised is the only sign of the problem, and the model still loads and runs. This is the limitation the 2017 announcement described. The default exporter refuses the same module: torch.export cannot decide the branch without a concrete value (the GuardOnDataDependentSymNode error) and stops instead of guessing. Rewriting the branch with torch.cond makes it part of the graph. The export contains an ONNX If node with both branches as subgraphs, and the file now handles both images correctly. The PyTorch documentation marks torch.cond as a prototype feature with limited support for input and output types.

Fixed shapes. Exported without dynamic_shapes, the convolution’s input has the fixed shape [1, 1, 32, 32], and ONNX Runtime rejects a batch of four images. A vision model can be exported with a dynamic batch dimension, and a fully convolutional network with dynamic height and width as well. In our reading, code that turns a shape into data, such as reshaping by a size computed from the input, is where dynamic dimensions are most likely to fail to export or to give graphs that work only at the example size; running the exported file at several sizes, as the minimal example does with three batch sizes, checks this directly.

Unsupported operators. An exporter can lack a translation for an ATen operator, and a runtime can lack a kernel for an ONNX operator or support it only in part. TensorRT 11.4’s operator matrix, for example, supports Resize only with some coordinate-transformation modes and without antialiasing, and Loop only when the scan output length is not dynamic. The PyTorch exporter’s custom_translation_table lets the user supply an ONNX Script function for a missing translation. A missing kernel can be replaced by a composition of standard operators, or provided as a custom operator in a vendor domain, which makes the model depend on runtimes that implement that domain.

Opset mismatch. The version converter produced valid opset 27 and opset 28 models, and the ONNX 1.23.2 checker accepted both, but ONNX Runtime 1.31.1 refused them with a message that calls both opsets under development and says its official support ends at opset 26. Its compatibility page says that ONNX Runtime supports all opsets of the latest released ONNX version, and the 1.31.0 release notes say that its ONNX dependency remains at 1.22.0, which introduced opset 27, after an attempt to integrate ONNX 1.23 was reverted. Neither statement would lead a reader to expect the limit at opset 26; we report it as observed with the installed package and do not know its cause. Other runtimes have their own ranges: the TensorRT 11.4 matrix covers opsets 9 to 24. The safe practice, which the PyTorch documentation also recommends, is to set opset_version to a version the target runtime supports rather than to the newest one.

Numerical parity. A model that loads is not yet a correct conversion; outputs must be compared with the original’s within a tolerance, as in the minimal example. PyTorch 2.10 added torch.onnx.testing.assert_onnx_program for this, and the exporter’s verify=True option runs the exported model in ONNX Runtime. In our reasoning, rendered or real images exercise the value ranges the model will see better than random noise does, and differences far above rounding level point to a semantic mismatch: an operator attribute translated differently, such as a resize or pooling mode, a model exported in training mode (the exporter warns about this), or preprocessing that differs between training and deployment.

Ecosystem and Integrations

ONNX Runtime is Microsoft’s inference engine for ONNX models, released under the MIT License. It loads a model, applies graph optimizations in three levels (basic rewrites such as constant folding and Conv–BatchNorm fusion, extended fusions, and layout optimizations), all enabled by default, and partitions the graph among execution providers. Each provider is an interface to a hardware library. Through a GetCapability() call, each provider claims the nodes or subgraphs it can run, in the priority order the user gives, and the default CPU provider runs the rest. As of October 2026 its documentation lists providers for NVIDIA CUDA and TensorRT, Intel OpenVINO and oneDNN, Windows DirectML, Qualcomm QNN, Android NNAPI, Apple Core ML, XNNPACK, AMD MIGraphX and Vitis AI, and WebGPU, among others; some are marked as previews. The Arm Compute Library provider was deprecated in 1.31.0. Graph optimization can be run once offline and the optimized model saved, which the documentation recommends only for the same hardware and options the model will run with. The installed 1.31.1 package used here offers the CPU, Core ML and Azure providers.

NVIDIA TensorRT runs models on NVIDIA GPUs and imports ONNX files through its ONNX parser, developed in the onnx/onnx-tensorrt repository. For TensorRT 11.4 (October 2026), the parser’s operator matrix covers opsets 9 to 24 and lists, for each operator, the supported types and restrictions. The matrix notes that TensorRT supports INT8, INT4, FP8 and FP4 only through quantization from a higher-precision type; quantized ONNX graphs carry the scales in explicit QuantizeLinear and DequantizeLinear nodes, as the quantization article describes.

Intel OpenVINO (release 2026.4.1, 1 October 2026) reads ONNX files directly with read_model or compile_model, or converts them into its own IR with convert_model or the ovc tool, which its documentation recommends when model loading time matters. It handles models with external data files.

OpenCV loads ONNX models in its dnn module and runs them without a deep-learning framework; in OpenCV 5.0 ONNX is the route for models from frameworks whose own importers were removed, as the OpenCV article describes.

Model sources. The ONNX repository’s README points to an onnx-community organization on Hugging Face for pretrained models in the format. Graphs can also be written directly in Python with ONNX Script or onnx.helper, and ONNX Script’s rewriter can replace patterns in an existing graph.

ONNX’s role is to give all of these runtimes one input; which one runs it depends on the target device, as described in deploying vision models and edge inference.

Alternatives

The alternatives are other ways of carrying a trained model out of its framework, described here as their own documentation describes them, without ranking.

• Framework-native formats. PyTorch’s torch.export produces an ExportedProgram that its own deployment tools consume (AOTInductor on servers, ExecuTorch on edge devices); TorchScript, the older format, is deprecated. TensorFlow uses the SavedModel format and, for devices, LiteRT. These are read by their own framework’s tools; the PyTorch and TensorFlow articles describe them.
• Runtime-specific formats. OpenVINO’s IR, an .xml graph with a .bin file of weights, is produced and consumed by OpenVINO, and its documentation describes converting ONNX files into it; such formats complement ONNX rather than replace it.
• Multi-format exporters. Keras 3’s Model.export writes TensorFlow SavedModel, ONNX, OpenVINO, LiteRT and PyTorch formats from the same model, which treats ONNX as one target among several.

ONNX’s distinguishing property is that it is a vendor-neutral specification with versioned operator semantics, written by many frameworks and read by many runtimes, rather than one tool’s internal format.

Versions and Compatibility

Facebook and Microsoft introduced the format on 7 September 2017, with support announced for Caffe2, PyTorch and the Cognitive Toolkit (onnx.ai news; Quiñonero Candela, 2017). On 10 October 2017, AMD, Arm, Huawei, IBM, Intel and Qualcomm announced support, and Amazon Web Services joined on 16 November 2017, adding Apache MXNet support. ONNX 1.0 followed on 6 December 2017, announced as production-ready by AWS, Facebook and Microsoft. The project’s open governance was announced in March 2018: a steering committee, special interest groups and working groups, modelled on Kubernetes, with steering committee seats filled by annual community election after the first year (ONNX open governance document). On 14 November 2019, ONNX joined the Linux Foundation’s LF AI Foundation, now LF AI & Data, as a graduated project (onnx.ai news).

The Version enumeration in onnx.proto dates the IR versions; the release table in the Versioning document gives the IR and opset versions of each release; GitHub gives the release dates:

ONNX release	Date	IR version	ai.onnx opset	Notable for this article
1.0	Dec 2017	3	1	First stable release
1.2	May 2018	3	7	Oldest opset ONNX Runtime supports
1.6.0	Sep 2019	6	11	Sequences and maps as types
1.7.0	May 2020	7	12	Training extension (IR 7)
1.10.0	Jul 2021	8	15	Optional type; symbolic shape inference
1.15.0	Oct 2023	9	20	Opset of the PyTorch 2.14 default export
1.19.0	Aug 2025	12	24	Newest opset in the TensorRT 11.4 parser
1.21.0	Mar 2026	13	26	Newest opset ONNX Runtime 1.31.1 accepted
1.22.0	Jun 2026	13	27	ONNX dependency of ONNX Runtime 1.31
1.23.0	Sep 2026	14	28	IR 14, opset 28
1.23.2	6 Oct 2026	14	28	Version used in this article

The 1.2 row gives the date of 1.2.1 (May 2018), the release that ONNX Runtime’s compatibility page names as the oldest it supports, with opset 7. ONNX 1.23.0 added IR version 14 with two 6-bit floating-point types and the SwiGLU operator as a function; 1.23.2 is a bug-fix release. From 1.20 (December 2025) to 1.23 (September 2026), a minor release appeared every two and a half to four months, which is our reading of the release dates rather than a stated policy.

Compatibility points as of October 2026:

• Opset ranges. The opset a producer writes must lie in the range the target accepts, ranges given above for PyTorch, tf2onnx, ONNX Runtime and TensorRT; the version converter can move a model between opsets when adapters exist for its operators.
• Backward compatibility of runtimes. ONNX Runtime’s compatibility page states that newer versions run all models that worked with earlier ones. The specification’s versioning rules make this possible, because an operator version, once published as stable, never changes its meaning.
• Licences. ONNX and its specification are under the Apache License 2.0, as are the tf2onnx, sklearn-onnx and onnx-tensorrt repositories of the ONNX organization (by their licence identifiers); ONNX Runtime and ONNX Script are under the MIT License.

Limitations

• Inference first. The format can describe training (IR 7 and later), but the repository describes its focus as inference, and the export paths described here produce inference graphs of models in evaluation mode.
• The operator set is a lowest common denominator. A model can be exported only if each of its operations maps onto standard operators, or onto a custom domain that the target runtime implements. Operators for newer architectures are added in later opsets (an Attention operator first appears in opset 23, for example); before that, or for a runtime that does not yet support them, such layers are exported as compositions of smaller operators or as vendor extensions.
• Support is per runtime and per operator. A model that runs in one runtime may not load in another, which implements a different opset range or only part of an operator; each target’s operator matrix has to be checked.
• Static graphs. Control flow must be written with graph operators (If, Loop, Scan) and dynamic shapes declared at export.
• Numerical results differ between runtimes. The specification defines what an operator computes and leaves its implementation to the runtime, so, in our reading, outputs can be expected to agree only within rounding, and less closely in reduced precision.
• Files over 2 GB must keep their weights in external data files, which travel with the model.
• Versions move quickly. Opsets are added several times a year, the PyTorch exporter changed its default in 2.9, and runtimes trail the newest opset; pinning the ONNX, exporter and runtime versions together keeps an export reproducible.