Vision Systems & Engineering
Robot Operating System (ROS)
The open-source robotics middleware ROS 2 as of its Jazzy distribution, covering nodes, topics, services and actions, DDS and the RMW layer, quality of service, discovery, launch and composition, tf2 and the REP 105 frames, the image and camera-info messages, message_filters, bags and distributions, with an example that ports the Python time synchronizers.
intermediate
The Robot Operating System (ROS) is not an operating system. It is open-source middleware and a set of libraries and tools for building robot software out of many cooperating programs. Each program, a node, publishes and subscribes to named, typed streams of messages. Cameras, depth sensors, detectors, trackers and planners all exchange data this way. This article covers ROS 2, the line that the ROS 2 documentation, the REP 2000 release schedule and the packages read here describe. The original ROS, now called ROS 1, appears only where those sources compare the two. The code is developed in public GitHub repositories, package by package, each under its own licence: the message_filters licence names Willow Garage and the Open Source Robotics Foundation as copyright holders.
For a vision engineer, ROS is the layer that carries frames and their metadata between processes and machines. Images travel as sensor_msgs/Image with a capture timestamp and a frame name; calibration travels beside them as CameraInfo; tf2 relates the camera’s frame to the robot and the world; and message_filters groups frames from several sensors that describe the same moment. Robot perception covers what robots do with such data. Multi-camera systems covers rig-level synchronization and calibration, and rotation representations covers REP 103’s axes, the _optical camera frames and the quaternion order of ROS messages. Facts below come from the Jazzy branches of the ROS 2 documentation (commit 929e37c), message_filters (05a6b5f), common_interfaces (a941f14) and rosbag2 (12b5e8b), and from the REP repository at commit 11ca24a, unless another source is named.
Purpose
A robot is a distributed system even when it fits in one chassis. A driver reads the camera, another the IMU, a third the wheels; perception, mapping, planning and control run at their own rates, often in different processes and sometimes on different computers. ROS gives these programs:
- a common way to find each other and exchange data (topics, services, actions), with automatic discovery;
- typed message definitions with agreed semantics, such as SI units and the image and camera-calibration messages;
- conventions for coordinate frames and their relations over time (REP 103, REP 105, tf2);
- tools to record, replay and inspect the traffic (
ros2 bag, and the command-line tools); - a release process, with distributions supported for fixed periods on named platforms (REP 2000).
The Topics page describes the result as “a strongly-typed, anonymous publish/subscribe system”: a subscriber need not know which publisher produced a message, so components can be swapped without touching the rest of the system.
Core Capabilities
Nodes, topics, services and actions
A node is “a participant in the ROS 2 graph” that uses a client library (rclcpp for C++, rclpy for Python) to communicate. Nodes can talk within one process, across processes or across machines, and the Nodes page recommends that each node “do one logical thing”. In C++, a node can create sub-nodes that share its context and graph identity under a sub-namespace, avoiding the discovery overhead of separate nodes.
ROS applications communicate through three kinds of interface, each described in a small interface definition language (.msg, .srv and .action files):
- Topics are for continuous data streams such as sensor data and robot state. Any number of publishers and subscribers may share a topic, and every subscriber receives what any publisher sends. Message fields are typed, and the core types carry fixed semantics; the Topics page’s example is the IMU message’s angular velocity, specified in radians per second.
- Services are remote procedure calls that should return quickly, because the client is usually waiting; a
.srvfile holds a request and a response separated by---. There should be one server per service name; which server answers is undefined if there are several. - Actions are long-running calls with feedback that can be cancelled or pre-empted, such as driving to a waypoint. An
.actionfile holds a request, a result and a feedback part, each separated by---.
Nodes also expose parameters that change their behaviour at run time.
DDS and the RMW layer
ROS 2 is built on DDS and its wire protocol RTPS, which provide discovery, serialization and transport. DDS is an industry standard with several implementations, and ROS 2 reaches them through an abstract ROS middleware interface, rmw, implemented once per vendor. The middleware page of the Jazzy documentation lists:
| Product | Licence | RMW implementation | Status |
|---|---|---|---|
| eProsima Fast DDS | Apache 2 | rmw_fastrtps_cpp |
Full support, default RMW, packaged with binary releases |
| Eclipse Cyclone DDS | Eclipse Public License v2.0 | rmw_cyclonedds_cpp |
Full support, packaged with binary releases |
| RTI Connext DDS | commercial, research | rmw_connextdds |
Full support; Connext installed separately |
| GurumNetworks GurumDDS | commercial | rmw_gurumdds_cpp |
Community support; GurumDDS installed separately |
Fast DDS is the default when installed; otherwise the implementation whose package name comes first alphabetically is used. Nodes on different RMW implementations can often communicate, but this is not guaranteed, and the page lists combinations that fail for WString; it suggests that every part of a distributed system use the same ROS version and the same RMW implementation.
Quality of service
Each publisher and subscription carries a QoS profile made of policies:
- history: keep last samples (with the depth as queue size) or keep all;
- reliability: best effort, which may lose samples, or reliable, which may retry;
- durability: volatile, or transient local, in which the publisher keeps samples for late-joining subscriptions;
- deadline, lifespan, liveliness and lease duration.
The defaults for publishers and subscriptions are keep last with depth 10, reliable, volatile and system-default liveliness, chosen to resemble ROS 1. The sensor data profile uses best effort and a smaller queue, because timely readings matter more for sensor data than receiving every one. The page notes that even reliable delivery in ROS 2 runs over UDP.
Compatibility follows a request-versus-offer model: a connection is made only if no policy the subscription requests is more stringent than what the publisher offers. A best-effort publisher and a reliable subscription are incompatible; a reliable publisher serves a best-effort subscription. A volatile publisher and a transient-local subscription do not communicate. When a pair is incompatible, no messages pass, and both sides can register callbacks for “incompatible QoS” events. In our reading, this is the first thing to check when a camera topic is visible but a subscriber receives nothing: a publisher using the best-effort sensor data profile and a subscription left at the reliable default never connect.
Discovery
Discovery is automatic, through the middleware. A starting node advertises itself to other nodes with the same ROS domain, set by the ROS_DOMAIN_ID environment variable; nodes answer, advertise periodically so that late arrivals are found, and announce when they go offline. Connections are made only between compatible QoS settings. In our reading, different domain IDs therefore separate robots, or experiments, that share a network.
Launch and composition
A system of many nodes is started from a launch file, written in XML, YAML or Python and run with ros2 launch. It describes which programs to run, where, with which arguments and configuration, and the launch system monitors the processes it started.
Composition decides at deployment time whether nodes run in separate processes, with fault isolation and easier debugging, or together in one process, with lower overhead and optionally more efficient intra-process communication. The recommended way to write C++ code is as a component, a node built into a shared library, similar to a ROS 1 nodelet, which a component container loads at run time. The Jazzy documentation lists three generic containers: component_container (one single-threaded executor), component_container_mt (one multi-threaded executor) and component_container_isolated (an executor per component). For image pipelines, in our reading, this is how large frames avoid serialization between a driver and the nodes that consume them; the Composition page itself states only the lower overhead and the optional intra-process path.
tf2 and coordinate frames
tf2 keeps track of coordinate frames over time. It stores the relations between frames as a tree buffered in time and transforms points, vectors and poses between any two frames at a requested time, answering questions such as where the head frame was relative to the world five seconds ago. Transforms are broadcast by many nodes, each for part of the robot, and gathered by listeners; static transforms, such as a camera’s mounting on the chassis, are published once and kept without history. The Tf2 page cites a TePRA 2013 paper on the library, “tf: The transform library” (not read). The page warns about direction: a transform published in geometry_msgs/msg/Transform describes the frame, which is the inverse of the transform that maps data from one frame into the other, and the library inverts it as it walks the tree.
Two REPs fix the conventions:
- REP 103 fixes SI units and right-handed systems, with forward, left and up for a body, and the
_opticalframes of cameras, with forward, right and down. TheImagemessage’s header comment repeats the optical convention: origin at the optical centre, to the right of the image, down, into the image. - REP 105 names the frames of mobile platforms.
base_linkis attached rigidly to the robot base.odomis world-fixed; a pose in it is continuous but drifts without bound, and it is typically computed from wheel, visual or inertial odometry.mapis world-fixed with up; a pose in it should not drift significantly but may jump when localization corrects it.earthis the ECEF origin, for several robots in different maps. Because each frame has exactly one parent, the tree isearth→map→odom→base_link, notmapandodomboth attached tobase_link.
A camera’s optical frame hangs below base_link through the robot’s mounting transforms. REP 105 assigns the odom → base_link transform to an odometry source, visual odometry among them, and the corrective map → odom transform to a localization component, whose jumps make map a poor frame for local sensing; in our reading, SLAM is one such component.
Images and camera calibration
sensor_msgs/Image carries an uncompressed image:
- a header whose stamp “should be acquisition time of image” and whose
frame_idshould be the camera’s optical frame; height(rows) andwidth(columns), with at the top-left corner;encoding, a string fromimage_encodings.hpp: for examplergb8,bgr8,mono8,mono16, OpenCV-style types such as16UC1and32FC1, Bayer patterns such asbayer_rggb8, and YUV formats such asuyvy,yuyv,nv21andnv24;is_bigendianandstep, the full row length in bytes, withdataof sizestep× rows.
In our reading of “full row length”, step may exceed width × bytes per pixel, so a reader must take each row from its own offset; the example below shows what happens otherwise.
Depth images follow REP 118. The canonical form is a 32-bit float per pixel giving depth along the camera’s axis in metres; drivers may instead publish 16-bit unsigned integers in millimetres on the image_raw topic, with 0 denoting an invalid depth. REP 118 keeps depth in the ordinary Image message, rather than a new type, so that image tools, viewers and image_transport work unchanged. Depth map covers the conversion of such images to points.
sensor_msgs/CameraInfo travels on a camera_info topic beside up to five image topics (image_raw, image, image_color, image_rect and image_rect_color). It holds:
- the calibrated image size and a distortion model with its parameters ; for
plumb_bob, the five parameters ; - , the 3 × 3 intrinsic matrix of the raw image, and , the rectification rotation for stereo, both row-major arrays of nine values;
- , the 3 × 4 projection matrix of the rectified image, row-major. For a monocular camera ; for the right camera of a horizontal stereo pair , with the baseline, as in stereo camera;
binning_x,binning_yand a region of interest, which may change without recalibration. Binning values of 0 mean 1, and the ROI is given in full-resolution, unbinned sensor coordinates (REP 104).
The message also says that an uncalibrated camera leaves , , and zeroed, and that clients may take to mean uncalibrated. Camera parameters shows how changes under binning and cropping.
Compression. CompressedImage carries a format string and a compressed buffer, and the values of format depend on the image transport used, that is, on the image_transport plugin that produced it. For the compressed_image_transport plugin the codec is one of JPEG, PNG or TIFF; for compressed_depth_image_transport the source is usually 16UC1 or 32FC1 and the codec PNG or RVL. In our reading, JPEG suits colour frames for viewing or logging, while depth needs a lossless codec; image file formats and image compression cover the codecs.
message_filters and time synchronization
Many vision nodes need frames from several sensors at once: a stereo pair, colour with depth, a camera with a LiDAR scan. message_filters provides filters that sit between subscriptions and a callback: a Subscriber source, a Cache that keeps the last messages for lookup by time, and synchronizers that call back with one message from each input. The index page of the documentation lists three policies for the C++ Synchronizer, each reading the timestamp from the message header:
- ExactTime calls back only when every channel has a message with exactly the same stamp;
- ApproximateEpsilonTime accepts stamps equal within an epsilon;
- ApproximateTime “uses an adaptive algorithm” to match messages.
The C++ synchronizers take up to nine channels.
The Python module implements two classes, TimeSynchronizer and ApproximateTimeSynchronizer. Their names differ from the C++ policies, and so does the approximate algorithm.
The Python classes (__init__.py, commit 05a6b5f). Both keep one dictionary per input, keyed by stamp in nanoseconds; a second message with the same stamp replaces the first. On every incoming message they apply these rules:
- Store the message in its input’s queue. While that queue holds more than
queue_sizemessages, delete the one with the smallest stamp, which is not necessarily the earliest to arrive. TimeSynchronizerfinds the stamps present in every queue, emits the set for the oldest of them, and then deletes from every queue all messages older than the emitted stamp.ApproximateTimeSynchronizertakesslop, documented as a delay in seconds. For each other input it collects the queued stamps whose distance to the new message’s stamp is at mostslop, sorted nearest first; if any input has none, it returns. It then walks the combinations of those candidates withitertools.product, in that nearest-first order, and emits the first combination whose spread, the newest stamp minus the oldest, is strictly less thanslop. It deletes only the emitted messages; unmatched older ones stay until the trim of rule 1 removes them.- Each class emits at most one set per incoming message.
The approximate search thus considers only messages that have already arrived, and pairs the newest message with them at once. It never waits for a later message that would match better. The number of combinations is the product of the candidate counts of the other inputs, each at most queue_size, so it grows exponentially with the number of inputs; in our reading, this keeps the Python class to a few inputs. The Python class can also accept headerless messages with allow_headerless=True, stamping them with the current ROS time; its docstring advises against this, “since the delays are unpredictable”.
The C++ ApproximateTime policy (sync_policies/approximate_time.h, same commit) is a different algorithm, and nothing in the example below tests it. It keeps a deque per channel and maintains a candidate set and a pivot channel. It compares a new candidate interval with the current one, weighting how much later its end is with age_penalty_, which defaults to 0.1. Intervals longer than max_interval_duration_ are rejected; that bound defaults to the largest int32_t number of seconds plus 999,999,999 ns and is changed with setMaxIntervalDuration. Optional inter-message lower bounds per channel let it prove that a candidate is optimal and publish it without waiting for more messages. Its constructor takes a queue size but no slop. It asserts a queue size above zero, with the comment that “at least 2 is recommended”. When a channel’s queue overflows, it drops the oldest message of that channel and marks the channel as having dropped messages.
The documentation of message_filters warns that each subscriber must use the same QoS profile as the corresponding publisher, otherwise the callback is never called and no warning is given. The QoS page’s rule is compatibility rather than equality; in our reading the advice is a safe way to meet it.
TUM RGB-D associates colour and depth stamps offline with its associate.py script: it forms every pair within a maximum difference and accepts them closest first. The ROS synchronizers instead decide online, in arrival order, from what is queued when each message arrives. Real-time vision pipelines covers the levels of synchronization and the clock domains these stamps come from.
Bags
ros2 bag record creates an ordinary subscriber to the requested topics and writes what it receives, without disturbing the other subscribers. rosbag2 stores bags through storage plugins. The repository provides mcap and sqlite3, and the README names mcap as the default. Recording can split files by size or duration and compress with zstd, by file or by message; the README notes that MCAP’s own compression keeps files indexable, whereas rosbag2’s compression does not. A snapshot mode keeps messages in an in-memory circular buffer and writes it to disk only when asked to, through a service on the recorder. ros2 bag play detects the storage format, and ros2 bag convert rewrites bags. A bag’s recording times are not the sensors’ stamps: EuRoC MAV’s tools re-stamp each message of its bags with its header timestamp, because the two differ.
Minimal Example
ROS 2 cannot be installed on the machine used here. The example therefore ports the two Python synchronizers from message_filters jazzy at commit 05a6b5f (src/message_filters/__init__.py) and runs them on synthetic streams whose capture instants are known. rclpy’s Time and Duration become integer nanoseconds, the lock and logging are removed, and a counter is added for messages deleted by the queue_size trim; the code carries the file’s BSD licence text. A second part packs Image-style buffers. Every outcome below tests our port, not ROS. The C++ policy is not tested.
Set-up. The streams last 60 s. Frame of a 30 Hz stream is captured at ns, rounded. The values below are our choices, not values from the sources: the drop rate, offsets, jitter, latencies, the correctness tolerance and the seed.
- A, a hardware-triggered pair: two 30 Hz cameras whose stamps equal their capture instants, identical in both streams. Each message is dropped independently with probability 0.05.
- A0, the base check: A without drops.
- B, a free-running pair: camera 1 at 30 Hz from time 0, camera 2 at 29.97 Hz from 7 ms. Each stamp is the capture instant plus independent integer jitter, uniform in ±1 ms.
- C, a camera with depth: a 30 Hz camera and a 15 Hz depth stream captured at the instants of the even camera frames, both with ±1 ms stamp jitter.
Stream 1 arrives 5 ms after capture. Stream 2 arrives either 5 ms after capture (no lag) or 105 ms after (lag), and messages are fed in order of arrival, stream 1 first on ties. Each policy is run with queue_size 2 and 10: ApproximateTime with slop 5, 17 and 40 ms, given directly in nanoseconds because rclpy’s conversion from seconds was not read, and ExactTime. A set is correct if its two capture instants differ by less than half the 30 Hz period, 16,666,667 ns. Each message carries its capture instant, so the script also counts the possible correct sets in each case. The data seed is 0.
Packing. A 37 × 53 colour image (R = (5c + 40) mod 256, G = (7r + 90) mod 256, B = (3r + 11c + 200) mod 256) and a 16-bit image (D = (4660 + 1777r + 613c) mod 65536) are packed as rgb8, bgr8 and 16UC1 in both byte orders. step is rounded up to a multiple of 64 bytes, and the padding is filled with 0xAB. Our decoder reads rows of step bytes and applies the byte order and channel order. P1 checks the round trips, P2 the data length and P3 the bytes of pixel (0, 0) against values written independently of the packer. P4 tries three wrong decodes: the bgr8 buffer read as rgb8, the big-endian buffer read as little-endian, and the rgb8 buffer read as if rows had no padding.
Hypotheses, written before any script ran, with the derivations that motivated them:
- C0: in A0, every policy at both queue sizes emits all 1,800 possible sets, all correct.
- H1a: ExactTime emits nothing in B and C. Jitter is drawn per message, so stamps never coincide.
- H1b: in A, ExactTime emits exactly the possible sets, all correct, at
queue_size10 with and without lag, and at 2 without lag. Its stamps are equal across streams, and partners arrive at most one frame apart. - H2a: every ApproximateTime set has a stamp spread below the slop. This follows from the code’s own test, so it checks the port only.
- H2b: in B without lag, slop 40 ms gives wrong sets at both queue sizes. With equal latencies, a new message can pair only with already-arrived messages of the other stream. The 30/29.97 Hz drift sweeps the phase through more than one period, so the already-arrived neighbour is sometimes more than half a period away yet within 40 ms, and it is taken before the right partner arrives.
- H2c: in B, slop 5 ms gives no wrong set under any condition. A stamp spread below 5 ms plus at most 1 ms of jitter per stamp bounds the capture spread below 7 ms.
- H2d: in A without lag, slop 40 ms gives wrong sets. When camera 1’s frame is dropped, camera 2’s frame waits in its queue, and camera 1’s frame finds it 33.3 ms away, within 40 ms.
- H2e: in A without lag, slop 5 and 17 ms give no wrong set, since stamps are equal or at least 33.3 ms apart.
- H3a: with the lag,
queue_size2 emits fewer sets than 10, for every case and slop and for ExactTime in A. A 100 ms lag is three frame periods, and a queue of two holds only the two newest stamps of the other stream when the late message arrives. - H3b: with the lag, the trim discards more messages at
queue_size2 than at 10, in every case and policy. - H4: every round trip is exact and every wrong decode differs from the original.
Planning and runs. Before the pilot, a planning script computed the ground truth of the same streams without running any synchronizer: the message counts, the possible sets (1,800, 1,611, 1,798 and 900) and the packing reference bytes. The expected counts in C0 and H1b were therefore written as numbers after that script printed them; the hypotheses themselves were written before. The batch plan’s outline of this example was extended before the pilot, and the editor approved these departures as piloted:
- the arrival-lag condition;
- the parameters listed above as our choices;
- ExactTime run without a slop;
- the printed counts of correct, wrong and possible sets and the data checks;
- the split hypotheses and those on case A;
- the byte-layout and wrong-decode checks;
- the slop given in nanoseconds.
The pilot and the run shown are the same script; the pilot and two runs printed identical output. It runs in under a second with NumPy 2.3.5.
import os
for v in ("OPENBLAS_NUM_THREADS", "OMP_NUM_THREADS", "MKL_NUM_THREADS"):
os.environ[v] = "1" # one BLAS thread, set before NumPy is imported
import itertools
import numpy as np
# TimeSync and ApproxSync below are ported from message_filters (ros2/message_filters, branch jazzy,
# commit 05a6b5f), src/message_filters/__init__.py: TimeSynchronizer.add and
# ApproximateTimeSynchronizer.add. Modified: rclpy Time and Duration are replaced by integer
# nanoseconds, the lock and logging are removed, messages are tuples, and a counter of messages
# deleted by the queue_size trim is added. The file carries this licence:
#
# Copyright (c) 2009, Willow Garage, Inc.
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of the Willow Garage, Inc. nor the names of its
# contributors may be used to endorse or promote products derived from
# this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
# POSSIBILITY OF SUCH DAMAGE.
class TimeSync: # port of TimeSynchronizer (exact stamps)
def __init__(self, n, queue_size):
self.queues = [{} for _ in range(n)] # one {stamp_ns: msg} per input
self.queue_size = queue_size
self.out, self.discarded = [], 0
def trim(self, my_queue):
while len(my_queue) > self.queue_size:
del my_queue[min(my_queue)] # the smallest stamp goes first
self.discarded += 1
def add(self, msg, i):
my_queue = self.queues[i]
my_queue[msg[2]] = msg
self.trim(my_queue)
common = set.intersection(*[set(q) for q in self.queues])
signaled = None
for t in sorted(common):
self.out.append(tuple(q[t] for q in self.queues))
for q in self.queues:
del q[t]
signaled = t
break
if signaled is not None: # drop everything older than the emitted stamp
for q in self.queues:
for s in list(q):
if s < signaled:
del q[s]
class ApproxSync(TimeSync): # port of ApproximateTimeSynchronizer
def __init__(self, n, queue_size, slop_ns):
TimeSync.__init__(self, n, queue_size)
self.slop = slop_ns
def add(self, msg, i):
stamp = msg[2]
my_queue = self.queues[i]
my_queue[stamp] = msg
self.trim(my_queue)
search_queues = self.queues[:i] + self.queues[i + 1:]
stamps = []
for queue in search_queues: # stamps within slop of the new one, nearest first
topic_stamps = []
for s in queue:
delta = abs(s - stamp)
if delta > self.slop:
continue
topic_stamps.append((s, delta))
if not topic_stamps:
return
topic_stamps = sorted(topic_stamps, key=lambda x: x[1])
stamps.append(topic_stamps)
for vv in itertools.product(*[list(zip(*s))[0] for s in stamps]):
vv = list(vv)
vv.insert(i, stamp)
qt = list(zip(self.queues, vv))
if (max(vv) - min(vv)) < self.slop and all(t in q for q, t in qt):
self.out.append(tuple(q[t] for q, t in qt))
for q, t in qt:
del q[t]
break # one set per incoming message
# ----- synthetic streams with exact ground truth (ours) -----
T_NS = 60 * 10**9 # 60 s of data
TOL = 16_666_667 # correct: capture instants < half a 30 Hz period apart
CASES = {"A0": 0, "A": 1, "B": 2, "C": 3}
def frames(rate, start_ns, stop_ns):
k, out = 0, []
while True:
t = int(round(start_ns + k * 1e9 / rate))
if t >= stop_ns:
return np.array(out, dtype=np.int64)
out.append(t); k += 1
def make_case(name):
"""Returns two streams, each (capture_ns, stamp_ns)."""
rng = np.random.default_rng([0, CASES[name]])
if name in ("A0", "A"): # hardware trigger: equal stamps
c1 = frames(30, 0, T_NS); c2 = c1.copy()
s1, s2 = c1.copy(), c2.copy()
if name == "A": # independent drops, p = 0.05 per stream
k1 = rng.random(c1.size) >= 0.05; k2 = rng.random(c2.size) >= 0.05
c1, s1, c2, s2 = c1[k1], s1[k1], c2[k2], s2[k2]
elif name == "B": # free-running 30 Hz and 29.97 Hz, jittered stamps
c1 = frames(30, 0, T_NS); c2 = frames(29.97, 7_000_000, T_NS)
s1 = c1 + rng.integers(-1_000_000, 1_000_001, c1.size)
s2 = c2 + rng.integers(-1_000_000, 1_000_001, c2.size)
else: # camera 30 Hz, depth 15 Hz on even camera frames
c1 = frames(30, 0, T_NS); c2 = c1[::2].copy()
s1 = c1 + rng.integers(-1_000_000, 1_000_001, c1.size)
s2 = c2 + rng.integers(-1_000_000, 1_000_001, c2.size)
return (c1, s1), (c2, s2)
LAT1, LAT2 = 5_000_000, {"no lag": 5_000_000, "lag": 105_000_000}
def feed(streams, lat2):
"""Messages (stream, capture, stamp, arrival) in arrival order; ties: stream 0 first."""
msgs = []
for i, ((c, s), lat) in enumerate(zip(streams, (LAT1, lat2))):
msgs += [(i, int(ci), int(si), int(ci) + lat) for ci, si in zip(c, s)]
return sorted(msgs, key=lambda m: (m[3], m[0]))
def run(sync, msgs):
for m in msgs:
sync.add(m, m[0])
sets = len(sync.out)
caps = [abs(a[1] - b[1]) for a, b in sync.out]
spreads = [abs(a[2] - b[2]) for a, b in sync.out]
correct = sum(d < TOL for d in caps)
return sets, correct, sets - correct, (max(spreads) if spreads else -1), sync.discarded
POLICIES = [("approx", 5_000_000), ("approx", 17_000_000), ("approx", 40_000_000), ("exact", None)]
def make_sync(pol, slop, q):
return ApproxSync(2, q, slop) if pol == "approx" else TimeSync(2, q)
# ----- data checks -----
print("NumPy", np.__version__)
data, possible = {}, {}
for name in CASES:
(c1, s1), (c2, s2) = data[name] = make_case(name)
d = np.abs(c1[:, None] - c2[None, :]); close = d < TOL
possible[name] = int(close.sum())
print("D1 %-2s messages %d + %d | max |stamp - capture| %d ns" % (
name, c1.size, c2.size, max(int(np.abs(s1 - c1).max()), int(np.abs(s2 - c2).max()))), end="")
if name == "A":
print(" | drops %d, %d" % (1800 - c1.size, 1800 - c2.size), end="")
if name == "B":
per = 1e9 / 30
print(" | phase drift %.3f periods" % ((c2[-1] - 7_000_000 - (c2.size - 1) * per) / per), end="")
if name == "C":
print(" | depth captures on camera captures %d" % int(np.isin(c2, c1).sum()), end="")
print()
print("D2 %-2s possible correct sets %d | max partners per message %d" % (
name, possible[name], max(int(close.sum(1).max()), int(close.sum(0).max()))))
print("D3 arrival latency: stream 1 %d ms; stream 2 %d ms (no lag), %d ms (lag)" % (
LAT1 // 10**6, LAT2["no lag"] // 10**6, LAT2["lag"] // 10**6))
# ----- synchronizer grid -----
hdr = "%-4s %-6s %-12s %3s %6s %7s %6s %7s %15s %9s" % (
"case", "lag", "policy", "q", "sets", "correct", "wrong", "sets/s", "max spread ns", "discarded")
print(hdr)
rows = {}
for name in ["A0", "A", "B", "C"]:
for lag in (["no lag"] if name == "A0" else ["no lag", "lag"]):
msgs = feed(data[name], LAT2[lag])
for pol, slop in POLICIES:
for q in (2, 10):
r = rows[(name, lag, pol, slop, q)] = run(make_sync(pol, slop, q), msgs)
label = "exact" if pol == "exact" else "approx %2d ms" % (slop // 10**6)
print("%-4s %-6s %-12s %3d %6d %7d %6d %7.2f %15d %9d" % (
name, lag, label, q, r[0], r[1], r[2], r[0] / 60, r[3], r[4]))
# ----- checks and hypotheses -----
A = [("approx", 5_000_000), ("approx", 17_000_000), ("approx", 40_000_000)]
c0 = all(rows[("A0", "no lag", p, s, q)][:3] == (possible["A0"], possible["A0"], 0)
for p, s in POLICIES for q in (2, 10))
print("C0 A0 every policy and q: sets == correct == %d, wrong 0:" % possible["A0"], c0)
h1a = [rows[(n, l, "exact", None, q)][0] for n in "BC" for l in ("no lag", "lag") for q in (2, 10)]
print("H1a exact sets in B, C (8 conditions):", h1a, "->", all(x == 0 for x in h1a))
h1b = [(l, q, rows[("A", l, "exact", None, q)][:3]) for l, q in (("no lag", 10), ("lag", 10), ("no lag", 2))]
print("H1b exact in A (sets, correct, wrong) vs possible %d:" % possible["A"], h1b, "->",
all(r == (possible["A"], possible["A"], 0) for _, _, r in h1b))
h2a = [(k, v[3]) for k, v in rows.items() if k[2] == "approx" and v[0] > 0 and not v[3] < k[3]]
print("H2a approx rows with spread >= slop:", len(h2a), "->", len(h2a) == 0)
for tag, case, slops, want in (("H2b", "B", [40_000_000], "wrong>0"), ("H2d", "A", [40_000_000], "wrong>0"),
("H2e", "A", [5_000_000, 17_000_000], "wrong=0")):
w = [rows[(case, "no lag", "approx", s, q)][2] for s in slops for q in (2, 10)]
ok = all(x > 0 for x in w) if want == "wrong>0" else all(x == 0 for x in w)
print("%s %s no lag, slop %s ms, q 2/10: wrong %s -> %s" % (
tag, case, "/".join(str(s // 10**6) for s in slops), w, ok))
w = [rows[("B", l, "approx", 5_000_000, q)][2] for l in ("no lag", "lag") for q in (2, 10)]
print("H2c B slop 5 ms, wrong over (no lag, lag) x (q 2, 10):", w, "->", all(x == 0 for x in w))
h3a, h3b = [], []
for n in "ABC":
for p, s in (A + ([("exact", None)] if n == "A" else [])):
a2, a10 = rows[(n, "lag", p, s, 2)], rows[(n, "lag", p, s, 10)]
h3a.append((n, p, s, a2[0], a10[0], a2[0] < a10[0]))
for p, s in POLICIES:
a2, a10 = rows[(n, "lag", p, s, 2)], rows[(n, "lag", p, s, 10)]
h3b.append((n, p, s, a2[4], a10[4], a2[4] > a10[4]))
print("H3a lag, sets q=2 < q=10:", [(n, s and s // 10**6, a, b) for n, p, s, a, b, _ in h3a], "->",
all(x[-1] for x in h3a))
print("H3b lag, discarded q=2 > q=10:", [(n, s and s // 10**6, a, b) for n, p, s, a, b, _ in h3b], "->",
all(x[-1] for x in h3b))
# ----- packing check: sensor_msgs/Image-style buffers (ours) -----
r, c = np.mgrid[0:37, 0:53]
rgb = np.stack([(5 * c + 40) % 256, (7 * r + 90) % 256, (3 * r + 11 * c + 200) % 256], -1).astype(np.uint8)
depth = ((4660 + 1777 * r + 613 * c) % 65536).astype(np.uint16)
BPP = {"rgb8": 3, "bgr8": 3, "16UC1": 2}
def pack(img, encoding, is_bigendian=0, align=64):
h, w = img.shape[:2]
row = w * BPP[encoding]
step = -(-row // align) * align
if encoding == "bgr8":
img = img[..., ::-1]
if encoding == "16UC1":
img = img.astype(">u2" if is_bigendian else "<u2")
buf = np.full((h, step), 0xAB, np.uint8)
buf[:, :row] = np.frombuffer(np.ascontiguousarray(img).tobytes(), np.uint8).reshape(h, row)
return dict(height=h, width=w, encoding=encoding, is_bigendian=is_bigendian, step=step, data=buf.tobytes())
def unpack(msg, ignore_step=False):
h, w, enc = msg["height"], msg["width"], msg["encoding"]
row = w * BPP[enc]
a = np.frombuffer(msg["data"], np.uint8)
a = a[:h * row].reshape(h, row) if ignore_step else a.reshape(h, msg["step"])[:, :row]
if enc == "16UC1":
return a.copy().view(">u2" if msg["is_bigendian"] else "<u2").reshape(h, w).astype(np.uint16)
a = a.reshape(h, w, 3)
return a[..., ::-1] if enc == "bgr8" else a
def maxdiff(x, y):
return int(np.abs(x.astype(np.int64) - y.astype(np.int64)).max())
msgs = {"rgb8": pack(rgb, "rgb8"), "bgr8": pack(rgb, "bgr8"),
"16UC1 BE": pack(depth, "16UC1", 1), "16UC1 LE": pack(depth, "16UC1", 0)}
want = {"rgb8": (40, 90, 200), "bgr8": (200, 90, 40), "16UC1 BE": (0x12, 0x34), "16UC1 LE": (0x34, 0x12)}
p1 = []
for k, m in msgs.items():
ref = depth if k.startswith("16") else rgb
p1.append(maxdiff(unpack(m), ref))
first = tuple(m["data"][:len(want[k])])
print("P %-8s step %3d | len(data) %4d = step x height %4d: %s | pixel (0,0) bytes %s, want %s: %s"
" | round trip max diff %d" % (k, m["step"], len(m["data"]), m["step"] * m["height"],
len(m["data"]) == m["step"] * m["height"], first, want[k], first == want[k], p1[-1]))
wrong = {"bgr8 read as rgb8": maxdiff(unpack(dict(msgs["bgr8"], encoding="rgb8")), rgb),
"BE read as LE": maxdiff(unpack(dict(msgs["16UC1 BE"], is_bigendian=0)), depth),
"rgb8 ignoring step": maxdiff(unpack(msgs["rgb8"], ignore_step=True), rgb)}
print("P4 wrong decodes, max diff:", wrong)
print("H4 round trips exact:", all(x == 0 for x in p1), "| every wrong decode differs:",
all(v > 0 for v in wrong.values()))
NumPy 2.3.5
D1 A0 messages 1800 + 1800 | max |stamp - capture| 0 ns
D2 A0 possible correct sets 1800 | max partners per message 1
D1 A messages 1695 + 1711 | max |stamp - capture| 0 ns | drops 105, 89
D2 A possible correct sets 1611 | max partners per message 1
D1 B messages 1800 + 1798 | max |stamp - capture| 999873 ns | phase drift 1.799 periods
D2 B possible correct sets 1798 | max partners per message 1
D1 C messages 1800 + 900 | max |stamp - capture| 999676 ns | depth captures on camera captures 900
D2 C possible correct sets 900 | max partners per message 1
D3 arrival latency: stream 1 5 ms; stream 2 5 ms (no lag), 105 ms (lag)
case lag policy q sets correct wrong sets/s max spread ns discarded
A0 no lag approx 5 ms 2 1800 1800 0 30.00 0 0
A0 no lag approx 5 ms 10 1800 1800 0 30.00 0 0
A0 no lag approx 17 ms 2 1800 1800 0 30.00 0 0
A0 no lag approx 17 ms 10 1800 1800 0 30.00 0 0
A0 no lag approx 40 ms 2 1800 1800 0 30.00 0 0
A0 no lag approx 40 ms 10 1800 1800 0 30.00 0 0
A0 no lag exact 2 1800 1800 0 30.00 0 0
A0 no lag exact 10 1800 1800 0 30.00 0 0
A no lag approx 5 ms 2 1611 1611 0 26.85 0 182
A no lag approx 5 ms 10 1611 1611 0 26.85 0 166
A no lag approx 17 ms 2 1611 1611 0 26.85 0 182
A no lag approx 17 ms 10 1611 1611 0 26.85 0 166
A no lag approx 40 ms 2 1652 812 840 27.53 33333334 99
A no lag approx 40 ms 10 1652 812 840 27.53 33333334 83
A no lag exact 2 1611 1611 0 26.85 0 10
A no lag exact 10 1611 1611 0 26.85 0 0
A lag approx 5 ms 2 22 22 0 0.37 0 3361
A lag approx 5 ms 10 1611 1611 0 26.85 0 169
A lag approx 17 ms 2 22 22 0 0.37 0 3361
A lag approx 17 ms 10 1611 1611 0 26.85 0 169
A lag approx 40 ms 2 175 3 172 2.92 33333334 3054
A lag approx 40 ms 10 1652 812 840 27.53 33333334 86
A lag exact 2 22 22 0 0.37 0 3345
A lag exact 10 1611 1611 0 26.85 0 0
B no lag approx 5 ms 2 461 461 0 7.68 4978765 2674
B no lag approx 5 ms 10 461 461 0 7.68 4978765 2658
B no lag approx 17 ms 2 1779 1754 25 29.65 16998142 38
B no lag approx 17 ms 10 1779 1754 25 29.65 16998142 22
B no lag approx 40 ms 2 1798 798 1000 29.97 34634043 1
B no lag approx 40 ms 10 1798 798 1000 29.97 34634043 0
B no lag exact 2 0 0 0 0.00 -1 3594
B no lag exact 10 0 0 0 0.00 -1 3578
B lag approx 5 ms 2 2 2 0 0.03 1461114 3593
B lag approx 5 ms 10 461 461 0 7.68 4978765 2661
B lag approx 17 ms 2 2 2 0 0.03 1461114 3593
B lag approx 17 ms 10 1776 1749 27 29.60 16998142 31
B lag approx 40 ms 2 406 0 406 6.77 39960556 2784
B lag approx 40 ms 10 1798 1745 53 29.97 18345132 0
B lag exact 2 0 0 0 0.00 -1 3594
B lag exact 10 0 0 0 0.00 -1 3578
C no lag approx 5 ms 2 900 900 0 15.00 1978528 898
C no lag approx 5 ms 10 900 900 0 15.00 1978528 890
C no lag approx 17 ms 2 900 900 0 15.00 1978528 898
C no lag approx 17 ms 10 900 900 0 15.00 1978528 890
C no lag approx 40 ms 2 900 900 0 15.00 1978528 898
C no lag approx 40 ms 10 900 900 0 15.00 1978528 890
C no lag exact 2 0 0 0 0.00 -1 2696
C no lag exact 10 0 0 0 0.00 -1 2680
C lag approx 5 ms 2 1 1 0 0.02 1271782 2696
C lag approx 5 ms 10 900 900 0 15.00 1978528 892
C lag approx 17 ms 2 1 1 0 0.02 1271782 2696
C lag approx 17 ms 10 900 900 0 15.00 1978528 892
C lag approx 40 ms 2 1 1 0 0.02 1271782 2696
C lag approx 40 ms 10 900 900 0 15.00 1978528 892
C lag exact 2 0 0 0 0.00 -1 2696
C lag exact 10 0 0 0 0.00 -1 2680
C0 A0 every policy and q: sets == correct == 1800, wrong 0: True
H1a exact sets in B, C (8 conditions): [0, 0, 0, 0, 0, 0, 0, 0] -> True
H1b exact in A (sets, correct, wrong) vs possible 1611: [('no lag', 10, (1611, 1611, 0)), ('lag', 10, (1611, 1611, 0)), ('no lag', 2, (1611, 1611, 0))] -> True
H2a approx rows with spread >= slop: 0 -> True
H2b B no lag, slop 40 ms, q 2/10: wrong [1000, 1000] -> True
H2d A no lag, slop 40 ms, q 2/10: wrong [840, 840] -> True
H2e A no lag, slop 5/17 ms, q 2/10: wrong [0, 0, 0, 0] -> True
H2c B slop 5 ms, wrong over (no lag, lag) x (q 2, 10): [0, 0, 0, 0] -> True
H3a lag, sets q=2 < q=10: [('A', 5, 22, 1611), ('A', 17, 22, 1611), ('A', 40, 175, 1652), ('A', None, 22, 1611), ('B', 5, 2, 461), ('B', 17, 2, 1776), ('B', 40, 406, 1798), ('C', 5, 1, 900), ('C', 17, 1, 900), ('C', 40, 1, 900)] -> True
H3b lag, discarded q=2 > q=10: [('A', 5, 3361, 169), ('A', 17, 3361, 169), ('A', 40, 3054, 86), ('A', None, 3345, 0), ('B', 5, 3593, 2661), ('B', 17, 3593, 31), ('B', 40, 2784, 0), ('B', None, 3594, 3578), ('C', 5, 2696, 892), ('C', 17, 2696, 892), ('C', 40, 2696, 892), ('C', None, 2696, 2680)] -> True
P rgb8 step 192 | len(data) 7104 = step x height 7104: True | pixel (0,0) bytes (40, 90, 200), want (40, 90, 200): True | round trip max diff 0
P bgr8 step 192 | len(data) 7104 = step x height 7104: True | pixel (0,0) bytes (200, 90, 40), want (200, 90, 40): True | round trip max diff 0
P 16UC1 BE step 128 | len(data) 4736 = step x height 4736: True | pixel (0,0) bytes (18, 52), want (18, 52): True | round trip max diff 0
P 16UC1 LE step 128 | len(data) 4736 = step x height 4736: True | pixel (0,0) bytes (52, 18), want (52, 18): True | round trip max diff 0
P4 wrong decodes, max diff: {'bgr8 read as rgb8': 254, 'BE read as LE': 64260, 'rgb8 ignoring step': 252}
H4 round trips exact: True | every wrong decode differs: True
Data checks. D1–D3 held. In A, the drops removed 105 and 89 messages. In B, the stamps stay within 999,873 ns of their capture instants, and the phase drifts by 1.799 periods, against by our arithmetic. In C, all 900 depth captures fall on camera captures. No message has more than one possible partner.
C0 held. In A0, every policy at both queue sizes emits 1,800 sets, all correct. Without drops or jitter, the ported classes recover the ground truth.
H1 held. H1a held: ExactTime emits no set in any of the eight B and C conditions, as expected from free-running stamps that never coincide. H1b held: in A, ExactTime emits 1,611 sets, all correct, at queue_size 10 with and without lag and at 2 without lag, which equals the 1,611 possible sets. In our port, exact matching suits hardware-triggered cameras that share a stamp and pairs nothing when stamps carry independent jitter.
H2a held, as a check of the port. No ApproximateTime row has a stamp spread at or above its slop.
H2b held. In B without lag at slop 40 ms, 1,000 of the 1,798 sets are wrong at both queue sizes. H2c held: at slop 5 ms in B, no set is wrong in any of the four conditions. Without lag, the price is the rate: 461 sets, 7.68 per second, against 1,798 possible.
H2d held. In A without lag at slop 40 ms, 840 of 1,652 sets are wrong at both queue sizes. The run emits more sets than there are possible correct ones (1,611). H2e held: at slop 5 and 17 ms the same streams give 1,611 sets with none wrong.
H3 held. H3a held in all ten comparisons. With the lag, queue_size 2 emits 22 sets in A at slop 5 and 17 ms and with ExactTime, against 1,611 at 10. At slop 40 ms the counts are 175 against 1,652 in A and 406 against 1,798 in B. In B at slop 5 and 17 ms they are 2 against 461 and 2 against 1,776. In C, queue_size 2 emits 1 set and 10 emits 900, at every slop. H3b held in all twelve comparisons: each count of messages discarded at queue_size 2 exceeds its count at 10, for example 3,361 against 169 in A at slop 5 ms, 2,784 against 0 in B at slop 40 ms, and 2,696 against 2,680 for ExactTime in C.
Rows without a prediction, reported as printed:
- B at slop 17 ms gives 25 wrong sets of 1,779 without lag (both queue sizes), and 27 of 1,776 with lag at
queue_size10. - B with lag at
queue_size10 and slop 40 ms gives 53 wrong sets of 1,798. - A at slop 40 ms gives 840 wrong sets of 1,652 with lag at
queue_size10, the same count as without lag. - A with lag at
queue_size2 and slop 40 ms gives 172 wrong sets of 175. - B with lag at
queue_size2 and slop 40 ms gives 406 sets, all wrong. - C without lag gives 900 sets, all correct, at every slop and both queue sizes.
H4 held, and the packing checks P1–P4 held. All four buffers round-trip with a maximum difference of 0; each holds step × height bytes (7,104 for the colour encodings, 4,736 for 16UC1); and the first pixel’s bytes are (40, 90, 200) for rgb8, (200, 90, 40) for bgr8, (0x12, 0x34) big-endian and (0x34, 0x12) little-endian. Each wrong decode differs from the original, with maximum differences of 254, 64,260 and 252. Reading bgr8 as rgb8 swaps red and blue. Reading big-endian data as little-endian swaps the bytes of every 16-bit value. Ignoring step shifts every row after the first by the accumulated padding. The 16-bit big-endian layout is the one PNG uses, as image file formats describes.
In our reading, the example shows that the approximate synchronizer’s slop is an acceptance window, not a guarantee of correct pairing. A window wider than half the frame period admits neighbouring frames whenever the true partner is missing or has not yet arrived. A queue shorter than the lag between streams removes true partners before they can be matched.
Ecosystem and Integrations
- Vision packages. The
CameraInfomessage namesimage_pipeline, whoseimage_procandstereo_image_procpackages produce the processed and rectified image topics fromimage_rawandcamera_info, andimage_geometry, which it recommends for projecting points with the calibration.CompressedImagenames thecompressed_image_transportandcompressed_depth_image_transportplugins ofimage_transport. - OpenCV and other libraries. The encodings in
image_encodings.hppinclude OpenCV’s type names (8UC3,16UC1,32FC1and others). REP 2000 lists each distribution’s dependency versions per platform at its first release, as a low watermark, among them OpenCV, PCL, Qt, Python and NumPy; for Jazzy on Ubuntu Noble these are OpenCV 4.6.0, PCL 1.14.0, NumPy 1.26.4 and Python 3.12.3. - Simulation. REP 2000 lists Gazebo Harmonic among Jazzy’s dependencies on Ubuntu, Debian and macOS. CARLA offers a native ROS 2 interface and a separate ROS bridge.
- Datasets. Several published datasets were recorded through ROS or ship ROS bags. TUM RGB-D was recorded with ROS Diamondback and its
openni_camerapackage, as described in the TUM RGB-D paper, and EuRoC MAV offered its sequences as bags. In drivable-area segmentation, the HybridNets repository includes a ROS package for path planning. - Tools. The Jazzy Concepts pages also include pages on the command-line tools and on RQt, which were not read for this article.
Alternatives
The sources do not rank middleware, and neither does this article.
- ROS 1. The ROS 2 pages contrast it in three places. ROS 1 connected every publisher and subscriber of the same type on the same topic and primarily used TCP. It offered queue sizes and latching, where ROS 2 has QoS policies. It had separate node and nodelet APIs, where ROS 2 has one component API. Nothing about ROS 1’s current support is claimed here.
- DDS directly. In our reading, since ROS 2 runs on DDS, a system that needs DDS features without the rest of the ROS ecosystem can use a DDS implementation’s own API, giving up ROS’s message packages, tools and conventions.
- Media pipelines. For pure video work, a media framework such as GStreamer moves frames between elements in one process. Real-time vision pipelines describes GStreamer’s queues and latency model.
- In-process code. A single program with threads and queues avoids serialization and discovery entirely. Real-time vision pipelines discusses its queue policies.
Versions and Compatibility
Distributions. As of November 2026, REP 2000 at commit 11ca24a (23 October 2025, the newest commit of the REP repository) gives this schedule:
- a new ROS 2 release every 12 months;
- in even years, an LTS release one month after the Ubuntu LTS, usually in May, supported for 4 years and 11 months;
- in odd years, a non-LTS release on the previous LTS’s Ubuntu, supported for 1.5 years.
The entries current at that commit are:
- Humble Hawksbill, May 2022 to May 2027;
- Jazzy Jalisco, May 2024 to May 2029;
- Kilted Kaiju, May 2025 to November 2026;
- Rolling Ridley, from June 2020, ongoing. It is a rolling development distribution which, as of May 2025, targets the same platforms as Kilted.
The copy lists no distribution released after Kilted, and none is described here. The facts in this article come from Jazzy branches.
Jazzy’s platforms (REP 2000):
- Tier 1: Ubuntu 24.04 (Noble) on amd64 and arm64, and Windows 10 (VS2019) on amd64.
- Tier 2: RHEL 9 on amd64.
- Tier 3, community-supported: Ubuntu 22.04, macOS, Debian Bookworm and OpenEmbedded/Yocto.
Tier 1 platforms get continuous integration, nightly and packaging jobs, and their errors are prioritized; Tier 3 platforms are those that community reports say work. Jazzy’s minimum language requirements are C++17 and Python 3.8. Its middleware support is:
- Tier 1: Fast DDS (the default), Cyclone DDS, and Connext on all architectures except arm64;
- Tier 2:
rmw_fastrtps_dynamic_cpp; - Tier 3: GurumDDS.
A middleware gets no better support than the tier of the platform it runs on.
Compatibility. Mixing RMW implementations, or ROS versions, across one system is not guaranteed to work (middleware page). In our reading, a bag recorded under one distribution is safest to replay under the same one.
Licences. ROS 2 has no single licence; each package carries its own. Among those read here, message_filters is BSD-3-Clause, sensor_msgs and rosbag2 are Apache-2.0, the documentation is CC BY 4.0, and the REPs are in the public domain. The middleware licences in the table above range from Apache 2.0 to commercial.
Versions used here. The example ran with NumPy 2.3.5. It ports message_filters jazzy at 05a6b5f; no ROS distribution was installed.
Limitations
- Pairing is only as good as the stamps. The synchronizers match header stamps, which the
Imagemessage requires to be acquisition times. A driver that stamps on arrival, or clocks that are not mapped into one domain, defeat every policy. - Approximate synchronization can pair the wrong frames, as the example shows for our port of the Python class; the C++ policy was not tested.
- QoS mismatches fail silently. An incompatible pair does not connect and a filter chain never calls back; only monitored incompatibility events reveal it.
- Large images cost bandwidth and copies.
Imagecarries raw pixels; compression needs animage_transportplugin, and composition offers an intra-process path. In our reading, how the middleware handles large messages should be measured on the target system. - Frame conventions must be followed. Optical frames, REP 105’s frame tree and the direction of tf2 transforms are conventions, not checks: a camera published with body-frame axes, or a transform inverted, produces data that look valid and are wrong. Rotation representations and camera pose cover common mistakes.
- A distribution is a moving target. Non-LTS distributions are supported for 1.5 years and LTS ones for about five, and binary support depends on the platform tier, so a deployed robot is tied to an Ubuntu release and a distribution’s end of life.
Related
- Real-Time Vision Pipelines
How a vision system turns a stream of camera frames into timely decisions, covering frame budgets, latency against throughput, pipelining and threads, queues and dropped frames, timestamps and clock synchronization, and how end-to-end latency is measured.
- Robot Perception
How robots use cameras, depth sensors, LiDAR, inertial and joint sensors and touch to estimate their own state and the world around them fast enough to act on it, covering the perception–action loop, state estimation, latency and robustness, and the main robot settings from mobile robots and arms to drones, legged robots and cars.
- TUM RGB-D
The TUM RGB-D SLAM dataset and benchmark, Kinect and Asus Xtion sequences at 30 Hz with motion-capture ground truth at 100 Hz; its sequence categories, file formats and calibration, timestamp association, and the absolute trajectory error and relative pose error as the official scripts compute them.
- Camera Intrinsic and Extrinsic Parameters
The parameters of the pinhole camera matrix P = K[R | t], split into intrinsics K, which map camera coordinates to pixels, and extrinsics R and t, which place the camera in the world.
- Image File Formats
What PNG, TIFF, JPEG files, OpenEXR and RAW containers store, including chunks, tags, sample types, bit depths and metadata; how far each one's dynamic range reaches; and what Pillow, imageio and tifffile return when they read them.
- EuRoC MAV
Eleven visual-inertial sequences flown by a hexacopter in an ETH Zurich machine hall and a motion-capture room, with stereo images, IMU data and laser-tracker or Vicon ground truth; the standard test set for visual-inertial odometry and SLAM, and how trajectory alignment changes the errors reported on it.
References
- ROS 2 documentation (Jazzy), Concepts: Nodes, Topics, Services, Actions, Interfaces, Discovery and Launch (source/Concepts/Basic), ros2/ros2_documentation at commit 929e37c (CC BY 4.0; read November 2026).
- ROS 2 documentation (Jazzy), Concepts: Quality of Service settings, Different ROS 2 middleware vendors, Composition and Tf2 (source/Concepts/Intermediate), ros2/ros2_documentation at commit 929e37c (CC BY 4.0; read November 2026).
- message_filters (jazzy): src/message_filters/__init__.py, include/message_filters/sync_policies/approximate_time.h, doc/index.rst and the Approximate Time Synchronizer (Python) tutorial, ros2/message_filters at commit 05a6b5f (BSD-3-Clause; read November 2026).
- sensor_msgs (jazzy): msg/Image.msg, msg/CameraInfo.msg, msg/CompressedImage.msg and include/sensor_msgs/image_encodings.hpp, ros2/common_interfaces at commit a941f14 (Apache-2.0 package licence; read November 2026).
- Foote, T. & Purvis, M. (2010). REP 103: Standard Units of Measure and Coordinate Conventions. ROS Enhancement Proposals, ros-infrastructure/rep at commit 11ca24a (read November 2026).
- Mihelich, P. (2010). REP 104: CameraInfo updates for Diamondback. ROS Enhancement Proposals, ros-infrastructure/rep at commit 11ca24a (read November 2026).
- Meeussen, W. (2010). REP 105: Coordinate Frames for Mobile Platforms. ROS Enhancement Proposals, ros-infrastructure/rep at commit 11ca24a (read November 2026).
- Mihelich, P. (2011). REP 118: Depth Images. ROS Enhancement Proposals, ros-infrastructure/rep at commit 11ca24a (read November 2026).
- Arguedas, M., Ragnarok, S., Thomas, D., Nash, A., Biggs, G. & Gutierrez, M. A. REP 2000: ROS 2 Releases and Target Platforms. ROS Enhancement Proposals, ros-infrastructure/rep at commit 11ca24a, 23 October 2025 (read November 2026).
- rosbag2 (jazzy): README.md, ros2/rosbag2 at commit 12b5e8b (Apache-2.0; read November 2026).
- tf: The transform library. TePRA 2013, as cited by the ROS 2 documentation's Tf2 page (not read).