Datasets & Benchmarks
EuRoC MAV
Eleven visual-inertial sequences flown by a hexacopter in an ETH Zurich machine hall and a motion-capture room, with stereo images, IMU data and laser-tracker or Vicon ground truth; the standard test set for visual-inertial odometry and SLAM, and how trajectory alignment changes the errors reported on it.
intermediate
EuRoC MAV is a set of eleven visual-inertial sequences recorded on board a small multicopter, a micro aerial vehicle (MAV), in two indoor environments at ETH Zurich: an industrial machine hall and a room equipped with a Vicon motion-capture system. Each sequence contains synchronized stereo images and inertial measurement unit (IMU) data, the calibration of the sensors, and ground-truth motion from a Leica laser tracker or from the Vicon system. Michael Burri, Janosch Nikolic, Pascal Gohl, Thomas Schneider, Joern Rehder, Sammy Omari, Markus Achtelik and Roland Siegwart of the Autonomous Systems Lab (ASL) described the data in The International Journal of Robotics Research, published online in January 2016. The data is often called the EuRoC dataset, and its sequences are named by environment and number, such as Machine Hall 01 (MH01) or Vicon Room 2 03 (V203).
EuRoC became the usual test set for visual odometry with an IMU (visual-inertial odometry, VIO) and for visual-inertial SLAM. The dataset has no official evaluation server or scoring script. Results on it are absolute trajectory errors computed by the authors of each method, so the alignment conventions they choose matter. This article describes the data, the ground truth and its known limits, and how the main sources evaluate on it.
The paper itself is closed access and was not available for this article. The descriptions below come from its abstract, the official dataset pages (current and archived), the official loader code, and the papers cited for each statement.
Purpose
The abstract gives two aims, one for each batch of sequences. The machine-hall sequences, recorded in an industrial environment with millimetre-accurate position ground truth from a laser tracker, support the design and evaluation of visual-inertial localization on real flight data. The Vicon-room sequences, with 6-D pose ground truth and a detailed 3-D scan of the room, support precise 3-D reconstruction. The flights range from slow ones in good visual conditions to dynamic ones with motion blur and poor illumination.
Delmerico and Scaramuzza (2018) called EuRoC the most appropriate dataset for flying-robot 6-DoF trajectories when they benchmarked VIO algorithms on it two years later.
Dataset Size
Platform and sensors
The official pages describe the following setup (as of October 2026; the archived former wiki page gives the component models):
- Platform: an AscTec Firefly hexacopter carrying a visual-inertial sensor unit.
- Cameras: a stereo pair of global-shutter WVGA monochrome cameras (Aptina MT9V034), each at 20 frames per second. Schubert et al. (2018, Table I) give the resolution as 752 × 480, and their text identifies the unit as a Skybotix stereo VI sensor.
- IMU: a MEMS IMU (ADIS16448) measuring angular rate and acceleration at 200 Hz. The TUM VI comparison table lists an ADIS16488; the archived former wiki page names the ADIS16448, and the current page names no model.
- Synchronization: “shutter-centric” temporal alignment. Both cameras run automatic exposure independently, and the mid-exposure times of the two cameras are aligned, so different shutter times do not break the stereo synchronization (former wiki page).
Sequences
The archived download table lists eleven sequences, with a difficulty label in each name. ORB-SLAM3’s authors describe them as sessions in three environments (Campos et al., 2021, Section VII-C).
| Environment | Ground truth | Sequences (official label) |
|---|---|---|
| Machine Hall | Leica MS50 laser tracker, 3-D position | MH_01_easy, MH_02_easy, MH_03_medium, MH_04_difficult, MH_05_difficult |
| Vicon Room 1 | Vicon motion capture, 6-D pose; Leica scan of the room | V1_01_easy, V1_02_medium, V1_03_difficult |
| Vicon Room 2 | Vicon motion capture, 6-D pose; Leica scan of the room | V2_01_easy, V2_02_medium, V2_03_difficult |
A twelfth download holds calibration recordings for users who want to run their own calibration. All the sequences are indoor flights, piloted manually according to Delmerico and Scaramuzza (2018), and difficulty rises with the sequence number within each environment (faster motion, poorer illumination). Schubert et al. (2018, Table I) total the eleven sequences at 0.9 km. Delmerico and Scaramuzza name a Vicon Room 2 sequence of 36 m as the shortest trajectory. The paper’s per-sequence durations and lengths were not checked for this article.
As of October 2026 the files are hosted on the ETH Research Collection. An archived copy of the record from June 2025 lists four ZIP archives: Machine Hall (11.81 GB), Vicon Room 1 (5.627 GB), Vicon Room 2 (5.600 GB) and the calibration datasets (4.112 GB). The former wiki page also offered each sequence as a ROS bag.
File format
The ASL dataset format stores each recording as folders: one per rigid body, marked by a body.yaml, with one subfolder per sensor. Each sensor folder holds a sensor.yaml with the sensor’s type and calibration and a data.csv with one row per measurement, as the official MATLAB loader in dataset_tools reads them. The first column is an integer timestamp, which evo and the rpg toolbox both convert from nanoseconds to seconds. The loader reads five measurement types (it also recognizes camera_target, for calibration-target observations, and pointcloud, for which it loads nothing):
imu: angular rate and acceleration.camera: a timestamp and an image file name.position: a 3-D position.pose: a position and a quaternion.visual-inertial: position, quaternion, velocity, and gyroscope and accelerometer biases.
Its variable names follow the frame convention of the sensor diagram, for example p_RS_R for the position of sensor frame S in reference frame R. By our reading of the type names, the laser tracker’s data is a position sensor, the Vicon’s a pose sensor, and the visual-inertial type the post-processed ground-truth estimate, which evo reads from state_groundtruth_estimate0/data.csv, taking the quaternion with its scalar first. For the ROS bags, the official tools include a script that re-stamps each message with its header timestamp, because the bags’ recording times differ from the sensor timestamps.
Annotation Types
Ground truth. In the machine hall, a Leica MS50 laser tracker follows a prism marker on the vehicle and measures its 3-D position only. In the Vicon rooms, the motion-capture system measures the full 6-D pose. The Leica MS50 was also used as a scanner to record a 3-D structure scan of the Vicon environment, which provides the geometry ground truth for reconstruction. Schubert et al. (2018, Table I) list the laser tracker at 20 Hz, the motion capture at 100 Hz, and an accuracy of about 1 mm for both.
Calibration. Each sequence includes the camera intrinsics, the camera–IMU extrinsics and ground truth aligned in space and time with the sensor data. The official sensor diagram shows the frames involved:
- The IMU defines the body frame .
- Each sensor (the two cameras and the prism marker) has a calibrated transform to the body frame.
- The tracking system measures the marker in its own frame .
- A world frame is drawn next to the gravity vector .
A raw ground-truth position therefore describes the prism marker in the tracker’s frame, not the camera or the IMU. It follows that an estimate of the camera or IMU pose has to be converted through the calibrated transforms before it can be compared with the raw ground truth. If the loader reading above is right, the post-processed ground truth carries a pose, a velocity and IMU biases.
Time synchronization. The sensor data and the motion ground truth were recorded on separate systems, and the Vicon system provided no device timestamps. The former wiki page states that the temporal offset was estimated as a state, covering both a fixed offset and clock drift, and that device timestamps were used where available to avoid jitter. The raw data is released so that users can try other synchronization schemes. The current page still lists synchronization as a known issue (as of October 2026).
Two kinds of ground truth are therefore in use: the raw tracker or Vicon measurements, and the processed estimate. Campos et al. (2021, Table II) mark which numbers in their comparison were computed against “raw GT” rather than “processed GT”, which shows that published EuRoC errors do not all use the same reference.
Splits
EuRoC defines no training, validation or test split. Every sequence comes with its ground truth, and methods are run on all eleven or on a stated subset. Within one environment the sequences share a world reference, so a single global alignment can score several sessions processed one after another; ORB-SLAM3’s multi-session experiments rely on this (Campos et al., 2021, Section VII-C). In our reading, a learned method that trains on EuRoC sequences has to choose its own held-out sequences, so its results are comparable only when that split is stated.
Tasks
- Visual-inertial odometry and SLAM, monocular-inertial or stereo-inertial, the main use.
- Pure visual odometry and SLAM, monocular or stereo, ignoring the IMU.
- Camera–IMU calibration, using the calibration recordings.
- 3-D reconstruction in the Vicon rooms, against the laser scan of the room.
Evaluation
The dataset’s authors provide data, calibration and a loader, but no evaluation script, server or leaderboard. Errors on EuRoC are almost always the RMSE of the translational absolute trajectory error (ATE), sometimes together with relative errors over sub-trajectories, computed by each paper’s authors. The metric and the alignment groups are defined in absolute trajectory error. For EuRoC, three choices decide the number:
- which transformation aligns the estimate to the ground truth;
- which poses that alignment is fitted to;
- which ground truth is used, raw or processed, and for which frames (every frame or only keyframes).
Conventions in the main sources
The sources disagree on the alignment for visual-inertial estimates:
- Zhang and Scaramuzza (2018) derive the alignment from what the sensors leave unobservable. With an IMU, gravity makes roll and pitch observable, so only a yaw about gravity and a translation remain: a 4-DoF alignment. Monocular vision alone needs Sim(3) and stereo needs SE(3) (their Table I).
- Delmerico and Scaramuzza (2018), from the same group a few months earlier, aligned monocular VIO estimates on EuRoC with a Sim(3) transformation (Umeyama’s method) before computing the RMSE. They also reported relative errors over segments of 7 to 35 m.
- Campos et al. (2021) aligned pure monocular estimates with Sim(3) and every other configuration, including monocular-inertial and stereo-inertial, with SE(3). They reported the median over 10 runs, measured scale error as from the Sim(3) fit, and used the full trajectory and the processed ground truth (arXiv version, Section VII). Their comparison table also collects other systems’ numbers that were computed against raw ground truth, on keyframe trajectories only, or by re-running the code; footnotes mark these cases (Table II).
All three are defensible. They answer different questions, and the example below shows that they give different numbers for the same estimate.
The rpg_trajectory_evaluation toolbox
Zhang and Scaramuzza released the toolbox that implements their tutorial. It is a third-party tool from the University of Zurich, not from the dataset’s authors, but its repository ships EuRoC example configurations, and it is the closest thing to a reference implementation of VIO evaluation on EuRoC. At commit 8c8ceec (September 2022, the latest as of October 2026) it computes the absolute error as follows; the code was read line by line and its alignment and error functions were also run:
- Association. Estimated and ground-truth timestamps (in seconds) are matched greedily, closest pair first, accepting differences below 0.02 s, with zero offset. The ground truth is not interpolated. Estimates with no match are dropped, and the matches are cached to a file (
trajectory_loading.py, with association code adapted from the TUM RGB-D benchmark’sassociate.py). - Alignment. A per-run
eval_cfg.yamlsetsalign_type(sim3,se3,posyawornone) andalign_num_frames(the first poses, or for all). Without the file, the toolbox aligns in Sim(3) over all poses. Theeval_cfg.yamlfiles of the repository’s EuRoC example results (34 runs of monocular and stereo VIO and of visual-inertial SLAM) all setposyawover all poses. - Fitting. Every multi-pose alignment uses Umeyama’s method on positions only. For
posyaw, the rotation is replaced by the yaw that maximizes with the cross-covariance of centred estimated and true positions. The code writes the solution as , which equals the tutorial’s Eq. 18 up to a multiple of . - First-pose alignment. With ,
se3uses (Eq. 19), andposyawtakes the yaw closest to that rotation (Eq. 20). The Sim(3) alignment needs at least two poses. - Errors. The translational error of pose is , where is the aligned estimated position. The rotational error is the angle of . The statistics (RMSE, mean, median, standard deviation, minimum, maximum, number of samples) are written to a YAML file named after the alignment type and the number of aligned poses.
Code and paper differ in one place. The tutorial’s Eq. 23 defines the position error as , with : the translation of the error transformation . The toolbox computes , the same quantity that Sturm et al.’s ATE reduces to. The two agree only when the rotation error is zero. The paper’s form also depends on where the world origin lies, as the example’s last two lines show. Numbers produced by the toolbox follow the code.
evo, the other widely used package, offers SE(3) or Sim(3) Umeyama alignment over all poses or the first , and a rigid first-pose (origin) alignment. Version 1.38.1 has no yaw-only alignment, so a 4-DoF evaluation needs the rpg toolbox or one’s own code.
Example: alignment on a synthetic flight
The script simulates a 120 s flight over a figure-eight in a room-sized volume, with poses at the cameras’ 20 Hz. The attitude is that of a multicopter: yaw follows the direction of travel, and roll and pitch tilt the thrust to produce the horizontal acceleration. A VIO-like estimate dead-reckons the true motion increments through an orientation error. Five estimates are compared:
- Jitter only: 1 cm and 0.2° of noise per pose and axis, with no drift.
- Yaw drift: the heading error grows at 0.05° per second. This error is unobservable for VIO, so the 4-DoF alignment should be allowed to remove a yaw offset.
- Tilt drift: the gravity (roll) error grows at 0.02° per second.
- Constant tilt: a fixed 1° gravity error.
- Scale +2%: the metric scale is 2% too large.
The last four carry the same jitter as the first (seed 0).
The tilt and scale errors are observable for a visual-inertial system, so an evaluation should charge for them. Each estimate is expressed in its own gravity-aligned frame, rotated 70° about the vertical and shifted, as a VIO system’s world frame would be. The alignment functions follow the tutorial’s equations. They agree with the toolbox’s alignTrajectory and compute_absolute_error at commit 8c8ceec to within over 75 case–alignment pairs, and the SE(3) and Sim(3) results agree with evo 1.38.1’s umeyama_alignment to within cm.
import numpy as np
def rot(axis, a):
c, s = np.cos(a), np.sin(a)
i, j = {"x": (1, 2), "y": (2, 0), "z": (0, 1)}[axis]
R = np.eye(3)
R[i, i], R[i, j], R[j, i], R[j, j] = c, -s, s, c
return R
def angle_deg(R):
return np.degrees(np.arccos(np.clip((np.trace(R) - 1) / 2, -1.0, 1.0)))
def best_yaw(C):
"""theta maximizing trace(Rz(theta) C) (Zhang & Scaramuzza 2018, eqs. 17-18, 20)."""
return np.arctan2(C[0, 1] - C[1, 0], C[0, 0] + C[1, 1])
def align(P, Q, Rs_e, Rs_g, mode, n=None):
"""s, R, t with gt ~ s R est + t, fitted on the first n poses (all if None)."""
if n == 1: # first pose only (eq. 19 / eq. 20)
C = Rs_e[0] @ Rs_g[0].T
R = rot("z", best_yaw(C)) if mode == "4-DoF" else Rs_g[0] @ Rs_e[0].T
return 1.0, R, Q[0] - R @ P[0]
P, Q = P[:n], Q[:n]
mp, mq = P.mean(0), Q.mean(0)
Pc, Qc = P - mp, Q - mq
if mode == "4-DoF": # Umeyama with a yaw-only rotation (eq. 18)
R, s = rot("z", best_yaw(Pc.T @ Qc)), 1.0
else: # Umeyama (Alg. 1)
U, D, Vt = np.linalg.svd(Qc.T @ Pc / len(P))
W = np.diag([1.0, 1.0, np.sign(np.linalg.det(U) * np.linalg.det(Vt))])
R = U @ W @ Vt
s = np.trace(np.diag(D) @ W) / np.mean(np.sum(Pc ** 2, 1)) if mode == "Sim(3)" else 1.0
return s, R, mq - s * R @ mp
def ate(est, gt, mode, n=None):
"""Translational ATE RMSE (cm) and rotational ATE RMSE (deg) after alignment."""
(Rs_e, P), (Rs_g, Q) = est, gt
s, R, t = align(P, Q, Rs_e, Rs_g, mode, n)
e_pos = Q - (s * P @ R.T + t)
e_rot = [angle_deg(Rg @ (R @ Re).T) for Re, Rg in zip(Rs_e, Rs_g)]
return 100 * np.sqrt(np.mean(np.sum(e_pos ** 2, 1))), np.sqrt(np.mean(np.square(e_rot)))
# Ground truth: a 120 s flight in a 6 m x 4 m x 1 m volume, poses at 20 Hz.
# Yaw follows the direction of travel; roll and pitch tilt the thrust to produce
# the horizontal acceleration, as on a multicopter.
g, T, hz = 9.81, 120.0, 20
t = np.arange(0, T, 1 / hz)
w = 2 * np.pi / 40
Q = np.column_stack([3 * np.sin(w * t), 2 * np.sin(2 * w * t), 1.5 + 0.5 * np.sin(3 * w * t)])
v = np.gradient(Q, t, axis=0)
a = np.gradient(v, t, axis=0)
Rs_g = []
for vi, ai in zip(v, a):
Rz = rot("z", np.arctan2(vi[1], vi[0]))
ab = Rz.T @ ai # acceleration in the heading frame
Rs_g.append(Rz @ rot("y", np.arctan2(ab[0], g + ab[2])) @ rot("x", -np.arctan2(ab[1], g + ab[2])))
gt = (Rs_g, Q)
N = len(t)
print(f"ground truth: {N} poses, {np.sum(np.linalg.norm(np.diff(Q, axis=0), axis=1)):.1f} m, "
f"max tilt {max(np.degrees(np.arccos(R[2, 2])) for R in Rs_g):.1f} deg")
def vio_estimate(err_axis="z", err_deg_per_s=0.0, tilt0_deg=0.0, scale=1.0, seed=None):
"""Dead-reckoned estimate: the true increments, rotated by an orientation error
that grows at a constant rate (about z: yaw drift; about x: gravity, i.e. tilt,
drift), times a scale, plus per-pose jitter. Expressed in the estimator's own
gravity-aligned frame (unknown yaw and origin)."""
E = [rot(err_axis, np.radians(err_deg_per_s * ti)) @ rot("x", np.radians(tilt0_deg)) for ti in t]
P = [Q[0]]
for i in range(N - 1):
P.append(P[-1] + scale * E[i] @ (Q[i + 1] - Q[i]))
P, Rs = np.array(P), [Ei @ R for Ei, R in zip(E, Rs_g)]
if seed is not None: # jitter: 1 cm and 0.2 deg per axis
r = np.random.default_rng(seed)
P = P + 0.01 * r.normal(size=P.shape)
Rs = [R @ rot("x", np.radians(0.2 * r.normal())) @ rot("y", np.radians(0.2 * r.normal()))
@ rot("z", np.radians(0.2 * r.normal())) for R in Rs]
G, o = rot("z", np.radians(70)), np.array([4.0, -2.0, 0.3])
return [G @ R for R in Rs], P @ G.T + o
cases = {
"jitter only": vio_estimate(seed=0),
"yaw drift 0.05 deg/s": vio_estimate("z", 0.05, seed=0),
"tilt drift 0.02 deg/s": vio_estimate("x", 0.02, seed=0),
"constant tilt 1 deg": vio_estimate(tilt0_deg=1.0, seed=0),
"scale +2%": vio_estimate(scale=1.02, seed=0),
}
modes = [("4-DoF", 1), ("SE(3)", 1), ("4-DoF", None), ("SE(3)", None), ("Sim(3)", None)]
names = ["4-DoF first", "SE(3) first", "4-DoF all", "SE(3) all", "Sim(3) all"]
print("\ntranslational ATE (cm) / rotational ATE (deg)")
print(f"{'':24s}" + "".join(f"{m:>15s}" for m in names))
for name, est in cases.items():
print(f"{name:24s}" + "".join("{:8.1f} /{:5.2f}".format(*ate(est, gt, m, n)) for m, n in modes))
print("\njitter only, seeds 0-9:")
for m, n, nm in [("4-DoF", 1, "4-DoF first"), ("4-DoF", None, "4-DoF all")]:
e = [ate(vio_estimate(seed=sd), gt, m, n)[0] for sd in range(10)]
print(f" {nm:12s} translational ATE {min(e):.1f}-{max(e):.1f} cm")
# Zhang & Scaramuzza's eq. 23 writes the position error as p - dR p', with dR the
# rotation error; unlike p - p' (used above and in their toolbox), it changes
# when the ground-truth world origin moves.
est = cases["constant tilt 1 deg"]
for shift in (0.0, 10.0):
Qs = Q + shift
s, R, tr = align(est[1], Qs, est[0], Rs_g, "4-DoF")
Pa, Ra = s * est[1] @ R.T + tr, [R @ Re for Re in est[0]]
e23 = [np.linalg.norm(q - Rg @ Re.T @ p) for q, p, Rg, Re in zip(Qs, Pa, Rs_g, Ra)]
print(f"constant tilt, 4-DoF all, world origin shifted {shift:4.1f} m: "
f"p - p' {100 * np.sqrt(np.mean(np.sum((Qs - Pa) ** 2, 1))):.1f} cm, "
f"eq. 23 {100 * np.sqrt(np.mean(np.square(e23))):.1f} cm")
Output:
ground truth: 2400 poses, 67.6 m, max tilt 1.2 deg
translational ATE (cm) / rotational ATE (deg)
4-DoF first SE(3) first 4-DoF all SE(3) all Sim(3) all
jitter only 1.9 / 0.35 2.0 / 0.45 1.7 / 0.34 1.7 / 0.34 1.7 / 0.34
yaw drift 0.05 deg/s 16.0 / 3.54 16.0 / 3.55 7.7 / 1.76 7.7 / 1.76 7.7 / 1.76
tilt drift 0.02 deg/s 4.1 / 1.43 3.6 / 1.21 3.9 / 1.43 2.5 / 0.78 2.5 / 0.78
constant tilt 1 deg 3.2 / 1.06 2.0 / 0.45 3.1 / 1.06 1.7 / 0.34 1.7 / 0.34
scale +2% 5.5 / 0.35 5.5 / 0.45 5.4 / 0.34 5.4 / 0.34 1.7 / 0.34
jitter only, seeds 0-9:
4-DoF first translational ATE 1.9-4.0 cm
4-DoF all translational ATE 1.7-1.8 cm
constant tilt, 4-DoF all, world origin shifted 0.0 m: p - p' 3.1 cm, eq. 23 3.5 cm
constant tilt, 4-DoF all, world origin shifted 10.0 m: p - p' 3.1 cm, eq. 23 28.2 cm
The two tilt rows show the main effect. A constant 1° gravity error displaces the estimate by up to a few centimetres across the room. The 4-DoF alignment cannot rotate it away, so the estimate is charged 3.1 cm and 1.06° instead of the 1.7 cm and 0.34° of jitter alone. SE(3) alignment rotates the whole trajectory back and reports exactly the jitter level, so a real attitude error disappears from both the translational and the rotational ATE. With tilt drift, SE(3) removes the mean tilt and charges only the variation around it. The same holds for scale: Sim(3) hides a 2% scale error that an IMU should have prevented. When the errors are only jitter and yaw drift, the three groups give the same values, because SE(3) and Sim(3) have nothing extra to absorb. A table that aligns some methods with SE(3) or Sim(3) and others with 4-DoF therefore compares like with like only when no method has an attitude or scale error, which is the case it is meant to detect.
The first-pose columns repeat the effect that Zhang and Scaramuzza measured on MH01 (below). Yaw drift costs about twice as much when the alignment uses only the first pose, because every later error is measured from that pose. The SE(3) first-pose alignment uses the first orientation, so it also forgives a constant tilt. The yaw-only first-pose alignment of their Eq. 20 does not. Over noise seeds 0–9 the first-pose ATE ranges from 1.9 to 4.0 cm against 1.7 to 1.8 cm for the all-pose fit, because it inherits the noise of a single pose. The last two lines show the code–paper difference: the tutorial’s Eq. 23 gives 3.5 cm with the original coordinates and 28.2 cm when the same ground truth is moved 10 m away from the origin, while the plain position difference does not change.
License
As of October 2026, the current dataset page states no licence. The ETH Research Collection record that hosts the files carries the rights statement In Copyright – Non-Commercial Use Permitted (InC-NC 1.0), in its DataCite metadata and on the archived June 2025 copy of the record page; the live record page refused automated access during this check. Under that statement, non-commercial use needs no permission from the rights holders, while other uses need their permission. The statement is not itself a licence. The former wiki page carried a footer placing the wiki’s own content under the Creative Commons Attribution-NonCommercial 4.0 licence “except where otherwise noted”. That footer covered the web pages, and it is unclear whether it was meant to cover the data. The terms for commercial use are therefore unclear, and users should ask the Autonomous Systems Lab.
The toolbox is under the MIT License.
Limitations and Bias
- Ground-truth accuracy during fast motion. The former wiki page notes that very dynamic motion is known to degrade the laser tracker’s accuracy. The manufacturer’s figures may then be optimistic, which complicates comparisons between very accurate methods. The start and end of each sequence, which have less motion, are assumed to be little affected. The current page lists this as a known issue.
- Synchronization. The sensor and ground-truth clocks were aligned by estimation, not by hardware (see above). It follows that a residual offset turns into a position error that grows with speed.
- Position-only ground truth in the machine hall. The Leica tracker gives no orientation. It follows that rotational errors in the MH sequences can only be measured against the processed estimate, not against a direct measurement.
- Independent exposure control. The two cameras change brightness independently, which makes stereo matching and feature tracking harder (official page).
- Missing frames. Campos et al. (2021) report that one camera drops frames in V203, which kept the stereo-inertial BASALT from completing that sequence.
- Small and dated. Schubert et al. (2018) wrote that EuRoC’s image resolution and bit depth were no longer state of the art. They also noted the limited number and variety of scenes, one machine hall and one lab room, the short sequences, and the lack of the photometric calibration that direct methods need, motivating the TUM VI benchmark.
- Saturation. With eleven short indoor sequences and results now at a few centimetres, average ATE differences between leading methods are small compared with the effect of the alignment choice shown in the example. In our reading this makes the convention as important as the method for any ranking.
Common Uses
- Benchmarking VIO and visual-inertial SLAM. Tables of RMS ATE per sequence, for MH01–MH05, V101–V103 and V201–V203, appear in most VIO papers. ORB-SLAM3’s Table II compares fifteen systems, ORB-SLAM3 included, in four sensor configurations (Campos et al., 2021).
- Visual-only methods. Stereo and monocular odometry and SLAM are evaluated on the images alone, with SE(3) or Sim(3) alignment. The visual odometry article cites DSO’s EuRoC results.
- Resource-constrained evaluation. Delmerico and Scaramuzza (2018) ran six VIO pipelines on a laptop, an Intel NUC and two embedded boards (UP Board and ODROID), playing the data back in real time, and recorded accuracy together with CPU load, memory and time per frame.
Historical Milestones
- 2016: release. The paper appeared online on 25 January 2016 and in print in September 2016 (IJRR 35(10)).
- 2018: benchmark of monocular VIO. Delmerico and Scaramuzza compared MSCKF, OKVIS, ROVIO, VINS-Mono (with and without loop closure), SVO+MSF and SVO+GTSAM on all eleven sequences and several computers. After Sim(3) alignment, translational RMSE across pipelines and platforms ranged from 3 cm to 2.28 m, and several pipeline–platform combinations failed on some sequences, most often V103 and V203 (Table I).
- 2018: alignment by observability. Zhang and Scaramuzza ran VINS-Mono on MH01 and aligned with the 4-DoF transformation. Fitting it on all 1,807 states gave 0.2795 m and 2.4935°, and fitting it on the first state only gave 0.4383 m. The translational ATE fell steadily as more initial states were used, while the rotational ATE showed no clear trend (Table III). They concluded that the states used for alignment must be the same for all compared methods and reported with the results.
- 2018: successor datasets. TUM VI was introduced with higher-resolution, photometrically calibrated images and longer, more varied sequences, citing EuRoC’s limitations (Schubert et al., 2018).
- 2021: centimetre level. Campos et al. reported an average RMS ATE of 3.5 cm for stereo-inertial ORB-SLAM3 over all eleven sequences (median of 10 runs per sequence, SE(3) alignment, processed ground truth) and 4.3 cm for monocular-inertial (Table II, arXiv version).
- 2024: archival hosting. The data moved to the ETH Research Collection with its own DOI (record dated 27 September 2024), and the dataset page now points there.
Related Datasets
- TUM RGB-D: handheld and robot-mounted Kinect sequences with motion-capture ground truth, from whose benchmark scripts the ATE and relative pose error come; it aligns in SE(3) because depth gives metric scale.
- TUM VI: a handheld stereo-inertial benchmark with hardware synchronization, 1024 × 1024 images, photometric calibration and motion capture at the start and end of most sequences (Schubert et al., 2018).
- KITTI odometry: stereo driving sequences with GPS/INS ground truth, evaluated with length-normalized relative errors on a server.
- ETH3D SLAM benchmark: RGB-D, stereo and visual-inertial sequences with a hidden test set and an online leaderboard.
Resources
- EuRoC MAV dataset page (Autonomous Systems Lab) and the ETH Research Collection record with the downloads.
- dataset_tools: the official MATLAB loader for the ASL format and the ROS bag re-stamping script.
- rpg_trajectory_evaluation: the University of Zurich toolbox with SE(3), Sim(3) and yaw-only alignment and EuRoC example configurations (third-party).
- evo: a package for trajectory evaluation that reads EuRoC ground-truth CSV files directly (third-party).
- No official leaderboard exists. Per-sequence results are compared in the papers that use the dataset, such as Table II of Campos et al. (2021).
Related
- Simultaneous Localization and Mapping (SLAM)
Building a map of an unknown environment while estimating the trajectory of the sensor moving through it, and using recognized revisits to keep both globally consistent.
- Camera Pose
The position and orientation of a camera as a rigid transform in SE(3), the two directions it is stored in (world-to-camera and camera-to-world), the axis conventions of OpenCV, COLMAP, OpenGL and NeRF, and how pose errors are measured.
- Camera Calibration
Estimating a camera's intrinsic parameters, lens distortion, and pose from images of a target with known geometry, usually by a closed-form initialization followed by minimizing reprojection error.
- Stereo Vision
Recovering depth from two images taken by cameras with a known relative pose, by measuring how far each point shifts between the views.
- KITTI
A benchmark suite of real driving data from Karlsruhe, with stereo, optical flow, scene flow, odometry, object detection, and tracking benchmarks built from camera, LiDAR, and GPS/IMU recordings.
- ETH3D
Laser-scanned indoor and outdoor scenes from ETH Zürich with high-resolution multi-view, low-resolution many-view and two-view stereo benchmarks, scored by accuracy, completeness and F1 at tolerances from 1 cm to 50 cm, plus an RGB-D SLAM benchmark.
References
- Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M. W. & Siegwart, R. (2016). The EuRoC Micro Aerial Vehicle Datasets. The International Journal of Robotics Research, 35(10), 1157–1163.
- Autonomous Systems Lab, ETH Zurich. EuRoC MAV Dataset (dataset page, accessed October 2026).
- Autonomous Systems Lab, ETH Zurich. The EuRoC MAV Dataset (former dataset wiki page, archived copy of December 2025, no longer maintained).
- Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M. W. & Siegwart, R. (2024). The EuRoC micro aerial vehicle datasets. ETH Zurich Research Collection, dataset record (rights statement and archives; DataCite metadata accessed October 2026).
- Autonomous Systems Lab, ETH Zurich. dataset_tools: loader for the ASL dataset format (GitHub repository, commit 8513020, 2016).
- Zhang, Z. & Scaramuzza, D. (2018). A Tutorial on Quantitative Trajectory Evaluation for Visual(-Inertial) Odometry. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 7244–7251.
- Robotics and Perception Group, University of Zurich. rpg_trajectory_evaluation (align_trajectory.py, align_utils.py, compute_trajectory_errors.py, trajectory.py, trajectory_loading.py, README and EuRoC example configurations; GitHub repository, commit 8c8ceec, 2022, MIT License).
- Grupp, M. (2017). evo: Python Package for the Evaluation of Odometry and SLAM. GitHub repository, version 1.38.1 (2026).
- Delmerico, J. & Scaramuzza, D. (2018). A Benchmark Comparison of Monocular Visual-Inertial Odometry Algorithms for Flying Robots. IEEE International Conference on Robotics and Automation (ICRA), 2502–2509.
- Schubert, D., Goll, T., Demmel, N., Usenko, V., Stückler, J. & Cremers, D. (2018). The TUM VI Benchmark for Evaluating Visual-Inertial Odometry. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 1680–1687.
- Campos, C., Elvira, R., Gómez Rodríguez, J. J., Montiel, J. M. M. & Tardós, J. D. (2021). ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM. IEEE Transactions on Robotics, 37(6), 1874–1890.