UMI Universal Manipulation Interface (Handheld Gripper) In-the-Wild Dataset
Provider：Stanford, Columbia and Toyota Research Institute (Cheng Chi et al.)
UMI uses a handheld gripper plus GoPro to collect in-the-wild human demos and reconstructs robot-agnostic action trajectories via SLAM; the official Hub totals about 774GB under MIT.
At a glance
- Readiness
- L1 · Rigid-body ready
- Rationale
- Manipulation L1: UMI is a handheld parallel-jaw gripper plus GoPro interface for in-the-wild data collection, recording human handheld demos as video, IMU and gripper state, then reconstructing robot-agnostic action trajectories via SLAM for Diffusion-Policy training and cross-platform (UR5/Franka) deployment; it is real data rather than simulation-loadable 3D models, hence L1 as a real behavior data layer, consistent with DROID and BridgeData V2.
- Scale
- Task-organized in-the-wild demos: cup arrangement (cup_in_the_wild), dynamic tossing, bimanual cloth folding, bimanual dish washing, lab cups and more; the official Hub release ships SLAM-reconstructed zarr replay buffers totalling about 774GB across 26,565 files, while secondary catalogs list around 2,543 trajectories (UMI has no single fixed total: the official Hugging Face repository (real-stanford/universal_manipulation_interface) is organized per task as zarr.zip replay buffers totalling about 774GB, whereas a secondary catalog (openbot.ai) totals around 2,543 trajectories; these are 'files/replay-buffers' versus 'task trajectories' bases and are not reconciled. Checked 2026-10-28.)
- License
- MIT — The official project repository (real-stanford/universal_manipulation_interface) carries an MIT LICENSE (Copyright 2023 Columbia Artificial Intelligence and Robotics Lab), covering code and dataset organization; the SLAM reconstruction pipeline (an ORB-SLAM3 fork) ships with the same repository. Defer to the official repository license before use.
- Access
- Open
- Formats
- other
- Runtimes
- other
- Physics properties
- Sensor attributesJoint structure
- Domain
- Other
- Published
- 2026-10-28
- Updated
- 2026-10-28
Why it is listed
Universal Manipulation Interface (UMI) is an open-source data-collection interface shaped as a handheld parallel-jaw gripper with a wrist-mounted GoPro, designed by researchers from Stanford, Columbia and the Toyota Research Institute. An operator holds the gripper and performs manipulation while the system records video, inertial data and gripper state, then reconstructs deployable robot actions from in-the-wild human demonstrations without needing the robot present during capture.
Readiness assessment
UMI pairs a carefully designed policy interface with inference-time latency matching and a relative-trajectory action representation, so the learned policies are hardware-agnostic and deployable across platforms such as UR5e and Franka as long as the parallel jaw stroke exceeds 85mm. The release includes a SLAM pipeline (an ORB-SLAM3 fork optimized for the workflow) that turns raw GoPro recordings into zarr replay buffers the training code reads, making it a practical path from in-the-wild human video to deployable Diffusion-Policy checkpoints.
Practical notes
The official repository license is MIT, covering code and dataset organization. Because only a wrist camera is used, UMI is calibration-free and robust to base movement, distractors and lighting changes, but SLAM success is the fragile step: the authors report about 99 percent of raw data yielding usable reconstructions under careful collection. The data is organized per task (cup_in_the_wild, dynamic_tossing, bimanual_cloth_folding, bimanual_dish_washing and lab cups), and the full official Hub release is about 774GB, so plan download and storage before training.
Disclaimer: license and usage terms follow the official source — verify them yourself before use. This site indexes and links only; it does not host or mirror any asset file.
License and usage terms are those of the official source — verify before use. No asset file is hosted or mirrored on this site.