Rigid-body ready Embodied / Robotics Mixed (check per asset) Open

RH20T Contact-Rich Multi-Sensor Robot Manipulation Dataset

Provider：MVIG Lab, Shanghai Jiao Tong University (Hao-Shu Fang et al.)

RH20T provides 110,000+ contact-rich manipulation sequences over 147 tasks, fusing vision, force, torque, audio and fingertip tactile sensing, with per-subset CC BY-SA and non-commercial licensing.

Get it from the source →

At a glance

Readiness
L1 · Rigid-body ready
Rationale
Manipulation L1: more than 110,000 real contact-rich manipulation sequences fuse RGB, depth, binocular infrared, joint angles and torques, 6-DoF force/torque, audio and fingertip tactile sensing, directly usable for vision-tactile policy learning; but the assets are real-world multimodal logs rather than simulation-loadable 3D models, hence L1 as a real behavior data layer, consistent with how BridgeData V2 and RLBench are graded.
Scale
More than 110,000 real contact-rich manipulation sequences over 147 tasks (48 RLBench, 29 MetaWorld and 70 self-proposed), across 7 robot configurations and 4 arm types (The official documentation states that each task contains millions of paired human-demonstration and robot-manipulation records, with the native release at roughly 40TB; a 640x360 resolution version is also provided, expanding to about 5TB for RGB and 10TB for RGBD, with depth images potentially degraded by compression. The task count follows the official split of 48 + 29 + 70 = 147. Checked 2026-10-26.)
License
Mixed (check per asset) — The dataset carries two licenses by subset: RH20T-C covering scene_0001 through scene_0005 is CC BY-SA 4.0, while RH20T-NC covering scene_0006 through scene_0010 is described in the official site text as CC BY-NC 4.0, with explicit prohibition on commercial use of that subset or of models trained on it. The hyperlink attached to that paragraph actually points to creativecommons.org/licenses/by-nc-sa/4.0/, which does not match the CC BY-NC 4.0 wording, so the authors should be asked whether the NC subset carries a ShareAlike obligation before use. The site also notes the dataset is partly funded by a company, hence the mixed licensing.
Access
Open
Formats
other
Runtimes
other
Physics properties
Collision geometryJoint structureSensor attributes
Domain
Other
Published
2026-10-26
Updated
2026-10-26

Why it is listed

RH20T was built around a gap the authors identified in imitation-learning data: most available demonstrations cover simple push and pick-place tasks under vision alone, while many real skills need both visual and tactile perception. The result is a dataset of over 110,000 contact-rich sequences collected in the real world, each paired with a corresponding human demonstration video so that one-shot imitation can transfer a learned policy to new tasks.

Readiness assessment

Each platform carries a force-torque-sensored arm, a gripper, one or two in-hand cameras, eight to ten global RGBD cameras and two microphones, all calibrated against the robot base frame and synchronized in time. The modality table is unusually broad for a public release: RGB, depth and binocular infrared at 10 Hz, joint angles and joint torques at 10 Hz, gripper Cartesian pose and 6-DoF force/torque at 100 Hz, audio at 30 Hz, and fingertip tactile arrays at 200 Hz on configuration 7 only. The authors also publish a parsing and visualization API and a rendered point-cloud view showing the robot model placed in the scene.

Practical notes

Licensing is split by subset: the commercial-use RH20T-C subset (scene_0001 to scene_0005) is CC BY-SA 4.0, while RH20T-NC (scene_0006 to scene_0010) is non-commercial and bars commercial use of models trained on it, with a link on the site that points to a ShareAlike variant rather than the stated plain non-commercial term, so clarify with the authors before building on that half. The dataset was contributed by volunteers, and the official notice asks that potentially sensitive footage be handled carefully and used only for model training. Prefer the resized 640x360 release for iteration, since the native release is around 40TB, and treat depth values in the compressed version as approximate.

Disclaimer: license and usage terms follow the official source — verify them yourself before use. This site indexes and links only; it does not host or mirror any asset file.

License and usage terms are those of the official source — verify before use. No asset file is hosted or mirrored on this site.