Data collection
UMI (Universal Manipulation Interface)
A hand-held gripper with a camera that lets people record robot training data anywhere, without a robot present.
How it works
A person carries a 3D-printed gripper with a GoPro and does the task by hand in real kitchens, offices or shops. The camera and gripper record what a robot would see and do. Because the gripper looks the same when mounted on a robot, a policy trained on this data (the authors use Diffusion Policy) can run on the real arm. The point is to collect varied data cheaply, so the resulting policy generalizes to new places and objects.
What it needs
A ~$300 hand-held gripper build, a GoPro, people willing to collect demonstrations; several hundred to a few thousand episodes for generalization. Robot needs a matching gripper and wrist camera.
Strengths
- Collect data without tying up a robot
- Policies generalized to unseen environments in the paper
- Hardware design and code are open
Limits
- Hardware-specific gripper on the robot
- Dynamic and bimanual tasks reached only ~70–88% success
- Latency matching between device and robot needs care
Where it has been shown to work (2)
| Task | Robot | Maturity | Success | n | Grade | Year |
|---|---|---|---|---|---|---|
| Cable and deformable handling | UR5e | Lab | 70% | 20 | B | 2024 |
| Tabletop benchmark tasks | UR5e | Lab | 100% | 20 | B | 2024 |