OpenVLA

How it works

Take a vision-language model (the kind that answers questions about pictures), and train it to output robot actions as tokens instead of words, using the Open X-Embodiment dataset of ~970,000 episodes from many robots. The result follows instructions like 'put the eggplant in the pot' on robots it has seen in training. For a new robot or task you fine-tune it on 10–150 demonstrations; low-rank adapters make that feasible on a single consumer GPU.

What it needs

Fine-tuning: 10–150 demonstrations per task and one GPU (LoRA). Inference needs a serious GPU; the released model runs at ~5–15 Hz. Works best when your robot resembles ones in Open X-Embodiment.

Strengths

• Open weights and training code
• Strong zero-shot results on robots in its training data
• Fine-tunes on small datasets

Limits

• Slow control rate for a 7B model
• Precision is limited; not suited to tight insertion as released
• Robots outside the pre-training mix need fine-tuning