Developing humanoid robots capable of complex object interactions requires coordination of whole-body balance, locomotion and precise hand manipulation to control the movement of both the robot and the object. Human demonstrations provide examples of such coordinated interactions, but transferring these skills to robots requires learning to establish and maintain effective contact across different embodiments and dynamics. Weave is introduced as a unified framework for teaching humanoid robots complex object interactions using recorded human demonstrations.
How the Weave Framework Works
Weave first converts recorded human–object interactions into reference motions that a robot can execute through contact-oriented retargeting and approach motion completion. At the heart of the system is a contact- and geometry-aware policy that jointly controls 29 body joints and 12 finger actuators across multiple objects and interaction sequences.
Test Results and Available Data
Evaluation on nine objects showed a success rate of 92,5% for trained interactions. Without additional training, the system achieved a success rate of 65,0% on sequences not encountered during training. The researchers also released around 9000 runs performed on physical hardware, covering approximately 23 hours of data. These data include robot–object trajectories with contact annotations, intended for further training of interaction policies and the generation of physically consistent motions in human–object interaction (HOI). The project has a website: https://xiaohu-art.github.io/Weave/.
Source: cs.RO updates on arXiv.org
