Paper arXiv Summary

ReForce

Learning Force-aware Retargeting for Dexterous Manipulation

Yuhang Wu, Lingqi Zeng, Changwei Jing, Jianglong Ye†, Xiaolong Wang†

† Equal advising.

Abstract

Human demonstrations offer a scalable data source for dexterous manipulation, but transferring them to robot actions remains challenging due to the embodiment gap. Today's retargeting is mostly kinematic, yet manipulation is decided by force, which governs how the hand interacts with the object and how the object moves. In this paper, we present ReForce, a Force-aware Retargeting method that turns human motion and forces into robot actions that reproduce the intended contact. ReForce predicts a residual on the kinematically retargeted action to reach the desired force, using a general force tracker trained on large-scale simulation interactions. It supports both online force-aware teleoperation and offline data translation. In simulation and on real hardware, ReForce achieves lower force-tracking error and stronger multi-finger contact engagement on contact-rich tasks such as paper-cup grasping and tongs manipulation.

Spring Pinch (Teleoperation)

Board Wiping (ACT)

Finger Release (Teleoperation)

Cup Pouring (ACT)

Overview of ReForce from human data and tactile feedback to stable robot manipulation.
ReForce uses human tactile references as contact intent, then adapts robot hand commands through fingertip force feedback.
~131k simulated interaction episodes
68.2M training samples after data augmentation

Simulation Data Rollout

Method

Training and inference pipeline for ReForce.
During training, randomized simulated interactions provide future-window joint and force targets. During inference, real tactile feedback closes the loop.

Human-guided System

Human-guided data collection and robot execution setup.

The hardware stack supports two uses: online teleoperation and data collection for ACT training. In both settings, a Manus glove captures human hand motion, while fingertip FSR sensors measure per-finger contact force.

For teleoperation, a VR headset tracks the human wrist motion used to drive the robot arm. During ACT data collection, AprilTags are used instead to recover the human wrist pose, while the glove and fingertip force sensors provide the hand pose and force reference streams.

Experiments

ReForce is evaluated against direct position replay and admittance control. We also show its performance under teleoperation and learned-reference execution with ACT.

Force Curve Comparisons

Tongs Squeezing Comparison with ACT policy

Position Control

Admittance Control

ReForce

ReForce is a force-aware retargeting method for contact-rich dexterous manipulation. It adapts human-guided motion and contact references during execution, allowing dexterous hands to respond to missing or excessive contact rather than simply replaying nominal trajectories. Across replay and imitation-learning experiments, our results suggest that force-aware retargeting improves force tracking and multi-finger contact engagement on contact-sensitive manipulation tasks. These findings support force-aware retargeting as a useful execution interface between human-guided trajectory generation and real-world dexterous contact control.