🔬 Research Highlights

DexCanvas pipeline from human motion capture to MANO hand reconstruction and physics simulation

DexCanvas [Paper] [Code] [Dataset]

DexCanvas introduces a real-to-simulation framework for reconstructing human hand–object interactions. We organize demonstrations around 21 grasp and in-hand manipulation types based on the Cutkosky taxonomy. Reinforcement learning reproduces these interactions in physics simulation, where we extract physically consistent contact forces and synthesize variations of the demonstrated motions. Combining the original demonstrations with these simulated rollouts yields a dataset of manipulation trajectories and contact annotations for robot learning.

Simulated unicycle swarmalators navigating through obstacles using local control

Navigation of Robotic Swarmalators [Paper] [Code] [Video]

We adapt the swarmalator model to robots with omnidirectional, unicycle, and bicycle dynamics, examining how motion constraints reshape collective behavior. Combining swarmalator planning with control barrier functions enables global and per-agent control for navigation through cluttered environments and object transport around obstacles.

MagFlow magnetic millirobot collectives reconfiguring between shapes and separating into eight color groups

MagFlow [Paper: Under review]

MagFlow controls collectives of passive magnetic millirobots through a shared electromagnetic field. Using camera-based density feedback, optimal transport guides the swarm toward target shapes, while a constrained control allocator converts the desired motion into feasible electromagnet commands. The system enables reset-free shape reconfiguration and separation of multiple groups. These capabilities are verified in both simulation and hardware experiments.

📄 Publications

^* Equal contribution.

2026

Nonreciprocal Swarmalators With Reconfigurable and Controllable Formations for Robot Collectives

Kush Patel^*, Xinyue Xu^*, Wei Xiao, and Steven Ceron.
Advanced Robotics Research, e70161, 2026.
Paper

3D Robotic Swarmalators That Reconfigure, Navigate, and Avoid Obstacles

Zehui Xu^*, Xinyue Xu^*, and Steven Ceron.
IEEE International Conference on Robotics and Automation (ICRA), 2026.
Paper

Xinyue Xu, Wei Xiao, and Steven Ceron.
IEEE Robotics and Automation Letters (RA-L), 11(2), 1066–1073, 2026.
Paper

2025

DexCanvas: Bridging Human Demonstrations and Robot Learning for Dexterous Manipulation

Xinyue Xu, Jieqiang Sun, Jing (Daisy) Dai, Siyuan Chen, Lanjie Ma, Ke Sun, Bin Zhao, Jianbo Yuan, Sheng Yi, Haohua Zhu, and Yiwen Lu.
arXiv preprint, arXiv:2510.15786, 2025.
Paper · Code · Dataset

Xinyue Xu 徐新越