EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration

Project Page | Paper

This project contains the implementation of our RA-L paper.

Authors: Linghao Chen, Yuzhe Qin, Xiaowei Zhou, Hao Su.

<img src="docs/pipeline_v5.png" width = 100% div align=left />

Update

• (2025.8.10) EasyHeC is pip installable and integrated into ManiSkill now! See here for details. Thanks to Stone Tao for the contribution!
• (2023.7.11) New prompt drawer for SAM has been supported! Checkout the newest code for more accurate results without training the PointRend!
• (2023.7.13) Franka Emika robot has been supported! See here for details. Thanks to Minghao Liu for the contribution!

Requirements

• Ubuntu 18.04+
• Python >3.7
• Nvidia GPU with mem >= 10G
• PyTorch 1.11.0+
• GCC<10 (may vary depending on your cuda version)

Install

See install.md

Usage

See usage.md

Citation

If you find this code useful for your research, please use the following BibTeX entry.

@article{chen2023easyhec,
  title={EasyHec: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration},
  author={Chen, Linghao and Qin, Yuzhe and Zhou, Xiaowei and Su, Hao},
  journal={IEEE Robotics and Automation Letters (RA-L)}, 
  year={2023}
}
@article{hong2024easyhec++,
  title={Fully Automatic Hand-Eye Calibration with Pretrained Image Models},
  author={Hong, Zhengdong and Zheng, Kangfu and Chen, Linghao},
  journal={International Conference on Intelligent Robots and Systems (IROS)},
  year={2024}
}