InterAct
[CVPR 2025] InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation
Install / Use
/learn @wzyabcas/InterActREADME
News
- [2025-04-20] Initial release of the InterAct dataset.
- [2025-07-08] Release the proessing code for unified SMPL-H representation.
- [2025-09-12] Publish the paper on arXiv.
- [2025-10-06] Release the hoi correction pipeline.
- [2025-10-06] Release the evaluation pipeline for text-to-hoi.
- [2025-10-29] Release corrected OMOMO data.
- [2025-11-23] Provide additional supports on ARCTIC and ParaHome.
- [2025-11-26] Release training code, pretrained model and evaluator checkpoints.
- [2025-11-26] Release augmentated data for InterAct-X.
- [2025-12-07] 🚀 Release the data conversion pipeline for bringing InterAct into simulation, specifically for InterMimic use.
- [2026-02-03] Release the pipeline for object-to-human.
- [2025-02-03] Release corrected OMOMO data V2.
TODO
- [x] Release comprehensive text descriptions, data processing workflows, visualization tools, and usage guidelines
- [x] Release the proessing code for unified SMPL-H representation
- [x] Publish the paper on arXiv
- [x] Release the evaluation pipeline for the benchmark
- [x] Release HOI correction pipeline
- [x] Release HOI correction data
- [x] Release augmentation data
- [x] Release baseline constructions for text2HOI.
- [x] Release the pipeline for constructing simulation ready data
- [x] Release baseline constructions for Object to Human
- [x] Release HOI correction data V2
- [ ] Release baseline constructions for the other HOI generative tasks
- [ ] Release the dataset with unified SMPL representation
- [ ] Release retargeted HOI dataset with unified human shape
General Description
We introduce InterAct, a comprehensive large-scale 3D human-object interaction (HOI) dataset, originally comprising 21.81 hours of HOI data consolidated from diverse sources, the dataset is meticulously refined by correcting contact artifacts and augmented with varied motion patterns to extend the total duration to approximately 30 hours. It includes 34.1K sequence-level detailed text descriptions.
Dataset Preparation
The InterAct dataset is consolidated according to the licenses of its original data sources. For data approved for redistribution, direct download links are provided; for others, we supply processing code to convert the raw data into our standardized format.
Please follow the steps below to download, process, and organize the data.
1. Request authorization
Please fill out this form to request non-commercial access to InterAct and InterAct-X. Once authorized, you'll receive the download links. Organize the data from NeuralDome, IMHD, CHAIRS, OMOMO, and its corrected and augmented data according to the following directory structure.
<details> <summary>Data structure</summary>data
│── neuraldome
│ ├── objects
│ │ └── baseball
│ │ ├── baseball.obj # object mesh
│ │ └── sample_points.npy # sampled object pointcloud
│ └── ...
│ ├── objects_bps
│ │ └── baseball
│ │ └── baseball.npy # static bps representation
│ └── ...
│ ├── sequences
│ │ └── subject01_baseball_0
│ │ ├── action.npy
│ │ ├── action.txt
│ │ ├── human.npz
│ │ ├── markers.npy
│ │ ├── joints.npy
│ │ ├── motion.npy
│ │ ├── object.npz
│ │ └── text.txt
│ └── ...
│ └── sequences_canonical
│ └── subject01_baseball_0
│ ├── action.npy
│ ├── action.txt
│ ├── human.npz
│ ├── markers.npy
│ ├── joints.npy
│ ├── motion.npy
│ ├── object.npz
│ └── text.txt
│ └── ...
│── imhd
│── chairs
│── omomo
└── annotations
2. Process from scratch
The GRAB, BEHAVE, INTERCAP, ARCTIC datasets are available for academic research under custom licenses from the Max Planck Institute for Intelligent Systems. Note that we do not distribute the original motion data—instead, we provide the processing code and annotations. Besides, we support ParaHome in addition to our original dataset. To download these datasets, please visit their respective websites and agree to the terms of their licenses:
<details> <summary>Licenses</summary> </details> <details> <summary>Please follow these steps to get started</summary>-
Download SMPL+H, SMPLX, DMPLs.
Download SMPL+H mode from SMPL+H (choose Extended SMPL+H model used in the AMASS project), DMPL model from DMPL (choose DMPLs compatible with SMPL), and SMPL-X model from SMPL-X. Then, please place all the models under
./models/. The./models/folder tree should be:models │── smplh │ ├── female │ │ ├── model.npz │ ├── male │ │ ├── model.npz │ ├── neutral │ │ ├── model.npz │ ├── SMPLH_FEMALE.pkl │ ├── SMPLH_MALE.pkl │ └── SMPLH_NEUTRAL.pkl └── smplx ├── SMPLX_FEMALE.npz ├── SMPLX_FEMALE.pkl ├── SMPLX_MALE.npz ├── SMPLX_MALE.pkl ├── SMPLX_NEUTRAL.npz └── SMPLX_NEUTRAL.pklPlease follow smplx tools to merge SMPL-H and MANO parameters.
-
Prepare Environment
-
Create and activate a fresh environment:
conda create -n interact python=3.8 conda activate interact pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118To install PyTorch3D, please follow the official instructions: Pytorch3D.
Install remaining packages:
pip install -r requirements.txt python -m spacy download en_core_web_sm bash install_human_body_prior.sh
- Prepare raw data
-
BEHAVE
Download the motion data from this link, and put them into ./data/behave/sequences. Download object data from this link, and put them into ./data/behave/objects.
Expected File Structure:
data/behave/ ├── sequences │ ├── data_name │ ├── object_fit_all.npz # object's pose sequences │ └── smpl_fit_all.npz # human's pose sequences └── objects └── object_name ├── object_name.jpg # one photo of the object ├── object_name.obj # reconstructed 3D scan of the object ├── object_name.obj.mtl # mesh material property ├── object_name_tex.jpg # mesh texture └── object_name_fxxx.ply # simplified object mesh -
OMOMO
Download the dataset from this link, and download the text annotations from this link.
Expected File Structure:
data/omomo/raw ├── omomo_text_anno_json_data # Annotation JSON data ├── captured_objects │ └── object_name_cleaned_simplified.obj # Simplified object mesh ├── test_diffusion_manip_seq_joints24.p # Test sequences └── train_diffusion_manip_seq_joints24.p # Train sequences -
InterCap
Dowload InterCap from the [the project website](https://intercap.is.tu
