<h2 align="center">HccePose (BF)</h2> <p align="center"> <a href="https://arxiv.org/abs/2510.10177"> <img src="https://img.shields.io/badge/arXiv-2510.10177-B31B1B.svg?logo=arxiv" alt="arXiv"> </a> <a href="https://huggingface.co/datasets/SEU-WYL/HccePose"> <img src="https://img.shields.io/badge/HuggingFace-HccePose-FFD21E.svg?logo=huggingface&logoColor=white" alt="HuggingFace"> </a> </p> <p align="center"> <a href="./README.md">English</b> | <a href="./README_CN.md">中文</a> </p>

🧩 Introduction

HccePose(BF) introduces a Hierarchical Continuous Coordinate Encoding (HCCE) mechanism that encodes the three coordinate components of object surface points into hierarchical continuous codes. Through this hierarchical encoding scheme, the neural network can effectively learn the correspondence between 2D image features and 3D surface coordinates of the object, while significantly enhancing its capability to learn accurate object masks. Unlike traditional methods that only learn the visible front surface of objects, HccePose(BF) additionally learns the 3D coordinates of the back surface, thereby establishing denser 2D–3D correspondences and substantially improving pose estimation accuracy.

<img src="/show_vis/fig2.jpg" width=100%>

✨ Update

⚠️ Note: All paths must be absolute paths to avoid runtime errors.
2025.10.27: We’ve released cc0textures-512, a lightweight alternative to the original 44GB CC0Textures library — now only 600MB! 👉 Download here
2025.10.28: s4_p1_gen_bf_labels.py has been updated. If the dataset does not contain a camera.json, the script will automatically create a default one.

🔧 Environment Setup

<details> <summary>Configuration Details</summary>

Download the HccePose(BF) Project and Unzip BOP-related Toolkits

# Clone the project
git clone https://github.com/WangYuLin-SEU/HCCEPose.git
cd HCCEPose

# Unzip toolkits
unzip bop_toolkit.zip
unzip blenderproc.zip

Configure Ubuntu System Environment (Python 3.10)

⚠️ A GPU driver with EGL support must be pre-installed.

apt-get update && apt-get install -y wget software-properties-common gnupg2 python3-pip

apt-get update && apt-get install -y libegl1-mesa-dev libgles2-mesa-dev libx11-dev libxext-dev libxrender-dev

pip install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu118

apt-get update && apt-get install pkg-config libglvnd0 libgl1 libglx0 libegl1 libgles2 libglvnd-dev libgl1-mesa-dev libegl1-mesa-dev libgles2-mesa-dev cmake curl ninja-build

pip install ultralytics==8.3.70 fvcore==0.1.5.post20221221 pybind11==2.12.0 trimesh==4.2.2 ninja==1.11.1.1 kornia==0.7.2 open3d==0.19.0 transformations==2024.6.1 numpy==1.26.4 opencv-python==4.9.0.80 opencv-contrib-python==4.9.0.80

pip install scipy kiwisolver matplotlib imageio pypng Cython PyOpenGL triangle glumpy Pillow vispy imgaug mathutils pyrender pytz tqdm tensorboard kasal-6d rich h5py

pip install bpy==3.6.0 --extra-index-url https://download.blender.org/pypi/

apt-get install libsm6 libxrender1 libxext-dev

python -c "import imageio; imageio.plugins.freeimage.download()"

pip install -U "huggingface_hub[hf_transfer]"

</details>

🧱 Custom Dataset and Training

🎨 Object Preprocessing

<details> <summary>Click to expand</summary>

Using the demo-bin-picking dataset as an example, we first designed the object in SolidWorks and exported it as an STL mesh file.
STL file link: 🔗 https://huggingface.co/datasets/SEU-WYL/HccePose/blob/main/raw-demo-models/multi-objs/board.STL

Then, the STL file was imported into MeshLab, and surface colors were filled using the Vertex Color Filling tool.

After coloring, the object was exported as a non-binary PLY file containing vertex colors and normals.

The exported model center might not coincide with the coordinate origin, as shown below:

To align the model center with the origin, use the script s1_p1_obj_rename_center.py.
This script loads the PLY file, aligns the model center, and renames it following BOP conventions.
The obj_id must be set manually as a unique non-negative integer for each object.
Example:

| input_ply | obj_id | output_ply | | :---: | :---: | :---: | | board.ply | 1 | obj_000001.ply | | board.ply | 2 | obj_000002.ply |

After centering and renaming all objects, place them into a folder named models with the following structure:

Dataset_Name
|--- models
      |--- obj_000001.ply
      ...
      |--- obj_000015.ply

</details>

🌀 Rotational Symmetry Analysis

<details> <summary>Click to expand</summary>

In 6D pose estimation tasks, many objects exhibit various types of rotational symmetry, such as cylindrical, conical, or polyhedral symmetry. For such objects, the KASAL tool is used to analyze and export symmetry priors in BOP format.

KASAL project: 🔗 https://github.com/WangYuLin-SEU/KASAL

Installation:

pip install kasal-6d

Launch the KASAL GUI with:

from kasal.app.polyscope_app import app
mesh_path = 'demo-bin-picking'
app(mesh_path)

KASAL automatically scans all PLY or OBJ files under mesh_path (excluding generated _sym.ply files).

In the interface:

Use Symmetry Type to select the symmetry category
For n-fold pyramidal or prismatic symmetry, set N (n-fold)
Enable ADI-C for texture-symmetric objects
If the result is inaccurate, use axis xyz for manual fitting

KASAL defines 8 symmetry types. Selecting the wrong one will result in visual anomalies, helping verify your choice.

Click Cal Current Obj to compute the object’s symmetry axis. Symmetry priors will be saved as:

Symmetry prior file: obj_000001_sym_type.json
Visualization file: obj_000001_sym.ply

</details>

🧾 Generating BOP-Format Model Information

<details> <summary>Click to expand</summary>

Run s1_p3_obj_infos.py to traverse all ply files and their symmetry priors in the models folder. This script generates a standard models_info.json file in BOP format.

Example structure:

Dataset_Name
|--- models
      |--- models_info.json
      |--- obj_000001.ply
      ...
      |--- obj_000015.ply

This file serves as the foundation for PBR rendering, YOLOv11 training, and HccePose(BF) model training.

</details>

🔥 Rendering the PBR Dataset

<details> <summary>Click to expand</summary>

Based on BlenderProc, we modified a rendering script — s2_p1_gen_pbr_data.py — for generating new datasets. Running this script directly in Python may cause a memory leak, which accumulates over time and gradually degrades rendering performance. To address this issue, we provide a Shell script — s2_p1_gen_pbr_data.sh — that repeatedly invokes s2_p1_gen_pbr_data.py, effectively preventing memory accumulation and improving efficiency. Additionally, several adjustments were made to BlenderProc to better support PBR dataset generation for new object sets.

Preparation Before Rendering

Before rendering, use s2_p0_download_cc0textures.py to download the CC0Textures material library.
After downloading, the directory structure should look like this:

HCCEPose
|--- s2_p0_download_cc0textures.py
|--- cc0textures

The cc0textures library occupies about 44GB of disk space, which is quite large. To reduce storage requirements, we provide a lightweight alternative called cc0textures-512, with a size of approximately 600MB. Download link: 👉 https://huggingface.co/datasets/SEU-WYL/HccePose/blob/main/cc0textures-512.zip

When running the rendering script, simply replace the cc0textures path with cc0textures-512 to use the lightweight material library. (It is sufficient to download only cc0textures-512; the original cc0textures is not required.)

Running the Renderer

The s2_p1_gen_pbr_data.py script is responsible for PBR data generation, and it is adapted from BlenderProc2.

Run the following commands:

cd HCCEPose
chmod +x s2_p1_gen_pbr_data.sh
nohup ./s2_p1_gen_pbr_data.sh 0 42 xxx/xxx/cc0textures xxx/xxx/demo-bin-picking xxx/xxx/s2_p1_gen_pbr_data.py > s2_p1_gen_pbr_data.log 2>&1 &

Folder Structure

After executing the above process, the program will:

Use materials from xxx/xxx/cc0textures;
Load 3D object models from xxx/xxx/demo-bin-picking/models;
Generate 42 folders, each containing 1000 PBR-rendered frames, under xxx/xxx/demo-bin-picking.

The resulting structure will be:

demo-bin-picking
|--- models
|--- train_pbr
      |--- 000000
      |--- 000001
      ...
      |--- 000041

</details>

🚀 Training the 2D Detector

<details> <summary>Click to expand</summary>

In 6D pose estimation tasks, a 2D detector is typically used to locate the object’s bounding box, from which cropped image regions are used for 6D pose estimation. Compared with directly regressing 6D poses from the entire image, the two-stage approach (2D detection → 6D pose estimation) offers better accuracy and stability. Therefore, **

HCCEPose

Install / Use

README

🧩 Introduction

<img src="/show_vis/fig2.jpg" width=100%>

✨ Update

🔧 Environment Setup

🧱 Custom Dataset and Training

🎨 Object Preprocessing

🌀 Rotational Symmetry Analysis

🧾 Generating BOP-Format Model Information

🔥 Rendering the PBR Dataset

Preparation Before Rendering

Running the Renderer

🚀 Training the 2D Detector