EEMTL: ℰffective and ℰfficient ℳulti-𝒯ask ℒearning for Dense Scene Understanding 🌇🌆🏙️ [ TPAMI2025, ACMMM2025 ]

:scroll: Introduction

This repository presents two complementary multi-task learning frameworks for dense scene understanding: BridgeNet, which achieves comprehensive feature interactions via bridge features across tasks, and HiTTs, which discovers effective supervision from partially annotated data using hierarchical task tokens. Task including:

| Task | NYUD-v2 | PASCAL-Context | |------------------------------|:---------:|:--------------:| | Semantic Segmentation | ✔️ | ✔️ | | Depth Estimation | ✔️ | - | | Surface Normal Estimation | ✔️ | ✔️ | | Human Parts Segmentation | - | ✔️ | | Saliency Estimation | - | ✔️ | | Edge Detection | ✔️ | ✔️ |

Please check the following papers for details:

🌉 BridgeNet: Comprehensive and Effective Feature Interactions via Bridge Feature for Multi-task Dense Predictions (paper)

Jingdong Zhang, Jiayuan Fan, Peng Ye, Bo Zhang, Hancheng Ye, Baopu Li, Yancheng Cai and Tao Chen

TPAMI 2025

<p align="center"> <img alt="img-name" src="BridgeNet/assets/teaser.png" width="900"> </p> <p align="center"> <img alt="img-name" src="BridgeNet/assets/compare_show.png" width="900"> </p>

🏷️ Multi-Task Label Discovery via Hierarchical Task Tokens for Partially Annotated Dense Predictions (paper)

Jingdong Zhang, Hanrong Ye, Xin Li, Wenping Wang and Dan Xu

ACM MM 2025

<p align="center"> <img alt="img-name" src="HiTTs/assets/vis_label_init.png" width="900"> </p>

📝 TODO

• ✅ May 27, 2025: Release code of BridgeNet.
• ✅ May 27, 2025: Release pretrain weight of BridgeNet.
• ✅ Nov 24, 2025: Release code of HiTTs.
• ✅ Release pretrain weight of HiTTs.

🛠️ Implemtation

Please refer to README files in BridgeNet and HiTTs for detailed implementation instructions.

Cite

BibTex:

@article{zhang2025bridgenet,
  title={BridgeNet: Comprehensive and Effective Feature Interactions via Bridge Feature for Multi-Task Dense Predictions},
  author={Zhang, Jingdong and Fan, Jiayuan and Ye, Peng and Zhang, Bo and Ye, Hancheng and Li, Baopu and Cai, Yancheng and Chen, Tao},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2025},
  publisher={IEEE}
}
@article{zhang2024multi,
  title={Multi-task label discovery via hierarchical task tokens for partially annotated dense predictions},
  author={Zhang, Jingdong and Ye, Hanrong and Li, Xin and Wang, Wenping and Xu, Dan},
  journal={arXiv preprint arXiv:2411.18823},
  year={2024}
}

Please do consider :star2: star our project to share with your community if you find this repository helpful!

Contact

Please contact Jingdong Zhang if any questions.

Related Project

SPGen: This project proposes SPGen, leveraging multi-layer Spherical Projection (SP) maps as a consistent and flexible representation to efficiently generate high-quality, topologically free 3D meshes from a single input image.

PDT: This project proposes Point Distribution Transformation (PDT) leveraging diffusion models to transform unstructured point clouds into semantically meaningful distributions for diverse 3D geometry processing tasks.