EasyDetect
[ACL 2024] An Easy-to-use Hallucination Detection Framework for LLMs.
Install / Use
/learn @zjunlp/EasyDetectREADME
An Easy-to-Use Multimodal Hallucination Detection Framework for MLLMs
$\color{red}{It's\ unfortunate\ that\ due\ to\ limited\ computational\ resources,\ we\ have\ suspended\ the\ online\ demo.}$ $\color{red}{If\ you'd\ like\ to\ try\ the\ demo,\ please\ contact\ sunnywcx @zju.edu.cn\ or\ zhangningyu @zju.edu.cn}$
<p align="center"> <a href="#citation">🌻Acknowledgement</a> • <a href="https://huggingface.co/datasets/openkg/MHaluBench">🤗Benchmark</a> • <a href="http://easydetect.zjukg.cn/">🍎Demo</a> • <a href="#overview">🌟Overview</a> • <a href="#modelzoo">🐧ModelZoo</a> • <a href="#installation">🔧Installation</a> • <a href="#quickstart">⏩Quickstart</a> • <a href="#citation">🚩Citation</a> <!-- <a href="#contributors">🎉Contributors</a> --> </p>
Table of Contents
- <a href="#acknowledgement">🌻Acknowledgement</a>
- <a href="#overview">🌟Overview</a>
- <a href="#unified-multimodal-hallucination">Unified Multimodal Hallucination </a>
- <a href="#dataset-mhallubench-statistic">Dataset: MHalluBench Statistic</a>
- <a href="#framework-uniHD-illustration">Framework: UniHD Illustration</a>
- <a href="#modelzoo">🐧ModelZoo</a>
- <a href="#installation">🔧Installation</a>
- <a href="#quickstart">⏩Quickstart</a>
- <a href="#citation">🚩Citation</a>
🔔News
News
- 2024-05-17 The paper Unified Hallucination Detection for Multimodal Large Language Models is accepted by ACL 2024 main conference.
- 2024-04-21 We replace all the base models in the demo with our own trained models, significantly reducing the inference time.
- 2024-04-21 We release our open-source hallucination detection model HalDet-LLAVA, which can be downloaded in huggingface, modelscope and wisemodel.
- 2024-02-10 We release the EasyDetect demo.
- 2024-02-05 We release the paper:"Unified Hallucination Detection for Multimodal Large Language Models" with a new benchmark MHaluBench! We are looking forward to any comments or discussions on this topic :)
- 2023-10-20 The EasyDetect project has been launched and is under development.
🌻Acknowledgement
Part implementation of this project were assisted and inspired by the related hallucination toolkits including FactTool, Woodpecker, and others. This repository also benefits from the public project from mPLUG-Owl, MiniGPT-4, LLaVA, GroundingDINO, and MAERec . We follow the same license for open-sourcing and thank them for their contributions to the community.
🌟Overview
EasyDetect is a systematic package which is proposed as an easy-to-use hallucination detection framework for Multimodal Large Language Models(MLLMs) like GPT-4V, Gemini, LlaVA in your research experiments.
Unified Multimodal Hallucination
Unified View of Detection
A prerequisite for unified detection is the coherent categorization of the principal categories of hallucinations within MLLMs. Our paper superficially examines the following Hallucination Taxonomy from a unified perspective:
<p align="center"> <img src="figs/view.png" width="60%" height="60%"> <img src="figs/intro.png" width="60%" height="60%"> </p>Figure 1: Unified multimodal hallucination detection aims to identify and detect modality-conflicting hallucinations at various levels such as object, attribute, and scene-text, as well as fact-conflicting hallucinations in both image-to-text and text-to-image generation.
Modality-Conflicting Hallucination. MLLMs sometimes generate outputs that conflict with inputs from other modalities, leading to issues such as incorrect objects, attributes, or scene text. An example in above Figure (a) includes an MLLM inaccurately describing an athlete's uniform , showcasing an attribute-level conflict due to MLLMs' limited ability to achieve fine-grained text-image alignment.
Fact-Conflicting Hallucination. Outputs from MLLMs may contradict established factual knowledge. Image-to-text models can generate narratives that stray from the actual content by incorporating irrelevant facts, while text-to-image models may produce visuals that fail to reflect the factual knowledge contained in text prompts. These discrepancies underline the struggle of MLLMs to maintain factual consistency, representing a significant challenge in the domain.
Fine-grained Detection Task Definition
Unified detection of multimodal hallucination necessitates the check of each image-text pair a={v, x}, wherein v denotes either the visual input provided to an MLLM, or the visual output synthesized by it. Correspondingly, x signifies the MLLM's generated textual response based on v or the textual user query for synthesizing v. Within this task, each x may contain multiple claims, denoted as ${c_i}_{i = 1 \cdots n}$. The objective for hallucination detectors is to assess each claim from a to determine whether it is "hallucinatory" or "non-hallucinatory", providing a rationale for their judgments based on the provided definition of hallucination. Text hallucination detection from LLMs denotes a sub-case in this setting, where v is null.
Dataset: MHalluBench Statistic
To advance this research trajectory, we introduce the meta-evaluation benchmark MHaluBench, which encompasses the content from image-to-text and text-to-image generation, aiming to rigorously assess the advancements in multimodal halluci- nation detectors. Further statistical details about MHaluBench are provided in below Figures.
<img src="figs/datasetinfo.jpg">Table 1: A comparison of benchmarks with respect to existing fact-checking or hallucination evaluation. "Check." indicates verifying factual consistency, "Eval." denotes evaluating hallucinations generated by different LLMs, and its response is based on different LLMs under test, while "Det." embodies the evaluation of a detector’s capability in identifying hallucinations.
<p align="center"> <img src="figs/饼图.png" width="40%" height="40%"> </p>Figure 2: Claim-Level data statistics of MHaluBench. "IC" signifies Image Captioning and "T2I" indicates Text-to-Image synthesis, respectively.
<p align="center"> <img src="figs/条形图.png" width="50%" height="50%"> </p>Figure 3: Distribution of hallucination categories within hallucination-labeled claims of MHaluBench.
Framework: UniHD Illustration
Addressing the key challenges in hallucination detection, we introduce a unified framework in Figure 4 that systematically tackles multimodal hallucination identification for both image-to-text and text-to-image tasks. Our framework capitalizes on the domain-specific strengths of various tools to efficiently gather multi-modal evidence for confirming hallucinations.
<img src="figs/framework.png">Figure 4: The specific illustration of UniHD for unified multimodal hallucination detection.
🐧ModelZoo
You can download two versions of HalDet-LLaVA, 7b and 13b, on three platforms: HuggingFace, ModelScope, and WiseModel. | HuggingFace | ModelScope | WiseModel | | ----------- | ----------- | ----------- | | HalDet-llava-7b | HalDet-llava-7b | HalDet-llava-7b | | HalDet-llava-13b | HalDet-llava-13b | HalDet-llava-13b |
The claim level results on validation dataset
- Self-Check(GPT-4V) means use GPT-4V with 0 or 2 cases
- UniHD(GPT-4V/GPT-4o) means use GPT-4V/GPT-4o with 2-shot and tool information
- HalDet (LLAVA) means use LLAVA-v1.5 trained on our train datasets
