A Survey on the Application of Large Language Model in Scenario-Based Testing of Automated Driving Systems

</div> <div align="center">

</div>

📢 News

[Dec 2025] 🎉 Our survey paper "A Survey on the Application of Large Language Models in Scenario-Based Testing of Automated Driving Systems" has been accepted for publication in IEEE Transactions on Intelligent Transportation Systems (T-ITS)!

Objective

This repository accompanies the survey paper "A Survey on the Application of Large Language Models in Scenario-Based Testing of Automated Driving Systems." It provides a continuously updated collection of academic and industrial studies on LLM applications across all phases of scenario-based ADS testing. Each study is summarized in structured tables with publication details, code links, and brief descriptions.

We welcome community participation. You can:

📝 Nominate new papers for inclusion
Use this Google Form to suggest recent studies for addition to the repository.
🧩 Open an Issue
Report errors, propose improvements, or suggest new features through the GitHub Issues page.
🔄 Submit a Pull Request
Contribute directly by updating tables, adding missing references, or refining documentation.

For direct communication, please contact:
📧 yongqi.zhao@tugraz.at or dong.bi@tugraz.at

1. Related Survey
2. Terminology
- Scenario
- Scenario Abstraction Level
3. LLM Applications by Phase
4. License

Usage

The repository categorizes and lists all references following the logical structure of the review paper, as shown in the Contents section of the last chapter. By selecting the topic of interest, you can easily navigate to the cutting-edge papers included within each module. The papers are presented in a tabular format consisting of five columns, as illustrated below: the first column shows the paper title, the second specifies the journal or conference, the third indicates the publication date, the fourth provides information about any available open-source resources, and the fifth offers a concise summary of the paper. <img width="1016" height="217" alt="image" src="https://github.com/user-attachments/assets/7c7d20e1-bbe3-4da7-8a5e-57f7208bab29" />

Citation

If you think this repo is useful, please cite our paper:

@ARTICLE{11361285,
  author={Zhao, Yongqi and Zhou, Ji and Bi, Dong and Mihalj, Tomislav and Hu, Jia and Eichberger, Arno},
  journal={IEEE Transactions on Intelligent Transportation Systems}, 
  title={A Survey on the Application of Large Language Models in Scenario-Based Testing of Automated Driving Systems}, 
  year={2026},
  volume={},
  number={},
  pages={1-23},
  keywords={Testing;Surveys;Terminology;Large language models;Trajectory;Taxonomy;Scenario generation;Hazards;Costs;Cognition;Generative AI;simulation test;safety assessment;automated vehicle;literature review},
  doi={10.1109/TITS.2026.3651004}}

1. Related Survey

Survey of LLM

| Title | Venue | Date | Code | Summary | |:--------|:--------:|:--------:|:--------:|:--------:| | A Comprehensive Overview of Large Language Models | arxiv.org | 2023-7 | - | This article offers a concise yet comprehensive overview of recent advancements in Large Language Models (LLMs), covering core concepts and cutting-edge topics to help researchers and practitioners navigate the rapidly evolving LLM landscape and foster further innovation in the field. | | A survey on large language models: Applications, challenges, limitations, and practical usage | arxiv.org | 2024-1 | GitHub | Hadi et al. offers a comprehensive overview of LLMs, covering their history, architecture, train-ing methods, applications, challenges, and future directions to support their development and real-world deployment. | | MM-LLMs: Recent Advances in MultiModal Large Language Models | arxiv.org | 2023 | Project | Zhang et al. systematically analyze 126 multimodal LLMs (MLLMs), with detailed comparison of model design and benchmark performances. | | Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models | arxiv.org | 2025-3 | Project | Su et al. review efficient reasoning strategies, categorizing them into model-based, output-based, and prompt-based approaches. | | A systematic survey of prompt engineering in large language models | arxiv.org | 2024 | - | Sahoo et al. provide a systematic overview of approaches used in prompt engineering, covering their methodologies, applications, models used and the datasets associated with each approach. offer a comprehensive survey on prompt engineering, including an evaluation of each methodology and a discussion on AI security, with a focus on vulnerability in prompt engineering. | | Unleashing the potential of prompt engineering in large language models: a comprehensive review | arxiv.org | 2023-10 | - | Chen et al. offer a comprehensive survey on prompt engineering, including an evaluation of each methodology and a discussion on AI security, with a focus on vulnerability in prompt engineering. | | Differentially private natural language models: Recent advances and future directions | arxiv.org | 2023-1 | - | Hu et al. review the use of differential privacy in natural language models. | | Privacy issues in large language models: A survey | arxiv.org | 2023 | GitHub | Neel and Chang present a survey on privacy issues in LLM more broadly. | | Privacy Preserving Prompt Engineering: A Survey | ACM Computing Surveys | 2025-5 | - | Edemacu and Wu focus specifically on privacy concerns related to in in-context learning and prompting mechanisms. |

Survey of Scenario-Based Testing

| Title | Venue | Date | Code | Summary | |:--------|:--------:|:--------:|:--------:|:--------:| | Scenario based testing of automated driving systems: A literature survey | FISITA web Congress | 2020 | - | This paper surveys 86 recent studies on scenario generation and evaluation methods for testing and validating Automated Driving Systems (ADS), highlighting the importance of simulation-based approaches to address safety, cost, and complexity challenges, and propos

LLM4ADSTest

Install / Use

README