CodeLLM Paper <a href=https://github.com/PurCL/CodeLLMPaper><img src='https://img.shields.io/github/stars/PurCL/CodeLLMPaper' width="120" height="26" /></a>

This repository provides a curated list of research papers focused on Large Language Models (LLMs) for code. It aims to facilitate researchers and practitioners in exploring the rapidly growing body of literature on this topic. The papers are systematically collected from various top-tier venues, categorized, and labeled for easier navigation.

Table of Contents

• A. Venues
• B. Selection Strategy
• C. Taxonomy
  • C.1. Application
  • C.2. Principle
  • C.3. Research Paradigm
• D. How to Contribute
  • D.1. PR Submission
  • D.2. Issue Submission
  • D.3. Request for Batch Updates
• E. Disclaimer and Contract

A. Venues

We have systematically selected papers from the following venues, which are top-tier conferences and journals in SE/PL/Sec/NLP communities.

• Software Engineering (SE)

  • ICSE2023, FSE2023, ASE2023, ISSTA2023, TSE2023, TOSEM2023
  • ICSE2024, FSE2024, ASE2024, ISSTA2024, TSE2024, TOSEM2024
  • ICSE2025, FSE2025, ISSTA2025

• Programming Languages (PL)

  • PLDI2023, OOPSLA2023
  • OOPSLA2024
  • PLDI2025, POPL2025, OOPSLA2025, CC2025

• Security (Sec)

  • S&P2023, USENIXSec2023, CCS2023, NDSS2023
  • S&P2024, USENIXSec2024, NDSS2024, CCS2024
  • S&P2025, NDSS2025

• Natural Language Processing (NLP)

  • ACL2023, EMNLP2023, NAACL2023
  • ACL2024, EMNLP2024, NAACL2024
  • NAACL2025

• Machine Learning (ML)

  • ICML2023, NeurIPS2023, ICLR2023
  • ICML2024, NeurIPS2024, ICLR2024
  • ICML2025, ICLR2025

Due to the large volume, we do not systematically collect the papers published in top-tier ML conferences (ICML, NeurIPS, and ICLR) and arXiv. However, we are keeping manually adding important works published in these venues. We plan to expand the collection over time, and contributions are welcome. For details, see the section How to Contribute.

B. Selection Strategy

1. Abstract Extraction: Extract the abstracts from bib files or HTML files. The bib and HTML files of the above listed venues are stored in the directory data/rawdata.

2. Keyword Matching: Filter abstracts that meet both of the following conditions:

   • Contains at least one keyword from: {"pretrain", "LLM", "large language model", "transformer", "code model"}.

   • Contains the keyword "code" or "program".

3. Relevance Check Using LLMs: Use LLMs to verify if the papers obtained in Step 2 are related to LLMs for code.

4. Manual Labeling: Manually assign labels to the papers based on domain knowledge.

All the selected papers along with the labels are maintained in the json file data/labeldata/labeldata.json. src/process.py is the python script used for selecting and labeling papers.

C. Taxonomy

The papers in this repository are categorized along three dimensions: Application, Principle, and Research Paradigm. Each paper is assigned multiple labels based on these categories. Note that categories are not necessarily disjoint.

C.1. Application

This category focuses on typical tasks in Software Engineering (SE) and Programming Languages (PL).

• General Coding Task (37)
• Code Generation (270)
  • Program Synthesis (108)
  • Code Completion (25)
  • Program Repair (70)
  • Program Transformation (42)
• Program Testing (99)
  • General Testing (7)
  • Fuzzing (31)
  • Library Testing (5)
  • DBMS Testing (1)
  • Compiler Testing (5)
  • GUI Testing (1)
  • Protocol Fuzzing (1)
  • Mutation Testing (2)
  • Unit Testing (12)
  • Differential Testing (6)
  • Debugging (16)
  • Bug Reproduction (6)
  • Vulnerability Exploitation (12)
• Static Analysis (208)
  • Syntactic Analysis (1)
  • Pointer Analysis (3)
  • Call Graph Analysis (4)
  • Data-flow Analysis (8)
  • Symbolic Execution (1)
  • Abstract Interpretation (2)
  • Type Inference (7)
  • Specification Inference (21)
  • Equivalence Checking (2)
  • Code Similarity Analysis (8)
  • Bug Detection (105)
  • Program Verification (28)
  • Program Optimization (8)
  • Program Decompilation (12)
  • Code Summarization (17)
  • Code Search (8)
  • Software Composition Analysis (3)
• Software Maintenance and Deployment (29)
  • Code Review (9)
  • Documentation Generation (4)
  • Commit Message Generation (5)
  • Software Configuration (1)
  • System Log Analysis (4)

C.2. Principle

This category concentrates on the LLMs' ability in understanding different forms of code and the non-functional properties of the LLMs (e.g., security and robustness). We also consider how to utilize the LLMs for general reasoning problems, such as typical agent-centric designs and specific PL designs for LLMs.

• Code Model (1)
  • Code Model Training (4)
    • Source Code Model (66)
    • IR Code Model (5)
    • Binary Code Model (15)
  • Code Model Security (33)
  • Code Model Robustness (11)
• Hallucination In Reasoning (16)
• PL Design For LLMs (3)
• Agent Design (72)
  • Prompt Strategy (44)
    • Retrieval-augmented Generation (14)
    • Reason With Code (17)
    • [Samp