SkillAgentSearch skills...

reference-verify

Verify references in an academic paper: check whether each BibTeX entry is real, whether in-text citations match the cited paper's actual content, and produce a structured verification report

Install / Use

npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill reference-verify

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

95/100

Supported Platforms

Universal

Tags

Our assessment of reference-verify

reference-verify scores 95/100 on our quality scale, 50th of 421 Education & Research skills we index (top 12%).

Its SKILL.md is 21 KB long, well organised into 51 sections with 9 code examples: a thorough specification that gives an agent plenty to work with.

With 4,360 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
20/20
Description
15/15
Adoption
15/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so reference-verify is actively maintained.
  • Our last check on 2026-11-15 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-29. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

reference-verify compared with similar skills

All 4 of these similar skills score higher than reference-verify; compare them before choosing.

SkillScoreStarsUpdatedFormat
reference-verify (this skill)by brycewang-stanford954.4k54d agoSKILL.md
last30days-skillby mvanhorn10068.0ktodayCLAUDE.md
x402by browser-use100121.0k3d agoSKILL.md
algorithmic-artby anthropics100177.9k55d agoSKILL.md
pptxby anthropics100177.9k55d agoSKILL.md

Frequently asked questions

How do I install reference-verify?
Run npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill reference-verify. The install tabs above show the steps for each supported agent.
Which AI agents does reference-verify work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is reference-verify safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is reference-verify still maintained?
The repository was last updated about 2 months ago, so reference-verify is actively maintained.

name: reference-verify description: "Verify references in an academic paper: check whether each BibTeX entry is real, whether in-text citations match the cited paper's actual content, and produce a structured verification report. Use when user says "验证参考文献", "ref verify", "check references", "核实引用", "引用是否正确", or wants to audit citations in a LaTeX manuscript." argument-hint: "[paper-directory-or-tex-file]" allowed-tools: Bash(*), Read, Write, Edit, Grep, Glob, Agent, WebSearch, WebFetch

Ref-Verify: 学术论文参考文献验证

Verify references for: $ARGUMENTS


Purpose

This skill performs a systematic audit of every reference cited in a LaTeX manuscript. For each citation it:

  1. Confirms the BibTeX entry describes a real, published paper (not hallucinated).
  2. Locates every place the reference is cited in the manuscript.
  3. Extracts the claim or viewpoint the author attributes to that reference.
  4. Judges whether the cited paper's actual content supports the attributed claim.
  5. Produces a structured verification report as a LaTeX table and a Markdown summary.

Constants

  • VERIFY_METHOD = web — How to verify references. Options: web (WebSearch + WebFetch for each entry), dblp (DBLP API first, WebSearch fallback), manual (skip online verification, only check internal consistency).
  • OUTPUT_FORMAT = excel — Output format. Options: excel (default, .xlsx with conditional formatting), markdown, latex, all (all three).
  • OUTPUT_DIR — Directory to save the report. Defaults to the same directory as the input .tex file.
  • LANGUAGE = zh — Report language. Options: zh (Chinese headers), en (English headers).
  • STRICT = false — When true, flag any citation where the attributed claim cannot be confirmed from the paper's abstract/title as 未确认. When false, give benefit of the doubt for plausible matches.
  • APPLY_FIXES = interactive — Whether to apply the suggested fixes after the report is generated. Options: interactive (default — show each proposed edit and ask user y/n/skip/edit), auto (apply every fix without asking; use only when user explicitly opts in), off (stop after the report; no edits to source files).

Inputs

The skill accepts one of:

  1. A directory containing .tex and .bib files (scans all .tex files for citations).
  2. A single .tex file (uses the \bibliography{} command to locate the .bib file).
  3. A .bib file alone (verifies entries exist but cannot check in-text citation context).

If no argument is provided, scan the current working directory.


Workflow

Step 1: Locate Files

  1. Find all .tex files in the target directory (or the single specified file).
  2. Find the .bib file(s) referenced by \bibliography{} or \addbibresource{}.
  3. Read and parse the .bib file to extract all BibTeX entries.
  4. Read all .tex files to extract all \cite{}, \citep{}, \citet{}, \citealt{}, \citealp{} commands.

Step 2: Build Citation Map

For each citation key found in the .tex files:

  1. Extract BibTeX metadata: author, title, journal, year, volume, pages.
  2. Locate all in-text occurrences: Record each file name, line number, and the surrounding sentence (±1 sentence for context).
  3. Extract attributed claim: From the surrounding text, identify what viewpoint, finding, or argument the author attributes to the cited work.

Produce an internal data structure:

[
  {
    "seq": 1,
    "key": "baker2006investor",
    "bib": { "author": "Baker, Malcolm and Wurgler, Jeffrey", "title": "Investor sentiment...", "journal": "Journal of Finance", "year": "2006" },
    "occurrences": [
      { "file": "sections/1_introduction.tex", "line": 12, "context": "...investor sentiment predicts cross-sectional stock returns...", "claim": "investor sentiment predicts cross-sectional returns" }
    ]
  },
  ...
]

Step 3: Verify Each Reference

For each BibTeX entry, perform verification in order:

3a. Existence Check (Is the paper real?)

Use this verification chain (stop at first success):

Method A: DBLP (if VERIFY_METHOD includes dblp)

# Search DBLP by title + first author
curl -s "https://dblp.org/search/publ/api?q=TITLE+FIRST_AUTHOR&format=json&h=3"
# If match found: VERIFIED

Method B: WebSearch

WebSearch: "EXACT_TITLE" author FIRST_AUTHOR_LASTNAME journal JOURNAL_NAME

Check the top 3 results. A paper is VERIFIED if:

  • The title matches (exact or near-exact, allowing minor capitalization/punctuation differences).
  • The author(s) match.
  • The journal/venue matches.
  • The year matches (±1 year tolerance for working paper → publication lag).

A paper is UNVERIFIED if no matching result is found after both methods.

Method C: CrossRef DOI (if a DOI is in the bib entry)

WebFetch: https://doi.org/DOI

3b. Metadata Accuracy Check

If the paper is verified, compare the BibTeX metadata against the verified source:

  • Is the year correct?
  • Is the journal name correct?
  • Are the volume and pages correct?
  • Are all authors listed correctly?

Flag any discrepancies.

3c. Citation Context Check (Is the claim supported?)

For each occurrence of the citation in the manuscript:

  1. Read the sentence(s) surrounding the citation.
  2. Identify the specific claim attributed to the cited paper.
  3. Determine whether the cited paper's title, abstract, and known content support that claim.

Classification:

  • 支撑 (Supported): The claim is clearly consistent with the cited paper's known content.
  • 部分支撑 (Partially supported): The claim is related but overstated, narrowed, or slightly reframed.
  • 不支撑 (Not supported): The claim does not match the cited paper's content.
  • 未确认 (Unconfirmed): Cannot determine from available information (title/abstract only).
  • 论文不存在 (Paper not found): The BibTeX entry could not be verified as a real publication.

Step 4: Generate Report

The default output is an Excel report (ref_verify_report.xlsx). Use Python + openpyxl to generate.

Excel Report (ref_verify_report.xlsx) — DEFAULT

The Excel workbook contains three sheets, structured as follows:

Sheet 1: 总览 (Overview)

| 项目 | 数值 | |------|------| | 论文标题 | [paper title] | | 验证日期 | [date] | | 验证方法 | [web/dblp/manual] | | 总引用数 | X | | 验证通过 | Y | | 存疑 | Z | | 不存在 | W | | 未引用条目 | U |

Sheet 2: 已验证文献 (Verified References)

This sheet covers references that can be found online. Focus is on citation–claim support analysis.

| 列 | 说明 | |----|------| | 序号 | 按文中首次出现顺序 | | 参考文献 | 作者 (年份), 期刊缩写 | | BibTeX Key | cite key | | 文中引用位置 | 文件名:行号,多处用分号分隔 | | 引用观点 | 作者在引用处表达的核心观点(≤30字) | | 支撑判定 | 支撑 / 部分支撑 / 不支撑 / 未确认 | | 支撑分析 | 详细说明:原文实际说了什么 vs 引用处声称了什么,差异在哪 | | 元数据问题 | 年份、期刊名、作者拼写等元数据错误(无则留空) | | 修改建议 | 针对不支撑/部分支撑/元数据错误的具体修改建议 |

Conditional formatting for this sheet:

  • 支撑 → row background light green (#C6EFCE)
  • 部分支撑 → row background light yellow (#FFEB9C)
  • 不支撑 → row background light red (#FFC7CE)
  • 未确认 → row background light gray (#D9D9D9)

Sheet 3: 问题文献 (Problematic References)

This sheet covers references that cannot be found or have serious issues. Each entry gets a detailed problem diagnosis.

| 列 | 说明 | |----|------| | 序号 | 按文中首次出现顺序 | | 参考文献 | BibTeX 中的作者 (年份), 期刊 | | BibTeX Key | cite key | | 文中引用位置 | 文件名:行号 | | 引用观点 | 作者在引用处表达的核心观点 | | 问题类型 | 见下方分类 | | 问题详情 | 详细描述搜索过程和发现 | | 可能原因 | 分析此条目出问题的原因 | | 修改建议 | 具体修改方案(替换文献/修正信息/删除引用) | | 严重程度 | 🔴 高 / 🟡 中 / 🟢 低 |

问题类型分类 (fill in "问题类型" column):

  • 论文不存在: 搜索不到任何匹配结果,可能是虚构文献
  • 作者不匹配: 找到同名论文但作者不同
  • 期刊/年份不匹配: 论文存在但发表期刊或年份与 bib 条目不符
  • 标题错误: 找到相似论文但标题有明显差异
  • 疑似混淆: 多篇相似论文,无法确定引用的是哪一篇
  • 预印本/未发表: 仅在 arXiv/SSRN 等找到,未正式发表
  • 孤立条目: bib 中存在但正文从未引用

Conditional formatting for this sheet:

  • 严重程度 🔴 高 → entire row bold red background (#FFC7CE), bold font
  • 严重程度 🟡 中 → row background light orange (#FFD9B3)
  • 严重程度 🟢 低 → row background light blue (#BDD7EE)

Excel Generation Code Pattern

Use this Python pattern via Bash to generate the xlsx:

import openpyxl
from openpyxl.styles import Font, PatternFill, Alignment, Border, Side
from openpyxl.utils import get_column_letter

wb = openpyxl.Workbook()

# --- Sheet 1: Overview ---
ws1 = wb.active
ws1.title = "总览"
# Write overview stats...

# --- Sheet 2: Verified References ---
ws2 = wb.create_sheet("已验证文献")
headers2 = ["序号", "参考文献", "BibTeX Key", "文中引用位置", "引用观点",
            "支撑判定", "支撑分析", "元数据问题", "修改建议"]
# Write headers with bold + background
# Write rows, apply conditional fill per 支撑判定 value
# Auto-adjust column widths

# --- Sheet 3: Problematic References ---
ws3 = wb.create_sheet("问题文献")
headers3 = ["序号", "参考文献", "BibTeX Key", "文中引用位置", "引用观点",
            "问题类型", "问题详情", "可能原因", "修改建议", "严重程度"]
# Write headers, rows, apply severity-based fills
# Auto-adjust column widths

# Global styling
for ws in [ws1, ws2, ws3]:
    ws.sheet_properties.pageSetUpPr = openpyxl.worksheet.properties.PageSetupProperties(fitToPage=True)
    ws.freeze_panes = "A2"  # Freeze header row

wb.save("OUTPUT_DIR/ref_verify_report.xlsx")

Important: Before running, ensure openpyxl is installed: pip install openpyxl

Markdown Report (ref_verify_report.md) — only when OUTPUT_FORMAT includes markdown or all

# 参考文献验证报告

**论文**: [paper title]
**验证日期**: [date]
**验证方法**: [web/dblp/manual]
**总引用数**: X | **验证通过**: Y | **存疑**: Z | **不存在**: W

## 一、已验证文献

| 序号 | 参考文献 | 文中引用位置 | 引用观点 | 支撑判定 | 支撑分析 | 修改建议 |
|------|----------|-------------|----------|----------|----------|----------|
| 1 | Baker and Wurgler (2006), JF | 1_intro.tex:L12 | investor sentiment predicts returns | ✅ 支撑 | 与原文主要发现一致 | — |

## 二、问题文献

| 序号 | 参考文献 | 问题类型 | 问题详情 | 可能原因 | 修改建议 | 严重程度 |
|------|----------|----------|----------|----------|----------|----------|
| 5 | Smith (2020), AER | 论文不存在 | 搜索Google Scholar/DBLP均无结果 | 可能为AI生成的虚构文献 | 删除该引用或替换为... | 🔴 高 |

## 三、未引用条目
...

## 四、总结与建议
...

LaTeX Report (ref_verify_report.tex) — only when OUTPUT_FORMAT includes latex or all

Generate a LaTeX longtable with the same two-section structure (已验证 + 问题文献).

Step 5: Summary and Recommendations

After the table, provide:

  1. Overall assessment: How many references pass verification, how many are flagged.
  2. Critical issues: Any references that appear fabricated or seriously misattributed.
  3. Minor issues: Metadata errors (wrong year, wrong journal name, missing pages).
  4. Recommendations: Specific fixes for each flagged entry.

Step 6: Apply Recommended Modifications

The verification report is only useful if its "修改建议" column actually reaches the source files. Step 6 walks every actionable suggestion back into the .tex and .bib files. Skip this step only when APPLY_FIXES = off.

Why this matters. Authors typically have 30–100 citations. Hand-applying every suggested fix from the report is tedious and error-prone — they will silently skip the "small" ones (a wrong year, an orphan entry) and the manuscript ships with the same defects. Closing the loop here is the difference between a report and a real audit.

6a. Build the fix queue

Re-use the in-memory verification data from Steps 2–3 — do not re-parse the xlsx. Build an ordered queue of rows that need action:

  1. All rows from Sheet 3 (问题文献) — every entry, regardless of severity.
  2. Rows from Sheet 2 (已验证文献) where 支撑判定 = 部分支撑 — the claim is overstated, narrowed, or reframed.

Rows where 支撑判定 ∈ {支撑, 未确认} are not in the queue. 不支撑 rows belong on Sheet 3 already (move them there if the report put them on Sheet 2). Process the queue in 序号 order so the user sees fixes flow in the same order as the manuscript.

6b. For each queued row, propose a concrete edit

The "修改建议" column says what to do at the policy level (e.g. "替换为 Baker and Wur

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars4.4k
CategoryEducation
Updated1mo ago
Forks527

Languages

Stata

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium