citation-audit
Zero-context verification that every bibliographic entry in the paper is real, correctly attributed, and used in a context the cited paper actually supports — catching hallucinated authors, wrong years, fabricated venues, version mismatches, and wrong-context citations
Install / Use
npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill citation-auditInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Customer SupportSupported Platforms
Our assessment of citation-audit
citation-audit scores 98/100 on our quality scale, 7th of 188 Customer Support skills we index (top 4%).
Its SKILL.md is 30 KB long, well organised into 43 sections with 10 code examples: a thorough specification that gives an agent plenty to work with.
With 16,644 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated 9 days ago, so citation-audit is actively maintained.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
citation-audit compared with similar skills
All 4 of these similar skills score higher than citation-audit; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| citation-audit (this skill)by wanshuiyin | 98 | 16.6k | 9d ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 85.8k | 12d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 74.0k | 1d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 73.4k | today | CLAUDE.md |
| CowAgentby zhayujie | 100 | 47.1k | today | CLAUDE.md |
Frequently asked questions
- How do I install citation-audit?
- Run
npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill citation-audit. The install tabs above show the steps for each supported agent. - Which AI agents does citation-audit work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is citation-audit safe to use?
- It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is citation-audit still maintained?
- The repository was last updated 9 days ago, so citation-audit is actively maintained.
Skill content
View source on GitHubname: citation-audit description: "Zero-context verification that every bibliographic entry in the paper is real, correctly attributed, and used in a context the cited paper actually supports — catching hallucinated authors, wrong years, fabricated venues, version mismatches, and wrong-context citations. Use when user says "审查引用", "check citations", "citation audit", "verify references", "引用核对", or before submission to ensure bibliography integrity." argument-hint: "[paper-directory-or-bib-file] [--uncited] [— soft-only]" allowed-tools: Bash(*), Read, Grep, Glob, Edit, Write, mcp__codex__codex, WebSearch, WebFetch
Citation Audit
🔒 Do not wrap this skill in
/loop,/schedule, orCronCreate. It is verdict-bearing — it judges bibliographic correctness. Re-running that verdict on a timer adds no new signal (it changes only when the bibliography changes). Schedule the external wait that precedes it — bibliography finalized → then audit once. Seeshared-references/external-cadence.md.
Verify every \cite{...} in a paper against three independent layers:
- Existence — the cited paper actually exists at the claimed arXiv ID / DOI / venue.
- Metadata correctness — author names, year, venue, and title match canonical sources (DBLP, arXiv, ACL Anthology, Nature, OpenReview, etc.).
- Context appropriateness — the cited paper actually supports the claim it is being used to support in the manuscript.
This skill is the fourth layer of \aris{}'s evidence-and-claim assurance, complementing experiment-audit (code), result-to-claim (science verdict), and paper-claim-audit (numerical claims). Together they form a bottom-up integrity stack from raw evaluation code to manuscript bibliography.
When to Use This Skill
Run before submission. The right gating point is:
- After
paper-writehas produced the LaTeX draft and bib file - After
paper-claim-audithas verified numerical claims - Before final
paper-compilefor submission
Do not run this on a half-written draft — most of the work is in cross-checking each \cite against context, which is wasted on placeholder text.
What This Skill Catches
The dangerous citation problems are not wildly fake citations — those are easy to spot. The dangerous ones are:
- Wrong-context citations: real paper, but the cited claim is not what that paper actually establishes (e.g., citing Self-Refine to support "self-feedback produces correlated errors" — Self-Refine actually argues the opposite).
- Author hallucinations: anonymous-author placeholders that slipped through, missing co-authors, wrong order.
- Title drift: arXiv v1 vs v3 with different titles silently merged.
- Venue confusion: arXiv preprint cited but the official venue is now CVPR/ICML/NeurIPS — using the wrong record.
- Year mismatch: arXiv 2023 preprint with 2024 conference acceptance, year reported inconsistently.
- Phantom DOIs: DOI looks real but does not resolve.
- Self-citation drift: your own prior work cited with year off by one.
Constants
- REVIEWER_MODEL =
gpt-6-astra— Used via Codex MCP. Default for cross-model review with web access. - CONTEXT_POLICY =
fresh— Each audit run uses a new reviewer thread (REVIEWER_BIAS_GUARD). Nevercodex-reply. - WEB_SEARCH = required — The reviewer must perform real web/DBLP/arXiv lookups, not pattern-match from memory.
- OUTPUT =
CITATION_AUDIT.md— Human-readable per-entry verdict report. - STATE =
CITATION_AUDIT.json— Machine-readable verdict ledger consumable by downstream tools. - SOFT_ONLY =
false— When true (set via— soft-only/— soft_onlyflag), the audit runs all three layers normally but forbids any.bibfile mutation. Findings that would otherwise mutate the bib (FIX / REPLACE / REMOVE) are translated into per-occurrence sentence-rewrite proposals against the citing*.texfiles. Used by/resubmit-pipelinePhase 1 to honor the user's hard "freeze the bib" constraint. - RENDER_HTML = true — When
true(default), auto-renderCITATION_AUDIT.mdto HTML after writing the report. Uses full Codex review gate (audit-class artifact — render-fidelity check matches the skill's cross-model audit invariant). Setfalseto skip, or pass— render html: false.
Workflow
Step 1: Discover bib file and section files
Locate:
references.bib(orpaper.bib/ similar) under the paper directory- All
*.texfiles containing\cite{...}calls (typicallysec/orsections/)
If multiple bib files exist, audit each separately.
Step 2: Extract all (cite-key, context) pairs
For each \cite{key1,key2,...} invocation in the paper:
- Record the cite key
- Record the file + line number
- Record the surrounding sentence (≥ 1 full sentence around the cite, for context check)
Output a flat list of (key, file, line, surrounding_sentence) tuples.
Also build the inverse: for each bib entry, the list of all places it is cited.
Define two protocol sets used throughout the rest of the workflow: cited_keys is the set of unique cite keys appearing in any \cite{...} invocation across the audited *.tex files (de-duplicated), and bib_keys is the set of keys parsed from the audited bib file(s). cited_keys drives Step 3 (audit only cited entries); bib_keys \ cited_keys is the uncited residual surfaced by the --uncited opt-in.
If the user passed --uncited, also compute the set difference bib_keys \ cited_keys here and stash it for use in Steps 5 and the JSON aggregation; see "Uncited Entry Detection (opt-in)" below for the protocol. The set-diff is a string operation only and does not consume reviewer budget.
Save the extracted contexts to paper/.aris/citation-audit/contexts.txt so the reviewer can read it directly. Use the paper-dir-relative path .aris/citation-audit/contexts.txt when recording the file in audited_input_hashes; do not stage under /tmp or other transient locations that the verifier cannot rehash later.
Step 3: Send each entry to fresh cross-model reviewer
For each cited bib entry — i.e., each key in cited_keys with at least one extracted citation context — invoke mcp__codex__codex (NOT codex-reply — fresh thread per entry, or batch with explicit per-entry isolation). Do not send entries in bib_keys \ cited_keys to the reviewer; those are detect-only and surface only when --uncited is explicitly enabled (see "Uncited Entry Detection" below).
mcp__codex__codex:
model: gpt-6-astra
config: {"model_reasoning_effort": "xhigh"}
sandbox: read-only
prompt: |
You are auditing a bibliographic entry. Use web/DBLP/arXiv search.
## Bib entry
@article{key2024example,
author = {...}, title = {...}, journal = {...}, year = {...}, ...
}
## Where this entry is cited in the paper
[paste extracted contexts]
For this entry, verify:
1. EXISTENCE: does this paper exist at the claimed arXiv ID / DOI / venue?
Output: YES / NO / UNCERTAIN, with the verifying URL.
2. METADATA: are author names, year, venue, title correct?
For each, output: correct / wrong: should be ... / typo: ...
3. CONTEXT: for each use, does the cited paper actually support the surrounding claim?
Output per-use: SUPPORTS / WEAK / WRONG, with one-sentence reasoning.
VERDICT: KEEP / FIX / REPLACE / REMOVE
- KEEP: entry is clean, all uses are appropriate
- FIX: metadata needs correction; uses are appropriate
- REPLACE: cite is wrong-context, find a different paper that actually supports the claim
- REMOVE: entry is hallucinated or unsupportable
Be honest. If you cannot verify online, say UNCERTAIN; do not guess.
Save the response to .aris/traces/citation-audit/<date>_runNN/<key>.md per the review-tracing protocol.
Step 4: Aggregate verdicts
Build CITATION_AUDIT.json following the schema defined in "Submission
Artifact Emission" below (single authoritative schema for this file).
Per-entry ledger data goes under details.per_entry, not under a
top-level entries field. The top-level verdict is a single overall
value (PASS / WARN / FAIL / NOT_APPLICABLE / BLOCKED / ERROR) derived
from per-entry verdicts per the decision table in "Submission Artifact
Emission"; the top-level summary is a one-line human-readable string.
Concretely, details carries the per-entry ledger:
"details": {
"total_entries": 29,
"counts": { "KEEP": 11, "FIX": 14, "REPLACE": 3, "REMOVE": 1 },
"per_entry": [
{
"key": "lu2024aiscientist",
"verdict": "KEEP",
"axis_failures": [],
"uses": [
{"file": "sections/1.intro.tex", "line": 11, "verdict": "SUPPORTS"},
{"file": "sections/6.related.tex", "line": 8, "verdict": "SUPPORTS"}
]
},
{
"key": "madaan2023selfrefine",
"verdict": "FIX",
"axis_failures": ["CONTEXT"],
"uses": [
{"file": "sections/2.overview.tex", "line": 42, "verdict": "WRONG",
"note": "Self-Refine demonstrates iterative improvement, not correlated errors"},
{"file": "sections/6.related.tex", "line": 13, "verdict": "SUPPORTS"}
]
}
]
}
See "Submission Artifact Emission" for the full artifact (top-level
fields audit_skill, verdict, reason_code, summary,
audited_input_hashes, trace_path, thread_id, reviewer_model,
reviewer_reasoning, generated_at, details).
Step 5: Generate human-readable report
Write CITATION_AUDIT.md:
# Citation Audit Report
**Date**: 2026-04-19
**Bib file(s)**: references.bib
**Total entries**: 29
## Summary
| Verdict | Count |
|---------|------|
| KEEP | 11 |
| FIX | 14 |
| REPLACE | 3 |
| REMOVE | 1 |
## Priority Fixes (CRITICAL — apply before submission)
### REMOVE: anon2025placeholder
- Author listed as "Anonymous" — canonical record exists with real authors and full title
- Title is incomplete
- ACTION: Replace key with the canonical citekey, update authors and title
### REPLACE-CONTEXT: example2023priorwork in sec/2.overview.tex:42
- Cited to support a specific technical claim
- The cited paper actually demonstrates a different (related but distinct) phenomenon
- ACTION: Rewrite the sentence; cite the prior work for what it actually establishes
[... continues for each entry ...]
## All-Clean Entries (no action needed)
[list of KEEP keys]
When --uncited is set, append the following section after "All-Clean Entries":
## Uncited Entries (opt-in)
The following bib entries are present in the audited bib file(s) but are not referenced by any `\cite{...}` in the paper body:
- `author2010example` — suggestion: prune (uncited; no local evidence of intent)
- `someone2015othercite` — suggestion: prune (uncited; no local evidence of intent)
- `third2024todo` — suggestion: check (a `% TODO: cite third2024todo` comment was found in `sections/3.related.tex`)
This section is detect-only; it does not change the top-level verdict.
Step 6: Apply fixes (interactive)
For each FIX/REPLACE/REMOVE verdict, prompt the user:
Fix [key]?
Change: <description of change>
Files affected: references.bib + sec/X.tex:Y
[Apply / Skip / Defer]
If AUTO_APPLY = true, apply all FIX-level changes (metadata corrections only). REPLACE and REMOVE always require human approval — they involve content changes.
Step 7: Recompile and verify
latexmk -C && latexmk -pdf -interaction=nonstopmode main.tex
Confirm:
- No new
Citation undefinedwarnings - No
Reference undefinedwarnings - Page count unchanged or only minimally affected by metadata fixes
Uncited Entry Detection (opt-in)
Default: disabled. Existing users see no behavior change — only \cite{...} keys are audited, and bib entries with no \cite reference in the manuscript are silent
Truncated for display — read the full file on GitHub.
Related Skills
Agent-Reach
85.8kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
headroom
74.0kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
ruflo
73.4k🌊 The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
CowAgent
47.1kOpen-source super AI assistant & Agent Harness. Plans tasks, runs tools and skills, self-evolves with memory and knowledge. Multi-agent, multi-model, multi-channel. Lightweight, extensible, one-line install.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
