SkillAgentSearch skills...

check-citations

Verify academic citations against CrossRef, Semantic Scholar, and OpenAlex. Detects AI-hallucinated references, chimeric citations, and suspicious patterns.

Install / Use

npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill 62-PHY041-claude-skill-citation-checker

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

91/100

Supported Platforms

Claude Code

Our assessment of check-citations

check-citations scores 91/100 on our quality scale, 132nd of 421 Education & Research skills we index (top 32%).

Its SKILL.md is 5.0 KB long, well organised into 20 sections with 7 code examples: a solid amount of guidance for an agent.

With 4,360 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
26/30
Structure
20/20
Description
15/15
Adoption
15/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so check-citations is actively maintained.
  • Our last check on 2026-10-28 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-29. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

check-citations compared with similar skills

All 4 of these similar skills score higher than check-citations; compare them before choosing.

SkillScoreStarsUpdatedFormat
check-citations (this skill)by brycewang-stanford914.4k54d agoSKILL.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
headroomby headroomlabs-ai10079.8ktodayCLAUDE.md
last30days-skillby mvanhorn10068.0ktodayCLAUDE.md
Scraplingby D4Vinci100103.6k2d agoMCP Server

Frequently asked questions

How do I install check-citations?
Run npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill check-citations. The install tabs above show the steps for each supported agent.
Which AI agents does check-citations work with?
It is written for Claude Code, as a SKILL.md file. Other agents that read the same format can often use it too.
Is check-citations safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is check-citations still maintained?
The repository was last updated about 2 months ago, so check-citations is actively maintained.

name: check-citations description: Verify academic citations against CrossRef, Semantic Scholar, and OpenAlex. Detects AI-hallucinated references, chimeric citations, and suspicious patterns. version: 1.0.0 metadata: openclaw: requires: bins: - python3 install: - kind: uv package: requests emoji: "\U0001F50D" homepage: https://github.com/PHY041/claude-skill-citation-checker

check-citations

Verify academic citations against CrossRef, Semantic Scholar, and OpenAlex. Detects AI-hallucinated references, chimeric citations (real title + wrong authors), and suspicious patterns before submission.

When to Use

  • After writing or editing a .bib file with AI assistance
  • Before submitting a paper, thesis, or report
  • When reviewing AI-generated literature sections
  • As a CI/CD check in LaTeX manuscript pipelines
  • When auditing existing bibliographies for dead or fabricated references

Background

  • 6-55% of AI-generated citations are fabricated (varies by model/domain)
  • 100+ hallucinated references found in NeurIPS 2025 accepted papers
  • Universities increasingly treat fake citations as academic misconduct
  • Three hallucination types: fully fabricated, chimeric (real title + wrong authors), modified real (slightly altered metadata)

Usage

Quick Check (Single File)

python scripts/citation_checker.py references.bib

Check All .bib Files in a Directory

python scripts/citation_checker.py path/to/report/

JSON Output (CI/CD Pipelines)

python scripts/citation_checker.py references.bib --json

Verbose Mode (Debug API Responses)

python scripts/citation_checker.py references.bib --verbose

How It Works

Cascading Multi-Source Verification

Each citation is checked against three independent databases:

| Source | Coverage | Strength | |--------|----------|----------| | CrossRef | 140M+ DOI-registered works | Best for journal/conference papers with DOIs | | Semantic Scholar | 200M+ papers | Best author disambiguation, arXiv coverage | | OpenAlex | 240M+ works | Broadest coverage, fully open |

Verification logic:

  • Found in 2+ sources with matching title → verified (high confidence)
  • Found in 1 source only → suspicious (manual check recommended)
  • Found in 0 sources → not_found (likely hallucinated)

Chimeric Detection

When a citation's title matches a real paper but the authors don't overlap at all, it's flagged as a possible chimeric hallucination — the most dangerous type because the title looks real on Google Scholar.

Red Flag Heuristics

  • Invalid DOI format (doesn't start with 10.xxxx/)
  • Suspiciously generic title patterns ("A Comprehensive Survey of...")
  • Future publication year
  • Missing author or year fields
  • Single-word author names (incomplete metadata)

Exit Codes

| Code | Meaning | |------|---------| | 0 | All citations verified | | 1 | One or more citations not found | | 2 | Suspicious citations only (no hard failures) |

Dependencies

pip install requests

No API keys required — uses free tiers of all three databases.

Accuracy (Tested)

| Category | Result | Description | |----------|--------|-------------| | Known-good | 9/10 (90%) | Famous ML papers (Vaswani, Devlin, Brown, He, etc.) | | Known-bad | 10/10 (100%) | Fabricated papers with plausible titles | | Chimeric | 5/5 (100%) | Real titles with wrong authors | | False positive rate | 10% | 1 miss: unpublished tech report without DOI | | False negative rate | 0% | No fake paper was ever verified |

The core guarantee: fake papers are never marked as real.

Limitations

  • Papers without DOI that have many derivatives (e.g., BERT without DOI) may not be found via title search alone — always include DOIs when available
  • Semantic Scholar free tier rate-limits at ~100 requests/5 minutes — batch verification is slower
  • Cannot verify papers behind paywalls or not indexed in any of the three databases
  • Book chapters, technical reports, and grey literature have lower coverage

Integration with LaTeX Workflows

Pre-commit Hook

#!/bin/bash
# .git/hooks/pre-commit
python scripts/citation_checker.py references.bib --json > /tmp/cite_check.json
NOT_FOUND=$(python3 -c "import json; d=json.load(open('/tmp/cite_check.json')); print(d['summary']['not_found'])")
if [ "$NOT_FOUND" -gt "0" ]; then
    echo "BLOCKED: $NOT_FOUND unfound citations. Run 'python scripts/citation_checker.py references.bib --verbose' to investigate."
    exit 1
fi

GitHub Actions

- name: Check citations
  run: |
    pip install requests
    python scripts/citation_checker.py references.bib --json > citation_report.json
    python -c "
    import json, sys
    r = json.load(open('citation_report.json'))
    if r['summary']['not_found'] > 0:
        print(f'FAIL: {r[\"summary\"][\"not_found\"]} citations not found')
        sys.exit(1)
    print(f'PASS: {r[\"summary\"][\"verified\"]} verified, {r[\"summary\"][\"suspicious\"]} suspicious')
    "

Related Skills

View on GitHub
GitHub Stars4.4k
CategoryEducation
Updated1mo ago
Forks527

Languages

Stata

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium