SkillAgentSearch skills...

quality-gate

Verify that every quantitative claim in the paper is traceable to an analysis output file, and that no important output was omitted. Make sure to use this skill whenever the user wants to check that the paper and analysis are consistent before submission.

Install / Use

npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill quality-gate

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

89/100

Supported Platforms

Universal

Tags

Our assessment of quality-gate

quality-gate scores 89/100 on our quality scale, 1544th of 4,698 Development & Engineering skills we index (top 33%).

Its SKILL.md is 5.3 KB long, well organised into 14 sections with 2 code examples: a solid amount of guidance for an agent.

With 4,360 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
26/30
Structure
18/20
Description
15/15
Adoption
15/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so quality-gate is actively maintained.
  • Our last check on 2026-10-31 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-29. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

quality-gate compared with similar skills

All 4 of these similar skills score higher than quality-gate; compare them before choosing.

SkillScoreStarsUpdatedFormat
quality-gate (this skill)by brycewang-stanford894.4k54d agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
ai-job-searchby ai-job-search-dev10072.1ktodayCLAUDE.md
claude-howtoby luongnv8910047.3k2d agoCLAUDE.md
lobehubby lobehub10085.6ktodayMCP Server

Frequently asked questions

How do I install quality-gate?
Run npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill quality-gate. The install tabs above show the steps for each supported agent.
Which AI agents does quality-gate work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is quality-gate safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is quality-gate still maintained?
The repository was last updated about 2 months ago, so quality-gate is actively maintained.

name: quality-gate description: >- Verify that every quantitative claim in the paper is traceable to an analysis output file, and that no important output was omitted. Make sure to use this skill whenever the user wants to check that the paper and analysis are consistent before submission. Triggers include: "run the quality gate", "check the paper matches the analysis", "verify consistency", "does the paper match my results", "check my numbers", "are my tables right", "quality check before submission", "verify my claims", "make sure everything is consistent", "double-check the paper against my output files", or any pre-submission integrity check between paper text and computed results. argument-hint: "[paper file path, or leave blank to auto-detect]" allowed-tools: ["Read", "Grep", "Glob", "Write", "Task", "AskUserQuestion"]

Quality Gate: Paper ↔ Analysis Consistency

Cross-check every numerical claim in the paper against analysis output files. Reports only — never edits.

Input: $ARGUMENTS — path to the paper draft, or leave blank to auto-detect.


Step 1: Locate the Paper Draft

If $ARGUMENTS is provided, use that path. Otherwise glob for:

  • manuscripts/**/*.tex
  • manuscripts/**/*.qmd
  • manuscripts/**/*.md

If multiple drafts found, use AskUserQuestion to let the user pick:

  • header: "Paper"
  • question: "Multiple drafts found. Which manuscript should I check?"
  • options: list up to 4 found files (label: filename, description: path and file size). If more than 4, group by directory.

Step 2: Extract Numerical Claims

Read the full manuscript and extract every quantitative claim:

  • Coefficients and standard errors — e.g., "The effect is 0.23 (SE = 0.04)"
  • Sample sizes — e.g., "N = 4,521 observations"
  • Percentages and proportions — e.g., "42% of firms..."
  • Means, medians, ranges — e.g., "average income of $45,000"
  • Table and figure references — e.g., "Table 2 shows...", "as seen in Figure 1"
  • p-values and significance statements — e.g., "statistically significant at the 1% level"

Record location (section, paragraph, line number if available) for each claim.


Step 3: Inventory Output Files

Glob for all output files:

  • output/tables/**/*.tex — regression and summary tables
  • output/tables/**/*.html — HTML versions
  • output/figures/**/*.pdf, output/figures/**/*.png — figures
  • output/**/*.rds, output/**/*.pkl, output/**/*.parquet, output/**/*.csv — saved objects

Build an inventory with file paths and sizes.


Step 4: Dispatch Verifier Agent

Dispatch the verifier agent via Task to perform the heavy verification work. Pass it:

  • The full list of numerical claims extracted in Step 2 (with locations)
  • The output file inventory from Step 3
  • The paper draft path
  • The bibliography file path
Task prompt: "You are the verifier agent. Paper draft: [path].
Bibliography: [bib path].

CLAIMS TO VERIFY:
[paste the full claims list from Step 2]

OUTPUT FILE INVENTORY:
[paste the inventory from Step 3]

Verify each claim against the output files. Then do a reverse check —
find output files NOT referenced in the paper. Then check all citation
keys against the bibliography. Follow the verifier agent instructions
and return your full verification report."

After the verifier completes, collect its results:

  • Claim verification table (MATCHED / UNVERIFIED / MISSING FILE per claim)
  • Unreferenced output files list
  • Missing citation keys

Step 5: Save Report

Save to quality_reports/quality_gate_[YYYY-MM-DD]_[paper-name].md:

# Quality Gate Report: [Paper Name]
**Date:** [YYYY-MM-DD]
**Paper:** [file path]

## Verdict: PASS / CONDITIONAL PASS / FAIL

PASS = all claims matched, no missing citations, no unexplained unreferenced outputs
CONDITIONAL PASS = minor unverified claims or informational unreferenced outputs
FAIL = unverified critical claims or missing citations

---

## Claim Verification

| Claim | Location | Found in Output? | Source File | Status |
|-------|----------|-----------------|-------------|--------|
| β = 0.23 (SE = 0.04) | Section 4, para 2 | Yes | output/tables/main_regs.tex | MATCHED |
| N = 4,521 | Table 2 note | Yes | output/tables/main_regs.tex | MATCHED |
| 42% of firms | Intro, para 1 | No | — | UNVERIFIED |

---

## Unreferenced Outputs

Files in output/ not referenced in the paper:

| File | Size | Recommended Action |
|------|------|-------------------|
| output/tables/robustness_het.tex | 4.2 KB | Reference in Section 7 or explain exclusion |

---

## Missing Citations

| Key | Used At | Status |
|-----|---------|--------|
| SmithJones2021 | Section 3, para 1 | NOT IN BIBLIOGRAPHY — CRITICAL |

---

## Summary

- Claims verified: N / M total
- Claims unverified: K (see table above)
- Unreferenced outputs: J
- Missing citations: L

## Recommended Actions (Priority Order)
1. [BLOCKING] ...
2. [RECOMMENDED] ...

Key Rules

  • Report only — never edit. All fixes are the user's responsibility after reviewing the report.
  • Tolerance: For inline numbers, a claim is MATCHED if the value appears in any output file within reasonable display rounding (±0.005 for 2-decimal numbers).
  • False positives are OK. Flag uncertainties as UNVERIFIED rather than guessing MATCHED.
  • Missing files are always BLOCKING — a figure reference pointing to a non-existent file is a FAIL.

Related Skills

View on GitHub
GitHub Stars4.4k
CategoryDevelopment
Updated1mo ago
Forks527

Languages

Stata

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium