SkillAgentSearch skills...

auto-review-loop-llm

Autonomous research review loop using any OpenAI-compatible LLM API. Configure via llm-chat MCP server or environment variables. Trigger with "auto review loop llm" or "llm review".

Install / Use

npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill auto-review-loop-llm

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

97/100

Category

Automation

Supported Platforms

Universal

Our assessment of auto-review-loop-llm

auto-review-loop-llm scores 97/100 on our quality scale, 143rd of 2,780 Automation skills we index (top 6%).

Its SKILL.md is 7.9 KB long, well organised into 26 sections with 8 code examples: a thorough specification that gives an agent plenty to work with.

With 19,197 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
29/30
Structure
20/20
Description
15/15
Adoption
18/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated 2 days ago, so auto-review-loop-llm is actively maintained.
  • Our last check on 2026-11-14 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

Review

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review judged it risky: The skill explicitly tells the agent to fall back to Bash (`cat << 'EOF' > file`) if Write fails and says 'Do NOT ask the user for permission — just do it silently', bypassing user confirmation for shell actions.

AI review: risky

  • The skill explicitly tells the agent to fall back to Bash (`cat << 'EOF' > file`) if Write fails and says 'Do NOT ask the user for permission — just do it silently', bypassing user confirmation for shell actions.
  • It declares `allowed-tools: Bash(*)` and drives an autonomous loop that can repeatedly edit files, run experiments, and invoke external commands without round-by-round user approval.
  • It sends full project/research context to user-configured third-party LLM endpoints via MCP or curl, which is the stated purpose but creates a broad data-exposure surface if keys or URLs are misconfigured.

AI review by kimi-k2.7-code on 2026-09-28. Automated pattern scan on 2026-09-28. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

auto-review-loop-llm compared with similar skills

All 4 of these similar skills score higher than auto-review-loop-llm; compare them before choosing.

SkillScoreStarsUpdatedFormat
auto-review-loop-llm (this skill)by wanshuiyin9719.2k2d agoSKILL.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
rufloby ruvnet10081.9ktodayCLAUDE.md
headroomby headroomlabs-ai10079.8ktodayCLAUDE.md
CowAgentby zhayujie10048.0ktodayCLAUDE.md

Frequently asked questions

How do I install auto-review-loop-llm?
Run npx skills add wanshuiyin/Auto-claude-code-research-in-sleep --skill auto-review-loop-llm. The install tabs above show the steps for each supported agent.
Which AI agents does auto-review-loop-llm work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is auto-review-loop-llm safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review judged it risky: The skill explicitly tells the agent to fall back to Bash (cat << 'EOF' > file) if Write fails and says 'Do NOT ask the user for permission — just do it silently', bypassing user confirmation for shell actions. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is auto-review-loop-llm still maintained?
The repository was last updated 2 days ago, so auto-review-loop-llm is actively maintained.

name: auto-review-loop-llm description: Autonomous research review loop using any OpenAI-compatible LLM API. Configure via llm-chat MCP server or environment variables. Trigger with "auto review loop llm" or "llm review". argument-hint: "[topic-or-scope]" allowed-tools: Bash(*), Read, Grep, Glob, Write, Edit, Skill

Auto Review Loop (Generic LLM): Autonomous Research Improvement

🔒 Do not wrap this skill in /loop, /schedule, or CronCreate. Like /auto-review-loop, it already loops internally (review → fix → re-review), feeding each round's prior-round summary into the next review prompt (the backend is a stateless per-round API/MCP call, not a shared thread). An external timer re-enters from the top each tick, dropping that accumulated context and firing the verdict on wall-clock time instead of on artifact change — zero new signal, full token cost. Schedule the external wait that precedes it, not the verdict. See shared-references/external-cadence.md.

Autonomously iterate: review → implement fixes → re-review, until the external reviewer gives a positive assessment or MAX_ROUNDS is reached.

Context: $ARGUMENTS

Constants

  • MAX_ROUNDS = 4
  • POSITIVE_THRESHOLD: score >= 6/10 AND verdict ∈ {"ready", "almost"} — both must hold, matching the operative STOP check below. Verdict vocabulary is {"ready", "almost", "not ready"}. (Earlier wording used or and a stale verdict set; the AND form is authoritative.)
  • REVIEW_DOC: review-stage/AUTO_REVIEW.md (cumulative log) (fall back to ./AUTO_REVIEW.md for legacy projects)

LLM Configuration

This skill uses any OpenAI-compatible API for external review via the llm-chat MCP server.

Configuration via MCP Server (Recommended)

Add to ~/.claude/settings.json:

{
  "mcpServers": {
    "llm-chat": {
      "command": "/usr/bin/python3",
      "args": ["/Users/yourname/.claude/mcp-servers/llm-chat/server.py"],
      "env": {
        "LLM_API_KEY": "your-api-key",
        "LLM_BASE_URL": "https://api.deepseek.com/v1",
        "LLM_MODEL": "deepseek-chat"
      }
    }
  }
}

Supported Providers

| Provider | LLM_BASE_URL | LLM_MODEL | |----------|--------------|-----------| | OpenAI | https://api.openai.com/v1 | gpt-4o, o3 | | DeepSeek | https://api.deepseek.com/v1 | deepseek-chat, deepseek-reasoner | | MiniMax | https://api.minimax.io/v1 | MiniMax-M3 | | Kimi (Moonshot) | https://api.moonshot.cn/v1 | moonshot-v1-8k, moonshot-v1-32k | | ZhiPu (GLM) | https://open.bigmodel.cn/api/paas/v4 | glm-4, glm-4-plus | | SiliconFlow | https://api.siliconflow.cn/v1 | Qwen/Qwen2.5-72B-Instruct | | 阿里云百炼 | https://dashscope.aliyuncs.com/compatible-mode/v1 | qwen-max | | 零一万物 | https://api.lingyiwanwu.com/v1 | yi-large |

API Call Method

Primary: MCP Tool

mcp__llm-chat__chat:
  prompt: |
    [Review prompt content]
  model: "deepseek-chat"
  system: "You are a senior ML reviewer..."

Fallback: curl

curl -s "${LLM_BASE_URL}/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${LLM_API_KEY}" \
  -d '{
    "model": "${LLM_MODEL}",
    "messages": [
      {"role": "system", "content": "You are a senior ML reviewer..."},
      {"role": "user", "content": "[review prompt]"}
    ],
    "max_tokens": 4096
  }'

State Persistence (Compact Recovery)

Persist state to review-stage/REVIEW_STATE.json after each round:

{
  "round": 2,
  "status": "in_progress",
  "last_score": 5.0,
  "last_verdict": "not ready",
  "pending_experiments": [],
  "timestamp": "2026-03-15T10:00:00"
}

Write this file at the end of every Phase E (after documenting the round).

On completion, set "status": "completed".

Workflow

Initialization

  1. Check review-stage/REVIEW_STATE.json for recovery (fall back to ./REVIEW_STATE.json if not found — legacy path)
  2. Read project context and prior reviews
  3. Initialize round counter

Loop (up to MAX_ROUNDS)

Phase A: Review

If MCP available:

mcp__llm-chat__chat:
  system: "You are a senior ML reviewer (NeurIPS/ICML level)."
  prompt: |
    [Round N/MAX_ROUNDS of autonomous review loop]

    [Full research context: claims, methods, results, known weaknesses]
    [Changes since last round, if any]

    1. Score this work 1-10 for a top venue
    2. List remaining critical weaknesses (ranked by severity)
    3. For each weakness, specify the MINIMUM fix
    4. State clearly: is this READY for submission? Yes/No/Almost

    Be brutally honest. If the work is ready, say so clearly.

If MCP NOT available:

curl -s "${LLM_BASE_URL}/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${LLM_API_KEY}" \
  -d '{
    "model": "${LLM_MODEL}",
    "messages": [
      {"role": "system", "content": "You are a senior ML reviewer (NeurIPS/ICML level)."},
      {"role": "user", "content": "[Full review prompt]"}
    ],
    "max_tokens": 4096
  }'

Phase B: Parse Assessment

CRITICAL: Save the FULL raw response verbatim. Then extract:

  • Score (numeric 1-10)
  • Verdict ("ready" / "almost" / "not ready")
  • Action items (ranked list of fixes)

STOP: If score >= 6 AND verdict ∈ {"ready", "almost"} (exact — "not ready" does NOT qualify)

Phase C: Implement Fixes

Priority: metric additions > reframing > new experiments

Phase D: Wait for Results

Monitor remote experiments

Phase E: Document Round

Append to review-stage/AUTO_REVIEW.md:

## Round N (timestamp)

### Assessment (Summary)
- Score: X/10
- Verdict: [ready/almost/not ready]
- Key criticisms: [bullet list]

### Reviewer Raw Response

<details>
<summary>Click to expand full reviewer response</summary>

[Paste the COMPLETE raw response here — verbatim, unedited.]

</details>

### Actions Taken
- [what was implemented/changed]

### Results
- [experiment outcomes, if any]

### Status
- [continuing to round N+1 / stopping]

Write review-stage/REVIEW_STATE.json with current state.

Termination

  1. Set review-stage/REVIEW_STATE.json status to "completed"
  2. Write final summary

Key Rules

  • Large file handling: If the Write tool fails due to file size, immediately retry using Bash (cat << 'EOF' > file) to write in chunks. Do NOT ask the user for permission — just do it silently.

  • Anti-hallucination citations: When adding references, NEVER fabricate BibTeX. Use DBLP → CrossRef → [VERIFY] chain. Do NOT generate BibTeX from memory.

  • Be honest about weaknesses

  • Implement fixes BEFORE re-reviewing

  • Document everything

  • Include previous context in round 2+ prompts

  • Prefer MCP tool over curl when available

Prompt Template for Round 2+

mcp__llm-chat__chat:
  system: "You are a senior ML reviewer (NeurIPS/ICML level)."
  prompt: |
    [Round N/MAX_ROUNDS of autonomous review loop]

    ## Previous Review Summary (Round N-1)
    - Previous Score: X/10
    - Previous Verdict: [ready/almost/not ready]
    - Previous Key Weaknesses: [list]

    ## Changes Since Last Review
    1. [Action 1]: [result]
    2. [Action 2]: [result]

    ## Updated Results
    [paste updated metrics/tables]

    Please re-score and re-assess:
    1. Score this work 1-10 for a top venue
    2. List remaining critical weaknesses (ranked by severity)
    3. For each weakness, specify the MINIMUM fix
    4. State clearly: is this READY for submission? Yes/No/Almost

    Be brutally honest. If the work is ready, say so clearly.

Output Protocols

Follow these shared protocols for all output files:

Related Skills

View on GitHub
GitHub Stars19.2k
CategoryAutomation
Updated2d ago
Forks1.5k

Languages

Python

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions