SkillAgentSearch skills...

scientific-critical-thinking

Evaluate research rigor. Assess methodology, experimental design, statistical validity, biases, confounding, evidence quality (GRADE, Cochrane ROB), for critical analysis of scientific claims.

Install / Use

npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill scientific-critical-thinking

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

92/100

Supported Platforms

Claude Code

Our assessment of scientific-critical-thinking

scientific-critical-thinking scores 92/100 on our quality scale, 98th of 421 Education & Research skills we index (top 24%).

Its SKILL.md is 23 KB long, well organised into 18 sections with 1 code example: a thorough specification that gives an agent plenty to work with.

With 4,360 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
17/20
Description
15/15
Adoption
15/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so scientific-critical-thinking is actively maintained.
  • Our last check on 2026-10-05 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-29. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

scientific-critical-thinking compared with similar skills

All 4 of these similar skills score higher than scientific-critical-thinking; compare them before choosing.

SkillScoreStarsUpdatedFormat
scientific-critical-thinking (this skill)by brycewang-stanford924.4k54d agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
last30days-skillby mvanhorn10068.0ktodayCLAUDE.md
Claude-Code-Game-Studiosby Donchitos10039.7k7d agoCLAUDE.md

Frequently asked questions

How do I install scientific-critical-thinking?
Run npx skills add brycewang-stanford/Auto-Empirical-Research-Skills --skill scientific-critical-thinking. The install tabs above show the steps for each supported agent.
Which AI agents does scientific-critical-thinking work with?
It is written for Claude Code, as a SKILL.md file. Other agents that read the same format can often use it too.
Is scientific-critical-thinking safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is scientific-critical-thinking still maintained?
The repository was last updated about 2 months ago, so scientific-critical-thinking is actively maintained.

name: scientific-critical-thinking description: "Evaluate research rigor. Assess methodology, experimental design, statistical validity, biases, confounding, evidence quality (GRADE, Cochrane ROB), for critical analysis of scientific claims." allowed-tools: [Read, Write, Edit, Bash]

<!-- ╔══════════════════════════════════════════════════════════════╗ ║ 本文件为开源 Skill 原始文档,收录仅供学习与研究参考 ║ ║ CoPaper.AI 收集整理 | https://copaper.ai ║ ╚══════════════════════════════════════════════════════════════╝ 来源仓库: https://github.com/K-Dense-AI/claude-scientific-writer 项目名称: claude-scientific-writer 开源协议: MIT License 收录日期: 2026-04-02 声明: 本文件版权归原作者所有。此处收录旨在为社会科学实证研究者 提供 AI Agent Skills 的集中参考。如有侵权,请联系删除。 -->

Scientific Critical Thinking

Overview

Critical thinking is a systematic process for evaluating scientific rigor. Assess methodology, experimental design, statistical validity, biases, confounding, and evidence quality using GRADE and Cochrane ROB frameworks. Apply this skill for critical analysis of scientific claims.

When to Use This Skill

This skill should be used when:

  • Evaluating research methodology and experimental design
  • Assessing statistical validity and evidence quality
  • Identifying biases and confounding in studies
  • Reviewing scientific claims and conclusions
  • Conducting systematic reviews or meta-analyses
  • Applying GRADE or Cochrane risk of bias assessments
  • Providing critical analysis of research papers

Visual Enhancement with Scientific Schematics

When creating documents with this skill, always consider adding scientific diagrams and schematics to enhance visual communication.

If your document does not already contain schematics or diagrams:

  • Use the scientific-schematics skill to generate AI-powered publication-quality diagrams
  • Simply describe your desired diagram in natural language
  • Nano Banana Pro will automatically generate, review, and refine the schematic

For new documents: Scientific schematics should be generated by default to visually represent key concepts, workflows, architectures, or relationships described in the text.

How to generate schematics:

python scripts/generate_schematic.py "your diagram description" -o figures/output.png

The AI will automatically:

  • Create publication-quality images with proper formatting
  • Review and refine through multiple iterations
  • Ensure accessibility (colorblind-friendly, high contrast)
  • Save outputs in the figures/ directory

When to add schematics:

  • Critical thinking framework diagrams
  • Bias identification decision trees
  • Evidence quality assessment flowcharts
  • GRADE assessment methodology diagrams
  • Risk of bias evaluation frameworks
  • Validity assessment visualizations
  • Any complex concept that benefits from visualization

For detailed guidance on creating schematics, refer to the scientific-schematics skill documentation.


Core Capabilities

1. Methodology Critique

Evaluate research methodology for rigor, validity, and potential flaws.

Apply when:

  • Reviewing research papers
  • Assessing experimental designs
  • Evaluating study protocols
  • Planning new research

Evaluation framework:

  1. Study Design Assessment

    • Is the design appropriate for the research question?
    • Can the design support causal claims being made?
    • Are comparison groups appropriate and adequate?
    • Consider whether experimental, quasi-experimental, or observational design is justified
  2. Validity Analysis

    • Internal validity: Can we trust the causal inference?
      • Check randomization quality
      • Evaluate confounding control
      • Assess selection bias
      • Review attrition/dropout patterns
    • External validity: Do results generalize?
      • Evaluate sample representativeness
      • Consider ecological validity of setting
      • Assess whether conditions match target application
    • Construct validity: Do measures capture intended constructs?
      • Review measurement validation
      • Check operational definitions
      • Assess whether measures are direct or proxy
    • Statistical conclusion validity: Are statistical inferences sound?
      • Verify adequate power/sample size
      • Check assumption compliance
      • Evaluate test appropriateness
  3. Control and Blinding

    • Was randomization properly implemented (sequence generation, allocation concealment)?
    • Was blinding feasible and implemented (participants, providers, assessors)?
    • Are control conditions appropriate (placebo, active control, no treatment)?
    • Could performance or detection bias affect results?
  4. Measurement Quality

    • Are instruments validated and reliable?
    • Are measures objective when possible, or subjective with acknowledged limitations?
    • Is outcome assessment standardized?
    • Are multiple measures used to triangulate findings?

Reference: See references/scientific_method.md for detailed principles and references/experimental_design.md for comprehensive design checklist.

2. Bias Detection

Identify and evaluate potential sources of bias that could distort findings.

Apply when:

  • Reviewing published research
  • Designing new studies
  • Interpreting conflicting evidence
  • Assessing research quality

Systematic bias review:

  1. Cognitive Biases (Researcher)

    • Confirmation bias: Are only supporting findings highlighted?
    • HARKing: Were hypotheses stated a priori or formed after seeing results?
    • Publication bias: Are negative results missing from literature?
    • Cherry-picking: Is evidence selectively reported?
    • Check for preregistration and analysis plan transparency
  2. Selection Biases

    • Sampling bias: Is sample representative of target population?
    • Volunteer bias: Do participants self-select in systematic ways?
    • Attrition bias: Is dropout differential between groups?
    • Survivorship bias: Are only "survivors" visible in sample?
    • Examine participant flow diagrams and compare baseline characteristics
  3. Measurement Biases

    • Observer bias: Could expectations influence observations?
    • Recall bias: Are retrospective reports systematically inaccurate?
    • Social desirability: Are responses biased toward acceptability?
    • Instrument bias: Do measurement tools systematically err?
    • Evaluate blinding, validation, and measurement objectivity
  4. Analysis Biases

    • P-hacking: Were multiple analyses conducted until significance emerged?
    • Outcome switching: Were non-significant outcomes replaced with significant ones?
    • Selective reporting: Are all planned analyses reported?
    • Subgroup fishing: Were subgroup analyses conducted without correction?
    • Check for study registration and compare to published outcomes
  5. Confounding

    • What variables could affect both exposure and outcome?
    • Were confounders measured and controlled (statistically or by design)?
    • Could unmeasured confounding explain findings?
    • Are there plausible alternative explanations?

Reference: See references/common_biases.md for comprehensive bias taxonomy with detection and mitigation strategies.

3. Statistical Analysis Evaluation

Critically assess statistical methods, interpretation, and reporting.

Apply when:

  • Reviewing quantitative research
  • Evaluating data-driven claims
  • Assessing clinical trial results
  • Reviewing meta-analyses

Statistical review checklist:

  1. Sample Size and Power

    • Was a priori power analysis conducted?
    • Is sample adequate for detecting meaningful effects?
    • Is the study underpowered (common problem)?
    • Do significant results from small samples raise flags for inflated effect sizes?
  2. Statistical Tests

    • Are tests appropriate for data type and distribution?
    • Were test assumptions checked and met?
    • Are parametric tests justified, or should non-parametric alternatives be used?
    • Is the analysis matched to study design (e.g., paired vs. independent)?
  3. Multiple Comparisons

    • Were multiple hypotheses tested?
    • Was correction applied (Bonferroni, FDR, other)?
    • Are primary outcomes distinguished from secondary/exploratory?
    • Could findings be false positives from multiple testing?
  4. P-Value Interpretation

    • Are p-values interpreted correctly (probability of data if null is true)?
    • Is non-significance incorrectly interpreted as "no effect"?
    • Is statistical significance conflated with practical importance?
    • Are exact p-values reported, or only "p < .05"?
    • Is there suspicious clustering just below .05?
  5. Effect Sizes and Confidence Intervals

    • Are effect sizes reported alongside significance?
    • Are confidence intervals provided to show precision?
    • Is the effect size meaningful in practical terms?
    • Are standardized effect sizes interpreted with field-specific context?
  6. Missing Data

    • How much data is missing?
    • Is missing data mechanism considered (MCAR, MAR, MNAR)?
    • How is missing data handled (deletion, imputation, maximum likelihood)?
    • Could missing data bias results?
  7. Regression and Modeling

    • Is the model overfitted (too many predictors, no cross-validation)?
    • Are predictions made outside the data range (extrapolation)?
    • Are multicollinearity issues addressed?
    • Are model assumptions checked?
  8. Common Pitfalls

    • Correlation treated as causation
    • Ignoring regression to the mean
    • Base rate neglect
    • Texas sharpshooter fallacy (pattern finding in noise)
    • Simpson's paradox (confounding by subgroups)

Reference: See references/statistical_pitfalls.md for detailed pitfalls and correct practices.

4. Evidence Quality Assessment

Evaluate the strength and quality of evidence systematically.

Apply when:

  • Weighing evidence for decisions
  • Conducting literature reviews
  • Comparing conflicting findings
  • Determining confidence in conclusions

Evidence evaluation framework:

  1. Study Design Hierarchy

    • Systematic reviews/meta-analyses (highest for intervention effects)
    • Randomized controlled trials
    • Cohort studies
    • Case-control studies
    • Cross-sectional studies
    • Case series/reports
    • Expert opinion (lowest)

    Important: Higher-level designs aren't always better quality. A well-designed observational study can be stronger than a poorly-conducted RCT.

  2. Quality Within Design Type

    • Risk of bias assessment (use appropriate tool: Cochrane ROB, Newcastle-Ottawa, etc.)
    • Methodological rigor
    • Transparency and reporting completeness
    • Conflicts of interest
  3. GRADE Considerations (if applicable)

    • Start with design type (RCT = high, observational = low)
    • Downgrade for:
      • Risk of bias
      • Inconsistency across studies
      • Indirectness (wrong population/intervention/outcome)
      • Imprecision (wide confidence intervals, small samples)
      • Publication bias
    • Upgrade for:
      • Large effect sizes
      • Dose-response relationships
      • Confounders would reduce (not increase) effect
  4. Convergence of Evidence

    • Stronger when:
      • Multiple independent replications
      • Different research groups and settings
      • Different methodologies converge on same conclusion
      • Mechanistic and empirical evidence align
    • Weaker when:
      • Single study or research group
      • Contradictory findings in literature
      • Publication bias evident
      • No replication attempts
  5. Contextual Factors

    • Biological/theoretical plausibility
    • Consistency with established knowledge
    • Temporality (cause precedes effect)
    • Specificity of relationship
    • Strength of association

Reference: See references/evidence_hierarchy.md for detailed hierarchy, GRADE system, and quality assessment tools.

5. L

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars4.4k
CategoryEducation
Updated1mo ago
Forks527

Languages

Stata

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium