SkillAgentSearch skills...

statistical-analyst

Applied statistics for business and product questions — test selection, assumption checks, power planning, effect sizes with intervals, multiplicity correction

Install / Use

npx skills add borghei/Claude-Skills --skill statistical-analyst

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

91/100

Supported Platforms

Universal

Our assessment of statistical-analyst

statistical-analyst scores 91/100 on our quality scale, 206th of 608 Data & Analytics skills we index (top 34%).

Its SKILL.md is 14 KB long, well organised into 19 sections with 3 code examples: a thorough specification that gives an agent plenty to work with.

It has 817 GitHub stars, a meaningful sign that others use it.

Substance
30/30
Structure
18/20
Description
15/15
Adoption
12/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so statistical-analyst is actively maintained.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-07. Automated pattern scan on 2026-10-07. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

statistical-analyst compared with similar skills

All 4 of these similar skills score higher than statistical-analyst; compare them before choosing.

SkillScoreStarsUpdatedFormat
statistical-analyst (this skill)by borghei9181757d agoSKILL.md
x402by browser-use100121.0k4d agoSKILL.md
algorithmic-artby anthropics100177.9k56d agoSKILL.md
pptxby anthropics100177.9k56d agoSKILL.md
prospectingby coreyhaines3110060.1k6d agoSKILL.md

Frequently asked questions

How do I install statistical-analyst?
Run npx skills add borghei/Claude-Skills --skill statistical-analyst. The install tabs above show the steps for each supported agent.
Which AI agents does statistical-analyst work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is statistical-analyst safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is statistical-analyst still maintained?
The repository was last updated about 2 months ago, so statistical-analyst is actively maintained.

name: statistical-analyst description: > Applied statistics for business and product questions — test selection, assumption checks, power planning, effect sizes with intervals, multiplicity correction. Use when interpreting an experiment, sizing a study, or vetting a claim. license: MIT + Commons Clause metadata: version: 1.0.0 author: borghei category: data-analytics domain: statistics updated: 2026-07-21 tags: [statistics, hypothesis-testing, effect-size, power-analysis, experimentation]

Statistical Analyst

Most bad statistics in business are not arithmetic errors. They are the wrong test on the right data, a null result reported as "no difference," a p-value mistaken for an effect size, or twenty comparisons run and the one that cleared 0.05 written up. This skill covers the applied path: pick the test the data shape actually calls for, check the assumptions that carry weight, size the study before running it, report effects with intervals rather than bare p-values, and say what you found to people who do not want a statistics lecture.

Everything here runs on the Python standard library — the t, chi-square, and normal distributions are implemented directly, so there is no scipy dependency between a question and an answer. Because those implementations are hand-rolled, stats_core.py --selftest verifies all of them against published reference values; run it once before trusting any result.

When to use this skill

  • An A/B test finished and someone needs to know whether to ship
  • A study is being designed and nobody has computed how many observations it needs
  • A stakeholder is quoting a p-value as if it were an effect size
  • A "no difference" result is about to be reported from a study that was underpowered
  • Several variants, metrics, or segments were compared and no correction was applied
  • The data are skewed or outlier-heavy and the default t test is about to be run anyway

Inputs the skill expects

  • The business question and what decision it will change
  • The outcome variable and its type (binary, continuous, count, ordinal, categorical)
  • Group structure — how many groups, independent or paired, randomized or observational
  • Observation counts per group, and the unit of measurement (user, session, order)
  • The baseline value and the smallest effect worth acting on
  • How many comparisons are in the family, and which metric was named primary before the data arrived

Clarify First

Before analyzing, confirm these inputs. If any is unknown or vague, ASK — do not assume:

  • [ ] The unit of measurement, and whether each unit appears once — clustering (many sessions per user counted as independent rows) deflates standard errors and manufactures significance; it invalidates every test below
  • [ ] The smallest effect worth acting on — without it there is no way to size the study or to say whether a significant result matters
  • [ ] How many comparisons are in the family, and which metric was primary — decides the correction and whether the result is confirmatory or exploratory
  • [ ] Whether anyone has already looked at the data while it accumulated — peeking invalidates a fixed-horizon test and changes the whole approach

Stop rule: ask only the 2-3 that most change the output. If the user says "just draft it," proceed and list your assumptions at the top of the artifact.

Workflows

Workflow 1 — Choose the test and size the study before collecting data

  1. Write down the business question and the decision it changes. If the answer changes nothing, stop — do not run the study.
  2. Fill in the outcome type, group structure, baseline, and the minimum effect worth shipping into a question spec.
  3. Run the selector. It returns one test, its load-bearing assumptions, the fallback when they fail, and the sample size the stated effect requires.
  4. If the achievable sample is below the required sample, say so before running. An underpowered study should be a conscious decision, not a discovery at write-up time.
  5. Record everything in assets/experiment_design_template.md.
python3 data-analytics/statistical-analyst/scripts/test_selector.py \
  --input data-analytics/statistical-analyst/assets/sample_question.json \
  --power 0.9

Workflow 2 — Run the test and report the effect, not the p-value

  1. Check independence first: count units versus count rows. More rows than units means clustering, and no test below is valid until that is handled.
  2. Inspect skew and outliers (mean vs median, top and bottom five values). If the data are heavy-tailed at small n, switch to Mann-Whitney.
  3. Run the test with the number of comparisons in the family declared, so the threshold is corrected.
  4. Read the effect size and the interval first. Report the effect in domain units, the interval in the same units, and the decision implication.
python3 data-analytics/statistical-analyst/scripts/stats_core.py --selftest

python3 data-analytics/statistical-analyst/scripts/run_test.py \
  --input data-analytics/statistical-analyst/assets/sample_experiment.json \
  --comparisons 3

Workflow 3 — Audit someone else's statistical claim

  1. Ask what the unit of measurement was and whether units repeat. This finds more real errors than every distributional check combined.
  2. Ask how many comparisons were run in total, including the ones not reported, and whether the primary metric was named before the data arrived.
  3. Re-run the test from the raw counts, with the true comparison count.
  4. If the claim is a null result, compute the interval and state the largest effect the study could have missed — "no difference" and "we could not tell" look identical in a significance test and are opposite conclusions.
python3 data-analytics/statistical-analyst/scripts/run_test.py \
  --input data-analytics/statistical-analyst/assets/sample_revenue.json \
  --test welch_t --comparisons 4 --format json

Decision frameworks

Test selection

| Question | Outcome | Groups | Test | Effect size | |----------|---------|--------|------|-------------| | Difference | Binary | 2 independent | [PROVEN] Two-proportion z | Absolute difference; Cohen's h | | Difference | Binary | 2 paired | [PROVEN] McNemar | Odds ratio on discordant pairs | | Difference | Binary/categorical | 3+ | [PROVEN] Chi-square of independence | Cramér's V | | Difference | Continuous, symmetric | 2 independent | [PROVEN] Welch's t | Mean difference; Hedges' g | | Difference | Continuous, skewed or n<15 | 2 independent | [PROVEN] Mann-Whitney U | Rank-biserial r | | Difference | Continuous | 3+ | [RECOMMENDED] One-way ANOVA | Eta-squared | | Difference | Count per exposure | 2 | [RECOMMENDED] Poisson rate ratio | Rate ratio | | Association | Two continuous | — | [PROVEN] Pearson, or Spearman if skewed | r, r² | | Change over time | Any | — | [RECOMMENDED] Interrupted time series | Level and slope change |

Use Welch's t, never Student's t, as the two-group default. It does not assume equal variances and costs a fraction of a degree of freedom when they are equal. Testing for equal variance first and then choosing is worse than always using Welch — the pre-test inflates the error rate of the whole procedure.

Reading an interval against your decision threshold

| Interval position | Reading | Action | |-------------------|---------|--------| | Entirely above the threshold | Real and big enough | Ship | | Above zero, straddles the threshold | Real, unclear if it clears the bar | Collect more, or decide on cost | | Straddles zero, narrow | Genuinely no meaningful effect | Do not ship — and this is the only case where "no difference" is honest | | Straddles zero, wide | Study could not answer the question | Report as inconclusive, state the upper bound |

The last two are identical in a significance test and are opposite conclusions. That is the strongest single argument for reporting intervals.

Sample size reality check

Per group, α = 0.05, power = 0.80:

| Baseline rate | Relative effect to detect | n per group | |---------------|---------------------------|-------------| | 2% | +10% | ~78,000 | | 8% | +10% | ~28,500 | | 8% | +25% | ~4,900 | | 20% | +10% | ~9,000 | | 20% | +25% | ~1,600 |

Most product experiments are sized by "how long can we wait," which is how underpowered studies get written up as "no difference."

Anti-Patterns

P-hacking by exploration

Mistake: Twenty metrics, six segments, and three time windows get compared; the one combination that clears p < 0.05 becomes the headline. Why it happens: It rarely feels like cheating. Each individual comparison is a reasonable question, the analyst is genuinely curious, and the tooling makes slicing free. Nobody counts the comparisons because nobody wrote them down. Instead: Name one primary metric before the data arrive and pre-register the subgroups you will examine. Everything else is exploratory, gets Benjamini-Hochberg correction, and is reported as hypothesis-generating rather than decisive. With α = 0.05 and 20 uncorrected comparisons, the chance of at least one false positive is 64% — a coin flip dressed as a finding.

Peeking at a running experiment

Mistake: The dashboard is checked daily and the test is stopped the moment p dips below 0.05. Why it happens: The data are right there, stopping early saves time and traffic, and each individual look feels harmless. The intuition that "more data can only help" is exactly backwards here. Instead: Fix the horizon, compute n up front, and do not look — or use a method built for continuous monitoring (group sequential with O'Brien-Fleming spending, or always-valid confidence sequences). Repeated peeking at an uncorrected fixed-horizon test drives the real false-positive rate to 20-30%: a random walk crosses the threshold eventually even when nothing is happening. If it has already happened, report the result as exploratory and re-run with a fixed horizon.

Reading a null result as "no effect"

Mistake: p = 0.31, so the memo says the change made no difference and the feature is killed. Why it happens: "Not significant" sounds like "no effect," and the alternative sentence — "we ran a study that could not answer the question" — is uncomfortable to write. Instead: Report the interval. If it runs from −0.2% to +4.1%, the study is consistent with a substantial gain and has ruled out almost nothing. State the largest effect you could have missed. Only a narrow interval around zero supports "no meaningful effect," and that distinction is invisible in the p-value.

Confusing significance with importance

Mistake: At n = 400,000 a 0.02% conversion difference reaches p < 0.001 and gets a roadmap slot. Why it happens: p-values conflate effect size with sample size, so at large n everything is significant. The number looks impressive precisely because the sample is large. Instead: Set the decision threshold from the economics — cost to ship divided by value per unit — before the analysis. Then compare the interval to that threshold, not to zero. The reciprocal error matters equally: at small n, an important effect can miss significance and get discarded.

Ignoring the unit of measurement

Mistake: A test on 50,000 sessions from 4,000 users treats every session as an independent observation. Why it happens: The event table has 50,000 rows, the tooling counts rows, and the resulting p-value is gratifyingly small. It is invisible unless someone explicitly compares row count to unit count. Instead: Aggregate to the unit of assignment before testing, or use a cluster-robust method. Treating clustered rows as independent can understate the standard error several-fold and turn pure noise into a highly significant result. This is the single most common invalidating error in applied product analytics, and the cheapest to ch

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars817
CategoryData
Updated1mo ago
Forks141

Languages

HTML

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium