SkillAgentSearch skills...

ab-test-setup-and-analysis

Designs statistically valid split tests for ads, audiences, landing pages, or bid strategies. Calculates required sample sizes before you start, monitors results during the test, and calls winners when statistical significance is actually reached — not when you feel like one is winning.

Install / Use

npx skills add irinabuht12-oss/marketing-skills --skill ab-test-setup-and-analysis

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

83/100

Category

Operations

Supported Platforms

Universal

Our assessment of ab-test-setup-and-analysis

ab-test-setup-and-analysis scores 83/100 on our quality scale, 598th of 763 Operations skills we index.

Its SKILL.md is 2.9 KB long, split into 7 sections and no code examples: a solid amount of guidance for an agent.

With 7,761 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
26/30
Structure
11/20
Description
15/15
Adoption
17/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated 26 days ago, so ab-test-setup-and-analysis is actively maintained.
  • Our last check on 2026-11-06 found the source still online.
  • No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
  • Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-10. Automated pattern scan on 2026-10-10. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

ab-test-setup-and-analysis compared with similar skills

All 4 of these similar skills score higher than ab-test-setup-and-analysis; compare them before choosing.

SkillScoreStarsUpdatedFormat
ab-test-setup-and-analysis (this skill)by irinabuht12-oss837.8k26d agoSKILL.md
taste-skillby Leonxlnx100120.0ktodayCLAUDE.md
Claude-Code-Game-Studiosby Donchitos10040.6k12d agoCLAUDE.md
algorithmic-artby anthropics100177.9k2mo agoSKILL.md
pptxby anthropics100177.9k2mo agoSKILL.md

Frequently asked questions

How do I install ab-test-setup-and-analysis?
Run npx skills add irinabuht12-oss/marketing-skills --skill ab-test-setup-and-analysis. The install tabs above show the steps for each supported agent.
Which AI agents does ab-test-setup-and-analysis work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is ab-test-setup-and-analysis safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is ab-test-setup-and-analysis still maintained?
The repository was last updated 26 days ago, so ab-test-setup-and-analysis is actively maintained.

name: ab-test-setup-and-analysis description: Designs statistically valid split tests for ads, audiences, landing pages, or bid strategies. Calculates required sample sizes before you start, monitors results during the test, and calls winners when statistical significance is actually reached — not when you feel like one is winning. Platform: Google and Meta. metadata: platform: Google and Meta

28/ A/B Test Setup and Analysis — Google + Meta

What it does

Designs statistically valid split tests for ads, audiences, landing pages, or bid strategies. Calculates required sample sizes before you start, monitors results during the test, and calls winners when statistical significance is actually reached — not when you feel like one is winning.

How it works

Claude takes your test hypothesis, current baseline metrics, and the minimum detectable effect you care about, then calculates how much traffic and time the test needs to produce a reliable result. During the test, it tracks results and tells you whether differences are statistically significant or just noise. It prevents premature test calls that waste the effort.

Practical example

You want to test a new headline variant against your current best performer. Your current ad gets a 2.8% CTR with 1,200 daily impressions. Claude calculates that to detect a 15% improvement (3.22% CTR) with 95% confidence, you need approximately 12,400 impressions per variant — about 10 days at current traffic levels. After 7 days, the new variant shows 3.1% CTR vs 2.7% for control. Claude tells you it's trending positive but not yet significant (p=0.14, need p<0.05) — keep running for 4 more days before making a call.

What you get back

  • Test design with clear hypothesis, control, variant, and success metric
  • Required sample size and estimated test duration before you start
  • Daily monitoring with current results, confidence level, and days remaining
  • Winner call only when significance is reached, with effect size and confidence interval
  • Post-test recommendations: roll out winner, iterate further, or test something else

When to use it

  • Before launching any creative, audience, or landing page test
  • When someone on the team wants to call a winner after 2 days of data
  • For structured testing programs where you need a pipeline of tests running sequentially
  • When clients ask "is this result real or just noise" and you need a definitive answer

Data access (Ryze MCP)

This skill works best with live account data. Connect the free Ryze MCP once and Claude reads your Google Ads, Meta Ads, GA4 and Search Console directly:

  • claude.ai / Claude Desktop: Settings → Connectors → Add custom connector → https://connector.get-ryze.ai/mcp
  • Claude Code: claude mcp add ryze --transport http https://connector.get-ryze.ai/mcp
  • Cursor: Settings → MCP → add the same URL

Setup guide: https://www.get-ryze.ai/how-to-connect-claude-to-google-meta-ads-mcp

Related Skills

View on GitHub
GitHub Stars7.8k
CategoryOperations
Updated26d ago
Forks579

Trust signals

88/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 medium