ln-45-benchmark-comparator
Compares tools or implementations through controlled benchmarks and independent correctness checks.
Install / Use
npx skills add levnikolaevich/claude-code-skills --skill ln-45-benchmark-comparatorInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
LegalSupported Platforms
Tags
Our assessment of ln-45-benchmark-comparator
ln-45-benchmark-comparator scores 82/100 on our quality scale, 169th of 209 Legal skills we index.
Its SKILL.md is 15 KB long, well organised into 11 sections and no code examples: a thorough specification that gives an agent plenty to work with.
It has 566 GitHub stars, a meaningful sign that others use it.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so ln-45-benchmark-comparator is actively maintained.
- Our last check on 2026-11-08 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-15. Automated pattern scan on 2026-10-15. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
ln-45-benchmark-comparator compared with similar skills
All 4 of these similar skills score higher than ln-45-benchmark-comparator; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| ln-45-benchmark-comparator (this skill)by levnikolaevich | 82 | 566 | 2mo ago | SKILL.md |
| x402by browser-use | 100 | 121.0k | 4d ago | SKILL.md |
| algorithmic-artby anthropics | 100 | 177.9k | 56d ago | SKILL.md |
| pptxby anthropics | 100 | 177.9k | 56d ago | SKILL.md |
| prospectingby coreyhaines31 | 100 | 60.1k | 6d ago | SKILL.md |
Frequently asked questions
- How do I install ln-45-benchmark-comparator?
- Run
npx skills add levnikolaevich/claude-code-skills --skill ln-45-benchmark-comparator. The install tabs above show the steps for each supported agent. - Which AI agents does ln-45-benchmark-comparator work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is ln-45-benchmark-comparator safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is ln-45-benchmark-comparator still maintained?
- The repository was last updated about 2 months ago, so ln-45-benchmark-comparator is actively maintained.
Skill content
View source on GitHubname: ln-45-benchmark-comparator description: "Compares tools or implementations through controlled benchmarks and independent correctness checks."
Benchmark Comparator
Goal: Compare alternatives under controlled, reproducible conditions. Correctness comes before speed, and measured data must remain separate from estimates, setup cost, and interpretation.
Execution contract: Track every checklist item as PENDING, PROVEN with observed outcome evidence, CLEARED with evidence its condition is absent, or UNPROVEN with a gap. Reading, delegation, tool failure, zero exit status, and self-reported success are not proof. Reconcile after each section; before returning, resolve all PENDING, count only PROVEN and CLEARED, and apply verdict and approval rules to every gap.
Preserve intent, scope, mode, and existing authorization. Complete authorized work before required external approval; ask only consequential unresolved choices. If nobody can answer, report the exact question and apply the skill's gap verdict without waiting or guessing. Cite any blocking instruction and unresolved boundary; never invent approval gates. Scale depth to material risk; preserve mandatory checks, dependencies, and safety order while choosing verification methods.
Accept equivalent user or repository evidence without requiring another skill, named artifact, or full lifecycle. Preserve source requirement and decision IDs. Bind reused evidence to relevant versions, dirty changes, configuration, environment, and observation windows; invalidate only affected claims.
Choose the primary workflow by outcome and explicit selection. Consume neighboring contracts and valid evidence without duplicating ownership or restarting audits. Split distinct deliverables by scope, not mandatory skill invocation. Broad independent assessments use audit workflows; focused domain reviews remain read-only.
On continuation, reconcile task, authorization, state, and gaps. Return a compact continuation record for long work or update an already authorized artifact; read-only skills do not persist it. Distinguish artifact readiness, verified behavior, and external-action authority.
Before retrieval, define the evidence question; read selectively without skipping applicable instructions. Use tools for mechanical processing; summarize bounded results, errors, coverage, and source paths, keeping full output outside context.
Before technical decisions, establish applicability and actual versions; verify version-dependent assumptions against official documentation or specifications. Search and read primary pages for current external recommendations; snippets are not evidence. Version-matched bundled documentation or valid prior evidence suffices. Record decision, version, source, and limitations. Mark unavailable evidence UNVERIFIED; block only essential dependent work. Prefer native mechanisms, justify added tools or abstractions with evidence, and reuse the existing rule owner.
Tool Routing
| Need | Preferred tool | Use it when | Fallback |
|---|---|---|---|
| Canonical workload and oracle | Repository fixtures, tests, expected diffs, schemas, or independently specified outcomes | Defining what success means before either candidate runs | Create the smallest deterministic fixture that represents the decision |
| Isolation | Clean Git worktrees, temporary directories, controlled environment, fixed seeds, and resettable caches | Preventing one candidate or run from contaminating another | Sequential clean-room setup with verified cleanup |
| Execution | The same shell runner and wrapper for every candidate | Capturing commands, exit status, stdout, stderr, timing, and artifacts consistently | Manual execution with an explicit reproducibility limitation |
| Activation proof | Logs, traces, command records, process metadata, or candidate-specific artifacts | Verifying the intended alternative actually ran and did not fall back | Treat the run as invalid when activation cannot be proven |
| Correctness grading | Tests, output parser, diff, schema validation, or independent oracle | Every scenario before cost comparison | Manual blind grading against written expectations |
| Performance and cost | Monotonic timer, resource metrics, token or usage telemetry, tool-call logs, and failure counts | Metrics are observable through the same method for all candidates | Label derived or estimated values and keep them out of measured aggregates |
| External semantics | Official documentation and specifications | Candidate configuration or claimed behavior needs current verification | Primary-source web research; otherwise mark the claim UNVERIFIED |
Do not tune the scenario after observing a preferred candidate, mix measurements from different workloads, or present internal estimates as externally measured facts. Benchmarking may create temporary worktrees and artifacts but must not change the source baseline or unapproved external state.
Evidence Rules
- Hold all non-tested variables constant or record and analyze the confounder.
- Correctness failure cannot be compensated by better speed, token use, or cost unless the decision explicitly allows degraded correctness.
- Use repeated runs and report raw values, center, spread, failures, and outliers; never headline the best run.
- Keep setup or indexing cost, steady-state cost, maintenance burden, and runtime cost separate.
- Report measured, derived, estimated, and qualitative evidence in distinct fields.
Checklist
1. Define the Decision and Experiment
- [ ] State the decision the benchmark must support, the candidates, intended users, representative workloads, and explicit non-goals.
- [ ] Include ordinary cases where a simpler or built-in candidate could reasonably win as well as cases exercising each candidate's claimed advantage; do not construct a feature demo for one side.
- [ ] Define scenario inputs, expected outcomes, correctness criteria, failure conditions, and an oracle independent of candidate self-report.
- [ ] Define primary and secondary metrics, units, measurement point, acceptance threshold, allowed tradeoffs, and tie or inconclusive rules.
- [ ] Define the treatment variable, then hold other relevant variables fixed: revision, model, prompts, permissions, runtime, hardware, data, caches, and network. A variable being compared cannot also be declared fixed.
- [ ] Predeclare repetitions using existing variability evidence or a bounded pilot, or a bounded sequential rule with fixed error tolerance and maximum runs. Before large-scale, long-running, or model-cost comparisons, agree the outcome, scope, quality criteria, and acceptable costs; prepare a small representative sample within authorized scope and scale only after its acceptance. Reuse unchanged authorization; ordinary necessary checks need no new approval. Preserve the repetition rule and return
INCONCLUSIVEif the budget cannot resolve the meaningful effect. - [ ] Freeze and hash the scenario text, fixtures, expectations, runner, parser, and decision rule before the first candidate result is inspected.
- [ ] Read repository instructions and inspect Git state before creating worktrees, temporary data, or runners.
- [ ] Start a run-owned resource ledger with every created absolute path, worktree, process ID, cache, account, dataset, report, and temporary artifact; never register pre-existing resources or credentials as cleanup targets.
- [ ] Require side-effect-free or idempotent scenarios and disposable accounts or datasets; define external-write authorization, cost and rate budgets, cleanup, and rollback evidence before execution.
- [ ] Return
BLOCKEDif candidates do not solve the same task, correctness cannot be independently graded, external effects cannot be isolated, or the decision rule is being chosen after results.
2. Build a Symmetric Harness
- [ ] Use the same runner, timeout, logging, environment construction, and artifact collection for every candidate.
- [ ] Create clean worktrees or equivalent isolated copies from the same commit and verify identical starting state.
- [ ] Inventory global and user-level instructions, hooks, plugins, settings, credentials, caches, and environment variables that can leak candidate behavior across supposedly isolated arms; disable, equalize, or record each one.
- [ ] Control seeds, clock, locale, concurrency, network access, dependency versions, cache state, warmup, and scenario order where they can influence results.
- [ ] Record exact candidate configuration, feature flags, prompts, command lines, permissions, and versions.
- [ ] Define a symmetric tuning policy and budget—default configuration, equally tuned configuration, or both—so one candidate is not optimized after seeing the other's result.
- [ ] Add an activation check that proves each candidate was used and identifies silent fallback, partial activation, or mixed execution.
- [ ] Validate the output parser, diff rules, test oracle, and metric collector on known pass and fail fixtures before benchmarking.
- [ ] Separate one-time setup, indexing, compilation, or download cost from steady-state execution and amortized cost.
- [ ] Define cleanup and failure recovery from the resource ledger so a crashed or timed-out run cannot contaminate later runs.
3. Execute and Capture Evidence
- [ ] Run candidates in a balanced or randomized order that avoids systematic warm-cache or temporal advantage.
- [ ] Capture start and end state, command, exit status, timing, resource metrics, logs, outputs, diffs, tests, and candidate-specific artifacts for every run.
- [ ] Verify activation before grading; mark unproven or fallback runs invalid rather than assigning them to the intended candidate.
- [ ] Grade correctness against the predefined oracle before examining performance and cost metrics.
- [ ] Grade task completeness separately from correctness and efficiency: verify every required outcome and prohibited side effect, rather than treating a smaller diff, lower token count, or successful subset as completion.
- [ ] Blind manual or qualitative graders to candidate identity and randomize presentation order; record disagreements instead of resolving them toward a preferred candidate.
- [ ] Record timeout, crash, malformed output, partial completion, tool error, and environmental failure as distinct failure classes.
- [ ] Follow the predeclared repetition or sequential stopping rule; count and preserve all attempts, including failed and invalid runs. Do not keep retrying until the desired number of successes appears.
- [ ] Pause when environmental drift, rate limits, external outages, background load, or runner defects make additional runs incomparable.
- [ ] After a harness fix, invalidate affected comparisons and rerun both candidates for those scenarios under the corrected harness; retain unaffected evidence and label prior invalid results.
- [ ] Treat setup, activation, parser, and environmental failures separately from task incorrectness, then state whether setup reliability is part of the actual product decision.
4. Analyze Validity and Results
- [ ] Exclude only runs that meet a predefined invalidation rule and record the reason, evidence, and whether exclusion changes the conclusion.
- [ ] Report per-scenario correctness, failures, latency, resource use, tokens or usage, tool calls, and other costs before aggregating.
- [ ] Use median, percentile, confidence interval, or another statistic appropriate to the sample and distribution; show spread and sample size.
- [ ] Keep metrics with different units or workloads separate and avoid a single composite score unless its weighting was defined before execution.
- [ ] Check whether differences exceed measurement noise and whether one scenario dominates the aggregate.
- [ ] Analyze setup cost, steady-state cost, maintenance complexity, portability, failure behavior, and operational burden separately from runtim
Truncated for display — read the full file on GitHub.
Related Skills
x402
121.0kSet up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key.
algorithmic-art
177.9kCreating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems.
pptx
177.9kUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an em…
prospecting
60.1kWhen the user wants to find and qualify prospects for outbound, or asks who to target first, which signal to prioritize, or where to spot buyers with the problem: buying signals, enrichment, verification, account research.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
