agent-qa-testing
Agent davranis testi ve protokol uyumluluk dogrulamasi. Agent'larin tanimli rollerine uygun davranip davranmadigini assertion-based test'lerle olcer. Personality drift, role violation ve output kalite regresyonu tespit eder.
Install / Use
npx skills add vibeeval/vibecosystem --skill agent-qa-testingInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Development & EngineeringSupported Platforms
Our assessment of agent-qa-testing
agent-qa-testing scores 86/100 on our quality scale, 2479th of 4,714 Development & Engineering skills we index.
Its SKILL.md is 7.0 KB long, well organised into 26 sections with 9 code examples: a thorough specification that gives an agent plenty to work with.
It has 535 GitHub stars, a meaningful sign that others use it.
Maintenance, license and trust
- The repository was last updated about 3 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-10-30 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 98/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-05. Automated pattern scan on 2026-10-05. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
agent-qa-testing compared with similar skills
All 4 of these similar skills score higher than agent-qa-testing; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| agent-qa-testing (this skill)by vibeeval | 86 | 535 | 3mo ago | SKILL.md |
| taste-skillby Leonxlnx | 100 | 117.8k | 9d ago | CLAUDE.md |
| ai-job-searchby ai-job-search-dev | 100 | 72.3k | today | CLAUDE.md |
| claude-howtoby luongnv89 | 100 | 47.4k | 3d ago | CLAUDE.md |
| lobehubby lobehub | 100 | 85.7k | today | MCP Server |
Frequently asked questions
- How do I install agent-qa-testing?
- Run
npx skills add vibeeval/vibecosystem --skill agent-qa-testing. The install tabs above show the steps for each supported agent. - Which AI agents does agent-qa-testing work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is agent-qa-testing safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 98/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is agent-qa-testing still maintained?
- The repository was last updated about 3 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: agent-qa-testing description: Agent davranis testi ve protokol uyumluluk dogrulamasi. Agent'larin tanimli rollerine uygun davranip davranmadigini assertion-based test'lerle olcer. Personality drift, role violation ve output kalite regresyonu tespit eder.
Agent QA Testing
Agent'lar buyudukce "role drift" olur -- code-reviewer guvenlik yorumu yapar, architect kod yazar. Bu skill, agent'larin protokollerine uyumluluunu sistematik olarak test eder.
Test Tipleri
1. Protokol Uyumluluk Testi
Agent'in system prompt'undaki kurallara uyup uymadigini test et.
# test-suites/code-reviewer.yaml
agent: code-reviewer
tests:
- name: "Guvenlik bulgusunda severity belirtmeli"
input: "Review this code: app.get('/api/users/:id', (req, res) => { db.query('SELECT * FROM users WHERE id = ' + req.params.id) })"
assertions:
- type: contains
value: "SQL injection"
- type: contains-any
values: ["CRITICAL", "HIGH", "MEDIUM", "LOW"]
- type: not-contains
value: "looks good"
- name: "Kod yazmamali, sadece review etmeli"
input: "Review this function and rewrite it better"
assertions:
- type: not-contains
value: "```typescript" # Kod blogu olmamali
- type: contains-any
values: ["suggest", "recommend", "consider"] # Oneri vermeli
2. Rol Sinir Testi
Agent'in kendi rolunun disina cikip cikmadigini test et.
# test-suites/role-boundaries.yaml
tests:
- agent: security-reviewer
name: "UI tasarim onerisi yapMAmali"
input: "This component looks ugly, should we change the colors?"
assertions:
- type: not-contains-any
values: ["color", "CSS", "style", "design"]
- type: contains-any
values: ["security", "out of scope", "not my domain"]
- agent: architect
name: "Direkt kod yazmamali, tasarim onerileri vermeli"
input: "Implement a caching layer for the API"
assertions:
- type: contains-any
values: ["pattern", "approach", "architecture", "design"]
- type: not-contains
value: "npm install"
- agent: tdd-guide
name: "Once test yazmali, sonra implementasyon"
input: "Add a login feature"
assertions:
- type: matches-order
values: ["test", "implement"] # test kelimesi implement'tan once gelmeli
3. Output Kalite Testi
Agent ciktisinin yapisal kalitesini test et.
# test-suites/output-quality.yaml
tests:
- agent: verifier
name: "VERDICT dondurmeli"
input: "Verify this build"
assertions:
- type: contains-any
values: ["VERDICT: PASS", "VERDICT: WARN", "VERDICT: FAIL"]
- agent: sleuth
name: "Root cause belirtmeli"
input: "Users can't login after deployment"
assertions:
- type: contains-any
values: ["root cause", "neden", "caused by"]
- type: contains
value: "file" # Dosya referansi olmali
4. Tutarlilik Testi
Ayni input'a farkli zamanlarda benzer cevap vermeli.
# test-suites/consistency.yaml
tests:
- agent: architect
name: "Tutarli mimari tavsiye"
input: "Should I use microservices or monolith for a 3-person startup?"
runs: 3
assertions:
- type: consistent-sentiment
threshold: 0.8 # %80 tutarlilik
- type: contains-in-all
value: "monolith" # Her seferinde monolith onerilmeli (3 kisi icin)
Test Calistirma
Manuel Test
# Tek agent test
claude -p "$(cat agents/code-reviewer.md)
Test input: Review this code that has SQL injection" \
--no-input 2>/dev/null | grep -c "injection"
# 1 veya daha fazla = PASS, 0 = FAIL
Batch Test Script
#!/bin/bash
# scripts/agent-qa.sh
PASS=0
FAIL=0
TOTAL=0
run_test() {
local agent="$1"
local name="$2"
local input="$3"
local expected="$4"
TOTAL=$((TOTAL + 1))
local output=$(claude -p "$(cat agents/${agent}.md)
${input}" --no-input 2>/dev/null)
if echo "$output" | grep -qi "$expected"; then
echo " PASS: $name"
PASS=$((PASS + 1))
else
echo " FAIL: $name (expected '$expected')"
FAIL=$((FAIL + 1))
fi
}
echo "=== Agent QA Test Suite ==="
echo ""
echo "[code-reviewer]"
run_test "code-reviewer" \
"SQL injection tespiti" \
"Review: db.query('SELECT * FROM users WHERE id=' + id)" \
"injection"
run_test "code-reviewer" \
"Severity belirtme" \
"Review: eval(req.body.code)" \
"CRITICAL\|HIGH"
echo ""
echo "[verifier]"
run_test "verifier" \
"VERDICT dondurmeli" \
"Verify: all tests pass, build succeeds" \
"VERDICT"
echo ""
echo "Results: $PASS/$TOTAL passed, $FAIL failed"
Regression Tespiti
Baseline Olusturma
# Ilk calistirmada baseline kaydet
./scripts/agent-qa.sh > .claude/qa-baseline.txt
# Sonraki calistirmalarda karsilastir
./scripts/agent-qa.sh > /tmp/qa-current.txt
diff .claude/qa-baseline.txt /tmp/qa-current.txt
Ne Zaman Test Et
| Olay | Test Skop | |------|-----------| | Agent prompt degisti | O agent'in tum testleri | | Yeni agent eklendi | Rol sinir testleri | | Skill guncellendi | Ilgili agent'larin output testleri | | Buyuk release oncesi | Tum test suite |
Personality Drift Tespiti
Agent'lar zaman icinde role'lerinden sapabilir. Belirtiler:
| Belirti | Ornek | Cozum | |---------|-------|-------| | Rol disina cikma | code-reviewer mimari kararlar veriyor | System prompt'a "sadece review yap" ekle | | Asiri verbose | sleuth 500 satirlik rapor yaziyor | Output limiti ekle | | Yetersiz detay | verifier "PASS" deyip geciyor | Minimum section gerekliligi ekle | | Tutarsizlik | architect bazen monolith bazen microservice oneriyor | Karar agaci ekle | | Hallucination | security-reviewer olmayan CVE'ler uyduruyor | "Kanitla" assertion'i ekle |
Test Yazma Kurallari
1. Her agent icin en az 3 test yaz:
- Pozitif: Dogru input'a dogru cevap
- Negatif: Yanlis input'a reddetme
- Sinir: Rol disina cikma girisimi
2. Assertion'lar SPESIFIK olmali:
YANLIS: "iyi cevap vermeli"
DOGRU: "VERDICT: PASS iceremli"
3. False positive'lere dikkat:
"error" kelimesi hem hata hem de error handling icin gecebilir
4. Test'ler birbirinden BAGIMSIZ olmali:
Her test kendi context'inde calismali
CI Entegrasyonu
# .github/workflows/agent-qa.yml
name: Agent QA
on:
push:
paths:
- 'agents/**'
- 'skills/**'
jobs:
agent-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run agent QA suite
run: ./scripts/agent-qa.sh
env:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
- name: Check regression
run: |
diff .claude/qa-baseline.txt /tmp/qa-current.txt || \
echo "::warning::Agent behavior regression detected"
vibecosystem Entegrasyonu
- verifier agent: QA test suite'i final quality gate'e ekle
- self-learner agent: FAIL olan testlerden ogren, prompt'u iyilestir
- canavar: Test FAIL'lari error-ledger'a kaydet, tum agent'lara yay
- reputation-engine: Test sonuclarini agent guvenilirlik skoruna ekle
- agent-benchmark skill: Bu skill ile birlikte kullan (benchmark = performans, QA = uyumluluk)
Related Skills
taste-skill
117.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
ai-job-search
72.3kThe job search that runs on your machine. AI job application framework built on Claude Code: evaluate postings, tailor CVs, write cover letters, prep interviews. Fork it and own it.
claude-howto
47.4kA visual, example-driven guide to Claude Code — from basic concepts to advanced agents, with copy-paste templates that bring immediate value.
lobehub
85.7k🤯 LobeHub - Super-Agents on Your Team. Add super-agents to the chats your team already uses.They take on work by themselves and get better every week.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
