Synkro

Synkro reads your policy documents, auto-generates diverse test scenarios, runs multi-turn conversations against your agent with a simulated user, and verifies every response against the policy rules. No hand-written test cases. No manual review.

Installation

pip install synkro

Simulate Your Agent

import synkro

def my_agent(messages):
    resp = openai.chat.completions.create(model="gpt-4o", messages=messages)
    return resp.choices[0].message.content

results = synkro.simulate(
    agent=my_agent,
    policy="All refunds require a receipt. Max refund is $500.",
    scenarios=10,
    turns=3,
)

print(results.pass_rate)  # 0.85
print(results.passed)     # 8 of 10 passed

for r in results:
    if not r.passed:
        print(r.scenario.description, r.issues)

Works with any agent — just pass a callable that takes messages and returns a string. Sync or async.

Generate Training Data

Use the same pipeline to generate training datasets for fine-tuning:

import synkro

dataset = synkro.generate(
    "All refunds require a receipt. Max refund is $500.",
    traces=100,
)
dataset.save("training.jsonl")

Or use the CLI:

synkro generate policy.pdf --traces 50

# Quick demo with built-in policy
synkro demo

Features

• Agent simulation — Test agents against auto-generated, policy-grounded scenarios
• Policy verification — Every conversation verified for rule compliance, contradictions, hallucinations
• Training data generation — Conversation, Instruction, Evaluation, and Tool Calling datasets
• Coverage tracking — Track scenario diversity like code coverage, identify gaps
• Any LLM — OpenAI, Anthropic, Google, Ollama, vLLM
• Any document — PDF, DOCX, TXT, Markdown, URLs

Documentation

Full documentation at synkro.sh/docs

• Quickstart
• Agent Simulation
• Coverage Tracking
• Dataset Types
• API Reference