SkillAgentSearch skills...

Automatic Speech Recognition (ASR)

Transcribe audio segments to text using Whisper models. Use larger models (small, base, medium, large-v3) for better accuracy, or faster-whisper for optimized performance. Always align transcription timestamps with diarization segments for accurate speaker-labeled subtitles.

Install / Use

npx skills add benchflow-ai/skillsbench --skill automatic-speech-recognition

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

85/100

Category

Automation

Supported Platforms

Zed

Tags

Our assessment of Automatic Speech Recognition (ASR)

Automatic Speech Recognition (ASR) scores 85/100 on our quality scale, 1871st of 2,790 Automation skills we index.

Its SKILL.md is 5.3 KB long, well organised into 24 sections with 7 code examples: a solid amount of guidance for an agent.

With 1,813 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
26/30
Structure
20/20
Description
15/15
Adoption
14/20
Freshness
11/15

Maintenance, license and trust

  • The repository was last updated about 4 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
  • Our last check on 2026-11-04 found the source still online.
  • It is released under the Apache-2.0 license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 98/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-02. Automated pattern scan on 2026-10-02. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

Automatic Speech Recognition (ASR) compared with similar skills

All 4 of these similar skills score higher than Automatic Speech Recognition (ASR); compare them before choosing.

SkillScoreStarsUpdatedFormat
Automatic Speech Recognition (ASR) (this skill)by benchflow-ai851.8k4mo agoSKILL.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
rufloby ruvnet10081.9ktodayCLAUDE.md
Claude-Code-Game-Studiosby Donchitos10039.7k7d agoCLAUDE.md
Scraplingby D4Vinci100103.6k2d agoMCP Server

Frequently asked questions

How do I install Automatic Speech Recognition (ASR)?
Run npx skills add benchflow-ai/skillsbench --skill "Automatic Speech Recognition (ASR)". The install tabs above show the steps for each supported agent.
Which AI agents does Automatic Speech Recognition (ASR) work with?
It is written for Zed, as a SKILL.md file. Other agents that read the same format can often use it too.
Is Automatic Speech Recognition (ASR) safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is Apache-2.0-licensed and scores 98/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is Automatic Speech Recognition (ASR) still maintained?
The repository was last updated about 4 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.

name: Automatic Speech Recognition (ASR) description: Transcribe audio segments to text using Whisper models. Use larger models (small, base, medium, large-v3) for better accuracy, or faster-whisper for optimized performance. Always align transcription timestamps with diarization segments for accurate speaker-labeled subtitles.

Automatic Speech Recognition (ASR)

Overview

After speaker diarization, you need to transcribe each speech segment to text. Whisper is the current state-of-the-art for ASR, with multiple model sizes offering different trade-offs between accuracy and speed.

When to Use

  • After speaker diarization is complete
  • Need to generate speaker-labeled transcripts
  • Creating subtitles from audio segments
  • Converting speech segments to text

Whisper Model Selection

Model Size Comparison

| Model | Size | Speed | Accuracy | Best For | |-------|------|-------|----------|----------| | tiny | 39M | Fastest | Lowest | Quick testing, low accuracy needs | | base | 74M | Fast | Low | Fast processing with moderate accuracy | | small | 244M | Medium | Good | Recommended balance | | medium | 769M | Slow | Very Good | High accuracy needs | | large-v3 | 1550M | Slowest | Best | Maximum accuracy |

Recommended: Use small or large-v3

For best accuracy (recommended for this task):

import whisper

model = whisper.load_model("large-v3")  # Best accuracy
result = model.transcribe(audio_path)

For balanced performance:

import whisper

model = whisper.load_model("small")  # Good balance
result = model.transcribe(audio_path)

Faster-Whisper (Optimized Alternative)

For faster processing with similar accuracy, use faster-whisper:

from faster_whisper import WhisperModel

# Use small model with CPU int8 quantization
model = WhisperModel("small", device="cpu", compute_type="int8")

# Transcribe
segments, info = model.transcribe(audio_path, beam_size=5)

# Process segments
for segment in segments:
    print(f"[{segment.start:.2f}s -> {segment.end:.2f}s] {segment.text}")

Advantages:

  • Faster than standard Whisper
  • Lower memory usage with quantization
  • Similar accuracy to standard Whisper

Aligning Transcriptions with Diarization Segments

After diarization, you need to map Whisper transcriptions to speaker segments:

# After diarization, you have turns with speaker labels
turns = [
    {'start': 0.8, 'duration': 0.86, 'speaker': 'SPEAKER_01'},
    {'start': 5.34, 'duration': 0.21, 'speaker': 'SPEAKER_01'},
    # ...
]

# Run Whisper transcription
model = whisper.load_model("large-v3")
result = model.transcribe(audio_path)

# Map transcriptions to turns
transcripts = {}
for i, turn in enumerate(turns):
    turn_start = turn['start']
    turn_end = turn['start'] + turn['duration']
    
    # Find overlapping Whisper segments
    overlapping_text = []
    for seg in result['segments']:
        seg_start = seg['start']
        seg_end = seg['end']
        
        # Check if Whisper segment overlaps with diarization turn
        if seg_start < turn_end and seg_end > turn_start:
            overlapping_text.append(seg['text'].strip())
    
    # Combine overlapping segments
    transcripts[i] = ' '.join(overlapping_text) if overlapping_text else '[INAUDIBLE]'

Handling Empty or Inaudible Segments

# If no transcription found for a segment
if not overlapping_text:
    transcripts[i] = '[INAUDIBLE]'
    
# Or skip very short segments
if turn['duration'] < 0.3:
    transcripts[i] = '[INAUDIBLE]'

Language Detection

Whisper can auto-detect language, but you can also specify:

# Auto-detect (recommended)
result = model.transcribe(audio_path)

# Or specify language for better accuracy
result = model.transcribe(audio_path, language="en")

Best Practices

  1. Use larger models for better accuracy: small minimum, large-v3 for best results
  2. Align timestamps carefully: Match Whisper segments with diarization turns
  3. Handle overlaps: Multiple Whisper segments may overlap with one diarization turn
  4. Handle gaps: Some diarization turns may have no corresponding transcription
  5. Post-process text: Clean up punctuation, capitalization if needed

Common Issues

  1. Low transcription accuracy: Use larger model (small → medium → large-v3)
  2. Slow processing: Use faster-whisper or smaller model
  3. Misaligned timestamps: Check time alignment between diarization and transcription
  4. Missing transcriptions: Check for very short segments or silence

Integration with Subtitle Generation

After transcription, combine with speaker labels for subtitles:

def generate_subtitles_ass(turns, transcripts, output_path):
    # ... header code ...
    
    for i, turn in enumerate(turns):
        start_time = format_time(turn['start'])
        end_time = format_time(turn['start'] + turn['duration'])
        speaker = turn['speaker']
        text = transcripts.get(i, "[INAUDIBLE]")
        
        # Format: SPEAKER_XX: text
        f.write(f"Dialogue: 0,{start_time},{end_time},Default,,0,0,0,,{speaker}: {text}\n")

Performance Tips

  1. For accuracy: Use large-v3 model
  2. For speed: Use faster-whisper with small model
  3. For memory: Use faster-whisper with int8 quantization
  4. Batch processing: Process multiple segments together if possible

Related Skills

View on GitHub
GitHub Stars1.8k
CategoryAutomation
Updated3mo ago
Forks368

Languages

PDDL

Trust signals

98/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

1 info