Eureka Audio

Eureka-Audio: A 1.7B lightweight audio–language model that matches 7B–30B models on ASR, audio understanding, and paralinguistic reasoning.

Generate Convert Improve

Install / Use

npx skills add Alittleegg/Eureka-Audio

Installs into whichever agent you are using.

About this skill

Quality Score

0/100

README

<img src="assets/eureka_logo_new.png" width="200"/> Eureka-Audio-Instruct <a href="https://huggingface.co/cslys1999/Eureka-Audio-Instruct"> <img src="https://img.shields.io/badge/🤗%20HuggingFace-Model-yellow" alt="HuggingFace"/> </a>   <a href="https://www.modelscope.cn/models/lys1999/Eureka-Audio-Instruct"> <img src="https://img.shields.io/badge/🤖%20ModelScope-Model-blue" alt="ModelScope"/> </a>   <a href="https://arxiv.org/abs/2602.13954"> <img src="https://img.shields.io/badge/📑%20arXiv-Paper-red" alt="Paper"/> </a>

We present Eureka-Audio, a compact yet high-performance audio language model that achieves competitive performance against models that are 4 to 18 times larger across a broad range of audio understanding benchmarks. Despite containing only 1.7B parameters, Eureka-Audio demonstrates strong performance on automatic speech recognition (ASR), audio understanding, and dense audio captioning, matching or surpassing multiple 7B to 30B audio and omni-modal baselines.

News

🎉 Apr 11, 2026: Our paper "MoE Adapter for Large Audio Language Models: Sparsity, Disentanglement, and Gradient-Conflict-Free" has been accepted by ACL-2026 Findings!
Feb 25, 2026: We release the inference code and model weights of Eureka-Audio-Instruct.
Feb 17, 2026: We release the technical report of Eureka-Audio.

Introduction
Architecture Overview
Quick Start
Evaluation
License
Acknowledgements
Citation

Introduction

Eureka-Audio is designed as a lightweight yet powerful audio foundation model capable of handling a wide variety of audio understanding tasks within a single unified framework. Key features include:

Lightweight yet Powerful: Achieve competitive results with only 1.7B parameters, delivering up to 3.7x faster decoding speed compared to larger models.
Universal Audio Understanding: Handle diverse tasks like automatic speech recognition (ASR), audio question answering, audio captioning, speech emotion recognition, and sound event classification.
Competitive Performance: Achieve competitive performance against models that are 4 to 18 times larger, matching or surpassing multiple 7B-30B audio and omni-modal baselines.
DataFlux Pipeline: A closed-loop audio instruction data synthesis and verification pipeline that constructs high-quality, logically consistent supervision from raw audio.
Sparse MoE Adapter: A novel sparsely activated Mixture-of-Experts adapter that explicitly accounts for audio heterogeneity and alleviates cross-modal optimization conflicts.
Open-Source: Release the code and model checkpoints for community research and development.

Architecture Overview

Eureka-Audio consists of three main components:

Audio Encoder: A Whisper-based audio encoder that encodes raw waveforms into high-temporal-resolution acoustic representations, capturing fine-grained perceptual and semantic information present in the audio signal.
Sparse MoE Adapter: A Mixture-of-Experts adapter that maps audio representations into the embedding space of the language model. This design explicitly models the heterogeneity of audio signals at both the semantic and acoustic levels, mitigating optimization conflicts while improving representational efficiency.
Language Model Backbone: Qwen3-1.7B-base serves as the language backbone. After alignment via the MoE Adapter, audio embeddings are concatenated with text token embeddings and jointly modeled by the backbone in a standard autoregressive manner.

Getting Started

Installation

git clone https://github.com/Alittleegg/Eureka-Audio.git
cd Eureka-Audio
pip install -r requirements.txt

Using HuggingFace Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model with trust_remote_code
model = AutoModelForCausalLM.from_pretrained(
    "cslys1999/Eureka-Audio-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

Quick Start

This example demonstrates basic usage for generating text from audio.

# ASR
python infer.py --model_path ${your_model_path} --audio_path ${your_audio_path} --task asr

# QA
python infer.py --model_path ${your_model_path} --audio_path ${your_audio_path} --task qa --question "Descript the audio."

# Caption
python infer.py --model_path ${your_model_path} --audio_path ${your_audio_path} --task caption

Evaluation

Eureka-Audio achieves competitive performance across a wide range of audio benchmarks despite having only 1.7B parameters.

Automatic Speech Recognition (ASR)

<table> <thead> <tr> <th>Datasets</th> <th>Type</th> <th>Model</th> <th>Size</th> <th>WER/CER ↓</th> </tr> </thead> <tbody> <tr> <td rowspan="11">LibriSpeech test-clean | test-other</td> <td rowspan="5">Omni</td> <td>Qwen3-Omni-Instruct</td> <td>30B-A3B</td> <td>1.60 | 2.93</td> </tr> <tr> <td>Ming-Lite-Omni-1.5</td> <td>19B-A2.8B</td> <td>1.90 | 3.54</td> </tr> <tr> <td>MiniCPM-o</td> <td>9B</td> <td>2.01 | 4.87</td> </tr> <tr> <td>Qwen2.5-Omni-7B</td> <td>7B</td> <td>1.53 | 3.19</td> </tr> <tr> <td>Qwen2.5-Omni-3B</td> <td>3B</td> <td>1.68 | 3.90</td> </tr> <tr> <td rowspan="4">Audio</td> <td>Step-Audio-2-mini</td> <td>8B</td> <td>1.41 | 2.76</td> </tr> <tr> <td>Audio Flamingo 3</td> <td>8B</td> <td>1.39 | 2.96</td> </tr> <tr> <td>Qwen2-Audio</td> <td>7B</td> <td>1.74 | 4.01</td> </tr> <tr> <td>Kimi-Audio-7B-Instruct</td> <td>7B</td> <td>1.33 | 2.57</td> </tr> <tr> <td rowspan="2">Ours</td> <td>Eureka-Audio-Base</td> <td>1.7B</td> <td>1.59 | 3.34</td> </tr> <tr> <td>Eureka-Audio-Instruct</td> <td>1.7B</td> <td>1.46 | 3.24</td> </tr> <tr> <td rowspan="11">Fleurs-en</td> <td rowspan="5">Omni</td> <td>Qwen3-Omni-Instruct</td> <td>30B-A3B</td> <td>5.04</td> </tr> <tr> <td>Ming-Lite-Omni-1.5</td> <td>19B-A2.8B</td> <td>5.82</td> </tr> <tr> <td>MiniCPM-o</td> <td>9B</td> <td>6.18</td> </tr> <tr> <td>Qwen2.5-Omni-7B</td> <td>7B</td> <td>5.49</td> </tr> <tr> <td>Qwen2.5-Omni-3B</td> <td>3B</td> <td>5.65</td> </tr> <tr> <td rowspan="4">Audio</td> <td>Step-Audio-2-mini</td> <td>8B</td> <td>4.51</td> </tr> <tr> <td>Audio Flamingo 3</td> <td>8B</td> <td>6.30</td> </tr> <tr> <td>Qwen2-Audio</td> <td>7B</td> <td>6.92</td> </tr> <tr> <td>Kimi-Audio-7B-Instruct</td> <td>7B</td> <td>6.11</td> </tr> <tr> <td rowspan="2">Ours</td> <td>Eureka-Audio-Base</td> <td>1.7B</td> <td>5.73</td> </tr> <tr> <td>Eureka-Audio-Instruct</td> <td>1.7B</td> <td>5.39</td> </tr> <tr> <td rowspan="10">AISHELL-2 ios</td> <td rowspan="5">Omni</td> <td>Qwen3-Omni-Instruct</td> <td>30B-A3B</td> <td>2.63</td> </tr> <tr> <td>Ming-Lite-Omni-1.5</td> <td>19B-A2.8B</td> <td>2.66</td> </tr> <tr> <td>MiniCPM-o</td> <td>9B</td> <td>3.42</td> </tr> <tr> <td>Qwen2.5-Omni-7B</td> <td>7B</td> <td>2.58</td> </tr> <tr> <td>Qwen2.5-Omni-3B</td> <td>3B</td> <td>2.77</td> </tr> <tr> <td rowspan="3">Audio</td> <td>Step-Audio-2-mini</td> <td>8B</td> <td>2.33</td> </tr> <tr> <td>Qwen2-Audio</td> <td>7B</td> <td>3.08</td> </tr> <tr> <td>Kimi-Audio-7B-Instruct</td> <td>7B</td> <td>2.80</td> </tr> <tr> <td rowspan="2">Ours</td> <td>Eureka-Audio-Base</td> <td>1.7B</td> <td>3.17</td> </tr> <tr> <td>Eureka-Audio-Instruct</td> <td>1.7B</td> <td>3.10</td> </tr> <tr> <td rowspan="10">WenetSpeech test-meeting | test-net</td> <td rowspan="5">Omni</td> <td>Qwen3-Omni-Instruct</td> <td>30B-A3B</td> <td>6.12 | 5.29</td> </tr> <tr> <td>Ming-Lite-Omni-1.5</td> <td>19B-A2.8B</td> <td>5.96 | 6.26</td> </tr> <tr> <td>MiniCPM-o</td> <td>9B</td> <td>15.53 | 7.68</td> </tr> <tr> <td>Qwen2.5-Omni-7B</td> <td>7B</td> <td>8.43 | 7.10</td> </tr> <tr> <td>Qwen2.5-Omni-3B</td> <td>3B</td> <td>8.53 | 7.14</td> </tr> <tr> <td rowspan="3">Audio</td> <td>Step-Audio-2-mini</td> <td>8B</td> <td>5.43 | 5.50</td> </tr> <tr> <td>Qwen2-Au