Aurio: Audio Stream Processing & Retrieval for .NET

Aurio is an open source .NET audio library for stream processing, analysis and retrieval.

<!-- nuget-exclude:start --> <p align="center"> <img src="aurio-icon.png" alt="Aurio logo" width="100"/> </p> <!-- nuget-exclude:end -->

Features

• 32-bit floating point audio stream processing engine
• File I/O (using NAudio and FFmpeg)
• FFT and iFFT (using Exocortex.DSP, FftSharp, FFTW, PFFFT)
• Resampling (using NAudio, libsamplerate, Soxr)
• Windowing, overlap-adding, STFT, iSTFT
• Chroma
• Dynamic Time Warping
• On-line Time Warping (Dixon, Simon. "Live tracking of musical performances using on-line time warping." Proceedings of the 8th International Conference on Digital Audio Effects. 2005.)
• Fingerprinting
  • Haitsma, Jaap, and Ton Kalker. "A highly robust audio fingerprinting system." ISMIR. 2002.
  • Wang, Avery. "An Industrial Strength Audio Search Algorithm." ISMIR. 2003.
  • Echoprint (Ellis, Daniel PW, Brian Whitman, and Alastair Porter. "Echoprint: An open music identification service." ISMIR. 2011.)
  • AcoustID Chromaprint
• Audio playback
• UI widgets

All audio processing (incl. fingerprinting) is stream-based and supports processing of arbitrarily long streams at constant memory use.

<!-- nuget-exclude:start -->

Getting Started

The easiest way of using Aurio is installing the packages through NuGet. For usage, check the code examples and example applications.

<!-- nuget-exclude:end -->

Packages

• Aurio: The core library.
• Aurio.Windows: Audio player and MP3 decoder. Windows only.
• Aurio.WaveControls: WPF controls and utilities for UIs. Windows only.
  • Spectrogram / Chromagram View, Spectrum / Graph View, VU Meter, Correlometer, Time Scale, Wave View

Decoders

| Name | Formats | Description | License | | --- | ----------- | --- | --- | | Aurio (core) | PCM Wave | Managed NAudio decoder. | MIT | | Aurio.Windows | MP3 | Uses Windows's ACM codec through NAudio. Windows only. | MIT | | Aurio.FFmpeg | many | Decodes a very wide range of media container formats and codecs through FFmpeg. Windows and Linux. | LGPL |

Resamplers

| Name | Description | Variable Rate Support | License | | --- | ----------- | --- | --- | | Aurio (core) | Managed NAudio WDL resampler. Recommended for cross-platform use. | yes | MIT | | Aurio.LibSampleRate | Native libsamplerate (a.k.a. Secret Rabbit Code) library via LibSampleRate.NET. Windows only*. | yes | BSD | | Aurio.Soxr | Native SoX Resampler library. Windows only*. | yes (depending on config) | LGPL |

(*) Linux binary not integrated yet.

FFTs

| Name | Description | In-Place Transform | Inverse Transform | License | | --- | ----------- | --- | --- | --- | | Aurio.Exocortex | Exocortex.DSP library. Fastest managed FFT, recommended for cross-platform use. | yes | yes | BSD | | Aurio.FftSharp | FftSharp library. Much slower than Exocortex. | no | yes | MIT | | Aurio.FFTW | Native FFTW (Fastest Fourier Transform in the West) library. Much faster than the managed implementations. Windows only*. | yes | no | GPL | | Aurio.PFFFT | Native PFFFT (pretty Fast FFT) library. Even faster than FFTW, recommended for high-performance use. Windows only*. | yes | yes | FFTPACK |

(*) Linux binary not integrated yet.

<!-- nuget-exclude:start -->

Run the Aurio.Test.FFTBenchmark tool for a more detailed performance comparison, or see native benchmark results here, here, and here.

What's new

See CHANGELOG.

Development

Requirements

• Windows: Visual Studio 2022 (with CMake tools)
• Linux: Ubuntu 22.04, CMake, Ninja
• .NET SDK 6.0

Build Instructions

Open ./nativesrc and ./src/Aurio.sln in Visual Studio, or check the CI workflow for the Windows and Linux CLI build command sequence.

<!-- nuget-exclude:end -->

Documentation

Not available yet. If you have any questions, feel free to open an issue!

Examples

Select Decoders

using Aurio;

// The stream factory automatically selects an appropriate decoder stream
// for a given input. Multiple decoders can be added. The PCM Wave decoder
// is added by default.

// Add Aurio.Windows MP3 decoder
AudioStreamFactory.AddFactory(new Windows.NAudioStreamFactory());
// Add Aurio.FFmpeg decoder
AudioStreamFactory.AddFactory(new FFmpeg.FFmpegAudioStreamFactory());

var stream = AudioStreamFactory.FromFileInfo(new FileInfo("./media.file"));

// Alternatively, decoders can be directly used without the factory
var stream = FFmpegSourceStream(new FileInfo("./media.file"));

Select Resampler

using Aurio.Resampler;
using Aurio.Streams;

// Only one resampler can be selected at a time.

// Use managed NAudio resampler from core
ResamplerFactory.Factory = new Aurio.NAudioWdlResamplerFactory();
// or Aurio.Soxr
ResamplerFactory.Factory = new Aurio.Soxr.ResamplerFactory();
// or Aurio.LibSampleRate
ResamplerFactory.Factory = new Aurio.LibSampleRate.ResamplerFactory();

// Needs resampler factory, throws otherwise
var stream = new ResamplingStream(...);

Select FFT

using Aurio;
using Aurio.FFT;

// Only one FFT can be selected at a time.

// Use Aurio.Exocortex
FFTFactory.Factory = new Exocortex.FFTFactory();
// or Aurio.Exocortex
FFTFactory.Factory = new FftSharp.FFTFactory();
// or Aurio.FFTW
FFTFactory.Factory = new FFTW.FFTFactory();
// or Aurio.PFFFT
FFTFactory.Factory = new PFFFT.FFTFactory();

// Needs FFT factory, throws otherwise
var stft = new STFT(...);

Stream Processing

// Read MKV movie with surround audio
var sourceStream = new FFmpegSourceStream(new FileInfo("media.mkv"));
// Convert to 32-bit
var ieee32BitStream = new IeeeStream(sourceStream);
// Downmix to stereo
var downmixStream = new SurroundDownmixStream(ieee32BitStream);
// Downmix to mono
var monoStream = new MonoStream(downmixStream);
// Concatenate with other streams
var concatStream = new ConcatenationStream(monoStream, anotherStream, ...);
// Turn volume down to 50%
var volumeStream = new VolumeControlStream(concatStream) { Volume = 0.5f, Balance = 1, Mute = false };
// Mix with another stream
var mixerStream = new MixerStream(concatStream.Properties.Channels, concatStream.Properties.SampleRate);
mixerStream.Add(volumeStream);
mixerStream.Add(yetAnotherStream);
// Skip the first 10 samples
var cropStream = new CropStream(mixerStream, mixerStream.Properties.SampleBlockByteSize * 10, mixerStream.Length);
// Clip samples at max volume
var clipStream = new VolumeClipStream(cropStream);
// Downsample to telephone sound quality
var resamplingStream = new ResamplingStream(clipStream, ResamplingQuality.Low, 8000);
// Write it to a WAV fil
var sinkStream = new NAudioSinkStream(resamplingStream);
WaveFileWriter.CreateWaveFile("telephone-audio.wav", sinkStream);

STFT

// Setup STFT with a window size of 100ms and an overlap of 50ms
var source = AudioStreamFactory.FromFileInfoIeee32(new FileInfo("audio.wav"));
var windowSize = source.Properties.SampleRate/10;
var hopSize = windowSize/2;
var stft = new STFT(source, windowSize, hopSize, WindowType.Hann);
var spectrum = new float[windowSize/2];

// Read all frames and get their spectrum
while (stft.HasNext()) {
    stft.ReadFrame(spectrum);
    // do something with the spectrum (e.g. build spectrogram)
}

FFT Equalizer

var source = new IeeeStream(...);
var target = new MemoryWriterStream(new System.IO.MemoryStream(), source.Properties);

var windowSize = 512;
var hopSize = windowSize/2+1; // for COLA condition
var window = WindowType.Hann;
var stft = new STFT(source, windowSize, hopSize, window);
var istft = new InverseSTFT(target, windowSize, hopSize, window);
var spectrum = new float[windowSize/2];

while (stft.HasNext()) {
    stft.ReadFrame(spectrum);
    // manipulate spectrum
    istft.WriteFrame(spectrum);
}
istft.Flush();

Generate fingerprints

// Setup the source (AudioTrack is Aurio's internal representation of an audio file)
var audioTrack = new AudioTrack(new FileInfo("somefilecontainingaudio.ext"));

// Setup the fingerprint generator (each fingerprinting algorithms has its own namespace but works the same)
var defaultProfile = FingerprintGenerator.GetProfiles()[0]; // the first one is always the default profile
var generator = new FingerprintGenerator(defaultProfile);

// Setup the generator event listener
generator.SubFingerprintsGenerated += (sender, e) => {
    // Print the hashes
    e.SubFingerprints.ForEach(sfp => Console.WriteLine("{0,10}: {1}", sfp.Index, sfp.Hash));
};

// Generate fingerprints for the whole track
generator.Generate(audioTrack);

Fingerprinting & Matching

// Setup the sources
var audioTrack1 = new AudioTrack(new FileInfo("somefilecontainingaudio1.ext"));
var audioTrack2 = new AudioTrack(new FileInfo("somefilecontainingaudio2.ext"));

// Setup the fingerprint generator
var defaultProfile = FingerprintGenerator.GetProfiles()[0];
var generator = new FingerprintGenerator(defaultProfile);

// Create a fingerprint store
var store = new FingerprintStore(defaultProfile);

// Setup the generator event listener (a subfingerprint is a hash with its temporal index)
generator.SubFingerprintsGenerated += (sender