WebAudioModules API

This repository includes API definition files for WebAudioModules (WAMs) written in TypeScript and abstract classes that conform to the API written in JavaScript. The API is designed for making web-based modular audio plugins and using them in compatible hosts.

The legacy WAM API is available in branch v10.

Installing the API

npm i -D @webaudiomodules/api

API definitions

The WAM API provides a specification which should be implemented in each WAM plugin or host. All the interfaces and types in the specification are documented in TypeScript in src/types.d.ts.

Similar to the VST, AudioUnit or AAX standards supported by desktop DAWs, WAMs are modular audio plugins which include a DSP component and a UI component along with some extra features such as parameter automation, MIDI message processing, and state saving/loading, etc. Plugins and hosts which conform to the standard defined by the API are guaranteed to be compatible, regardless of their underlying implementations.

VSCode IntelliSense will take the types into account by using JSDoc or TypeScript import. For example:

// JavaScript
/** @typedef {import('@webaudiomodules/api').WamEvent} IWamEvent */

// TypeScript
import { WamEvent } from '@webaudiomodules/api';

Features

The API supports these primary features:

• Getting information about the WAM by fetching a JSON file.

• Getting the WAM plugin constructor by fetching an ECMAScript Module file.

• Getting a WebAudio AudioNode that can be inserted into an existing audio graph.

• Saving and restoring the plugin's state.

• Getting parameter information from both main thread and audio thread (via AudioWorklet).

• Scheduling automation events of plugin parameters from both threads.

• Scheduling transport, MIDI, and OSC events from both threads.

• Managing event connections between WAM plugins.

• Emitting events to downstream WAM plugins.

• Cleaning up when the plugin instance is destroyed.

• Facilitating an alternative to import statements on the audio thread.

• Allowing hosts to directly access plugin processor instances on the audio thread.

API Overview

The interfaces defined are:

• WebAudioModule, which is the main entry point of a WAM plugin instance.

• WamDescriptor, which contains general information about the plugin.

• WamNode, which extends WebAudio's AudioNode and can be inserted into the host's audio graph.

• WamProcessor, which extends WebAudio's AudioWorkletProcessor and processes signals in the audio thread.

• WamParameterInfo, which provides parameter metadata and convenience methods.

• WamParameter, which provides parameter state information.

• WamEvent, which provides information for scheduling or emitting WAM related events like automation or MIDI messages.

• WamGroup, which maintains graph information for hosts and sub-hosts on the audio thead.

• WamEnv, which manages WamGroups, registers WamProcessors, and stores plugin dependencies on the audio thread.

WebAudioModule interface

A WAM distribution should include at least a JSON descriptor file and a JavaScript file that exports by default a WebAudioModule constructor. The constructor should provide statically:

• isWebAudioModuleConstructor getter that returns true.

• createInstance method that asynchronously instantiates the WebAudioModule.

  This method is a short hand for calling the constructor then the initialize method, and should return a Promise that resolves the WebAudioModule constructed and initialized.

• the new constructor.

  The WAM instance constructed by the new operator is only usable after calling initialize method.

After importing the default export from the ESM module, the host can first do a type check using the isWebAudioModuleConstructor getter, then construct the WAM instance using the createInstance method. The following example demonstrates the steps required for a host to create a WAM using the WAM SDK:

/** @typedef {typeof import('@webaudiomodules/api').WebAudioModule} WebAudioModuleConstructor */
(async () => {
    const audioCtx = new AudioContext();
    // Init WamEnv
    const { VERSION: apiVersion } = await import("@webaudiomodules/api");
    const { addFunctionModule, initializeWamEnv, initializeWamGroup } = await import("@webaudiomodules/sdk");
    await addFunctionModule(audioContext.audioWorklet, initializeWamEnv, apiVersion);
	const hostGroupId = 'example-host'; // will be known by host's WAMs
	const hostGroupKey = performance.now().toString(); // should be kept secret from host's WAMs
	await addFunctionModule(audioContext.audioWorklet, initializeWamGroup, hostGroupId, hostGroupKey);

    // Init WAM
    const initialState = {};
    const imported = await import('./path_to_wam/index.js');
    /** @type {WebAudioModuleConstructor} */
    const WAM = imported.default;
    const isWAM = typeof WAM === 'function' && WAM.isWebAudioModuleConstructor;
    if (!isWAM) return;
    const wam = await WAM.createInstance(audioCtx, initialState);
    return wam;
})();

Here,

const wam = await WAM.createInstance(audioCtx, initialState);

is equivalent to

const wam = new WAM(audioCtx);
await wam.initialize(initialState);

The following getters and methods should also be implemented.

• isWebAudioModule getter that returns true.

• audioContext getter that returns the current BaseAudioContext the WAM belongs to.

• audioNode getter that returns the AudioNode to be inserted into an audio graph.

• initialized getter that returns false before initialized, and true after.

• groupId getter that returns an identifier for the WAM instance's WamGroup.

• moduleId getter that returns an identifier for the WAM, usually composed by its vender + its name.

• instanceId getter that returns the unique identifier for the WAM instance.

• descriptor getter that returns a WamDescriptor containing the same information found in the WAM's JSON file.

• name getter that returns the WAM's name.

• vendor getter that returns the WAM vendor's name.

• createAudioNode method that asynchronously instantiates an AudioNode (which may or may not be a Wamnode which will be inserted into the host's audio graph.

• initialize method that asynchronously initializes the newly constructed WAM and creates its AudioNode via createAudioNode. After initialization, the WAM will be ready to connect its AudioNode to the host's audio graph.

• createGui method that asynchronously creates an Element containing the WAM's GUI which can be attached to the HTML Document.

  There could be multiple GUIs controlling the same WAM, for example if the host generates its own controls to adjust plugin parameters. Make sure the WAM's primary GUI can both control the WAM and responding to any state changes that might occur via interactions with the host.

• destroyGui method that cleans up the WAM's existing but no longer useful GUI element created via createGui.

For example, a host can get and append to the document the WAM's GUI by doing following:

(async () => {
    const container = document.getElementById('wam-container');
    const wamGui = await wam.createGui();
    container.appendChild(wamGui);
})();

and remove it by:

wamGui.remove();
wam.destroyGui(wamGui);

To connect an initialized WAM to an audio graph:

(async () => {
    const defaultConstraints = {
        audio: {
            echoCancellation: false,
            mozNoiseSuppression: false,
            mozAutoGainControl: false,
        },
    };
    const stream = await navigator.mediaDevices.getUserMedia(defaultConstraints);
    const inputNode = audioCtx.createMediaStreamSource(stream);

    const { audioNode } = wam;
    inputNode.connect(audioNode);
    audioNode.connect(audioCtx.destination);
})();

WamDescriptor interface

The WAM descriptor contains information that can be used by the host to properly categorize, display, and load a WAM. The WamDescriptor interface is an object used in the WAM's descriptor JSON file and in its instance's descriptor property. It has the following fields:

• name: the WAM's name.
• vendor: the WAM vendor's name.
• version: the WAM's version (string).
• apiVersion: the WAM API version used (string).
• thumbnail: a URL containing an image for the WAM's thumbnail.
• keywords: an array of keyword strings.
• isInstrument: true if the WAM is a MIDI instrument (boolean).
• description: text describing the behavior of the WAM.
• website: a URL of the WAM's development website.

The WamDescriptor also contains a set of boolean properties indicating the WAM's IO support. They are optional in the descriptor JSON, but mandatory in the descriptor getter under the WebAudioModule interface. These properties will affect the WAM's behavior in the host when it receives audio or events from upstream WAMs.

• hasAudioInput
• hasAudioOutput
• hasMidiInput
• hasMidiOutput
• hasAutomationInput
• hasAutomationOutput
• hasMpeInput
• hasMpeOutput
• hasOscInput
• hasOscOutput
• hasSysexInput
• hasSysexOutput

WamNode interface

WamNode extends WebAudio's AudioNode. Instances are accessed via the audioNode getter under the WebAudioModule interface.

A WAM host will use its native (or overridden) connect and disconnect methods to r