UAPMD: next-gen, cross-platform, multi-format, liberally licensed audio plugin host engine with fully equipped virtual MIDI 2.0 devices provider

UAPMD (Ubiquitous Audio Plugin MIDI Device) is a cross-platform, multi-track audio plugin host sequencer that exposes their control points as virtual MIDI 2.0 devices. Your can use arbitrary MIDI 2.0 client apps to:

• play MIDI 2.0 instruments with 32-bit precision; use Assignable Controllers (NRPNs) to change plugin parameters in 32-bit values (velocity in 16-bit).
• retrieve parameter list as Assignable Controllers and presets as Program List, as long as they are exposed via the plugin APIs. Thus you don't have to remember which controller index maps to the parameter you want, or which program number maps to the tone you need.
• save and load the plugin's binary state (.vstpreset etc.), just like how you use them in a DAW.

We also develop midicci, an fully featured MIDI 2.0 software keyboard that leverages the full potential of this project.

uapmd-app, our proof-of-concept sequencer application, and its engine uapmd-engine, is currently capable of:

• managing a master track and multiple tracks, where
• each track has a linear list of audio plugins, and
• contains audio clips and MIDI clips together
• importing SMF into multiple tracks
• importing audio file into multiple tracks using STEM separator (Demucs so far)
• save and load them as a project

UAPMD targets the following platforms:

| platform | plugin formats | missing features | |-|-|-| | Linux desktop | VST3,LV2,CLAP | | | macOS | VST3,AU(v2/v3),LV2,CLAP | | | Windows | VST3,LV2,CLAP | MIDI 2.0 virtual devices (WIP) | | Android | AAP | MIDI 2.0 virtual devices (but AAP itself supports it statically) | | iOS | AUv3 | | | Web (Emscripten) | WebCLAP | |

Note that there are handful of experimental formats.

UAPMD is based on its own plugin hosting foundation remidy, and released under the MIT license (except for Android which brings in a lot of ApacheV2-licensed libraries). uapmd-app is built strictly on the libraries with liberal licenses.

Build + Install

There is an application uapmd-app that performs almost all features UAPMD provides.

packages

uapmd offers Linux packages on the release pages and GitHub Actions build artifacts, in .deb, .rpm and .tar.xz (They are based on CPack packaging tasks). On macOS the package target generates a DMG image ready to distribute and the build also emits a standalone uapmd-app.app bundle you can drag to Applications. On Windows, running the same target produces a ZIP archive, and if NSIS is installed you also get a standard installer executable.

uapmd offers Homebrew package as well. You can install it as: brew install atsushieno/oss/uapmd then run /opt/homebrew/bin/uapmd-app or use those libraries the package offers. Our package settings are stored at atsushieno/homebrew-oss.

building from source

This uapmd Git repository provides the simple normative cmake build:

$ cmake -B build -G Ninja -DCPM_SOURCE_CACHE=~/.cache/CPM/uapmd    # you can skip -DCPM_SOURCE_CACHE
$ cmake --build build
$ cmake --build build --target package # if you prefer package files

If you are using Windows:

$ cmake -B build -G "Visual Studio 17 2022" -DBUILD_SHARED_LIBS=OFF -DREMIDY_BUILD_CONFIG=Release -DUAPMD_ENABLE_WINMIDI=ON -DCPM_SOURCE_CACHE=%HOME%\.cache\CPM\uapmd    # you can skip -DCPM_SOURCE_CACHE
$ cmake --build build
$ cmake --build build --target package # if you prefer package files

After successful build on those desktop platforms, the artifacts are found like: *.deb, *.rpm, *.tar.xz, uapmd-*.zip, or *.dmg (under build directory)

If you target Android:

$ cd android && ./gradlew build

If you target iOS:

$ bash build-ios-sim.sh CPM_SOURCE_CACHE=~/.cache/CPM/uapmd   # you can skip CPM_SOURCE_CACHE

If you target Web:

$ bash build-wasm-imgui.sh CPM_SOURCE_CACHE=~/.cache/CPM/uapmd   # you can skip CPM_SOURCE_CACHE

Then you can run it like:

npx http-server cmake-build-wasm/source/tools/uapmd-app

Screenshots

I put them on the wiki pages (note that other than the latest ones they are more like historical records).

What's the point of these tools?

v0.1: With UAPMD, You do not have to wait for MIDI 2.0 synthesizers in the market; existing audio plugins should work as virtual MIDI 2.0 devices. We have timidity++ or fluidsynth, Microsoft GS wavetable synth, YAMAHA S-YXG etc. for MIDI 1.0. UAPMD will take a similar place for MIDI 2.0.

v0.2: UAPMD works more like a multitrack sequencer that lets you organize audio and MIDI 2.0 clips with audio plugins, to play all together or record statically into audio files.

v0.3: UAPMD works everywhere on desktop, mobile, and web (virtual MIDI 2.0 devices as long as the platform is eligible).

Usage

This repository contains one primary executable uapmd-app.

There are supplemental tools for diagnosing problems we encounter.

uapmd-app

It is the primary multitrack sequencer, virtual MIDI 2.0 device service controller. Currently the public command line options are hacky:

$ uapmd-app (plugin-name) (format-name) (api-name)

plugin-name is match by std::string::contains() within display name, case-sensitive.

format-name is one of VST3 AU, LV2, or CLAP.

api-name so far accepts only PIPEWIRE (on Linux) to use PipeWire, and uses default available API otherwise.

We have some users guide documentation.

uapmd-scan

uapmd-scan is a standalone entry point for the scan-only mode that also powers uapmd-app --scan-only. It always performs a fresh rescan via the remote scanner worker (equivalent to uapmd-app --scan-only --force-rescan --full --remote), enforces per-bundle timeouts (--timeout <seconds>, default 120) so that hung plugins are terminated and skipped, persists the cache to (local app data)/remidy-tooling/plugin-list-cache.json (local app data depends on the platform), and prints the JSON report to stdout.

uapmd-apply

uapmd-apply is an offline rendering engine for *.uapmdz project files. It instantiates all the plugins used in the project, then render a WAV without GUI. You can achieve the same functionality using uapmd-app.

Documentation

ALL docs under docs are supposed to describe design investigation and thoughts.

We are moving quick and may not reflect the latest state of union, or describe our plans correctly.

There are some notable docs:

• Plugin catalog (listing) and instantiation
• State
• GUI support and main thread constraints
• Parameters
• Presets

Code modules

There are two primary libraries in this repository:

remidy

remidy offers plugin API abstraction layer at lower level that primarily involves application agnostic audio and event processing. Apart from parameter API, it adopts UMP for event inputs, including parameter support via NRPN (AC, Assignable Controller) and Per-Note AC. It is an opinionated layer towards MIDI 2.0 i.e. events are parsed into timed parameter changes and other events on the plugins.

remidy-tooling

remidy-tooling offers higher level API to build audio plugin hosting tools like plugin scanning and instancing in the common manner. What this layer introduces in practice is a set of filters; various existing specific plugin products and vendors are filtered by "safe for multithreaded access to the plugin API," "plugin scanning requires the UI thread," or "crashes remidy" kind of information.

uapmd, uapmd-file, uapmd-data, and uapmd-engine

uapmd and uapmd-engine provide reusable foundation for constructing virtual MIDI 2.0 devices upon plugin hosting layer (only remidy so far). uapmd serves AllCtrlList MIDI-CI standard property for plugin parameters as Assignable Controllers (NRPNs), ProgramList MIDI-CI standard property for the indexed presets as Program Change, and saves and loads states in MIDI-CI property manner.

uapmd itself contains core engine behind MIDI-CI processing for audio plugin hosting and instancing API without implementation and usable backends. uapmd-engine goes one step further to establish the premise that there is single audio processing backend, multiple MIDI 2.0 devices, audio graphs, and so on to make everything in usable form.

uapmd-file implements supplemental cross-platform file/document API.

uapmd-data defines the sequencer model such as tracks and clips, then implements audio processing data provider for realtime processing.

uapmd-app

uapmd-app is a plugin host that you can list the installed plugins, instantiate plugins, process audio with a UMP keyboard, adjust parameters, select presets, launch the GUI, save, and restore the states. It also exposes those plugins as platform virtual MIDI 2.0 devices, translating UMP inputs into event inputs to those in each plugin API, as well as exposing some plugin features using MIDI-CI property exchange.

It likely works with MIDI 1.0 inputs (translated, depending on the platform) to control plugins.

License and Dependencies

Sources in this repository are released under the MIT license.

There are third-party (and first-party) dependency libraries (git submodules, CMake FetchContent, or directly included):

• lv2/lv2kit (serd, sord, sratom, lilv, zix): th