SkillAgentSearch skills...

BogJS

JavaScript framework for object-based audio rendering in modern browsers from IRT

GenerateConvertImprove

Install / Use

/learn @IRT-Open-Source/BogJS
About this skill

Quality Score

0/100

Supported Platforms

Universal

README

bogJS - JS framework for object-based audio rendering in browsers

With the introduction of HTML5 and the Web Audio API, an important prerequisite was made for native rendering of object-based audio in modern browsers. Object-based audio is a revolutionary approach for creating and deploying interactive, personalised, scalable and immersive content, by representing it as a set of individual assets together with metadata describing their relationships and associations. This allows media objects to be assembled in ground-breaking ways to create new user experiences.

Demo

See bogJS in action: https://lab.irt.de/demos/object-based-audio/RadioDrama/

Basic usage and sample scene

<script
  src="http://code.jquery.com/jquery-3.2.1.min.js"
  integrity="sha256-hwg4gsxgFZhOsEEamdOYGBf13FyQuiTwlAQgxVSNgt4="
  crossorigin="anonymous"></script>

<script src="dist/bogJS-latest.js"></script>
<!-- Only if you want to use the UI relevant parts: -->
<script src="dist/bogJS-ui-latest.js"></script>

<script>    
var om;
$(document).on('om_ready', function(){
    om.start();
});

om = new ObjectManager('path/to/sceneFile.spatdif');
</script>
sceneFile.spatdif
/spatdif/time 0.0
/spatdif/source/Drums/position 0.01 1.24 0
/spatdif/source/Drums/interpolate 0
/spatdif/source/Drums/active 1
/spatdif/source/Drums/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/drums
/spatdif/source/Drums/gain 1
/spatdif/source/Drums/interactive 0
/spatdif/source/Bass/position -0.85 2.55 0
/spatdif/source/Bass/interpolate 0
/spatdif/source/Bass/active 1
/spatdif/source/Bass/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/bass
/spatdif/source/Bass/gain 1
/spatdif/source/Bass/interactive 0
/spatdif/source/Vocal/position -1.06 0.52 0
/spatdif/source/Vocal/interpolate 0
/spatdif/source/Vocal/active 1
/spatdif/source/Vocal/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/vocal
/spatdif/source/Vocal/gain 1
/spatdif/source/Vocal/interactive 0
/spatdif/source/Accordion/position 1.82 1.47 0
/spatdif/source/Accordion/interpolate 0
/spatdif/source/Accordion/active 1
/spatdif/source/Accordion/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/accordion
/spatdif/source/Accordion/gain 1
/spatdif/source/Accordion/interactive 0
/spatdif/source/Guitar/position 1.8 2.23 0
/spatdif/source/Guitar/interpolate 0
/spatdif/source/Guitar/active 1
/spatdif/source/Guitar/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/guitar
/spatdif/source/Guitar/gain 1
/spatdif/source/Guitar/interactive 0
/spatdif/source/Trumpet/position -1.92 -0.59 0
/spatdif/source/Trumpet/interpolate 0
/spatdif/source/Trumpet/active 1
/spatdif/source/Trumpet/track_mapping http://akamai-progressive.irt.de/demos/bogJS/debo/trumpet
/spatdif/source/Trumpet/gain 1
/spatdif/source/Trumpet/interactive 0

Installation

Using the latest version

For your convenience, the dist/ folder contains the latest build files of bogJS.

Building by yourself

You only need npm to build this framework. Once you have npm installed, you must run

npm install

to install the required dependencies.

Now, you can build the project with several commands:

To build and bundle the whole project, run this command in the project folder. This will create another subfolder "dist/" with the files as follows:

npm run build

--> dist/bogJS-latest.js
--> dist/bogJS-latest.js.map
--> dist/bogJS-latest.min.js
--> dist/bogJS-latest.min.js.map
--> dist/bogJS-ui-latest.js
--> dist/bogJS-ui-latest.js.map
--> dist/bogJS-ui-latest.min.js
--> dist/bogJS-ui-latest.min.js.map

To immediately build every change in one of the source files, use this command. The core bundle will then be constantly updated in dist/bogJS-dev.js

npm run watch

--> dist/bogJS-dev.js

To build and bundle a new relese version of the project, use this command. The files will be created under "dist/":

npm run build-release

--> bogJS-vX.X.X.js
--> bogJS-vX.X.X.js.map
--> bogJS-vX.X.X.min.js
--> bogJS-vX.X.X.min.js.map
--> bogJS-ui-vX.X.X.js
--> bogJS-ui-vX.X.X.js.map
--> bogJS-ui-vX.X.X.min.js
--> bogJS-ui-vX.X.X.min.js.map

Note: This command will also add a new Git tag with increased patch number and will also update the package.json file accordingly. To increase the version tag with a minor or even major, execute

npm version minor

or

npm version major

See also https://docs.npmjs.com/cli/version

Documentation

To create the documentation, you will need jsdoc3:

jsdoc -c jsdoc.conf

The default folder for the generated documentation is doc/ .

Version Info

With every build process, the version info is included to the bundle and saved as global javascript variables

__BROWSERIFY_META_DATA__CREATED_AT
// --> "Tue Feb 16 2016 21:13:38 GMT+0100 (CET)"

__BROWSERIFY_META_DATA__GIT_VERSION
// --> "1448b3e v0.2.8"

Basic concepts

The scene file

All object and scene relevant information is stored and read from a scene file. The format currently used is kind of a OSC string combined with SpatDIF commands.

The SceneReader class can be replaced as long as the ObjectManager class instance retrieves the scene and object info as defined. See the documentation for further info.

For the future, it is planned to implement a JSON representation of the ADM format.

Implemented object descriptors

The following parameters can be currently assigned to the objects:

  • Gain [float]: Values for the gain of the audio signal, connected to the PannerNode. Values must be between 0 and 1.
  • Position [float, cartesian, right-hand]: X, Y and Z values represent the position of the objects. See here for further info regarding the coordinate system.
  • Interactive [boolean]: This parameter is intended to be used if the object shall or shall not offer any interactive usage by the user. Example use case might be the adjustment of speech to music level.
  • Active [boolean]: This parameter can be used if the object is in the scene but should not be heard. It is kind of similar to the gain parameter.

It is further planned to implement more sophisticated parameters of ADM in the future.

To load and play audio signals, you can use three different options:

  • Single audio objects (rather short signals up to a few minutes), starting with playback from the very beginning or at a dedicated playback time. These objects will be loaded via XMLHttpRequests and decoded with the Web Audio API built-in decodeAudioData() function. The representation of those signals will be a AudioBufferSouceNode. The bogJS wrapper for this is the AudioData class. A typical use case would be a effect or speech object.
  • Multiple single objects that have the identical duration and that shall be played back simultaneaously. For those objects, there is a dedicated class (IRTPlayer) which can or should be used for this purpose. The IRTPlayer class uses several AudioData instances to load and play files.
  • One or more audio objects with longer duration (> 3 minutes). These audio signals shall have the same duration as the audio scene. A typical usage would be one or multiple audio beds (containing e.g. atmo and music) that can have a file or stream representation. The HTML5 media element is used for this purpose and connected to the Web Audio API via a MediaElementSourceNode. The media file or stream can have multiple channels. Depending on the browser and codec, different maximum track numbers are possible. More than eight channels are currently only possible with a .wav container. See also here.

The channel order detection class

If you use an audio bed for your scene, the ChannelOrderTest() class is used to detect the order of the decoded channels. To make use of this functionality, you have to encode these test files with your preferred encoder and settings. The test set contains uncompressed wav files from 2 channels up to 16 chs with a duration of 48000 samples (1 second@48kHz). For each channel, a different sinus tone is used to detect the order after the encoding and decoding again. Further, you need to upload those files to a folder on the server where the HTML file using bogJS is located. Otherwise you will likely run into a CORS issue. For OS X user, there is a encoding bash script enclosed under /tools.

/* Simple example */
var ch = new ChannelOrderTest("mp4", 6);
ch.testChs();
// -> [0, 2, 1, 4, 5, 3]

/* Advanced example */
var ctx = new AudioContext();
var ch;
$(ch).on('order_ready', function(e, order){
    console.log("Got channel order: " + order);
    doSomething();
});
ch = new ChannelOrderTest("ogg", 4, ctx, "path/to/testfiles/");
// -> Got channel order: [0, 3, 2, 1]

Container and file extensions

If no file extension is given in the scene file, the AudioData() class and the ObjectManager() class will firstly detect the capabilities of the used browser and then try to load the given file with the preferred extension. The order can be read and changed here:

  1. .mp4
  2. .opus
  3. .ogg
  4. .mp3
  5. .wav

Interpolation

Especially for object movements, we need interpolated position updates of the values. The Web Audio API offers with the AudioParam interface the possibility to do this in certain ways. Unfortunately, the PannerNode (used for object positioning) does not support this interface. Only the n

IRT-Open-Source

View profile

View on GitHub
GitHub Stars21
CategoryDevelopment
Updated9mo ago
Forks2
IRT-Open-Source/bogJS

Languages

JavaScript

Security Score

82/100

Audited on Jun 9, 2025

No findings