Rawl

This is a fork of Chip Player JS that focuses on music theory analysis of MIDI and NES tracks. All credits for tremendous original work go to Matt Montag and all contributors to libraries that he managed to wire together. I'm just writing a tiny layer on top. The licensing is the same.

My stuff lives primarily in src/components/rawl, I also removed some features from the UI to maximize useful display area for analysis. Annotations live in src/corpus/analyses.json.

Your own annotations currently won't be saved, but I can fix that - please, contact me.

Setup instructions

Try yarn install and yarn start. If it doesn't work, try reading https://github.com/mmontag/chip-player-js/. Don't compile wasm, I pre-compiled it and added it to the repo. .

12 Colors

https://twitter.com/vitalypavlenko/status/1771820942680830417

Input formats

MIDI

MIDI files can be dropped into https://rawl.rocks/ for visualization. Saving for user files isn't supported yet (contact me). Additionally, there's a decent built-in MIDI library coming from Chip Player JS project: rock/pop, classical, jazz

TODO

This content should be transformed so that it's discoverable via search engines on queries like "yellow submarine analysis".

Also, the environment should make it super easy to record a YouTube video with the analysis, and even the one synchronized with real audio instead of a MIDI rendering.

Also, we should probably explore other MIDI datasets, eg. MetaMIDI, Los Angeles, 130k.

Also, this should probably be available on Ultimate Guitar.

Digital sheet music

There should be a tool to color noteheads of MusicXML and MuseScore files in 12 colors. It should be deployed as a drag-n-drop web app. It should have an instantly playable preview (eg. using https://opensheetmusicdisplay.org/)

We can upload some colored files to MuseScore for the discoverability of our system. I'm not sure how to balance and not to abuse this marketing channel.

Apparently, some system of coloring will be shipped with MuseScore very soon. We don't know yet how customizable that will be.

PDF sheet music

There should be an Optical Music Recognition (OMR) pipeline to color noteheads in place. We can probably do that by tweaking https://github.com/BreezeWhite/oemer

Audio

There should be a tool that colors spectrogram in 12 colors. See a prototype on Chromatone.

Also, there should be a splitter / recongition pipeline. See transcription notes

Video game music

The early VGM music is in unique position: it allows a precise extraction of a piano roll and it's transmitted in rips which sound almost exactly like composers designed them to.

Early consoles used simple oscillators (NES, Sega Game Gear, Sega Master System). Sega Genesis used FM synthesis. It's easy to extract pitches from those. Starting from SNES, I'm not so sure - it may require per-channel pitch recognition or a geeky digging into SNES DSP internals.

Live performance

Eg. a solo piano performance made from a digital piano should take MIDI notes and visualize them in real-time. Someone should take care of switching a tonic.

NES

Hypotheses

Languages on NES are clusters that are far away from each other. Some games that are unbound by external influences try to make tracks for levels so that all levels are as far from each other as possible. To achieve that, they employ the following languages:

• common practice (periods, V/V and other applied dominants)
• jazz (this one is rare)
• 12-bar blues or blues scale (I-IV or stasis)
• chiptune (constant structures, modulations with exact repetitions, mixolydian shuttles, Super Mario cadence or phrygian stasis, parallel keys, quartal)
• hijaz - surprisingly, without any Near East connotations
• chromatic
• atonal
• pentatonic, including W-arpeggios - specifically when depicting China or Japan

Two things are interesting:

• these languages most likely don't blend within a single track, but may well go together in different levels of the same game
• other world language are absent: hemitonic pentatonics, gamelan, maqamat. all tunings are 12-edo

Another hypothesis is that Chinese developers largely used Chinese traditional pentatonic-based tonic-ambiguos language with guzheng imitations, disregarding the game content. (Or did they describe it in their own language, which I as a Westerner don't understand?)

What if every bootleg country has some unique musical language?

Also, it feels like there was a gradual development of cool timbral hacks and compositional form. Games from 1986 are simpler than games from 1993.

Composers

Certain composers for NES can have very recognizable style and musical toolchain. List of sound drivers may also help. This is also a good way to listen to "something similar to that cool track that I've found".

Levels of tailored education

Let's build education around asking a student to analyze pieces of music. Then the levels of pieces of increasing attractiveness:

1. Genres that the student doesn't listen to much, in a form that's hard to navigate. Eg. common-practice classical music in sheet music
2. 12-colored piano rolls of random pop pieces available in MIDI. It's for the brave who want to get into the unknown
3. Same for favorite artists for which MIDIs are available.
4. Same but music only is available, so the MIDI is crafted via automatic transcription.