Introduction

A collection of programs for working with various games in and out of the BEMANI series. This could be untangled quite a bit into various modules that provide simpler pieces. However, this is how it ended up evolving over time. This repository includes utilities for unpacking (and sometimes repacking) various file formats, emulating network services for various games, utilities for sniffing, redirecting and reconstructing network packets, utilities for gathering information about various game music databases and associated tooling that makes developing the previous utilities easier. It is meant to be a complete ecosystem for somebody looking to provide hobby network services to themselves in order to preserve a particular era of gaming that is no longer officially supported.

Thanks to Tau for the great writeup on the binary network format. Thanks to some rando on stack overflow for RC4 code for Python. Thanks to some other rando on stack overflow for sample sniffer code for Python. Thanks to Tau again for the great logging in easerver to compare my original output to. Thanks to PKGINGO for encouragement and well-received excitement about progress. Thanks to Sarah and Alice for being good RE partners for several games, and sharing good finds. Thanks to helvetica for helping with game RE and retweeting cute anime ladies onto my feed every goddamn night.

2dxutils

A utility for unpacking and repacking .2dx audio container files. Can extract RIFF WAV audio out of an existing .2dx file, update an existing file or create a new file from scratch given some WAV files. This isn't the best utility and I think there are more complete and more accurate programs out there. However, they all lack source as far as I could tell, so I developed this. Run it like ./2dxutils --help to see help output and determine how to use this.

afputils

Utilities for working with several animation formats found across a vast range of games. This includes a TXP2 container parser and repacker, a GE2D shape parser and an AFP/BSI parser. Together, they make a set of utilities that attempts to work with AFP, the fork of SWF that handles animations in various games. This utility is capable of rendering animations out of IFS and TXP2 files as well as providing decompiled pseudocode for the flash-like bytecode found in many animation files. It is also capable of unpacking and repacking TXP2 containers with new texture files. Note that this format is based on SWF and thus very complicated. Therefore, it is unlikely that these tools will correctly handle all animations from all games that it encounters. Run it like ./afputils --help to see help output and determine how to use it.

api

Development version of this repository's BEMAPI implementation. This serves as the REST-like API layer for inter-network federation of game song catalogs, scores, profiles and rivals. Run it like ./api --help to see help output and determine how to use this. Much like "services" and "frontend", this should be pointed at the development version of your services config file which holds information about the MySQL database that this should connect to as well as what game series are supported. See config/server.yaml for an example file that you can modify.

Do not use this utility to serve production traffic. Instead, see bemani/wsgi/api.wsgi for a ready-to-go WSGI file that can be used with a Python virtualenv containing this project and its dependencies, uWSGI and nginx. Note that if you do not wish to make your network available for federation then this entire service can be omitted.

arcutils

A utility for unpacking .arc files. This does not currently repack files. However, the format is so trivial that adding such a feature would be fairly easy. Run it like ./arcutils --help to see help output and determine how to use this.

assetparse

A utility which takes a particular game's asset directory and converts files for use on the frontend. This optionally enables things such as customization previews on the frontend. Much like "read", this requires a properly setup config file which points the frontend as well as this utility at the correct location to store converted assets. See config/server.yaml for an example file that you can modify. Unlike "read", this utility is entirely optional. However, if you do not convert assets for games that you are running, you will miss out on preview graphics on the frontend. Run it like ./assetparse --help to see help output and determine how to use this.

bemanishark

A wire sniffer that can decode eAmuse packets and print them. Run it on a computer that can sniff traffic between an eAmusement server and a supported game and it will spit out the requests and responses XML-formatted identically to the XML output in the logs of "services". This works on both binary and XML traffic. Note that it does not have the capability to sniff SSL-encrypted traffic, so don't even bother trying to run this at an arcade with official support.

Run it like sudo ./bemanishark to invoke. Will run indefinitely until killed (Ctrl-C will suffice). Run like ./bemanishark --help for options. Without options, it assumes you want to sniff port 80 for all addresses. Note that it doesn't support the Base64 binary blob formats found in SN1 and 2. Note also that over time it will start to lose packets. This is a bug that I never figured out, and it appears to be the OS failing to send over some packets resulting in a failure to reassemble the TCP stream.

This utility might be better if rewritten to be a plugin for Wireshark instead of a standalone sniffing utility, but I don't have the time.

binutils

A utility for unpacking raw binxml data (files that use the same encoding scheme as the binary network protocol) to their XML representation. This is useful for examining raw binary blobs or digging into unknown file formats that contain binxml. Run it like ./binutils --help to see help and learn how to use this.

bootstrap

A utility for quickly bootstrapping a local setup's music database from an already running BEMAPI-compatible server that has been set up for federation. This is better documented in the below "Database Initialization" section. Note that this utility assumes no omnimix support and will bootstrap only normal game databases. The BEMAPI federation protocol does support omnimix, so if you are bootstrapping against a running instance that has omnimix databases and you wish to support omnimixes as well, you can look at the source to this and manually run commands for the games in question.

cardconvert

A command-line utility for converting between card numbers written on the back of a card and the card ID stored in the RFID of the card. Run it like ./cardconvert --help to see how to use this. This will sanitize input, so you can feed it card numbers with or without spaces, and you can mix up 1 and I as well as 0 and O, and it will properly handle decoding. This supports both new and old style e-Amusement cards but does not support the cross-play network cards with five groups of digits on the back of the card.

dbutils

A command-line utility for working with the DB used by "api", "services" and "frontend". This utility includes options for creating tables in a newly-created DB, granting and revoking admin rights to the frontend, generating migration scripts for production DBs, and upgrading production DBs based on previously created migration scripts. Its driven by alembic under the hood. You will use create on initial setup to generate a working MySQL database. If you change the schema in code, you can use this again with the generate option to generate a migration script. Whenever you run an upgrade to your production instance, you should run this against your production DB with the upgrade option to bring your production DB up to sync with the code you are deploying. Run it like ./dbutils --help to see all options. The config file that this works on is the same that is given to "api", "services" and "frontend".

formatfiles

A simple wrapper frontend to black, the formatter used on this project. Running this will auto-format all of the python code that might need formatting, leaving the rest out. When submitting pull requests make sure to run this so that your code conforms to the style used by this project! Run this like ./formatfiles to fix up all files in the repository.

frontend

Development version of a frontend server allowing for account and server administration as well as score viewing and profile editing. Run it like ./frontend --help to see help output and determine how to use this. Much like "services" and "api", this should be pointed at the development version of your services config file which holds information about the MySQL database that this should connect to as well as what game series are supported. See config/server.yaml for an example file that you can modify.

Do not use this utility to serve production traffic. Instead, see bemani/wsgi/api.wsgi for a ready-to-go WSGI file that can be used with a Python virtualenv containing this project and its dependencies, uWSGI and nginx. Note thati this shares a config file with "services" and "api" but is independent, sharing state with them using the production DB only.

ifsutils

A mediocre utility that can extract .ifs files. This has a lot of baked in assumptions and is not nearly as good as other open-source utilities for extracting files. It also cannot repack files yet. This is included for posterity, and because some bootstrapping code requires it in order to fully start a production server. Run it like ./ifsutils --help to see help output and learn how to use it.

iidxutils

A utility for patching IIDX music database files. Note that this currently can only apply a "hide leggendarias from normal folders" patch, although it would be trivial to extend for other uses