Libbbf
Bound Book Format: A high-performance container format for comics and manga
Install / Use
/learn @ef1500/LibbbfREADME
libbbf: Bound Book Format
Bound Book Format (.bbf) is a binary container format intended for the ordered storage of page-based media assets. BBF was designed principally for comics, manga, artbooks, and similar sequential image collections.
Bound Book Format additionally enables paginated retrival, asset deduplication, fast indexed access to page order and metadata, and MKV-like sectioning.
BBF is most suited for the sharing, reading, archival and streaming of completed volumes, collections of media, or series (as opposed to ongoing works).
Getting Started
Prerequisites (Muxer, Library, General Use)
- C++17 compliant compiler (GCC/Clang/MSVC), and optionally CMake
- xxHash library
Prerequisites (Debugging/Benchmarking)
Prerequisites (WASM)
- xxHash library
- Emscripten
- CMake
Compilation
CMake
cmake -B build
cmake --build build
sudo cmake --install build
Manual Compilation
Linux
g++ -std=c++17 -O2 -I./src -I./src/muxer -I./src/vend src/muxer/bbfmux.cpp src/bbfcodec.cpp src/muxer/dedupemap.cpp src/muxer/stringpool.cpp src/vend/xxhash.c -o bbfmux
Windows
g++ -std=c++17 -O2 -I./src -I./src/muxer -I./src/vend src/muxer/bbfmux.cpp src/bbfcodec.cpp src/muxer/dedupemap.cpp src/muxer/stringpool.cpp src/vend/xxhash.c -o bbfmux
Alternatively, if you need python support, use libbbf-python.
Technical Details
BBF files are footer-indexed, and the footer can be at either after the header (petrified) for fast reading, or at the bottom (default).
Feature Comparison: BBF v3 (libbbf) and Common Comic Storage Containers
Legend: ✅ = specified in-format; ⚠️ = optional / conventional; ❌ = not in spec
| Capability | BBF v3 (libbbf) | CBZ (ZIP) | CBR (RAR) | PDF | EPUB | Folder | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Memory-mapped parsing | ✅ | ✅ [A] | ✅ [B] | ✅ [C] | ✅ [A][D] | ✅ | | Adjustable alignment | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | Variable alignment (size-based packing) | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | | Content-hash deduplication | ✅ | ❌ | ❌ | ⚠️ [E] | ⚠️ [F] | ⚠️ [G] | | Per-asset integrity value stored with each asset | ✅ (XXH3-128) | ⚠️ (CRC32) [H] | ✅ (CRC32/BLAKE2sp) [I] | ❌ | ⚠️ (CRC32) [H] | ❌ | | Single index-region checksum | ✅ (XXH3-64) | ❌ [J] | ✅ [K] | ❌ | ❌ | ❌ | | Index-first layout / linearization | ✅ (Petrification) | ❌ [L] | ⚠️ [M] | ✅ [N] | ❌ [L] | ❌ | | Arbitrary metadata | ✅ | ⚠️ [O] | ⚠️ [P] | ✅ [Q] | ✅ [R] | ⚠️ [S] | | Hierarchical sectioning / chapters | ✅ | ❌ | ❌ | ✅ [T] | ✅ [U] | ⚠️ [V] | | Customizable page ordering independent of filenames | ✅ | ❌ | ❌ | ✅ [W] | ✅ [U] | ❌ | | Customizable expansion entries | ✅ (expansion table) | ✅ [X] | ✅ [Y] | ✅ [Z] | ✅ [AA] | ❌ |
<sub> [A] Only in STORE mode, which CBZ is typically in.<br/> [B] Only in STORE mode, which CBR is typically in.<br/> [C] PDFs often store page content/images in filtered (compressed) streams; mmap helps parsing.<br/> [D] EPUB is a ZIP-based container with additional rules; it inherits ZIP's container properties.<br/> [E] PDF can reuse already-stored objects/resources by reference, but there is no hash-based deduplication mechanism.<br/> [F] EPUB can reference the same resource from multiple content documents (file-level reuse), but does not have hash-based a deduplication mechanism.<br/> [G] Folder-level deduplication is a filesystem matter (hardlinks, block-dedupe, etc.).<br/> [H] ZIP/EPUB store CRC-32 per member.<br/> [I] RAR stores per-file checksums; RAR5 can use BLAKE2sp instead of CRC32.<br/> [J] ZIP has per-entry CRC32; an archive-wide checksum is not a baseline ZIP requirement (there are optional signing/encryption features).<br/> [K] RAR has header/file checksums.<br/> [L] ZIP's "end of central directory" structure is tail-located by design; the canonical index is at the end.<br/> [M] RAR5 may include an optional "quick open record" placed at the end to accelerate listing, but it is not index-first linearization.<br/> [N] PDF defines "Linearized PDF" (fast web view) as a standardized layout.<br/> [O] ZIP provides extra fields and stores application-defined manifest files.<br/> [P] RAR provides "extra area" records and archive comments; structured metadata is not standardized as key:value pairs.<br/> [Q] PDF includes a metadata model (e.g., XMP) and document structure/navigation mechanisms; it is rich, but correspondingly intricate.<br/> [R] EPUB's package document supports metadata and a defined reading order via the spine.<br/> [S] Folder metadata is may differ by operating system.<br/> [T] PDF may contain a document outline (bookmarks) as a navigational hierarchy.<br/> [U] EPUB defines a default reading order via the spine (and navigation documents for the table of contents).<br/> [V] A folder can simulate sectioning with subdirectories, but no interoperable "chapter table" exists.<br/> [W] PDF page order is defined by the page tree structure.<br/> [X] ZIP extra fields are a per-record entry and not table-based.<br/> [Y] RAR extra area records are entries with forward-compatible skipping rules.<br/> [Z] PDF defines formal extension rules (name classes, extension mechanisms).<br/> [AA] EPUB permits additional resources and metadata. </sub>Graphical Comparison (BBF vs. CBZ)
A graphical representation of bbfbench is given below. To verify (or to see BBF's performance metrics on your particular system), use the bbfbench program provided in ./src/bench/.
Features
Memory Mapping
Libbbf uses memory mapping to load files into memory and read them quickly.
Adjustable Alignment (Ream Size)
You can adjust the alignment between assets. The default alignment is $2^{12}$ (4096 bytes).
Deduplication
Libbbf deduplicates assets using XXH128, and stores the asset hash for future
verification purposes.
XXH3-64 Footer Checksum
Libbbf uses an XXH3-64 hash to checksum the asset, page, and string tables.
Linearization (Petrification)
For those self-hosting, BBF can relocate the index and footer to immediately follow the header. This allows readers to parse the header and footer with a single initial block read of 320 bytes.
Arbitrary Metadata
Libbbf supports arbitrary key:value pairs with optional parent labels (key:value[:parent]).
MKV-Like Sectioning
Support for nested chapters and groupings using parent-labeled sections, allowing for table-of-contents structures.
Customizable Page-Ordering
Logical reading order is decoupled from physical asset storage by using a dedicated Page Table, so custom demuxing/page orders can be specified.
Customizable Expansion Entries
Libbbf's spec includes expansion entries, which can be used to expand the functionality of bbf files.
Variable Alignment (Variable Ream Size)
BBF files support power-of-two alignment (up to 2^16). To minimize internal fragmentation, a "Variable Ream" flag allows smaller assets to be packed to 8-byte boundaries while larger assets remain aligned to the primary ream size.
CLI Usage: bbfmux
The included bbfmux tool is a reference implementation for muxing/demuxing BBF files.
========[ BBFMUX v3.0 ]====================================================
| Bound Book Format Muxer Developed by EF1500 |
===========================================================================
USAGE: bbfmux <INPUT_DIR|BBF_FILE> [MODE] [OPTIONS]...
MODES (Mutually Exclusive):
(Default) Mux folder contents into a BBF container
--info Display headers, metadata, and statistics
--verify Validate XXH3-128/64 hashes
--extract Unpack contents to disk
--petrify Linearize BBF file for faster reading
MUXER OPTIONS:
--meta=K:V[:P] Add metadata (Key:Value[:Parent])
--metafile=<FILE> Read K:V:P entries from file
--section=N:T[:P] Add section (Name:Target[:Parent])
--sections=<FILE> Read section entries from file
--ream-size=<N> Ream size exponent override (2^N)
--alignment=<N> Byte alignment exponent override (2^N)
--variable-ream-size Enable variable ream sizing (reccomended)
VERIFY / EXTRACT OPTIONS:
--section="NAME" Target specific section
--rangekey="KEY" Stop extraction on key substring match
--asset=<ID> Target specific asset ID
--outdir=[PATH] Extract asset(s) to directory
--write-meta[=F] Dump metadata to file [default: path.txt]
--write-hashes[=F] Dump hashes to file [default: hashes.txt]
INFO FLAGS:
--hashes, --footer, --sections, --counts, --header, --metadata, --offsets
CLI Features
The bbfmux utility provides an example interface for managing Bound Book
