6cy
High-performance, streaming-first container format with per-block codec polymorphism and robust data recoverability. Reference implementation in Rust.
Install / Use
/learn @byte271/6cyREADME
.6cy Container Format
v1.0.0 · Reference implementation in Rust
</div>.6cy is a binary archive format built around five hard guarantees:
- Every block is self-describing. Magic, version, codec UUID, sizes, and two independent checksums live in each 84-byte block header. A reader can parse any single block in isolation.
- Checksums are mandatory. A CRC32 covers the header; a BLAKE3 covers the content. Neither can be disabled. Corruption is caught before any allocation.
- Codec identity is frozen. Each codec is identified by a permanent 128-bit UUID stored verbatim on disk. Short numeric IDs are an in-process optimization only and are never written to files.
- No runtime negotiation. The superblock declares all required codec UUIDs upfront. A decoder either has every one or fails immediately — no fallback, no partial decode.
- Reconstructible index. The FILE INDEX is written last. Because every DATA block header embeds the file ID, file offset, and original size, the complete block list can be rebuilt by reading headers forward from byte 256 — without the index and without decompressing any payload. Index corruption is not archive corruption.
Benchmark — 6cy (LZMA) vs 7-Zip (LZMA2 level 1)
These numbers are from v0.3.0. v1.0.0 benchmarks have not yet been run. v1.0.0 contains no changes to the core codec, I/O, or compression pipeline, so these figures remain representative. Updated numbers will be published once v1.0.0 benchmarks are completed.
Tested on AMD Ryzen 9 6900HX (8C/16T, 3301 MHz), 16 GB RAM,
Windows 11 Home, 10 GiB synthetic binary file, 3 runs each.
Full methodology in BENCHMARK.md.
| Metric | 6cy LZMA | 7z LZMA2 L1 | |--------|-------------|-------------| | Pack time (avg) | 13.0 s | 34.6 s | | Unpack time (avg) | 47.0 s | 8.6 s | | Archive size | 960 KiB | 1 527 KiB | | Pack throughput | 0.767 GiB/s | 0.289 GiB/s | | Unpack throughput | 0.213 GiB/s | 1.162 GiB/s | | Pack CPU (avg) | 76.7 % | 97.4 % | | Compression ratio | 10 919 : 1 | 6 868 : 1 |
6cy packs 2.66× faster and produces a 37% smaller archive.
7z decompresses 5.46× faster — 7z's LZMA2 decompressor is a mature
hand-optimized C++ implementation; lzma-rs is pure Rust (correctness-first).
A future release will evaluate an optional liblzma FFI backend for the
decompression path.
Features
Core format
- Content-addressable deduplication — identical 4 MiB chunks are written
once; subsequent references cost only an 84-byte
BlockRef. No codec pass for duplicate chunks. - Four codecs — Zstd (default), LZ4, Brotli, LZMA. Each identified by a frozen UUID; short IDs never leave the process.
- Solid mode — multiple files compressed together as one block for maximum ratio on small/similar files.
- AES-256-GCM block encryption — Argon2id key derivation (64 MiB, 3 passes). The archive UUID serves as the KDF salt so the same password yields a different key for every archive.
- Chunked streaming — files of any size are split into configurable chunks (default 4 MiB). Random access spans chunk boundaries correctly.
- Reconstructible index — the FILE INDEX is written last. If it is missing
or corrupt,
6cy scanrebuilds the file list by reading only block headers forward from byte 256, without decompressing any payload. - RecoveryMap checkpoints — a RecoveryMap is appended after the INDEX block.
It records a checkpoint (archive offset, last file ID, timestamp) after each
complete file is written. If a write is interrupted before
finalize(), the RecoveryMap narrows the scan range during recovery, avoiding a full forward scan from byte 256 in large archives. - Plugin C ABI — third-party codecs load via a frozen C ABI
(
plugin_abi/sixcy_plugin.h, ABI version 1). Explicit buffer contracts, declared thread safety, no shared allocator.
GUI (6cy Archive Suite desktop app — v1.0.0)
The desktop application (sixcy-app, built with Tauri + React) ships nine
panels. All panels added or promoted to full feature status in v1.0.0 are
marked NEW.
| Panel | Description |
|-------|-------------|
| Pack | Drag-and-drop or browse files → compress into .6cy. Codec selector (Zstd/LZ4/Brotli/LZMA/None), per-codec level bounds enforced, chunk size, solid mode, encryption password. Keyboard shortcut ⌘/Ctrl+Enter. |
| Unpack | Extract archive with BLAKE3 integrity verification. Shows file count, total size, and per-file extraction log. |
| List | Tabular archive contents with filename filter, compression ratio per file, total original/compressed size summary. |
| Info | Superblock fields, required codec UUIDs, root BLAKE3 hash — all copyable to clipboard. |
| Scan | Index-bypass block header scan. Reconstructs file list without INDEX block. Filename filter on results. |
| Recover (NEW) | Index-bypass full recovery. Animated block-health grid (healthy / corrupt / truncated / unknown codec). Summary card showing quality rating (Full / Partial / HeaderOnly / Catastrophic), per-category block counts, health-score progress bar, and recoverable MiB salvaged. |
| Optimize (NEW) | Re-compress all blocks at a chosen Zstd level (1–19). Before/after size and savings percentage displayed as stat cards. |
| Merge (NEW) | Combine multiple .6cy archives into one with cross-archive CAS deduplication. Drag-to-reorder source list. Output codec selector. |
| Bench (NEW) | RLE pre-filter benchmark. Drag-drop or browse any file; measures encode time, decode time, savings percentage, and round-trip correctness. Persists run history table across benchmarks within the session. |
Additional GUI improvements in v1.0.0:
- Dark mode — full dark theme, persists across sessions.
- Recent archives — last 10 used paths surfaced as a collapsible quick-picker in every panel.
- Keyboard navigation — Alt+1–9 switches panels; ⌘/Ctrl+Enter triggers Pack.
- Copy-to-clipboard — one-click copy on UUIDs, archive paths, and BLAKE3 root hashes in the Info panel.
- Log export — terminal output can be saved to a
.txtfile. - Codec tooltips — hovering a codec badge shows a description of its strengths.
- Input validation — level and chunk-size fields are clamped to valid ranges per codec; invalid values are rejected before any backend call.
- Binary prefix fix —
fmtBytesnow uses correct binary thresholds (2³⁰ for GiB, 2²⁰ for MiB) instead of the SI values (10⁹, 10⁶) that were previously mislabelled as binary units.
Project Layout
Sixcy_CAS/
├── Cargo.toml # version 1.0.0, Apache-2.0
├── LICENSE # Apache-2.0 (code)
├── LICENSE-SPEC # CC BY 4.0 (spec.md)
├── README.md # this file
├── BENCHMARK.md # detailed benchmark report (v0.3.0 data)
├── CHANGELOG.md # version history
├── CONTRIBUTING.md # how to contribute
├── SECURITY.md # threat model and disclosure policy
├── spec.md # binary format specification (CC BY 4.0)
├── plugin_abi/
│ └── sixcy_plugin.h # frozen C ABI for codec plugins
└── src/
├── main.rs # CLI (6cy binary)
├── lib.rs # crate root + re-exports
├── archive.rs # high-level Archive API
├── block.rs # block header encode/decode
├── superblock.rs # superblock (offset 0, 256 bytes)
├── plugin.rs # Rust wrapper for C plugin ABI
├── perf.rs # parallel chunk compression, write buffer, RLE pre-filter
├── codec/mod.rs # frozen UUID registry + built-in codecs
├── crypto/mod.rs # AES-256-GCM + Argon2id
├── index/mod.rs # FileIndex, BlockRef
├── io_stream/mod.rs # SixCyWriter, SixCyReader, scan_blocks
└── recovery/
├── mod.rs # RecoveryMap + re-exports
└── scanner.rs # extract_recoverable, BlockHealth, RecoveryReport
Getting Started
Prerequisites
- Rust stable (1.70+)
- No C toolchain required — all dependencies are pure Rust (except
zstd, which wraps the zstd C library)
Build
git clone https://github.com/cyh/sixcy.git
cd sixcy
cargo build --release
# binary: target/release/6cy (Linux/macOS)
# binary: target\release\6cy.exe (Windows)
CLI Reference
pack — create an archive
# Single file, Zstd (default)
6cy pack -o archive.6cy -i file.bin
# Multiple files, LZMA codec
6cy pack -o archive.6cy -i a.bin -i b.bin -i c.bin --codec lzma
# Solid block (all inputs compressed together)
6cy pack -o archive.6cy -i *.txt --codec zstd --solid
# Encrypted (AES-256-GCM, Argon2id key derivation)
6cy pack -o archive.6cy -i secret.bin --password "my passphrase"
# Custom chunk size (default 4096 KiB = 4 MiB)
6cy pack -o archive.6cy -i huge.bin --chunk-size 8192
# Full options
6cy pack --output archive.6cy \
--input file1.bin --input file2.bin \
--codec lzma \
--level 3 \
--chunk-size 4096 \
--solid \
--password "secret"
Available codecs: zstd (default) · lz4 · brotli · lzma · none
unpack — extract an archive
# Extract to current directory
6cy unpack archive.6cy
# Extract to specific directory
6cy unpack archive.6cy -C output/
# Extract encrypted archive
6cy unpack archive.6cy -C output/ --password "my passphrase"
list — list contents
6cy list archive
