FileTrove

<p align="center"> <img src="https://github.com/steffenfritz/FileTrove/assets/16431534/b8c1456d-08bb-48bb-afcf-5e99db8466b9" width="300"> </p> <p align="center"> <img alt="Build Status" src="https://github.com/steffenfritz/FileTrove/actions/workflows/buildstatus.yml/badge.svg"> <a href="https://www.gnu.org/licenses/agpl-3.0"><img alt="License: AGPL v3" src="https://img.shields.io/badge/License-AGPL_v3-blue.svg"></a> <a href="https://pkg.go.dev/github.com/steffenfritz/FileTrove"><img alt="Go Reference" src="https://pkg.go.dev/badge/github.com/steffenfritz/FileTrove.svg"></a> <a href="https://scorecard.dev/viewer/?uri=github.com/steffenfritz/FileTrove"><img alt="OpenSSF Scorecard" src="https://api.scorecard.dev/projects/github.com/steffenfritz/FileTrove/badge"></a> <a href="https://www.bestpractices.dev/projects/8952"><img alt="OpenSSF Best Practices" src="https://www.bestpractices.dev/projects/8952/badge"></a> </p>

VERSION: v1.0.0-BETA-4

---

FileTrove walks a directory tree, identifies every file, computes metadata, and writes all results into a SQLite database with TSV export support.

What it collects

| Category | Details | |----------|---------| | File type | MIME type, PRONOM identifier, format version, identification proof/note, extension — via siegfried | | File & directory timestamps | Creation, modification, and access times | | Hashes | MD5, SHA1, SHA256, SHA512, BLAKE2B-512 | | Entropy | Shannon entropy (files up to 1 GB) | | Extended attributes | xattr from ext3/ext4, btrfs, APFS, and others | | EXIF metadata | Extracted from image files | | YARA-X | Match results from your own rule files | | NSRL | Flags known software files via the National Software Reference Library | | Dublin Core | Optional session-level descriptive metadata |

Each file and directory gets a UUIDv4 as a unique identifier. All results land in a SQLite database and can be exported to TSV.

Installation

1. Get the binary — download a release from the releases page, or compile from source (see BUILDING.md). Both a standard dynamic binary (ftrove) and a static binary (e.g. ftrove_amd64_linux_static) are provided.

2. Run the installer from the directory where you want FileTrove to live:

   ./ftrove --install .
   

   This creates a db/ directory, downloads the siegfried signature database, and optionally downloads the NSRL database (1.4 GB compressed). If you already have an NSRL database, copy it into db/ afterwards.

3. You're ready.

YARA-X

YARA-X scanning requires a C library that is not bundled with FileTrove. It is built automatically during task build if not already present. See BUILDING.md for setup instructions.

• Example rule files: testdata/yara/
• When a rule matches, the rule name, session UUID, and file UUID are recorded in the yara table. The rule file itself is not stored.

NSRL custom databases

You can build your own NSRL-style database from any newline-delimited list of SHA1 hashes using admftrove, which is built alongside ftrove.

Running a scan

./ftrove -i $DIRECTORY

FileTrove walks $DIRECTORY recursively. Run ./ftrove -h for all available flags.

Viewing results

List all sessions and export one to TSV:

./ftrove -l
./ftrove -t 926be141-ab75-4106-8236-34edfcf102f2

You can also query the SQLite database directly:

• CLI: sqlite3 db/filetrove.db
• GUI: sqlitebrowser
• Visualisation: Sqliteviz

Background

FileTrove is the successor of filedriller, based on the iPres 2021 paper Marrying siegfried and the National Software Reference Library.