Logidze

Logidze provides tools for logging DB records changes when using PostgreSQL. Just like audited and paper_trail do (but faster).

Logidze allows you to create a DB-level log (using triggers) and gives you an API to browse this log. The log is stored with the record itself in JSONB column. No additional tables required.

🤔 How is Logidze pronounced?

Other requirements:

Ruby ~> 2.7
Rails >= 6.0 (for Rails 4.2 use version <=0.12.0, for Rails 5.x use version <= 1.2.3)
PostgreSQL >= 10.0

Installation

Add Logidze to your application's Gemfile:

gem "logidze", "~> 1.1"

Install required DB extensions and create trigger function:

bundle exec rails generate logidze:install

This creates a migration for adding trigger function and enabling the hstore extension.

Run migrations:

bundle exec rails db:migrate

NOTE: Logidze uses DB functions and triggers, hence you need to use SQL format for a schema dump:

# application.rb
config.active_record.schema_format = :sql

Using with schema.rb

Logidze seamlessly integrates with [fx][] gem to make it possible to continue using schema.rb for the database schema dump.

Add fx gem to your Gemfile and run the same Logidze generators: rails g logidze:install or rails g logidze:model.

If for some reason Logidze couldn't detect the presence of Fx in your bundle, you can enforce it by passing --fx option to generators.

On the other hand, if you have fx gem but don't want Logidze to use it—pass --no-fx option.

Configuring models

Run the following migration to enable changes tracking for an Active Record model and adding a log_data::jsonb column to the table:

bundle exec rails generate logidze:model Post
bundle exec rails db:migrate

This also adds has_logidze line to your model, which adds methods for working with logs.

By default, Logidze tries to infer the path to the model file from the model name and may fail, for example, if you have unconventional project structure. In that case, you should specify the path explicitly:

bundle exec rails generate logidze:model Post --path "app/models/custom/post.rb"

Backfill data

To backfill table data (i.e., create initial snapshots) add backfill option to the generator:

bundle exec rails generate logidze:model Post --backfill

Now your migration should contain and UPDATE ... statement to populate the log_data column with the current state.

Otherwise a full snapshot will be created the first time the record is updated.

You can create a snapshot manually by performing the following query:

UPDATE <my_table> as t
SET log_data = logidze_snapshot(to_jsonb(t))

Or by using the following methods:

Model.create_logidze_snapshot

# specify the timestamp column to use for the initial version (by default the current time is used)
Model.create_logidze_snapshot(timestamp: :created_at)

# filter columns
Model.create_logidze_snapshot(only: %w[name])
Model.create_logidze_snapshot(except: %w[password])

# or call a similar method (but with !) on a record

my_model = Model.find(params[:id])
my_model.create_logidze_snapshot!(timestamp: :created_at)

A snapshot is only created if log_data is null.

Log size limits

You can provide the limit option to generate to limit the size of the log (by default it's unlimited):

bundle exec rails generate logidze:model Post --limit=10

Tracking only selected columns

You can log only particular columns changes. There are mutually exclusive except and only options for this:

# track all columns, except `created_at` and `active`
bundle exec  rails generate logidze:model Post --except=created_at,active
# track only `title` and `body` columns
bundle exec rails generate logidze:model Post --only=title,body

Logs timestamps

By default, Logidze tries to get a timestamp for a version from record's updated_at field whenever appropriate. If your model does not have that column, Logidze will gracefully fall back to statement_timestamp().

To change the column name or disable this feature completely, you can use the timestamp_column option:

# will try to get the timestamp value from `time` column
bundle exec rails generate logidze:model Post --timestamp_column time
# will always set version timestamp to `statement_timestamp()`
bundle exec rails generate logidze:model Post --timestamp_column nil # "null" and "false" will also work

Undoing a Generated Invocation

If you would like to re-do your rails generate anew, as with other generators you can use rails destroy to revert it, which will delete the migration file and undo the injection of has_logidze into the model file:

bundle exec rails destroy logidze:model Post

IMPORTANT: If you use non-UTC time zone for Active Record (config.active_record.default_timezone), you MUST always infer log timestamps from a timestamp column (e.g., when back-filling data); otherwise, you may end up with inconsistent logs (#199). In general, we recommend using UTC as the database time unless there is a very strong reason not to.

Using with partitioned tables

Logidze supports partitioned tables for PostgreSQL 13+ without any additional configuration. For PostgreSQL 11/12, you should use after triggers. To do that, provide the --after-trigger option to the migration:

bundle exec rails generate logidze:model Post --after-trigger

NOTE: Record changes are written as a full snapshot if the partition has changed during the update.

IMPORTANT: Using Logidze for partitioned tables in PostgreSQL 10 is not supported.

Storing history data in a separate table

By default, Logidze stores history data in the log_data column in the origin record table, which might lead to table bloat. If it concerns you, you may configure Logidze to store history data in a separate table instead.

First, generate the shared logidze_data table (only once per project):

bundle exec rails generate logidze:migration:logs

Then, create your model migration with the --detached option:

bundle exec rails generate logidze:model Post --detached

You can also configure Logidze to always store history data in a separate table for all models:

# config/initializers/logidze.rb

Logidze.log_data_placement = :detached

NOTE: You may need to upgrade your logdize_logger function. Check upgrading for more details

IMPORTANT: Using --detached mode for storing historic data slightly decreases performance. Check [bench results] for the details.

Usage

Basic API

Your model now has log_data column, which stores changes log.

To retrieve record version at a given time use #at or #at! methods:

post = Post.find(27)

# Show current version
post.log_version #=> 3

# Show log size (number of versions)
post.log_size #=> 3

# Get copy of a record at a given time
post.at(time: 2.days.ago)

# or revert the record itself to the previous state (without committing to DB)
post.at!(time: "2018-04-15 12:00:00")

# If no version found
post.at(time: "1945-05-09 09:00:00") #=> nil

You can also get revision by version number:

post.at(version: 2)

NOTE: If log_data is nil, #at(time:) returns self and #at(version:) returns nil. You can opt-in to return nil for time-based #at as well by setting Logidze.return_self_if_log_data_is_empty = false.

It is

Logidze

Install / Use

README

Logidze

Links

Table of contents

Installation

Using with schema.rb

Configuring models

Backfill data

Log size limits

Tracking only selected columns

Logs timestamps

Undoing a Generated Invocation

Using with partitioned tables

Storing history data in a separate table

Usage

Basic API