Coffer

Simply ACID* key-value database. At the medium or even low latency it tries to provide greater throughput without losing the ACID properties of the database. The database provides the ability to create record headers at own discretion and use them as transactions. The maximum size of stored data is limited by the size of the computer's RAM.

*is a set of properties of database transactions intended to guarantee validity even in the event of errors, power failures, etc.

Properties:

• high throughput
• tolerated latency
• high reliability

ACID:

• good durabilty
• compulsory isolation
• atomic operations
• consistent transactions

Table of Contents

• Usage
• Examples
• API
  • Methods
• Config
  • Handler
    • Example of a handler without using an argument
    • Example of a handler using an argument
• Launch
  • Start
  • Follow
• Data storage
  • Data loading after an incorrect shutdown
• Error codes
  • Code List
  • Code checks through methods
• Benchmark
• Dependencies
• ToDo

Usage

package main

import (
	"fmt"

	"github.com/claygod/coffer"
)

const curDir = "./"

func main() {
	// STEP init
	db, err, wrn := coffer.Db(curDir).Create()
	switch {
	case err != nil:
		fmt.Println("Error:", err)
		return
	case wrn != nil:
		fmt.Println("Warning:", err)
		return
	}
	if !db.Start() {
		fmt.Println("Error: not start")
		return
	}
	defer db.Stop()

	// STEP write
	if rep := db.Write("foo", []byte("bar")); rep.IsCodeError() {
		fmt.Sprintf("Write error: code `%d` msg `%s`", rep.Code, rep.Error)
		return
	}

	// STEP read
	rep := db.Read("foo")
	if rep.IsCodeError() {
		fmt.Sprintf("Read error: code `%v` msg `%v`", rep.Code, rep.Error)
		return
	}
	fmt.Println(string(rep.Data))
}

Examples

Use the following links to find many examples how use the transactions:

• Quick start https://github.com/claygod/coffer/tree/master/examples/quick_start
• Finance https://github.com/claygod/coffer/tree/master/examples/finance

API

Started DB returns reports after has performed an operation. Reports containing:

• code (error codes here: github.com/claygod/coffer/reports/codes)
• error
• data
• other details Reporting structures read here:: github.com/claygod/coffer/reports

Methods

• Start
• Stop
• StopHard
• Save
• Write
• WriteList
• WriteListUnsafe
• Read
• ReadList
• ReadListUnsafe
• Delete
• DeleteListStrict
• DeleteListOptional
• Transaction
• Count
• CountUnsafe
• RecordsList
• RecordsListUnsafe
• RecordsListWithPrefix
• RecordsListWithSuffix

Pay attention!

All requests which names contain Unsafe can be usually executed in both cases: when the database is running or stopped (not running). In the second case (when DB is stopped), should not make requests in parallel, because in this case the consistency of DB can be compromised and data lost.

Other methods work only if the database is running.

Start

The Follow interactor turns on while running a database. It controls the relevance of the current checkpoint.

Stop

Stop DB. If you want to periodically stop and start the database in your application, probably, you may want to create a new client when the DB has been stopped.

Write

Write a new record in a database specifying the key and value. Their length must satisfy the requirements specified in configuration files.

WriteList

Write several records to the database specifying corresponding map in the arguments.

Strict mode (strictMode=true): The operation performs if there are no records with these keys. A list of existed records is returned.

Optional mode (strictMode=false): The operation performs regardless of whether there are records with these keys or not. A list of existed records is returned.

Important: this argument is a reference argument; it cannot be changed in the called code!

WriteListUnsafe

Write several records to the database specified the corresponding map in the arguments. This method exists in order to fill the database faster before it starts. The method is not for parallel use.

Read

Read one record from the database. In the received report there will be a result code. If it is positive, that means that the value in the right data field.

ReadList

Read several records. There is a limit on the maximum number of readable records in the configuration. Except found records the list of not found records is returned.

ReadListUnsafe

Read several records. The method can be called when the database is stopped (not running). The method is not for parallel use.

Delete

Remove a single record.

DeleteList

Strict mode (true): Delete several records. It is possible only if all records are in the database. If at least there is a lack of one record, none of records will be deleted.

Optional mode (false): Delete several records. All found records from the list in will be deleted in DB.

Transaction

Make a transaction. The transaction should be added in the database at the stage of creating and configuring. The user of the database is responsible for the consistency of the functionality of transaction handlers between different runs of the database. The transaction returns new values which are stored in the DB.

Count

Get the number of records in the database. A request can be made only when the database has started.

CountUnsafe

Get the number of records in the database. Requests to a stopped (or not running), database cannot be made in parallel!

RecordsList

Get a list of all database keys. With a large number of records in the database, the request will be slow. Use it only at great need to avoid problems. The method works only when the database is running.

RecordsListUnsafe

Get a list of all database keys. With a large number of records in the database, the request will be slow. Use it only at great need to avoid problems. The method is not for parallel use while using a request when database is stopped (or not running).

RecordsListWithPrefix

Get a list of all keys with prefix specified in the argument (prefix is the begging of record string).

Config

If you specify the path to the database directory all configuration parameters will be reset to the default:

cof, err, wrn := Db(dirPath) . Create()

Default values can be found in the /config.go file. But each of the parameters can be configured:

	Db(dirPath).
	BatchSize(batchSize).
	LimitRecordsPerLogfile(limitRecordsPerLogfile).
	FollowPause(100*time.Second).
	LogsByCheckpoint(1000).
	AllowStartupErrLoadLogs(true).
	MaxKeyLength(maxKeyLength).
	MaxValueLength(maxValueLength).
	MaxRecsPerOperation(1000000).
	RemoveUnlessLogs(true).
	LimitMemory(100 * 1000000).
	LimitDisk(1000 * 1000000).
	Handler("handler1", &handler1).
	Handler("handler2", &handler2).
	Handlers(map[string]*handler).
	Create()

Db

Specify the work directory where the database will store files. In case of a new database the directory should not contain files with the “log”, “check”, “checkpoint” extensions.

BatchSize

The maximum number of records which database can add at a time (this applies to setting up internal processes; this does not apply to the number of records added at a time). Decreasing of this parameter slightly improves the latency (but not too much). Increasing of this parameter slightly degrades the latency, but at the same time increases the throughput.

LimitRecordsPerLogfile

A number of operations which is going to be written to one log file. Small number forces the database creates new files very often, and it adversely affects the speed of the database. A big number reduces the number of pauses while creating files, but the size of files increases.

FollowPause

The size of the time interval for starting the Follow interactor, which analyzes old logs and periodically creates new checkpoints.

LogsByCheckpoint

The option specifies after how many full log files it is necessary to create a new checkpoint (the smaller number, the more often it should be created). For good productivity, it’s better not to do it too often.

AllowStartupErrLoadLogs

The option allows the database works at startup, even if the last log file was completed incorrectly, i.e. the last record is corrupted (a typical situation for an abnormal shutdown). By default, the option is enabled.

MaxKeyLength

This is the maximum allowable key length.

MaxValueLength

This is the maximum size of the value length.

MaxRecsPerOperation

This is the maximum number of records that is possible per operation.

RemoveUnlessLogs

The option is for deleting old files. After Follow has created a new checkpoint, with the permission of this option, it removes unnecessary operations logs. If for some reason it’s needed to store the whole log of operations, this option can be disabled. But be ready that this will increase the consumption of disk space.

LimitMemory

This is the minimum size of free RAM. When