Autodiff

Documentation

Autodiff is a numerical optimization and linear algebra library for the Go / Golang programming language. It implements basic automatic differentation for many mathematical routines. The documentation of this package can be found here.

Scalars

Autodiff defines three different scalar types. A Scalar contains a single mutable value that can be the result of a mathematical operation, whereas the value of a ConstScalar is constant and fixed when the scalar is created. Automatic differentiation is implemented by MagicScalar types that allow to compute first and second order derivatives. Autodiff supports the following scalars types:

| Scalar | Implemented interfaces |--------------|------------------------------------------------------ | | ConstInt8 | ConstScalar | | ConstInt16 | ConstScalar | | ConstInt32 | ConstScalar | | ConstInt64 | ConstScalar | | ConstInt | ConstScalar | | ConstFloat32 | ConstScalar | | ConstFloat64 | ConstScalar | | Int8 | ConstScalar, Scalar | | Int16 | ConstScalar, Scalar | | Int32 | ConstScalar, Scalar | | Int64 | ConstScalar, Scalar | | Int | ConstScalar, Scalar | | Float32 | ConstScalar, Scalar | | Float64 | ConstScalar, Scalar | | Real32 | ConstScalar, Scalar, MagicScalar | | Real64 | ConstScalar, Scalar, MagicScalar |

The ConstScalar, Scalar and MagicScalar interfaces define the following operations:

| Function | Description | |--------------|------------------------------------------------------ | | GetInt8 | Get value as int8 | | GetInt16 | Get value as int16 | | GetInt32 | Get value as int32 | | GetInt64 | Get value as int64 | | GetInt | Get value as int | | GetFloat32 | Get value as float32 | | GetFloat64 | Get value as float64 | | Equals | Check if two constants are equal | | Greater | True if first constant is greater | | Smaller | True if first constant is smaller | | Sign | Returns the sign of the scalar |

The Scalar and MagicScalar interfaces define the following operations:

| Function | Description | |--------------|------------------------------------------------------ | | SetInt8 | Set value by passing an int8 variable | | SetInt16 | Set value by passing an int16 variable | | SetInt32 | Set value by passing an int32 variable | | SetInt64 | Set value by passing an int64 variable | | SetInt | Set value by passing an int variable | | SetFloat32 | Set value by passing an float32 variable | | SetFloat64 | Set value by passing an float64 variable |

The Scalar and MagicScalar interfaces define the following mathematical operations:

| Function | Description | | ------------ | ----------------------------------------------------- | | Min | Minimum | | Max | Maximum | | Abs | Absolute value | | Sign | Sign | | Neg | Negation | | Add | Addition | | Sub | Substraction | | Mul | Multiplication | | Div | Division | | Pow | Power | | Sqrt | Square root | | Exp | Exponential function | | Log | Logarithm | | Log1p | Logarithm of 1+x | | Log1pExp | Logarithm of 1+Exp(x) | | Logistic | Standard logistic function | | Erf | Error function | | Erfc | Complementary error function | | LogErfc | Log complementary error function | | Sigmoid | Numerically stable sigmoid function | | Sin | Sine | | Sinh | Hyperbolic sine | | Cos | Cosine | | Cosh | Hyperbolic cosine | | Tan | Tangent | | Tanh | Hyperbolic tangent | | LogAdd | Addition on log scale | | LogSub | Substraction on log scale | | SmoothMax | Differentiable maximum | | LogSmoothMax | Differentiable maximum on log scale | | Gamma | Gamma function | | Lgamma | Log gamma function | | Mlgamma | Multivariate log gamma function | | GammaP | Lower incomplete gamma function | | BesselI | Modified Bessel function of the first kind | | LogBesselI | Log of the Modified Bessel function of the first kind |

Vectors and Matrices

Autodiff implements dense and sparse vectors and matrices that support basic linear algebra operations. The following vector and matrix types are provided by autodiff:

| Type | Scalar | Description | |--------------------------|--------------|----------------------------------------| | DenseInt8Vector | Int8 | Dense vector of Int8 scalars | | DenseInt16Vector | Int16 | Dense vector of Int16 scalars | | DenseInt32Vector | Int32 | Dense vector of Int32 scalars | | DenseInt64Vector | Int64 | Dense vector of Int64 scalars | | DenseIntVector | Int | Dense vector of Int scalars | | DenseFloat32Vector | Float32 | Dense vector of Float32 scalars | | DenseFloat64Vector | Float64 | Dense vector of Float64 scalars | | DenseReal32Vector | Real32 | Dense vector of Real32 scalars | | DenseReal64Vector | Real64 | Dense vector of Real64 scalars | | SparseInt8Vector | Int8 | Sparse vector of Int8 scalars | | SparseInt16Vector | Int16 | Sparse vector of Int16 scalars | | SparseInt32Vector | Int32 | Sparse vector of Int32 scalars | | SparseInt64Vector | Int64 | Sparse vector of Int64 scalars | | SparseIntVector | Int | Sparse vector of Int scalars | | SparseFloat32Vector | Float32 | Sparse vector of Float32 scalars | | SparseFloat64Vector | Float64 | Sparse vector of Float64 scalars | | SparseReal32Vector | Real32 | Sparse vector of Real32 scalars | | SparseReal64Vector | Real64 | Sparse vector of Real64 scalars | | SparseConstInt8Vector | ConstInt8 | Sparse vector of ConstInt8 scalars | | SparseConstInt16Vector | ConstInt16 | Sparse vector of ConstInt16 scalars | | SparseConstInt32Vector | ConstInt32 | Sparse vector of ConstInt32 scalars | | SparseConstInt64Vector | ConstInt64 | Sparse vector of ConstInt64 scalars | | SparseConstIntVector | ConstInt | Sparse vector of ConstInt scalars | | SparseConstFloat32Vector | ConstFloat32 | Sparse vector of ConstFloat32 scalars | | SparseConstFloat64Vector | ConstFloat64 | Sparse vector of ConstFloat64 scalars | | DenseInt8Matrix | Int8 | Dense matrix of Int8 scalars | | DenseInt16Matrix | Int16 | Dense matrix of Int16 scalars | | DenseInt32Matrix | Int32 | Dense matrix of Int32 scalars | | DenseInt64Matrix | Int64 | Dense matrix of Int64 scalars | | DenseIntMatrix | Int | Dense matrix of Int scalars | | DenseFloat32Matrix | Float32 | Dense matrix of Float32 scalars | | DenseFloat64Matrix | Float64 | Dense matrix of Float64 scalars | | DenseReal32Matrix | Real32 | Dense matrix of Real32 scalars | | DenseReal64Matrix | Real64 | Dense matrix of Real64 scalars | | SparseInt8Matrix | Int8 | Sparse matrix of Int8 scalars | | SparseInt16Matrix