Tensorflex

The paper detailing Tensorflex was presented at NeurIPS/NIPS 2018 as part of the MLOSS workshop. The paper can be found here.

Contents

• How to run
• Documentation
• Examples
• Pull Requests Made

How to run

• You need to have the Tensorflow C API installed. Look here for details.
• You also need the C library libjpeg. If you are using Linux or OSX, it should already be present on your machine, otherwise be sure to install (brew install libjpeg for OSX, and sudo apt-get install libjpeg-dev for Ubuntu).
• Simply add Tensorflex to your list of dependencies in mix.exs and you are good to go!:

{:tensorflex, "~> 0.1.2"}

In case you want the latest development version use this:

{:tensorflex, github: "anshuman23/tensorflex"}

Documentation

Tensorflex contains three main structs which handle different datatypes. These are %Graph, %Matrix and %Tensor. %Graph type structs handle pre-trained graph models, %Matrix handles Tensorflex 2-D matrices, and %Tensor handles Tensorflow Tensor types. The official Tensorflow documentation is present here and do note that this README only briefly discusses Tensorflex functionalities.

• read_graph/1:

  • Used for loading a Tensorflow .pb graph model in Tensorflex.

  • Reads in a pre-trained Tensorflow protobuf (.pb) Graph model binary file.

  • Returns a tuple {:ok, %Graph}.

  • %Graph is an internal Tensorflex struct which holds the name of the graph file and the binary definition data that is read in via the .pb file.

• get_graph_ops/1:

  • Used for listing all the operations in a Tensorflow .pb graph.

  • Reads in a Tensorflex %Graph struct obtained from read_graph/1.

  • Returns a list of all the operation names (as strings) that populate the graph model.

• create_matrix/3:

  • Creates a 2-D Tensorflex matrix from custom input specifications.

  • Takes three input arguments: number of rows in matrix (nrows), number of columns in matrix (ncols), and a list of lists of the data that will form the matrix (datalist).

  • Returns a %Matrix Tensorflex struct type.

• matrix_pos/3:

  • Used for accessing an element of a Tensorflex matrix.

  • Takes in three input arguments: a Tensorflex %Matrix struct matrix, and the row (row) and column (col) values of the required element in the matrix. Both row and col here are NOT zero indexed.

  • Returns the value as float.

• size_of_matrix/1:

  • Used for obtaining the size of a Tensorflex matrix.

  • Takes a Tensorflex %Matrix struct matrix as input.

  • Returns a tuple {nrows, ncols} where nrows represents the number of rows of the matrix and ncols represents the number of columns of the matrix.

• append_to_matrix/2:

  • Appends a single row to the back of a Tensorflex matrix.

  • Takes a Tensorflex %Matrix matrix as input and a single row of data (with the same number of columns as the original matrix) as a list of lists (datalist) to append to the original matrix.

  • Returns the extended and modified %Matrix struct matrix.

• matrix_to_lists/1:

  • Converts a Tensorflex matrix (back) to a list of lists format.

  • Takes a Tensorflex %Matrix struct matrix as input.

  • Returns a list of lists representing the data stored in the matrix.

  • NOTE: If the matrix contains very high dimensional data, typically obtained from a function like load_csv_as_matrix/2, then it is not recommended to convert the matrix back to a list of lists format due to a possibility of memory errors.

• float64_tensor/2, float32_tensor/2, int32_tensor/2:

  • Creates a TF_DOUBLE, TF_FLOAT, or TF_INT32 tensor from Tensorflex matrices containing the values and dimensions specified.

  • Takes two arguments: a %Matrix matrix (matrix1) containing the values the tensor should have and another %Matrix matrix (matrix2) containing the dimensions of the required tensor.

  • Returns a tuple {:ok, %Tensor} where %Tensor represents an internal Tensorflex struct type that is used for holding tensor data and type.

• float64_tensor/1, float32_tensor/1, int32_tensor/1, string_tensor/1:

  • Creates a TF_DOUBLE, TF_FLOAT, TF_INT32, or TF_STRING constant value one-dimensional tensor from the input value specified.

  • Takes in a float, int or string value (depending on function) as input.

  • Returns a tuple {:ok, %Tensor} where %Tensor represents an internal Tensorflex struct type that is used for holding tensor data and type.

• float64_tensor_alloc/1, float32_tensor_alloc/1, int32_tensor_alloc/1:

  • Allocates a TF_DOUBLE, TF_FLOAT, or TF_INT32 tensor of specified dimensions.

  • This function is generally used to allocate output tensors that do not hold any value data yet, but will after the session is run for Inference. Output tensors of the required dimensions are allocated and then passed to the run_session/5 function to hold the output values generated as predictions.

  • Takes a Tensorflex %Matrix struct matrix as input.

  • Returns a tuple {:ok, %Tensor} where %Tensor represents an internal Tensorflex struct type that is used for holding the potential tensor data and type.

• tensor_datatype/1:

  • Used to get the datatype of a created tensor.

  • Takes in a %Tensor struct tensor as input.

  • Returns a tuple {:ok, datatype} where datatype is an atom representing the list of Tensorflow TF_DataType tensor datatypes. Click here to view a list of all possible datatypes.

• load_image_as_tensor/1:

  • Loads JPEG images into Tensorflex directly as a TF_UINT8 tensor of dimensions image height x image width x number of color channels.

  • This function is very useful if you wish to do image classification using Convolutional Neural Networks, or other Deep Learning Models. One of the most widely adopted and robust image classification models is the Inception model by Google. It makes classifications on images from over a 1000 classes with highly accurate results. The load_image_as_tensor/1 function is an essential component for the prediction pipeline of the Inception model (and for other similar image classification models) to work in Tensorflex.

  • Reads in the path to a JPEG image file (.jpg or .jpeg).

  • Returns a tuple {:ok, %Tensor} where %Tensor represents an internal Tensorflex struct type that is used for holding the tensor data and type. Here the created Tensor is a uint8 tensor (TF_UINT8).

  • NOTE: For now, only 3 channel RGB JPEG color images can be passed as arguments. Support for grayscale images and other image formats such as PNG will be added in the future.

• loads_csv_as_matrix/2:

  • Loads high-dimensional data from a CSV file as a Tensorflex 2-D matrix in a super-fast manner.

  • The load_csv_as_matrix/2 function is very fast-- when compared with the Python based pandas library for data science and analysis' function read_csv on the test.csv file from MNIST Kaggle data (source), the following execution times were obtained:

    • read_csv: 2.549233 seconds
    • load_csv_as_matrix/2: 1.711494 seconds

  • This function takes in 2 arguments: a path to a valid CSV file (filepath) and other optional arguments opts. These include whether or not a header needs to be discarded in the CSV, and what the delimiter type is. These are specified by passing in an atom :true or :false to the header: key, and setting a string value for the delimiter: key. By default, the header is considered to be present (:true) and the delimiter is set to ,.

  • Returns a %Matrix Tensorflex struct type.

• run_session/5:

  • Runs a Tensorflow session to generate predictions for a given graph, input data, and required input/output operations.

  • This function is the final step of the Inference (prediction) pipeline and generates output for a given set of input data, a pre-trained graph model, and the specified input and output operations of the graph.

  • Takes in five arguments: a pre-trained Tensorflow graph .pb model read in from the read_graph/1 function (graph), an input tensor with the dimensions and data required for the input operation of the graph to run (tensor1), an output tensor allocated with the right dimensions (tensor2), the name of the input operation of the graph that needs where the input data is fed (input_opname), and the output operation name in the graph where the outputs are obtained (output_opname). The input tensor is generally created from the matrices manually or using the load_csv_as_matrix/2 function, and then passed through to one of the tensor creation functions. For image classification the load_image_as_tensor/1 can also be used to create the input tensor from an image. The output tensor is created using the tensor allocation functions (generally containing alloc at the end of the function name).

  • Returns a List of Lists (similar to the matrix_to_lists/1 function) containing the generated predictions as per the output tensor dimensions.

• `add_scalar_to_matrix