GPc

Gaussian process code in C++ including some implementations of GP-LVM and IVM.

Gaussian Process Software

This page describes how to compile and gives some examples of use of the C++ Gaussian Process code.

Release Information

Current release is 0.001.

Design Philosophy

The software is written in C++ to try and get a degree of flexibility in the models that can be used without a serious performance hit. This was difficult to do in MATLAB as users who have tried version 1 (which was fast but inflexible) and version 2 (which was flexible but slow) of the MATLAB software will appreciate.

The software is mainly written in C++ but relies for some functions on FORTRAN code by other authors and the LAPACK and BLAS libraries.

As well as the C++ code some utilities are supplied in the corresponding MATLAB code for visualising the results.

Compiling the Software

The software was written with gcc on ubuntu.

Part of the reason for using gcc is the ease of interoperability with FORTRAN. The code base makes fairly extensive use of FORTRAN so you need to have g77 installed. The software is compiled by writing

$ make gp

at the command line. Architecture specific options are included in the make.ARCHITECTURE files. Rename the file with the relevant architecture to make.inc for it to be included.

Optimisation

One of the advantages of interfacing to the LAPACK and BLAS libraries is that they are often optimised for particular architectures.

General Information

The way the software operates is through the command line. There is one executable, gp. Help can be obtained by writing

$ ./gp -h

which lists the commands available under the software. Help for each command can then be obtained by writing, for example,

$ ./gp learn -h

All the tutorial optimisations suggested take less than 1/2 hour to run on my less than 2GHz Pentium IV machine. The first oil example runs in a couple of minutes. Below I suggest using the highest verbosity options -v 3 in each of the examples so that you can track the iterations.

Examples

The software loads in data in the <a href="http://svmlight.joachims.org/">SVM light</a> format. This is to provide compatibility with other <a href="https://github.com/SheffieldML/GPmat/">Gaussian Process software</a>. Anton Schwaighofer has written <a href="http://staffwww.dcs.shef.ac.uk/people/N.Lawrence/software/svml_toolbox.html"> a package</a> which can write from MATLAB to the SVM light format.

One Dimensional Data Data

Provided with the software, in the examples directory, is a one dimensional regression problem. The file is called spgp1d.svml.

First we will learn the data using the following command,

$ ./gp -v 3 learn -# 100 examples/sinc.svml sinc.model

The flag -v 3 sets the verbosity level to 3 (the highest level) which causes the iterations of the scaled conjugate gradient algorithm to be shown. The flag -# 100 terminates the optimisation after 100 iterations so that you can quickly move on with the rest of the tutorial.

The software will load the data in sinc.svml. The labels are included in this file but they are <i>not</i> used in the optimisation of the model. They are for visualisation purposes only.

Gnuplot

The learned model is saved in a file called sinc.model. This file has a plain text format to make it human readable. Once training is complete, the learned covariance function parameters of the model can be displayed using

$ ./gp display sinc.model

Loading model file.
... done.
Standard GP Model: 
Optimiser: scg
Data Set Size: 40
Kernel Type: 
Scales learnt: 0
X learnt: 0
Bias: 0.106658 

Scale: 1 

Gaussian Noise: 
Bias on process 0: 0
Variance: 1e-06
compound kernel:
rbfinverseWidth: 0.198511
rbfvariance: 0.0751124
biasvariance: 1.6755e-05
whitevariance: 0.00204124

Notice the fact that the covariance function is composed of an RBF kernel, also known as squared exponential kernel or Gaussian kernel; a bias kernel, which is just a constant, and a white noise kernel, which is a diagonal term. This is the default setting, it can be changed with flags to other covariance function types, see ./gp learn -h for details.

For your convenience a gnuplot file may generated to visualise the data. First run

$ ./gp gnuplot -r 400 examples/sinc.svml sinc.model sinc

The sinc supplied as the last argument acts as a stub for gnuplot to create names from, so for example (using gnuplot vs 4.0 or above), you can write

$ gnuplot sinc_plot.gp

Note: for this to work on OSX you may have to

$ brew install gnuplot --with-x

Then you should obtain the plot shown below

<center><img src="sinc.png"><br> Gaussian process applied to sinc data.</center><br>

The other files created are sinc_error_bar_data.dat, which produces the error bars and sinc_line_data.dat which produces the mean as well as sinc_scatter_data.dat which shows the training data.

Other Data

You might also want to try a larger data set.

$ ./gp -v 3 learn -# 100 examples/spgp1d.svml spgp1d.model

MATLAB and OCTAVE

While MATLAB can be slow (and very expensive for non-academic users) it can still be a lot easier to code the visualisation routines by building on MATLAB's graphics facilities. To this end you can load in the results from the MATLAB/OCTAVE GPmat toolbox for further manipulation. You can download the toolbox from <a href="https://github.com/SheffieldML/GPmat/">here</a>. Once the relevant toolboxes (you need all the dependent toolboxes) are downloaded you can visualise the results in MATLAB using

>> [y, X] = svmlread('sinc.svml')
>> gpReadFromFile('sinc.model', X, y)
>>

where we have used the <a href="http://staffwww.dcs.shef.ac.uk/people/N.Lawrence/software/svml_toolbox.html">SVML toolbox</a> of Anton Schwaighofer to load in the data.

IVM Software

This page describes how to compile and gives some examples of use of the C++ Informative Vector Machine Software (IVM).

Design Philosophy

The software is written in C++ to try and get a degree of flexibility in the models that can be used without a serious performance hit.

The software is mainly written in C++ but relies for some functions on FORTRAN code by other authors and the LAPACK and BLAS libraries.

Compiling the Software

The software was written with gcc vs 3.2.2. There are definitely Standard Template Library issues on Solaris with gcc 2.95, so I suggest that at least version 3.2 or above is used.

Part of the reason for using gcc is the ease of interoperability with FORTRAN. The code base makes fairly extensive use of FORTRAN so you need to have g77 installed. The software is compiled by writing

$ make ivm

at the command line. Architecture specific options are included in the make.ARCHITECTURE files. Rename the file with the relevant architecture to make.inc for it to be included.

Optimisation

One of the advantages of interfacing to the LAPACK and BLAS libraries is that they are often optimised for particular architectures. The file make.atlas includes options for compiling the ATLAS optimised versions of lapack and blas that are available on a server I have access to. These options may vary for particular machines.

Cygwin

For Windows users the code compiles under cygwin. However you will need version s of the lapack and blas libraries available (see <a href="http://www.netlib.org">www.netlib.org</a>). This can take some time to compile, and in the absence of any pre-compiled versions on the web I've provided some pre-compiled versions you may want to make use of (see the cygwin directory). Note that these pre-compiled versions are <i>not</i> optimised for the specific architecture and therefore do not give the speed up you would hope for from using lapack and blas.

Microsoft Visual C++

As of Release 0.101 the code compiles under Microsoft Visual Studio 7.1. A project file is provided in the current release in the directory MSVC/ivm. The compilation makes use of f2c versions of the FORTRAN code and the C version of LAPACK/BLAS, CLAPACK. Detailed instructions on how to compile are in the readme.msvc file. Much of the work to convert the code (which included ironing out several bugs) was done by William V. Baxter for the GPLVM code.

General Information

The way the software operates is through the command line. There is one executable, ivm. Help can be obtained by writing

$ ./ivm -h

which lists the commands available under the software. Help for each command can then be obtained by writing, for example,

$ ./ivm learn -h

All the tutorial optimisations are suggested take less than 1/2 hour to run on my less than 2GHz Pentium IV machine. The first oil example runs in a couple of minutes. Below I suggest using the highest verbosity options -v 3 in each of the examples so that you can track the iterations.

Bugs

Victor Cheng writes:

<i>" ... I've tested your IVM C++ Gaussian Process tool (IVMCPP0p12 version). It is quite useful. However, the gnuplot function seems has a problem. Every time I type the command: "Ivm gnuplot traindata name.model", an error comes out as: "Unknown noise model!". When I test this function with IVMCPP0p11 IVM, its fine, but IVMCPP0p11 has another problem that it gives "out of memory" error in test mode! So I use two vesions simultaneously. "</i>

I'm working (as of 31/12/2007) on a major rewrite, so it's unlikely that these bugs will be fixed in the near future, however if anyone makes a fix I'll be happy to incorporate it! Please let me know.

Examples

The software loads in data in the <a href="http://svmlight.joachims.org/">SVM light</a> format. Anton Schwaighofer has written <a href="http://staffwww.dcs.shef.ac.uk/people/N.Lawrence/software/svml_toolbox.html"> a package</a> which can write from MATLA