.. -- mode: rst --

================================================================== Helsinki Finite-State Technology (library and application suite)

This package contains a bridging library for multiple FST libraries and toolkits and set of tools for processing of finite-state automata especially for linguistic systems. HFST library and tools are licensed under GNU GPL licence version 3, you may read the full licence in the file named COPYING. The authors specified in AUTHORS file may be contacted about licensing issues.

For additional information, see the project page <https://hfst.github.io>_.

.. image:: https://travis-ci.org/hfst/hfst.svg :target: https://travis-ci.org/hfst/hfst

Installation

Installation method depends on operating system and the version you want to install. For stable versions there exists packages for some of the better operating system and package manager combinations. For bleeding edge newest versions, development and non-supported operating systems and versions you will have to perform installation from the sources_.

Installation packages for Debian and Ubuntu

Debian packages for HFST are distributed via Apertium project <http://apertium.projectjj.com/apt/nightly/pool/main/h/hfst/>. This folder contains debian packages for HFST API library, command line tools and Python bindings. The debian packages are experimental; the requirements of debian or ubuntu installations are same as main packages. SFST is excluded from the packages as it has portability issues with hash_maps and hash_sets. For installation instructions, see downloads <https://hfst.github.io/downloads/>.

Binaries for Windows

Binaries for Windows are distributed via Apertium project <http://apertium.projectjj.com/win32/nightly/>. This folder contains ready-compiled HFST library and command line tools. For installation instructions, see downloads <https://hfst.github.io/downloads/>. Python bindings for Windows are currently available as wheels for 32-bit Python versions 2.7, 3.4, 3.5 and 3.6 on our PyPI page <https://pypi.python.org/pypi/hfst>. Performing installation from the sources is also possible on Windows with MinGW and Cygwin, if Python bindings are not needed.

Binaries for Mac OS X

Binaries for OS X are distributed via Apertium project <http://apertium.projectjj.com/osx/nightly/>. This folder contains ready-compiled HFST library and command line tools. For installation instructions, see downloads <https://hfst.github.io/downloads/>. Python bindings for OS X are currently available as wheels for Python versions 2.7, 3.4, 3.5 and 3.6 on our PyPI page <https://pypi.python.org/pypi/hfst>. Performing installation from the sources is also possible on Mac.

Installation for Gentoo Linux

HFST software is available in science overlay. For portage, use::

layman -a science

To add the repository. Then::

emerge -av hfst

to install. Parts of the spell-checking tools are in Finnish overlay. If you use Paludis, just try to cave resolve sci-misc/hfst to get the current instructions.

Installation for other systems

For rest of the systems, HFST needs to be installed from the source, the traditional GNU/linux way. To begin your installation, you need to start by gathering the dependencies, as adviced in the following chapter.

Dependencies

Please note that the dependencies for the library can be set during the compile time. Settings that determine dependencies relate to features the resulting library will have; e.g. if you disable openfst you cannot use weighted finite-state automata. If you enable foma, you will be able to read foma format automata as HFST files and use foma's algorithms to process automata. The command ./configure --help lists all features that can be controlled with configure switches and whether they are enabled or disabled by default.

Compilation requirements

OpenFST_ version 1.7.9 or older (newer does not work) (default)
- to disable OpenFST support, configure switch --without-openfst may be used (however, this seriously limits the use of HFST)
- if you have problems linking, make sure openfst is compiled with c++17 compilers
To use the SFST_ backend (default):
- the SFST library version 1.4.6g is bundled with HFST and included by default when building HFST
- SFST requires readline and ncurses
- The SFST frontend a.k.a. the SFST-PL parser a.k.a. hfst-sfstpl2fst does not require the SFST library to be installed; the library is only used for library-stuff like reading SFST format automata.
- to disable SFST backend, configure switch --without-sfst must be used
foma_ version 0.10.0 or newer (default)
- to disable foma backend, configure switch --without-foma may be used
- the hfst-xfst frontend does not require foma binary or foma library
ICU version 50 or newer
To compile corpus processing tool hfst-proc, you need to use the configure switch --enable-proc (or --enable-all-tools)
To use the Python interface:
- Compiling the bindings requires swig
- NOTE: the Python API is not under autotools, you must compile it yourself; for more information, see file python/README

Note that if you did install dependent libraries, such as libxml or ICU to your home directory instead of using your system's package manager (or supported default location):

If you only have a local version of a library, you can use it with appropriate LDFLAGS and CXXFLAGS, eg. ./configure LDFLAGS=-L/path/to/local/lib --prefix=/path/to/local/installation and make CXXFLAGS=-I/path/to/local/headers

If you are building a development version you loaded from the version control system, you must have new brand of GNU development tools installed:

autoconf >=2.62
automake >=1.11.1
libtool >=2.2
=gettext-0.17
GNU tool-chain is also needed with distributed packages if the user wishes to modify Makefile.am or configure.ac files.
Mac OS X users are advised to use MacPorts; Mac OS X 10.6 with XCode 2.3.2 at least is not sufficient
A package loaded from hfst web site does not have these requirements

The source codes loaded from the version controls system will also require parse generator system:

GNU flex 2.5.35 or compatible and
GNU bison 1.31 (2.4 suggested) or compatible
flex 2.5.4-2.5.33 will choke on perfectly valid rules used in hfst
bison older than 1.31 do not support name-prefix needed for having multiple parsers in one library
A package loaded from hfst web site does not have these requirements
source code loaded from version control system requires them only to bootstrap; if you use systems with archaic versions of flex or bison and cannot install updates, you might be able to get the needed files from somewhere

When running HFST software or using HFST libraries from HFST-enabled software:

If the executable is dynamically linked (almost always), the operating system must be able to find hfst libraries
If you install the libraries in non-standard paths, you need to ensure that operating system is aware of this; In linux this may happen by setting LD_LIBRARY_PATH, on Mac DYLD_LIBRARY_PATH
the hfst-xfst frontend needs GNU compatible getopt, or basic getopts without GNU-incompatible getopt installed
the hfst-sfstpl2fst frontend does not require SFST libraries or binaries, you do not need to enable SFST libraries (via the switch --with-sfst) in order to compile SFST-PL scripts to HFST automata.

Installation from the sources

INSTALL describes the GNU build system in detail, but for most users the usual::

./configure
make
(as root) make install

should result in a local installation and::

(as root) make uninstall

in its uninstallation.

If you aren't going to be linking to the library after building it and don't need to be able to send debugging information, you can save a considerable amount of space and memory by doing::

make install-strip

instead of make install. This strips all the symbols from the binaries, reducing sizes by a factor of 5-10.

If you would rather install in e.g. your home directory (or aren't the system administrator), you can tell ./configure::

    ./configure --prefix=$HOME

The HFST library may link to numerous FST handling backends with varying licences. If you are going to redistribute the HFST library you compiled, make sure there are no clashes in the licences of the linked libraries.

If you are checking out the development versions from GIT you must first create and install the necessary autotools files from the host system:

autoreconf -i

It is common practice to keep generated files out of version control <https://www.gnu.org/software/automake/manual/automake.html#CVS>_.

For further installation instruction refer to file INSTALL, which contains the standard installation instructions for GNU autoconf based software.

C-API

HFST includes an experimental, very limited C API. The API is built into it's own library, aptly named libhfst_c. On Linux systems, pkg-config should locate the necessary parts, like the libhfst_c.so shared library file, and the libhfst_c.h header file.

Versioning of the C API is unrelated to the versions of hfst itself, use the version of hfst_c instead. For now, it is pre-1.0, that is, it's not stable. Changes must be expected.

Note that there are separate bindings to Python from HFST, so there is no need to go through the C API for using HFST from Python.

Troubleshooting

In this section we list the error

Hfst

Install / Use

README