ELFEN - Efficient Linguistic Feature Extraction for Natural Language Datasets

This Python package provides efficient linguistic feature extraction for text datasets (i.e. datasets with N text instances, in a tabular structure).

For a full overview of the features available, check the overview table, for further details and tutorials check the documentation. Some example use cases are illustrated in Jupyter notebooks in this repository

The multilingual support is documented in the multilingual support table.

Check out our roadmap for planned features and improvements. Check out the changelog for recent changes.

Installation

Install this package using the current PyPI version

python -m pip install elfen

Install this package from source

python -m pip install git+https://github.com/mmmaurer/elfen.git

If you want to use the spacy backbone, you will need to download the respective model, e.g. "en_core_web_sm":

python -m spacy download en_core_web_sm

To use wordnet features, download open multilingual wordnet using:

python -m wn download omw:1.4

Note that for some languages, you will need to install another wordnet collection. For example, for German, you can use the following command:

python -m wn download odenet:1.4

For more information on the available wordnet collections, consult the wn package documentation.

Note that if you have used elfen or the wn package before, you may encounter errors from the wn package about an incompatible database schema. In that case, you can use the following command to reset the database:

  import wn
  wn.reset_database(rebuild=True)

[!IMPORTANT] Some of the external resources used for feature extraction (e.g., NRC lexicons) have to be downloaded manually due to licensing restrictions. For this, please see this guide. Note that without these resources, only a subset of features will be available.

Multiprocessing and limiting the numbers of cores used

The underlying dataframe library, polars, uses all available cores by default. If you are working on a shared server, you may want to consider limiting the resources available to polars. To do that, you will have to set the POLARS_MAX_THREADS variable in your shell, e.g.:

export POLARS_MAX_THREADS=8

Usage of third-party resources usable in this package

The extraction of psycholinguistic, emotion/lexicon and semantic features relies on third-party resources such as lexicons. Please refer to the original author's licenses and conditions for usage, and cite them if you use the resources through this package in your analyses.

For an overview which features use which resource, and how to export all third-party resource references in a bibtex string, consult the documentation.

Acknowledgements

While all feature extraction functions in this package are written from scratch, the choice of features in the readability and lexical richness feature areas (partially) follows the readability and lexicalrichness Python packages.

We use the wn Python package to extract Open Multilingual Wordnet synsets.

Citation

If you use this package in your work, please cite the System Demonstration paper describing the package, to appear in the proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026):

@inproceedings{maurer-2026-elfen,
    title = "elfen: A Python Package for Efficient Linguistic Feature Extraction for Natural Language Datasets",
    author = "Maurer, Maximilian",
    editor = "Croce, Danilo  and
      Leidner, Jochen  and
      Moosavi, Nafise Sadat",
    booktitle = "Proceedings of the 19th Conference of the {E}uropean Chapter of the {A}ssociation for {C}omputational {L}inguistics (Volume 3: System Demonstrations)",
    month = mar,
    year = "2026",
    address = "Rabat, Marocco",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.eacl-demo.5/",
    pages = "61--74",
    ISBN = "979-8-89176-382-1"
}