Introduction

This repository is for the ACL 2024 workshop paper Statements: Universal Information Extraction from Tables with Large Language Models for ESG KPIs.

In this paper, we propose STATEMENTS as a new knowledge model for storing quantiative information in a domain agnotic, uniform structure. The task of converting a raw input (table or text) to Statements is called Statement Extraction (SE). The statement extraction task falls under the category of universal information extraction.

Links:

• Data
• Code
• Arxiv Paper

Following notebooks are provided:

• Data preprocessing
• Statements Preview + Convert labels table to statement
• Tree Similarity Score

Following scripts are provided:

• Model training
• Model inferencing
• Statement extraction (if task was indirect statement extraction, see below)
• Evaluation

SemTabNet

Data

The SemTabNet data was originally prepared from annotating 84,890 table cells in 1107 tables from multiple ESG Reports. Each cell in our dataset is classified into one of the following 16 categories:

• Property, Property Value
• Sub-property
• Subject, Subject Value
• Unit, Unit Value
• Time, Time Value
• Key, Key Value
• Header 1, Header 2, Header 3
• Empty, Rubbish

Thus, each original table gives rise to another table (same shape and size) consisting only of the annotation labels. We call this the labels table. Using a set of rules, it is now possible to write a program which takes in the original table and the labels table to produce the statements.

In addition, we augment the original tables to give rise to at-most 130 more tables. This leads to the final SemTabNet which contains over 120k tables.

Experiments

In our paper, we consider three experiments, for statement extraction from tables, which are described below:

| Alias | Task | Input | Output| |---|----- |-----|---------| |ud2sd_table| SE Direct | Table in markdown format | Statement in markdown format| | tca1d |SE Indirect 1D | Individual table cell content | Classification label| | tca2d |SE Indirect 2D | Table in markdown format | Labels table in markdown format|

DATA Samples

You can see data samples in this file. These are generated using this notebook.

Citation

@misc{mishra2024statementsuniversalinformationextraction,
      title={Statements: Universal Information Extraction from Tables with Large Language Models for ESG KPIs}, 
      author={Lokesh Mishra and Sohayl Dhibi and Yusik Kim and Cesar Berrospi Ramis and Shubham Gupta and Michele Dolfi and Peter Staar},
      year={2024},
      eprint={2406.19102},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2406.19102}, 
}