GRASP - Generic Reasoning and SPARQL Generation across Knowledge Graphs

News

October 20th 2025:
- GRASP can now be used for entity linking, in particular for cell entity annotation
- Workshop paper published for SemTab 2025 challenge
- Preview of camera-ready version available here
August 28th 2025:
- Demo paper of GRASP has also been accepted to ISWC 2025
- Preview of camera-ready version available here
July 31st 2025:
- GRASP has been accepted to ISWC 2025
- Preview of camera-ready version available here
July 14th 2025:
- arXiv preprint available at arxiv.org/abs/2507.08107
July 10th 2025:
- Code release
- Data release

Overview and directory structure

Links:

Public demo available at grasp.cs.uni-freiburg.de
Data available at ad-publications.cs.uni-freiburg.de/grasp

apps/
  evaluation/                     # Streamlit app for evaluation
  grasp/                          # Svelte web app compatible with GRASP server
  grisp/                          # Svelte web app compatible with GRISP server
bash/                             # Bash scripts to run and evaluate GRASP
configs/
  run.yaml                        # Config to run GRASP with a single KG
  serve.yaml                      # Config to run GRASP with all available KGs
  grisp/                          # Configs for the GRISP baseline
  notes/                          # Configs for note-taking
queries/                          # Custom index data and info SPARQL queries
                                    for various knowledge graphs
scripts/                          # Various helper scripts
data/                          
  benchmark/                      # Benchmarks grouped by knowledge graph
    [knowledge-graph]/
      [benchmark]/                   
        test.jsonl                # Test set with input and ground truth
        train-example-index/      # Index based on train set for few-shot learning
                                    (needs to be downloaded)
        outputs/
          [model].jsonl           # Model output
          [model].config.json     # Model config
          [model].evaluation.json # Evaluation against ground truth
  kg-index/                       # KG indices (need to be downloaded)
    wikidata/
    freebase/
    ...
src/                              # Source code for GRASP
Makefile                          # Makefile for building benchmarks

Quickstart

Follow these steps to run GRASP. If you want to use Docker, see section Run GRASP with Docker below.

Run GRASP

Install GRASP

# Via git (recommended, up-to-date version)
pip install git+https://github.com/ad-freiburg/grasp.git@main

# From PyPI (not recommended as of now, at least as long
# as GRASP is under heavy development)
pip install grasp-rdf

Set the GRASP_INDEX_DIR env variable. Defaults to $HOME/.grasp/index if not set. We set it to $PWD/data/kg-index, but you can choose any directory you like.
Get indices for the knowledge graphs you want to use. All indices are available publicly. For example, to get the indices for Wikidata:

# Change to index directory
cd $GRASP_INDEX_DIR
# Download Wikidata index
wget https://ad-publications.cs.uni-freiburg.de/grasp/kg-index/wikidata.tar.gz
# Extract index
tar -xzf wikidata.tar.gz

Optionally, you can also download example indices for few-shot learning. Example indices are always built from the train set of a benchmark and called train-example-index. For example, to get the example index for QALD-10 on Wikidata:

# Change to benchmark directory
cd data/benchmark/wikidata/qald10
# Download example index
wget https://ad-publications.cs.uni-freiburg.de/grasp/benchmark/wikidata/qald10/train-example-index.tar.gz
# Extract example index
tar -xzf train-example-index.tar.gz

Run GRASP:

# Note, that if you e.g. run OpenAI models, you also need to set the
# OPENAI_API_KEY env variable (see section about supported models below).

# Tip: Set --log-level DEBUG to show the individual steps of GRASP
# (reasoning and function calls) in a nicely formatted way.

# Run GRASP on an input and output the result to stdout as JSON with metadata.
# Actual output for the task is in the "output" field of that JSON object.

# Input from stdin:
echo "Where was Angela Merkel born?" | grasp run configs/run.yaml

# Input via CLI argument:
grasp run configs/run.yaml --input "Where was Angela Merkel born?"

# You can run different tasks with GRASP (default is sparql-qa).
# Depending on the task, the expected input format and output format
# will differ. For general-qa, the input is also a natural language
# question, same as for sparql-qa, but the output will be just a natural
# language answer instead of a SPARQL query.
echo "Where was Angela Merkel born?" | grasp run configs/run.yaml --task general-qa

# For cell entity annotation (cea), the input is a JSON object with a "table"
# field containing "header" and "data". The task links table cells to entities
# in the knowledge graph.
grasp run configs/run.yaml --task cea --input-format json \
  --input '{"table": {"header": ["Country", "Capital"], "data": [["France", "Paris"]]}}'

# Show all available options:
grasp run -h

# You can also run GRASP on multiple inputs (in JSONL format).
# In the following, we show an example to run GRASP on the QALD-10 
# test set over Wikidata.

# Input from stdin:
cat data/benchmark/wikidata/qald10/test.jsonl | grasp file configs/run.yaml

# Input via CLI argument:
grasp file configs/run.yaml --input-file data/benchmark/wikidata/qald10/test.jsonl

# Save output to a file instead of stdout and show progress bar:
grasp file configs/run.yaml \
  --input-file data/benchmark/wikidata/qald10/test.jsonl \
  --output-file data/benchmark/wikidata/qald10/outputs/test.jsonl \
  --progress

# Show all available options:
grasp file -h

# You can also run GRASP in a client-server setup. This is also the server
# that powers the corresponding web app.
# To start a GRASP server, by default on port 8000, just run:
grasp serve configs/run.yaml

# For convenience, we also provide a config to run the server with all
# available knowledge graphs (make sure to download all indices first):
grasp serve configs/serve.yaml

# Show all available options:
grasp serve -h

# Evaluate GRASP output with F1 score (for sparql-qa task) by executing
# predicted and ground truth SPARQL queries and comparing results:
grasp evaluate f1 wikidata \
  data/benchmark/wikidata/qald10/test.jsonl \
  data/benchmark/wikidata/qald10/outputs/gpt-41.search_extended.jsonl

# Use a judge model to pick the best output from multiple prediction files.
# The last argument is the output evaluation file:
grasp evaluate judge configs/run.yaml \
  data/benchmark/wikidata/qald10/test.jsonl \
  data/benchmark/wikidata/qald10/outputs/model1.jsonl \
  data/benchmark/wikidata/qald10/outputs/model2.jsonl \
  data/benchmark/wikidata/qald10/outputs/judge.evaluation.json

# Show all available options:
grasp evaluate f1 -h
grasp evaluate judge -h

# Build an example index for few-shot learning from a JSONL file of examples:
grasp examples data/benchmark/wikidata/qald10/train.jsonl \
  data/benchmark/wikidata/qald10/train-example-index

# Cache entity and property information for a knowledge graph (speeds up
# runtime by pre-fetching info SPARQL query results):
grasp cache wikidata

# Merge data from multiple knowledge graphs into a single combined KG.
# The first KG is the primary one; entities/properties from subsequent KGs
# are added to it. For example used to combine language-specific indices
# of the same knowledge graph:
grasp merge wikidata-en wikidata-de wikidata-fr wikidata-multilingual

# Note-taking: run GRASP on a knowledge graph to produce notes that
# can be included in the config to improve performance.
# Take notes by running GRASP on exemplary task samples:
grasp notes samples configs/notes/samples.yaml notes/
# Take notes from existing GRASP output files:
grasp notes outputs configs/notes/outputs.yaml notes/
# Take notes by freely exploring a knowledge graph (no task samples needed):
grasp notes explore configs/notes/explore.yaml notes/

Server API

When running grasp serve, the server exposes the following HTTP endpoints (by default on port 8000).

| Endpoint | Method | Description | |---|---|---| | /knowledge_graphs | GET | Returns list of available KG names | | /config | GET | Returns the server configuration | | /run | POST | Run GRASP on a single input |

`POST /run`

Request body:

{
  "task": "sparql-qa",
  "input": "Where was Angela Merkel born?",
  "knowledge_graphs": ["wikidata"],
  "past": null
}

task: one of "sparql-qa", "general-qa", "cea", "wikidata-query-logs"
input: the task input (string for most tasks, JSON object for "cea")
knowledge_graphs: list of one or more KG names available on the server
past (optional): conversation history for multi-turn interactions, with fields messages (list of previous messages) and known (set of known entity/property IRIs)

Response: GRASP output as a JSON object

Error codes: 400 invalid KG selection, 429 rate limit exceeded, 503 server busy, 504 generation timeout

Run GRASP with Docker

Build the Docker image:

docker build -t grasp .

The entrypoint for the Docker image is the grasp CLI

Grasp

Install / Use

README