DeepKE
[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction
Install / Use
/learn @zjunlp/DeepKEREADME
DeepKE is a knowledge extraction toolkit for knowledge graph construction supporting cnSchema,low-resource, document-level and multimodal scenarios for entity, relation and attribute extraction. We provide documents, online demo, paper, slides and poster for beginners.
- ❗Want to use Large Language Models with DeepKE? Try DeepKE-LLM and OneKE, have fun!
- ❗Want to train supervised models? Try Quick Start, we provide the NER models (e.g, LightNER(COLING'22), W2NER(AAAI'22)), relation extraction models (e.g., KnowPrompt(WWW'22)), relational triple extraction models (e.g., ASP(EMNLP'22), PRGC(ACL'21), PURE(NAACL'21)), and release off-the-shelf models at DeepKE-cnSchema, have fun!
- We recommend using Linux; if using Windows, please use
\\in file paths; - If HuggingFace is inaccessible, please consider using
wisemodelormodescape.
If you encounter any issues during the installation of DeepKE and DeepKE-LLM, please check Tips or promptly submit an issue, and we will assist you with resolving the problem!
Table of Contents
- Table of Contents
- What's New
- Prediction Demo
- Model Framework
- Quick Start
- Tips
- To do
- Reading Materials
- Related Toolkit
- Citation
- Contributors
- Other Knowledge Extraction Open-Source Projects
What's New
June, 2025We integrate the MCP service tools into DeepKE, enabling knowledge extraction through large language models (LLMs) as tool callers for lightweight models.December, 2024We open source the OneKE knowledge extraction framework, supporting multi-agent knowledge extraction across various scenarios.April, 2024We release a new bilingual (Chinese and English) schema-based information extraction model called OneKE based on Chinese-Alpaca-2-13B.Feb, 2024We release a large-scale (0.32B tokens) high-quality bilingual (Chinese and English) Information Extraction (IE) instruction dataset named IEPile, along with two models trained withIEPile, baichuan2-13b-iepile-lora and llama2-13b-iepile-lora.Sep 2023a bilingual Chinese English Information Extraction (IE) instruction dataset calledInstructIEwas released for the Instruction based Knowledge Graph Construction Task (Instruction based KGC), as detailed in here.June, 2023We update DeepKE-LLM to support knowledge extraction with KnowLM, ChatGLM, LLaMA-series, GPT-series etc.Apr, 2023We have added new models, including CP-NER(IJCAI'23), ASP(EMNLP'22), PRGC(ACL'21), PURE(NAACL'21), provided event extraction capabilities (Chinese and English), and offered compatibility with higher versions of Python packages (e.g., Transformers).Feb, 2023We have supported using LLM (GPT-3) with in-context learning (based on EasyInstruct) & data generation, added a NER model W2NER(AAAI'22).
-
Nov, 2022Add data annotation instructions for entity recognition and relation extraction, automatic labelling of weakly supervised data (entity extraction and relation extraction), and optimize multi-GPU training. -
Sept, 2022The paper DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population has been accepted by the EMNLP 2022 System Demonstration Track. -
Aug, 2022We have added data augmentation (Chinese, English) support for low-resource relation extraction. -
June, 2022We have added multimodal support for entity and relation extraction. -
May, 2022We have released DeepKE-cnschema with off-the-shelf knowledge extraction models. -
Jan, 2022We have released a paper DeepKE: A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population -
Dec, 2021We have addeddockerfileto create the enviroment automatically. -
Nov, 2021The demo of DeepKE, supporting real-time extration without deploying and training, has been released. -
The documentation of DeepKE, containing the details of DeepKE such as source codes and datasets, has been released.
-
Oct, 2021pip install deepke -
The codes of deepke-v2.0 have been released.
-
Aug, 2019The codes of deepke-v1.0 have been released. -
Aug, 2018The project DeepKE startup and codes of deepke-v0.1 have been released.
Prediction Demo
There is a demonstration of prediction. The GIF file is created by Terminalizer. Get the code. <img src="pics/demo.gif" width="636" height="494" align=center>
<br>Model Framework
<h3 align="center"> <img src="pics/architectures.png"> </h3>- DeepKE contains a unified framework for named entity recognition, relation extraction and attribute extraction, the three knowledge extraction functions.
- Each task can be implemented in different scenarios. For example, we can achieve relation extraction in standard, low-resource (few-shot), document-level and multimodal settings.
- Each application scenario comprises of three components: Data including Tokenizer, Preprocessor and Loader, Model including Module, Encoder and Forwarder, Core including Training, Evaluation and Prediction.
Quick Start
DeepKE-LLM
In the era of large models, DeepKE-LLM utilizes a completel
Related Skills
YC-Killer
2.7kA library of enterprise-grade AI agents designed to democratize artificial intelligence and provide free, open-source alternatives to overvalued Y Combinator startups. If you are excited about democratizing AI access & AI agents, please star ⭐️ this repository and use the link in the readme to join our open source AI research team.
groundhog
398Groundhog's primary purpose is to teach people how Cursor and all these other coding agents work under the hood. If you understand how these coding assistants work from first principles, then you can drive these tools harder (or perhaps make your own!).
last30days-skill
13.8kAI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary
000-main-rules
Project Context - Name: Interactive Developer Portfolio - Stack: Next.js (App Router), TypeScript, React, Tailwind CSS, Three.js - Architecture: Component-driven UI with a strict separation of conce
