CIAN

Character-level Intra Attention Network

Implementation of the paper Character-level Intra Attention Network (CIAN) in proceddings of the RepEval Workshop in The 2017 Conference on Empirical Methods on Natural Language Processing.

Architecture of the model:

<p align="center"><img src="https://github.com/yanghanxy/CIAN/blob/master/figure/architecture.png" height="387" width="459"></p>

Requirements

Code is written in python 2.7 and requires Keras 2.

Data

Dataset could be downloaded at MultiNLI and SNLI.

How to run

First to do a modification with Keras, see the following section.

Dataset should be be put in folder ./data

To run the model, use

python ./model.py

The result and log file will be saved in ./log folder.

Modification with keras

In #Python_Path\Lib\site-packages\keras\preprocessing\text.py, line 39,

CHANGE

    text = text.translate(maketrans(filters, split * len(filters)))

TO

    try:
        text = unicode(text, "utf-8")
    except TypeError:
        pass
    translate_table = {ord(c): ord(t) for c, t in zip(filters, split * len(filters))}
    text = text.translate(translate_table)

Result

<p align="center"><img src="https://github.com/yanghanxy/CIAN/blob/master/figure/training.png" height="428" width="783"></p>

Visualization of Attention

PairID 192997e, label Entailment

<p align="center"><img src="https://github.com/yanghanxy/CIAN/blob/master/figure/PairID_192997e.PNG" height="764" width="594"></p>

PairID 254941e, label Entailment

<p align="center"><img src="https://github.com/yanghanxy/CIAN/blob/master/figure/PairID_254941e.PNG" height="760" width="603"></p>

Reference

[1] Character-Aware Neural Language Models

[2] Intra Attention Mechanism