The code and data for the paper "A More Fine-Grained Aspect-Sentiment-Opinion Triplet Extraction Task"

ASOTE

Aspect-Sentiment-Opinion Triplet Extraction (ASOTE) extracts aspect term, sentiment and opinion term triplets from sentences. In the triplet extracted by ASOTE the sentiment is the sentiment of the aspect term and opinion term pair. For example, given the sentence, "The atmosphere is attractive , but a little uncomfortable.", ASOTE extracts two triplets, ("atmosphere", positive, "attractive") and ("atmosphere", negative, "uncomfortable").

Differences between ASOTE and ASTE

Aspect Sentiment Triplet Extraction (ASTE) (Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis) also extracts triplets from sentences. Each triplet extracted by ASTE contains an aspect term, <u>the sentiment that the sentence expresses toward the aspect term</u>, and one opinion term associated with the aspect. For example, In the third sentence, the negative sentiment toward the aspect term “food” is expressed without an annotatable opinion.

Data

Dataset Construction

We build four datasets for the ASOTE task: 14res, 14lap, 15res, 16res.

Specifically, we first construct four ASTE datasets by merging four ATSA datasets from three SemEval tasks (SemEval-2014 Task 4, SemEval-2015 Task 12, SemEval-2016 Task 5) and four TOWE datasets like Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis, then manually annotate the sentiment of the aspect and opinion pair in the ASTE datasets.

For example, given the sentence, "The atmosphere is attractive, but a little uncomfortable.", its annotations in SemEval-2014 Task 4 are as follows:

<sentence id="32464601#418474#1">
    <text>The atmosphere is attractive, but a little uncomfortable.</text>
    <aspectTerms>
        <aspectTerm term="atmosphere" polarity="conflict"  from="4" to="14"/>
    </aspectTerms>
</sentence>

its annotations in TOWE datasets are as follows:

32464601#418474#1	The atmosphere is attractive , but a little uncomfortable .	The\O atmosphere\B is\O attractive\O ,\O but\O a\O little\O uncomfortable\O .\O	The\O atmosphere\O is\O attractive\B ,\O but\O a\O little\O uncomfortable\B .\O

We align the above annotations of the sentence and get an ASTE sample:

{
    "sentence": "The atmosphere is attractive , but a little uncomfortable .", 
    "words": [
        "The", 
        "atmosphere", 
        "is", 
        "attractive", 
        ",", 
        "but", 
        "a", 
        "little", 
        "uncomfortable", 
        "."
    ], 
    "polarity": "conflict", 
    "opinions": [
        {
            "aspect_term": {
                "start": 1, 
                "end": 2, 
                "term": "atmosphere"
            }, 
            "opinion_term": {
                "start": 3, 
                "end": 4, 
                "term": "attractive"
            }
        }, 
        {
            "aspect_term": {
                "start": 1, 
                "end": 2, 
                "term": "atmosphere"
            }, 
            "opinion_term": {
                "start": 8, 
                "end": 9, 
                "term": "uncomfortable"
            }
        }
    ], 
    "aspect_term": {
        "start": 1, 
        "end": 2, 
        "term": "atmosphere"
    }
}

We then annotate the sentiments of the aspect term and opinion term pairs in the ASTE sample and get an ASOTE sample:

{
    "sentence": "The atmosphere is attractive , but a little uncomfortable .", 
    "words": [
        "The", 
        "atmosphere", 
        "is", 
        "attractive", 
        ",", 
        "but", 
        "a", 
        "little", 
        "uncomfortable", 
        "."
    ], 
    "polarity": "conflict", 
    "opinions": [
        {
            "aspect_term": {
                "start": 1, 
                "end": 2, 
                "term": "atmosphere"
            }, 
            "opinion_term": {
                "start": 3, 
                "end": 4, 
                "term": "attractive"
            }, 
            "polarity": "positive"
        }, 
        {
            "aspect_term": {
                "start": 1, 
                "end": 2, 
                "term": "atmosphere"
            }, 
            "opinion_term": {
                "start": 8, 
                "end": 9, 
                "term": "uncomfortable"
            }, 
            "polarity": "negative"
        }
    ], 
    "aspect_term": {
        "start": 1, 
        "end": 2, 
        "term": "atmosphere"
    }
}

Note that, in our ASOTE datasets, triplets belonging to different aspect terms in the same sentence are in different json lines.

Data version

v1

14res, 14lap, 15res, 16res

The v1 datasets only includes sentences which contain at least one aspect term.

v2

14res, 14lap, 15res, 16res

The v2 datasets includes both sentences which contain aspect terms and sentences which do not contain aspect terms. That is, the v2 datasets include all sentences in the corresponding ATSA datasets from the three SemEval tasks (SemEval-2014 Task 4, SemEval-2015 Task 12, SemEval-2016 Task 5). For example, although the sentence, "It gets very crowded so I would suggest that you get their early .", do not contain aspect terms, it also is included in the v2 datasets. We think datasets including these sentences can better evaluate the performance of methods, since methods can encounter this kind of sentences in real-world scenarios.

Is it necessary to annotate the sentiments of the aspect term and opinion term pairs in the ASTE datasets for obtaining our ASOTE datasets?

Wang et al. (2016, 2017) have annotated the opinions and thier sentiments of the sentences in the restaurant and laptop datasets from SemEval-2014 Task 4 and the restaurant dataset from SemEval-2015 Task 12. Is it necessary to annotate the sentiments of the aspect term and opinion term pairs in the ASTE datasets for obtaining our ASOTE datasets? The answer is yes. The reasons are as follows:

• The sentiments of aspect term and opinion term pairs are different from the sentiments of opinions.
• Wang et al. (2016, 2017) only annotated the opinions with positive sentiments or negative sentiments. That is, the opinions with neutral sentiments were not annotated.
• The opinions annotated by Wang et al. (2016, 2017) are different from the opinions annotated in the TOWE datasets which are used to construct our ASOTE datasets. For example, given this sentence, "those rolls were big , but not good and sashimi wasn't fresh.", the opinions and their sentiments annotated by Wang et al. (2016, 2017) are big +1, good +1, and fresh +1 (+1 indicates positive sentiment), while the opinions annotated in the TOWE datasets are "big", "not good" and "wasn't fresh". We think the opinions annotated in the TOWE datasets are more appropriate for aspect-sentiment-opinion triplets.

The datasets provided by Wang et al. (2016, 2017) can be found here:

• Recursive-Neural-Conditional-Random-Field
• Coupled-Multi-layer-Attentions

Baselines

Some models have been proposed to extract opinion triplets from sentences. However, these models were only evaluated on the ASTE task. We have evaluated the performances of a few opinion triplet extraction models on the ASOTE task:

• Grid Tagging Scheme for Aspect-oriented Fine-grained Opinion Extraction. Zhen Wu, Chengcan Ying, Fei Zhao, Zhifang Fan, Xinyu Dai, Rui Xia. In Findings of EMNLP, 2020. original_repository GTS_on_ASOTE
• A Multi-task Learning Framework for Opinion Triplet Extraction. Chen Zhang, Qiuchi Li, Dawei Song, Benyou Wang. In Findings of EMNLP, 2020. original_repository OTE-MTL_on_ASOTE
• Position-Aware Tagging for Aspect Sentiment Triplet Extraction. Lu Xu, Hao Li, Wei Lu, Lidong Bing. In EMN