HUSCAP logo Hokkaido Univ. logo

Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >

Multi-Emotion Estimation in Narratives from Crowdsourced Annotations

Files in This Item:
jcdl2015.pdf946.5 kBPDFView/Open
Please use this identifier to cite or link to this item:

Title: Multi-Emotion Estimation in Narratives from Crowdsourced Annotations
Authors: Duan, Lei Browse this author
Oyama, Satoshi Browse this author →KAKEN DB
Sato, Haruhiko Browse this author →KAKEN DB
Kurihara, Masahito Browse this author →KAKEN DB
Keywords: Multi-emotion annotation
Emotional Consistency
Contextual cue
Human computation
Issue Date: 2015
Publisher: ACM
Citation: JCDL '15 Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, ISBN: 978-1-4503-3594-2
Start Page: 91
End Page: 100
Publisher DOI: 10.1145/2756406.2756910
Abstract: Emotion annotations are important metadata for narrative texts in digital libraries. Such annotations are necessary for automatic text-to-speech conversion of narratives and affective education support and can be used as training data for machine learning algorithms to train automatic emotion detectors. However, obtaining high-quality emotion annotations is a challenging problem because it is usually expensive and time-consuming due to the subjectivity of emotion. Moreover, due to the multiplicity of “emotion”, emotion annotations more naturally fit the paradigm of multi-label classification than that of multi-class classification since one instance (such as a sentence) may evoke a combination of multiple emotion categories. We thus investigated ways to obtain a set of high-quality emotion annotations ({instance, multi-emotion} paired data) from variable-quality crowdsourced annotations. A common quality control strategy for crowdsourced labeling tasks is to aggregate the responses provided by multiple annotators to produce a reliable annotation. Given that the categories of “emotion” have characteristics different from those of other kinds of labels, we propose incorporating domain-specific information of emotional consistencies across instances and contextual cues among emotion categories into the aggregation process. Experimental results demonstrate that, from a limited number of crowdsourced annotations, the proposed models enable gold standards to be more effectively estimated than the majority vote and the original domain-independent model.
Conference Name: ACM/IEEE-CS Joint Conference on Digital Libraries
Conference Sequence: 15
Conference Place: Knoxville, TN
Rights: ©2015 ACM. This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in JCDL '15 Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, ISBN: 978-1-4503-3594-2, 2015
Type: proceedings (author version)
Appears in Collections:情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)

Submitter: 小山 聡

Export metadata:

OAI-PMH ( junii2 , jpcoar )


Feedback - Hokkaido University