|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Multi-Emotion Estimation in Narratives from Crowdsourced Annotations
| Title: | Multi-Emotion Estimation in Narratives from Crowdsourced Annotations |
| Authors: | Duan, Lei Browse this author | | Oyama, Satoshi Browse this author →KAKEN DB | | Sato, Haruhiko Browse this author →KAKEN DB | | Kurihara, Masahito Browse this author →KAKEN DB |
| Keywords: | Multi-emotion annotation | | Emotional Consistency | | Contextual cue | | Crowdsourcing | | Human computation |
| Issue Date: | 2015 |
| Publisher: | ACM |
| Citation: | JCDL '15 Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, ISBN: 978-1-4503-3594-2 |
| Start Page: | 91 |
| End Page: | 100 |
| Publisher DOI: | 10.1145/2756406.2756910 |
| Abstract: | Emotion annotations are important metadata for narrative texts in digital libraries. Such annotations are necessary for automatic text-to-speech conversion of narratives and affective education support and can be used as training data for machine learning algorithms to train automatic emotion detectors. However, obtaining high-quality emotion annotations is a challenging problem because it is usually expensive and time-consuming due to the subjectivity of emotion. Moreover, due to the multiplicity of “emotion”, emotion annotations more naturally fit the paradigm of multi-label classification than that of multi-class classification since one instance (such as a sentence) may evoke a combination of multiple emotion categories. We thus investigated ways to obtain a set of high-quality emotion annotations ({instance, multi-emotion} paired data) from variable-quality crowdsourced annotations. A common quality control strategy for crowdsourced labeling tasks is to aggregate the responses provided by multiple annotators to produce a reliable annotation. Given that the categories of “emotion” have characteristics different from those of other kinds of labels, we propose incorporating domain-specific information of emotional consistencies across instances and contextual cues among emotion categories into the aggregation process. Experimental results demonstrate that, from a limited number of crowdsourced annotations, the proposed models enable gold standards to be more effectively estimated than the majority vote and the original domain-independent model. |
| Conference Name: | ACM/IEEE-CS Joint Conference on Digital Libraries |
| Conference Sequence: | 15 |
| Conference Place: | Knoxville, TN |
| Rights: | ©2015 ACM. This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in JCDL '15 Proceedings of the 15th ACM/IEEE-CS Joint Conference on Digital Libraries, ISBN: 978-1-4503-3594-2, 2015 http://doi.acm.org/10.1145/2756406.2756910 |
| Type: | proceedings (author version) |
| URI: | http://hdl.handle.net/2115/65265 |
| Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
Submitter: 小山 聡
|
