HUSCAP logo Hokkaido Univ. logo

Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >

Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion

This item is licensed under: Creative Commons Attribution 4.0 International

Files in This Item:

The file(s) associated with this item can be obtained from the following URL:https://doi.org/10.3390/s20072146


Title: Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion
Authors: Moroto, Yuya Browse this author
Maeda, Keisuke Browse this author
Ogawa, Takahiro Browse this author →KAKEN DB
Haseyama, Miki Browse this author →KAKEN DB
Keywords: tensor analysis
visual attention
change with time
feature fusion
convolutional neural network
Issue Date: Apr-2020
Publisher: MDPI
Journal Title: Sensors
Volume: 20
Issue: 7
Start Page: 2146
Publisher DOI: 10.3390/s20072146
Abstract: The paper proposes a method of visual attention-based emotion classification through eye gaze analysis. Concretely, tensor-based emotional category classification via visual attention-based heterogeneous convolutional neural network (CNN) feature fusion is proposed. Based on the relationship between human emotions and changes in visual attention with time, the proposed method performs new gaze-based image representation that is suitable for reflecting the characteristics of the changes in visual attention with time. Furthermore, since emotions evoked in humans are closely related to objects in images, our method uses a CNN model to obtain CNN features that can represent their characteristics. For improving the representation ability to the emotional categories, we extract multiple CNN features from our novel gaze-based image representation and enable their fusion by constructing a novel tensor consisting of these CNN features. Thus, this tensor construction realizes the visual attention-based heterogeneous CNN feature fusion. This is the main contribution of this paper. Finally, by applying logistic tensor regression with general tensor discriminant analysis to the newly constructed tensor, the emotional category classification becomes feasible. Since experimental results show that the proposed method enables the emotional category classification with the F1-measure of approximately 0.6, and about 10% improvement can be realized compared to comparative methods including state-of-the-art methods, the effectiveness of the proposed method is verified.
Rights: https://creativecommons.org/licenses/by/4.0/
Type: article
URI: http://hdl.handle.net/2115/78881
Appears in Collections:情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)

Export metadata:

OAI-PMH ( junii2 , jpcoar )

MathJax is now OFF:


 

 - Hokkaido University