|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion
This item is licensed under:Creative Commons Attribution 4.0 International
| Title: | Tensor-Based Emotional Category Classification via Visual Attention-Based Heterogeneous CNN Feature Fusion |
| Authors: | Moroto, Yuya Browse this author | | Maeda, Keisuke Browse this author | | Ogawa, Takahiro Browse this author →KAKEN DB | | Haseyama, Miki Browse this author →KAKEN DB |
| Keywords: | tensor analysis | | visual attention | | change with time | | feature fusion | | convolutional neural network |
| Issue Date: | Apr-2020 |
| Publisher: | MDPI |
| Journal Title: | Sensors |
| Volume: | 20 |
| Issue: | 7 |
| Start Page: | 2146 |
| Publisher DOI: | 10.3390/s20072146 |
| Abstract: | The paper proposes a method of visual attention-based emotion classification through eye gaze analysis. Concretely, tensor-based emotional category classification via visual attention-based heterogeneous convolutional neural network (CNN) feature fusion is proposed. Based on the relationship between human emotions and changes in visual attention with time, the proposed method performs new gaze-based image representation that is suitable for reflecting the characteristics of the changes in visual attention with time. Furthermore, since emotions evoked in humans are closely related to objects in images, our method uses a CNN model to obtain CNN features that can represent their characteristics. For improving the representation ability to the emotional categories, we extract multiple CNN features from our novel gaze-based image representation and enable their fusion by constructing a novel tensor consisting of these CNN features. Thus, this tensor construction realizes the visual attention-based heterogeneous CNN feature fusion. This is the main contribution of this paper. Finally, by applying logistic tensor regression with general tensor discriminant analysis to the newly constructed tensor, the emotional category classification becomes feasible. Since experimental results show that the proposed method enables the emotional category classification with the F1-measure of approximately 0.6, and about 10% improvement can be realized compared to comparative methods including state-of-the-art methods, the effectiveness of the proposed method is verified. |
| Rights: | https://creativecommons.org/licenses/by/4.0/ |
| Type: | article |
| URI: | http://hdl.handle.net/2115/78881 |
| Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
|
