|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Human-Centric Emotion Estimation Based on Correlation Maximization Considering Changes With Time in Visual Attention and Brain Activity
This item is licensed under:Creative Commons Attribution 4.0 International
| Title: | Human-Centric Emotion Estimation Based on Correlation Maximization Considering Changes With Time in Visual Attention and Brain Activity |
| Authors: | Moroto, Yuya Browse this author | | Maeda, Keisuke Browse this author | | Ogawa, Takahiro Browse this author →KAKEN DB | | Haseyama, Miki Browse this author →KAKEN DB |
| Keywords: | Multimodal approach | | CCA | | changes with time | | tensor analysis | | eye gaze data | | fNIRS |
| Issue Date: | 12-Jan-2021 |
| Publisher: | IEEE (Institute of Electrical and Electronics Engineers) |
| Journal Title: | IEEE Access |
| Volume: | 8 |
| Start Page: | 203358 |
| End Page: | 203368 |
| Publisher DOI: | 10.1109/ACCESS.2020.3036908 |
| Abstract: | A human-centric emotion estimation method based on correlation maximization with consideration of changes with time in visual attention and brain activity when viewing images is proposed in this paper. Owing to the recent developments of many kinds of biological sensors, many researchers have focused on multimodal emotion estimation using both eye gaze data and brain activity data for improving the quality of emotion estimation. In this paper, a novel method that focuses on the following two points is introduced. First, in order to reduce the burden on users, we obtain brain activity data from users only in the training phase by using a projection matrix calculated by canonical correlation analysis (CCA) between gaze-based visual features and brain activity-based features. Next, for considering the changes with time in both visual attention and brain activity, we obtain novel features based on CCA-based projection in each time unit. In order to include these two points, the proposed method analyzes a fourth-order gaze and image tensor for which modes are pixel location, color channel and the changes with time in visual attention. Moreover, in each time unit, the proposed method performs CCA between gaze-based visual features and brain activity-based features to realize human-centric emotion estimation with a high level of accuracy. Experimental results show that accurate human emotion estimation is achieved by using our new human-centric image representation. |
| Rights: | https://creativecommons.org/licenses/by/4.0/ |
| Type: | article |
| URI: | http://hdl.handle.net/2115/80140 |
| Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
|
