|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Time-Lag Aware Latent Variable Model for Prediction of Important Scenes Using Baseball Videos and Tweets
| Title: | Time-Lag Aware Latent Variable Model for Prediction of Important Scenes Using Baseball Videos and Tweets |
| Authors: | Hirasawa, Kaito Browse this author | | Maeda, Keisuke Browse this author | | Ogawa, Takahiro Browse this author →KAKEN DB | | Haseyama, Miki Browse this author →KAKEN DB |
| Keywords: | latent variable model | | prediction of important scenes | | Twitter | | sports video | | time lags |
| Issue Date: | 23-Mar-2022 |
| Publisher: | MDPI |
| Journal Title: | Sensors |
| Volume: | 22 |
| Issue: | 7 |
| Start Page: | 2465 |
| Publisher DOI: | 10.3390/s22072465 |
| Abstract: | In this study, a novel prediction method for predicting important scenes in baseball videos using a time-lag aware latent variable model (Tl-LVM) is proposed. Tl-LVM adopts a multimodal variational autoencoder using tweets and videos as the latent variable model. It calculates the latent features from these tweets and videos and predicts important scenes using these latent features. Since time lags exist between posted tweets and events, Tl-LVM introduces the loss considering time lags by correlating the feature into the loss function of the multimodal variational autoencoder. Furthermore, Tl-LVM can train the encoder, decoder, and important scene predictor, simultaneously, using this loss function. This is the novelty of Tl-LVM, and this work is the first end-to-end prediction model of important scenes that considers time lags to the best of our knowledge. It is the contribution of Tl-LVM to realize high-quality prediction using latent features that consider time lags between tweets and multiple corresponding previous events. Experimental results using actual tweets and baseball videos show the effectiveness of Tl-LVM. |
| Type: | article |
| URI: | http://hdl.handle.net/2115/85169 |
| Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
|
