Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Text-to-image GAN-based Scene Retrieval and Re-ranking Considering Word Importance
This item is licensed under:Creative Commons Attribution 4.0 International
Title: | Text-to-image GAN-based Scene Retrieval and Re-ranking Considering Word Importance |
Authors: | Yanagi, Rintaro Browse this author | Togo, Ren Browse this author | Ogawa, Takahiro Browse this author →KAKEN DB | Haseyama, Miki Browse this author →KAKEN DB |
Keywords: | Text-to-image generative adversarial network | multimedia information retrieval | scene retrieval | re-ranking |
Issue Date: | 11-Nov-2019 |
Publisher: | IEEE |
Journal Title: | IEEE Access |
Volume: | 7 |
Issue: | 1 |
Start Page: | 169920 |
End Page: | 169930 |
Publisher DOI: | 10.1109/ACCESS.2019.2952676 |
Abstract: | In this paper, we propose a novel scene retrieval and re-ranking method based on a text-toimage Generative Adversarial Network (GAN). The proposed method generates an image from an input query sentence based on the text-to-image GAN and then retrieves a scene that is the most similar to the generated image. By utilizing the image generated from the input query sentence as a query, we can control semantic information of the query image at the text level. Furthermore, we introduce a novel interactive reranking scheme to our retrieval method. Specifically, users can consider the importance of each word within the first input query sentence. Then the proposed method re-generates the query image that reflects the word importance provided by users. By updating the generated query image based on the word importance, it becomes feasible for users to revise retrieval results through this re-ranking process. In experiments, we showed that our retrieval method including the re-ranking scheme outperforms recently proposed retrieval methods. |
Rights: | https://creativecommons.org/licenses/by/4.0/ |
Type: | article |
URI: | http://hdl.handle.net/2115/76281 |
Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
Submitter: 藤後 廉
|