HUSCAP logo Hokkaido Univ. logo

Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >

Text-to-image GAN-based Scene Retrieval and Re-ranking Considering Word Importance

This item is licensed under:Creative Commons Attribution 4.0 International

Files in This Item:
08895780.pdf1.86 MBPDFView/Open
Please use this identifier to cite or link to this item:http://hdl.handle.net/2115/76281

Title: Text-to-image GAN-based Scene Retrieval and Re-ranking Considering Word Importance
Authors: Yanagi, Rintaro Browse this author
Togo, Ren Browse this author
Ogawa, Takahiro Browse this author →KAKEN DB
Haseyama, Miki Browse this author →KAKEN DB
Keywords: Text-to-image generative adversarial network
multimedia information retrieval
scene retrieval
re-ranking
Issue Date: 11-Nov-2019
Publisher: IEEE
Journal Title: IEEE Access
Volume: 7
Issue: 1
Start Page: 169920
End Page: 169930
Publisher DOI: 10.1109/ACCESS.2019.2952676
Abstract: In this paper, we propose a novel scene retrieval and re-ranking method based on a text-toimage Generative Adversarial Network (GAN). The proposed method generates an image from an input query sentence based on the text-to-image GAN and then retrieves a scene that is the most similar to the generated image. By utilizing the image generated from the input query sentence as a query, we can control semantic information of the query image at the text level. Furthermore, we introduce a novel interactive reranking scheme to our retrieval method. Specifically, users can consider the importance of each word within the first input query sentence. Then the proposed method re-generates the query image that reflects the word importance provided by users. By updating the generated query image based on the word importance, it becomes feasible for users to revise retrieval results through this re-ranking process. In experiments, we showed that our retrieval method including the re-ranking scheme outperforms recently proposed retrieval methods.
Rights: https://creativecommons.org/licenses/by/4.0/
Type: article
URI: http://hdl.handle.net/2115/76281
Appears in Collections:情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)

Submitter: 藤後 廉

Export metadata:

OAI-PMH ( junii2 , jpcoar_1.0 )

MathJax is now OFF:


 

 - Hokkaido University