open access

https://hdl.handle.net/2115/87060

Multi-modal shared module that enables the bottom-up formation of map representation and top-down map reading

英語

symbol grounding

2021-11-02

Taylor & Francis

Advanced Robotics

36(1-2)

85-99

Humans create internal models of an environment (i.e. cognitive maps) through subjective sensorimotor experiences and can also understand spatial locations by looking at an external map as a symbol of an environment. We simulate the development of the cognitive map from sensorimotor experiences and grounding of the external map in a single deep neural network model. Our proposed network has a shared module that processes the features of multiple modalities (i.e. vision, hearing, and touch) and even external maps in the same manner. The multiple modalities are encoded into feature vectors by modality-specific encoders, and the encoded features are processed by the same shared module. The proposed network was trained to predict the sensory inputs of a simulated mobile robot. After the predictive learning, the spatial representation was developed in the internal states of the shared module, and the same spatial representation was used for predicting multiple modalities, including the external map. The network can also perform spatial navigation by associating the external map with the cognitive map. This implies that the external maps are grounded in subjective sensorimotor experiences, being bridged through the developed internal spatial representation in the shared module.

学術雑誌論文

Accepted Manuscript

This is an Accepted Manuscript of an article published by Taylor & Francis in Advanced robotics on 02 Nov 2021, available online: https://www.tandfonline.com/doi/10.1080/01691864.2021.1993334

0169-1864

AA10679021