open access

https://hdl.handle.net/2115/77798

Effective neural network training with adaptive learning rate based on training loss

英語

Beam search

2018-05

Elsevier

NEURAL NETWORKS

101

68-78

A method that uses an adaptive learning rate is presented for training neural networks. Unlike most conventional updating methods in which the learning rate gradually decreases during training, the proposed method increases or decreases the learning rate adaptively so that the training loss (the sum of cross-entropy losses for all training samples) decreases as much as possible. It thus provides a wider search range for solutions and thus a lower test error rate. The experiments with some well-known datasets to train a multilayer perceptron show that the proposed method is effective for obtaining a better test accuracy under certain conditions. (c) 2018 Elsevier Ltd. All rights reserved.

学術雑誌論文

Accepted Manuscript

Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

©2018. This manuscript version is made available under the CC-BY-NC-ND 4.0 license https://creativecommons.org/licenses/by-nc-nd/4.0/

0893-6080