閲覧可能ファイル 

ファイル	フォーマット	サイズ	閲覧回数	説明
Tomoumi Takase	pdf	2.36 MB	3,714

論文情報

ファイル出力 EndNote Basic出力 Mendeley出力

アクセス権	open access
URI	https://hdl.handle.net/2115/71558
タイトル	Why Does Large Batch Training Result in Poor Generalization? A Comprehensive Explanation and a Better Strategy from the Viewpoint of Stochastic Optimization
著者	Takase, Tomoumi ; Oyama, Satoshi ; Kurihara, Masahito
著者名 Takase, Tomoumi
e-Rad_Researcher 30346100 著者名 Oyama, Satoshi
e-Rad_Researcher 50133707 著者名 Kurihara, Masahito
言語	英語
キーワード	Non-convex optimization
	Gradient descent
	Neural network
	Batch training
	Randomized algorithm
発行日	2018-07
出版者	MIT
収録物名	Neural Computation
巻(号)	30(7)
ページ	2005-2023
抄録	We present a comprehensive framework of search methods, such as simulated annealing and batch training, for solving non-convex optimization problems. These methods search a wider range by gradually decreasing the randomness added to the standard gradient descent method. The formulation that we define on the basis of this framework can be directly applied to neural network training. This produces an effective approach that gradually increases batch size during training. We also explain why large batch training degrades generalization performance, which was not clarified in previous studies.
資源タイプ	学術雑誌論文
出版タイプ	Version of Record
権利情報	© 2018 Massachusetts Institute of Technology
PISSN	0899-7667
EISSN	1530-888X
関連情報	PMID 29652590
	URI https://www.mitpressjournals.org/loi/neco
関連情報 (isIdenticalTo)	出版社版DOI https://doi.org/10.1162/neco_a_01089

ホームへ戻る