|
Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Adapting the Learning Rate of the Learning Rate in Hypergradient Descent
| Title: | Adapting the Learning Rate of the Learning Rate in Hypergradient Descent |
| Authors: | Itakura, Kazuma Browse this author | | Atarashi, Kyohei Browse this author | | Oyama, Satoshi Browse this author →KAKEN DB | | Kurihara, Masahito Browse this author →KAKEN DB |
| Keywords: | optimization | | hypergradient descent | | adjusting learning rate |
| Issue Date: | Dec-2020 |
| Publisher: | IEEE |
| Journal Title: | 2020 Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems (SCIS-ISIS) |
| Start Page: | 1 |
| End Page: | 6 |
| Publisher DOI: | 10.1109/SCISISIS50064.2020.9322765 |
| Abstract: | Gradient descent is a widely used optimization method. The adjustment of the learning rate is an important factor in improving its performance, and many researchers have investigated methods for automatically adjusting the learning rate. One such method, hypergradient descent, automatically adjusts the learning rate by using gradient descent. However, it introduces the “learning rate of the learning rate,” and an appropriate value for the learning rate of the learning rate must be chosen in order to effectively adjust the learning rate. We investigated the use of two datasets and two optimization methods for doing this and achieved an effective adjustment of the learning rate when the objective function was convex and L -smooth. |
| Conference Name: | Joint 11th International Conference on Soft Computing and Intelligent Systems and 21st International Symposium on Advanced Intelligent Systems |
| Conference Sequence: | 2020 |
| Conference Place: | Hachijo island, Tokyo |
| Rights: | © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. |
| Type: | proceedings |
| URI: | http://hdl.handle.net/2115/80339 |
| Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
Submitter: 小山 聡
|
