Hokkaido University Collection of Scholarly and Academic Papers >
Graduate School of Information Science and Technology / Faculty of Information Science and Technology >
Peer-reviewed Journal Articles, etc >
Holmes : A Hardware-Oriented Optimizer Using Logarithms
Title: | Holmes : A Hardware-Oriented Optimizer Using Logarithms |
Authors: | Yamagishi, Yoshiharu Browse this author | Kaneko, Tatsuya Browse this author | Akai-Kasaya, Megumi Browse this author →KAKEN DB | Asai, Tetsuya Browse this author →KAKEN DB |
Keywords: | optimizer | edge computing | neural network | nonvolatile mem-ory | quantization |
Issue Date: | Dec-2022 |
Publisher: | IEICE - Institute of the Electronics, Information and Communication Engineers |
Journal Title: | IEICE transactions on information and systems |
Volume: | E105D |
Issue: | 12 |
Start Page: | 2040 |
End Page: | 2047 |
Publisher DOI: | 10.1587/transinf.2022PAP0001 |
Abstract: | Edge computing, which has been gaining attention in re-cent years, has many advantages, such as reducing the load on the cloud, not being affected by the communication environment, and providing excellent security. Therefore, many researchers have attempted to implement neural networks, which are representative of machine learning in edge computing. Neural networks can be divided into inference and learning parts; however, there has been little research on implementing the learning component in edge computing in contrast to the inference part. This is because learning requires more memory and computation than inference, easily exceeding the limit of resources available for edge computing. To overcome this prob-lem, this research focuses on the optimizer, which is the heart of learning. In this paper, we introduce our new optimizer, hardware-oriented logarith-mic momentum estimation (Holmes), which incorporates new perspectives not found in existing optimizers in terms of characteristics and strengths of hardware. The performance of Holmes was evaluated by comparing it with other optimizers with respect to learning progress and convergence speed. Important aspects of hardware implementation, such as memory and oper-ation requirements are also discussed. The results show that Holmes is a good match for edge computing with relatively low resource requirements and fast learning convergence. Holmes will help create an era in which advanced machine learning can be realized on edge computing. |
Rights: | Copyright ©2022 The Institute of Electronics, Information and Communication Engineers | https://search.ieice.org/ |
Type: | article (author version) |
URI: | http://hdl.handle.net/2115/87632 |
Appears in Collections: | 情報科学院・情報科学研究院 (Graduate School of Information Science and Technology / Faculty of Information Science and Technology) > 雑誌発表論文等 (Peer-reviewed Journal Articles, etc)
|
Submitter: 浅井 哲也
|