Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | 서지원 | - |
dc.date.accessioned | 2022-03-21T07:19:56Z | - |
dc.date.available | 2022-03-21T07:19:56Z | - |
dc.date.issued | 2020-07 | - |
dc.identifier.citation | 2020 57th ACM/IEEE Design Automation Conference (DAC), page. 1-6 | en_US |
dc.identifier.isbn | 978-1-7281-1085-1 | - |
dc.identifier.issn | 0738-100X | - |
dc.identifier.uri | https://ieeexplore.ieee.org/document/9218518 | - |
dc.identifier.uri | https://repository.hanyang.ac.kr/handle/20.500.11754/169267 | - |
dc.description.abstract | Training a deep neural network (DNN) is expensive, requiring a large amount of computation time. While the training overhead is high, not all computation in DNN training is equal. Some parameters converge faster and thus their gradient computation may contribute little to the parameter update; in nearstationary points a subset of parameters may change very little. In this paper we exploit the parameter convergence to optimize gradient computation in DNN training. We design a light-weight monitoring technique to track the parameter convergence; we prune the gradient computation stochastically for a group of semantically related parameters, exploiting their convergence correlations. These techniques are efficiently implemented in existing GPU kernels. In our evaluation the optimization techniques substantially and robustly improve the training throughput for four DNN models on three public datasets. | en_US |
dc.description.sponsorship | This work is supported by Samsung Research, Samsung Electronics Co., LTd, by the National Research Foundation of Korea (NRF) grant (No. 2018R1D1A1B07050609), and by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No.2013-0-00109, WiseKB: Big data based self-evolving knowledge base and reasoning platform). We thank Jinwon Lee for the preliminary experiments. The corresponding authors are Jiwon Seo and Yongjun Park. | en_US |
dc.language.iso | en | en_US |
dc.publisher | IEEE | en_US |
dc.subject | Convergence | en_US |
dc.subject | Training | en_US |
dc.subject | Neurons | en_US |
dc.subject | Correlation | en_US |
dc.subject | Monitoring | en_US |
dc.subject | Kernel | en_US |
dc.subject | History | en_US |
dc.title | Convergence-Aware Neural Network Training | en_US |
dc.type | Article | en_US |
dc.identifier.doi | 10.1109/DAC18072.2020.9218518 | - |
dc.relation.page | 1-6 | - |
dc.contributor.googleauthor | Oh, Hyungjun | - |
dc.contributor.googleauthor | Yu, Yongseung | - |
dc.contributor.googleauthor | Ryu, Giha | - |
dc.contributor.googleauthor | Ahn, Gunjoo | - |
dc.contributor.googleauthor | Jeong, Yuri | - |
dc.contributor.googleauthor | Park, Yongjun | - |
dc.contributor.googleauthor | Seo, Jiwon | - |
dc.relation.code | 20200058 | - |
dc.sector.campus | S | - |
dc.sector.daehak | COLLEGE OF ENGINEERING[S] | - |
dc.sector.department | SCHOOL OF COMPUTER SCIENCE | - |
dc.identifier.pid | seojiwon | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.