Full metadata record
DC Field | Value | Language |
---|---|---|
dc.contributor.author | 최정욱 | - |
dc.date.accessioned | 2021-03-08T02:45:28Z | - |
dc.date.available | 2021-03-08T02:45:28Z | - |
dc.date.issued | 2019-12 | - |
dc.identifier.citation | Advances in Neural Information Processing Systems 32 (NeurIPS 2019), Page. 1-10 | en_US |
dc.identifier.issn | 1049-5258 | - |
dc.identifier.uri | https://proceedings.neurips.cc/paper/2019/hash/65fc9fb4897a89789352e211ca2d398f-Abstract.html | - |
dc.identifier.uri | https://repository.hanyang.ac.kr/handle/20.500.11754/160272 | - |
dc.description.abstract | Reducing the numerical precision of data and computation is extremely effective in accelerating deep learning training workloads. Towards this end, 8-bit floating point representations (FP8) were recently proposed for DNN training. However, its applicability was only demonstrated on a few selected models and significant degradation is observed when popular networks such as MobileNet and Transformer are trained using FP8. This degradation is due to the inherent precision requirement difference in the forward and backward passes of DNN training. Using theoretical insights, we propose a hybrid FP8 (HFP8) format and DNN end-to-end distributed training procedure. We demonstrate, using HFP8, the successful training of deep learning models across a whole spectrum of applications including Image Classification, Object Detection, Language and Speech without accuracy degradation. Finally, we demonstrate that, by using the new 8 bit format, we can directly quantize a pre-trained model down to 8-bits without losing accuracy by simply fine-tuning batch normalization statistics. These novel techniques enable a new generations of 8-bit hardware that are robust for building and deploying neural network models. | en_US |
dc.description.sponsorship | This research is realized by generous collaborations across IBM Research. | en_US |
dc.language.iso | en | en_US |
dc.publisher | Neural Information Processing Systems Foundation | en_US |
dc.title | Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks | en_US |
dc.type | Article | en_US |
dc.relation.page | 1-10 | - |
dc.contributor.googleauthor | Sun, Xiao | - |
dc.contributor.googleauthor | Choi, Jungwook | - |
dc.contributor.googleauthor | Chen, Chia-Yu | - |
dc.contributor.googleauthor | Wang, Naigang | - |
dc.contributor.googleauthor | Venkataramani, Swagath | - |
dc.contributor.googleauthor | Srinivasan, Vijayalakshmi (Viji) | - |
dc.contributor.googleauthor | Cui, Xiaodong | - |
dc.contributor.googleauthor | Zhang, Wei | - |
dc.contributor.googleauthor | Gopalakrishnan, Kailash | - |
dc.relation.code | 20190015 | - |
dc.sector.campus | S | - |
dc.sector.daehak | COLLEGE OF ENGINEERING[S] | - |
dc.sector.department | DEPARTMENT OF ELECTRONIC ENGINEERING | - |
dc.identifier.pid | choij | - |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.