경량 딥러닝 모델의 초저정밀도 양자화를 위한 학습 방식의 개선
- Title
- 경량 딥러닝 모델의 초저정밀도 양자화를 위한 학습 방식의 개선
- Other Titles
- Improving training method for very low bit weight quantization of Light Deep Learning Model
- Author
- 최정욱
- Issue Date
- 2020-11
- Publisher
- 대한전자공학회
- Citation
- 2020년도 대한전자공학회 추계학술대회 논문집, Page. 601-604
- Abstract
- Deep Learning Model Quantization is the most effective technique to make a model much lighter and cost efficient in terms of computation.
Above many quantization algorithms, PROFIT[1] is a specialized algorithm for sub 4-bit mobile network quantization. But this method has sudden accuracy degradation in 2-bit width precision.
In this paper, we propose a better training method to deal with this problem in 2-bit weight quantization. We adopt AIWQ, a metric for the activation’s instability induced by weight quantization [1] and make threshold value with this metric. Using threshold value, we stop training some quantized layers which have high sensitivity to weight quantization and fine-tune the rest of the quantized layers with different learning rate and scheduler. With this advanced training method, we improved 2-bit weight quantization accuracy of light deep learning models including EfficientNetB0 and MobilenetV2.
- URI
- https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE10521871https://repository.hanyang.ac.kr/handle/20.500.11754/172399
- Appears in Collections:
- COLLEGE OF ENGINEERING[S](공과대학) > ELECTRONIC ENGINEERING(융합전자공학부) > Articles
- Files in This Item:
There are no files associated with this item.
- Export
- RIS (EndNote)
- XLS (Excel)
- XML