DNN compression by ADMM-based joint pruning
- Title
- DNN compression by ADMM-based joint pruning
- Author
- 이기천
- Keywords
- Alternative direction method of multipliers (ADMM); Structured pruning; Neural network compression; Unstructured pruning
- Issue Date
- 2022-03
- Publisher
- Elsevier B.V.
- Citation
- Knowledge-Based Systems, v. 239, article no. 107988, Page. 1-11
- Abstract
- The success of deep neural networks (DNNs) has motivated pursuit of both computationally and memory efficient models for applications in resource-constrained systems such as embedded devices. In line with this trend, network pruning methods reducing redundancy in over-parameterized models are being studied actively. Previous works on this research have demonstrated the ability to learn a compact network by imposing sparsity constraints on the parameters, but most of them have difficulty not only in identifying both connections and neurons to be pruned, but also in converging to optimal solutions. We propose a systematic DNN compression method where weights and network architectures are jointly optimized. We solve the joint problem using alternating direction method of multipliers (ADMM), a powerful technique capable of handling non-convex separable programming. Additionally, we provide a holistic pruning approach, an integrated form of our method, for automatically pruning networks without specific layer-wise hyper-parameters. To verify our work, we deployed the proposed method to a variety of state-of-the-art convolutional neural networks (CNNs) on three image classification benchmark datasets: MNIST, CIFAR-10, and ImageNet. Results show that the proposed pruning method effectively compresses the network parameters and reduces the computation cost while preserving prediction accuracy.
- URI
- https://www.sciencedirect.com/science/article/pii/S0950705121011047?via%3Dihubhttps://repository.hanyang.ac.kr/handle/20.500.11754/178076
- ISSN
- 0950-7051;1872-7409
- DOI
- 10.1016/j.knosys.2021.107988
- Appears in Collections:
- COLLEGE OF ENGINEERING[S](공과대학) > INDUSTRIAL ENGINEERING(산업공학과) > Articles
- Files in This Item:
There are no files associated with this item.
- Export
- RIS (EndNote)
- XLS (Excel)
- XML