DEEPTOOLS: Compiler and Execution Runtime Extensions for RAPiD AI Accelerator

Abstract

The ubiquitous adoption of systems specialized for AI requires bridging two seemingly conflicting challenges-the need to deliver extreme processing efficiencies while employing familiar programming interfaces, making them compelling even for nonexpert users. We take a significant first step towards this goal and present an end-to-end software stack for the RAPID AI accelerator developed by IBM Research. We present a set of software extensions, called DEEPTOOLS, that leverage and work within popular deep learning frameworks. DEEPTOOLS requires no additional user input and enables aggressive, accelerator-specific performance optimization akin to a full, custom framework. DEEPTOOLS has two key components: 1) a compiler runtime called DeepRT, which automatically identifies how best to execute a given DNN graph on RAPID and constructs the requisite program binaries; and 2) an execution runtime called RAPiDLiB, which triggers and manages the execution of compute and data-transfer operations on RAPID. We integrate DEEPTOOLS with TensorFlow and map popular DNNs (AlexNet, VGG, ResNet, LSTM) to RAPID. We demonstrate substantial improvement in performance over hand-tuned mappings.