AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution

Abstract

Although image super-resolution (SR) problem has ex-perienced unprecedented restoration accuracy with deep neural networks, it has yet limited versatile applications due to the substantial computational costs. Since differ-ent input images for SR face different restoration difficul-ties, adapting computational costs based on the input image, referred to as adaptive inference, has emerged as a promising solution to compress SR networks. Specifically, adapting the quantization bit-widths has successfully re-duced the inference and memory cost without sacrificing the accuracy. However, despite the benefits of the resul-tant adaptive network, existing works rely on time-intensive quantization-aware training with full access to the origi-nal training pairs to learn the appropriate bit allocation policies, which limits its ubiquitous usage. To this end, we introduce the first on-the-fly adaptive quantization frame-work that accelerates the processing time from hours to sec-onds. We formulate the bit allocation problem with only two bit mapping modules: one to map the input image to the image-wise bit adaptation factor and one to obtain the layer-wise adaptation factors. These bit mappings are cali-brated and fine-tuned using only a small number of calibration images. We achieve competitive performance with the previous adaptive quantization methods, while the processing time is accelerated by × 2000. Codes are available at https://github.com/Cheeun/AdaBM.