Efficient ResNets: Residual Network Design

Aditya Thakur, Harish Chauhan, Nikunj Gupta
6/21/2023
cs.CVcs.LG

Official code: Nikunj-Gupta/Efficient_ResNets (27 stars, MIT, PyTorch, last commit 2024-07-23).

Abstract

ResNets (or Residual Networks) are one of the most commonly used models for image classification tasks. In this project, we design and train a modified ResNet model for CIFAR-10 image classification. In particular, we aimed at maximizing the test accuracy on the CIFAR-10 benchmark while keeping the size of our ResNet model under the specified fixed budget of 5 million trainable parameters. Model size, typically measured as the number of trainable parameters, is important when models need to be stored on devices with limited storage capacity (e.g. IoT/edge devices). In this article, we present our residual network design which has less than 5 million parameters. We show that our ResNet achieves a test accuracy of 96.04% on CIFAR-10 which is much higher than ResNet18 (which has greater than 11 million trainable parameters) when equipped with a number of training strategies and suitable ResNet hyperparameters. Models and code are available at https://github.com/Nikunj-Gupta/Efficient_ResNets.

View on arXivView PDF

Code Implementations(1)

Nikunj-Gupta/Efficient_ResNetsOfficial100%
PyTorchMIT

A Residual Network Design with less than 5 million trainable parameters achieving an accuracy of 96.04% on CIFAR-10.

272Shell, PythonMar 4, 20222 years agoMIT
cnndeep-learningoptimizationpytorchresnet

Similar Papers

RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network

85% match2023

Improving Resnet-9 Generalization Trained on Small Datasets

83% match2023

AdaResNet: Enhancing Residual Networks with Dynamic Weight Adjustment for Improved Feature Integration

80% match1 repo22024

Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit

80% match2026

Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition

79% match2023

Efficiera Residual Networks: Hardware-Friendly Fully Binary Weight with 2-bit Activation Model Achieves Practical ImageNet Accuracy

79% match1 repo12024

Cite this paper

@article{thakur2023efficient,
  title  = {Efficient ResNets: Residual Network Design},
  author = {Aditya Thakur and Harish Chauhan and Nikunj Gupta},
  year   = {2023},
  eprint = {2306.12100},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.12100v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.12100.svg)](https://codeofpaper.com/paper/2306.12100)

Discussion