Improving Resnet-9 Generalization Trained on Small Datasets

Omar Mohamed Awad, Habib Hajimolahoseini, Michael Lim, Gurpreet Gosal, Walid Ahmed +2 more
9/7/2023
cs.LGcs.CLcs.CV

No confident code implementation found for this paper. 4 possible matches can be shown below.

Abstract

This paper presents our proposed approach that won the first prize at the ICLR competition on Hardware Aware Efficient Training. The challenge is to achieve the highest possible accuracy in an image classification task in less than 10 minutes. The training is done on a small dataset of 5000 images picked randomly from CIFAR-10 dataset. The evaluation is performed by the competition organizers on a secret dataset with 1000 images of the same size. Our approach includes applying a series of technique for improving the generalization of ResNet-9 including: sharpness aware optimization, label smoothing, gradient centralization, input patch whitening as well as metalearning based training. Our experiments show that the ResNet-9 can achieve the accuracy of 88% while trained only on a 10% subset of CIFAR-10 dataset in less than 10 minuets

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 4 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Efficient ResNets: Residual Network Design

83% match1 repo272023

Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)

79% match2023

Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization

78% match2024

RRR-Net: Reusing, Reducing, and Recycling a Deep Backbone Network

78% match2023

Toward Errorless Training ImageNet-1k

78% match2025

ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks

77% match2024

Cite this paper

@article{awad2023improving,
  title  = {Improving Resnet-9 Generalization Trained on Small Datasets},
  author = {Omar Mohamed Awad and Habib Hajimolahoseini and Michael Lim and Gurpreet Gosal and Walid Ahmed and Yang Liu and Gordon Deng},
  year   = {2023},
  eprint = {2309.03965},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.03965v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.03965.svg)](https://codeofpaper.com/paper/2309.03965)

Discussion