Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation

Xin Chen, Lingxi Xie, Jun Wu, Qi Tian
4/29/2019
cs.CVcs.LG

Official code: chenxin061/pdarts (364 stars, last commit 2020-02-18).

Abstract

Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time (~7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdarts.

View on arXivView PDF

Code Implementations(1)

chenxin061/pdartsOfficial100%

Codes for our paper "Progressive Differentiable Architecture Search:Bridging the Depth Gap between Search and Evaluation"

36485PythonApr 29, 20196 years agoNOASSERTION

Similar Papers

Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

90% match1 repo3642019

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

90% match1 repo1.4k2018

FTSO: Effective NAS via First Topology Second Operator

88% match2023

Searching for A Robust Neural Architecture in Four GPU Hours

88% match1 repo1.6k2019

NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size

87% match1 repo1.6k2020

sharpDARTS: Faster and More Accurate Differentiable Architecture Search

87% match1 repo152019

Cite this paper

@article{chen2019progressive,
  title  = {Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation},
  author = {Xin Chen and Lingxi Xie and Jun Wu and Qi Tian},
  year   = {2019},
  eprint = {1904.12760},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1904.12760}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1904.12760.svg)](https://codeofpaper.com/paper/1904.12760)

Discussion