Identity Mappings in Deep Residual Networks

Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
3/16/2016
cs.CVcs.LG

Official code: KaimingHe/resnet-1k-layers (938 stars, last commit 2017-05-24).

Abstract

Deep residual networks have emerged as a family of extremely deep architectures showing compelling accuracy and nice convergence behaviors. In this paper, we analyze the propagation formulations behind the residual building blocks, which suggest that the forward and backward signals can be directly propagated from one block to any other block, when using identity mappings as the skip connections and after-addition activation. A series of ablation experiments support the importance of these identity mappings. This motivates us to propose a new residual unit, which makes training easier and improves generalization. We report improved results using a 1001-layer ResNet on CIFAR-10 (4.62% error) and CIFAR-100, and a 200-layer ResNet on ImageNet. Code is available at: https://github.com/KaimingHe/resnet-1k-layers

View on arXivView PDF

Code Implementations(1)

KaimingHe/resnet-1k-layersOfficial100%

Deep Residual Networks with 1K Layers

938251LuaApr 5, 20169 years ago

Cite this paper

@article{he2016identity,
  title  = {Identity Mappings in Deep Residual Networks},
  author = {Kaiming He and Xiangyu Zhang and Shaoqing Ren and Jian Sun},
  year   = {2016},
  eprint = {1603.05027},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1603.05027}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1603.05027.svg)](https://codeofpaper.com/paper/1603.05027)

Discussion