Learning Spatiotemporal Features with 3D Convolutional Networks

Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, Manohar Paluri
12/2/2014
cs.CV

Official code: facebookarchive/C3D (1,182 stars, last commit 2019-07-31).

Abstract

We propose a simple, yet effective approach for spatiotemporal feature learning using deep 3-dimensional convolutional networks (3D ConvNets) trained on a large scale supervised video dataset. Our findings are three-fold: 1) 3D ConvNets are more suitable for spatiotemporal feature learning compared to 2D ConvNets; 2) A homogeneous architecture with small 3x3x3 convolution kernels in all layers is among the best performing architectures for 3D ConvNets; and 3) Our learned features, namely C3D (Convolutional 3D), with a simple linear classifier outperform state-of-the-art methods on 4 different benchmarks and are comparable with current best methods on the other 2 benchmarks. In addition, the features are compact: achieving 52.8% accuracy on UCF101 dataset with only 10 dimensions and also very efficient to compute due to the fast inference of ConvNets. Finally, they are conceptually very simple and easy to train and use.

View on arXivView PDF

Code Implementations(1)

facebookarchive/C3DOfficial100%

C3D is a modified version of BVLC caffe to support 3D ConvNets.

1,182498C++, CSSApr 10, 20157 years agoNOASSERTION

Similar Papers

Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?

86% match1 repo4.0k2017

TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition

81% match1 repo4402017

Spatiotemporal Modeling Encounters 3D Medical Image Analysis: Slice-Shift UNet with Multi-View Fusion

81% match2023

4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks

81% match1 repo3.0k2019

A Decade of Deep Learning for Remote Sensing Spatiotemporal Fusion: Advances, Challenges, and Opportunities

80% match1 repo702025

Hierarchical Interactive Reconstruction Network For Video Compressive Sensing

80% match2023

Cite this paper

@article{tran2014learning,
  title  = {Learning Spatiotemporal Features with 3D Convolutional Networks},
  author = {Du Tran and Lubomir Bourdev and Rob Fergus and Lorenzo Torresani and Manohar Paluri},
  year   = {2014},
  eprint = {1412.0767},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1412.0767}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1412.0767.svg)](https://codeofpaper.com/paper/1412.0767)

Discussion