Edge Devices Inference Performance Comparison

R. Tobiasz, G. Wilczyński, P. Graszka, N. Czechowski, S. Łuczak
6/21/2023
cs.LGcs.CV

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

In this work, we investigate the inference time of the MobileNet family, EfficientNet V1 and V2 family, VGG models, Resnet family, and InceptionV3 on four edge platforms. Specifically NVIDIA Jetson Nano, Intel Neural Stick, Google Coral USB Dongle, and Google Coral PCIe. Our main contribution is a thorough analysis of the aforementioned models in multiple settings, especially as a function of input size, the presence of the classification head, its size, and the scale of the model. Since throughout the industry, those architectures are mainly utilized as feature extractors we put our main focus on analyzing them as such. We show that Google platforms offer the fastest average inference time, especially for newer models like MobileNet or EfficientNet family, while Intel Neural Stick is the most universal accelerator allowing to run most architectures. These results should provide guidance for engineers in the early stages of AI edge systems development. All of them are accessible at https://bulletprove.com/research/edge_inference_results.csv

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

On the Sustainability of AI Inferences in the Edge

89% match2025

Benchmarking Deep Learning Models for Object Detection on Edge Computing Devices

82% match2024

Spiker+: a framework for the generation of efficient Spiking Neural Networks FPGA accelerators for inference at the edge

82% match2024

Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment

82% match2024

Evolution of Convolutional Neural Network (CNN): Compute vs Memory bandwidth for Edge AI

81% match2023

Channel-Adaptive Edge AI: Maximizing Inference Throughput by Adapting Computational Complexity to Channel States

81% match2026

Cite this paper

@article{tobiasz2023edge,
  title  = {Edge Devices Inference Performance Comparison},
  author = {R. Tobiasz and G. Wilczyński and P. Graszka and N. Czechowski and S. Łuczak},
  year   = {2023},
  eprint = {2306.12093},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.12093v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.12093.svg)](https://codeofpaper.com/paper/2306.12093)

Discussion