A Performance Evaluation of a Quantized Large Language Model on Various Smartphones

Tolga Çöplü, Marc Loedi, Arto Bendiken, Mykhailo Makohin, Joshua J. Bouw +1 more
12/19/2023
cs.LGcs.AIcs.PF

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

This paper explores the feasibility and performance of on-device large language model (LLM) inference on various Apple iPhone models. Amidst the rapid evolution of generative AI, on-device LLMs offer solutions to privacy, security, and connectivity challenges inherent in cloud-based models. Leveraging existing literature on running multi-billion parameter LLMs on resource-limited devices, our study examines the thermal effects and interaction speeds of a high-performing LLM across different smartphone generations. We present real-world performance results, providing insights into on-device inference capabilities.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

MobileMoE: Scaling On-Device Mixture of Experts

89% match2026

On-Device Language Models: A Comprehensive Review

87% match2024

Revolutionizing Mobile Interaction: Enabling a 3 Billion Parameter GPT LLM on Mobile

87% match2023

AIvaluateXR: An Evaluation Framework for on-Device AI in XR with Benchmarking Results

86% match2025

LinguaLinked: A Distributed Large Language Model Inference System for Mobile Devices

86% match2023

Understanding Large Language Models in Your Pockets: Performance Study on COTS Mobile Devices

86% match2024

Cite this paper

@article{pl2023performance,
  title  = {A Performance Evaluation of a Quantized Large Language Model on Various Smartphones},
  author = {Tolga Çöplü and Marc Loedi and Arto Bendiken and Mykhailo Makohin and Joshua J. Bouw and Stephen Cobb},
  year   = {2023},
  eprint = {2312.12472},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.12472v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.12472.svg)](https://codeofpaper.com/paper/2312.12472)

Discussion