Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions

Amine Barrak, Emna Ksontini
1/30/2025
cs.DCcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

As data-intensive applications grow, batch processing in limited-resource environments faces scalability and resource management challenges. Serverless computing offers a flexible alternative, enabling dynamic resource allocation and automatic scaling. This paper explores how serverless architectures can make large-scale ML inference tasks faster and cost-effective by decomposing monolithic processes into parallel functions. Through a case study on sentiment analysis using the DistilBERT model and the IMDb dataset, we demonstrate that serverless parallel processing can reduce execution time by over 95% compared to monolithic approaches, at the same cost.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

FSD-Inference: Fully Serverless Distributed Inference with Scalable Cloud Communication

88% match2024

Optimizing Distributed Deployment of Mixture-of-Experts Model Inference in Serverless Computing

81% match2025

ServerlessLLM: Low-Latency Serverless Inference for Large Language Models

81% match1 repo7112024

Efficient Multi-stage Inference on Tabular Data

81% match2023

A Survey of Serverless Machine Learning Model Inference

81% match2023

Inference Acceleration for Large Language Models on CPUs

78% match2024

Cite this paper

@article{barrak2025scalable,
  title  = {Scalable and Cost-Efficient ML Inference: Parallel Batch Processing with Serverless Functions},
  author = {Amine Barrak and Emna Ksontini},
  year   = {2025},
  eprint = {2502.12017},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.12017v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.12017.svg)](https://codeofpaper.com/paper/2502.12017)

Discussion