Bauplan: zero-copy, scale-up FaaS for data pipelines

Jacopo Tagliabue, Tyler Caraza-Harter, Ciro Greco
10/22/2024
cs.DBcs.LGcs.OS

No official code found. Best implementation: BauplanLabs/WoSC-2024 (1 star, last commit 2025-12-30).

Abstract

Chaining functions for longer workloads is a key use case for FaaS platforms in data applications. However, modern data pipelines differ significantly from typical serverless use cases (e.g., webhooks and microservices); this makes it difficult to retrofit existing pipeline frameworks due to structural constraints. In this paper, we describe these limitations in detail and introduce bauplan, a novel FaaS programming model and serverless runtime designed for data practitioners. bauplan enables users to declaratively define functional Directed Acyclic Graphs (DAGs) along with their runtime environments, which are then efficiently executed on cloud-based workers. We show that bauplan achieves both better performance and a superior developer experience for data workloads by making the trade-off of reducing generality in favor of data-awareness

View on arXivView PDF

Code Implementations(1)

BauplanLabs/WoSC-2024Community100%
10Python, DockerfileDec 29, 20239 months ago

Similar Papers

A Framework for SLO, Carbon, and Wastewater-Aware Sustainable FaaS Cloud Platform Management

73% match2024

In Search of a Fast and Efficient Serverless DAG Engine

72% match1 repo782019

Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces

71% match1 repo382023

A lightweight design for serverless Function-as-a-Service

71% match2 repos10.8k2020

Word Frequency Counting Based on Serverless MapReduce

71% match2026

High-Performance Serverless Computing: A Systematic Literature Review on Serverless for HPC, AI, and Big Data

70% match1 repo22026

Cite this paper

@article{tagliabue2024bauplan,
  title  = {Bauplan: zero-copy, scale-up FaaS for data pipelines},
  author = {Jacopo Tagliabue and Tyler Caraza-Harter and Ciro Greco},
  year   = {2024},
  eprint = {2410.17465},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.17465v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.17465.svg)](https://codeofpaper.com/paper/2410.17465)

Discussion