Cascade: A Platform for Delay-Sensitive Edge Intelligence

Weijia Song, Thiago Garrett, Yuting Yang, Mingzhao Liu, Edward Tremel +4 more
11/29/2023
cs.OScs.AI

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Interactive intelligent computing applications are increasingly prevalent, creating a need for AI/ML platforms optimized to reduce per-event latency while maintaining high throughput and efficient resource management. Yet many intelligent applications run on AI/ML platforms that optimize for high throughput even at the cost of high tail-latency. Cascade is a new AI/ML hosting platform intended to untangle this puzzle. Innovations include a legacy-friendly storage layer that moves data with minimal copying and a "fast path" that collocates data and computation to maximize responsiveness. Our evaluation shows that Cascade reduces latency by orders of magnitude with no loss of throughput.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Intelligent Task Offloading: Advanced MEC Task Offloading and Resource Management in 5G Networks

82% match2025

Idle is the New Sleep: Configuration-Aware Alternative to Powering Off FPGA-Based DL Accelerators During Inactivity

78% match2024

Adaptive Serverless Resource Management via Slot-Survival Prediction and Event-Driven Lifecycle Control

78% match2026

SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads

78% match2023

Synergy: Towards On-Body AI via Tiny AI Accelerator Collaboration on Wearables

77% match2023

HyperFlexis: Joint Design of Algorithms and Systems for Multi-SLO Serving and Fast Scaling

77% match2025

Cite this paper

@article{song2023cascade,
  title  = {Cascade: A Platform for Delay-Sensitive Edge Intelligence},
  author = {Weijia Song and Thiago Garrett and Yuting Yang and Mingzhao Liu and Edward Tremel and Lorenzo Rosa and Andrea Merlina and Roman Vitenberg and Ken Birman},
  year   = {2023},
  eprint = {2311.17329},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.17329v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.17329.svg)](https://codeofpaper.com/paper/2311.17329)

Discussion