AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization

Genghan Zhang, Shaowei Zhu, Anjiang Wei, Zhenyu Song, Allen Nie +4 more
11/19/2025
cs.LGcs.CL

No confident code implementation found for this paper.

Abstract

We present AccelOpt, a self-improving large language model (LLM) agentic system that autonomously optimizes kernels for emerging AI acclerators, eliminating the need for expert-provided hardware-specific optimization knowledge. AccelOpt explores the kernel optimization space through iterative generation, informed by an optimization memory that curates experiences and insights from previously encountered slow-fast kernel pairs. We build NKIBench, a new benchmark suite of AWS Trainium accelerator kernels with varying complexity extracted from real-world LLM workloads to evaluate the effectiveness of AccelOpt. Our evaluation confirms that AccelOpt's capability improves over time, boosting the average percentage of peak throughput from $49\%$ to $61\%$ on Trainium 1 and from $45\%$ to $59\%$ on Trainium 2 for NKIBench kernels. Moreover, AccelOpt is highly cost-effective: using open-source models, it matches the kernel improvements of Claude Sonnet 4 while being $26\times$ cheaper.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Towards Automated Kernel Generation in the Era of LLMs

87% match1 repo2442026

FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems

84% match2 repos2512026

CuTeGen: An LLM-Based Agentic Framework for Generation and Optimization of High-Performance GPU Kernels using CuTe

84% match2026

Autocomp: A Powerful and Portable Code Optimizer for Tensor Accelerators

83% match2025

SwizzlePerf: Hardware-Aware LLMs for GPU Kernel Performance Optimization

83% match2025

Large Language Model Agent for Hyper-Parameter Optimization

82% match1 repo322024

Cite this paper

@article{zhang2025accelopt,
  title  = {AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization},
  author = {Genghan Zhang and Shaowei Zhu and Anjiang Wei and Zhenyu Song and Allen Nie and Zhen Jia and Nandita Vijaykumar and Yida Wang and Kunle Olukotun},
  year   = {2025},
  eprint = {2511.15915},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2511.15915v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2511.15915.svg)](https://codeofpaper.com/paper/2511.15915)

Discussion