Compiler generated feedback for Large Language Models

Dejan Grubisic, Chris Cummins, Volker Seeker, Hugh Leather
3/18/2024
cs.PLcs.LG

No confident code implementation found for this paper. 4 possible matches can be shown below.

Abstract

We introduce a novel paradigm in compiler optimization powered by Large Language Models with compiler feedback to optimize the code size of LLVM assembly. The model takes unoptimized LLVM IR as input and produces optimized IR, the best optimization passes, and instruction counts of both unoptimized and optimized IRs. Then we compile the input with generated optimization passes and evaluate if the predicted instruction count is correct, generated IR is compilable, and corresponds to compiled code. We provide this feedback back to LLM and give it another chance to optimize code. This approach adds an extra 0.53% improvement over -Oz to the original model. Even though, adding more information with feedback seems intuitive, simple sampling techniques achieve much higher performance given 10 or more samples.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 4 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Large Language Models for Compiler Optimization

91% match2023

Meta Large Language Model Compiler: Foundation Models of Compiler Optimization

85% match2024

Towards Optimizing with Large Language Models

85% match1 repo7612023

Large Language Models for Tuning Evolution Strategies

84% match2024

LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generation

84% match2024

Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Models

84% match2024

Cite this paper

@article{grubisic2024compiler,
  title  = {Compiler generated feedback for Large Language Models},
  author = {Dejan Grubisic and Chris Cummins and Volker Seeker and Hugh Leather},
  year   = {2024},
  eprint = {2403.14714},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2403.14714v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2403.14714.svg)](https://codeofpaper.com/paper/2403.14714)

Discussion