GeckOpt: LLM System Efficiency via Intent-Based Tool Selection

Michael Fore, Simranjit Singh, Dimitrios Stamoulis
4/24/2024
cs.LGcs.AI

No confident code implementation found for this paper.

Abstract

In this preliminary study, we investigate a GPT-driven intent-based reasoning approach to streamline tool selection for large language models (LLMs) aimed at system efficiency. By identifying the intent behind user prompts at runtime, we narrow down the API toolset required for task execution, reducing token consumption by up to 24.6\%. Early results on a real-world, massively parallel Copilot platform with over 100 GPT-4-Turbo nodes show cost reductions and potential towards improving LLM-based system efficiency.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

iServe: An Intent-based Serving System for LLMs

84% match2025

ToolTalk: Evaluating Tool-Usage in a Conversational Setting

83% match1 repo892023

Large Language Models as Tool Makers

83% match1 repo1.1k2023

Look Before You Leap: Towards Decision-Aware and Generalizable Tool-Usage for Large Language Models

82% match2024

Balancing Accuracy and Efficiency in Multi-Turn Intent Classification for LLM-Powered Dialog Systems in Production

82% match2024

Efficient Intent-Based Filtering for Multi-Party Conversations Using Knowledge Distillation from LLMs

82% match2025

Cite this paper

@article{fore2024geckopt,
  title  = {GeckOpt: LLM System Efficiency via Intent-Based Tool Selection},
  author = {Michael Fore and Simranjit Singh and Dimitrios Stamoulis},
  year   = {2024},
  eprint = {2404.15804},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.15804v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.15804.svg)](https://codeofpaper.com/paper/2404.15804)

Discussion