Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?

Nirjhor Rouf, Fin Amin, Paul D. Franzon
6/19/2024
cs.LG

No official code found. Best implementation: FinAminToastCrunch/LoRA-KD-Microelectronic-Reasoning (1 star, last commit 2025-07-11).

Abstract

In this work, we present empirical results regarding the feasibility of using offline large language models (LLMs) in the context of electronic design automation (EDA). The goal is to investigate and evaluate a contemporary language model's (Llama-2-7B) ability to function as a microelectronic Q & A expert as well as its reasoning, and generation capabilities in solving microelectronic-related problems. Llama-2-7B was tested across a variety of adaptation methods, including introducing a novel low-rank knowledge distillation (LoRA-KD) scheme. Our experiments produce both qualitative and quantitative results.

View on arXivView PDF

Code Implementations(1)

FinAminToastCrunch/LoRA-KD-Microelectronic-ReasoningCommunity67%

The official repository of "Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?" Published in LLM-Aided Design (LAD '24)

10Jun 27, 20241 years ago

Similar Papers

Large Language Models (LLMs) for Electronic Design Automation (EDA)

85% match2025

A Survey of Research in Large Language Models for Electronic Design Automation

84% match2025

LLMs for Analog Circuit Design Continuum (ACDC)

84% match2025

White-Box Reasoning: Synergizing LLM Strategy and gm/Id Data for Automated Analog Circuit Design

82% match2025

CIRCUIT: A Benchmark for Circuit Interpretation and Reasoning Capabilities of LLMs

82% match2025

Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillation

82% match2024

Cite this paper

@article{rouf2024lowrank,
  title  = {Can Low-Rank Knowledge Distillation in LLMs be Useful for Microelectronic Reasoning?},
  author = {Nirjhor Rouf and Fin Amin and Paul D. Franzon},
  year   = {2024},
  eprint = {2406.13808},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.13808v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.13808.svg)](https://codeofpaper.com/paper/2406.13808)

Discussion