RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability

Vishwesh Sangarya, Jung-Eun Kim
12/20/2024
cs.LGcs.AIcs.CV

Official code: JEKimLab/AAAI2025RESQUE (0 stars, MIT, last commit 2025-01-03).

Abstract

As a strategy for sustainability of deep learning, reusing an existing model by retraining it rather than training a new model from scratch is critical. In this paper, we propose REpresentation Shift QUantifying Estimator (RESQUE), a predictive quantifier to estimate the retraining cost of a model to distributional shifts or change of tasks. It provides a single concise index for an estimate of resources required for retraining the model. Through extensive experiments, we show that RESQUE has a strong correlation with various retraining measures. Our results validate that RESQUE is an effective indicator in terms of epochs, gradient norms, changes of parameter magnitude, energy, and carbon emissions. These measures align well with RESQUE for new tasks, multiple noise types, and varying noise intensities. As a result, RESQUE enables users to make informed decisions for retraining to different tasks/distribution shifts and determine the most cost-effective and sustainable option, allowing for the reuse of a model with a much smaller footprint in the environment. The code for this work is available here: https://github.com/JEKimLab/AAAI2025RESQUE

View on arXivView PDF

Code Implementations(1)

JEKimLab/AAAI2025RESQUEOfficial100%
MIT

This is the official implementation of “RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability” published in AAAI Conference on Artificial Intelligence 2025

01PythonDec 15, 20241 years agoMIT

Similar Papers

Aggregate Representation Measure for Predictive Model Reusability

88% match2024

Estimating Environmental Cost Throughout Model's Adaptive Life Cycle

85% match1 repo2024

Cost-Effective Retraining of Machine Learning Models

84% match2023

Sustainable Machine Learning Retraining: Optimizing Energy Efficiency Without Compromising Accuracy

84% match1 repo12025

Minimizing Energy Costs in Deep Learning Model Training: The Gaussian Sampling Approach

83% match2024

Accuracy is not the only Metric that matters: Estimating the Energy Consumption of Deep Learning Models

83% match1 repo22023

Cite this paper

@article{sangarya2024resque,
  title  = {RESQUE: Quantifying Estimator to Task and Distribution Shift for Sustainable Model Reusability},
  author = {Vishwesh Sangarya and Jung-Eun Kim},
  year   = {2024},
  eprint = {2412.15511},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.15511v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.15511.svg)](https://codeofpaper.com/paper/2412.15511)

Discussion