Knowledge Distillation for Anomaly Detection

Adrian Alan Pol, Ekaterina Govorkova, Sonja Gronroos, Nadezda Chernyavskaya, Philip Harris +3 more
10/9/2023
cs.LG

No official code found. Best implementation: yuxin-jiang/MRKD (47 stars, last commit 2025-12-23).

Abstract

Unsupervised deep learning techniques are widely used to identify anomalous behaviour. The performance of such methods is a product of the amount of training data and the model size. However, the size is often a limiting factor for the deployment on resource-constrained devices. We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity. Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.

View on arXivView PDF

Code Implementations(1)

yuxin-jiang/MRKDCommunity75%

This is an official implementation of “A Masked Reverse Knowledge Distillation Method Incorporating Global and Local Information for Image Anomaly Detection” (MRKD) with PyTorch, accepted by knowledge-based systems.

470PythonDec 24, 202310 months ago
anomaly-detectionknowledge-distillationmasked-image-modeling

Similar Papers

Large Language Model Guided Knowledge Distillation for Time Series Anomaly Detection

84% match2024

Scale-Aware Contrastive Reverse Distillation for Unsupervised Medical Anomaly Detection

83% match1 repo72026

Deep Learning for Network Anomaly Detection under Data Contamination: Evaluating Robustness and Mitigating Performance Degradation

82% match2024

Representation Learning in Anomaly Detection: Successes, Limits and a Grand Challenge

82% match2023

Adaptive Anomaly Detection in Evolving Network Environments

82% match2025

Deep Learning Advancements in Anomaly Detection: A Comprehensive Survey

82% match2025

Cite this paper

@article{pol2023knowledge,
  title  = {Knowledge Distillation for Anomaly Detection},
  author = {Adrian Alan Pol and Ekaterina Govorkova and Sonja Gronroos and Nadezda Chernyavskaya and Philip Harris and Maurizio Pierini and Isobel Ojalvo and Peter Elmer},
  year   = {2023},
  eprint = {2310.06047},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.06047v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.06047.svg)](https://codeofpaper.com/paper/2310.06047)

Discussion