Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes

Jeremias Ferrao, Niclas Müller-Hof, Iustin Sîrbu, Traian Rebedea, Yftah Ziser
6/25/2026
cs.CL

No official code found. Best implementation: Jazhyc/aims-safety (0 stars, last commit 2026-06-23).

Abstract

We argue that safety classifiers should model user intent as an explicit signal between the prompt and the final label. To study this, we introduce AIMS, a human-annotated dataset of 1,724 difficult safety prompts, each paired with an intent description and harm label. We use AIMS to evaluate intent-aware training across supervised fine-tuning, preference learning, reasoning distillation, and reinforcement learning. Despite its size, AIMS enables competitive safety classifiers across training regimes: DPO from model-generated intent errors improves over SFT, and intent-conditioned distillation outperforms reasoning-only distillation in most teacher-student pairs. Most notably, directly rewarding intent faithfulness with GRPO yields the strongest average performance across five external safety benchmarks, while our intent-aware models form the inference latency-F1 Pareto frontier. These results show that faithful intent modeling is a compact, high-quality supervision signal for more robust safety classifiers.

View on arXivView PDF

Code Implementations(1)

Jazhyc/aims-safetyCommunity65%

Project page for 'Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes'

00Jun 23, 20263 months ago

Similar Papers

Can Vision-Language Models Stay Helpful When Facing Implicit Risks? Intent-Privilege OPSD for Efficient Safety-Helpfulness Alignment

89% match2026

SIA: Enhancing Safety via Intent Awareness for Vision-Language Models

87% match1 repo2025

Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models

85% match1 repo12026

OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Sets

85% match2026

DT-Guard: Intent-Driven Reasoning-Active Training for Reasoning-Free LLM Safety Guardrail

84% match2026

SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models

84% match1 repo102025

Cite this paper

@article{ferrao2026paved,
  title  = {Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes},
  author = {Jeremias Ferrao and Niclas Müller-Hof and Iustin Sîrbu and Traian Rebedea and Yftah Ziser},
  year   = {2026},
  eprint = {2606.27210},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2606.27210v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2606.27210.svg)](https://codeofpaper.com/paper/2606.27210)

Discussion