Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness

Hoang Cuong Nguyen, Mark Dras, Usman Naseem
9/3/2026
cs.CL

Official code: hoangcuongnguyen2001/Beyond-Shallow-Alignment (0 stars, last commit 2026-08-31).

Abstract

How do the methods used to train language models to refuse harmful requests shape how that refusal actually works inside the model? We compare three post-training methods - supervised fine-tuning, reasoning-augmented fine-tuning (training on reasoning chains that justify a safety decision), and preference optimization (ORPO) - across three architecturally distinct models (Llama-3.1-8B, Gemma-2-9B, Qwen3-8B). We find that training method, not just data, reshapes how refusal is computed internally: reasoning-augmented training consistently produces a distinct kind of refusal computation, visible across all three models, while architecture independently shapes internal structure and how reliably refusal can be steered. Most importantly, no method we study achieves all three properties we would want from safe alignment at once: refusal that isn't concentrated in a few fragile components, safety gains that don't cost general capability, and safety behavior correctable through small, targeted edits. We caution against treating current post-training methods as a solved, reliable defense, especially for security-critical use. Code and models are available in https://github.com/hoangcuongnguyen2001/Beyond-Shallow-Alignment.

View on arXivView PDF

Code Implementations(1)

hoangcuongnguyen2001/Beyond-Shallow-AlignmentOfficial100%

Repository for the 2026 EMNLP Main Conference paper "Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness"

00Python, Jupyter NotebookAug 26, 20261 months ago

Similar Papers

Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Models

88% match1 repo112024

Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training

88% match1 repo722024

Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior

88% match2025

Refusal geometry reflects refusal training: diverse refusal prefixes can raise stable rank and weaken refusal vector ablation attacks

88% match2026

Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

88% match2026

Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning

87% match1 repo22025

Cite this paper

@article{nguyen2026beyond,
  title  = {Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness},
  author = {Hoang Cuong Nguyen and Mark Dras and Usman Naseem},
  year   = {2026},
  eprint = {2609.03887},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2609.03887v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2609.03887.svg)](https://codeofpaper.com/paper/2609.03887)

Discussion