$\nabla$NABLA: Neighborhood Adaptive Block-Level Attention

Dmitrii Mikhailov, Aleksey Letunovskiy, Maria Kovaleva, Vladimir Arkhipkin, Vladimir Korviakov +4 more
7/17/2025
cs.CV

Official code: gen-ai-team/Wan2.1-NABLA (31 stars, Apache-2.0, last commit 2025-07-28).

Abstract

Recent progress in transformer-based architectures has demonstrated remarkable success in video generation tasks. However, the quadratic complexity of full attention mechanisms remains a critical bottleneck, particularly for high-resolution and long-duration video sequences. In this paper, we propose NABLA, a novel Neighborhood Adaptive Block-Level Attention mechanism that dynamically adapts to sparsity patterns in video diffusion transformers (DiTs). By leveraging block-wise attention with adaptive sparsity-driven threshold, NABLA reduces computational overhead while preserving generative quality. Our method does not require custom low-level operator design and can be seamlessly integrated with PyTorch's Flex Attention operator. Experiments demonstrate that NABLA achieves up to 2.7x faster training and inference compared to baseline almost without compromising quantitative metrics (CLIP score, VBench score, human evaluation score) and visual quality drop. The code and model weights are available here: https://github.com/gen-ai-team/Wan2.1-NABLA

View on arXivView PDF

Code Implementations(1)

gen-ai-team/Wan2.1-NABLAOfficial100%
Apache-2.0

Wan: Open and Advanced Large-Scale Video Generative Models

311Shell, PythonJun 19, 20251 years agoApache-2.0

Similar Papers

SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer

91% match2026

Bidirectional Sparse Attention for Faster Video Diffusion Training

90% match2025

FG-Attn: Leveraging Fine-Grained Sparsity In Diffusion Transformers

90% match2025

SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention

89% match1 repo3592025

Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers

89% match1 repo522025

TIMERIPPLE: Accelerating vDiTs by Understanding the Spatio-Temporal Correlations in Latent Space

89% match2025

Cite this paper

@article{mikhailov2025nablanabla,
  title  = {$\nabla$NABLA: Neighborhood Adaptive Block-Level Attention},
  author = {Dmitrii Mikhailov and Aleksey Letunovskiy and Maria Kovaleva and Vladimir Arkhipkin and Vladimir Korviakov and Vladimir Polovnikov and Viacheslav Vasilev and Evelina Sidorova and Denis Dimitrov},
  year   = {2025},
  eprint = {2507.13546},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.13546v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.13546.svg)](https://codeofpaper.com/paper/2507.13546)

Discussion