Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds

Xianzhe Fan, Shengliang Deng, Xiaoyang Wu, Yuxiang Lu, Zhuoling Li +5 more
1/31/2026
cs.CVcs.RO

No official code found. Best implementation: XianzheFan/Any3D-VLA (64 stars, MIT, last commit 2026-02-07). 1 more below.

Abstract

Existing Vision-Language-Action (VLA) models typically take 2D images as visual input, which limits their spatial understanding in complex scenes. How can we incorporate 3D information to enhance VLA capabilities? We conduct a pilot study across different observation spaces and visual representations. The results show that explicitly lifting visual input into point clouds yields representations that better complement their corresponding 2D representations. To address the challenges of (1) scarce 3D data and (2) the domain gap induced by cross-environment differences and depth-scale biases, we propose Any3D-VLA. It unifies the simulator, sensor, and model-estimated point clouds within a training pipeline, constructs diverse inputs, and learns domain-agnostic 3D representations that are fused with the corresponding 2D representations. Simulation and real-world experiments demonstrate Any3D-VLA's advantages in improving performance and mitigating the domain gap. Our project homepage is available at https://xianzhefan.github.io/Any3D-VLA.github.io.

View on arXivView PDF

Code Implementations(2)

XianzheFan/Any3D-VLACommunity63%
MIT

Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds

643Shell, PythonFeb 3, 20268 months agoMIT
XianzheFan/Any3D-VLA-real-world-controllerCommunity62%
MIT

Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds

20Python, DockerfileFeb 3, 20268 months agoMIT

Similar Papers

PointVLA: Injecting the 3D World into Vision-Language-Action Models

87% match2025

AugVLA-3D: Depth-Driven Feature Augmentation for Vision-Language-Action Models

85% match2026

3D-VLA: A 3D Vision-Language-Action Generative World Model

84% match2 repos1.1k2026

VLA Models Are More Generalizable Than You Think: Revisiting Physical and Spatial Modeling

83% match2025

GeneralVLA: Generalizable Vision-Language-Action Models with Knowledge-Guided Trajectory Planning

82% match1 repo122026

VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction

82% match1 repo4452025

Cite this paper

@article{fan2026anydvla,
  title  = {Any3D-VLA: Enhancing VLA Robustness via Diverse Point Clouds},
  author = {Xianzhe Fan and Shengliang Deng and Xiaoyang Wu and Yuxiang Lu and Zhuoling Li and Mi Yan and Yujia Zhang and Zhizheng Zhang and He Wang and Hengshuang Zhao},
  year   = {2026},
  eprint = {2602.00807},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2602.00807v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2602.00807.svg)](https://codeofpaper.com/paper/2602.00807)

Discussion