ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

Nghia Hieu Nguyen, Tho Thanh Quan, Ngan Luu-Thuy Nguyen
10/18/2024
cs.CVcs.CL

No official code found. Best implementation: nghiangh/ViConsFormer (1 star, last commit 2025-01-24).

Abstract

Text-based VQA is a challenging task that requires machines to use scene texts in given images to yield the most appropriate answer for the given question. The main challenge of text-based VQA is exploiting the meaning and information from scene texts. Recent studies tackled this challenge by considering the spatial information of scene texts in images via embedding 2D coordinates of their bounding boxes. In this study, we follow the definition of meaning from linguistics to introduce a novel method that effectively exploits the information from scene texts written in Vietnamese. Experimental results show that our proposed method obtains state-of-the-art results on two large-scale Vietnamese Text-based VQA datasets. The implementation can be found at this link.

View on arXivView PDF

Code Implementations(1)

nghiangh/ViConsFormerCommunity100%
10PythonJun 18, 20241 years ago

Similar Papers

ViTextVQA: A Large-Scale Visual Question Answering Dataset for Evaluating Vietnamese Text Comprehension in Images

85% match1 repo92024

LiGT: Layout-infused Generative Transformer for Visual Question Answering on Vietnamese Receipts

84% match2025

Advancing Vietnamese Visual Question Answering with Transformer and Convolutional Integration

83% match1 repo212024

AutoViVQA: A Large-Scale Automatically Constructed Dataset for Vietnamese Visual Question Answering

83% match1 repo2026

Enhancing Vietnamese VQA through Curriculum Learning on Raw and Augmented Text Representations

82% match1 repo52025

ViConBERT: Context-Gloss Aligned Vietnamese Word Embedding for Polysemous and Sense-Aware Representations

77% match1 repo32025

Cite this paper

@article{nguyen2024viconsformer,
  title  = {ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering},
  author = {Nghia Hieu Nguyen and Tho Thanh Quan and Ngan Luu-Thuy Nguyen},
  year   = {2024},
  eprint = {2410.14132},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.14132v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.14132.svg)](https://codeofpaper.com/paper/2410.14132)

Discussion