Exploring Models and Data for Image Question Answering

Mengye Ren, Ryan Kiros, Richard Zemel
5/8/2015
cs.LGcs.AIcs.CL

Official code: renmengye/imageqa-public (82 stars, MIT, last commit 2016-03-23).

Abstract

This work aims to address the problem of image-based question-answering (QA) with new models and datasets. In our work, we propose to use neural networks and visual semantic embeddings, without intermediate stages such as object detection and image segmentation, to predict answers to simple questions about images. Our model performs 1.8 times better than the only published results on an existing image QA dataset. We also present a question generation algorithm that converts image descriptions, which are widely available, into QA form. We used this algorithm to produce an order-of-magnitude larger dataset, with more evenly distributed answers. A suite of baseline results on this new dataset are also presented.

View on arXivView PDF

Code Implementations(1)

renmengye/imageqa-publicOfficial100%
MIT

Code for paper "Exploring Models and Data for Image Question Answering"

8229PythonSep 23, 201510 years agoMIT

Similar Papers

ChiQA: A Large Scale Image-based Real-World Question Answering Dataset for Multi-Modal Understanding

87% match1 repo342022

OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese

86% match1 repo62023

Focal Visual-Text Attention for Visual Question Answering

86% match1 repo332018

BLIVA: A Simple Multimodal LLM for Better Handling of Text-Rich Visual Questions

86% match1 repo2612023

A Comprehensive Survey on Visual Question Answering Datasets and Algorithms

85% match2024

Multi-Clue Reasoning with Memory Augmentation for Knowledge-based Visual Question Answering

85% match2023

Cite this paper

@article{ren2015exploring,
  title  = {Exploring Models and Data for Image Question Answering},
  author = {Mengye Ren and Ryan Kiros and Richard Zemel},
  year   = {2015},
  eprint = {1505.02074},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1505.02074}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1505.02074.svg)](https://codeofpaper.com/paper/1505.02074)

Discussion