Visual Semantic Relatedness Dataset for Image Captioning

Ahmed Sabir, Francesc Moreno-Noguer, Lluís Padró
1/20/2023
cs.CLcs.CV

No official code found. Best implementation: ahmedssabir/Textual-Visual-Semantic-Dataset (10 stars, last commit 2024-02-27).

Abstract

Modern image captioning system relies heavily on extracting knowledge from images to capture the concept of a static story. In this paper, we propose a textual visual context dataset for captioning, in which the publicly available dataset COCO Captions (Lin et al., 2014) has been extended with information about the scene (such as objects in the image). Since this information has a textual form, it can be used to leverage any NLP task, such as text similarity or semantic relation methods, into captioning systems, either as an end-to-end training strategy or a post-processing based approach.

View on arXivView PDF

Code Implementations(1)

ahmedssabir/Textual-Visual-Semantic-DatasetCommunity100%

Visual Semantic Relatedness Dataset for Captioning. CVPRW 2023

102Python, Jupyter NotebookAug 17, 20222 years ago

Similar Papers

ClipCap: CLIP Prefix for Image Captioning

86% match1 repo1.4k2021

A Comprehensive Analysis of Real-World Image Captioning and Scene Identification

86% match2023

Retrieval-Augmented Transformer for Image Captioning

85% match2022

DEVICE: Depth and Visual Concepts Aware Transformer for OCR-based Image Captioning

85% match2023

Beyond Vision: Contextually Enriched Image Captioning with Multi-Modal Retrieval

85% match2025

Image Captioners Sometimes Tell More Than Images They See

84% match2023

Cite this paper

@article{sabir2023visual,
  title  = {Visual Semantic Relatedness Dataset for Image Captioning},
  author = {Ahmed Sabir and Francesc Moreno-Noguer and Lluís Padró},
  year   = {2023},
  eprint = {2301.08784},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.08784v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.08784.svg)](https://codeofpaper.com/paper/2301.08784)

Discussion