ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning

Duc-Tai Dinh, Duc Anh Khoa Dinh
7/28/2025
cs.CLcs.IR

Official code: ductai05/ZSE-Cap (0 stars, GPL-3.0, last commit 2025-08-01).

Abstract

We present ZSE-Cap (Zero-Shot Ensemble for Captioning), our 4th place system in Event-Enriched Image Analysis (EVENTA) shared task on article-grounded image retrieval and captioning. Our zero-shot approach requires no finetuning on the competition's data. For retrieval, we ensemble similarity scores from CLIP, SigLIP, and DINOv2. For captioning, we leverage a carefully engineered prompt to guide the Gemma 3 model, enabling it to link high-level events from the article to the visual content in the image. Our system achieved a final score of 0.42002, securing a top-4 position on the private test set, demonstrating the effectiveness of combining foundation models through ensembling and prompting. Our code is available at https://github.com/ductai05/ZSE-Cap.

View on arXivView PDF

Code Implementations(1)

ductai05/ZSE-CapOfficial100%
GPL-3.0

ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning

00Jupyter NotebookJun 29, 20251 years agoGPL-3.0

Similar Papers

IFCap: Image-like Retrieval and Frequency-based Entity Filtering for Zero-shot Captioning

85% match1 repo162024

The Solution for the CVPR2024 NICE Image Captioning Challenge

81% match2024

MeaCap: Memory-Augmented Zero-shot Image Captioning

80% match1 repo562026

TPCap: Unlocking Zero-Shot Image Captioning with Trigger-Augmented and Multi-Modal Purification Modules

80% match2025

ReCap: Event-Aware Image Captioning with Article Retrieval and Semantic Gaussian Normalization

79% match2025

ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishing

79% match1 repo762026

Cite this paper

@article{dinh2025zsecap,
  title  = {ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning},
  author = {Duc-Tai Dinh and Duc Anh Khoa Dinh},
  year   = {2025},
  eprint = {2507.20564},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2507.20564v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2507.20564.svg)](https://codeofpaper.com/paper/2507.20564)

Discussion