MDMMT: Multidomain Multimodal Transformer for Video Retrieval

Maksim Dzabraev, Maksim Kalashnikov, Stepan Komkov, Aleksandr Petiushko
3/19/2021
cs.CV

Official code: papermsucode/mdmmt (26 stars, last commit 2021-06-28).

Abstract

We present a new state-of-the-art on the text to video retrieval task on MSRVTT and LSMDC benchmarks where our model outperforms all previous solutions by a large margin. Moreover, state-of-the-art results are achieved with a single model on two datasets without finetuning. This multidomain generalisation is achieved by a proper combination of different video caption datasets. We show that training on different datasets can improve test results of each other. Additionally we check intersection between many popular datasets and found that MSRVTT has a significant overlap between the test and the train parts, and the same situation is observed for ActivityNet.

View on arXivView PDF

Code Implementations(1)

papermsucode/mdmmtOfficial100%

MDMMT: Multidomain Multimodal Transformer for Video Retrieval

264PythonMar 31, 20215 years ago

Similar Papers

Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval

89% match1 repo3802021

Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval

87% match2025

Dual Learning with Dynamic Knowledge Distillation and Soft Alignment for Partially Relevant Video Retrieval

86% match1 repo192025

MV-Adapter: Multimodal Video Transfer Learning for Video Text Retrieval

85% match1 repo142024

Dual Learning with Dynamic Knowledge Distillation for Partially Relevant Video Retrieval

85% match1 repo192023

Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning

85% match1 repo212024

Cite this paper

@article{dzabraev2021mdmmt,
  title  = {MDMMT: Multidomain Multimodal Transformer for Video Retrieval},
  author = {Maksim Dzabraev and Maksim Kalashnikov and Stepan Komkov and Aleksandr Petiushko},
  year   = {2021},
  eprint = {2103.10699},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2103.10699}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2103.10699.svg)](https://codeofpaper.com/paper/2103.10699)

Discussion