A Comparative Study of Text Retrieval Models on DaReCzech

Jakub Stetina, Martin Fajcik, Michal Stefanik, Michal Hradis
11/19/2024
cs.IRcs.AI

No confident code implementation found for this paper.

Abstract

This article presents a comprehensive evaluation of 7 off-the-shelf document retrieval models: Splade, Plaid, Plaid-X, SimCSE, Contriever, OpenAI ADA and Gemma2 chosen to determine their performance on the Czech retrieval dataset DaReCzech. The primary objective of our experiments is to estimate the quality of modern retrieval approaches in the Czech language. Our analyses include retrieval quality, speed, and memory footprint. Secondly, we analyze whether it is better to use the model directly in Czech text, or to use machine translation into English, followed by retrieval in English. Our experiments identify the most effective option for Czech information retrieval. The findings revealed notable performance differences among the models, with Gemma22 achieving the highest precision and recall, while Contriever performing poorly. Conclusively, SPLADE and PLAID models offered a balance of efficiency and performance.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Exploring Information Retrieval Landscapes: An Investigation of a Novel Evaluation Techniques and Comparative Document Splitting Methods

87% match1 repo12024

FiCo-ITR: bridging fine-grained and coarse-grained image-text retrieval for comparative performance analysis

84% match1 repo22024

Beyond Content Relevance: Evaluating Instruction Following in Retrieval Models

82% match1 repo82026

IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarios

81% match1 repo12024

Do We Need Bigger Models for Science? Task-Aware Retrieval with Small Language Models

81% match2026

Benchmarking Information Retrieval Models on Complex Retrieval Tasks

81% match2025

Cite this paper

@article{stetina2024comparative,
  title  = {A Comparative Study of Text Retrieval Models on DaReCzech},
  author = {Jakub Stetina and Martin Fajcik and Michal Stefanik and Michal Hradis},
  year   = {2024},
  eprint = {2411.12921},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.12921v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.12921.svg)](https://codeofpaper.com/paper/2411.12921)

Discussion