DateLogicQA: Benchmarking Temporal Biases in Large Language Models

Gagan Bhatia, MingZe Tang, Cristina Mahanta, Madiha Kazi
12/17/2024
cs.CLcs.AI

No official code found. Best implementation: gagan3012/EAIS-Temporal-Bias (0 stars, last commit 2024-12-01).

Abstract

This paper introduces DateLogicQA, a benchmark with 190 questions covering diverse date formats, temporal contexts, and reasoning types. We propose the Semantic Integrity Metric to assess tokenization quality and analyse two biases: Representation-Level Bias, affecting embeddings, and Logical-Level Bias, influencing reasoning outputs. Our findings provide a comprehensive evaluation of LLMs' capabilities and limitations in temporal reasoning, highlighting key challenges in handling temporal data accurately.

View on arXivView PDF

Code Implementations(1)

gagan3012/EAIS-Temporal-BiasCommunity100%
00Python, Jupyter NotebookNov 11, 20241 years ago

Similar Papers

Temporal Referential Consistency: Do LLMs Favor Sequences Over Absolute Time References?

85% match2025

Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning

85% match2026

LTLBench: Towards Benchmarks for Evaluating Temporal Reasoning in Large Language Models

84% match1 repo22024

What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?

84% match2026

A Study into Investigating Temporal Robustness of LLMs

83% match2025

When Benchmarks Age: Temporal Misalignment through Large Language Model Factuality Evaluation

83% match1 repo42025

Cite this paper

@article{bhatia2024datelogicqa,
  title  = {DateLogicQA: Benchmarking Temporal Biases in Large Language Models},
  author = {Gagan Bhatia and MingZe Tang and Cristina Mahanta and Madiha Kazi},
  year   = {2024},
  eprint = {2412.13377},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.13377v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.13377.svg)](https://codeofpaper.com/paper/2412.13377)

Discussion