Semantic similarity prediction is better than other semantic similarity measures

Steffen Herbold
9/22/2023
cs.CLcs.LG

Official code: aieng-lab/stsscore (7 stars, Apache-2.0, last commit 2024-01-17).

Abstract

Semantic similarity between natural language texts is typically measured either by looking at the overlap between subsequences (e.g., BLEU) or by using embeddings (e.g., BERTScore, S-BERT). Within this paper, we argue that when we are only interested in measuring the semantic similarity, it is better to directly predict the similarity using a fine-tuned model for such a task. Using a fine-tuned model for the Semantic Textual Similarity Benchmark tasks (STS-B) from the GLUE benchmark, we define the STSScore approach and show that the resulting similarity is better aligned with our expectations on a robust semantic similarity measure than other approaches.

View on arXivView PDF

Code Implementations(1)

aieng-lab/stsscoreOfficial100%
Apache-2.0
70Jupyter NotebookSep 20, 20232 years agoApache-2.0

Similar Papers

Advancing Semantic Textual Similarity Modeling: A Regression Framework with Translated ReLU and Smooth K2 Loss

85% match1 repo32024

How Small Transformation Expose the Weakness of Semantic Similarity Measures

84% match2025

Advances and Challenges in Semantic Textual Similarity: A Comprehensive Survey

84% match2025

Annotating Training Data for Conditional Semantic Textual Similarity Measurement using Large Language Models

82% match1 repo12025

Reasoning before Comparison: LLM-Enhanced Semantic Similarity Metrics for Domain Specialized Text Analysis

82% match2024

Magnitude Matters: a Superior Class of Similarity Metrics for Holistic Semantic Understanding

81% match2025

Cite this paper

@article{herbold2023semantic,
  title  = {Semantic similarity prediction is better than other semantic similarity measures},
  author = {Steffen Herbold},
  year   = {2023},
  eprint = {2309.12697},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.12697v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.12697.svg)](https://codeofpaper.com/paper/2309.12697)

Discussion