Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement

Diego Rossini, Lonneke van der Plas
1/27/2026
cs.CL

No official code found. Best implementation: DiegoRossini/binary-mwe-detection (3 stars, MIT, last commit 2026-01-15).

Abstract

We present a comprehensive approach for multiword expression (MWE) identification that combines binary token-level classification, linguistic feature integration, and data augmentation. Our DeBERTa-v3-large model achieves 69.8% F1 on the CoAM dataset, surpassing the best results (Qwen-72B, 57.8% F1) on this dataset by 12 points while using 165x fewer parameters. We achieve this performance by (1) reformulating detection as binary token-level START/END/INSIDE classification rather than span-based prediction, (2) incorporating NP chunking and dependency features that help discontinuous and NOUN-type MWEs identification, and (3) applying oversampling that addresses severe class imbalance in the training data. We confirm the generalization of our method on the STREUSLE dataset, achieving 78.9% F1. These results demonstrate that carefully designed smaller models can substantially outperform LLMs on structured NLP tasks, with important implications for resource-constrained deployments.

View on arXivView PDF

Code Implementations(1)

DiegoRossini/binary-mwe-detectionCommunity70%
MIT

Code for "Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement" (EACL 2026)

30Shell, PythonJan 12, 20269 months agoMIT

Similar Papers

CoAM: Corpus of All-Type Multiword Expressions

81% match2024

Multilingual Multiword Expression Identification Using Lateral Inhibition and Domain Adaptation

79% match2023

Romanian Multiword Expression Detection Using Multilingual Adversarial Training and Lateral Inhibition

78% match2023

Towards Fair and Efficient De-identification: Quantifying the Efficiency and Generalizability of De-identification Approaches

77% match2026

Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection

77% match2024

Investigating Large Language Models for Complex Word Identification in Multilingual and Multidomain Setups

77% match2024

Cite this paper

@article{rossini2026binary,
  title  = {Binary Token-Level Classification with DeBERTa for All-Type MWE Identification: A Lightweight Approach with Linguistic Enhancement},
  author = {Diego Rossini and Lonneke van der Plas},
  year   = {2026},
  eprint = {2601.19360},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.19360v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.19360.svg)](https://codeofpaper.com/paper/2601.19360)

Discussion