Team UTSA-NLP at SemEval 2024 Task 5: Prompt Ensembling for Argument Reasoning in Civil Procedures with GPT4

Dan Schumacher, Anthony Rios
4/2/2024
cs.CL

No confident code implementation found for this paper.

Abstract

In this paper, we present our system for the SemEval Task 5, The Legal Argument Reasoning Task in Civil Procedure Challenge. Legal argument reasoning is an essential skill that all law students must master. Moreover, it is important to develop natural language processing solutions that can reason about a question given terse domain-specific contextual information. Our system explores a prompt-based solution using GPT4 to reason over legal arguments. We also evaluate an ensemble of prompting strategies, including chain-of-thought reasoning and in-context learning. Overall, our system results in a Macro F1 of .8095 on the validation dataset and .7315 (5th out of 21 teams) on the final test set. Code for this project is available at https://github.com/danschumac1/CivilPromptReasoningGPT4.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Archimedes-AUEB at SemEval-2024 Task 5: LLM explains Civil Procedure

86% match2024

NLP at UC Santa Cruz at SemEval-2024 Task 5: Legal Answer Validation using Few-Shot Multi-Choice QA

83% match1 repo2024

eagerlearners at SemEval2024 Task 5: The Legal Argument Reasoning Task in Civil Procedure

82% match2024

LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights

79% match2024

CALRK-Bench: Evaluating Context-Aware Legal Reasoning in Korean Law

79% match1 repo12026

Towards Supporting Legal Argumentation with NLP: Is More Data Really All You Need?

79% match2024

Cite this paper

@article{schumacher2024team,
  title  = {Team UTSA-NLP at SemEval 2024 Task 5: Prompt Ensembling for Argument Reasoning in Civil Procedures with GPT4},
  author = {Dan Schumacher and Anthony Rios},
  year   = {2024},
  eprint = {2404.01961},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.01961v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.01961.svg)](https://codeofpaper.com/paper/2404.01961)

Discussion