Analyzing Large Language Models for Classroom Discussion Assessment

Nhat Tran, Benjamin Pierce, Diane Litman, Richard Correnti, Lindsay Clare Matsumura
6/12/2024
cs.CL

No official code found. Best implementation: nhattlm95/LLM_for_Classroom_Discussion (3 stars, MIT, last commit 2024-05-08).

Abstract

Automatically assessing classroom discussion quality is becoming increasingly feasible with the help of new NLP advancements such as large language models (LLMs). In this work, we examine how the assessment performance of 2 LLMs interacts with 3 factors that may affect performance: task formulation, context length, and few-shot examples. We also explore the computational efficiency and predictive consistency of the 2 LLMs. Our results suggest that the 3 aforementioned factors do affect the performance of the tested LLMs and there is a relation between consistency and performance. We recommend a LLM-based assessment approach that has a good balance in terms of predictive performance, computational efficiency, and consistency.

View on arXivView PDF

Code Implementations(1)

nhattlm95/LLM_for_Classroom_DiscussionCommunity100%
MIT

Analyzing Large Language Models for Classroom Discussion Assessment

30PythonMay 8, 20242 years agoMIT

Similar Papers

Evaluating Large Language Models in Analysing Classroom Dialogue

88% match2024

LLM-as-a-Grader: Practical Insights from Large Language Model for Short-Answer and Report Evaluation

85% match2025

Are Large Language Models Good Essay Graders?

85% match2024

An Evaluation on Large Language Model Outputs: Discourse and Memorization

84% match2023

How well do Large Language Models Recognize Instructional Moves? Establishing Baselines for Foundation Models in Educational Discourse

84% match2025

A Report on the llms evaluating the high school questions

84% match2025

Cite this paper

@article{tran2024analyzing,
  title  = {Analyzing Large Language Models for Classroom Discussion Assessment},
  author = {Nhat Tran and Benjamin Pierce and Diane Litman and Richard Correnti and Lindsay Clare Matsumura},
  year   = {2024},
  eprint = {2406.08680},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.08680v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.08680.svg)](https://codeofpaper.com/paper/2406.08680)

Discussion