Strategies for Creating Uncertainty in the AI Era to Trigger Students Critical Thinking: Pedagogical Design, Assessment Rubric, and Exam System

Abstract

Generative AI challenges traditional assessments by allowing students to produce correct answers without demonstrating understanding or reasoning. Rather than prohibiting AI, this work argues that one way to integrate AI into education is by creating uncertain situations with the help of AI models and using thinking-oriented teaching approaches, where uncertainty is a central pedagogical concept for stimulating students critical thinking. Drawing on epistemology and critical thinking research studies, we propose designing learning activities and assessments around the inherent limitations of both AI models and instructors. This encourages students to reason, question, and justify their final answers. We show how explicitly controlling AI behavior during exams (such as preventing direct answers or generating plausible but flawed responses) prevents AI from becoming a shortcut to certainty. To support this pedagogy, we introduce MindMosaicAIExam, an exam system that integrates controllable AI tools and requires students to provide initial answers, critically evaluate AI outputs, and iteratively refine their reasoning. We also present an evaluation rubric designed to assess critical thinking based on students reasoning artifacts collected by the exam system.