Principal Investigator: Ross Higashi
CoPrincipal Investigator(s): Jean Oh
Organization: Carnegie-Mellon University
NSF Award Information: Using AI to Focus Teacher-Student Troubleshooting in Classroom Robotics
Abstract:
Maintaining effective instructional interactions between teachers and students around content is challenging, especially in open-ended problem-solving domains such as computer programming. Troubleshooting student programs at the classroom scale becomes difficult, even more so in remote or hybrid instructional contexts. Yet an instructor’s adaptability, insight, rapport with students, and leadership role in the classroom remain indispensable. This project will explore the use of Machine Learning (ML) algorithms to offload the time-consuming tasks of finding and deciphering student errors while also focusing teacher-student troubleshooting interactions around algorithmically identified episodic “clips” of student work-in-progress. This approach differs from the current state of the art in that it neither replaces nor simply informs the teacher, but instead convenes students and instructors around instructionally rich portions of the students’ own code and output. Design, development, and refinement of a prototype Convening AI system for middle school robotics programming will directly impact more than a dozen educators and 2000 of their students, including several schools serving underrepresented minority populations. It will also produce generalizable know-how about the design of Convening AI systems for other educational domains and ultimately inform future directions for the design of human-AI systems.

This project will address the technical and sociotechnical integration challenges of an AI-driven convener through design-based research by developing a proof-of-concept Formative Assessment Suggestion Tool (FAST) in the context of middle school robotics programming. FAST will compare the efficacy of different probabilistic and neural network-based self-supervised learning approaches in identifying a student’s intended solution from their source code and simulated robot run telemetry, e.g., by comparing the plan generated by a student with that by an optimal planner. It then uses a rollback planner to identify the point at which the student’s current implementation no longer has a likely path to that solution, such that this point can be expressed to the teacher. FAST’s ML models are initially trained on an archival data set of 35,000 student code submissions to isomorphic robot programming scenarios. Each source file is re-simulated in an instrumented environment to reconstruct position, collision, and other information. Additional data including longitudinal student code-writing behavior will be collected using instrumentation upgrades developed and deployed to the simulator curriculum’s active user base during the project. Data from classroom observation will be used to model and monitor proportions of time spent engaged in different instructional actions with and without the tool. User experience around convening will be refined through participatory co-design with teachers and students. Structural equation modeling will be used to test a theory of action around uptake of the tool into classroom practice: faster, more accurate troubleshooting increases student learning and engagement as well as teacher satisfaction, leading to acceptance and continued use of the technology in a virtuous cycle.

This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.