Principal Investigator: Zachary Alstad
CoPrincipal Investigator(s): Jodi Asbell-Clarke, Ibrahim Dahlstrom-Hakki
Organization: TERC Inc
NSF Award Information: Using Augmented Reality to Enhance Attention in STEM Learning for Students with Executive Function Disabilities
Abstract:
Using Augmented Reality to Enhance Attention in Science, Technology, Engineering and Mathematics Learning for Students with Executive Function Disabilities

Executive functioning represents a broad skill set that is central to STEM learning and academic success, including skills related to attention, persistence, emotion regulation, and inhibition control. For students with executive function difficulties, focusing during homework and other independent learning tasks is a challenge. Many existing assistive technologies supplementing a student’s organizational capabilities are expensive and exclusionary and rely on students’ ability to organize or initiate their own task monitoring. This project provides an innovative, scalable intervention that enables individuals who struggle with executive functioning to persist and thrive in academic and workplace settings by using Augmented Reality technology to support an individual’s ability to self-monitor their attention and re-engage with the content when they are off task. In the long term, the system has the potential to become a widely used learning technology that improves outcomes for students with and without executive functioning difficulties. The project will serve the public interest by increasing the participation of a population that is currently underrepresented in STEM education and the STEM workforce.

This project will make a novel contribution to both the computer sciences and the learning sciences, achieved through two overarching project goals. First, using a temporal analysis of head position, head orientation, and gaze orientation, the project team will develop an open-source tool based on deep learning algorithms that can detect off-task behavior as undergraduate students work on math homework problems. Second, the team will use augmented reality to provide appropriate feedback to students in the form of redirection or breaks related to their own level of focus and distractibility. Development of this system will involve an iterative prototyping and testing process, and will conclude with an early efficacy study. To explore the most effective prompts, students with executive functioning issues will participate as co-designers of the system. The deep learning algorithm for the detection of off-task behavior is a novel contribution to the computer science field that has potential applications for other subject areas and other types of learning difficulties. Furthermore, the resultant self-monitoring and redirection prompts are a contribution to the learning sciences that can benefit most learners. Thus, this research will be a significant development in the application of augmented reality to learning as it explores the most productive means of student interaction with educational technology in a complex problem space.

This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.