Principal Investigator: David Mostow
CoPrincipal Investigator(s): Jessica Nelson-Taylor
Organization: Carnegie-Mellon University

Abstract:
The big question the PIs are addressing in this project is how to unobtrusively track silent reading of novice readers so as to be able to use an intelligent tutoring system to aid reading comprehension. This EAGER project focuses on the first steps in answering that question. This pilot project builds on previous work in vision and speech technology, sensor fusion, machine learning, user modeling, intelligent tutors, and eye movements in an effort to identify the feasibility of using eye tracking techniques, along with other information collected from an intelligent reading tutor, to predict reading difficulties of novice/young readers. In particular, the work plan includes collecting gaze data in real-world conditions in the context of using the existing Reading Tutor, designing software to display those traces so that accuracy can be guauged, testing gazepoint accuracy and detecting gaze-speech discrepancies, and using that data to develop heuristics for detecting tracking errors in real time and calibrating eye tracking data to noisy school environments, a primary environment where the augmented Reading Tutor would ultimately be used. The intellectual merit of this project is in identifying and addressing challenges in relating children’s gaze data to their silent reading, in making technical contributions to calibrating eye trackers so that they can be used in normal everyday applications, and in setting the stage for intelligent tutors across diverse domains to exploit gaze more broadly.

The project’s most important potential broader impacts is in establishing a foundation for exploiting gaze input to build intelligent computing systems that can be used to help children with reading difficulties learn to read and read to learn. If successful, the PIs will develop a larger project that will extent the successful Project Listen Reading Tutor so that it can track readers as they are reading silently and help them with their comprehension — both comprehension of text itself and strategies for coming to deep understanding.