Claire Christensen, Senior Education Researcher at SRI International, shares more on her most recent NSF-funded project, EAGER: Technology to Review Online Videos for Education (TROVE) (#2139219).
Team: Anirban Roy, Sarah Gerard, Maddie Cincebeaux
Exposure to online videos through various digital media platforms has become increasingly prevalent among young children. In particular, YouTube has become one of the most popular video streaming platforms regularly consumed. Because there are so many online videos and they are released at such an astonishing rate, it is challenging to understand the specific content that children are exposed to. Together, Dr. Claire Christensen, an education researcher, and Co-PI Dr. Anirban Roy, a machine-learning (ML) researcher, have formed an interdisciplinary team to create a machine learning-based tool that can identify early mathematics content in YouTube videos.
What inspired your project?
With the portability of digital devices, it has become nearly impossible for adults, caregivers, or educators to monitor every video a child engages with on YouTube. This has made it challenging for adults to distinguish between educational and non-educational content consumption on the platform.
On average, approximately 500 hours of video content are released every minute on YouTube, and there are no existing automated approaches to classifying this content. Existing approaches rely on human reviews to identify video content, but it is difficult to keep up with coding a rapidly growing collection of YouTube content and apply those reviews to more broadly understand children’s media consumption. To support children’s healthy media use we need an efficient, accurate way to understand what they watch on YouTube. So we are working to develop TROVE, Technology to Review Online Videos for Education. It is a tool that will use ML algorithms to identify and better understand the mathematics content in online videos. In the future, researchers could use this tool to understand how exposure to different types of media content affects their development. And developers could use this tool to create or direct children to high-quality educational content online.
What are you most excited about or looking forward to accomplishing in the first year of your project and beyond?
Our prior internally funded research and development allowed us to work with parents and educators to pave the direction of the project. Now, in the first year of our NSF EAGER funding, the priority is to accurately train a ML algorithm to understand kindergarten- and preschool-level mathematics content in YouTube videos. Human annotators will use a rubric we developed to identify Common Core-aligned early mathematics content in YouTube videos. Then we will use their annotations to train a machine learning model and judge its accuracy. We will also have the support of an external advisory board that has expertise in children’s media, learning sciences, and mathematics learning.
We envision several future applications for TROVE. For example, we have submitted a collaborative NIH/NICHD proposal around using TROVE as part to better measure children’s media exposure and measure its longitudinal correlations with their academic, social-emotional, and physical development. Another potential future application is as a tool for parents, caregivers, or educators to get a more nuanced measure of their child’s exposure than what they currently have access to (e.g., total screen time, name of videos, name of apps). Paired with research that looks at outcomes, TROVE could then be used to flag children’s media use as a predictor of positive or negative outcomes and provide guidance to adults on interventions, if needed.
What do you foresee being your biggest obstacle to meeting this goal?
There may never be enough data to train the model, so we need to explore novel approaches to machine learning with small datasets. For ML researchers, education content is novel and nuanced in ways that other content topics are not. In education, you can teach a domain in a multitude of ways whereas more other machine learning tasks, like classifying objects, are more clear-cut. Human reviewers can identify the many myriad ways a video teaches a skill, but training an ML algorithm to be this flexible is challenging and requires a lot of data. And this challenge is part of why I love working on this project at SRI, with colleagues who are world-class ML researchers. Through this NSF grant we have opportunities to apply existing technologies developed in other fields to piece together the best possible approach to this problem. I can’t wait to see what we learn!
