Principal Investigator: Deborah Fields
CoPrincipal Investigator(s): Sarah Brasiel, Taylor Martin
Organization: Utah State University

Abstract:
This Cyberlearning: Transforming Education Exploration Project is designed to advance understanding of how to personalize feedback and advice to learners as they engage in exploratory and creative activities in constructionist technology-enhanced learning environments. During such activities, learners often engage in programming (using, e.g., Scratch, Alice) with the goal of creating a model or an animation of their own choosing. Assessment of learner capabilities and conceptions would allow automated personalization of advice to learners, facilitate self-reflection, and help teachers or mentors to know the range of capabilities and understanding across a classroom. This project brings together a PI who is expert at promoting learning in the context of constructionist learning activities and another who is expert at educational data mining to identify indicators of young learners’ (middle schoolers) conceptions of computational concepts and programming capabilities. The project uses a data analytic approach; data mining methods are used to mine the thousands of operations learners carry out to find patterns that might indicate understanding and capability, qualitative methods are used to analyze what learners were intending and thinking as they were carrying out those operations, patterns are identified in the observational data, and the two streams of data are matched to identify the ways conceptions and capabilities show themselves while learners are programming. The intellectual activity focuses both on the combination of data mining and ethnographic methods for such purposes and on the specifics of those indicators.

Automating assessment is difficult in a project-based learning environment where learners are creating products of their own choosing. Because the activity is quite unconstrained, collecting and analyzing the data necessary for providing help and feedback to the learner is quite difficult. This project uses a combination data analytic and ethnographic approach to find indicators of the conceptions and capabilities of middle schoolers as they are using Scratch to create models and animations of their choosing. The results of this project will make contributions in several areas: (i) advancing methods for automating assessment for learners using the Scratch programming language, (ii) advancing methods for data collection and analysis for personalizing feedback in a relatively open-ended programming environment (iii) broadening understanding of how to assess computational thinking in the context of open-ended programming assignments, and (iv) advancing methodology for automatically assessing capabilities and understanding when learners are engaged in open-ended kinds of assignments.