Principal Investigator: Eli Meir
CoPrincipal Investigator(s): Eric Klopfer, Joel Abraham, Zhushan Li
Organization: SimBiotic Software

Abstract:
This project seeks to develop a dynamic formative assessment method for use with virtual labs. The research focuses on how to constrain a virtual lab experience to be amenable to automated feedback on relatively open-ended responses students are generating while still giving students an appropriate exploratory experience. The technology innovation question is how to do that with validity and reliability. The team is focusing on how to use available artificial intelligence technologies to make it possible to provide good feedback, both to learners working in these environments and to their teachers. PIs are adding dynamic formative assessment capabilities to virtual lab experiences that are already extensively used in undergraduate and high-school biology classes.

This is an automated assessment project, focusing on assessing learner understanding and capabilities in situations where learners are exploring, having, and using ideas as they are learning STEM content and practices. There are several ways one could approach automated assessment for situations where learners are acting in a fairly unconstrained way — design algorithms that can interpret and make inferences from free text, or find ways to design the environment in such a way that learners can explore, develop, and record as needed for deep learning but where they have a more constrained way of expressing themselves or limitations in what they can do that don’t constrain the learning or engagement. This project seeks to find a sweet spot — a happy medium where learners can explore, try things out, have ideas, refine ideas, and use ideas with significant freedom but just enough constraint for already-existing artificial intelligence algorithms to interpret what learners are doing, why they are doing it, and what they mean to express. Learning how to do this is essential to designing the learning environments of the future.

There is broad acknowledgement that more attention must be given in STEM fields to the teaching of higher-order thinking skills, including experimental design, data interpretation and evidence-based judgment. Timely formative assessment is a crucial component of such learning, but formative assessment is impossible for a teacher to do for a whole class of individuals at the time when it will have the most effect (when students are engaging in or have just finished engaging in such activities) and too labor intensive to be done regularly in large high school and introductory-level college classes. This project is therefore developing techniques for automatically providing immediate formative assessment as students are conducting simulation-based experiments and reasoning about their results. The investigation focuses on helping students learn to conduct experiments and interpret results within the discipline of biology; lessons learned will be applicable across STEM domains at the high-school and college levels.