Interactions with AI Systems: How Do We Avoid Risk and Bias?

AI-enabled conversational agents first engaged learners in text-based dialogue in Carbonell’s 1970 SCHOLAR system. Since then, both the technology behind conversational agents and the learning sciences have advanced. Current conversational agents are able to sense, react, and interact through multiple modalities. This session will explore how teachers and learners can interact with an AI agent, what the agents may bring to the learning process, the potential risks that can occur using AI agents, research to understand the risks around biases and threats to fairness, and how to potentially mitigate risks. The format for the session will include brief presentations followed by questions from attendees. The four experts in the session will discuss their research on 1) collaborative game-based learning environments that detect when students are off-task from their text-based dialogue interactions, 2) conversational agents as facilitators being adapted for use with low-SES adult learners who are seeking certification in fields of software development, 3) a human-in-the-loop AI-powered system that helps teachers write meaningful and directed feedback to students on open-ended questions in the context of mathematics, and 4) emerging AI-technologies with potential to support teachers and students as they undertake sequences of varied collaborative activities (classroom orchestration) in classrooms. Each context has unique challenges and the experts will share how they work to avoid risks, biases, and threats to fairness within the situations they work in as well as in the models and methods they use.