Emerging Technology Adoption for Educators
by Pati Ruiz and Eleanor Richard
Key Ideas
- Bringing together school leaders, teachers, and emerging technology experts to create a framework for adopting emerging technologies in schools led to new solutions not imagined by the different groups on their own.
- Working intentionally to gain common understanding of goals while allowing differences in perspectives helped the group move forward.
- Structuring the group into various configurations, sometimes small group meetings, sometimes individual, enabled everyone to participate.
Background
Research has shown that the evaluation of emerging technologies, including those that involve artificial intelligence/machine learning (AI/ML) systems, often requires a significant amount of expert knowledge and historical context (Mohamed et al., 2020). Yet school leaders often must evaluate and make procurement decisions about AI/ML-enabled tools and systems without adequate training and guidance. We aimed to design an evaluation tool that would be useful for emerging technologies, both for researchers and educators.
To date, a number of AI/ML evaluation tools have been created (Holmes et al., 2021; The EdTech Equity Project, n.d.; The MITRE Corporation, 2019; Connor and Nelson 2021); however, few of them are specifically designed for U.S. school districts, and none of them have been widely adopted in the U.S. To address this issue, CIRCLS established the Emerging Technology Advisory Group for PK-12 to develop a framework for schools to evaluate and make procurement decisions about emerging technologies, including AI/ML systems. The advisory group worked together to identify needs and create tools for educators in U.S. PK-12 education settings. The overarching research question addressed by the group was:
Story of the Partnership
We begin with an overview of the partnership’s work from beginning to end. Throughout this work, we created a partnership among researchers, teachers, and administrators who are passionate about reviewing and assessing the emerging technology that enters schools and classrooms. We recruited school leaders, teachers, and emerging technology experts through the CIRCLS newsletter, which currently has more than 3500 subscribers. For all interested participants, we asked about their connection to PK-12 education and why they were interested in and felt they would be able to support this work. From these answers, 11 participants from diverse U.S. school districts were selected, including representatives from large public school districts, small public school districts, independent schools, and higher education. All participants had strong connections to PK-12 technology. They also all had knowledge and experiences that would contribute significantly to the creation of a framework that considered diverse perspectives and was designed for U.S. education settings.
In between advisory group meetings, participants were asked to read scholarly and media articles about emerging technology and AI/ML-enabled tools in PK-12 education. Based on discussions of the readings, consideration of artifacts prepared by participants, and additional collective work, the advisory group drafted a framework. Once a draft of the framework was complete, we shifted from group work to one-on-one conversations to finalize it. Members of our advisory group reviewed the framework and suggested modifications to the framework through comments or in interviews with researchers. Researchers met individually with group members to resolve the comments and suggestions. The final product from this work at the time of this writing is a preliminary draft version of the emerging technology adoption framework, and we welcome comments and recommendations for modifications of this draft framework.
Now, we will proceed to describe stages of the partnership in more detail, describing (1) launching the partnership, (2) valuing what the communities are already doing and building off of those practices and knowledge, (3) listening to all members of the group with a goal of transformation, and (4) valuing capacity building as well as the limits experienced by our participants. In the next sections, we will describe how we did this.
1. Launching the Advisory Group
Our partnership was very intentional about creating an explicit agreement defining how the group would work together. It took several meetings to agree upon our project’s overarching goals communally. Our effort to reach a partnership consensus allowed all of our members to share a common belief in the value of our work. Additionally, our partners were asked to contribute their expertise and experiences within their unique contexts to make sure that we developed a broadly applicable product. We additionally allowed for the variation of our community members and their experiences to challenge or push forward the groups’ ideas. For example, when partners within this work had differing beliefs about the collection, use, and dissemination of different types of student data, we discussed each of these perspectives and worked to include them all within our framework, allowing districts to evaluate emerging technologies and make their own decisions. The partners on this project brought their perspectives and approached the process of creating solutions in different ways, and that variety significantly enhanced our work.
2. Valuing What Communities Are Already Doing
Within this project, we worked to center the expertise of our advisory group members from the onset. To do this, we explicitly asked participants to share experiences from their local context of procuring and integrating emerging technologies, including challenges that they faced within the technology procurement process. By making connections to what our advisory group members were navigating within their contexts, we were able to see commonalities, such as a desire to balance ease of use with data privacy and student needs. We also noted a repeated concern over whether or not funding for new technology initiatives would allow for continued use of new technology for many years. Further, there were recurring concerns about policies that don’t explicitly block technologies without appropriate data safe-guarding practices.
Examining what these school communities were already practicing in regard to procuring and integrating emergency technology also allowed us to see commonalities and differences. Some of the differences we noted were the duration and level of detail required within the vetting process of new technology, the level of accepted involvement of administrators and school boards, and what types of data are determined to be acceptable for technology to collect (visual data, location data, biometric data). We were able to identify a large number of commonalities and differences within the practices that communities are already doing because we recruited administrators and educators on the ground who have diverse experiences with technology procurement within varying types of schools.
3. Listening to All Members of the Group
In an effort to listen to all members of the advisory group, two cohorts were formed, with each meeting twice a month for five months. The cohorts were co-led by the authors. The first meeting was held in June 2022 and consisted of introductions, the co-creation of group goals, and norm development. The overarching goal was to understand how schools and districts evaluate and make procurement decisions in relation to educational AI systems and create tools to assist in these processes, while paying attention to ethical questions about how student and family data is gathered, used, and stored as well as how AI/ML-powered educational tools and systems are based on data gathered from previous learners and not always designed to work for diverse users (Lecher & Varner, 2021; Wang, 2021).
The smaller cohorts allowed participants to have a chance to share their voices and knowledge through specific examples. One participant, for example, was able to share how their school had been provided with facial recognition surveillance cameras and how they had ethical dilemmas with implementing that type of emerging technology in their settings. All participants in the cohort were able to share their perspectives on the ethics of this surveillance technology and how their districts might be equipped or not to handle a similar adoption. After these sessions, one participant reflected: “I am excited about working and learning with people from different backgrounds.” Another commented, “This was fantastic! Look forward to future conversations.”
Our time with the advisory group led to the creation of a preliminary version of the emerging technology adoption framework. The League of Innovative Schools is an organization of 125 districts serving more than 3.8 million students. As part of their 2022 Fall Convening, we invited League members to comment on this preliminary framework during a 45-minute session as well as through individual feedback forms. As a result of this feedback, the Emerging Technology Adoption Framework: For PK-12 Education was modified. Feedback and comments are currently being accepted by the community regarding the framework as well as its implementation. The team’s next step is to test the framework with a district and integrate the various forms of feedback in ways that transform the framework into a more useful tool that is adaptable in a variety of communities and settings.
4. Valuing Capacity Building and Limits Experienced by Our Participants
While the cohort model allowed us to provide a variety of meeting times for our advisors, we also provided individual meeting options for advisory board members when they were not able to join the larger group. We followed up with advisors via email after every meeting to request additional written feedback, and we also kept a copy of all of that information on an unlisted webpage on the CIRCLS website. At the request of our advisors, we made adjustments to our meeting plans, including adding more individualized check-ins about the framework towards the end; advisors found it easier to meet with the authors and talk through different framework sections while we listened rather than giving feedback with the whole group present. Advisors also requested that we invite outside experts and researchers to give perspectives that might be missing from the group, so we invited a researcher to one of our meetings and also shared drafts of the framework with a range of outside experts, including policy experts, researchers, technology integration specialists, and educators in different contexts.
In the process of creating our advisory group partnership, we encountered some challenges. With so many group members with different contextual needs and within different time zones, finding time to meet among competing priorities was difficult. To overcome this challenge, we asked for detailed weekly availability from each member, then made two smaller working groups and allowed members to switch to the other group if needed due to scheduling. We also adjusted participation to include asynchronous feedback sessions.
The nature of our virtual meeting format might allow participants to multitask rather than focus on the task. To overcome this challenge, we worked to make meetings concise and applicable to the contexts of all members and had individual meetings with advisory board members who seemed distracted or were off task to see how we might better support their participation. To determine that all partnership members were invested and felt this work applied to their contexts and beliefs, we communally agreed upon the goals of the project. Reaching consensus among the participants took time and several meetings; however, once consensus was reached, all members shared a unified belief in what we were attempting to accomplish and supported our work.
Another challenge we faced throughout this project was addressing the needs of all types of contexts (large/small districts, rural/urban/suburban needs, family/community engagement, and independent/public schools). This challenge highlighted that different contexts sometimes have competing needs and desires for technology tools, and school technology ecosystems may be compromised by incompatible tools. While we were able to consider many of these issues in our work to create a framework, we acknowledged within our partnership that some elements of the framework may apply to some contexts more than others.
A final challenge was that we found that districts had varying levels of acceptance of the collection, use, and dissemination of different types of student data; that is, an emerging technology that was considered acceptable in one district may not be acceptable in another. We are still working to address this challenge within our larger partnership.
Lessons Learned and Next Steps
We convened a diverse group of individuals over a five-month period to develop our emerging technology framework. Above we provided extensive details about how we structured the meetings, as a way of providing an example for the field of what it looks like to organize meetings with practitioner partners. As we reflected on our process, we identified the following key ideas for our successful partnership among researchers, teachers, and administrators:
- Collectively agreeing on goals as a starting point for creating working partnerships
- Asking partners to share specific examples of what they have experienced in their local contexts
- Identifying commonalities and differences
- Building off of what is already being done while also allowing for variation across contexts
- Using flexible grouping strategies to accommodate various preferences for remote work
By employing these strategies, we were able to produce a draft framework that can be applied in a multitude of settings and include many context considerations. Through our work with the advisory group, we saw the significant value of districts bringing ideas and proposed changes to their local contexts, and saw value in reflecting as a group on how the implementation is going and what changes might be beneficial. Our future research and design will incorporate both of these practices.
