Project SCHOLAR: Supporting Children with Hyperrealistic Oracles for Learning through Augmented Reality (Award #: H327S240009)

Four panel comic displaying a young girl having her facial expression read by an AI tutor which is reacting to her reading with praise.

Introduction:

The ability to read is essential for success, yet gaps in reading achievement are disquieting and stubbornly large for students with disabilities (SWD). SWD, on average, perform significantly lower than their peers without disabilities. Many SWD, but particularly those with SLD and dyslexia, have difficulties in metacognition, working memory, processing speed, and attention. These difficulties can negatively affect the acquisition of literacy skills. For example, difficulties in phonological processing directly impact phonological awareness in learning to read. Thus, SWD require more explicit, systematic, and intensive instruction than their typically developing peers.

AI solutions like SCHOLAR can provide highly scalable, replicable, and refinable interventions that could fundamentally change special education. Project SCHOLAR will (1) refine and test an AR-based intelligent tutoring system that incorporates EBP, universal design for learning (UDL), and culturally responsive pedagogy (CRP), (2) increase access to EBP for SWD using technology, (3) increase literacy for SWD and extend the literature on one-on-one tutoring as an effective reading support for SWD, (4) improve educator use and knowledge of an AI tool to leverage EBP with SWD, (5) improve educator collaboration and professional learning opportunities focused on improving outcomes for SWD, and (6) improve educator and family engagement.

 

Primary Components of the SCHOLAR Technology:

Multimodal Observations of Learners Inform AI Remediations

Overview of multimodal data, like facial expression, that is observed by the AI agent, and the internal process of observing, interpreting, decision making, and then gesturing or speaking.

Learner Model Informed by Co-design with Educators

Side by side images: One with sliders describing a student's personality, the other a scaffolded worksheet for describing a student's characteristics, strengths, barriers, and ways AI might serve them.

Decision Intelligence (DI) Driven Remediation

Visual breakdown of Decision Intelligence into four parts: Actions, Externals, Intermediates, and Outcomes.

Conclusions from Year 1:

When introducing this technology to new sites, there were many critical conversations that could help overcome roadblocks for AI technology integration into spaces. These topics included:

  • Human-Centered AI

  • Building Healthy AI Skeptism

  • AI’s Potential in Rural Spaces

  • AI’s Potential for Supporting Students with or at-risk for Disabilities

When asking: “What challenges and barriers do you expect SCHOLAR will face?” and “How might we improve SCHOLAR’s Quality, Relevance, and Usefulness?”

  • SCHOLAR agent’s look, dialog, expressions, and overall persona

  • the Decision Intelligence design, and limitations we would have to innovate to overcome,

  • Alignment with Science of Reading (UFLI curriculum)

  • Designing for for learner variability, special needs, and classroom realities

Our team 1) developed a survey to assess school implementations of reading programs and began to administer it, 2) conducted initial interviews and focus groups with elementary reading teachers and administrators, and 3) developed and tested a series of test cases to improve and verify the quality of the software.