UB Awarded $20M Grant to Establish National Institute to Develop AI Technologies to Assist Children with Speech and Language Disabilities – UBNow: News and views for UB Faculty and Staff

UB has received a highly competitive grant from the National Science Foundation to establish a national institute to develop artificial intelligence systems that identify and support young children with language and/or language processing difficulties.

The five-year, $20 million effort will address the nation’s shortage of speech-language pathologists and provide services to children ages 3 to 10 who are at increased risk of falling behind in their academic and socio-emotional development — problems caused by exacerbated by the COVID-19 pandemic.

The award, with which UB will establish the AI ​​Institute for Extraordinary Education, will also advance fundamental AI technologies, human-centric AI design and learning science that improve educational outcomes for young children. It is one of the largest federal research grants to be awarded to UB, New York’s flagship university.

“The University at Buffalo is deeply committed to using our research and educational excellence for the greater good, and our faculty has long been at the forefront of the development of cutting-edge artificial intelligence systems,” said President Satish K. Tripathi. “Being selected by the National Science Foundation to lead this impactful initiative underscores the critical role our world-class researchers play in addressing society’s most pressing challenges, including creating equitable and inclusive environments that enable young learners to unfold.”

Venu Govindaraju, Vice President for Research and Economic Development, is the principal investigator of the grant.

“There simply aren’t enough speech therapists in the United States, and as a result, children aren’t getting life-changing surgeries early enough,” said Govindaraju, SUNY Distinguished Professor in the Department of Computer Science and Engineering, School of Engineering and Applied Sciences. “Our multidisciplinary team will develop advanced artificial intelligence systems that address this critical problem, enabling earlier diagnosis and tailored interventions that close educational gaps and create more inclusive learning environments where children thrive both academically and socially.”

The AI ​​Institute for Exceptional Education will be focused on serving millions of children across the country who need language and language services under the Education Act for Persons with Disabilities.

Specifically, it will develop two advanced AI solutions: the AI ​​Screener for early detection of potential speech and/or language disabilities and disorders, and the AI ​​Orchestrator, which will act as a virtual teaching assistant by providing students with skill-based interventions.

The AI ​​screener will listen to and observe children in the classroom by collecting speech samples, facial expressions, gestures and other data from children. It will create weekly summaries of these interactions, cataloging each child’s vocabulary, pronunciation, video snippets, and more. These summaries help teachers monitor their students’ language and language processing skills and, when necessary, suggest formal assessment by a speech therapist.

This is critical because the earlier language and speech issues are addressed, the greater the likelihood that children will excel academically and socio-emotionally.

The AI ​​Orchestrator is an app that will help speech therapists, most of whom have an average caseload so large that they are forced to offer group-based interventions for children instead of individual care. The app addresses this by recommending personalized content tailored to student needs. It continues to monitor student progress and adjust lesson plans to ensure interventions are working.

The research team ensures that the data collected by the AI ​​Screener and the AI ​​Orchestrator comply with ethics and privacy policies. Information about these plans will be shared with participants. Initially, the team intends to deploy prototypes of each system in approximately 80 classrooms and reach 480 kindergarten children.

The institute will consist of more than 30 researchers from nine universities including UB; the University of Illinois Urbana-Champaign; University at Stanford; the University of Washington; Cornell University; the University of Nevada, Reno; the University of Texas at El Paso; Penn State University; and the University of Oregon.

These scientists specialize in artificial intelligence, natural language processing, social robotics, communication disorders, diversity and inclusivity, learning science, communication, and other fields.

Govindaraju and UB researchers Jinjun Xiong and Srirangaraj Setlur will coordinate the team’s work.

Xiong, Empire Innovation Professor in the Department of Computer Science and Engineering, is Co-Principal Investigator and will serve as the Institute’s Scientific Director and Co-Director. Setlur, Principal Research Scientist at the Department of Computer Science and Engineering, will serve as executive director of the institute.

Additional Co-Principal Investigators are David Feil-Seifer, Associate Professor in the Department of Computer Science & Engineering at the University of Nevada, Reno; Pamela Hadley, professor and department chair in the Department of Speech and Hearing Studies at the University of Illinois Urbana-Champaign; and Julie Kientz, professor and chair of the Department of Human Centered Design & Engineering at the University of Washington.

In addition to Govindaraju, Xiong and Setlur, the institute will include 13 UB researchers representing the School of Engineering and Applied Sciences, the College of Arts and Sciences and the Graduate School of Education.

Letitia Thomas, Associate Dean for Diversity at the Faculty of Engineering and Applied Sciences, will lead the ‘Participation and Diversity’, ‘Equity’ and ‘Inclusion’ areas of the scholarship.

X. Christine Wang, associate dean for research at the Graduate School of Education and director of the Fisher Price Foundation Research Center for Early Childhood Research at UB, will lead the education and human resource development portion of the research and facilitate field testing.

“The pedagogical tools that this interdisciplinary research team will develop have the potential to have a profound impact on the classroom, for both educators and students,” said Wang, professor of early childhood education and learning sciences at the Graduate School of Education.

The award builds on a rich history of AI research at UB dating back decades to when computer scientists including Govindaraju developed a handwriting recognition system that has saved the US Postal Service hundreds of millions of dollars by automating mail sorting.

More recently, UB has made strategic investments in both people and programs to bolster the university’s national leadership in AI, including the establishment of the Institute for Artificial Intelligence and Data Science and the Center for Information Integrity.

“The Institute will introduce new, innovative courses at the intersection of AI and education to prepare the next generation of undergraduate and graduate students to excel in multidisciplinary research that can have broad societal impact,” said Provost A. Scott Weber. “The institute’s groundbreaking research will raise UB’s profile as an AI powerhouse, attracting the best students from around the world and helping the university to raise its rank.”

Tripathi commended Congress leaders for their continued support of UB, whose mission is to bring the benefits of its research and science to local and global communities in ways that impact and positively transform the world.

“We would like to thank US Senate Majority Leader Charles Schumer, Sen. Kirsten Gillibrand and Congressman Brian Higgins for their many years of support of federally funded university research. They have shown time and time again how much they value the ability of academic research to solve real-world problems, and their engagement creates opportunities for people in our state and nation to thrive,” he said.

Other partners are the Buffalo Public Schools; Amherst Central School District, Amherst, New York; Sweet Home Central School District, Amherst, New York; Washoe County School District, Reno, Nevada; The Summit Center, Amherst, New York; Buffalo-Area Minority Engineering; Nevada Robotics, Institute for Desert Research; Gigi’s Playhouse, Buffalo, New York; Western New York STEM Hub; and UB Early Childhood Research Center.