How AI can Unlock STEM Content for Students with Disabilities
By Julianna Wright, posted on May 19, 2021This Global Accessibility Awareness Day, Benetech highlights STEM content as the next frontier in information access
Maya is trying her best to complete her algebra homework, but she encounters barrier after barrier. Her digital math textbook is supposed to be accessible; however, she knows that she is missing key information. Maya is blind and uses a screen reader to navigate her textbook. She opens the section that her teacher assigned, and the computer reads aloud:
“We use the quadratic formula to find the solutions to a quadratic equation.”
But instead of reading the formula, the computer reads “Image.” The formula is a graphic that is unreadable by the computer, and there are no alternative text descriptions of the formula.
“Some help that is,” she thinks to herself. She’ll have to google the formula later.
She listens to the rest of the chapter, looking for context clues that might help her figure out the missing formula. No luck. When she gets to the page of homework problems, things only get worse: “Find the solutions to the following equations, using the quadratic formula. One. Image. Two. Image. Three. Image.” There’s no way that she can do this assignment by herself. She’ll have to wait until after her Dad finishes cooking dinner to have him describe the images and equations.
Maya routinely receives the highest marks in her class. However, because she often must rely on others to read the equations, her teachers and classmates already doubt the integrity of her work. She knows that if she could reliably read the equations and problem sets with her screen reader, she could complete this work independently and prove her mastery.
Accessible STEM Materials: The Next Frontier
Education in Science, Technology, Engineering, and Math (STEM) is essential. However, too many students like Maya encounter barrier upon barrier when pursuing an education in these fields because the information is not accessible to them. For 20 years, Benetech has been focused on making information accessible for people with disabilities and learning differences.
When Jim Fruchterman first launched Benetech’s Bookshare platform, he had the vision that optical character recognition software could be used to convert printed text into digital accessible books. Since that time, we’ve been able to significantly advance the conversion of a novel like Harry Potter, or War and Peace, to the point where it is essentially a solved problem. STEM is the next frontier.
Converting STEM education materials remains a challenge. These books are full of images of math and inline math equations that must be properly read aloud for students with disabilities or learning differences to be able to truly access and learn the material. Transforming these books has traditionally required extensive individual labor. The average math textbook contains over 5,000 equations, and it can take three to four months for a human to transform a print math book into accessible formats.
Artificial Intelligence is a Gamechanger
With artificial intelligence, we can reduce the conversion time for accessible math books from months to minutes. The Bookshare engineers have built and trained a neural network to scan images and text in books to identify math. It then feeds the images into a math scanner application that generates the equation in MathML, the accessible math code, with a confidence rating. The equations with the highest confidence ratings can be fed back into the book’s digital file immediately, and the remaining can be reviewed by a human proofreader. The result: a scalable solution for transforming math books into better accessible formats so that when students like Maya open their math textbooks, the computer will read the math correctly, and they’ll be able to complete their work independently, just like their peers.
Only the Beginning
This is only the beginning of the journey. What is starting with algebra equations can expand further to describe chemistry equations, simple graphs, and common diagrams. It is also just as important for publishers to improve their workflows to ensure that the their textbooks are Born Accessible from the start.
With these innovations in artificial intelligence, we are now at the precipice of a new accessibility revolution, one that will ensure that students with disabilities are not limited and discouraged from studying STEM due to access barriers, and are instead empowered to explore STEM courses and career paths to become tomorrow’s great innovators.
Benetech would like to thank our funders for this initiative, including GM, Amazon Web Services, and the US Department of Education Office of Special Education Programs. We also acknowledge and thank the Andrew W. Mellon Foundation for their support of this work through Federated Repositories of Accessible Materials for Higher Education II.
Want more updates on inclusive STEM education? Sign up for our monthly newsletter.
Watch a video of Product Manager, Nick Bowen, to learn more about how Benetech is working to automatically transform STEM content into accessible formats.