Audio Haptic Exploration

In the past few decades, STEM opportunities for the general population have increased exponentially. However, insufficient resources, lack of proper infrastructure, shortage of specially skilled teachers, and socio-economic issues restrict the BVI from accessing STEM education beyond middle school. This deprives the BVI children of acquiring fundamental problem solving and critical thinking skills, crucial for academic and professional development.

The project aims at enabling accessible STEM learning for the blind and visually-impaired (BVI) in the Global South through the use of assistive technologies.

Our goal is to facilitate mathematical thinking by developing a low-cost, user-driven, cooperative, playful, and accessible learning tool that can run on economically viable smartphones running Android OS. Taking inspiration from the fundamentals of ludic design, we leverage sensory perceptions of sound (spatial audio), as well as touch (haptic feedback) in BVI to facilitate a gamified, playful, and engaging learning experience. During the span of the project, we aim to achieve the following three goals. First, to conduct a preliminary investigation by teaching vector basics through sound (direction) and haptics (magnitude) and evaluating the learning outcomes. Second, we will investigate and evaluate the learning outcomes of educating the BVI about 2D and 3D geometry. Finally, we will design a ludic design accessible interaction framework for basic and advanced mathematical learning.