A visual-first literacy system for Deaf & Hard of Hearing children
SignBuddy explores how early literacy can be taught to Deaf and Hard of Hearing children without relying on phonics or audio-based instruction. The project translates research from Deaf education into a repeatable digital learning system and validates it through a working proof of concept.
Role: Product Creator & Lead Researcher Tech Stack: Google AI Studio for the POC
<aside> 💡
My journey into accessible education didn't start with code; it started with paper. Previously, I co-authored and designed a physical sign language book aimed at helping beginners learn basic vocabulary. While the book was a valuable resource, I observed a critical limitation during user testing: Sign language is 4-dimensional (3D space + time).
The Books I co-authored and designed
</aside>
For most children, language develops through constant, passive exposure. They overhear conversations at home, in public spaces, and through media. This process often called incidental learning builds vocabulary and meaning long before formal reading instruction begins.
For Deaf and Hard of Hearing (DHH) children, this pathway is often inaccessible. Many cannot rely on spoken language in their environment and therefore miss out on the background exposure that hearing children take for granted. This gap is not caused by cognitive limitations, but by lack of access to language during early development. The impact is most severe between the ages of four and seven, when the brain is highly receptive to forming linguistic and literacy-related structures. When accessible language is delayed during this window, children face long-term challenges in vocabulary growth, reading comprehension, and academic confidence.
The challenge is compounded by the fact that most DHH children are born to hearing parents who may not know sign language at the time of diagnosis. Unlike hearing children, they cannot rely on ambient language while support systems gradually come into place. As a result, early literacy difficulties often emerge before schooling even begins. SignBuddy approaches this as a design problem rather than a learning deficit one that requires rethinking how language and literacy are introduced in a visual-first way from the earliest years.
Most existing tools fail Deaf and Hard of Hearing children not because they lack good intentions, but because they are designed around the wrong assumptions.
Adult-focused sign language apps prioritize grammar, sentence structure, and conversational fluency. They assume reading ability, sustained attention, and abstract reasoning. For pre-literate children, especially those still forming a first language, this creates unnecessary cognitive load and turns learning into a task rather than an exploration.
Research Paper: *Fingerspelling and Print: Understanding Word Reading of Deaf Children* discusses fingerspelling and reading literacy in DHH children.
Research indicates that alternate coding systems like fingerspelling are being explored because traditional sound-based literacy approaches often lack efficacy for DHH learners. (Link to Scott et al., 2019)
At the opposite end are “baby sign” apps and static sign dictionaries. These treat signs as isolated gestures meant for basic communication rather than as part of a complete linguistic system. While they may help with early expression, they do not support vocabulary depth, progression, or the transition from sign to written language.
Mainstream literacy apps introduce a different barrier. They rely heavily on phonics—sound-to-letter associations that presume auditory access. Even when visually adapted, sound remains the conceptual foundation. For DHH children, this makes literacy indirect and abstract rather than grounded in accessible language.
