The SmartGlove BISINDO Wearable Sensor and Large Language Model–Based Sign Language-to-Speech System for Users With Communication Disorders: Design Science Research (Preprint)
Suhartono Suhartono, Bambang Yulianto, Burhanuddin Arafah, Kusubakti Andajani, Abdul Kholiq, Muhammad Ridha Anugrah Latief, Bambang Prastio, Diding Wahyudin Rohaedi, Deshinta Arrova Dewi, Taqdir Taqdir · JMIR Rehabilitation and Assistive Technologies · 2026
Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.
JMIR Rehabilitation and Assistive Technologies · JMIR Publications · article · 2026
Abstract
Background: In Indonesia, people with communication disorders predominantly communicate using Bahasa Isyarat Indonesia (BISINDO). Nevertheless, all current systems for BISINDO recognition generate fragmented, word-by-word output and lack natural language generation (NLG) capabilities, thereby fundamentally constraining their communicative utility. Objective: The aim of this study is to develop, validate, and field-evaluate SmartGlove BISINDO, an AI-assisted wearable sign language-to-speech system that embeds large language model (LLM)-powered NLG into a BISINDO translation pipeline for the first time.
Methods: This study followed the design science research (DSR) framework, covering phases 1 through 5 (problem identification, solution objectives, prototype design and development, demonstration, and evaluation). Data were collected from 12 purposively selected informants (6 domain experts and 6 deaf education companions/teachers) across 3 Indonesian cities (Malang, Surabaya, and Makassar), using a needs-analysis questionnaire and an expert validation sheet, and analyzed through statistical triangulation of agreement percentage, content validity ratio (CVR), and Aiken V.
The validated prototype was field-tested in Makassar and Malang with 15 participants with communication disorders and 4 observers or companions; it was evaluated by a disability expert in Surabaya. Results: <.05), with 3 items-gesture differentiation, AI contextual relevance, and settings configuration ease-falling below the threshold and indicating areas for refinement. During the testing phase, the system received a lower practicality score (mean 3.35, SD 0.16; 67%), particularly for clarity of speech output. However, observers and companions gave a significantly higher score of 88%, with a mean of 4.40 (SD 0.08).
A disability expert also gave a score of 81% (with a mean value calculated across the 20 questionnaire items of 4.05, SD 0.60). Regarding language, the AI-generated output underwent three transformation processes: addition, deletion, and substitution. Of the 3, substitution is the most likely to alter the intended meaning. Conclusions: SmartGlove BISINDO is a wearable sign language-to-speech system that introduces the BISINDO research landscape to LLM-based NLG connect-and-translate approaches and demonstrates strong content validity.
This study presents preliminary evidence toward a replicable model for interdisciplinary DSR-based prototype development that combines augmentative and alternative communication (AAC) theory, systemic functional linguistics, and design science methodology, intended as a reference for other underrepresented sign language contexts in Southeast Asia and beyond. Field evaluation further indicated favorable practicality overall, alongside a gap between expert-perceived feasibility and user-rated experience that will guide subsequent design refinement.
Open the resolved locator · 10.2196/96258
Metadata licence: OpenAlex metadata (CC0)