The development of a wearable device for Sign Language Gesture Recognition (SLGR) by using Conceptual Design Approach (CDA)

Sarika Zuhri; Syahriza Syahriza; Teuku Andhika Malik Rahman; Rizki Agam Syahputra; Iskandar Hasanuddin

doi:10.30811/jpl.v21i4.3744

The development of a wearable device for Sign Language Gesture Recognition (SLGR) by using Conceptual Design Approach (CDA)

Sarika Zuhri, Syahriza Syahriza, Teuku Andhika Malik Rahman, Rizki Agam Syahputra, Iskandar Hasanuddin

Abstract

In recent years, there has been a significant focus on researching and developing Sign Language Gesture Recognition (SLGR) for people with hearing and speaking impairments. This is especially important in Indonesia, where there are approximately two million people with these disabilities. However, current research and developments on SLGR devices are specifically designed to understand only particular sign language systems, such as Korean Standard Sign Language (KSDSL) and American Sign Language (ASL), each of which has its unique gestures and models. As a result, no device has been developed to recognize the gestures of the Indonesian sign language system, known as Sistem Isyarat Bahasa Indonesia (SIBI). Therefore, this study aims to develop an SLGR device that can recognize and translate SIBI gestures into output images via text and speech. The development of the SLGR device in this study is conducted by using the Conceptual Design Approach (CDA) methodology. Where in this case, previous research on SLGR devices is first observed as a benchmark for comparison. Furthermore, the benchmark is used as the basis for the function, sub-function, and specification of the proposed design. Based on these stages of benchmarking and concept development, the study concluded that the final design of the proposed SLGR device is constructed by using 5 flex and Gyroscope sensors connected wirelessly to the Raspberry microcontroller. The device is equipped with a voice system and LCR RPi as the output system for translation. Based on the combination of these sensors, the device is able to identify any particular gestures that correspond to words and phrases in SIBI and translate them into speech via the designated speaker and text display on the LCD screen. To fully understand the performance of the device, experimental tests are conducted by analyzing the input of 26 alphabets in the SIBI system. As a result, the device demonstrated an average of 92% accuracy to convert sign language into voice and text, which demonstrates the usefulness of the proposed device

Keywords

Sign Language, Sign Language Gesture Translator (SLGR), Conceptual Design, SIBI

Full Text:

PDF

References

H. M. Hurlbut, “The Signed Languages of Indonesia: An Enigma,” 2014.

A. Q. Baktash, S. L. Mohammed, and H. F. Jameel, “Multi-Sign Language Glove based Hand Talking System,” IOP Conf Ser Mater Sci Eng, vol. 1105, no. 1, p. 012078, Jun. 2021, doi: 10.1088/1757-899X/1105/1/012078.

Ridwang, Syafaruddin, A. A. Ilham, and I. Nurtanio, “Indonesian Sign Language Letter Interpreter Application Using Leap Motion Control based on Naïve Bayes Classifier,” IOP Conf Ser Mater Sci Eng, vol. 676, no. 1, p. 012012, Nov. 2019, doi: 10.1088/1757-899X/676/1/012012.

A. Z. Shukor, M. F. Miskon, M. H. Jamaluddin, F. bin Ali@Ibrahim, M. F. Asyraf, and M. B. bin Bahar, “A New Data Glove Approach for Malaysian Sign Language Detection,” Procedia Comput Sci, vol. 76, pp. 60–67, 2015, doi: 10.1016/j.procs.2015.12.276.

[[5] S. A. Mehdi and Y. N. Khan, “Sign language recognition using sensor gloves,” in Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP ’02., 2002, pp. 2204–2206 vol.5. doi: 10.1109/ICONIP.2002.1201884.

[[6] T. Handhika, R. I. M. Zen, Murni, D. P. Lestari, and I. Sari, “Gesture recognition for Indonesian Sign Language (BISINDO),” J Phys Conf Ser, vol. 1028, p. 012173, Jun. 2018, doi: 10.1088/1742-6596/1028/1/012173.

G. Pahl, W. Beitz, J. Feldhusen, and K.-H. Grote, Engineering Design. London: Springer London, 2007. doi: 10.1007/978-1-84628-319-2.

H. Ghael, “A Review Paper on Raspberry Pi and its Applications,” 2020, doi: 10.35629/5252-0212225227.

L. R. Nendauni, “THE DEVELOPMENT OF SIGN LANGUAGE: A SYNOPSIS OVERVIEW.” Aug. 2021. doi: 10.13140/RG.2.2.19207.93609.

A. Hiddinga and O. Crasborn, “Signed languages and globalization,” Language in Society, vol. 40, no. 4, pp. 483–505, Sep. 2011, doi: 10.1017/S0047404511000480.

L. A. F. Fernandes, V. F. Pamplona, J. L. Prauchner, L. P. Nedel, and M. M. Oliveira, “A Conceptual Image-Based Data Glove for Computer-Human Interaction,” Revista de Informática Teórica e Aplicada, vol. 15, no. 3, pp. 75–94, Apr. 2009, doi: 10.22456/2175-2745.8265.

Y. R. Garda et al., “Flex Sensor Based Biofeedback Monitoring for Post-Stroke Fingers Myopathy Patients,” J Phys Conf Ser, vol. 1007, p. 012069, Apr. 2018, doi: 10.1088/1742-6596/1007/1/012069.

S. Alapati and S. N. Yeole, “A Review on Applications of Flex Sensors,” International Journal of Emerging Technology and Advanced Engineering, vol. 7, pp. 97–100, Aug. 2017.

K. T. Ulrich and S. D. Eppinger, Product Design and Development: Fifth Edition, Fifth Edit. 2012. doi: 10.1016/B978-0-7506-8985-4.00002-4.

A. Z. Shukor, M. F. Miskon, M. H. Jamaluddin, F. bin Ali Ibrahim, M. F. Asyraf, and M. B. bin Bahar, “A New Data Glove Approach for Malaysian Sign Language Detection,” in Procedia Computer Science, 2015, vol. 76, pp. 60–67. doi: 10.1016/j.procs.2015.12.276.

M. S. and M. S. and K. J. and A. C. Kushwah, “Sign Language Interpretation Using Pseudo Glove,” in Proceeding of International Conference on Intelligent Communication, Control and Devices, 2017, pp. 9–18.

[[17] Syahriza et al 2020, IOP Conf. Ser.: Mater. Sci. Eng. 931 012003. DOI 10.1088/1757-899X/931/1/012003

M. A. Ahmed, B. B. Zaidan, A. A. Zaidan, Mahmood, M. Salih, Z. T. Al- Qaysi, A. H. Alamoodi. (2020). Based on Wearable Sensory Device in 3D printed Humanoid : A New Real Time Sign Languange Recognition System, 168. https://doi.org/10.1016/j.measurement.2020.108431

[[19] Kushwah, M. S., Manish Sharma, Kunal Jain, Anish Chopra (2017). Sign Languange Interpretation Using Pseudo Glove. In : Singh, R., Choudhury, S. (eds) Proceeding of International Conference on Intelligent Communication, Control and Devices. Advances in Intelligent Systems and Computing, vol 479. Springer, Singapore. https://doi.org/10.1007/978-981-10-1708-7_2

[[20] Braun, Andreas & Wichert, Reiner & Kuijper, Arjan & Fellner, Dieter. (2014). A Benchmarking Model for Sensors in Smart Environments. 8850. 10.1007/978-3-319-14112-1_20.

DOI: http://dx.doi.org/10.30811/jpl.v21i4.3744