ArSL21L: Arabic Sign Language Letter Dataset Benchmarking and an Educational Avatar for Metaverse Applications

Ganzorig Batnasan, Munkhjargal Gochoo, Munkh Erdene Otgonbold, Fady Alnajjar, Timothy K. Shih

研究成果: 書貢獻/報告類型會議論文篇章同行評審

摘要

It is complicated for the PwHL (people with hearing loss) to make a relationship with social majority, which naturally demands an interactive auto computer systems that have ability to understand sign language. With a trending Metaverse applications using augmented reality (AR) and virtual reality (VR), it is easier and interesting to teach sign language remotely using an avatar that mimics the gesture of a person using AI (Artificial Intelligence)-based system. There are various proposed methods and datasets for English SL (sign language); however, it is limited for Arabic sign language. Therefore, we present our collected and annotated Arabic Sign Language Letters Dataset (ArSL21L) consisting of 14202 images of 32 letter signs with various backgrounds collected from 50 people. We benchmarked our ArSL21L dataset on state-of-the-art object detection models, i.e., 4 versions of YOLOv5. Among the models, YOLOv5l achieved the best result with COCOmAP of 0.83. Moreover, we provide comparison results of classification task between ArSL2018 dataset, the only Arabic sign language letter dataset for classification task, and our dataset by running classification task on in-house short video. The results revealed that the model trained on our dataset has a superior performance over the model trained on ArSL2018. Moreover, we have created our prototype avatar which can mimic the ArSL (Arabic Sign Language) gestures for Metaverse applications. Finally, we believe, ArSL21L and the ArSL avatar will offer an opportunity to enhance the research and educational applications for not only the PwHL, but also in general real and virtual world applications. Our ArSL21L benchmark dataset is publicly available for research use on the Mendeley.

原文???core.languages.en_GB???
主出版物標題Proceedings of the 2022 IEEE Global Engineering Education Conference, EDUCON 2022
編輯Mohammed Jemni, Ilhem Kallel, Abdeljalil Akkari
發行者IEEE Computer Society
頁面1814-1821
頁數8
ISBN(電子)9781665444347
DOIs
出版狀態已出版 - 2022
事件13th IEEE Global Engineering Education Conference, EDUCON 2022 - Tunis, Tunisia
持續時間: 28 3月 202231 3月 2022

出版系列

名字IEEE Global Engineering Education Conference, EDUCON
2022-March
ISSN(列印)2165-9559
ISSN(電子)2165-9567

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???13th IEEE Global Engineering Education Conference, EDUCON 2022
國家/地區Tunisia
城市Tunis
期間28/03/2231/03/22

指紋

深入研究「ArSL21L: Arabic Sign Language Letter Dataset Benchmarking and an Educational Avatar for Metaverse Applications」主題。共同形成了獨特的指紋。

引用此