The Institut Teknologi Sepuluh Nopember Student Branch in Indonesia has set out to help the Yayasan Pembinaan Anak Cacat (YPAC) Surabaya, which is a medical rehabilitation center. The team took note that there were roadblocks that may prevent someone from receiving the best care they can.
The team saw that due to travel costs, patients were not receiving the amount of consistent care they needed. Also, due to the pandemic, there was a restriction on non-urgent medical care to prevent the transmission of COVID-19. They also mention that the therapies offered may have bored the patients, or not encouraged them to come back. To help support this organization, the team created Smart Mobile Inclusive Learning (SMILE): an AI-driven multi-purpose therapy for disabled children.
The SMILE device is meant to provide therapy for both psychomotor and cognitive disabilities. SMILE also has a game-like aspect to make the therapy process more interesting for children. The device uses a human-pose-detection method with machine learning for the psychomotor aspect.
The device uses a camera, mini PC, and a monitor screen. The camera captures the child’s movements when interacting with objects displayed on the monitor. The detection process will use a machine learning method for pose detection, which was developed with the MediaPipe library using the Python programming language.
For the cognitive aspect of SMILE, the device uses audio-visual media based on 3D animation. The media adapts based on the needs of the individual child. For example, the content of the material in the 3D animation learning media will be adjusted to the demands of therapy. There is a clear organization and structure of the media. The material moves at a speed that supports learning. There is clarity in the instruction, along with colors and shapes that give an attractive appearance to keep the attention of the patients.
To ensure the sustainability and maintenance of SMILE, a user’s manual will be provided, and application training. Also, the team hopes to expand and publish the project, and further develop with feedback.
This project was given a grant of $5,950.00 as part of EPICS in IEEE’s Access and Abilities Competition.