DEEP LEARNING FOR GESTURE RECOGNITION IN GYM TRAINING PERFORMED BY A VISION-BASED AUGMENTED REALITY SMART MIRROR

Authors

Bernardo Lanza, Università di BresciaFollow
Cristina Nuzzi, University of BresciaFollow
Simone Pasinetti, University of Brescia, ItalyFollow
Matteo Lancini, University of Brescia, ItalyFollow

Category

Imaging

Document Type

Paper

Abstract

This paper illustrates the development and the validation of a smart mirror for sport training. The application is based the skeletonization algorithm MediaPipe and runs on an embedded device Nvidia Jetson Nano equipped with two fisheye cameras. The software has been evaluated considering the exercise biceps curl. The elbow angle has been measured by both MediaPipe and the motion capture system BTS (ground truth), and the resulting values have been compared to determine angle uncertainty, residual errors, and intra-subject and inter-subject repeatability. The uncertainty of the joints’ estimation and the quality of the image captured by the cameras reflect on the final uncertainty of the indicator over time, highlighting the areas of improvements for further developments.

Recommended Citation

Lanza, Bernardo; Nuzzi, Cristina; Pasinetti, Simone; and Lancini, Matteo (2022) "DEEP LEARNING FOR GESTURE RECOGNITION IN GYM TRAINING PERFORMED BY A VISION-BASED AUGMENTED REALITY SMART MIRROR," ISBS Proceedings Archive: Vol. 40: Iss. 1, Article 87.
Available at: https://commons.nmu.edu/isbs/vol40/iss1/87