Transformer-Guided Monocular 3D Fitness Pose Analysis with Biomechanical Constraints and Privacy Preservation

Abstract

Monocular 3D pose estimation holds great promise in fitness and rehabilitation settings, but various challenges arise, including privacy concerns, interpretability limitations, and user and environment-dependent posture differences. The proposed approach is a comprehensive framework for fitness pose analysis, based on monocular RGB input, adopting the transformer architecture for 3D pose estimation, as well as a biomechanical classification and privacy-guarding output for the whole pipeline. The methodology includes image enhancement, reconstruction using a transformer, evaluation of the joint angles, watermarking and encryption of the data for postural assessment and secure data management. The framework was evaluated with a customised dataset of squat, plank and push-up exercises using correct and incorrect forms to determine the reliability of the system in performing real-life tasks. The proposed system was able to attain an accuracy of 72.73% with the implementation of XGBoost using a subject-independent 5-fold cross-validation strategy. This accuracy may seem average, but it is still under the controlled condition where there is a significant inter-subject variability. While the previous approaches relied on a controlled dataset and powerful deep learning classifiers, the proposed system places emphasis on the interpretability of these classifiers, computational efficiency, and practicality of deployment. Furthermore, it is shown that the framework can be applied to cross domains, specifically within the zero-shot evaluations of Human3.6M and MPI-INF-3DHP datasets, which is an important test, since no “re-training” operations are performed. This generalisation capability is further validated in a real-world, zero-shot setting via qualitative evaluations based on images not seen before that are publicly available. This shows the feasibility of developing such lightweight, easily-interpretable and privacy-preserving 3D fitness analysis systems without the need for specialised sensing devices.