Intelligent Body Pose Tracking for Posture Assessment

Most existing training applications offer good programmes for guiding users to achieve individual fitness goals, some even come with guided video workouts led by professional trainers. However, such applications lack or have limited capability to assess whether the correct posture is maintained during exercise - poor posture can reduce exercise effectiveness and may even cause injury, e.g. arched back during push-ups.

This solution is a synergistic combination of video/image processing, human pose recognition, and machine learning technologies to deliver a solution that addresses the twin challenges of accurate count and correct execution of exercises in an automated manner, without having to wear any additional hardware/sensors. The software-only solution is able to advise users on the correct execution of repetitive movement sequences, e.g. sit-ups, and push-ups, and is deployable on a wide range of affordable camera-enabled hardware devices such as mobile phones, tablets, and laptops, and it can be easily integrated into existing applications to enhance functionality. It is applicable to the sports and healthcare industry to help users perform exercises correctly and effectively in an unencumbered manner.

This software-only solution is able to identify the correct or incorrect execution of repetitive movement sequences using camera inputs from devices such as mobile phones, tablets, or laptops. The solution can be deployed on any Python and JavaScript-capable platform; providing flexibility for deployment across a range of devices and operating systems, including web browsers.

Video/Image Processing

• Pre-processing of video frames as inputs for Human Pose Estimation and Machine Learning
• Built-in algorithm used to assess real-time movement direction, i.e., up, down, right and left, thereby providing inputs for a more accurate count

Human Pose Estimation

• Based on BlazePose, OpenCV and TensorFlow for real-time tracking of up to 32 human pose keypoints, e.g. shoulder, wrist, hip, knee, ankle, etc
• Measurement of angles between pose keypoints as input parameters to determine correct posture, e.g. is the user's back straight, are the knees bent
• Enhances original image from camera with accentuated pose keypoints and connections between keypoints, i.e., stickman diagram, used as input AI classification to determine correct posture.

Machine Learning

• Train and test AI models using human subject video footage for correct and automatic classification of exercises
• AI models classify exercise based on angles and distance of pose landmarks and enhanced images with accentuated pose landmarks and connections between landmarks
   

• Primarily for fitness and healthcare (rehabilitation) industries - can also be applied for any activity that requires assessment of posture
• Repetitive exercise-specific counting e.g. push-up, sit-up, additional exercises can be included, requiring specific customisation
• Posture assessment and analysis of the movement of human subjects - personalised calibration to initial posture can be explored 
• Enables intelligent coaching functionality in fitness/healthcare applications to promote healthy or active living

• Enables real-time posture assessment and monitoring
• Does not require additional body-worn sensors or wearables; simply requires a camera-enabled mobile device
• Provides value added-service to an existing healthcare/fitness application to help end-users perform exercises correctly and effectively in an unencumbered manner 
• Provides remote assessment of exercise without any over-reliance on the expertise of a professional coach

The technology partner is looking for test-bedding opportunities in the fitness and rehabilitation industries, which would involve studying specific exercises individually. Additionally, the technology owner is looking for co-development with companies that have adjacent use for this posture assessment software, e.g. heavy load lifting, combining the software with additional body-worn sensors to detect uneven weight distribution on the workman's back.