Spinometer

Spinal Alignment / Posture Estimator Based on Face-to-display Distance and Head Tilting Angle

This program estimates spine posture based on facial distance and angle. The algorithm is a proprietary patent, and this software serves as a demonstration of this patented technology.

Features

The demo app:

  • Requires only webcam and is completely non-invasive. No need for X-Ray, spinal mouse or any other special devices.

  • Made with Unity and Sentis.

The algorithm:

  • Estimates spinal alignment based on eye-to-display distance and head tilting angle.

Download

Usage

  1. Upon launching, you will see the initial setup screen.

  2. Follow the setup process:

    1. Select your webcam device from the dropdown and press "Next".
    2. Set the relative angle between the webcam and your face (face the camera directly and press "Next").
    3. Set the webcam's field of view (position your face 50cm from the camera and press "Next").

  3. After setup, the program will begin normal operation.

  4. To redo the setup, press "Settings" in the bottom right corner.

Note: Setup information is saved in the registry. If you uninstall the program, this data will preserved. To remove it completely, use the Registry Editor.

How It Works

  1. Preparation

    1. An image frame is grabbed from the webcam.
    2. The image frame is preprocessed (i.e. scaling, flipping and gray scaling).
    3. The face is fit in the frame, the head pose (distance from eyes to camera, and tilting angle of the head) is calculated.

  2. Spinal alignment estimation by the proprietary algorithm

    1. Based on the distance and the tilting angle calculated in previous step, the spinal sagittal (front/back direction) alignment is estimated. See Background section (below) for more information. This software is optimized to the situation that the user is using a laptop in a sitting position.

  3. Rendering

    1. The estimated spinal alignment is converted to absolute positions and angles of each spinal segment by forward kinematics.
    2. The skeleton is rendered, the arm positions and angles are calculated by inverse kinematics while hands, hips and legs are fixed.

Background

Spinal sagittal alignment

Humans, unlike other vertebrates, are bipedal animals, standing and walking upright. The human spine has what is called a physiological curve, which, when viewed from the side, includes a forward curve in the cervical spine, a backward curve in the thoracic spine, and a forward curve in the lumbar spine. This balance allows the heavy head to be positioned directly above the pelvis, enabling good posture without strain. This specific alignment of the spine in the anterior-posterior direction is referred to in medical terms as "spinal sagittal alignment."

In recent years, abnormalities in spinal sagittal alignment have been recognized as contributing to various health conditions. Especially in the wake of the COVID-19 pandemic, the surge in remote work and web meetings has significantly increased the time spent sitting and using computers. Poor posture during these activities has been linked to a range of problems, including neck and lower back pain.

Accurate measurement of spinal sagittal alignment traditionally required devices such as X-rays and fluoroscopy, which use radiation and are impractical for continuous monitoring in daily life or work environments. While various wearable devices have been developed to detect neck tilt and similar metrics, none have undergone rigorous medical validation.

Formulation

In collaboration with Dr. Ko Hashimoto, a chief spine surgeon from the Department of Orthopaedic Surgery at Tohoku University, one of Japan's leading national universities, we have established an algorithm that estimates spinal sagittal alignment by calculating the distance between the screen and the eyes (visual distance) and the tilt angle of the head (head angle) during PC use. This algorithm was derived through the use of a 3D motion analysis device with a spatial resolution of 1 mm and a temporal resolution of 10 ms, as well as actual measurements from X-ray images, and multivariate analysis incorporating subject attributes such as gender and body type.

Additionally, we have successfully developed software that estimates spinal sagittal alignment solely from webcam images, without the need for any wearable devices, by integrating a processing mechanism that uses AI to recognize both the distance and head tilt angle from facial images captured by a webcam. This technology has potential applications not only for individuals working at a PC but also for posture estimation in various scenarios.

From the experiments described above, we have derived a unique estimation formula that predicts the sagittal alignment of various sections of the spine based on explanatory variables such as visual distance, head angle, and various physical characteristics. This has enabled us to achieve high accuracy and responsiveness in posture estimation, as demonstrated in the software. The results of this research have been submitted for academic publication and are expected to be released soon.

Furthermore, the technology and algorithms have already been patented in Japan (Patent No. 7007777), with patent applications pending in the United States, Europe, and China.

Patents

  • JP7007777B1,"A computer-readable recording medium recording a spinal alignment estimation device, a spinal alignment estimation system, a method of operating the spinal alignment estimation system, a spinal alignment estimation program, and a spinal alignment estimation program.", Yusei Takahashi, Ko Hashimoto, Tohoku University NUC, 2022-01-25, Japan.

Thanks

  • opentrack authors and maintainers, as we adopted their code of neuralnet tracker module as a crucial part of Spinometer.

Download

Releases

Source Code

The source code is available on GitHub (https://github.com/Spinometer/spinometer-demo).

In addition to Unity and related packages, the app depends on following third party assets and files: