Quattro: Motion Detection

Overview

The Quattro project is a concept proof that approaches Shake or Tremor Sensing using Motion Detection on Apple Devices, specifically Apple Watch.

Based on the generic nature of the metrics available from CMDeviceMotion samples on both iOS and watchOS, although the target clinical objective is wearable on wrist Watch data, the phone provides a useful test instrument.

This includes detection of motion parameters using CoreMotion including acceleration, rotation, pitch, roll - otherwise known as Intertial Measurement Unit (IMU) data sets.

Problem

The clinical context would be data capture of motion artifacts such is in the context of a neurological disorder such as Parkinson’s Disease or Essential Tremors.

Approach

After a substantial amount of background research model has been implemented in Python, converted to a CoreML package and wired up for live inference at time of capture on the Apple Watch, returning results.

What is unique about the inference model is that it is a hybrid neural network and the algorithm takes several of the IMU values into account. The objective and intent was to take a carefully chosen, narrow window to shorten inference time interval - window and (performance) cost.

The result was distilled down to a SHAKE yes or no with a confidence number on device. The application system also included the ability to upload to a backend for orhestrated post processing including a repeated inference pipeline run with the original Python native model implementaiton.

Outcome

This was built using SwiftUI, Swift, CoreMotion, and a CoreML model package for inference implemented in Python and converted. The web user interface was designed and implemented in TypeScript and JavaScript using astro.js.

Read more about it at a summary page located here.