Search papers, labs, and topics across Lattice.
To eliminate the need for cumbersome wearable sensors while overcoming severe visual self-occlusion during continuous turning tasks, this paper presents a cross-modal subspace distillation framework for Parkinson's freezing-of-gait (FOG) detection. The authors train a dual-stream visual model that probabilistically balances skeletal graph nodes and raw pixel representations under the supervision of latent topologies extracted from a pre-trained IMU kinematic oracle. Evaluated on multimodal continuous $360^\circ$ turning sequences, the approach dynamically shifts reliance to continuous pixel boundaries during joint-tracking failures, achieving wearable-grade FOG prediction accuracy in entirely camera-only deployment settings.
Wearable-free video models can match the precision of physical IMU sensors during severe turning-induced self-occlusion by distilling kinematic topologies directly into visual latent spaces.
Parkinson's disease is a progressive neurodegenerative disorder characterised by gradual deterioration of movement control. Automated freezing-of-gait (FOG) detection supports the objective assessment of gait-related motor impairment. Two common approaches are used for FOG prediction: (i) analysing video recordings of the patient's movements and (ii) analysing data collected using inertial measurement unit (IMU) wearable sensors attached to the patient's lower limbs. Video-based approaches may suffer detection errors during continuous turning-in-place tasks because the lower limbs undergo substantial geometric self-occlusion, degrading pose-estimation accuracy. IMU-based approaches are generally less affected by visual occlusion; however, they are difficult to deploy outside clinical or laboratory settings, as the sensors must be attached securely and remain in place throughout the assessment. Motivated by this, we propose a cross-modal subspace distillation framework to mitigate the limitations of unimodal FOG detection by combining IMU accuracy with video-based practicality. We extract invariant latent topologies from a pre-trained kinematic oracle to structurally supervise a non-encoded visual architecture during training. To resolve periods of severe spatial occlusion, a dual-stream visual model probabilistically fuses skeletal graph nodes and continuous spatial pixels, dynamically shifting reliance to uninterrupted pixel boundaries as joint tracking confidence drops. Evaluated against a public, multi-modal sequence dataset of Parkinson's individuals executing continuous $360^\circ$ turns, empirical results demonstrate that applying sensory boundary topologies strictly mitigates tracking evaluation entropy. Our constrained optimisation confirms that highly precise FOG prediction bounds can be achieved over zero-wearable inference environments.
