Graduate work with the Mixed Reality Group for Stroke Rehabilitation



Context

Interactive neurorehabilitation (INR) refers to the application of any interactive system to neurorehabilitation therapies (e.g. for stroke, traumatic brain injury, Parkinson’s disease). INR systems track and analyze movement of a patient practicing functional activities (like reaching to grasp an object) and provide interactive feedback on performance (for example, how directly one reached a target). The benefit of INR systems is that they offer accurate and detailed means for evaluating movement quality, and based on such analysis, generate augmented feedback on aspects of movement that are otherwise potentially difficult for a patient to monitor individually or in parallel (e.g. monitoring wrist location with respect to torso movement while reaching).

While large-scale evidence supporting the best methodology for INR system design is currently lacking, it is proposed that information from other disciplines can be used to accelerate design and testing of INR.

Contribution

My graduate work focused on designing feedback for INR by applying interdisciplinary concepts derived from external but related fields, such as motor learning and rehabilitation, interactive learning, media arts, and interactive computing. My dissertation proposed these concepts and applies them with compositional guidelines (a series of suggested design constraints and approaches) for authoring visual media for INR systems for supervised training, unsupervised training, and assisted reflection.

While the interdisciplinary concepts provided (such as multimodality, representation, adaptability, form coherence) are specifically applied to INR and stroke rehabilitation, such an approach may also assist the development of media for other interactive learning contexts. INR presents a diverse sampling of design challenges, including a user base of various levels of ability, and activity training that ranges in terms of type, supervision level, location, and timescale.

Compositional guidelines are proposed to help media designers (1) tailor feedback to a particular phase of interactive learning while still support an experience that both (2) fits within the overall rehabilitation training continuum and (3) allows for movement in any direction along this continuum. The desired aggregate effect of using systems designed with these guidelines is for the patient to gradually achieve a complex learning goal while reducing dependence on augmented feedback.

Implementation specific contributions to the INR systems shown above

System for supervised training: A visual summary of the patient’s trajectory error displayed following the AMRR system’s real-time particle feedback (designed by Loren Olson) to enhance the patient’s offline reflection on his reach performance of an individual reach and grasp. The summary was designed to translate a continuous experience (particles forming an image) into a static summary that could communicate magnitude and direction of error over time. More information on the overall system design can be found here and here, and results comparing this system to traditional rehabilitation can be found here.

alt text

alt text

System for unsupervised training: A multi-level visual environment that responds to multiple aspects of the patient’s movement performance across various types of training tasks, structures of activity, and training sensitivities. The visual environment also uses various feedback design strategies to communicate movement performance, ranging from direct performance display to abstract inference of a narrative, to help facilitate self-evaluation over longer timescales of training. Multiple contexts of training are linked using an overarching narrative to connect physical action to several levels of visual summary, as well as gradually facilitate the patient’s self-assessment. More information on the overall system design can be found here. A paper on the results from a pilot study evaluating this system is forthcoming.

alt text




Related papers

Dissertation

  • N Lehrer, Applied Interdisciplinary Concepts for Designing Visual Media Within Interactive Neurorehabilitation Systems, Arizona State University, Dec 2014. Link

Journal

  • M Baran, N Lehrer, M Duff, V Venkataraman, P Turaga, T Ingalls, Z Rymer, SL Wolf, T Rikakis, “Interdisciplinary concepts for design and implementation of mixed reality interactive neurorehabilitation systems for stroke.” Physical Therapy Nov 2014 (E-pub), March 2015 (Print)

  • T Rikakis, A Kelliher, N Lehrer, “Experiential Media and Digital Culture.” IEEE Computer 46(1): 46-54, Jan 2013. Link

  • N Lehrer, S Attygalle, SL Wolf, T Rikakis. “Exploring the bases for a mixed reality stroke rehabilitation system, Part I: A unified approach for representing action, quantitative evaluation, and interactive feedback.”, Journal of NeuroEngineering and Rehabilitation, 8(51) 2011. online

  • N Lehrer, Y Chen, M Duff, SL Wolf, T Rikakis. “Exploring the bases for a mixed reality stroke rehabilitation system, Part II: Design of Interactive Feedback for upper limb rehabilitation.” Journal of NeuroEngineering and Rehabilitation, 2011 8(54) 2011. online

  • Y Chen, M Duff, N Lehrer, SM Liu, P Blake, SL Wolf, H Sundaram, and T Rikakis, “A Novel Adaptive Mixed Reality System for Stroke Rehabilitation: Principles, Proof of Concept and Preliminary Application in Two Patients”, Topics in Stroke Rehabilitation 2011, 18(3):212–230. pdf

Conference

  • V Venkataraman, P Turaga, N Lehrer, M Baran, T Rikakis, and SL Wolf, “Attractor-Shape for Dynamical Analysis of Human Movement: Applications in Stroke Rehabilitation and Action Recognition”, IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2013. pdf

  • D Siwiak, N Lehrer, M Baran, Y Chen, M Duff, T Ingalls, and T Rikakis, “A Home-based Adaptive Mixed Reality Rehabilitation System,” Proceedings of ACM Multimedia International Conference (ACM MM), Scottsdale, AZ. November 28 - December 1, 2011. pdf

  • N Lehrer, T Rikakis, and SL Wolf, Presentation: “Experiential Biofeedback & Adaptive Learning Using Mixed Reality for Stroke”, Association for Applied Psychophysiology and Biofeedback (AAPB) 42nd Annual Meeting, March 11, 2011, New Orleans.

  • M Baran, N Lehrer, D Siwiak, Y Chen, M Duff, T Ingalls, and T Rikakis. “Design of a home-based adaptive mixed reality rehabilitation system for stroke survivors.” 33rd Annual International IEEE EMBS Conference, Boston, Massachusetts, Aug. 30 - Sept. 3, 2011. pdf

  • Y Chen, M Duff, N Lehrer, H Sundaram, J He, SL Wolf, and T Rikakis T, “A Computational Framework for Quantitative Evaluation of Movement during Rehabilitation”, International Symposium on Computational Models for Life Sciences, Tokyo, Japan, 22-24 June 2011. pdf

  • Y Chen, N Lehrer, H Sundaram and T Rikakis. “Adaptive Mixed Reality Stroke Rehabilitation: System Architecture and Evaluation Metrics”. 1st Multimedia Systems Conference (MMSys 2010). Feb. 2010. Scottsdale, AZ. pdf