Authors:
(1) Clement Lhos, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA;
(2) Emek Barıs¸ Kuc¸uktabak, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA;
(3) Lorenzo Vianello, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA;
(4) Lorenzo Amato, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and The Biorobotics Institute, Scuola Superiore Sant’Anna, 56025 Pontedera, Italy and Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, 56127 Pisa, Italy;
(5) Matthew R. Short, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA;
(6) Kevin Lynch2, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA;
(7) Jose L. Pons, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA. Table of Links Abstract and I Introduction II Methods III Results IV Discussion V. Conclusion, Acknowledgment, and References V. CONCLUSION In this study, we demonstrated the feasibility of employing deep learning in gait state detection for exoskeleton control, eliminating the necessity of ground reaction force sensors. Despite challenges such as a limited dataset, realtime constraints, and issues related to error propagation, these obstacles were successfully addressed. Evaluation of our model’s closed-loop performance versus typical sensors (i.e., force plates, FSR footplates) highlights the system’s applicability across new users and several walking speeds and conditions. To address our model limitations, future work could include training datasets encompassing various walking speeds, overground walking and other activities (e.g. ramps, stairs). Furthermore, this work could be used to implement a state machine that utilizes an estimated stance interpolation factor, selecting impedance parameters in the mid-level controller for exoskeleton control and tailored assistance during walking. The proposed framework will be validated in future on additional healthy individuals as well as individuals with lower-limb impairments (e.g., stroke, spinal cord injury). ACKNOWLEDGMENT This work was supported by the National Science Foundation / National Robotics Initiative (Grant No: 2024488). We would like to thank Tim Haswell for his technical support on the hardware improvements of the ExoMotus-X2 exoskeleton. REFERENCES [1] R. Baud, A. R. Manzoori, A. Ijspeert, and M. Bouri, “Review of control strategies for lower-limb exoskeletons to assist gait,” Journal of NeuroEngineering and Rehabilitation, vol. 18, no. 1, Jul. 2021. [Online]. Available: https://doi.org/10.1186/s12984-021-00906-3 [2] J. de Miguel-Fernandez, J. Lobo-Prat, E. Prinsen, J. M. Font-Llagunes, and L. Marchal-Crespo, “Control strategies used in lower limb exoskeletons for gait rehabilitation after brain injury: a systematic review and analysis of clinical effectiveness,” Journal of NeuroEngineering and Rehabilitation, vol. 20, no. 1, Feb. 2023. [3] E. Basalp, P. Wolf, and L. Marchal-Crespo, “Haptic training: Which types facilitate (re)learning of which motor task and for whom? answers by a review,” IEEE Transactions on Haptics, vol. 14, no. 4, pp. 722–739, 2021. [4] M. Kim, A. M. Simon, and L. J. Hargrove, “Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees,” Wearable Technologies, vol. 3, 2022. [5] W. Huo, S. Mohammed, Y. Amirat, and K. Kong, “Fast gait mode detection and assistive torque control of an exoskeletal robotic orthosis for walking assistance,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1035–1052, 2018. [6] A. Ortlieb, R. Baud, T. Tracchia, B. Denkinger, Q. Herzig, H. Bleuler, and M. Bouri, “An active impedance controller to assist gait in people with neuromuscular diseases: Implementation to the hip joint of the autonomyo exoskeleton,” in 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), 2018. [7] E. B. Kuc¸¨ uktabak, Y. Wen, S. J. Kim, M. R. Short, D. Ludvig, L. Hargrove, E. J. Perreault, K. M. Lynch, and J. L. Pons, “Haptic transparency and interaction force control for a lower-limb exoskeleton,” IEEE Transactions on Robotics, pp. 1–19, 2024. [8] M. Mundt, A. Koeppe, S. David, F. Bamer, W. Potthast, and B. Markert, “Prediction of ground reaction force and joint moments based on optical motion capture data during gait,” Medical Engineering & Physics, vol. 86, pp. 29–34, 2020. [9] A. Karatsidis, G. Bellusci, H. Schepers, M. de Zee, M. Andersen, and P. Veltink, “Estimation of ground reaction forces and moments during gait using only inertial motion capture,” Sensors, vol. 17, no. 12, p. 75, Dec. 2016. [Online]. Available: https://doi.org/10.3390/s17010075 [10] L. Mourot, L. Hoyet, F. L. Clerc, and P. Hellier, “Underpressure: Deep learning for foot contact detection, ground reaction force estimation and footskate cleanup,” Computer Graphics Forum, vol. 41, no. 8, pp. 195–206, 2022. [11] J.-Y. Jung, W. Heo, H. Yang, and H. Park, “A neural network-based gait phase classification method using sensors equipped on lower limb exoskeleton robots,” Sensors, vol. 15, no. 11, pp. 27 738–27 759, 2015. [12] D.-X. Liu, X. Wu, W. Du, C. Wang, and T. Xu, “Gait phase recognition for lower-limb exoskeleton with only joint angular sensors,” Sensors, vol. 16, no. 10, p. 1579, Sep. 2016. [13] C. Camardella, F. Porcini, A. Filippeschi, S. Marcheschi, M. Solazzi, and A. Frisoli, “Gait phases blended control for enhancing transparency on lower-limb exoskeletons,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5453–5460, 2021. [14] V. Lippi, C. Camardella, A. Filippeschi, and F. Porcini, “Identification of gait phases with neural networks for smooth transparent control of a lower limb exoskeleton,” in Proceedings of the 18th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,, INSTICC. SciTePress, 2021, pp. 171–178. [15] J. Fong, E. B. Kuc¸uktabak, V. Crocher, Y. Tan, K. M. Lynch, J. L. Pons, and D. Oetomo, “Canopen robot controller (corc): An open software stack for human robot interaction development,” in Wearable Robotics: Challenges and Trends, J. C. Moreno, J. Masood, U. Schneider, C. Maufroy, and J. L. Pons, Eds. Cham: Springer International Publishing, 2022, pp. 287–292. [16] F. Labarriere, E. Thomas, L. Calistri, V. Optasanu, M. Gueugnon, ` P. Ornetti, and D. Laroche, “Machine learning approaches for activity recognition and/or activity prediction in locomotion assistive devices—a systematic review,” Sensors, vol. 20, no. 21, 2020. [17] M. Kim and L. J. Hargrove, “Deep-learning to map a benchmark dataset of non-amputee ambulation for controlling an open source bionic leg,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 597–10 604, 2022. [18] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. [19] E. B. Kuc¸uktabak, Y. Wen, M. Short, E. Demirbas¸, K. Lynch, and J. Pons, “Virtual physical coupling of two lower-limb exoskeletons,” in 2023 International Conference on Rehabilitation Robotics (ICORR), 2023, pp. 1–6. [20] S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, 2020. [21] R. A. Alamro, A. E. Chisholm, A. M. M. Williams, M. G. Carpenter, and T. Lam, “Overground walking with a robotic exoskeleton elicits trunk muscle activity in people with high-thoracic motor-complete spinal cord injury,” Journal of NeuroEngineering and Rehabilitation, vol. 15, no. 1, Nov. 2018. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Clement Lhos, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA; (2) Emek Barıs¸ Kuc¸uktabak, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA; (3) Lorenzo Vianello, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA; (4) Lorenzo Amato, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and The Biorobotics Institute, Scuola Superiore Sant’Anna, 56025 Pontedera, Italy and Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, 56127 Pisa, Italy; (5) Matthew R. Short, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA; (6) Kevin Lynch2, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA; (7) Jose L. Pons, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA. Authors: Authors: (1) Clement Lhos, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA; (2) Emek Barıs¸ Kuc¸uktabak, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA; (3) Lorenzo Vianello, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA; (4) Lorenzo Amato, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and The Biorobotics Institute, Scuola Superiore Sant’Anna, 56025 Pontedera, Italy and Department of Excellence in Robotics & AI, Scuola Superiore Sant’Anna, 56127 Pisa, Italy; (5) Matthew R. Short, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA; (6) Kevin Lynch2, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA; (7) Jose L. Pons, Legs and Walking Lab of Shirley Ryan AbilityLab, Chicago, IL, USA, Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA and Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA. Table of Links Abstract and I Introduction Abstract and I Introduction II Methods II Methods III Results III Results IV Discussion IV Discussion V. Conclusion, Acknowledgment, and References V. Conclusion, Acknowledgment, and References V. CONCLUSION In this study, we demonstrated the feasibility of employing deep learning in gait state detection for exoskeleton control, eliminating the necessity of ground reaction force sensors. Despite challenges such as a limited dataset, realtime constraints, and issues related to error propagation, these obstacles were successfully addressed. Evaluation of our model’s closed-loop performance versus typical sensors (i.e., force plates, FSR footplates) highlights the system’s applicability across new users and several walking speeds and conditions. To address our model limitations, future work could include training datasets encompassing various walking speeds, overground walking and other activities (e.g. ramps, stairs). Furthermore, this work could be used to implement a state machine that utilizes an estimated stance interpolation factor, selecting impedance parameters in the mid-level controller for exoskeleton control and tailored assistance during walking. The proposed framework will be validated in future on additional healthy individuals as well as individuals with lower-limb impairments (e.g., stroke, spinal cord injury). ACKNOWLEDGMENT This work was supported by the National Science Foundation / National Robotics Initiative (Grant No: 2024488). We would like to thank Tim Haswell for his technical support on the hardware improvements of the ExoMotus-X2 exoskeleton. REFERENCES [1] R. Baud, A. R. Manzoori, A. Ijspeert, and M. Bouri, “Review of control strategies for lower-limb exoskeletons to assist gait,” Journal of NeuroEngineering and Rehabilitation, vol. 18, no. 1, Jul. 2021. [Online]. Available: https://doi.org/10.1186/s12984-021-00906-3 [2] J. de Miguel-Fernandez, J. Lobo-Prat, E. Prinsen, J. M. Font-Llagunes, and L. Marchal-Crespo, “Control strategies used in lower limb exoskeletons for gait rehabilitation after brain injury: a systematic review and analysis of clinical effectiveness,” Journal of NeuroEngineering and Rehabilitation, vol. 20, no. 1, Feb. 2023. [3] E. Basalp, P. Wolf, and L. Marchal-Crespo, “Haptic training: Which types facilitate (re)learning of which motor task and for whom? answers by a review,” IEEE Transactions on Haptics, vol. 14, no. 4, pp. 722–739, 2021. [4] M. Kim, A. M. Simon, and L. J. Hargrove, “Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees,” Wearable Technologies, vol. 3, 2022. [5] W. Huo, S. Mohammed, Y. Amirat, and K. Kong, “Fast gait mode detection and assistive torque control of an exoskeletal robotic orthosis for walking assistance,” IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1035–1052, 2018. [6] A. Ortlieb, R. Baud, T. Tracchia, B. Denkinger, Q. Herzig, H. Bleuler, and M. Bouri, “An active impedance controller to assist gait in people with neuromuscular diseases: Implementation to the hip joint of the autonomyo exoskeleton,” in 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), 2018. [7] E. B. Kuc¸¨ uktabak, Y. Wen, S. J. Kim, M. R. Short, D. Ludvig, L. Hargrove, E. J. Perreault, K. M. Lynch, and J. L. Pons, “Haptic transparency and interaction force control for a lower-limb exoskeleton,” IEEE Transactions on Robotics, pp. 1–19, 2024. [8] M. Mundt, A. Koeppe, S. David, F. Bamer, W. Potthast, and B. Markert, “Prediction of ground reaction force and joint moments based on optical motion capture data during gait,” Medical Engineering & Physics, vol. 86, pp. 29–34, 2020. [9] A. Karatsidis, G. Bellusci, H. Schepers, M. de Zee, M. Andersen, and P. Veltink, “Estimation of ground reaction forces and moments during gait using only inertial motion capture,” Sensors, vol. 17, no. 12, p. 75, Dec. 2016. [Online]. Available: https://doi.org/10.3390/s17010075 [10] L. Mourot, L. Hoyet, F. L. Clerc, and P. Hellier, “Underpressure: Deep learning for foot contact detection, ground reaction force estimation and footskate cleanup,” Computer Graphics Forum, vol. 41, no. 8, pp. 195–206, 2022. [11] J.-Y. Jung, W. Heo, H. Yang, and H. Park, “A neural network-based gait phase classification method using sensors equipped on lower limb exoskeleton robots,” Sensors, vol. 15, no. 11, pp. 27 738–27 759, 2015. [12] D.-X. Liu, X. Wu, W. Du, C. Wang, and T. Xu, “Gait phase recognition for lower-limb exoskeleton with only joint angular sensors,” Sensors, vol. 16, no. 10, p. 1579, Sep. 2016. [13] C. Camardella, F. Porcini, A. Filippeschi, S. Marcheschi, M. Solazzi, and A. Frisoli, “Gait phases blended control for enhancing transparency on lower-limb exoskeletons,” IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 5453–5460, 2021. [14] V. Lippi, C. Camardella, A. Filippeschi, and F. Porcini, “Identification of gait phases with neural networks for smooth transparent control of a lower limb exoskeleton,” in Proceedings of the 18th International Conference on Informatics in Control, Automation and Robotics - Volume 1: ICINCO,, INSTICC. SciTePress, 2021, pp. 171–178. [15] J. Fong, E. B. Kuc¸uktabak, V. Crocher, Y. Tan, K. M. Lynch, J. L. Pons, and D. Oetomo, “Canopen robot controller (corc): An open software stack for human robot interaction development,” in Wearable Robotics: Challenges and Trends, J. C. Moreno, J. Masood, U. Schneider, C. Maufroy, and J. L. Pons, Eds. Cham: Springer International Publishing, 2022, pp. 287–292. [16] F. Labarriere, E. Thomas, L. Calistri, V. Optasanu, M. Gueugnon, ` P. Ornetti, and D. Laroche, “Machine learning approaches for activity recognition and/or activity prediction in locomotion assistive devices—a systematic review,” Sensors, vol. 20, no. 21, 2020. [17] M. Kim and L. J. Hargrove, “Deep-learning to map a benchmark dataset of non-amputee ambulation for controlling an open source bionic leg,” IEEE Robotics and Automation Letters, vol. 7, no. 4, pp. 10 597–10 604, 2022. [18] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. [19] E. B. Kuc¸uktabak, Y. Wen, M. Short, E. Demirbas¸, K. Lynch, and J. Pons, “Virtual physical coupling of two lower-limb exoskeletons,” in 2023 International Conference on Rehabilitation Robotics (ICORR), 2023, pp. 1–6. [20] S. Levine, A. Kumar, G. Tucker, and J. Fu, “Offline reinforcement learning: Tutorial, review, and perspectives on open problems,” arXiv preprint arXiv:2005.01643, 2020. [21] R. A. Alamro, A. E. Chisholm, A. M. M. Williams, M. G. Carpenter, and T. Lam, “Overground walking with a robotic exoskeleton elicits trunk muscle activity in people with high-thoracic motor-complete spinal cord injury,” Journal of NeuroEngineering and Rehabilitation, vol. 15, no. 1, Nov. 2018. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

New Study Shows Exoskeletons Now Adapt to You, Not the Other Way Around

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Discussion

EXOSELF.tech

Too Long; Didn't Read

Get Free Unlimited IP Geo Data With IPinfo Lite

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Conclusion

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

How to Test the Performance of an Exoskeleton

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Introduction

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Methods

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Results

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Discussion

How to Test the Performance of an Exoskeleton

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Introduction

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Methods

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Results

Weight Distribution Estimation in Lower-Limb Exoskeletons via Deep Learning: Discussion

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps