The goal of MediHopps is to develop an intelligent full body holoportation for stand-alone XR glasses. These precisely capture body postures and poses and transfer them to digital avatars in the virtual world.
1
2
The intelligent assistance system classifies the detected movements and recognizes incorrectly performed exercises.
3
In this way, MediHopps creates an easily accessible, supervised, qualitatively controlled and location-independent rehabilitation sports training including positive group dynamic effects.
In a nutshell
What?
The MediHopps project realizes for the first time a full body holoportation for XR glasses, which enables precise tracking of the entire body in XR without the need for high-priced external hardware. Wearing the multisensory glasses enables precise tracking of body postures & movements and transferring them to digital avatars. An XR platform supports collaborative execution of rehabilitation sports sessions, through quality coaching from a trainer.
In combination with an AI-supported rehabilitation sports assistance system, rehabilitation exercises are classified with regard to correct exercise. Users and trainers receive the assessment as real-time feedback. The goal of the development is to create a sensor-reduced modular XR group collaboration room for medically precise telerehabilitation. The project initially focuses on dynamic rehabilitation exercises in the application fields “mobilization” and “strengthening” due to their effectiveness. However, the solution can be used for all equipment-free exercises, e.g. also in stretching or balance exercises.
How?
The execution of rehabilitation sports units with appropriate XR sensor technology (integrated cameras, IMU sensors) collects and processes training data for the implementation of the project. Existing XR goggles have several integrated, cleverly placed cameras that capture the entire front of the wearer’s body in the field of view. Thus, they are very well suited for the realization of a full body holoportation system. This data, consisting of movements and body poses, is used for training an artificial intelligence.
The evaluation of visual information (angle of view, visible body parts), IMU information (acceleration, velocity and angle information) as well as a time series analysis (change over defined time axes) enables the derivation of a body pose and movement. MediHopps develops an essential innovation module of the Holoportation system based on the latest approaches of the 3D human body model for anatomically correct and precise transfer of body postures and movements to 3D avatars. This innovation module enables a collaborative, virtual and controlled rehabilitation sport in XR training rooms. As a value-added service, a Rapid Content Creation module offers low-threshold creation of new virtual rehab exercises by the trainer in the system.
Initial situation
More and more people are suffering from chronic degenerative diseases and are dependent on continuous care. The supply bottlenecks in rural regions pose a particular challenge.
Demographic change is increasing the need for medical rehabilitation sports, both in aftercare and in prevention. People with limited mobility often do not have the opportunity to participate in suitable offers, especially in peripheral areas.
Telemedical solutions
against supply bottleneck
Community spirit
in the virtual training room
The August-Wilhelm Scheer Institute develops the following components:
Through the competences in the fields of artificial intelligence as well as multisensor technology, we take over the conception and development of the artificial intelligence for full body holoportation as well as the classification and recognition of rehabilitation exercises. In addition, we define the requirements for the necessary training data collected in the data collections and support the organization and implementation of the data collections.
Partner
FUNDING NOTICE
The MediHopps project is funded by the German Federal Ministry of Education and Research.
Funding code: 16SV9058
Duration: 01.08.2022 – 31.07.2025