–  We have developed a new embodied AI platform, called MyoSuite, that applies machine learning (ML) to biomechanical control problems by unifying motor and neural intelligence. 

–  The musculoskeletal models in MyoSuite are up to 4,000x faster than other simulators at meeting the data requirements of modern ML algorithms

– This research could have significant impact in areas such as developing prosthetics and post-injury rehabilitation, since we can generate physiologically realistic movements, such as twirling a pen or manipulating Baoding balls, in full detail.

– As we build for the metaverse, these models will help developers build avatars that move more realistically, making the experience more expressive and immersive.

The more intelligent an organism is, the more complex the motor behavior it can exhibit. So an important question to consider, then, is — what enables such complex decision-making and the motor control to execute those decisions? To explore this question, we’ve developed MyoSuite: a set of musculoskeletal models and tasks that enable the application of ML to solve biomechanic control problems. MyoSuite unifies the two facets of intelligence: motor and neural. We are releasing MyoSuite and open-sourcing a set of comprehensive benchmarks for the ML community to enable continued research in this area.

Human biomechanics is a complex multijoint, multiactuator musculoskeletal system: A single muscle contraction can bend multiple joints, and each joint movement is controlled by multiple muscles (see image below). Synthesis and expression of intelligent behaviors in such a complex biomechanical system require effective coordination between the central nervous system, where decisions are synthesized by networks of billions of neurons, and the peripheral musculoskeletal system, which translates these intentions into actions.

The field of artificial intelligence has sought to emulate intelligent behaviors via neural structures or neural networks. At the same time, and mostly independently, the biomechanics community has been developing musculoskeletal systems via in-vivo and ex-vivo studies to understand peripheral actuation. With MyoSuite, we’re making progress toward unraveling the details of sensory-motor control, which relies on diverse proprioceptive signals to coordinate the full-body motor control required to exhibit intelligent behaviors in interaction-rich settings. 

MyoSuite’s physiologically realistic musculoskeletal models are orders of magnitude more computationally efficient and scalable than existing models. This allows us to simulate the large number of muscles the human body contains. Finally, we developed and solved a few of the most complex motor control behaviors that haven’t been done before — including dexterous manipulation, like twirling a pen in your hand or turning a key, and have made significant progress on challenging tasks such as the simultaneous manipulation of two Baoding balls in one hand. Now that we can generate physiologically realistic movements in full detail, we anticipate that our results will lead to progress in complex real-world problems like rehabilitation, prosthesis, and ergonomics. 

Applying ML to musculoskeletal motor control 

Modern ML paradigms are helping solve high-dimensional complex problems like Alphago or MuZero, where close-form analytical solutions are not known. ML algorithms leverage the data and scalability of computation to evolve solutions that are otherwise not feasible for human-interpretable solution techniques. But these algorithms are rarely used to explore complex motor control situations such as musculoskeletal control. Why? 

While existing in-silico frameworks such as OpenSim contain physiologically detailed musculoskeletal models, they lack the capability for the complex interaction with the physical world outside the agent’s body. These existing frameworks are neither embedded in complex and skillful motor tasks nor computationally effective or scalable enough to satisfy the data needs of ML algorithms. MyoSuite bridges these gaps.

MyoSuite, an ecosystem for musculoskeletal motor control

MyoSuite is designed from the ground up as a comprehensive platform for investigating the physiological details behind musculoskeletal motor control. The MyoSuite ecosystem includes a comprehensive set of well-tuned, physiologically accurate musculoskeletal models that support temporal interaction-rich musculoskeletal dynamics, such as muscle fatigue or sarcopenia, and carefully designed behavioral tasks exposing motor control challenges that are aligned with real-life situations such as daily tasks, injury rehabilitation, and prosthetic/exoskeleton assistance.

Real-world implications 

Not only can MyoSuite synthesize behaviors, but the comprehensive platform can also facilitate applications with real-life implications, such as rehabilitation, surgery, and shared autonomy assistive devices. 

Let’s consider a tendon-tear injury. First, MyoSuite can easily model tendon-transfer, a well-known technique for recovering functionality due to a torn tendon. MyoSuite also has the ability to simulate the outcome, in terms of mobility, achieved by a given surgical procedure and the impact it will have on the functional rehabilitation of the affected parts, following the surgical procedure. The MyoSuite models can also help inform splint support designs and rehabilitation procedures by highlighting activation patterns and muscle groups.  

Akin to how Pytorch became the de facto platform for neural decision making, we are hoping that MyoSuite will become the de facto platform for unifying motor and neural intelligence. Although we have made progress, we are barely scratching the surface of MyoSuite’s potential. Over time, MyoSuite aspires to become the de facto platform for various fields and domains focusing on AI-driven solutions that leverage musculoskeletal motor control.

Today, we are releasing MyoSuite with physiologically realistic simulations of human anatomy of the arm and hand. These models are embedded in a series of tasks that go well beyond what the biomechanics field is currently capable of. We are also open-sourcing a set of comprehensive benchmarks for the ML community that are an order of magnitude bigger than existing ones in the field. This allows the community to use a larger set of reference models to compare different data-driven solutions. 

To further this ambitious undertaking, we will need help and contributions from the wider community. In addition to open-sourcing MyoSuite, we will soon be launching MyoChallenge, a NeurIPS competition track where we invite the community to participate in solving two of the most difficult dexterity challenges — die reorientation and simultaneous rotation of two Baoding balls. Although the ML community has been able to solve very complex and dexterous tasks, the physiologically realistic tasks that MyoChallenge offers are unique and will prompt a new type of challenge where solutions can be directly measured against human solutions.

MyoSuite is a collaboration between Meta AI and the University of Twente.