AR in the OR: First augmented reality shoulder replacement surgery in Texas
May 27, 2021
For several years, augmented reality (AR) has been a crucial part of pre-operative planning for shoulder replacement. Creating 3D models with sophisticated software, we have been able to develop precise arthroplasty procedures that preserve more of the patient's natural anatomy.
Now, we're taking this innovation a step further: In April 2021, we performed the first augmented reality shoulder replacement in Texas.
Wearing AR headsets in the operating room, we were able to virtually overlay the 3D surgical plan over the patient's anatomy in real time, providing an intricately personalized procedure with the highest level of precision.
UT Southwestern is one of just 15 surgical centers in the world using this next-generation arthroplasty technique, which was approved by the U.S. Food and Drug Administration (FDA) in July 2020. Augmented reality technology opens the gateway to a new gold-standard in surgical approach to shoulder replacement.
Pre-op 3D modeling for precise surgical planning
The process begins when we enter the patient’s preoperative shoulder CT scan into the BLUEPRINT™ software, and it generates a 3D model that we can manipulate in the virtual space:
- Examining the patient's anatomy
- Planning incision points
- Marking the points for the anchor pins
- Adjusting the prosthetic device
Then we can conduct a virtual “dry run,” implanting the shoulder in the software environment so we can determine what works best for that patient's anatomy and pathology. Ultimately, this virtual dry run becomes our surgical plan for the operating room.
The software also allows us to run a virtual model of the patient's arm and shoulder through range of motion exercises to further refine the procedure. By estimating the range of motion each surgical touch provides, we can maximize patient function and potentially optimize their recovery.
Essentially, we can perform the entire surgery and estimate post-surgical mobility and function in virtual reality before we even touch the patient in the OR. What's more, I can show patients what we'll do prior to surgery, which helps them feel more informed and engaged in their care.
AR in the operating room
Wearing the HoloLens AR glasses during surgery makes it possible to see, navigate, and manipulate the entire surgical plan during the procedure. The surgeon can scroll through it and zoom in or out while comparing it in real time to the patient's anatomy.
It's almost like what people see when playing Pokémon Go – you see the real world, with a cartoon-like overlay of the surgical plan. Rather than an immersive VR experience, it's a supplement to reality.
Being able to reference the surgical plan during surgery adds another layer of checks and balances to already highly precise shoulder replacement procedures. Combined with the decades of orthopedic surgical experience of our team, this technology allows us to deliver safe, effective, and precise surgeries using one robust toolkit.
'Almost like a Star Wars-type thing'
Monte Perkins was among the first patients to experience AR shoulder replacement at UT Southwestern.
By the time he arrived at my office, the pain in his right shoulder was so intense it was “driving him into the ground,” he said. He had fallen in his front yard in December 2020, and over the next few months he and his wife, Lynda, had visited several doctors and a pain specialist in search of some relief.
Mr. Perkins, a third-generation jeweler who has owned a shop in Irving for more than 50 years, was referred to me and he was eager to get his life and business back on track. When I showed him, via MRI, that he had two torn tendons in his shoulder, he told me “I'm just ready to have it fixed.”
So the 74-year-old opted for shoulder replacement, which I had recommended.
When he and Lynda learned about the augmented reality technique, they were "blown away" by what was possible.
"It’s almost like a Star Wars-type thing,” said Mr. Perkins. “They were able to layer everything together and make sure it all fit perfectly – just like a puzzle.”
The results have exceeded their expectations. “When Monte went to his first physical therapy appointment, they could not believe the movement he had in his arm already,” said Mrs. Perkins. “And every day, we notice he’s able to do something he wasn’t able to do before the surgery.”
Mrs. Perkins, whose husband has also battled lung and liver cancer since 2018, said their family feels fortunate to have an academic medical center nearby because it means patients get access to the most up-to-date treatments.
“Monte’s proud to be one of the first people in Texas to have had an augmented reality shoulder replacement done” said Mrs. Perkins. “I joke with him that 'I’m living with a celebrity!' ”
More enhancements on the horizon
Because this technology is in its infancy, the potential long-term benefits regarding post-surgical pain and recovery times haven’t been studied yet.
However, we do know that better planning leads to better surgeries. Research shows that 3D modeling can result in highly accurate restoration of the patient's anatomy and precise positioning of the implant.
In the future, we will conduct clinical studies of device survivorship and outcomes. For now, we are performing this procedure for every patient who has had CT imaging that is properly formatted to suit the software – we've been doing imaging in this manner for all recent UT Southwestern patients interested in shoulder replacement.
We anticipate that the technology will also expand educational opportunities for future orthopedic surgeons. Versions of AR surgical software allow for clinical "spectators" to see the procedure virtually through the surgeon's eyes.
This is the first iteration of AR surgical technology. We anticipate future improvements will only enhance the accuracy of placing the implants, cutting the bone, and determining the trajectory of the screws in the hardware. Augmented reality certainly feels like a game-changing advancement for shoulder repair and replacement surgery.
To talk with a surgeon about augmented reality shoulder replacement, call 214-645-8300 or request an appointment online.
