AR Supported Repair using Automatic Guided Learning (AGL) makes untrained techs into electronics repair superstars
AR Laptop Repair Minions
HOLLYWOOD, CA (GoshRobin) 2022/11/14 – As Augmented Reality product tech strategist for the world’s largest PC maker, I designed a series of PoC (Proof-of-Concept) prototypes to show off what our ThinkReality AR glasses can do. We wanted to show key accounts, research partners and potential customers the new capabilities our Lenovo ThinkReality AR glasses offer to meet their business needs.
Supported Repair, the Automatic Guided Learning (AGL) Concept
Lenovo has large service centers to repair Thinkpad laptops and Motorola mobile phones, products they manufacture. It typically takes a lot of training to prepare a tech. Lenovo has a myriad of product models that present a great many repair variations. A tech following the repair directions for the wrong model may make mistakes.
What if a tech who has never seen a particular model of laptop could be walked through the repair process automatically, using AR with AI?
Here’s what happens as a laptop tech wearing AR glasses is walked through the disassembly and assembly procedures on a specific model-by-model basis by AI.
- The tech turns over a broken laptop, and the AR glasses capture the barcode or use computer vision OCR to read the printed serial number to determine the model
- The 3D CAD drawings matching that device are automatically retrieved and visually locked to the physical device, giving the tech x-ray vision
Thinkpad Exploded View
- Moveable markers float in the air to show where screws need to be removed and where the case should be pried for disassembly
- A checklist appears floating in mid-air, for the tech to follow step-by-step to diagnose and repair the device
3D Checklist
- If the tech gets stuck, because the AGL encounters an unexpected repair situation, the tech may call for Remote Assistance from an expert engineer
- The remote engineer appears to the tech as a video hologram floating in mid-air
Lenovo Remote Assistance
- The remote engineer sees on his screen (typically a PC or tablet, not AR) what the tech sees
- The remote engineer guides the tech, manipulating remote markers that appear in the tech’s vision, to point the tech where to be looking, as the engineer may be saying, “Pry under the component I’ve marked”
- Once the unexpected repair obstacle is overcome, the tech may return to Automatic Guided Learning (AGL)
- The unexpected repair is noted in the company’s incident tracking system, a ticket is created to inform developers that the AGL needs to be improved, to handle this issue automatically in the future
AGL for Network Operations Center (NOC) Maintenance
Lenovo ThinkReality AR NOC Maintenance
At Lenovo a similar concept was developed for NOC maintenance, to guide technicians to the equipment rack needing service. As shown in this Lenovo ThinkReality video…
AGL for Medical Equipment Repair
At the UN World Health Organization, I designed MCARE, a AGL metaverse for training doctors to save lives.
WHO MCARE 3D Hospital Simulation
Going a step further, I conceived of combining telemedicine and Supported Repair in MCARE.
WHO has provided sophisticated medical equipment to low resource hospitals across the world. When this life-saving equipment breaks, it often gets pushed into a closet. It stays there, with no local facility or technician able to repair an MRI machine.
Although advanced medical equipment is both expensive and repairable, a low resource hospital a continent away lacks the funds to ship equipment internationally, to send it to the manufacturer for repairs, and then to have it shipped back again. And if the hospital does do that, it takes months.
With medical AGL Supported Repair, MCARE can enable local mechanics to repair exotic medical equipment despite them having never worked on such equipment before.
Repair Certification Using AR
With expert engineers reviewing repairs via live Remote Assistance, or later by recorded AR video, field repairs can be certified as meeting OEM repair standards and upholding product warranties. The need for such repair certification is not limited to fixing MRI machines and laptops, however.
AR for construction site remote inspection
Building construction is another field that AR can streamline, by enabling inspectors to do their job faster and better, remotely, without coming on the construction site.
About Robin Rowe
I’m Robin Rowe. People call me Robin or Rob, sometimes professor. I work in advanced product design and innovation management.
As UN WHO Augmented Reality Group Manager and XR Games Producer, I developed a medical metaverse, an AR/VR hospital simulation to train doctors worldwide to save lives.
As chief technologist at multi-billion dollar systems integrator SAIC, I was the founding director of their AI research lab and an enterprise manager with P&L responsibility for commercial and defense divisions. I constructed the Chicago NBC-TV studios, advanced national critical infrastructure by adding Smart Cities capability to the U.S. traffic control system, created real-time AI for DoD national defense crisis management, and developed the real-time motion-capture animation system used to create digital stunt doubles in superhero films such as Disney Marvel Spider-Man.
I led the team that won the Novartis Biome innovation prize in 2019, using AI computer vision to analyze skin disease. I’ve taught C++ as a Computer Science professor at the Naval Postgraduate School and the University of Washington. Former DARPA PI for AI and digital video. Former navy research scientist for VR war gaming and vision research. I’ve chaired innovation committees at ANSI/ISO and The CFO Alliance. I’ve developed financial software for Fortune 500 companies and large NGOs. I’m a thesis advisor to graduate and doctoral students in metaverse R&D.
I founded my first start-up, a car company, when I was 16. My family is in real estate and agriculture, owns the largest organic farm in Illinois. Not wishing to run the family business, I went another way. I have 30 years experience in product design and financial systems. And, with trading stocks and now crypto on my own account, I’ve made high ROI year over year.