O'Neil Manufacturing wanted a new way train new personnel on performing maintenance on basic machinery. They contacted the GRILL to create for them an AR maintenance trainer so trainees can see instructions in 3D space.
GRILL students set out to build an AR maintenance trainer that overlays 3D models to real-life objects. With Unity and the HoloLens 2, equipment can be scanned into the software, while the hardware tracks the users' hand movements and eyes when performing maintenance.
Wright Scholars used Unity Engine to create a program for the HoloLens 2 capable of recognizing real world maintenance targets, and showing maintenance steps via 3D animations. They also implemented eye tracking and gesture recognition in order to make the simulation immersive and interactive.
Students created a smooth program interface with manually-animated 3D models that break down the equipment by each component. The software walks the user through the maintenance process—based on what they are fixing—by consulting a live instructional data sheet on a local WAMPServer.
Future work would include implementing text-to-speech for instructions, improving database security, and adding more maintenance targets.