AR/MR Technology for Infrastructure Inspection: Finding the Human-Machine Balance
by Dr. Alan Smith, Project Consultant at SGH
While infrastructure across the country increasingly ages and deteriorates our inspection practices have largely remained the same. Still using pen and paper, inspectors visit each element of a structure and meticulously document their condition. This works for each case but is rapidly losing ground on both volume and accuracy to what is required. Emergent technologies such as computer vision and machine learning have long promised to revolutionize this work approach (the “workflow”) but why haven’t they?
The Balancing Act: Human and Machine Collaboration
Traditional inspection methods involving pen, paper, and tape measures can be greatly improved and hence are ripe for disruption. Beyond the tedious paperwork, these methods often result in transcription errors and alarming inconsistency between inspectors—sometimes exceeding 100% variation in measurements (Washer et. al., 2020).
Having said that, our recent research evaluating Augmented Reality (AR) bridge inspection tools revealed something fascinating about automation: more isn’t always better. When comparing different levels of human-machine collaboration, we found an optimal balance exists between human input and computer automation.
Our collected data showed that as automation increases, inspectors experience reduced workload and faster task completion—but with a critical caveat. Without some degree of human oversight, measurement accuracy decreases significantly, especially for detailed features like crack width measurements. Our experiments with the HoloLens 2 demonstrated that hybrid approaches allowing users to correct computer vision results produced the most favorable outcomes, optimizing for both efficiency and accuracy.
The Future is Hybrid
Our research confirms that the optimal path forward isn’t replacing human inspectors with technology, but creating thoughtful integration. Our field studies with HoloLens 2 revealed several practical considerations for successful implementation:
Implementation Challenges and Solutions
Our research identified several key factors that must be addressed for successful AR/MR integration:
- Hardware limitations in outdoor environments affect hand-tracking and cause sensor saturation, though these issues should diminish as technology matures
- Automation balance is critical—high automation reduced task time by 20% but introduced errors, while semi-automated approaches with human verification delivered optimal results
- Experience dynamics reveal telling patterns—inspectors’ years of traditional experience overshadowed brief AR training benefits, while novices found immediate workload reduction, suggesting they better represent AR’s true value
- Computer vision robustness remains challenging—algorithms performed inconsistently in variable field conditions (especially for crack width measurements), highlighting the need for research in realistic environments rather than controlled laboratory settings
Integration Pathway
Implementation should follow a phased approach:
- Begin with digital documentation that conforms to existing standards (our interface was validated by inspectors as more intuitive than current systems)
- Introduce measurement assistance with full human verification
- Gradually expand the scope beyond cracks to include corrosion, spalling, and delamination detection
- Develop seamless data export and integration with existing bridge management software
By addressing these practical implementation factors alongside technical capabilities, we can create inspection systems that enhance human abilities rather than attempt to replace them—ultimately improving inspection speed, safety, accuracy, and long-term deterioration tracking.
References
Washer, G. A., Hammed, M. M., Jensen, P., & Connor, R. J. (2020). Quality of element-level bridge inspection data. Transportation Research Record, 2674(2), 252-261.