Learn how industrial giant ABB is using Unity and augmented reality to transform field maintenance procedures into a completely paperless process.
We recently invited members of ABB’s IS Innovation and Digital Scouting team, Maciej Włodarczyk and Rafał Kielar, to walk us through how they used Unity to develop a new digital field operator system. This multiplatform application runs on mobile devices and the HoloLens, and it’s designed to improve the efficiency and safety of the field operators maintaining and servicing equipment on industrial sites.
Learn more in their presentation on a recent Unity webinar, including their migration process to the HoloLens 2. In the webinar, Microsoft’s Mixed Reality Academy lead engineer Nick Klingensmith also shares how Microsoft’s new device will take AR-enabled training, guidance, and maintenance to the next level.
Let’s explore the problems ABB sought to solve for its clients and some of their key learnings from the development process.
For ABB’s customers, two key personnel are involved in this process: the field operators responsible for maintaining and servicing equipment, and control room operators who supervise the process and are located in another part of the facility.
Due to the dangerous nature of the tasks involved, field operators have traditionally undergone time-consuming, expensive training programs before working on-site.
Once they are in the field, however, it is difficult to assign and track performed service procedures, which are done on paper. This leads to further communication issues between field and control room operators, who often need to exchange information in real-time.
ABB used the Unity Editor and Microsoft’s Mixed Reality Toolkit (MRTK) to test prototypes quickly and eventually build a production-ready software application called ABB Ability™ Augmented Field Procedures. This multiplatform application completely digitizes the field operator experience with remote-enabled augmented reality technologies.
The application provides several advantages over the traditional, paper-based workflow. This system:
Based on their experiences, Włodarczyk and Kielar from ABB shared numerous best practices for those developing similar applications related to training, guidance, and maintenance use cases. In this post, we focus on several best practices for the HoloLens application, centered around the user interface (UI) and ergonomy of interactions.
Check out the webinar for a complete list of ABB’s recommendations, including location/device recognition, hologram positions, and more.
As seen in the image above, the UI should not obscure the user’s view. UI elements that block the real-life objects that the user needs to interact with can pose a safety hazard.
To minimize clutter in the user’s field of view, allow navigation menus to be accessed on request rather than being omnipresent. In the video above, notice how the user controls the visibility of the menu with a gesture.
In order to avoid blocking the field of view, some may think smaller menus and buttons make sense. On the contrary, these should be large enough to be easily targeted by gaze and selected by gestures.
Włodarczyk and Kielar needed to make their app easier and more convenient to use than the paper-based Standard Operating Procedures (SOP) their clients were used to.
That led them to automate as much of the experience as possible in order to limit the number of interactions the user needed to perform (e.g., having a window automatically appear following a gesture, as shown in the video above). It’s also important to provide clear instructions (e.g., the “tap to dock” message shown in the video) to ensure that the next step is always clear.
Users should also be given the flexibility to select the interaction mode of their choice, such as voice commands, gaze, and gestures.
Since most field operators have limited experience with AR but will be the end-users of these applications, it’s critical to share the app with a test group comprised of these individuals. They will be a source of valuable feedback that will help you to reduce complexity and streamline your app to its core components.
---
For more best practices from ABB, sign up to watch our on-demand webinar.
You can also check out ABB’s presentation at Unite Copenhagen.
Check out Unity Industry and learn how to get started developing XR applications with Unity.
