Learn how Unity and HoloLens 2 have become essential tools at one of the world’s largest automakers to streamline processes, increase understanding, and save time.
One of the core principles of Toyota Motor Corporation is Kaizen (continuous improvement). In both production equipment and work procedures, Kaizen seeks to drive maximum quality, efficiency gains, and elimination of waste.
Toyota often turns to technology to deliver these improvements, which is why the automaker was an early adopter of 3D data for digital engineering and later embraced real-time 3D technology. Toyota uses Unity’s real-time 3D development platform in many ways across its automotive lifecycle.
Its virtual pipeline starts by importing vehicle data into Unity using Pixyz. This process quickly converts Toyota’s large computer-aided design (CAD) assemblies into lightweight content suitable for real-time 3D.
The company then uses Unity to develop applications tailored to its needs and deploy them to various platforms, whether it’s conducting training sessions in virtual reality (VR), creating stunningly realistic car configurators for its luxury Lexus brand, or condensing inspection workflows from days to hours with HoloLens.
Toyota has used Unity to create and deploy mixed reality applications to Microsoft’s revolutionary device across its automotive production process. Naturally, its team was eager to expand their mixed reality capabilities with HoloLens 2, the next generation of Microsoft’s wearable holographic computer.
Watch the talk below from Koichi Kayano, the project leader of mixed reality for automotive digital engineering at Toyota, which introduces several proof of concept cases in progress. Learn how Unity and Microsoft’s new mixed reality devices are helping Toyota achieve Kaizen in several aspects of design, manufacturing, and field service.
Here are some of the many ways Toyota is saving time, reducing costs and driving efficiencies with mixed reality.
Previously an arduous task, CFD analysis is now made simpler with the assistance of mixed reality. Toyota uses Unity and HoloLens 2 to capture and display CFD analysis on a vehicle in real-time to streamline the design review process.
Going around a stationary vehicle, the user can simulate and analyze how its design affects aerodynamics. And using multiple HoloLens 2 devices, Toyota’s team can share their view with one another to better communicate and collaborate during a review process.
Using Unity and HoloLens 2, Toyota captures and displays CFD analysis on a vehicle in real-time
Once a vehicle is assembled, it is challenging to explain the functionality of hidden mechanisms within the vehicle. This is made more difficult when the function requires the vehicle to be in motion.
Using HoloLens 2 and Unity, users can now move around and inspect the inner workings of a “moving” vehicle – making a task that was once impossible now able to be easily and safely performed.
Users can see how the vehicle operates at startup, upon acceleration and deceleration in mixed reality
The possibility of human error is always present, even among expert technicians. Simple errors such as a loose coolant cap carry consequences if not detected and corrected immediately.
With the help of machine learning, Unity, and HoloLens 2, engineers are being guided to recognize and remedy inconsistencies that are easily missed by ordinary inspection.
Toyota’s team can easily spot mistakes that the human eye would miss with mixed reality. In this case, HoloLens 2 detects that the oil level gauge is improperly installed and highlights it in red text.
Trying to reduce human error previously required a lot of human effort, however. To train machine learning models on Microsoft Azure to recognize these simple mistakes, Toyota’s team needed to take 20,000 photos and spend 200 hours annotating the photos manually.
Manual annotation was a time-consuming process for Toyota and required its team to label tens of thousands of photos to help the AI system detect errors accurately.
Using Unity, Toyota created 3D models of the vehicle and the body parts under the hood, varied the model’s position in 3D space, and automatically captured a large volume of labeled images to train its machine learning models.
Toyota used Unity to automatically capture and tag images to train its machine learning models to recognize errors after body parts are installed
Compared to the traditional workflow that took 200 hours, this approach generated the needed amount of auto-labeled images in just 30 minutes—a 400X improvement in speed. This synthetic data also proved just as effective at training the machine learning models as the manually annotated photographs. For Toyota, this kind of time and cost savings is an ideal example of Kaizen in action.
Proper electrical wiring configuration is crucial for ensuring vehicles operate as intended. In a finished vehicle, however, inspection of connector positions and pin assignments is a serious challenge.
Instead of relying on 2D diagrams, Toyota’s team now has the ability to visualize the entire three-dimensional electrical wiring diagram inside the engine, doors, dashboard, or any other part of the car they require. This allows Toyota’s field service engineers to gain contextual understanding and to visualize the location of the wiring systems without the labor and time needed to remove physical parts.
Toyota field service engineers can visualize the entire three-dimensional electrical wiring diagram inside the vehicle.
Toyota also leveraged mixed reality applications from Dynamics 365 for two additional use cases.
--
As a global company, Toyota needs to connect field support engineers and experts across various locations. This presents numerous challenges and hinders effective collaboration and training.
With HoloLens 2 and Dynamics 365 Remote Assist, Microsoft’s mixed reality distance collaboration tool, two or more participants can share the same view, communicate, and collaborate no matter their location. This solution allows remote staff to inspect work, educate, and train field engineers, while saving considerable time and money and improve overall results.
Toyota uses HoloLens 2 and Microsoft’s collaboration tools to seamlessly connect offsite experts to remote support engineers
Step-by-step tutorials are critical to the ability of Toyota’s service engineers to make repairs effectively, but creating these manuals is time- and resource-intensive. In the past, a digital version of a work procedure app would take up to ten days and require an on-site computer graphics engineer.
With Dynamics 365 Guides, Microsoft’s virtual training, performance, and instruction solution, the same task now takes 90 percent less time—just one day. This automated process allows anyone with basic training to create necessary applications through detailed instructions and guidance, freeing up programmers to focus on other tasks.
Dynamics 365 Guides reduced person-hours needed to create training content for car repairs by 90 percent
--
Learn more about tools for developing for HoloLens 2 with Unity. Unity is the leading platform for creating content for augmented reality and virtual reality applications – subscribe to Unity Industrial Collection to get started today or learn more about our solutions for your business. Unity Technologies is the author of this blog post; Toyota Motor Corporation is not responsible for its content.
