Whether it’s a full routing, a sheet of assembly instructions, an assembly diagram, or a manufacturer’s manual, the use of augmented reality makes it possible to present an operator with information about the tasks to be done intuitively and with little ambiguity (matching contextual information and the system being assembled is more natural than when using a paper plan, for example). In this context, the augmentations are generally presented sequentially so as to reflect the various steps of the assembly (or disassembly) process. The information and augmentations integrated into the real world may relate to:

These operator assistance solutions are particularly attractive in businesses with high levels of turnover or seasonal employment, because they reduce the time needed to train the operators.

One benefit of such a device lies in the ability to track and record the progress of assembly operations. The use of the augmented reality device may therefore help the traceability of operations.

In a context of assembly assistance, it is generally desirable to implement hands-free display solutions. For this reason, the solutions deployed are generally:

One of the major challenges of these applications lies in the goal of accurately placing the augmentations with respect to the current assembly. Depending on the usage context, the desired viewing details may vary from a millimeter to several centimeters. For this reason, the methods and algorithms used for locating the augmented reality device in space also vary, but all include a method that makes it possible to spatially co-reference the digital models (augmentations) with the real world (object to be assembled).

The current issues and technological obstacles for this type of use case particularly relate to improving the process of creating routings in AR, and locating moving objects. This is because many of these processes are already digitized, but require automation tools in order to correctly adapt them to augmented reality interfaces.

Finally, with respect to the technological acceptability of the devices, the needs expressed relate to improving the ergonomics and usability of the display devices, and especially ensuring over the long term that the optical devices cause no harm to the operator from intensive use.

One of the first use cases deployed in business was at Boeing in the 90s, with the use of AR to visually superimpose electrical schematics and the corresponding documentation, directly onto plywood boards onto which electrical wiring was assembled and stranded.

In 2011, DCNS experimented with a system for assisting workers at shipyards to track fixtures on the ship’s walls. Using a video projector, this projective AR application makes it possible to display the CAN model of the ship directly onto the walls during the construction phases. This is expected to save assembly time and make the positioning more accurate.

The company SUNNA DESIGN, for example, uses a fixed screen across from the operator at his or her workstation to present the positioning instructions for the parts to be assembled.

Another example is the company SL Laser, which, using a scanning laser projector, provides a system to assist in positioning when drape-forming a carbon composite part.

Owing to its ability to realistically incorporate a digital model in the real world, augmented reality makes it possible to assist in the assembly control process. The visual comparison offered to the human makes it easier to search for positioning defects or reference errors. At the same time, an assembly completeness check is possible. If a fault is detected, the operator can generally take a photo from the application and fill out a tracking form, which will later be added back to the company’s information system.

The current state of technology does not make it possible to automatically detect defects and therefore constitutes a (non-automatic) visual assistance system that can accelerate the control process while increasing its performance and exhaustiveness.

For quality control assistance, needs for hands-free display solutions are rarely expressed. Additionally, the potentially long time that controls take, and the need to enter textual information, generally steer solutions toward tablets or fixed screens. In some cases, it may be wise to use projective systems (with a video projector) because they have the benefit of displaying a large quantity of information at once, and thereby do not require that the operator looking for defects examine the environment through his or her tablet screen, whose field of vision is limited to a few dozen degrees.

The technological obstacles and challenges for this use primarily relate to seeking accurate enough augmentation positioning to be compatible with the positioning control task to be performed. Location algorithms must both estimate the device’s movements sufficiently accurately while enabling precise spatial co-referencing between the digital model and the real object.

In 2013, as part of the innovation program related to the shipyard of the future, STX initiated an approach to integrate augmented viewing technologies at the shipyard. The use cases explored included the idea of an assembly completeness control system. The issue is critical, because these errors prove to be particularly difficult to correct at later stages of assembly, particularly aboard the ship.

Test campaigns relied on using a tablet combined with a system of visual markers for the initial spatial references. The system enables an operator to view through the tablet’s screen, the digital model representing the final integrated assembly state in viewing the actual model. He or she may thereby visually check the progress of the shipyard and the business features that enable him or her to generate error reports when a non-compliance is detected.

Initially developed by AIRBUS then sold by TESTIA, the MIRA (Mixed Reality Application) solution was created in 2009, to improve the productivity of control operations. Using augmented reality, on a tablet and sensor specially developed for the application, operators can view parts of the cabin and detect assembly errors from digital models integrated into the real world. Used on the production of the A380, the application considerably reduced control times, as well as undetected errors.

CNES uses augmented reality to control the assembly operations of satellites. They may thereby reduce the time needed to compare paper plans and the actual state of assembly while simultaneously reducing the rate of undetected errors.

Augmented reality may be a response to the problems encountered by operators and technicians in the field. Such workers must, for instance, complete maintenance tasks (corrective or preventive) or inspection rounds in large environments with a lot of equipment present. Based on the operator’s level of expertise and the site of intervention, those activities may prove particularly complex.

In this context, augmented reality enables valuable assistance by presenting information drawn from the company’s document system, in an intuitive and contextualized manner. Maintenance procedures, technical documentation, map of inspection rounds, data from sensors, etc. All of this information may be presented to the user, given the place and common activity.

Information from the supervisory control and data acquisition (SCADA) system may also be viewed in augmented reality, so as to view values drawn from sensors and connected industrial objects (Industrial IoT). This data may thereby be viewed in spatially consistent manner with respect to the equipment, or even to the sensors.

In large environments or those particularly populated with equipment, augmented reality combined with a powerful location system may also be used like a GPS to visually guide the operator to the equipment that they need to service or inspect.

Finally, the most advanced augmented reality may include remote assistance systems. These systems enable the operator in the field to share what they’re observing with an expert (via video transmission or 3D digitization), and to receive information and instructions from the expert in the form of information in his or her visual field, information presented in a way temporally and spatially consistent with reality in the best of cases.

The desired devices are generally goggles, so as to leave the user’s hands free. Despite this fact, the solutions currently deployed are most commonly based on tablets, because these devices are more mature and robust than currently available head-mounted devices.

To assist technicians in the field, there are numerous solutions that may be wrongly likened to augmented reality solutions. Without questioning its usefulness, a system based on an opaque heads-up display that shows instruction sheets to the user could not be considered an augmented reality system. If there’s no spatial-temporal consistency between the augmentations and reality, it isn’t augmented reality.

The company ThyssenKrupp collaborated with Microsoft in 2017 to develop an augmented reality system that would assist technicians. Based on a Hololens headset, the system enables the technician to get expert help from an engineer in another location, while keeping his or her hands free. The remote expert sees everything the technician sees, and can point out elements as though they were right next to each other. For example, he or she can "draw" on the real world, share documents, or tell the technician what to do to disassemble a component. The pilot deployment study with 50 Hololens headsets made it possible to measure a time reduction of about 20 minutes to 2 hours on a typical operation through the use of the system.

In 2016, the company Caterpillar demonstrated an augmented reality system for one of its products, a generator, which is typically leased out via its network of dealers. In this demo, a user equipped with a tablet (iPad) could point to the generator and see step-by-step instructions for a preventive maintenance procedure in 3D animated form.

When designing new production equipment, testing its integration into its intended environment through augmented reality may be valuable. These simulations make it possible, for example, to identify any interference between the future equipment and its environment, to assess its impact on flows of people and materials, or to confirm that users will be safe when implementing moving machinery such as an industrial robot. To do so, it is necessary to be able to view in a common frame of reference both the expected production equipment, available digitally only, and the environment for it to fit into, which is real and physical. Although 3D scanning solutions that make it possible to digitize the environment exist, the compromise between the accuracy of the scan and the acquisition time make them of little use for a simple visualization. Augmented reality technologies, on the other hand, offer the ability to incorporate the digital model into the actual environment without any additional scanning.

From a hardware perspective, these applications generally require a wide field of vision to be able to perceive the modeled production equipment in full. As of this writing, head-mounted devices are not yet able to meet this requirement, and as s result only hand-held devices (generally tablets) can suffice. From a software perspective, as the environment is generally available only in digital form, and accuracy takes precedence over aesthetics, the digital model’s positioning is generally done by visual markers made up of geometric patterns with a high degree of contrast.

These simulations are helpful mainly at identifying potential problems of integration at a very early stage in the process of designing production machinery. These problems may thereby be anticipated and resolved long before installation itself, which greatly reduces the number of adaptations needed on-site. These on-site modifications are generally very expensive, because even outside the direct cost of the modifications and the need to service the machine under more delicate conditions than in the shop, production must also be adapted or even stopped during servicing. These defects must therefore be identified and corrected as soon as possible in the design cycles.

Furthermore, viewing the future means of production directly on-site is an extremely powerful way of communicating, which among other things enables better mediation between designers and operators.

Before it was bought out in 2015, the company Metaio sold a solution that could perform such simulations. One example implementation was publicized with the automaker Opel during the design of the Adam. The principle consists of superimposing the vehicle’s digital model onto an existing production line and checking to ensure there is no physical interference. The analysis is carried out by means of square markers attached to different objects in the physical environment. It is thereby possible to move the virtual vehicle interactively and to study potential interference with the actual environment.

Although the use of Augmented Reality is still emerging, it does appear to be a promising source of time savings, particularly out of a desire to optimize order picking and preparing operations.

In this context, it enables superior anticipation of the order schedule and load management by connecting with management systems. Its visual assistance enables workers to find their way around the site more quickly, using geolocation mechanisms that are compatible with the accuracy requirements of a large-scale indoor location scenario.

The visual instructions may also contain other information that is useful for completing the task, like the number of items in the order or the parts references.

Once the task is complete, validation by the device causes stock management and operations supervisory system (WMS) to be updated in real time.

Augmented reality systems that make it possible to assist operators during palleting operations have also been created.

The challenges of technological acceptability primarily relate to the ergonomics and battery life of devices that must be worn by the user for long periods of time. User feedback also indicates that the cameras on augmented reality goggles generally perform too poorly to read bar codes remotely.

It is expected to limit errors while also saving time, particularly for novice staff.

In 2015, DHL, in partnership with Ubimax, has developed a pilot project that would enable operators equipped with monocular augmented reality goggles to get assistance during picking. The system enables heads-up viewing of instructions, guidance to different locations needed for the order, and a barcode-reading system to confirm input. The disclosed results indicate 25% less time is spent than with solutions already in place, as well as an error rate reduced to 0%.

SAP and Vuzix have taken the same idea to an operator visual guidance project. Using monocular goggles, they move within the warehouse, and scanning barcodes to learn the contents of an order guide them visually to their destination.

Similar proofs of concept have also been created by the company Generix for its client Oscaro.com and by the Hardis Group, which just made a proof of concept for Celio, in a warehouse where operators pick orders for multiple stores, with four boxes to fill at a time.

Augmented reality applied to the field of training is of great interest to learners, thanks to its visualization and natural interaction capabilities. It enables understandings of phenomena and procedures facilitated by integration of scriptable virtual elements into the real world. Its capabilities include, for instance, offering a learner "X-ray vision" of a piece of equipment to observe its internal operation. It also makes it possible to learn how to carry out complex procedures, by directly showing the different steps of assembly on an artifact, and by extension the future movements to perform. Instant visual feedback of how well the learned action was performed is also possible (speed of movement, positioning of a tool, etc.). Augmented reality also offers the chance to train without consuming or damaging materials, for welding or paint-spraying, to give two examples. Finally, it enables risk-free learning situations for tasks that could be hazardous in real life (such as operating an overhead crane).

For training purposes, display-related technological approaches are numerous, and depend on the goals and corresponding usage setting:

The company Sunna Design, has incorporated an augmented reality device into its production system in order to better train its operators. The stated results are very positive, because the company estimates that the training time needed has been reduced to 80%.?

Boeing has been interested in augmented reality for several years. In 2015 the company presented an augmented reality ROI for training. The study related to people who had no knowledge of aeronautics who needed to carry out assembly using an augmented reality device, compared to using a screen and traditional instruction sheets. At the end of this test, people who use the augmented system committed only a single error during 50 steps of assembly, where other people committed eight errors.?

The solution SOLDAMATIC makes it possible, for example, to train in welding. Using actual parts and an augmented reality headset, which make it possible to view the virtual molten pool, the learner can practice various gestures and procedures, while obtaining real-time visual feedback about the quality of their attempts. The stated gains speak for themselves: 68% consumables saved, 56% of time saved on actual in-shop training.

Marketing and sales were the primary vectors for the use of augmented reality over the past few years, particularly among the general public. This technology appeared to be a solution for business issues encountered in various sectors of activity, including industry.

With respect to marketing, augmented reality is mainly used to:

With respect to sales, augmented reality has primarily been used to boost sales and individual purchases by:

The company Augment offers an augmented reality solution of the same name, which can be used to place objects like refrigerators or POSs directly where they will be installed. It claims sales growth figures of 30% thanks to augmented reality. This figure is drawn from pilot projects Augment has led with Colgate and Coca-Cola.

With regard to sales assistance devices, we can also take the example of the company Lego, which has put augmented reality terminals in several of its stores. The intent is for customers to be able to see inside the packaging held in their hands while facing a fixed screen, their future assembled Lego model, in order to convince them to buy.

Another example is the company IKEA and its 2013 catalog enhanced with an augmented reality app. IKEA’s goal was to increase its remote sales by allowing individuals to directly view furniture in their own home in augmented reality form. Currently no ROI is publicly available for this campaign, but it should be noted that IKEA has gained attention in communications and innovation from this pioneering, widely available augmented reality app. More recently, in June 2017, IKEA announced that it would partner with Apple to come out with a new version of its augmented catalog.