Skip to main content

Hitachi
Contact InformationContact Information

    Highlight

    Remote monitoring systems for industrial and other machinery have increased in sophistication along with advances in IoT technology. If these systems are to evolve beyond merely monitoring and reporting on current conditions and instead acquire the ability to predict future outcomes and advise on how to resolve faults when they do occur, it will require the development of AI-based smart maintenance systems that enable people and the IoT to work together. Since it first incorporated a remote intelligent diagnostic system for elevators into its maintenance operations in 1994, Hitachi has been engaged in the development of such technologies as AI-based fault recovery support, the use of wearable devices for maintenance quality improvement and digitalization, and a dashboard that provides maintenance information to building owners. This article describes Hitachi’s current work and future plans for smart maintenance systems for elevators and escalators.

    Table of contents

    Author introduction

    Norimi Kodaira

    Norimi Kodaira

    • Maintenance Technology Development Department, Maintenance Technology Development Division, Field Service Division, Hitachi Building Systems Co., Ltd. Current work and research: Development of diagnostic devices for elevators and escalators.

    Masanori Miyoshi

    Masanori Miyoshi

    • Digital Engineering Department, Maintenance Technology Development Division, Field Service Division, Hitachi Building Systems Co., Ltd. Current work and research: Analytics of operational technology data and development of dashboard for building equipment. Society memberships: The Information Processing Society of Japan (IPSJ).

    Youichi Kumakura

    Youichi Kumakura

    • System Development Department, Maintenance Technology Development Division, Field Service Division, Hitachi Building Systems Co., Ltd. Current work and research: Development of remote monitoring terminal and remote control system.

    Masaharu Takahashi

    Masaharu Takahashi

    • Maintenance Technology Development Department, Maintenance Technology Development Division, Field Service Division, Hitachi Building Systems Co., Ltd. Current work and research: Digitalization of maintenance work by utilizing wearable terminals.

    Introduction

    Hitachi Building Systems Co., Ltd. first incorporated a remote intelligent diagnostic system for elevators into its maintenance operations in 1994, with aims that included enhancing convenience and safety for elevator users, providing a wider range of services, and improving maintenance productivity(1).

    The features of this system were as follows:

    1. Remote intelligent diagnostic system devices collect operational data from elevator controllers at regular intervals and forward it via public line to servers at the Hitachi Building Systems control center. This data is then used for remote inspection whereby diagnostic algorithms on the servers checked and evaluated equipment operation and degradation, with the scheduling of maintenance work by field engineers (FEs) being done automatically.
    2. When a fault indication is received from an elevator, an alarm is immediately passed on to the control center based on the nature of the fault. Fault information is also forwarded from the control center to the relevant maintenance depot for prompt action. The system has proved particularly useful for addressing faults that leave people stuck inside an elevator, with such incidents able to be dealt with very quickly through coordinated action by the control center and maintenance depot.
    3. The system is also equipped with functions that enable in-depth fault analysis. Along with logging the on/off status of equipment control signals right up until a fault occurs, these also include a fault finding monitor function for identifying the causes of faults.
    4. A full range of functions is also provided for service improvement, including remote automatic recovery from faults and automatic recovery after earthquakes.

    While these features helped the system make elevator use safer and expand service functionality, the experience and judgement of FEs was still needed for the maintenance work that kept elevators in a reliable condition. Unfortunately, factors such as the labor shortages that come with a shrinking population and the retirement of experienced engineers make it difficult to keep up high-quality maintenance services that rely on the skills built up by individuals. In response, Hitachi has embarked on the development and deployment of a smart maintenance system that enables people and the Internet of Things (IoT) to work together and that is built on the foundation of this existing system, transforming it into an even higher quality maintenance service.

    This article describes the core technologies used in the new system, namely the use of artificial intelligence (AI) to support fault recovery and the use of wearable devices for maintenance quality improvement and digitalization. It also describes a dashboard that provides building owners with maintenance information for the elevators maintained by the system.

    Smart Maintenance System in which People and IoT Work Together

    AI Support for Fault Recovery

    The mean age of the elevators and escalators maintained by Hitachi Building Systems is currently 16.9 years, with many of these being older models that are overdue for modernization. FEs provide a callout service that responds promptly when faults occur on these machines. Past practice was for FEs to receive training in the general and specialist knowledge required for their work while also undergoing on-the-job training where they acquire the practical expertise to deal with a wide variety of faults by working alongside more experienced staff. Unfortunately, changes in Japanese society as a whole have upset the numerical balance between newly recruited and experienced FEs, with the retirement of older staff making it difficult to adequately pass on expertise in legacy models. Maintaining the quality and safety of these older models now poses a fresh challenge for the future.

    This heightens the risk of elevators being out of service for extended periods when, for example, it takes a long time to identify the cause of a rare or unusual fault or to restore operation when such faults occur on an older model of elevator.

    In response, Hitachi Building Systems is developing a fault recovery support system that uses AI. When an elevator develops a fault, the system automatically obtains control signal logs for key items of equipment collected by the remote intelligent diagnostic system.

    The system likewise acquires the situation assessment data entered by the FE into their device when they arrive on site. Next, it uses the approximately 900,000 historic fault response records to perform a pattern matching analysis on the control signals, on-site fault reports, and fault response records to list the possible causes in order of likelihood, presenting this information on the FE’s device along with a flowchart instructing them on how to go about fault finding.

    Figure 1 shows the functions of the AI fault recovery support system and the sequence of operation. Notification of the elevator fault is passed on to the control center based on the error code generated by the control unit (when it is able to identify the fault cause) and the telephone call from the building owner. In the case of older models in particular, the remote intelligent diagnostic system is not always available and there may be no fault data in situations such as when an elevator is making an abnormal noise. In cases like these, the system indicates how to perform fault finding with a high level of precision.

    Fig. 1—Block Diagram of AI Fault Recovery Support System Fig. 1—Block Diagram of AI Fault Recovery Support System The system searches approximately 900,000 historic fault response records to perform a pattern matching analysis on control signals, on-site fault reports, and fault response records. The possible fault causes are displayed on the FE’s device in order of likelihood along with a flowchart instructing them on how to go about fault finding.

    In cases when the fault cause cannot be determined from the error code, or on older models from which the remote intelligent diagnostic system is unable to provide fault information, the system cross references the situation assessment data entered by the FE into their device against the more than 900,000 fault response text records (see Figure 2). Having pre-processed the text-format fault response records to identify fault details and establish the associated cause-and-effect relationships, the system is able to narrow down the likely causes based on the fault information collected on-site and present these in order of likelihood.

    Fig. 2—Example Procedure for Identifying Location of Fault Cause Using Machine Learning Fig. 2—Example Procedure for Identifying Location of Fault Cause Using Machine Learning Text-format fault response records are pre-processed to identify fault details and establish the cause-and-effect relationships between these details and the circumstances of the fault. These relationships are used together with fault information entered on-site to narrow down the likely causes and present them in order of likelihood.

    The system is also designed to improve its accuracy over time through machine learning that involves adding new faults into the system and routinely conducting cause-and-effect analyses. When used in practice, the system is able to identify fault causes quickly.

    Use of Wearable Devices for Maintenance Quality Improvement

    While use of the AI fault recovery support system makes it possible to respond to faults quickly and accurately, this will only result in better maintenance quality if the FEs perform their work correctly. FEs are provided with a portable device for use in their work. When the AI fault recovery support system identifies the cause of a fault, for example, the device instructs the FE on which equipment to inspect and the procedure for doing so. As this involves looking at the device screen in the confined spaces where elevator maintenance work is performed, work safety considerations such as body posture and lighting need to be taken into account.

    Advances in wearable devices are making a variety of new functions possible. Hitachi Building Systems is developing ways of utilizing this new technology to improve work safety and the accuracy of maintenance work. This involves replacing the notebook computers previously used for this purpose with wearable computers, augmented reality (AR) devices, wearable sensors, and smartphones, with wearable computers and smartphones being used to provide network connectivity (see Figure 3). These smartphones or other network-capable devices connect to intranet servers, with information being processed by Lumada algorithms. The system has the following features.

    Fig. 3—Use of Wearable Devices for Maintenance Quality Improvement Fig. 3—Use of Wearable Devices for Maintenance Quality Improvement Hitachi is utilizing these new technologies to improve work safety and the accuracy of maintenance work and is working to develop the relevant elemental techniques.

    1. In response to an FE verbally entering the required information, the system identifies the appropriate instructions and displays them on an AR device. The FE follows the displayed instructions, thereby ensuring that the maintenance work is done properly. When questions arise during this work, the FE can receive direct instructions from support staff via two-way video and audio.
    2. The system ensures that maintenance work has been done properly. Along with displaying relevant information such as operating instructions for the equipment being maintained, images of the completed work taken using the camera on the AR device and sent to the system are used to automatically assess how well the site has been tidied up (something that in the past was left to the FE’s own judgement).
    3. A range of wearable sensors are provided to monitor the physical condition of the FE and provide this information to the support staff. These include Global Positioning System (GPS), accelerometer, heart rate monitor, and body temperature sensors. This provides for the early detection and response to any FE health concerns and prevents serious accidents by avoiding conditions such as heat stroke.
    4. Safety can be improved by monitoring the movements of a person fitted with wearable sensors and automatically detecting anomalous movements by comparing their actual actions against the task they are meant to be performing, issuing a warning to both the FE and support staff when this occurs.

    These wearable devices are also proving useful for encouraging safety awareness, being used to produce virtual reality (VR) depictions of workplace injuries that can provide simulated experience in safety training. Hitachi plans to further develop these elemental techniques in order to implement systems that can be used in practice.

    Digitalization of Maintenance Work

    Experienced FEs can sense the condition of equipment, such as faults or deterioration, from small changes in things like sound, vibration, temperature, smell, or color. This draws on the sense of something about an elevator being different from normal and is acquired intuitively from the cumulative experience of conducting routine maintenance and working on faults. To help the AI fault recovery support system identify the causes of problems with greater accuracy, there is a need to convert this FE intuition into a digital form that can be incorporated into the diagnostic process. To this end, Hitachi is working on implementing the following technique.

    1. Elevator operational data collected by the remote intelligent diagnostic system, sensor data from the FE’s wearable devices, and data on elevator vibration and motion noise is collated in a database together with the assessment by an FE as to whether the elevator for which the data was collected is operating normally or abnormally.
    2. Using the FE’s assessment, the correlations between the various data (operational data, wearable device data, and vibration and noise data) are determined and grouped into separate datasets based on whether the assessment was normal or abnormal.
    3. New data is classified as normal or abnormal depending on which dataset it is closer to (determined by calculating its Mahalanobis distance* from each dataset).

    The idea is that, by calculating the relationships between different data on the basis of human judgements, this will provide a diagnostic technique that replicates human intuition. The intention is to also provide datasets in which there is a clearly defined correlation between the inputs and outputs (“teaching data”) so as to further improve the technique by learning these patterns.

    *
    A distance measure used in statistics that is based on the correlations between multiple variables and used in multivariable analysis.

    Dashboards that Present Equipment Data to Building Owners

    Fig. 4—BUILLINK App Screenshots Fig. 4—BUILLINK App Screenshots These screenshots are from the dashboard for building owners and managers that present information from the smart maintenance system on a personal computer, smartphone, or other such device. The intention is to make elevators “visible, connectable, and operable,” providing not only operational information but also a wide choice of options that include control of elevator operation and the ability to change the information displayed in elevator cars.

    In November 2019, Hitachi launched BUILLINK, a new Lumada solution for the commercial property sector that provides building owners and managers with a dashboard for personal computer or smartphone that shows details of the operation and maintenance of building facilities managed by the smart maintenance system(2). This is intended to meet the needs of building owners and managers who need direct access to information such as on the operation of elevators and other building facilities and their operational status in the event of a major disaster (see Figure 4). The dashboard provides a wide choice of options in addition to operational information, including equipment status data and the ability to change the information displayed in elevators, the aim being to make elevators “visible, connectable, and operable.” It also supports paperless practices for greater administrative efficiency.

    BUILLINK also has access to fault recovery information, meaning that along with alerting building owners immediately when people are trapped in an elevator, it also provides realtime updates on services such as those for remote automatic recovery from such incidents and automatic recovery after earthquakes and on the dispatch of FEs. This keeps building owners informed as to when their elevators will be back in operation. Along with making the work of building owners and managers more efficient, it is anticipated that this will also provide greater convenience for elevator users and raise asset values for owners.

    Conclusions

    This article has described an AI-based fault recovery support system, the use of wearable devices for maintenance quality improvement and digitalization, and a dashboard that provides building owners with maintenance information about the elevators managed by the system.

    Along with transforming the intuition of experienced FEs into digital form so as to develop diagnostic techniques that replicate human intuition, Hitachi’s future plans also include making these diagnostic techniques available as teaching data to further improve the AI-based fault recovery support system.