Railroad operators must perform periodic inspection and maintenance of railway equipment to ensure safe and secure railroad transportation. For more than 50 years, Hitachi has been involved in developing and commercializing inspection and measurement devices for railway equipment that use optical technologies and various sensors, helping to ensure the safety of railroad transportation. In this project, Hitachi developed a wayside equipment monitoring system based on eight 4K cameras installed on the roof of rolling stock, and a system that enables inspection work at the office via ground-based devices, using the video data retrieved by the wayside equipment monitoring system. This system started operating in April 2020, and the retrieved high-frequency video data is being utilized in wayside equipment inspections, contributing to more efficient inspection work, higher inspection quality, and enhanced safety for maintenance staff. In the future, Hitachi will continue working toward expanding the target range of inspections and determining abnormality locations automatically.
Railroad operators must perform periodic inspections and continuous maintenance to ensure the safety and security of railways, which are a vital means of human transportation. Since the 1960s, Hitachi has contributed to ensuring the safety of railroad transportation for more than 50 years now through the development and commercialization of technologies for track geometry inspection, overhead line inspection, and wayside equipment monitoring, applying optical technologies, such as lasers and light-emitting diodes (LEDs), along with various types of sensors that Hitachi has cultivated in a variety of fields.
Recently, the railroad industry is facing major issues such as labor shortages due to the declining birthrate and aging population, and the mass retirement of experienced workers who have inspection and maintenance know-how. Finding countermeasures for these issues and making maintenance work more efficient have become urgent tasks. Furthermore, as has been seen in many fields of social infrastructure, operators are strongly considering shifting from time based maintenance (TBM), where parts replacement and other work is performed according to a predetermined cycle, to condition based maintenance (CBM), where the acquisition of condition data from equipment is automated to predict equipment deterioration from the large amount of data obtained, enabling parts replacement and other remedial work to be performed more efficiently.
Hitachi’s revenue car-mounted wayside equipment monitoring system [overhead electric equipment monitoring system (hereinafter “wayside equipment monitoring system”)] can be mounted on in-service rolling stock. This system further improves the efficiency and inspection quality of the series of railway equipment maintenance operations from inspection to maintenance. As a result, Hitachi will contribute to the realization of a mobility society where railway users can move securely and comfortably.
This article provides an overview of the wayside equipment monitoring system and describes activities for improving railroad maintenance operations using artificial intelligence (AI) technology.
Railroad wayside equipment has conventionally been inspected visually by a maintenance worker foot patrol, but the huge amount of equipment means that this inspection method is extremely labor intensive. In response, Hitachi developed inspection devices and systems to increase the efficiency of wayside equipment inspections by capturing images of the wayside equipment from video cameras installed on in-service rolling stock, enabling staff to check the video data at their office and shorten inspection times(1).
During development, trial runs were performed with cameras installed on in-service rolling stock to determine the specifications. Other technologies were also used to improve inspection efficiency, such as automatic inspection image extraction using AI, and identifying the location of the captured images using the global navigation satellite system (GNSS).
Table 1 — Wayside Equipment Monitoring System Specifications
High-resolution video can be recorded by 4K cameras installed in the system.
Table 1 shows the specifications of the wayside equipment monitoring system and Figure 1 shows the overall structure of the system.
The system is comprised of two parts; an onboard system and a ground-based system. 4K cameras installed as part of the onboard system, enable high-resolution videos and location information from GNSS to be recorded on a solid-state drive (SSD). At the ground-based system, the images and locations required for inspection are extracted from the recorded video information and converted into data by AI processing and location identification processing on the control terminal.
Figure 1 — Overall Structure of Wayside Equipment Monitoring System
The wayside equipment monitoring system is comprised of two systems; an onboard system and a ground-based system.
To visualize all the inspection target equipment in the video, a total of eight cameras are installed on the roof, comprising two forward/rearward cameras, four side cameras, and two catenary cameras. These are processed simultaneously to capture seamless 360-degree images (see Figure 2). At the same time, location information is linked to the images by recording signals such as GNSS information, mast detection sensors, and distance pulses from the rolling stock. The large amount of video data is compressed using H.265 [High Efficiency Video Coding (HEVC)] and recorded on SSD.
Also, a mobile wireless communication device using Long-Term Evolution (LTE)* is installed as a system support function. This adds capabilities such as recording start and stop control commands, ground-based system error reporting, and remote control.
Figure 2 — Images Recorded by Each Camera
A total of eight cameras positioned on the roof can capture seamless 360-degree images.
On the ground-based system side, analyzing the spectrum of each image frame of the recorded data to automatically determine the clearest day of inspection, assists staff in selecting the best images for their inspection. AI processing and location identification processing converts the selected recorded data into a set of extracted images and location data required for inspection.
Also, as a result of conducting reinforcement learning of AI through the analysis of the detected equipment targets for inspection, the performance of automatic inspection equipment target extraction was improved, achieving labor savings by further shortening inspection times.
As previously described, periodic inspections of overhead electric equipment used to be performed visually whilst moving on foot, but after deploying this system, staff can inspect equipment from the distributed image data instead of going on foot patrol.
This system supports two inspection methods: “Walking speed inspection,” which enables the video playback speed to be freely adjusted down to a minimum of 1 km/h to mimic walking speed, and “Spot inspection,” where images related to specific wayside equipment are extracted by AI processing and location identification processing to shorten inspection times.
When the spot inspection method is used, the data is listed based on the locations of the wayside equipment from the masts, and the equipment to be inspected can be selected from this list to display its images (see Figure 3). Up to four camera images can be simultaneously displayed from the eight cameras, eliminating blind spots. Also, the display color of selected or inspected wayside equipment changes on the selection screen to prevent any inspection omissions.
An image zoom function is available to further improve inspection accuracy (see Figure 4). High-resolution 4K enables zooming up to 1 mm/pixel making it possible to confirm the status of equipment screws or corrosion.
This wayside equipment monitoring system has been installed on the rolling stock of Kyushu Railway Company and is being utilized in maintenance operations(2).
Figure 3 — Example of Spot Inspection
To shorten inspection times related to specific wayside equipment, images extracted by AI processing and location identification processing are inspected.
Figure 4 — Zoom Function for Inspection Images
An image zoom function is available to improve inspection accuracy.
The deployment of this system has enabled operations to be shifted from visual inspections performed by on-site foot patrols to office-based video inspections. This has saved labor by eliminating the travel time for getting to the work site and the effort required in walking the line for inspections.
Performing inspections at the office has also reduced the risk of accidents while patrolling, enhancing the safety of maintenance and inspections. Furthermore, the automatic extraction of equipment for inspection has shortened inspection times to three-quarters that of conventional methods, and by accumulating the video captured when lighting conditions were most favorable, inspections could be performed without being impacted by the time of day or weather. The installation of eight 4K cameras has enabled wayside equipment to be captured in detail from a wide range of angles so that image inspection that can discover the signs of equipment deterioration far earlier than before.
This article has described the background of the development of technology for the wayside equipment monitoring system and the activities for improving social value such as safety and economic value by increasing maintenance and inspection efficiency in railroad maintenance operations through the use of AI technology and location identification technology.
From here on, Hitachi plans to expand the range of targets for inspections with additional inspection objects and inspection routes, while also studying the feasibility of shifting the judgments of equipment acceptability that are currently performed by human staff to an automatic determination by image analysis using AI technology. This will be done step-by-step from the equipment and inspection items for which technical methods have already been established. Hitachi positions the deployment of this technology that contributes to safe and secure railroad operations, and efficient maintenance, as an example of its contribution to social infrastructure, and it will continue to develop such technology to help realize a safe and secure society based on robust infrastructure.
