Hitachi Review

Image Analysis System Using Cameras to Detect Deviations in Worker Activities and Signs of Failures in Production Line

Collaborative Creation Technology that Innovates Manufacturing from the Front Lines

Skip to main content

Hitachi

Hitachi Review

CUTTING EDGE 2017

Image Analysis System Using Cameras to Detect Deviations in Worker Activities and Signs of Failures in Production Line

Collaborative Creation Technology that Innovates Manufacturing from the Front Lines

In advanced manufacturing workplaces, a way of linking and visualizing the performance of man, machine and material is needed to improve quality and increase work efficiency. Hitachi and Daicel Corporation therefore developed an image analysis system which, using various types of camera as sensing methods, links manufacturing performance data with image analysis technology to prevent human error and detects signs of operational failures in production line facilities. In the future, Hitachi will pursue initiatives aimed at optimizing not only the front lines of manufacturing but the entire global supply chain.

Detection of Human Error by Identifying Deviations from Standard Behaviors

What kind of problems was the image analysis system developed to solve?

IsakaDaicel had been examining how the Internet of Things (IoT) could be used to innovate the production process for automotive airbag inflators (gas generators). Seeing the various elemental technologies at Hitachi Central Research Laboratory, Daicel showed an interest in image analysis technology using cameras, and Hitachi and Daicel began collaborative research. When we analyzed the manufacturing performance data of plants from the perspective of the 3Ms (man, machine and material), we realized that if we could quantitatively determine the status of machines, materials, and worker activities on the front lines of manufacturing, this would help improve product quality, and increase productivity and the accuracy of traceability. We then rallied members with the technology and knowhow necessary for the new system from across the Hitachi Group and worked with the customer to find ways of solving their problems.

Describe how the system works.

Example of sensing of frontline workers and production line facilities using image analysis system

NagayoshiI was previously working on research into banknotes validation and image recognition for surveillance to keep people safe and secure. The technology developed based on this knowhow is the one that detects deviations in worker activities. It uses human joint positions such as hands, elbows, and shoulders of workers as inputs. It then compares the actual movements of workers with the standard behavioral model and raises the alert if it detects activities that deviate from the standard ones. To first gain an understanding of what constituted deviations in front-line worker activities, we put much effort to identify and list up all possible deviations by questioning the customer at great length and identifying all work processes. We also included new technology that normalizes individual differences arising from physical size and selectively uses information of the joints related to the work process to build a system for accurately determining deviations.

EndoWe also developed technology for detecting anomalies with welding machines, which play an important role in the inflator manufacturing process, using technologies and experiences built up in non-destructive inspection of power generation plant facilities. The light generated during welding contains, in fact, the distribution of the light constantly changes in a fraction of a second and has a characteristic appearance when a defect arises. If this knowledge is used in conjunction with light analysis of the light-emitting part and data on the current and voltage of existing production facilities, it is possible to detect problems based on differences from normal operations. By gleaning knowledge from the customer about how metal is welded together, we solved problems one by one.

Configuration of image analysis system

Shift in the Management of Product Information from “Point” to “Plane”

Describe the difficulties you encountered during development and your impressions of collabo­rative creation with the customer.

YoshikawaThe technology we developed is based on image analysis, which means its accuracy can’t be verified without image data from the customer’s front lines of manufacturing. In research and development, a process of trial and error is necessary for accuracy improvement. However, we were unable to stop the actual production line for research and development purposes. We therefore created a mock-up of the front lines of manufacturing in the laboratory. Daicel sent us some veteran workers and we conducted our research and development based on the data gathered from work demonstrated by these workers.

YamadaA major premise of this project was that the introduction of this system would not increase the workload on the front lines and we went to great lengths to come up with ways of linking analysis to the existing work processes. With Daicel’s cooperation, we conducted repeated tests and we were really pleased when we succeeded in linking the analysis image technology to the existing frontline system using logic based on the work time data of each production line facility. It was collaborative creation in the truest sense.

EndoI was impressed by the fact that Daicel and Hitachi were able to conduct development from the same perspective as we are both engaged in manufacturing, thinking about what we needed to do or what we could do for the customers and their end users. There was a deep sense of unity between the engineers of various departments at Hitachi. Moreover, we also got a great deal of pleasure from working together with the customer.

What effect did the system have and how did the customer rate the system?

IsakaPrior to introduction of the system, Daicel could only inspect the quality of workers’ activities and facilities on the inflator production line at certain “points” through the visual inspection of individual lots by supervisors. However, introduction of the image analysis system, which analyzes all man, machine and material processes by capturing them as continuous images, gave the customer an understanding of product information at “all points on a plane.” The system was highly rated by the customer because it prevented human error and significantly improved in-process product quality assurance. In the future, Daicel plans to introduce the system not only to its domestic plants but also to six plants overseas, and hopes to translate this into improvement in quality and productivity globally and optimization of the entire supply chain.

Aiming to Develop the Technology into a Solution to Problems in a Wide Range of Industries

How does Hitachi intend to develop this technology?

YamadaFirst, we intend to package the image analysis system so that it can be used on different front lines of manufacturing. At the same time, we are also working on image analysis technology as a key component of Hitachi’s IoT platform Lumada, and intend to use the technology to help develop solutions that quickly solve the individual problems faced by customers.

NagayoshiThis technology could also be used in areas such as preventing risks arising from workers’ activities and teaching workers how to operate machines correctly both in the manufacturing industry and other sectors. In the future, we intend to combine artificial intelligence (AI) with image analysis to develop technology that can assist workers and propose more efficient ways of working.

Hideya Isaka

  • Senior Engineer,
  • Industrial System Upper Engineering Center,
  • Industrial Manufacturing Solution Division,
  • Industrial Solutions Division,
  • Industry & Distribution Business Unit,
  • Hitachi, Ltd.

Hiroshi Yoshikawa

  • Chief Engineer,
  • Energy Systems Department,
  • Matsudo Research Center,
  • Corporate Technology,
  • Industry and Water Business Administration Division,
  • Hitachi, Ltd.

Hiroto Nagayoshi

  • Senior Researcher,
  • Media Systems Research Department,
  • Center for Technology Innovation − Systems Engineering,
  • Research & Development Group,
  • Hitachi, Ltd.

Hisashi Endo

  • Senior Researcher,
  • Applied Energy Systems Research Department,
  • Center for Technology Innovation – Energy,
  • Research & Development Group,
  • Hitachi, Ltd.

Toshihiro Yamada

  • Senior Engineer,
  • Industrial System Upper Engineering Center,
  • Industrial Manufacturing Solution Division,
  • Industrial Solutions Division,
  • Industry & Distribution Business Unit,
  • Hitachi, Ltd.
Download Adobe Reader
In order to read a PDF file, you need to have Adobe® Reader® installed in your computer.