Featured customer case
Lumada customer case code: UC-01726S
AI-based solution that identifies human posture and actions
2022-06-02
Workers engage in manual operations on manufacturing shop floors, so factory supervisors (including managers) need to prevent operating mistakes and ensure consistent product quality. This document explains use cases in which you can use an AI-based solution to analyze camera footage from a shop floor and identify workers’ postures and work-related actions. Supervisors can use this solution to detect operating mistakes and thereby help ensure consistent product quality. The solution can also provide data-based visualization of the productivity of individual workers. As a result, the solution can improve the productivity of the entire shop floor.
In this AI-based solution, software analyzes the videos taken by ordinary cameras. The software identifies and collects information about the skeletal positions of workers and identifies their postures and actions. Supervisors can use the results to detect operating mistakes and deviations from standard actions. The solution is applicable to any location using workers, not only in manufacturing companies but also in transportation companies, medical institutions, and nursing care facilities.
Co-creation with Lumada—Using videos taken by ordinary cameras on the manufacturing shop floor
This solution helps ensure consistent product quality. For example, a major cause of errors on the manufacturing shop floor is due to manual work performed incorrectly. By using this solution to identify the causes of such errors, you can prevent such errors from occurring.
Workers who perform manual work on the manufacturing shop floor make mistakes. For example, they might not follow the correct work procedure or might not carry out all the necessary steps in a procedure. Reasons include the following:
Such worker errors contribute to a higher product defect rate and can result in poor product quality.
Compared to experienced workers, novice workers might need more time to follow a difficult work procedure. To improve work efficiency, measures must be taken to ensure that the novice workers can complete their tasks as quickly as experienced workers.
To improve the quality of manual work and ensure consistent product quality, factory managers need to implement measures that reduce operating mistakes and ensure that workers are working efficiently.
The solution uses videos, taken by ordinary cameras, of the work performed on a manufacturing shop floor. Using AI (artificial intelligence), the solution software processes the video data and collects information about the skeletal positions of workers. Based on this data, the solution can then identify the postures of workers (for example, the movement of their hands and the directions their bodies are facing) and thereby identify their work-related actions. The solution detects changes in the actions of workers and the positions of parts, tools and other objects, identifies what the workers are doing, and then notifies an application of the workers’ actions.
Using this system, a factory supervisor can detect whether workers are following standard work procedures. The system can also issue an alert if the system detects any deviation from a standard work procedure: for example, if a worker is not carrying out all necessary steps or is not performing the correct work procedure. In this way, the system can address operating mistakes and ensure that workers follow the correct procedures. This ensures consistent product quality.
The system also analyzes information about the skeletal positions of workers and displays a chart indicating the time spent on each task. The chart enables factory supervisors to compare the time spent on each task and view videos of the work performed by workers. Supervisors can then analyze the factors to find out why a task takes a long time. Based on the analysis results, supervisors can provide guidance to individual workers to improve work efficiency.
In addition, factory supervisors can examine videos of workers who work efficiently to understand their creative efforts and body actions, and incorporate this best-practice knowledge into employee training programs to improve the productivity of the entire shop floor.
Generating data about the skeletal positions of workers from video footage and identifying actions by using a special analysis feature
This solution can estimate what workers are doing and can be used to improve the efficiency of manual tasks that formerly required intensive manual labor. The solution uses videos taken by ordinary cameras and does not require the expensive cameras or professional devices (such as sensors and markers) that were previously necessary to recognize human actions.
The AI-based solution identifies human posture and actions from videos taken by ordinary cameras. With this solution, video images are processed to generate data about the skeletal positions of workers, and a special analysis feature identifies the worker actions.
Key tasks of factory supervisors include reducing work-related injuries, ensuring consistent product quality, preventing production delays, and resolving other problems that might occur on the manufacturing shop floor. We provide support for these key tasks by using the solution to identify the causes of problems and then verifying the situation after steps are taken to resolve the problem. In addition to providing solutions, we also provide support for pre-implementation PoC (proof of concept) projects and actual implementations on the shop floor.
Three analysis models are employed in the AI-based solution that identifies human posture and actions. (This link provides details in Japanese.) These models are used to understand worker actions.
The figure below shows an example of how our solution can improve safety on a manufacturing shop floor.
When loading, carrying, or unloading parts on the manufacturing shop floor, good posture is essential. For example, workers should lift by bending their knees, squatting down, grasping the object, and then straightening the knees while keeping their back straight. Stooping over or adopting an otherwise poor posture increases physical strain and can result in injuries and accidents. To avoid such problems, we need to improve work processes to ensure, for example, that workers work with a sound posture.
The AI-based solution that identifies human posture and actions can analyze videos from cameras, collect information about the skeletal positions of workers, and automatically detect their postures. This solution also verifies whether the detected posture matches the posture predefined for the work. If a detected posture is deemed inappropriate for the work, the system issues an alert to prevent the worker from adopting a posture that might result in injury or an accident. This contributes to ensuring safety at work.
For details of this solution, follow the links below.
In this AI-based solution, software analyzes the videos taken by ordinary cameras. The software identifies and collects information about the skeletal positions of workers and identifies their postures and actions. Supervisors can use the results to detect operating mistakes and deviations from standard actions. The solution is applicable to any location using workers, not only in manufacturing companies but also in transportation companies, medical institutions, and nursing care facilities.
