Hitachi opened its Center for Technology Innovation – Artificial Intelligence in April 2020 in response to rising expectations for the practical application of AI. The center is drawing on a long history of AI research to bring about the DX of corporate activity and improvements in people’s QoL. This article presents examples of this work, including projects that make use of Hitachi AIs for natural language processing and video analysis that have demonstrated their capabilities by gaining top prizes in international competitions at TRECVID, CoNLL, and SemEval. One example involves making the decision-making process at financial institutions more efficient while another deals with monitoring the physical load on workers at manufacturing workplaces. The article also covers the development of Lumada CPSs that operate across both physical and cyber space to ensure that the benefits of DX and enhanced QoL have the resilience to withstand sudden changes in external circumstances.
Internationally, artificial intelligence (AI) is currently experiencing its third boom. The earlier booms in the 1960s and 1980s proved to be disappointments, failing to deliver research outcomes that lived up to society’s expectations at a practical level. With this third boom, however, the emergence of deep learning has brought reports of AI research that has succeeded in surpassing the capabilities of humans within certain narrow fields, accelerating the use of AI in social deployments.
Hitachi has a long history of AI development aimed at delivering customer value in the form of improvements in the quality of people’s private and working lives as well as digital transformation (DX) to improve productivity and efficiency at companies and elsewhere. AI research at Hitachi dates back to an intelligent robot exhibited at a Hitachi Technology Exhibition in 1970. The robot showed off its ability to read assembly drawings and identify the shapes of building blocks on a desktop, and to assemble the blocks autonomously. This was followed by further research and development as well as progress on commercialization in applications such as document analysis, conceptual search, sign language recognition, image search, speech synthesis and recognition, machine translation, geographical information processing, remote sensing, finger vein authentication, and video analysis. The period since 2015 has seen research teams on sensor networks, robotics, consumer devices, and healthcare informatics coming together, leading ultimately to the opening of the Center for Technology Innovation – Artificial Intelligence in April 2020. Many researchers are based at the newly established Kyōsō-no-Mori facility at Hitachi’s Central Research Laboratory in Kokubunji, Tokyo where they work on AI-related topics. This has included the development of numerous technologies that have gained positive recognition from the academic community and at the Text Retrieval Conference Video Retrieval Evaluation (TRECVID), Conference on Computational Natural Language Learning (CoNLL), and Semantic Evaluation (SemEval) international competitions.
No matter how good the technology, putting AI to use in business has tended to take a lot of time and money. Hitachi has been working to overcome this problem. The Lumada Data Science Laboratory was established in April 2020 to bring research and business together to advance commercialization as well as research and development. The laboratory makes use of agile research and development processes to put its research to practical use at an early stage, having a pool of talent that can respond dynamically and using open innovation to bring in outside expertise. In this way, it can quickly put together proprietary Hitachi solutions that combine AI with domain knowledge in operational technology (OT) acquired through collaborative creation (co-creation) with customers.
This article presents examples of AI developed by Hitachi to provide two-fold customer value in the forms of delivering DX and improving quality of life (QoL). It also covers the associated work on cyber-physical systems (CPSs) that acquire data from the physical spaces where companies and people go about their activities, using recognition and analysis to build models in cyberspace and supplying the results back to the physical world as feedback.
The following sections present two examples in which AIs developed by Hitachi are used for the DX of corporate activity.
Figure 1 — Demand Prediction AI
The system models demand cycles, patterns of fluctuation, and external influences, and uses artificial intelligence (AI) to consolidate their predictions.
Failures to correctly anticipate demand can lead to higher costs and lost opportunities at distribution centers due to over or under stocking or excess labor and freight overheads. Another issue is that such estimates rely heavily on the experience of expert staff. The number of products handled by distribution centers typically runs into the tens of thousands, with demand fluctuations and the influence of external factors such as events or the weather varying widely from product to product. In response, Hitachi has built a number of models for predicting the influence of particular external factors as well as the long- and short-term cycles and the patterns of sudden fluctuations in the volume of goods. AI techniques have been developed that combine these to predict demand on a product-by-product basis with a level of accuracy that was difficult to achieve in the past. Their use in practice is also promoted by automatically attaching explanations to the AI predictions so that they will make sense to the people who use them. In a trial of distribution center operation based on demand predictions made using these techniques, Hitachi Transport System, Ltd. succeeded in reducing operating costs (see Figure 1).
Figure 2 — Financial Reporting Documents for Security
The shaded text is extracted using an AI. The difficulty of extracting text based on specified keywords and text patterns means that a syntax analysis of the surrounding text is also needed if a practical level of performance is to be achieved.
Financial institutions use information from sources such as the large number of financial reporting documents released by the Financial Services Agency as a basis for making decisions. A thousand or more of these documents are released each month, with some being more than 300 pages in length. Past practice was for staff to read these documents and extract 130 more items of information, work that could take months. In response, Hitachi has developed a way of identifying and extracting the information needed for investment and lending decisions from the text of financial reporting documents like these (see Figure 2).
A key role is played by StruAP(1), a tool for identifying relationships that analyzes the syntactic dependencies in the text, abstracting them in the form of a syntax tree to generate tree structure patterns. It then performs pattern matching on the syntax tree to obtain information about the relationships between phrases. By doing so, information can be extracted from reports in less than half a day, with better and more uniform data quality among the other recognized benefits. The technology won a Field Innovation Award from the Japanese Society for Artificial Intelligence (JSAI) for its use in achieving DX at a customer’s business.
In addition to making businesses more efficient, AIs developed by Hitachi are also used to improve the QoL of workers. The following sections describe two such applications.
Figure 3 — Experimental Screen for Comparing and Assessing Differences from Actions of Model Worker
The screen shows appropriateness scorings for actions based on a body load assessment (top left), a video of an action deemed to be inappropriate (top right), a trace graph of scores for different parts of the body (bottom left), and the scores for actions by each part of the body (bottom right).
AIs that use video and sensors to recognize the movements of people are deployed in manufacturing, maintenance, logistics, and other such workplaces. As recognition accuracy in these applications varies depending on what sort of work the people are doing, it is necessary to provide a range of recognition options to suit different circumstances. Hitachi supplies solutions that enhance worker safety as well as improve productivity and quality by combining worker position and path recognition, posture and action recognition, and fine-grained movement recognition(2).
Figure 3 shows an example of multimodal recognition of worker actions that is intended to improve safety. While it is important in factories and other workplaces for staff to learn appropriate working practices and skills, one of the challenges of skills transfer is that such practices are often difficult to document in words. In response, Hitachi has built an AI for identifying how highly skilled staff differ from ordinary workers in terms of where they perform actions and which parts of their body they use. This is done by first having the participants wear a special suit (wearable device) to record data on the bodily movements of skilled staff and the actions of ordinary workers. The data is then used to train a Siamese network, a type of neural network used in AI. By identifying the places and body parts where significant differences occur, this provides a way to determine the physical stress on workers, identify dangerous work postures, and highlight where practices can be improved. The technique was developed through joint research with the German Research Center for Artificial Intelligence.
Figure 4 — Relationship between Parasympathetic Nerve Activity and Accident Risk
Heart rate variation is used to estimate autonomic nervous function, an indicator of fatigue, so as to determine its cause-and-effect relationship with accident risk (near-miss incidents).
Hitachi has developed a technique that highlights the cause-and-effect relationships between work productivity or human error and people’s tiredness or stress estimated by using vital signs data from sources such as heart rate and pulse sensors. This has potential uses in preventing traffic accidents, for example, by collecting sensing data from people doing delivery work and analyzing the correlation with accident risk. Hitachi has conducted an analysis of sensing data and found a correlation between parasympathetic nerve activity and accident risk (see Figure 4). The results of this analysis have been used in a function for estimating driver fatigue from heart rate variations that can alert the driver or their supervisor when necessary. It has been implemented on 1,200 commercial vehicles operating out of 80 locations in the delivery operations of Hitachi Transport System. Future plans include its use to prevent errors and accidents in factory or social infrastructure maintenance work.
There is an expectation that the benefits of DX or improved QoL will be resilient, meaning their provision will not be interrupted by sudden changes in external circumstances. This section describes the research and development of the Lumada CPSs that provide this resilience.
Hitachi is directing its efforts at creating Lumada CPSs that bring the cyber and physical spaces together to achieve innovation by combining OT and AI (see Figure 5). The features of these CPSs are that they have separate growth and evolution loops and that they can all be treated uniformly regardless of what they are used for and the scale of the products, systems, or society involved. Here, “growth” means promoting growth by rapidly working through a loop in which physical space is replicated in cyberspace where AI provides ways of dealing with things more efficiently, and then using this as feedback to the physical world to facilitate taking action. Similarly, “evolution” relates to cases where existing systems are no longer able to cope due to changes in external circumstances, objectives, or the mix of stakeholders. It refers to the use of cyberspace to simulate things like how to mitigate constraints or switch from one system to another so as to identify solutions that satisfy key performance indicators (KPIs), using this as a means of building a consensus among stakeholders to upgrade the relevant parts of the physical world. These two loops can deliver ongoing value enhancement even in the face of sudden changes in complex societies.
This research is at the forefront of work on world-leading technology, being undertaken in partnership with the Montreal Institute for Learning Algorithms (MILA), Stanford University, and the University of California, Berkeley. A practical application being worked on by Hitachi is collabotics for facilitating commissioning and modification in the automation of industrial robots.
Figure 5 — Lumada CPSs
By linking the cyber and physical spaces and working through separate growth and evolution loops, Lumada CPSs can continue delivering consistent benefits in the face of complex societal challenges that are subject to high levels of uncertainty.
AI plays a central role in both the CPS growth loop and evolution loop. The particular forms of AI are those for sensing analysis, for prediction and solution searching, and for explaining the action to be taken.
Sensing analysis AIs that replicate the physical world in cyberspace work by using a geographic information system (GIS) as a platform on which to combine data acquired from sensing with data on the spatial structure of the physical space. One example would be to perform high-level recognition of people’s actions by overlaying architectural drawings with the locations of people acquired by techniques such as laser range finding.
In the case of AIs for prediction and solution searching that generate predictions from historic data, the accuracy of machine learning models varies widely depending on the skill of the person who built them. In response, Hitachi is developing techniques for the automatic identification of feature values to overcome this human dependence and for incorporating customer domain expertise into its prediction models. Hitachi is also engaged in research that uses reinforcement learning to seek solutions to real-world challenges, this technique having demonstrated better-than-human judgement in games like shogi and chess.
Finally, explainable AI is able to explain why an AI made the decision it did. When making use of AI decision-making in the real world, there is a need to explain the reasons for decisions and to achieve a consensus among the stakeholders involved. Unfortunately, because AI techniques such as deep learning models are extremely complex, people will not feel confident about using their decisions in the workplace if they do not understand the reasoning behind them. In response, Hitachi is developing techniques for explaining the reasons behind a wide variety of AI decisions in ways that are easy to understand.
This article has given a history of AI research at Hitachi and described the activities of its Center for Technology Innovation – Artificial Intelligence together with examples of practical applications.
As well as working with the Lumada Data Science Laboratory to hasten the social deployment of Hitachi AI, Hitachi also intends to live up to the expectations of society by combining this AI with the OT and IT it has built up with customers.
