Factors such as the spread of COVID-19 are amplifying concerns about resilience, meaning the flexibility to adapt to changes in the social environment. This article describes edge analytics from Hitachi that transcend the constraints of time, space, and people to facilitate the transition to remote work and other aspects of the new normal, considering this in the context of emerging new technologies such as 5G and 6G mobile communications, AI, AR/VR, data centers, and the cloud as well as how the pandemic has brought a change in the direction of events. It also describes example solutions and other work being done in the areas of practical combinatorial optimization problems, urban infrastructure maintenance, the use of odor sensing for spatial or other condition measurement, and the automation of industrial work processes.
Figure 1 — Lumada CPS with Computing System for Running Edge Analytics
The true worth of edge analytics is manifest when 5G/6G edge computing systems are used in tandem with Lumada in cyberspace and physical space.
The COVID-19 pandemic has encouraged working from home and other measures for providing places where people can remain safe. Numerous apps intended to make people’s lives easier have become available, with the global market for non-contact and other remote sensing services forecasted to reach USD21.62 billion in 2022(1). Recent years have seen a relentless rise in the technologies underpinning this trend, including IT, fifth- and sixth-generation mobile communications (5G and 6G), artificial intelligence (AI), and cross reality (XR) [a general term that combines both augmented reality (AR) and virtual reality (VR)]. 5G enables the sending and receiving of video or workplace data at such high speeds that users are unaware of any delays. AI is being used to replace a wide variety of manual tasks and jobs, being able to rapidly extract the desired information from workplace data. XR, meanwhile, along with its role in lifestyle innovations such as eSport, is also used for modeling and controlling construction machinery or factory equipment and to assist with the automatic operation of electric vehicles or railways.
The Internet of Things (IoT) makes full use of these new technologies to link devices together for data collection (sense) and analysis (think), and to make use of the associated results in control (act)(2). Unfortunately, simply using networks to store data in the cloud does not on its own make it any easier to work through the steps of “sense, think, and act.” This is because the quantity of data involved is huge and the processing needed to deliver practical solutions takes too long.
This has directed more attention toward edge analytics, which means shifting the load of data analysis out to the network edge (see Figure 1). Edge analytics involves the collection of data generated by operational technology (OT), including audio and video as well as data from measurements of temperature, humidity, pressure, distortion, vibration, magnetism, or odor. The data is then analyzed by edge AI or machine learning and the results are put to use in controlling equipment. By providing an impetus for digital transformation (DX) in ways that make people’s lives better, edge analytics transcend the constraints of time, space, and people to make remote work or remote control feel more natural, to provide safety and comfort in new living practices, and to improve productivity through workplace automation.
In considering Hitachi’s edge analytics, this article places a particular focus on resilience in the sense of robustness and the flexibility to adapt to change, presenting examples that use Hitachi’s own computing, sensing, and AI technologies.
The following sections present examples in which edge analytics is used to solve problems involving the development of a combinatorial optimization technique for use in the workplace, infrastructure maintenance, environmental measurement, and the automation of work.
The requirements for business continuity during the pandemic are continually changing. This makes it difficult to run a business using plans formulated on the basis of predictive models. What is needed, rather, is the ability to react promptly to unexpected circumstances and to choose the best course of action at any given time. Many problems that involve seeking the best course of action can be solved by combinatorial optimization, a technique for maximizing particular key performance indicators (KPIs) within the constraints set by the available resources (time, cost, and so on). The difficulty is that the computation time increases dramatically the more combinations there are that need to be considered.
Hitachi has developed complementary metal-oxide semiconductor (CMOS) annealing as a means of overcoming this explosive growth in computation time. In this context, problem size is indicated by the number of elements (called “spins”) that are involved in the computation. Current practice is for combinatorial optimization computations to run not at the edge, but in the cloud, which has the advantage of occupying a system-wide vantage point. On the other hand, high-speed execution on the edge is required when optimization needs to respond immediately to changing conditions in applications such as the detailed control of robots and factory equipment, which are significantly affected by latency differences between the cyber and real worlds.
At the 2019 International Solid-State Circuits Conference (ISSCC), the world’s foremost forum in the field of semiconductor integrated circuits, Hitachi announced a CMOS annealing machine with what at the time was a world-leading number of spins (approximately 60,000). The annealing machine was implemented as a business-card-sized device suitable for edge applications [see Figure 2 (a)]. Computational resources and space availability are limited at the edge of IoT systems. While the business card size of the device overcomes these particular constraints, actual constraints vary widely between sites and there is a need for scalability to cope with the largest problem sizes within the resources available in each case.
Accordingly, Hitachi went on to announce a prototype at ISSCC 2021 that features nine chips per board and provides a world-leading 144,000 spins [see Figure 2 (b)]. Hitachi has also demonstrated how the number of spins can be augmented by connecting boards together in a tile configuration to build nodes. This means that systems can start with a single-chip configuration and then be scaled up with theoretically no limit(3). This makes CMOS annealing available in a form that can cope with the many constraints of edge computing, thereby opening up the potential for further development of edge-based optimization applications. More information about the current state of application development can be found in another article in this issue of Hitachi Review(4).。
Figure 2 — CMOS Annealing Machine for Edge Applications and How it Handles Large Problem Sizes
Board-to-board connectors are used to connect CMOS annealing processor chips together in a nine-chip (3 × 3) configuration, thereby enabling the computation of large optimization problems.
Figure 3 — Use of Vibration Sensing for Water Leak Detection
Leaks are detected by measuring vibrations in underground water pipes.
Problems with infrastructure become more frequent as it ages. Along with outwardly visible instances such as roads being affected by ground subsidence or the fracturing of the steelwork in bridges, other forms of degradation that are not directly visible are also starting to happen with greater frequency, including leaks from underground water pipes or short-circuit faults in street lighting. Past practice for infrastructure maintenance has been for experienced staff or inspection vehicles to visit sites and conduct routine checks at intervals of one to several years. Unfortunately, higher costs resulting from the delayed identification of problems and other such issues combine with the shortage of experienced staff to make the provision of effective maintenance services increasingly difficult.
To overcome this problem, Hitachi has built an infrastructure maintenance platform that uses IoT sensing(5). This works by installing sensors at the site of infrastructure to collect data and identify leaks or electrical faults in real time. The edge sensors connect to a cloud server by means of low-power wide-area (LPWA) or other IoT communications and the data is analyzed and presented on a monitoring platform. This can detect various different forms of degradation and enables the provision of timely maintenance.
In the case of water leak detection, Hitachi has developed its own ultra-sensitive vibration sensing technique that features a proprietary algorithm (see Figure 3). The effectiveness of the technique has been demonstrated in trials where it was able to detect not only very small underground water leaks, but also leaks in noisy environments such as near major highways or railway lines that even experienced inspectors find difficult to detect. Hitachi also succeeded in cutting costs by providing a long battery life, thereby reducing the frequency with which the sensors themselves require maintenance.
Figure 4 — Air Sensing Using Smartphone and Sensor
Data collected by a gas sensor that plugs into a smartphone is uploaded to the cloud where it is analyzed to detect what is in the air.
Demand for safe places and clean air has been growing around the world as a result of the COVID-19 pandemic. There is a need for ways to measure the cleanliness of the environment in enclosed spaces such as trains or elevators in order to facilitate access restrictions and ventilation to prevent indoor infection. Past uses of air sensing have included fire prevention (in the form of fire alarms and gas chromatographs), the control of air purification systems, and the use of scent to provide comfortable living spaces. Unfortunately, these past methods did not offer much flexibility, requiring fixed sensors to be installed and limited in terms of the range of gases or odors they can detect.
Hitachi has developed a small and lightweight multi-semiconductor gas sensor that is portable and able to detect a wide variety of odors(6). Figure 4 shows a prototype and its potential applications. The sensor module, which is roughly the same width as a smartphone, is used by connecting it to the smartphone connection port. The device collects data from five semiconductor gas sensors and forwards this data via the smartphone to the cloud, where machine learning is used to classify many different odors with a sensitivity on the order of parts-per-million (ppm).
A solution incorporating this technology is used for odor monitoring and air quality control at a wide variety of sites. At industrial plants, it is used for quality management and to detect anomalies on production lines or in warehouses. Potential environmental measurement applications include use in waste management. In human measurement, it has uses in checking the physical condition of staff, health management, and the quantification of feelings to determine the suitability of switching work tasks. Other potential applications that relate to site resilience include air quality monitoring in elevators or smoking areas, access control, and air cleaning control solutions.
Figure 5 — Use of Motion Planning Accelerator to Speed up Robot Operation
The robot motion planning accelerator enables the automation of computationally heavy non-routine tasks such as an AGV navigating around obstacles.
The shift to remote working and the shrinking workforce have put growing momentum behind the use of robots to perform complex manual tasks. One such example is the rapid recognition, sorting, and picking of objects. The handling of standardized objects such as boxes is comparatively simple and can be implemented without too much difficulty using conventional deep learning. Non-routine tasks such as an automatic guided vehicle (AGV) navigating around obstacles, on the other hand, are much more difficult, with the very long computation times needed prior to execution estimated to be 100 times or more what is required for routine tasks.
In response, Hitachi has developed a robot motion planning accelerator to speed up computation for non-routine tasks(7) (see Figure 5). The hardware implementation of the accelerator uses a field-programmable gate array (FPGA) equipped with central processing unit (CPU) cores. Motion planning is done by: (1) identifying any obstacles in the intended robot path, (2) generating a map of the area around the objects that the robot will manipulate in performing its task, and (3) searching the map to identify the optimal path that avoids collisions with obstacles. Use of the above hardware and its dynamic circuit reconfiguration capability speeds up the three computations used for motion planning. Search times are also shortened by coordinating the operation of software and hardware so that multiple map searches can run concurrently.
In a trial that combined the above technologies, robot motion planning was completed in 500 ms, more than 100-times faster than the software-based technique used previously. Hitachi intends to continue upgrading this technology to facilitate the automation of a wide variety of complex tasks in the future, including by combining the accelerator with high-speed recognition AI to further improve obstacle recognition performance. The aim is to use it to provide system-wide monitoring and the automation of equipment in smart factories and supply chains.
This article has presented examples of edge analytics at Hitachi, including a technique for the edge implementation of combinatorial optimization and solutions that are used for infrastructure maintenance, environmental measurement, and industrial resilience.
In the future, Hitachi plans to develop edge semiconductors that are tailored to the needs of customers and to deploy edge analytics that make use of 5G and 6G mobile communications to achieve mission critical performance in the sense of not shutting down when confronted with obstacles.
The sections describing combinatorial optimization and automation by robots draw on work done in a project (JPNP16007) commissioned by the New Energy and Industrial Technology Development Organization (NEDO). The author would like to take this opportunity to express his deep gratitude to everyone involved.
