The Digital Innovation Platform of Lumada, which supports the sustainable development and operation of social infrastructure systems, provides not only data collection, storage, processing, analysis, and visualization functions, but also following for: (1) Cooperation between heterogeneous systems, (2) Supporting collaborative creation (co-creation) with customers, (3) Collaboration with partner software development environments, (4) Management of application execution location, and (5) Sharing and re-use of developed solutions. This article describes the background and purpose of introducing microservices APIs and container technology, and the outline of the system-of-systems technology provided by the platform.
With the rapid spread of the Internet since the 1980s, internal IT systems have been interconnected with the IT systems of customers or other organizations. In addition, with the advent of cloud services since 2000, new businesses have been created using the huge amount of data stored on the Internet. Today, many companies are looking to invest in IT to create new businesses rather than to streamline existing businesses.
Emerging companies such as Amazon.com Inc. and Google LLC are creating innovative new services while developing new IT systems to support them. Furthermore, these companies have generalized their IT systems and commercialized them as a digital service platform for enterprises.
On the other hand, Hitachi has experience in providing a wide range of solutions. These include both operational technology (OT) systems for railways, buildings, power plants, and other social infrastructure and IT systems for the public-sector and the financial and manufacturing industries. In addition, through experience in developing and operating these social infrastructure systems, Hitachi has built a field support system that can respond to individual customer needs.
Therefore, Hitachi aims to create a service that provides new value to society by the “customer co-creation” method. In the first place, innovations for achieving the Sustainable Development Goals (SDGs) and improving quality of life (QoL) are difficult to achieve by a single company acting alone. In the coming era, the world that depends only on economic growth and free competition cannot continue. Based on social consensus on topics such as achievement of the SDGs and QoL improvement, it is necessary to seek wisdom from various organizations and to collaborate to solve problems.
Furthermore, from the solutions developed through this customer co-creation, useful software components are cut out, refactored so that they can be used universally, and their accumulation, sharing, and re-use are encouraged. As a result, while accelerating the evolution of software, not only IT, but also Hitachi’s strength, OT, will be combined to continuously develop a service platform that supports cyber-physical systems. At the same time, Hitachi would like to provide the service platform as the Digital Innovation Platform to customers and partner companies.
To promote the approach described in the previous section, it is necessary to link existing systems with new systems for artificial intelligence (AI) and Internet of Things (IoT) applications, as shown in Figure 1. Unlike the digital world, due to the existing giant social infrastructure systems that include many physical field devices and their control software, it is not realistic to try to replace whole systems at once.
However, as shown in Figure 1, the technologies used by the existing OT/IT system and the new system differ greatly in hardware and software. In addition, the existing system on the left side of Figure 1 is based on the culture of waterfall development whereas the right side is based on the culture of agile development(1). Bridging these two different cultures is an important role of the Digital Innovation Platform.
Fig. 1—Linking Existing System and New System
Some means is needed to link new IoT, AI, or other such systems developed using agile development to existing systems created by waterfall methodologies in a way that can bridge these distinct development cultures and the different programming languages, tools, and methodologies they use.
Fig. 2—Positioning of Digital Innovation Platform
The Digital Innovation Platform enables loosely coupled integration with existing OT and IT systems, thereby serving as a basis for the development, accumulation, and re-use of applications in various different business domains and providing the functions for utilizing these as a resource for collaborative creation with customers.
Through working with customers on collaborative creation, Lumada aims to create value from data and accelerate digital innovation. To support this value creation, Lumada’s architecture is designed to enable the rapid and efficient development of solutions and services that utilize advanced digital technologies. In the following, Figure 1 shows too much implementation detail for the system and so a simplified version is provided in Figure 2. In Figure 2, the existing system is separated into OT and IT and placed on the left and right.
As shown in Figure 2, the Digital Innovation Platform is included in Lumada along with the customer co-creation methodology “NEXPERIENCE” developed by Hitachi and solutions and customer cases for each business domain. Originally, the Digital Innovation Platform also included physical products such as advanced sensors and railway vehicles. The Digital Innovation Platform is a general-purpose software platform that does not depend on individual business domains.
Lumada is comprised of six main types of functional component: Edge, Core, Data Management, Analytics, Studio, and Foundry.
However, these functions alone cannot solve the problems described in the previous section, namely cooperation between heterogeneous systems, co-creation with customers, the accumulation and re-use of solution development results, and the establishment of an ecosystem that encourages this. In addition to the above, the Digital Innovation Platform provides the following features as a solution to these problems.
The container technology and microservice(4) APIs introduced on this platform are closely related to the realization of functions (a) to (e) above. In the following, this article will explain in detail focusing on (a) in particular. For the details of (b) to (e) above, refer to the articles, Hitachi’s Use of Node-RED for Rapid Solution Development and Associated OSS Activities” and “Lumada Solution Hub for Accelerated Development and Deployment of Digital Solutions.
Fig. 3—Digital Innovation Platform and Existing OT and IT Systems
Given that data is sourced from existing OT and IT systems, ease of connectivity with data analytics applications that run on the Digital Innovation Platform is an important requirement.
Figure 3 shows the interconnection between the Digital Innovation Platform and existing systems. It is the existing OT and IT systems that actually generate the data. Therefore, in order to analyze those data with a new application system running on this platform, these systems must be integrated into one system (a system-of-systems(5)).
The existing system part in Figure 3 often runs on the customer’s on-premises environment or on the public cloud of another company. Therefore, Lumada uses the global IT infrastructure of public cloud providers such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP). Hitachi will also provide a multi-cloud operation management platform that enables integrated management of the entire application system across multiple clouds.
Fig. 4—Realization of System-of-systems by Microservice APIs
The architecture of Lumada is designed to link different systems together into a “system-of-systems” that spans multiple clouds, with the industry-standard HTTP/REST protocol and JSON data formats used to facilitate this integration with external systems.
Next to the connection of the lower layer described above, the interconnection of the upper layer (application layer) must be realized. In the case of interconnecting systems that have been independently developed and operated, the system-of-systems architecture based on loose coupling using message communication shown in Figure 4 is generally used. In recent years, microservice APIs using the HTTP/REST(6) protocol and JSON(7) data format have become the mainstream means of providing this message communication. With the spread of the Web and smartphones, these technologies are widely used for linking applications, and have already become de facto standards.
The Digital Innovation Platform recommends the use of microservice APIs for interconnecting the application layer to ensure ease of connection with applications and software as a service (SaaS) provided by other companies. While existing systems, as shown in Figure 4, will not necessarily offer a microservice API, they will provide an external interface of some sort and this can be wrapped by API management(8) and converted into a microservice API.
The microservice APIs described in the previous section only define the framework (communication protocols and data formats). In order to actually connect systems, the detailed semantics of communication messages are required along with data format definitions. Although various organizations have been discussing standardization and open source software (OSS) development, flexible implementation of multiple standard specifications is indispensable when globally linking systems that span different business fields. Therefore, the Digital Innovation Platform provides the following functions to facilitate the development of applications that link multiple different systems.
Specifically, the Digital Innovation Platform provides the components shown in Table 1 in order to facilitate linkage of heterogeneous systems. Node-RED*1, (9) belongs to Studio and provides a function that allows users to connect systems to each other at the API level using a graphical user interface (GUI). Pentaho and the digital twin solution(10) belong to Data Management and enable the division, conversion, and combination of data required to connect heterogeneous systems. Hitachi Data Hub(10) belongs to Edge and enables the protocol conversion that is essential for data collection from IoT devices and other systems. Hitachi Application Framework/Event Driven Computing (HAF/EDC)(11) provides performance scaling according to the expansion of system scale and data volume, and reliability enhancement by redundant configuration. In addition, Lumada Solution Hub(12) makes it possible to store, share, and re-use developed solutions.
Table 1—Lumada’s System-of-systems Technology
The system-of-systems technology available on Lumada provides rapid and flexible ways of working with multiple international or industry standards, an essential requirement when implementing new cross-industry services on a global scale.
The situation in which the “hypervisor method is the best choice for virtual machines” changed dramatically in the last five to six years with the advent of container technology and the establishment of the Cloud Native Computing Foundation (CNCF)(13) under the Linux Foundation(14). Today, not only the pioneer Google, but also major vendors such as Amazon, Microsoft Corporation, Red Hat Inc., and VMware, Inc. are providing serverless computing environments using container technology.
The advantage of container technology is that it consumes less memory than hypervisor technology and its performance is superior. Container technology also solves the version inconsistency problem of operating system (OS) and common libraries that has plagued engineers when using a mix of different middleware and applications. In addition to these strengths, Hitachi has introduced container technology to the Digital Innovation Platform for the following purposes.
In addition to Java and C/C ++ that have long been used for system development, Python has become mainstream in the data analysis and AI fields, and JavaScript is the main programming language used in Web application development. In the OT field, engineering software such as ANSYS*2, MATLAB/Simulink*3, and SolidWorks*4 has become popular.
Using container technology and microservice APIs makes it easy to connect applications created in these various languages and development environments by message communication and to integrate them into one system. By applying the system-of-systems technology described in Chapter 4 in this way, it is possible not only to link old and new systems, but also to easily connect software components by overcoming the differences in programming languages and development environments used for application development.
As the importance of data is widely recognized, there are trends to regulate the movement of data. The European Union (EU) General Data Protection Regulation is one example. However, if container technology is applied, it is possible to transfer and execute the application near the data storage location without moving the data.
In agile development of applications that use IoT and AI technology, there are many cases where it is difficult to predict the necessary computing performance and communication traffic. Applications that did not cause problems at the proof-of-concept (PoC) stage may exceed allowable delay times in a production environment where the number of connected devices is higher, or may suffer from high costs due to the increased level of communication traffic. However, use of container technology can improve the flexibility of system design by, for example, using distributed execution of applications near the edge.
This article has explained the background issues, technologies that should be introduced, and functions that should be provided, from the perspective of an architect, regarding the design concept of the Digital Innovation Platform for Lumada. In the design of this platform, Hitachi has sought to promote continuous and gradual software evolution. In order to put this into practice, the concept of a system-of-systems is also recognized as important. This involves linking existing systems to new systems and interconnecting applications created in different programming languages and development environments.
Node-RED and Lumada Solution Hub were beyond the scope of this article and are instead covered by subsequent articles in this issue of Hitachi Review. These provide functions such as support for co-creation with customers, cooperation with partner software development environments, management of application execution location, and support for the sharing and re-use of developed software.
