Hitachi Review

Innovation of Software Development Process across Hitachi Group

Skip to main content

Hitachi

Hitachi Review

Foundations of Next-generation MONOZUKURI Transformed by Digital Technology

Innovation of Software Development Process across Hitachi Group

Highlight

Providing new value to society on the basis of data by incorporating software and networking into a wide variety of products is one of Hitachi’s business strategies. In the increasingly complex field of software development that is growing exponentially in scale, achieving this call for innovations that will put management practices in place that are fit for the digital era. This article describes a new company-wide scheme that was established in FY2018 to enhance software development capabilities. It also describes leading edge technologies that support development process innovation, namely a technique for putting software development processes in place that draw on synergies from across the Hitachi Group, which has extensive experience with software development, and the development of a risk identification technique for development projects using artificial intelligence. It also gives a practical example of process innovation in the form of the implementation of keyword-driven testing in continuous integration.

Table of contents

Author introduction

Hideto Ogawa, Ph.D.

  • System Productivity Research Department, Center for Technology innovation – Systems Engineering, Research & Development Group and Software Modernization Laboratory, Research & Development Group, Hitachi, Ltd. Current work and research: Research and application of software engineering technology. Society memberships: The Information Processing Society of Japan (IPSJ), The Institute of Electronics, Information and Communication Engineers (IEICE), and the Japan Society for Software Science and Technology (JSSST).

Masumi Kawakami

  • System Productivity Research Department, Center for Technology innovation – Systems Engineering, Research & Development Group and Software Modernization Laboratory, Research & Development Group, Hitachi, Ltd. Current work and research: Research and application of software engineering technology. Society memberships: IPSJ, the Society of Project Management (SPM), IEEE.

Hiroto Kaga

  • System Productivity Research Department, Center for Technology innovation – Systems Engineering, Research & Development Group, Hitachi, Ltd. Current work and research: Research and application of software engineering technology. Society memberships: JSSST.

Tatsuaki Osafune, Ph.D.

  • System Productivity Research Department, Center for Technology innovation – Systems Engineering, Research & Development Group, Hitachi, Ltd. Current work and research: Research of software engineering technology. Society memberships: IEEE.

1. Introduction

Software development is an essential feature of the today's MONOZUKURI (manufacturing). As exemplified by terms such as the Internet of Things (IoT) and “connected-X,” software not only provides functionality similar to what already exists, in a world where all sorts of different product incorporate software and networking, it is also crucial to the delivery of enhanced customer value, such as more complex and precise control and high levels of optimization based on data analysis.

To achieve this, the scale and complexity of the software contained in products is growing exponentially (see Figure 1). A survey by the Information-technology Promotion Agency, Japan (IPA)(1) identified the improvement of design quality, boosting of development capabilities (in terms of volume), and keeping up with technological trends as the top three issues in embedded software development, with the challenge being to achieve both quality and volume in software development while also dealing with the steady stream of new technologies that are emerging. Innovation in software development will be an essential management practice for creating customer value using the next generation of technology.

In FY2018, Hitachi launched a new company-wide initiative for software development innovation. The initiative involves establishing practices that will drive organizational innovation and developing technology that will drive development process innovation. This article presents examples of organizational innovation in Section 2 and of development process innovation in Sections 3 and 4.

Figure 1—Increasing Size of Embedded SoftwareThe size and complexity of the software contained in products is rising exponentially.

2. Company-wide Scheme for Enhancing Software Development Capabilities

Hitachi includes a MONOZUKURI Strategy Division and Corporate Quality Assurance Division, two company-wide specialists in manufacturing and quality assurance that routinely engage in improvement activities across the Hitachi Group. Embedded systems improvement work has also been ongoing since 2004 to enhance software development throughout Hitachi(2). While these company-wide activities have continued to deliver improvements, Hitachi also revamped the measures it has in place at a company-wide level to enhance software development capabilities in FY2018, drawing on the rapid advance of software technology over recent years to make fundamental changes in how it goes about software development (see Figure 2).

Figure 2—Measures in Place at Hitachi for Enhancing Software Development CapabilitiesHitachi is adopting company-wide measures that draw on group know-how.

2.1 Software Development Capability, Reliability Enhancement Project

The Software Development Capability, Reliability Enhancement Project, the “Software Project” for short, was launched as a headquarters initiative in FY2018. The mission of the Software Project is to enhance development capabilities by drawing on synergies from across the Hitachi Group, which has extensive experience of different types of software development. This involves working with the software enhancement leaders appointed in each division to assess the state of software development in their respective organizations. The project also collects details of the issues confronting the various divisions together with leading examples and know-how so that this knowledge can be spread across other divisions to overcome the challenges they face.

2.2 Software Modernization Laboratory

The Research & Development Group established the Software Modernization Laboratory (SML) to operate across different research departments. The mission of the SML is to promote the modernization of Hitachi's software development techniques. It consists of researchers with expertise in software development and pursues the use and creation of advanced software development techniques. It also works with the Software Project described above to modernize development through the use of advanced technologies from inside and outside Hitachi.

The following sections describe some of the innovative techniques developed by SML and give examples of process innovation in the business divisions.

3. Techniques for Software Development Innovation

An effective way to go about innovation in software development is to work through a loop that involves observing how development teams work in practice, using this as a basis for understanding what is actually happening, identifying actions based on this understanding, and then putting these into practice. Also vital to the success of projects is continual monitoring and assessment while they are in progress to identify development risks at an early stage, and to reduce risks by taking prompt action in response (see Figure 3).

The SML drives innovation in software development by developing techniques to support organizational-level improvement and real-time action on projects while they are in progress.

For organizational-level improvement, it developed the best practice sharing portal service to assist with reviewing the software development capabilities of organizations (situation assessment) and with the formulation of development processes (policy making).

To facilitate real-time action, the SML developed a project management artificial intelligence (AI) which collects information on in-progress projects (observation) , identify risks (situation assessment), and recommending action (policy making).

Figure 3—Scheme for Enhancing Development CapabilitiesThe scheme works through a loop of observation, situation assessment, policy making, and implementation in terms of both organizational-level improvement and the taking of real-time action at the project level.

3.1 Best Practice Sharing Portal for Improvement at Organizational Level

A common approach to raising the level of software development capabilities is to define a standard development process and utilize it across different product developments. When a company is involved in such a diverse range of products as Hitachi, however, there will be instances where having a single defined development process will prove inadequate to the circumstances that pertain to a particular product. Accordingly, what is needed to raise the level of software development capabilities across the Hitachi Group is a way to observe and assess the situation for particular products and to establish a development process that is tailored to each product.

To facilitate this, a best practice sharing portal was developed as a web service to support development process innovation. The portal provides a self-assessment platform for reviewing the software development capabilities of an organization and a mechanism for recommending practices to put together a development process based on the assessment.

The portal defines activity standards for reviewing the level of development capabilities in an organization in terms of development process, architecture, and design and development methodologies respectively. Development capabilities can be reviewed by comparing the organization's current activities with these defined standards.

The portal also has a library of practices (specific methods and examples of how activities are put into practice) and can recommend specific actions for improving development capabilities. These practices are made up of leading examples and know-how collected by the company-wide scheme described earlier, utilizing the extensive knowledge of Hitachi to enable new processes to be put in place (see Figure 4).

Figure 4—Overview of Best Practice Sharing Portal ServiceThe service supports evaluation of the activities undertaken by a project and the establishment of processes using the collected practices.

3.2 Project Management AI that Facilitates Realtime Actions

With systems such as those for ticket management or product management being adopted to make software development more efficient, an increasingly common approach is for project managers to use the collection of information from these different systems as a basis for running their projects. The problem, unfortunately, is that it can be difficult for project managers to acquire accurate information in projects with 100 or more developers or that span multiple sites, making them impossible to manage effectively.

In response, the project management AI helps keep track of development projects by collecting data during the project and analyzing it using repository mining techniques. The AI provides useful information to project managers by presenting it in a variety of different ways (views), evaluating the state of the development project in a comprehensive manner to identify any risks that have emerged, and issuing risk alerts and recommending actions that the project manager can take. To implement these functions, Hitachi developed techniques for combining and revising project management and product management information, and also risk identification techniques that utilize statistics and machine learning (see Figure 5).

By allowing them to base their decisions on data, the project management AI enables these managers to keep their development projects on track.

Figure 5—Example of Project VisualizationThe project management AI analyzes project data and provides project managers with information about how the project is going, also predicting project risks and informing managers and support departments of these so that prompt action can be taken.

4. Examples of Development Process Innovation for Rapid Development

This section describes examples of process innovation in software development that the SML has put into practice in business divisions.

While product releases work on a rapid cycle in order to keep up with market requirements, the increasing scale of development imposes a heavy workload. Continuous integration (CI), a practice whereby software is frequently tested during development, has attracted attention as a way of reducing development workloads without compromising quality, allowing developments to proceed with finite human resources. In the CI development process, quality checks are performed whenever functional enhancements or modifications are made. This involves building the software, implementing the runtime environment, conducting automated testing (see Figure 6).

While CI involves frequent quality checks, there is also a high workload associated with producing the test scripts used for automated testing. Accordingly, the SML set out to reduce the amount of work needed to produce these CI test scripts.

Figure 6—Continuous IntegrationBy working through an ongoing sequence of building the software, implementing the runtime environment, and conducting automated testing, continuous integration enables development and quality checks to be performed with high turnover.

4.1 Implementation of Keyword-driven Testing when Using CI

Figure 7—Example of Test Script DuplicationThe past practice of testing each function independently meant that some testing was duplicated, resulting in unnecessary development.

Figure 8—Keyword-driven TestingBy developing test operations and associated keywords beforehand, test scripts can be created by combining keywords.

The tests for a program under development are generally implemented by the testing staff in the form of test scripts. Unfortunately, sharing of these test scripts is difficult if the testing staff write them in different programming languages. Furthermore, a lack of familiarity with existing test scripts may result in staff writing scripts that duplicate existing tests (see Figure 7).

To overcome these problems and encourage the reuse of test scripts within a development project, it is necessary to standardize the rules for script coding. Keyword-driven testing is recognized as best practice for test script coding, providing a way to overcome this problem. It works by defining keywords for the procedures and operations used in testing, using sequences of these keywords to specify the tests to be performed (see Figure 8).

Robot Framework is an open-source test framework for keyword-driven testing(5). It provides a framework for the execution of test scripts written in the prescribed format for keyword-driven testing. Test scripts are split into test case and keyword sections respectively, with the operations corresponding to each keyword defined in the keyword section being executed in the order specified in the test case section. This use of keywords to code the test scripts means that the part of test scripts can be shared at the level of individual keywords.

To encourage this sharing of keywords, a system was developed that automatically extracts existing keywords from collections of test scripts and lists them on the web (see Figure 9). Use of this system helps make testing staff aware of the keywords that already exist.

In this way, the use of keyword-driven testing to produce test scripts in CI facilitates the sharing of the keywords that correspond to the individual procedures in existing test cases. It is anticipated that the reuse of keywords will reduce the workload associated with writing test scripts.

Figure 9—Keyword Listing SystemThe system helps testing staff reuse keywords and scripts by automatically extracting keywords from collections of existing test scripts.

5. Conclusions

This article has described the technology developments and the measures that Hitachi has put in place at a company-wide level to facilitate process innovation in software development. Meanwhile, a new paradigm is emerging for software development called Software 2.0. This involves the use of machine learning to build systems. Existing development techniques may not be useful when applied to inductive learning based on data, since the existing techniques assume a deductive approach to software development. Whereas the conventional approach to software testing, for example, has involved formulating test cases using the specification as a test oracle, the lack of such test-oracle specifications in machine learning makes testing difficult. Along with working on research into development techniques suitable for Software 2.0, Hitachi also intends to contribute to the establishment of the next generation of system development techniques through the activities of organizations such as the Consortium of Quality Assurance for Artificial-Intelligence-based Products and Services and the Special Interest Group on Machine Learning Systems Engineering.

REFERENCES

1)
Information-Technology Promotion Agency, Japan, “Survey Report ‘Trend Survey on Embedded Software,” (Mar. 2018) in Japanese.
2)
S. Koizumi et al., “Enforcement of Ability to Develop Embedded Systems Applying Group Synergy of Hitachi,” Hitachi Hyoron, 91, 5, pp. 418–421 (May 2009) in Japanese.
3)
J. Humble et al., “Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation,” Addison-Wesley Professional, USA, (Jul. 2010).
4)
IEEE 29119-5: 2016, ISO/IEC/IEEE International Standard, “Software and Systems Engineering — Software Testing — Part 5: Keyword-Driven Testing,” (Nov. 2016)
5)
Robot Framework
6)
The Consortium of Quality Assurance for Artificial-Intelligence-based products and services in Japanese.
7)
Special Interest Group on Machine Learning Systems Engineering
Download Adobe Reader
In order to read a PDF file, you need to have Adobe® Reader® installed in your computer.