One Hitachi Initiatives

Driven by rapid growth in demand from applications such as generative artificial intelligence (AI), data centers have become the predominate facilities for providing large volumes of computing resources and storage in a controlled environment. Hitachi has identified data centers as a key business. This includes the use of containerized data centers that make efficient use of space and enable rapid deployment to offer one-stop services that encompass the planning, facilities design, installation, operation, and maintenance that customers want. It is also developing practices for use at the planning stage to make the processes of agreeing on a plan with the customer, pricing, and design more efficient while still maintaining a capacity for rapid accommodation of the diverse requirements that customers have for containerized data centers. Specifically, these are: (1) Estimation of the quantity of equipment needed for each containerized data center configuration from the base IT equipment requirements, which include their power consumption, need for system redundancy, and so on. This equipment includes utilities like air conditioning equipment, chillers, and uninterruptible power supplies (UPSs). (2) Use of generative AI and the metaverse for three-dimensional (3D) modelling of the component parts of containerized data centers to make these easier to view.

Hitachi intends to contribute to the rapid deployment of data centers by customers through further development of technology to address a variety of needs.

Demand for data centers has been growing in response to the increasing sophistication and wider adoption of generative AI and other AI technologies over recent years. The rising performance of the graphics processing unit (GPU) servers and other IT equipment used for AI computing has been accompanied by a massive increase in their power density and heat dissipation. Accompanying this has been a strengthening of environmental regulation worldwide, making the improvement of energy efficiency an important consideration for data centers.

Hitachi has been working on research and development that aims to optimize both AI processing performance and cooling energy efficiency through integrated operation that encompasses everything from the AI processing running on the data center to its IT and cooling systems. This has involved utilizing the efficiency characteristics of GPUs, cooling, and other systems that Hitachi has obtained from its own data centers to develop techniques for, (1) distributed AI processing to smooth heat generation, and (2) coordinated control of cooling operation based on the AI processing load. Unit testing of prototypes has been used to estimate the AI processing performance gains from smoothing heat generation and the energy savings from reducing cooling when AI processing loads are low.

With a view to future commercial deployment, Hitachi is proceeding with the installation and testing of integrated systems in environments that include a mix of both next-generation GPUs and heterogenous cooling systems.

While the adoption of renewable energy is accelerating as part of efforts to achieve carbon neutrality, challenges remain in the form of generation curtailment due to a lack of operating margin on the grid and the question of how to provide the firming capacity needed to cope with the fluctuating output of renewable generation. The rise in data center electricity demand due to the rapid uptake of generative AI is also leading to longer delays in getting facilities connected to the grid, not to mention the extra costs to society of upgrading electricity infrastructure. To address these challenges, data centers are being located in regions with abundant renewables and their workloads managed in a way that facilitates the use of locally sourced renewable energy.

Through collaborative creation with TEPCO Power Grid, Inc., Hitachi has developed a high-speed workload shifting technique that is integrated with electricity grid operation. It works by identifying where and how to shift loads based on the supply and demand for electric power, data center location, the speed of the network between data centers, server loads, and workload characteristics. In a demonstration involving three geographically separated data centers, the technique successfully relocated workloads between the facilities with almost no downtime, even when running generative AI applications that consume large amounts of electric power. The demonstration also showed that the technique has potential in the ancillary services market as a means of grid frequency stabilization. Hitachi intends to deploy this technology in the future to build an energy ecosystem with distributed data centers.

Amid the current rapid acceleration in the adoption of generative AI, accommodating the explosive growth in the movement of data will require the construction of next-generation all-photonics networks (APNs) that combine high capacity and low latency with low power consumption. As APNs transmit optical signals over long distances without any intermediate electrical processing, the technology offers a dramatic boost to the performance of a telecommunications infrastructure that is already approaching its limits. The main obstacles to implementing APNs in practice are the need to ensure security in infrastructure control and that a high level of specialist expertise is needed for control of the analog behaviors distinctive of optical signals.

To overcome these challenges, Hitachi has developed an AI agent that uses a large language model (LLM) to combine physical layer monitoring data with natural language instructions. It has also devised techniques that enable the agent to run in an on-premises environment without the need for external cloud hosting.

By using natural language instructions to specify optimal settings for the optical devices in APNs (including high-capacity networks that link data centers), this provides flexible network control without needing to be an expert.

While GPUs are essential to the development of generative AI, it is anticipated that the amount of heat generated by these devices will approach 2,000 W in the near future. As a result, a transition is underway to replace the GPU air cooling systems used in the past with new more efficient liquid cooling systems.

The Research & Development Group at Hitachi has extensive experience developing cooling techniques over many years. Liquid cooling was a feature on the M880 mainframe computer launched in 1991, while the MP5800 model of 1995, featuring high-performance liquid cooling capable of handling 140 W per chip, placed it at the forefront of technology at that time. In a world first*¹, Hitachi commercialized direct-cooled power modules in 2007 as part of development work on hybrid electric vehicles (HEVs) and other electrically driven products. It also led the world with the development in 2013 of power modules with double-sided direct water cooling*² that delivered a 35% improvement in cooling performance compared to previous modules. Hitachi also incorporated loop thermosiphons into commercial products for server cooling in 2010, using two-phase liquid cooling, a method that utilizes the latent heat of vaporization. This reduced fan power consumption by 70%, using high-performance boiling surfaces to minimize thermal resistance.

Hitachi intends to leverage this liquid cooling expertise, cultivated over many years, to provide solutions for the escalating thermal issues of GPUs.