Skip to main content
While the popularity of unstructured data has been growing, network attached storage (NAS) has been attracting attention as an infrastructure for storage and management of data. Although the ease of implementing NAS keeps its popularization growing, it is becoming a problem that large amounts of NAS are flooding organizations.
At Hitachi, we have developed new NAS products—based on the concept of a "cloud on-ramp"—as a solution to the problems involved with NAS. Under a new way of thinking, namely, utilizing NAS as "an entrance to the cloud," we seek to expand the potential of NAS.
NEMOTOIt is a way of thinking; namely, NAS is positioned as the entrance to the cloud. To put it concretely, the cloud on-ramp aims to gather data stored in NAS at various locations and departments into cloud storage and to manage that data.
It would be better to think of the cloud on-ramp as a function whereby the cloud and NAS are combined suitably and data is utilized efficiently.
Figure 1: Conceptual diagram of cloud on-ramp function
NEMOTOAs NAS continues to gain in popularity and various manufacturers launch NAS products, Hitachi's NAS products must discriminate from other companies' products. Given that need, we decided to make products that meet customers' needs in a form unlike that of conventional NAS. At that time, that which drew our attention was solving the so-called "silo problem" associated with NAS.
In comparison to other kinds of storage, NAS has the merit that it can be easily implemented. Thanks to the ease by which it can be used right after being connected to the network, it can be introduced in small units at divisional or departmental level. However, in large organizations, on implementing NAS in such small units, NAS will flood the organization, and management tasks (namely, capacity monitoring and backup) become a bigger burden. As a solution to the "silo problem" associated with NAS implementation in this manner, we thought up the idea of utilizing the "cloud".
Although the cloud has many special features, a key one is "flexibility of capacity"—that is, storage capacity can be secured in just the amount required, when required. By exploiting this feature, we think it is possible to liberate system managers from troublesome management tasks.
Figure 2: "Silo problem" concerning NAS
NEMOTOA key feature of the cloud on-ramp function is reducing the load of systems managers in charge of backup and capacity management.
We devised a configuration under which NAS automatically determines the storage location of data in accordance with the utilization rate of data. In this manner, capacity management of each location is simplified. Data no longer used is moved from NAS to cloud storage, and only information indicating that "Such and such data is on the cloud." is kept. The timing under which the data is moved is automatically judged by the system in consideration of the status of the capacity of NAS. As a result, it is beneficial that system managers need have little awareness regarding the capacity of NAS. What's more, by means of file-virtualization technology, backup data stored in the cloud can be accessed from NAS; therefore, there is no need for a user to be aware that a storage location has changed.
Figure 3: Tiering by applying file-virtualization technology
NEMOTOAs for restoring data, we are offering a function called "on-demand restore". As the name suggests, this is a function that sequentially restores the data that the user wants.
As for the conventional restoration method, services could only be restarted after restoring all data. With on-demand restore, however, services can be restarted after restoring the top-level directory only. Although the first to be restored is the top-level directory, by applying file-virtualization technology, it is possible to show data as if it appears to have been restored even if there are no actual data in the directory. What's more, in the conventional manner, each set of data can be accessed. And files that have been accessed can be restored in order. Narrowing down the "range that is restored at the start" makes the time until services resume shorter.
This function came about because when investigating the format of backup data, we considered that "rapid restoration will surely become necessary." Not being able to use backup data when required would also be a wasted opportunity. These days a key point drawing attention is how quickly services can be restarted after natural disaster. On this point, we think that on-demand restore will also be useful.
Figure 4: Image of on-demand restore
NEMOTOAs for development of cloud on-ramp, we placed particular importance on speeding up backup. As for backup, not only making it as automatic but also determining how to speed it up is a key point. But we struggled because we could not attain the performance we wanted.
NEMOTOFirst, we investigated the scope of backup. In the case of the conventional method, all information in NAS (such as directory tree structure, metadata, etc.) is backed up. In this case, however, backup is performed while all directories are checked, so it took a very long time. So we devised a backup method whereby only modified files and directory structures are backed up. Targeting only modified data in this manner significantly increased the speed of backup.
Figure 5: Speeding up automatic backup
NEMOTODoing that alone, however, was not enough to attain satisfactory performance. So next we investigated the format of the backup data.
In regard to cloud storage, data is not stored in units of "files"; instead, it is stored in units of "objects." Since objects can have a variety of formats, it was necessary to investigate the most suitable method of storage. It was very tricky determining the scope of information to make into a single object, and we tested various methods and evaluated their characteristics. Since development had to be completed in a short time, we had a tough time accomplishing the development with our approach, namely, theoretically calculating the performance of each method while simultaneously creating a prototype and measuring its performance.
NEMOTOAs a product featuring the cloud on-ramp function, the "Hitachi Data Ingestor" was released in the worldwide. "Hitachi Virtual File Platform" released in Japan also adopts it. Both Hitachi Data Ingestor and Hitachi Virtual File Platform received very good reviews from our customers. Receiving such good reviews was the result of developing this product with a mind to distinguish it from products of other companies.
NEMOTOYes, it was. Since we developed a product with a special function that is not possessed by products of other companies, we can say that it has become easier to appeal to customers in the business field.
What's more, through this development, we recognized the importance of not only creating this function but also anticipating and developing other necessary functions. As for automatic backup, we created a function called "on-demand restore" by investigating the units and method for backup while considering the inextricably linked "restore" function. Moreover, the on-demand restore function is also being utilized for data migration from existing NAS to Hitachi Data Ingestor or Hitachi Virtual File Platform. It is becoming possible to expand a single technology to a variety of services.
NEMOTOAt the moment, although data is only being stored on the cloud, from now onwards, how to utilize that data is becoming more important. And studies on using cloud-stored data to get valuable information are on-going. For example, in the case that sales data is accumulated from a number of branches, ways of doing business at each branch can be reviewed by analyzing that data. Like this, stored data can be utilized in order to help decision making in business.
Although we are still studying how to utilize metadata, if it were made possible to rapidly retrieve target items from within various kinds of information, that possibility would probably contribute to utilizing data stored on the cloud.
NEMOTOThat's right. I have been responsible for a variety of NAS fields such as application and operations management. From now on, I aspire to be a researcher who can utilize past experience and think about the storage required by customers from a broad standpoint.
And what I feel on being involved in a wide range of fields is that evaluating ideas quantitatively is very important. As is true of research and development on the cloud on-ramp function, prototyping and verifying ideas is the ideal state as a researcher. Although there are various pressures, I want to continue to pursue research while keeping this "ideal state" in mind.
(Publication: June 5, 2012)