Skip to main content

Hitachi

Corporate InformationResearch & Development

How can we make a projector that projects images that are easy to see even in bright places? I In order to meet this challenge, Hitachi, Ltd. has pursued technological development to enhance the visibility of a projector. Through this development, the Company has given consideration to the Retinex theory that describes the mechanism of human "eyes" and "brain." As a result, Hitachi has realized a projector that is groundbreakingly easy-to-see as by considering the characteristics of human eyes.

What is meant by "easy to see?"

Images are offering increasingly high picture quality.

Photo: NAKAJIMA Mitsuo

NAKAJIMAThe phrase "high picture quality" started to gather attention in the 1990s when high-definition TVs entered the stage. This was also around the time that Hitachi started its endeavors in the development of high-picture-quality technology for TVs and image equipment in general.

As we accumulate technologies year by year, we have focused on the development of "frame-rate conversion technology" and "super-resolution technology" to generate clearer motion pictures over the past 4 to 5 years. More recently, the spotlight is placed on "visibility" for making images easier to see.

Figure 1: Development of high-picture-quality technology by Hitachi

What is required to enhance visibility?

NAKAJIMAThrough our development endeavors and discussions to date, we organized the evaluation factors of image quality into the six items shown in the following figure. For example, "shade" means a sharp contrast of bright and dark portions, "tone" means smoothness, and "signal-to-noise (S/N) ratio" means low noise, among other things.

Of these, the three factors of "clarity," "shade" and "vividness" that are shown in the upper half of the figure below are essential for enhancing visibility. In particular, "shade" is the key.

Figure 2: Evaluation factors for image quality and corresponding items for image correction

KAMIMURAHowever, it is not enough to simply enhance the shade. The figure above shows there are other factors on the opposite side with contrary characteristics. For example, "shade" and "tone" are in a trade-off relationship. In other words, if shade is enhanced and the contrast between bright and dark portions gets too sharp, the images will be rough and lose tone (smoothness), possibly becoming less easy to see. Therefore, it is important to maintain a balance between the respective factors on the opposite sides.

Projectors are weak in bright places

What challenges have been encountered in enhancing the visibility of projectors?

NAKAJIMAYou may have experienced that projector images are rather hard to see in bright places. Previously, it was taken for granted that projectors should be used in dark or darkened places. Nowadays, however, projectors are used in a diversified manner and there is a growing demand for using them under bright light. Thus, to enhance visibility under bright lighting conditions has been a major issue we have had to tackle.

Why are the images harder to see in bright places?

Photo: FUKUDA Nobuhiro

FUKUDAIt is deeply related to human visual characteristics.

The first characteristic is the dynamic range of human eyes. A dynamic range refers to the difference between the brightest and the darkest portions people can recognize, and is indicated using a ratio. The ratio is said to be 1:100 in the case of human eyes. The figure of 100 represents the number of levels that the range (width) is divided into for recognition, and the number of levels represents "resolution." However, the dynamic range in the natural world is 1:10,000,000,000. As this indicates, the level of brightness that human eyes can discern is limited.

The second characteristic is that these levels are not absolutely fixed but can vary depending on the environment.

…I know this is rather hard to grasp. Let me explain by using a specific example. Please look at this picture.

Photo 1: Picture to demonstrate the contrast of brightness *

*
Taken from standard motion picture images of the Institute of Image Information and Television Engineers

FUKUDANow, look at the shops in the picture. Seen from outside, the internal areas of the shops are completely dark and the interiors cannot be seen clearly. But what will happen when you step into the shops? Most likely you will be able to clearly see the fittings and other objects in the shop.

This phenomenon occurs because human eyes recognize brightness by dividing it into 100 levels within the surrounding environment. When you are outside, the difference between the brightest and darkest portions is so great that your eyes recognize the brightness of the lighting inside the shops as almost black. Once you are inside the shops, however, the difference between the brightest and darkest portions is not as great as outside, and the range of recognizable brightness is more finely divided. That is why you can recognize things more vividly when inside the shop compared to when you are outside.

Are you saying that human eyes perceive brightness in a relative manner?

FUKUDAExactly. Now, let's look at the situation in the case of a projector. In a dark place, there is no light except that which comes out of the projector, and human eyes distinguish only the light from the projector by dividing it into 100 levels. If there is some external light, the overall brightness increases and the range (width) of the light is enhanced. However, the intensity (range) of the light from the projector remains unchanged. This causes the resolution of the image portions to decrease, making the image harder to recognize. In other words, the contrast is lowered.

Is this why contrast correction is required?

NAKAJIMAThat is correct. However, if contrast is corrected for the entire image, corrections are made to both the brightest and darkest portions. This results in the white portions becoming even whiter and the black portions even blacker, and the image will be collapsed (whiteout and blackout) in some areas.

In order to minimize the collapsed areas, an effective measure is "local contrast correction" that corrects each part of the image one by one. By conducting local contrast correction, we can increase the contrast of respective parts while keeping the bright and dark tones within the image. One of the theories used to achieve this local contrast correction is the Retinex theory.

Utilizing "eye" and "brain" mechanisms

Retinex does not seem to be a familiar term. Can you explain this theory?

Photo: NAKAJIMA Mitsuo

NAKAJIMARetinex is a coined word coming from "retina" and "cortex," and is a theory describing how light is perceived by human "eyes" and the "brain."

The Retinex theory has its roots in theories that were discovered about 60 years ago. However, the theory deals with "sense" or "feel" that cannot be quantified, and thus no conclusions have yet been drawn. Currently, respective researchers including experts in medical fields are conducting studies by creating their own models to research how visual effects and calculation efficiency can be improved.

The Retinex theory started from the theories on "color constancy" and "lightness constancy." In making contrast corrections in our recent development, we researched an algorithm applying "lightness constancy." According to this theory, images are described by "illumination light × reflectance" and people can somewhat sense "reflectance." In the case of a projector, because light from outside causes visibility to decrease, improving "reflectance" proves effective.

According to the Retinex theory, we should be able to calculate reflectance by locally estimating the volume of illumination light from the images. So we intended to enhance the resolution of the images by amplifying the reflectance and applying this to the images. As such, we were able to improve the ease to see the entire image by conducting local processing based on the Retinex theory throughout the images.

Doesn't conducting local adjustments cause problems in terms of the overall balance?

FUKUDAIt is no wonder that you may suspect there might be a lack of consistency with other portions or a reversal of brightness might occur if local adjustments are conducted. But there is no need to worry.

Let's do a small experiment. Look at the figure below. How does the brightness of the square marked "A" and "B" compare to each other?

Figure 3: Adelson's checker-shadow illusion

FUKUDAThe "B" looks brighter, doesn't it? In fact, the two sections have the same brightness. When we talked about the dynamic range, we explained that "people recognize brightness in a relative manner." In the same way, people distinguish luminance in relation with the brightness of surrounding areas.

Human beings cannot perceive absolute luminance. Therefore, locally changing the contrast does not cause any big problems. Rather, it is a reasonable measure as it leads to mimicking the mechanism of human eyes.

How much will images become easier to see through using the Retinex theory?

KAMIMURAFor evaluation, we conduct two types of measurements. One is quantitative evaluation and the other is subjective evaluation of the human eye. Through quantitative evaluation, we have proven in calculations that the resolution that decreased due to light from outside has been locally restored. As for the subjective evaluation, we asked the test subjects to evaluate the six evaluation factors, which we explained at the beginning, using five level scoring so that we can measure whether visibility had been improved or not.

You may be able to clearly see the difference by looking at actual images. Please look at the pictures below. The lower images are corrected images by utilizing technologies in which the Retinex theory has been applied. The gloss is different particularly with the strawberries and cherries, isn't it? We should be able to say that controlling the reflectance has enhanced their gloss, clarity and shade, resulting in improved visibility.

Photo 2: Images corrected with technologies applying the Retinex theory* (corrected images are lower)

*
Image patterns provided by ShibaSoku Co., Ltd.

NAKAJIMAAside from the six factors for image quality, there is another important factor, which is the "feel of a material." Visibility is largely dependent on whether you can tell if the material is like a soft blanket or a hard iron. For this reason, we also gave attention to the "feel of a material" in the latest development. However, we found it difficult to evaluate the results, because the "feel of a material" varies depending on the person.

Actualizing theory in products

Can the theories examined be directly applied?

Photo: KAMIMURA Junji

KAMIMURAThis isn't possible because projectors have problems specific to them. First, we must consider keystone correction. Projector images may be distorted into a trapezoid shape when they are projected on to a screen at an angle. Keystone correction is a funtion to correct such a trapezoid shape into a rectangular shape. We need to take into account the distortion when adjusting the intensity of corrections that apply the Retinex theory.

In addition, as projectors are used in a variety of environments, we must be able to appropriately adjust the scope of corrections in proportion to the dynamic range of respective environments.

We reconstruct the theories after taking into account such aspects, and then apply them to products.

Processing must be hard when a lot of corrections need to be made.

KAMIMURAThat's right. It is also important to decide which method to use for processing. Broadly categorized, there are CPUs (central processing units) and GPUs (graphic processing units) that conduct processing through software and FPGAs (field-programmable gate arrays) and ASICs (application-specific ICs) that are specially designed to apply the functions. We need to decide after considering the characteristics of the products.

In the latest development, we considered realizing high-definition images with a screen size of 1,920 x 1,200 pixels or larger to be used for high-end projectors. Moreover, since the local spatial filtering was to be conducted, the greater number of processing is required as the image is larger, so the processing speed should also be faster. Because of this, we decided to use FPGAs for the latest development project as we judged that, while focusing on the processing speed, we should use what could be changed relatively easily in case something happend.

Figure 4: Examination of measures to achieve processing

KAMIMURAOn the other hand, reducing the processing burden is also important in terms of controlling costs. Processing will take time if we try to obtain accurate theoretical values. Therefore, we decided to use many formulas that calculate "approximate values" instead of 100%-accurate values to reduce the processing burden and increase the processing speed. In doing so, we investigated the most appropriate approximation formulas by setting the acceptable error compared with the theoretical values to be within 1%, as the resolution of human eyes is 100.

NAKAJIMAIf we pursue ideals, we should be able to achieve various effects. However, what is quite important is whether "the theories can really be applicable to products." For commercialization, it is essential to reconcile ideals with reality. In this regard, we believe we were able to reconcile them at a high level in the latest development and achieve extremely high effects. The Company started selling the commercialized products in May 2013 in overseas markets, and we have received raving reviews from the sales people commenting that "the advantage of the products can be easily demonstrated as they have great visual effects."

Pursuing technologies that contribute to people and society

What future developments do you think the "visibility enhancement technology" will produce?

NAKAJIMAIn the latest development, we successfully enhanced the feel of a material by locally conducting contrast correction. Going forward, we want to try to conduct contrast correction on a wider scale in addition to local corrections. The realization of images with even higher visibility by simultaneously enhancing the contrast and the feel of a material for the bright and dark portions in images can be expected.

Moreover, the visibility enhancement technology is utilized in increasingly wider areas including not only projectors and TVs but also vehicle monitors and medical equipment. Looking ahead, such developments could be of help in such areas as well as in projectors.

The horizons are expanding?

Photo: FUKUDA Nobuhiro

FUKUDAA recent trend focuses on the thinking and technologies for "supporting the five senses." Our latest development supports "eyesight" by applying the Retinex theory which describes the relationship between human eyes and the brain. As we move ahead, we want to continue developing technologies that can support the five human senses, as was done this time.

Whatever technologies we develop and however excellent they may be, they won't make sense if they are of no use to people or to society. We will continue to strive to be able to develop technologies that contribute to people and society.

Photo: KAMIMURA Junji

KAMIMURAMy hope is that such advanced technologies as this will come to the attention of more people, and that they would be utilized in even wider areas.

Recently, events are being held at stations and other venues utilizing the technology called projection mapping. I think such playful initiatives will bring about new demand and encourage the birth of new technologies. Responding to such changes in society, we want to create more and more products with a sense of playfulness.

(Publication: December 3, 2013)

Notification

  • Publication: December 3, 2013
  • Professional affiliation and official position are at the time of publication.
  • Page top

Related contents

  • Page top