computer generated hiologram for real time display

ABSTRACT

Holography is a truly three- dimensional digital imaging medium. Recent work in electro-holography or "holovideo" demonstrates that the two crucial technologies - computation and optical modulation - can be scaled up to produce larger, interactive, color holographic images. Synthetic images and images based on real-world scenes are quickly converted into holographic fringe patterns using newly developed computational algorithms. Holovideo has numerous potential applications in the fields of visualization, entertainment, and information.

Keywords: Holovideo, Bipolar Intensity

1. INTRODUCTION
Computer graphics is confined chiefly to flat images. Images may look three-dimensional (3-D), and sometimes create the illusion of 3-D when displayed, for example, on a stereoscopic display . Nevertheless, when viewing an image on most display systems, the human visual system (HVS) sees a flat plane of pixels. Volumetric displays can create a 3-D computer graphics image, but fail to provide many visual depth cues (e.g., shading, texture gradients) and cannot provide the powerful depth cue of overlap (occlusion). Discrete parallax displays (such as lenticular displays) promise to create 3-D images with all of the depth cues, but are limited by achievable resolution. Only a real-time electronic holographic (“holovideo”) display can create a truly 3-D computer graphics image with all of the depth cues (motion parallax, ocular accommodation, occlusion, etc.) and resolution sufficient to provide extreme realism. Holovideo displays promise to enhance numerous applications in the creation and manipulation of information, including telepresence, education, medical imaging, interactive design, and scientific visualization.
2. ELECTROHOLOGRAPHY BASICS
Optical holography, used to create 3-D images, begins by using coherent light to record an interference pattern. Illumination light is modulated by the recorded holographic fringe pattern (called a ``fringe''), subsequently diffracting to form a 3-D image. A fringe region that contains a low spatial frequency component diffracts light by a small angle. A region that contains a high spatial frequency component diffracts light by a large angle. In general, a region of a fringe contains a variety of spatial frequency components and therefore diffracts light in a variety of directions.
The real-time display of holographic images has recently become a reality. In any real-time display system, a computer-generated hologram (CGH) must be computed as quickly as possible in order to provide for dynamic and interactive images. However, numerical synthesis of a holographic interference pattern demands an enormous amount of computation, making rapid (_1 second) generation of holograms of even limited size impossible with conventional computers.
A holographic fringe pattern is computed by numerically simulating the physical phenomena of light diffraction and interference. In general, light diffracts from a three-dimensional object to the hologram plane. Since the analytical expressions that model diffractive propagation through free space resemble the Fourier transform integral, computation of holographic interference patterns often utilizes the Fast Fourier Transform (FFT) algorithm. Though relatively fast, this approach is useful only for images possessing discrete depth surfaces and becomes slow when applied to images that extend throughout an image volume.
A more general approach is a ray-tracing method in which the contribution from each object point source is computed at each point in the hologram plane. This method can produce arbitrary three-dimensional (3D) images, but is slow, since it requires one calculation per image point per hologram sample. As presented in this paper, the application of several methods of reducing computation complexity leads to computation times as short as one second on a data-parallel-processing supercomputer. First, a “bipolar intensity” representation of the holographic interference pattern is developed and shown to eliminate unwanted image artifacts and simplify calculations without loss of image quality or generality. Second, a look-up table approach is described and shown to provide further speed increase, though image resolution and quantization noise become issues.



Fig.1. General Geometry for HPO CGH
3. DISPLAY SYSTEMS
The display systems that we have developed during the past five years used the combination of an acousto-optic modulator (AOM) and a series of lenses and scanning mirrors to assemble a real 3-D holographic image at video frame rates. This time-multiplexed SLM approach is sometimes called the "Scophony geometry" after the early contender for television displays. A partial schematic is shown in the following figure. The viewer sees a real 3-D image located just in front of the output lens of the system. The viewer experiences the depth cue of horizontal motion parallax in this HPO image. Vertical parallax is sacrificed to simplify the display. (This restriction does not limit the display's usefulness in most applications.)


Fig. 2 Partial schematic diagram (top view) of an holovideo display.

The scanning mirror system angularly multiplexes the image of the modulated light. A vertical scanning mirror (not shown) positions each hololine vertically. Electronic control circuits synchronize the scanners with the incoming holographic signal.
4. HOLOGRAPHIC IMAGING SPECIFICS

This paper focuses on the computation of off-axis transmission holograms possessing horizontal parallax only (HPO), a quality of the “rainbow”. It is possible to represent an HPO hologram with a vertically stacked array of one-dimensional holographic lines. Consider an HPO hologram made optically using a reference beam with a horizontal angle of incidence. Spatial frequencies are large in the horizontal direction (_ 1000 lp/mm) and increase with the reference beam angle. However, by limiting the view zone to only a single vertical view, vertical spatial frequencies are low (_ 10 lp/mm). It is evident that elimination of vertical parallax provides a factor of 100 (or greater) reduction of CGH size. During reconstruction of this hologram, diffraction occurs predominantly in the horizontal direction. It is appropriate to represent this holographic pattern with a relatively low vertical sample spacing (or “pitch”), roughly that used in a two-dimensional (2D) imaging system. In the horizontal dimension, however, the sampling pitch must be very high in order to accurately represent the holographic information.

For each horizontal plane (“scan-plane”), the associated horizontal line of the hologram diffracts light to form image points in that plane only. Therefore, the 2D holographic pattern representing an HPO 3D image can be thought of as a stack of one-dimensional (1D) holograms or “holo-lines”. The goal of this paper, then, is to compute these 1D holographic lines as quickly as possible.


The images to be generated are approximated as a collection of self-luminous points of light located in x,y, and z locations. Each point possesses a magnitude and a phase. The square of the magnitude is proportional to the desired brightness of an image point, and the phase is relative to the reference beam. Each point radiates a fan-shaped wavefront that is a horizontal slice of an anisotropic spherical wave. It is important to be able to assign a range of propagation angles for each point of image light in order to limit the spatial frequencies contained in the holographic fringe pattern. At one extreme, “light” used to compute the CGH must have an angle of incidence that is greater than that of the reference beam to prevent overlapping real and virtual reconstructed images (image aliasing). At the other extreme, the total angle subtended by the incident reference and object beams cannot be so large as to give rise to spatial frequencies that cannot be adequately represented by the discretized numerical representation. If the horizontal sampling pitch is physical distance d, then the maximum spatial frequency (f max) that can be represented is 1/2d, according to the Nyquist Sampling Theorem. Higher spatial frequencies cause aliasing, thus destroying image quality. Anti-aliasing is therefore accomplished by limiting the minimum and maximum angles of incidence of object light. In addition to anti-aliasing, the range of direction of object light propagation is used for the purposes of image occlusion and advanced image lighting models, resulting in a more realistic looking image.

The information content of a CGH must be reduced to a size and format that can be manipulated by existing computers. Consider a typical CGH pattern: it is composed of a large but finite 2D array of numbers representing the intensity of the computed total wavefront. The horizontal sampling pitch puts an upper limit on the maximum angle of diffraction of the CGH and consequently the maximum range of image viewing angles. Reducing the range of viewing angles reduces the required horizontal sampling rate and therefore the amount of data (space-bandwidth
product) in the CGH. Furthermore, by reducing the size of the hologram (and therefore of the image), the data content of the CGH is as low as a few megabytes. These size reductions are an undesirable trade-off used only when no further information reduction is possible. Quantization is also an important consideration. Each sample must represent an arbitrary physical value, but digital electronics commonly limit the number of quantization levels available when manipulating data. For example, the output device used in the current MIT system is a framebuffer capable of storing 6 megasamples, each represented by one byte (8 bits), giving the output data 256 possible quantized values. Therefore, a computed holographic interference pattern must be normalized to fit within this range. It is then quantized, increasing image noise due to the loss of accuracy.
Quantization is also important when considering computation speeds, since less accurate representations of values (fewer bits) can be used to increase speed, but also sacrifice image quality.


5. COMPUTATION USING POINT SOURCE SUMMATION

In general, the physics of optical holography are as follows. The object light and the reference light are incident at the plane of the hologram. Each beam is represented with a complex time-harmonic electric field vector, E o and E R . It is assumed that both are mutually coherent sources of monochromatic light. For this analysis, the units of an electric field amplitude are normalized so that the square of its magnitude corresponds to optical intensity; the polarizations are assumed identical and for simplicity are not specified. The object beam E o is generally a superposition of light scattered from locations throughout the object volume. The total time-harmonic electric field incident upon the hologram is E O +E R , which represents the interference of the total object light and the reference light.
The resulting intensity pattern is

I TOTAL = E O 2 + E R 2 + 2Re{E O E R * } (1)

and is a real physical light distribution comprised of three components. The second term is the reference beam intensity and represents an essentially constant or “DC” bias which increases the value of the intensity uniformly over the hologram. In computational holography, it can be left out, since normalization will subtract any DC bias present in the total holographic pattern. The first term is the object self-interference: a spatially varying pattern that is generated when interference occurs between light scattered from two or more object locations. During image reconstruction, this component of the holographic pattern is unnecessary and often produces unwanted image artifacts. In optical holography, a common solution is to spatially separate the self-interference artifacts from the desired image by increasing the reference beam angle to at least three times the angle subtended by the object. However, in computational holography, a large reference beam angle is a luxury that one does not have. Therefore, the obvious solution is to exclude this object self-interference term during computation. Finally, it is the third term that contains all of the necessary and useful holographic information, and is referred to as I F .

The numerical computation of a holographic pattern is now examined, beginning with the simple physics of point-source light propagation. The hologram is positioned at the z = 0 plane, and has horizontal and vertical axes of x and y respectively. Each object point emits light from position (x p ,y p ,z p ). The fan-shaped object sources expose a limited width of a particular holographic line (“holo-line”). For the HPO CGH considered henceforth, an object point contributes only on the holo-line that is at the same vertical position (y p =y). (To be more accurate, one must account for vertical foreshortening, absent due to the elimination of vertical parallax. Depending on specific display geometries, a more general image-point selection criterion is to include on holo-line y each point with y p + µ( z p - z view )= y, where z view is the intended view distance from the hologram, and µ= y/zview is the slope of the path of light from the holo-line to the viewer.) Throughout the remainder of this discussion, the computation of a single holo-line is analyzed, and a full 2D CGH is computed simply by generating an array of holo-lines for each value of y. Only the x-dependence of E O and E R and other physical quantities need to be
considered in computing a single holo-line.

For the purposes of computation, each object point is treated as an angularly truncated two-dimensional point source. Each has a complex amplitude of A p = a p exp(i? p ), where the real-valued magnitude is a p and the realvalued relative phase of point source number p is ? p . Within the region of contribution, the phase of the object wavefront, ?p(x), is approximated as a spherical wave[9] centered at the point source location:

? p (x)= k r p (x) + ? p where rp(x)=[(x-x p ) 2 + z p 2 ] 1/2
where r P (x) is the oblique distance to a location on the holo-line and is a function of x. The wavenumber is k= 2?/?, where ? is the free-space wavelength of the light. The time-harmonic representation of the total object field for a single holo-line is
N
E o (x)= å a p (x)r p -1 (x)exp[i ? p (x)] (2) P=1
where N is the number of object points contributing to this particular y-valued holo-line. The added dependence of a p on x facilitates anti-aliasing and occlusion simply by not including contributions outside of specific ranges of x. Finally, to avoid singularities, it is assumed that the magnitude of z p is never less than some small amount, e.g., 10 ?.

The reference beam E R is a point source at some specific location (x R , y R = y,z R ) with a horizontal angle of incidence q R = arctan(x R /z R ) and curvature in the x dimension only, i.e., collimated in the y dimension. The time-harmonic representation of the reference beam field at any holo-line is
E R (x)= a R exp[i? R (x)] (3)
where a R is the magnitude (assumed constant versus x) of the reference wave at the hologram plane and
q R (x)= k[(x- x R ) 2 + z R 2 ] 1/2

Note that all magnitudes and phases are real quantities.

6. BIPOLAR INTENSITY

The third term of Equation 1, called I F (x), contains all of the information needed to reconstruct the image in a given horizontal plane. Note that it is real-valued; it represents the combined intensity variations (“fringes”) resulting from each object point interfering with the reference beam. Since it contains negative values as well as non-negative values, it is a “bipolar intensity” which exists physically only when superimposed on the first two bias terms in Equation 1. Computationally, however, I F (x) can range both positive and negative since it is represented numerically, and is later offset during normalization.
The bipolar interference pattern I F (x) has the advantage of containing no object self-interference or bias components, and is numerically simpler to compute. I F (x) is further simplified:
N
I F (x)= 2Re{[ å a p (x)r p -1 (x)exp{i? p (x)}][a R exp(i? R (x)]
P=1
N
= 2a R å Re{a p (x)r p -1 (x)exp[i? p (x)- i? R (x)]}
p=1
N
= 2a R å a p (x)r p -1 (x)cos[? p (x)- ? R (x)] (4)
p=1

The right-hand side of Equation 4 is simply a scaled sum of the real-valued cosinusoidal fringe pattern resulting from the interference of point source p with the reference beam. Each of these constituent fringes is summed to obtain the full bipolar fringe pattern.

The advantages of this approach are readily seen by comparison to computation of the full interference pattern I TOTAL (x) which requires keeping track of both the real and imaginary parts of the object light. Each point requires a function call to both sine and cosine, and complex-value arithmetic must be used. In the bipolar intensity approach,
the real-valued cosinusoidal fringes need simply to be summed to achieve the desired interference pattern. Each point requires only a single cosine function call. Therefore, a factor of two speed-up is expected. A subtler advantage is revealed by considering numerical precision. The integer multiples of 2? spanned by ? p (x) must be calculated but are discarded when computing the cosine or the sine of the object light phase. Typically,
? p (x)is represented by a floating-point number composed of four 8-bit bytes possessing a precision of roughly 7 decimal digits. Values for ? p (x) often exceed 10 7 and must therefore make use of double-precision floating point representation, decreasing computation speed. In computing only the bipolar intensity component I F (x), ? R (x)
(and any arbitrary integer) is first subtracted from ? R (x) before applying the cosine function, reducing the number of required significant digits; thus, the important fractional phase information is adequately represented with a single-precision floating-point expression.

After an intensity pattern has been computed, it must be normalized in order to satisfy the output device requirements of the CGH display system. Since normalizing generally scales the entire pattern, the leading factor of 2a R on the right-hand side of Equation 4 is hereafter excluded. The reference beam intensity (the square of a R ) is no longer meaningful. This makes physical sense when considering that in optical holography, the purpose of choosing the ideal reference beam intensity ratio (relative to the object light) is to provide a sufficient DC offset and scaling to the interference fringes in order to keep them within the range of sensitivity of the recording medium. Computationally, offset and scaling are provided automatically during normalization. With the factor of 2 a R set
arbitrarily to unity, (and substituting the definition of ? P (x)) Equation 4 becomes
N
I F (x) = å a p (x)r p -1 (x)cos[kr p (x) - ? R (x)+ ? p ] (5)
p=1

which is hereafter called the bipolar fringe method of CGH computation. No reference beam ratio needs to be specified during computation, and bias buildup is not an issue. Compare this bipolar intensity method to the physical process occurring in some photorefractive crystals[10] (e.g. lithium niobate), in which uniform (“DC”) intensity is
not recorded due to the material's negligible response to intensity patterns with low spatial frequencies. Researchers exploit this absence of bias build-up in order to sequentially expose multiple holographic intensity patterns.

7. PRECOMPUTED ELEMENTAL FRINGES: THE LOOK-UP TABLE APPROACH

Continuing with the bipolar intensity summation approach, further improvements in computation speed are gained through the use of precomputed look-up tables containing all possible elemental fringes. Consider a two-dimensional display which requires no computation (other than normalization and perhaps logarithmic correction) in order to
display a two-dimensional image. This simple fact is due to the one-to-one correspondence between each image element and each display element, both often referred to ambiguously as a “pixel”. To illuminate a particular image pixel, simply display some non-zero value in the corresponding display pixel. In a three-dimensional holographic display, this simple correspondence between each image element and each display element does not exist. In this case, a 3D image element is a point of light in some (x, y, z) location with a brightness and relative phase. A display element corresponds to a numerical sample of a line of a holographic pattern modulating a beam of light. By determining how each possible image element relates to the display elements (holographic pattern), computation is reduced to a minimum.

Specifically, it is possible to precompute the contributions to I F (x) of an image point of unity magnitude for each possible value of (x p , z p ). Since each holo-line is computed using the same E R (x), the precomputed tables are used in the computation of each holo-line. Rather than having to compute the cosinusoidal fringe each time it are needed, a large precomputed lookup table maps each (x p ,z p ) to the appropriate elemental fringe pattern contribution. To define these tables, Equation 5 is expanded.
N
I F (x)= å {a P (x) cos ? p r p -1 (x) cos[kr p (x)–? R (x)] + a P (x) sin? p r p -1 (x)sin[kr p (x)–? R (x)]}
P=1
(6)
Other than the dependence of a p on x, all spatial dependence of Equation 6 is in the following two expressions, used to define the two look-up tables:

TABLE c [x,X i, Z i ] = r i -1 (x) cos [k r i (x) - ? R (x)]
TABLEs[x,X i ,Z i ] = r i -1 (x) sin[? R (x) – kr i (x)]

where r i (x)=[(x-X i ) 2 + Z 2 ] 1/2 . For both of these tables, the first index is x, which is already discretized due the sampled representation of the CGH. However, image point positions (x p ,z p ) are not explicitly discretized, and must be rounded off to generate the X i and Z i source location indices. Before the tables are generated, the X i and Z i resolutions must be chosen in order to discretize the image volume. Since the acuity of the human visual system is limited, it is possible to chose resolutions that do not visibly degrade the image.

The first table looks like an array of cosinusoid-like fringes that have an approximately linear chirp in spatial frequency with respect to x. The rate of chirp is a function of the point source depth z p , and the horizontal position of the fringe is a function of x p . The second table is essentially the same but in quadrature to the first, i.e., with a p /2 phase difference, needed in order to represent any arbitrary point source phase. The dependence of a p on x due to anti-aliasing is conveniently included in the two tables simply by leaving zeroes in all table locations in which there is no contribution. It is then assumed that any further variation in ap will be dealt with during computation time, leaving a p independent of x. Thus, I F (x), expressed in terms of precomputed tables, is
N
I F (x) = å {(a p cos ? p ) TABLE c [x,X p, Z p ] + (a p sin ? p ) TABLE s [x,X p, Z p ] } (7)
P=1
Computation for a given holo-line at vertical position y is as follows:
• For every point with yp _ y (i.e. on given scan-plane):
– Round off _xp_ zp_ to _Xp_ Zp_ to index the desired elemental fringe.
– For each x sample in IF _x_:
• Scale TABLE c [x,X p ,Z p ] by a p cos ? p .
• Scale TABLEs[x,X p ,Z p ] by a p sin ? p .
• Accumulate these scaled values in I F (x).
After each holo-line is computed (at each value of y), then normalizing and output is performed depending on the specific display system.

8. COMPUTATIONAL COMPLEXITY

Computational complexity is dramatically reduced through the use of the two precomputed look-up tables. For a single object point contributing to a particular hologram point, the amount of computation required is two multiplications and two additions. In comparision, without the tables, computation involves a minimum of five additions, five multiplications, one square root, and one cosine function call( i.e using the Bipolar Fringe summation method and a precomputed ? R (x) known as each x). Full complex computation of I TOTAL (x) would require still more computational steps, at least twice as many. Therefore, an order of magnitude of speedup is expected through use of the precomputed tables.

9. RESULT

Using the methods of bipolar intensity summation and precomputed elemental fringe patterns, hologram computation has been implemented. A Connection Machine Model 2 employs a data-parallel approach in order to perform real-time CGH computation. This means that each x location on the hologram is assigned to one of 32k virtual processors. (The 16k physical processors are internally programmed to imitate 32k “virtual” processors.) A Sun 4 workstation is used as a front-end for the CM2, and the parallel data programming language C Paris is used to implement holographic computation.

The following table presents a comparision between the time taken by various methods for displaying the above described image on a parallel supercomputer and a serial workstation:
Computation Method CM2 Sun4
Full(Complex) I TOTAL 2.180ms 943.4ms
Bipolar Intensity 1.135ms 486.3ms
Look-Up Tables 0.174ms 39.0ms
Single Look-Up table 0.084ms 22.1ms


As is evident the Look Up Table approach is the best in terms of time efficiency. The Look Up table method can be extended to produce full parallax images. Although, there would be an increase in the number of table look-ups and more computations required the this would become simpler as the computing speed increases.
10. APPLICATIONS

I divide interactive computer-graphics applications into two extreme modes of interaction: the ``arm's reach'' mode, and the ``far away'' mode. An arm's-reach application involves interacting with scenes in a space directly in front of the user, where the user constantly interacts, moving around it to gain understanding. In this mode, all of the visual depth cues are employed, particularly motion parallax, binocular disparity, convergence, and ocular accommodation. These applications warrant the expense of holovideo and the extreme realism and three-dimensionality of its images: computer-aided design, multi-dimensional data visualization, virtual surgery, teleoperation, training and education (e.g., holographic virtual textbooks on anatomy, molecules, or engines).
At the other extreme, a far-away application involves scenes that are beyond arm's reach and are generally larger. The imagery of such applications - e.g., flight simulation, virtual walk-throughs - make adequate use of the kinetic depth cue, pictorial depth cues, and other depth cues associated with flat display systems. A high-resolution 2-D display may be a more cost-effective solution for far-away applications.
11. PRESENT... AND FUTURE
Currently there are no off-the-shelf holographic displays. Holographic display technology is in a research stage, analogous to the state of 2-D display technology in the 1920s. What, then, does the future hold? The future promises exactly what holovideo needs: more computing power, higher-bandwidth optical modulation, and improvements in holographic information processing.
Computing power continues to increase. A doubling of computing power at a constant cost - a trend that continues at a rate of every 18 months - effectively doubles the interactive image volume of a holographic display . Inexpensive computation - around $100 per gigaMAC - is the most crucial enabling technology for practical holovideo.
Although optical modulation has borrowed from existing technologies (e.g., transmissive LCDs, AOMs), new technologies will fuel the development of larger, more practical holovideo displays. Because bandwidth is most important, we use as a figure of merit the number of bits that can be modulated in the latency time of the Human Visual System (typically 20 ms). An AOM can modulate about 16 Mb in this time interval, at a cost of about $2000 (or $120 per Mb), including the associated electronics. The DMD, a new technology for high-end 2-D video projection technology, delivers approximately 100 Mb in 20 ms, for a cost of about $3000, or $30 per Mb. Future mass-production could reduce the cost further. Reflective LCDs are another possible technology. Several researchers create small reflective LCDs directly on a semiconductor chip using VLSI technology .

12. CONCLUSION

Experimental results demonstrated that a horizontal parallax only off axis transmission hologram can be computed in under 1s. Computation was fast enough to provide dynamic holographic images at interactive rates. The interactive display of holographic images added the additional sensations of depth and tangibility that accompany moving 3-D images.
The overall speedup demonstrated here is remarkable. CGH computation that traditionally would require several minutes or hours on a mainframe computer is reduced to 1s.

REFERENCES:

1. S. A. Benton, Experiments in holographic video imaging
2. www.media.mit.edu/spi/ holoVideo All.htm
3. www. holovideo .com