The tremendous advances in genomic expression, molecular design, and protein synthesis demonstrate the significance and prospect of bio-technologies. Molecular biology addresses biomolecular structures, properties, functions, relationships, and mechanisms governing bio-chemical processes. Biomolecular datasets have grown at an exponential rate as increasingly large experiments are performed to gather information using emerging acquisition devices. Without effective data processing and visual analysis methods, biologists can not comprehend large amounts of data, since this would require painstaking cognitive reconstruction. Molecular visualization takes advantage of computer graphics, image processing, virtual reality, and even cognitive psychology to provide biologists with a deep insight into complex structures, fine features, and obscure patterns in large-scale datasets. Data visualization is becoming an indispensable component in modern biological research.

Conventional wide-field microscopy employs a condenser lens to uniformly illuminate the wide area of a thick volume of specimen. Out-of-focus light rays emitted from non-focal planes interfere with the focal slice and thus deteriorate the resulting image contrast and resolution, particularly when thick and / or living specimens are used. In general, thin tissue sections and careful manipulations are required to obtain useful data. Confocal microscopy [1] (Fig. 1) addresses this problem by incorporating a specifically designed dichromatic mirror, an objective lens, and a confocal aperture to prevent out-of-focus light rays from reaching the photomultiplier detector. This configuration suppresses out-of-focus blurring and thus generates images with increased resolution and improved contrast from thick or even whole-mount specimens. A slice is acquired by letting the laser beam scan across the specimen. A volume is collected as the focal plane is shifted in the longitudinal direction. A transmitted light image can be captured from an unstained specimen, but a specimen is usually labeled with fluorescent probes to get a multi-channel image.

Iso-surface extraction and direct volume rendering (volume rendering) are two classical methods widely used for visualizing volumetric data. Iso-surface techniques such as marching cubes [2], dividing cubes [3], and marching tetrahedrons [4] need to fit intermediate graphical primitives (e.g., triangles) to reconstructed surfaces prior to surface rendering. Volume rendering techniques such as ray casting [5], ray tracing [6], splatting [7], shear-warp [8], and hardware-based texture mapping [9] eliminate the construction of geometric representations, but instead operate directly on voxels by using a light absorption-transmission model and a transfer function to assign colors and opacities to voxels that are then composited along view directions. As evidenced by numerous publications [3], [5], [9], iso-surface and volume rendering are well established in visualizing medical data (e.g., CT, MRI). However, there have been only a few reports on visualization of confocal microscopic biomolecular data [10], [11], [12], [13]. The problems that this kind of data brings to volume visualization are drawing more and more attention.

The widespread adoption of confocal microscopy in molecular biology is attributed to out-of-focus light elimination, increased resolution, improved contrast, multi-channel imaging of co-localized objects, and non-invasive "optical sectioning" of thick specimens. However, there are still a variety of factors in combination with inherent optical characteristics, which may affect image quality and numerical accuracy during data acquisition. Furthermore, intangible amorphous micro-structures and lack of a priori knowledge make it a daunting task to visualize confocal microscopic biomolecular data.

First, low signal-to-noise ratio and low image contrast require effective data processing and image enhancement techniques to treat both slices and the volume to retain small structures and restore fuzzy features. Second, the longitudinal resolution is at least 3 ~ 4 times lower than the lateral resolution and therefore the volume is smeared in the longitudinal direction. Interpolation may introduce artifacts if the raw data is simply re-sampled to generate regular voxels. Third, large-scale datasets require efficient algorithms to achieve interactive visualization for trial-and-error exploration that is usually needed for an unfamiliar dataset. The size of a three-channel volume data with a resolution of 512 × 512 × 64 is 48MB and it is even much larger when time-varying scenarios are considered. Given the current computing capability and commodity graphics cards, it is still difficult to visualize a large dataset in real time using computationally intensive visualization methods like volume rendering.

Medical datasets tend to contain shape-invariant organs, smooth and well-defined boundaries, and only a few continuous objects that may be easily segmented. However, biomolecular datasets usually have amorphous structures, ragged contours, porous surfaces, and numerous disconnected small objects. These properties make it extremely difficult to extract and track features in time-varying datasets. Compared to medical datasets for which a considerable amount of a priori knowledge is available to validate visualization results and quickly tune parameters to find features of interest, biomolecular datasets with amorphous structures often leave the user uncertain about visualization results. It is also difficult to distinguish artifacts introduced in acquisition or visualization from real structures.

Finally, a confocal microscopic dataset is longitudinally thin (e.g., 512 × 512 × 15), which affects perception of depth. Due to the aforementioned problems, visualization techniques well suited for medical data may not produce satisfactory results when used for confocal microscopic data. Novel methods need to be developed to address these challenges.

Visualization of confocal microscopic biomolecular data has been applied to study intra-cellular / extra-cellular structures, cellular processes of protein transport, neuromorphology, and time-varying structures of polymerized actin [14]. Janacek and Kubinova [15] employed Iris eXplorer to extract, display, and measure iso-surfaces from tobacco cell chain and capillary bed in terminal vilus of human placenta. Kraemer [16] developed Topologizer using VTK and Perl to visualize neuropils olfactory glomeruli in a beetle brain. Object-based segmentation and measured-concentration based segmentation were proposed to study dynamic topological properties that an object retains under deformation. Other packages such as OpenDX, MicroVoxel, and VoxelView were also used for image segmentation and registration, geometry reconstruction, and measurement of surface areas and enclosed volumes.

Kaufman et al. [10] discussed some key issues for visualizing confocal microscopic data and proposed solutions for surface discrimination and shading. They developed BioCube, a visualization environment for volumetric exploration of actin cytoskeleton structures. Brady et al. [11] developed a CAVE-based system called Crumbs for immersive data visualization, identification and tracking of fibrous Drosophila embryos, and segmentation of intricate cartilage in a horse fetlock. Monks et al. [12] applied iso-surface, volume rendering, and virtual reality techniques to provide an enhanced visual understanding of communications between cells of an immune system by displaying co-locality of 3D proteins capable of interacting with one another. Fang et al. [17] integrated a shear-warp algorithm based on 2D texture mapping, an adjustable transfer function design based on image processing, and a boundary detection method in IVIE system for interactive volume visualization of noisy confocal microscopic data.

Confocal microscopic data is still subject to noise and artifacts and even susceptible to longitudinal smearing, which complicates image segmentation and feature extraction. Razdan et al. [18] adopted some methods to remove noise, suppress artifacts, and enhance edges in the pre-processing stage to improve image quality. TINVIZ was developed to visualize multi-channel data using ray casting and a voxel-level merging approach. Kyan et al. [13] employed local energy as an image statistics for surface detection, and Kohoonen self-organizing feature map, which is an image segmentation method based on neural networks, for effective feature extraction from translucent chromosomes. Mullick et al. [19] presented a fast algorithm driven by physically defined parameters to show internal structures without using a traditional segmentation process which might cause artifacts. Maddah et al. [20] proposed an automatic 3D center-line extraction algorithm based on distance transform and path planning to extract elongated vessels from branching structures of rat brains. Connected thin center-lines can be accurately and rapidly generated without user interaction or a priori knowledge of the object shape.

Visualization of time-varying confocal microscopic data helps interpret complex bio-chemical processes. De Leeuw et al. [21] used iso-surface, volume rendering, and animation techniques to show the functioning of living cells. They then developed an interactive visualization and feature tracking system to investigate the movement of chromatin during de-condensation. Features are defined as points in a multi-dimensional attribute space while the distance between two points is taken as a metric to evaluate the feature correspondence [22]. Recently they adopted landmark-based and voxel-based methods in retaining only internal cell movement while removing acquisition-induced external motion from time-dependent chromatin datasets [23].

We are working on visualization of biomolecular structures from large-scale confocal microscopic data to help biologists

• devise an accurate mapping of structure-function relationship that defines the activity of individual proteins and the molecular mechanisms through which they operate and

• determine changes in protein expression patterns in response to pathogens and environmental stress.

One goal is to develop a CAVE-based visualization system to facilitate the discovery of similarities and discrepancies between structures, to help solve connectivity and docking problems (proteins, enzymes, drugs), and to allow interactive manipulation of the structures to study properties to predict the behavior of new molecules and specifically-designed RNA in an immersive environment. Amira [24] is a modular, object-oriented geometry reconstruction, data visualization, and pattern analysis package supporting Tool Command Language (TCL) and component (data objects and computational / rendering modules) networks. It offers a shortcut to the cutting edge of biomolecular visualization so that research efforts can be directly focused on algorithm design and problem solving. Amira allows us to add in custom modules such as data readers, data writers, data processing and visualization routines, and scene rendering programs based on Open Inventor to enhance the already powerful visualization capability. AmiraVR, an important extension of Amira for virtual reality applications, enables us to quickly develop an immersive biomolecular data visualization system based on a four-wall CAVE.

Biologists at the University of Southern Mississippi, our collaborators, acquired a multi-channel volume dataset (dimensions: 512 x 512 x 15; rectilinear voxel size: 0.0557892um x 0.0557892um x 0.3642286um) using confocal microscopy from a budding yeast saccharomyces cerevisiae containing two transcriptional activators: protein GCR1 and protein GCR2. GCR1 was labeled with transgenetic Cyan Fluorescent Protein (CFP) and excited by a 458nm laser to generate channel #1 while GCR2 was labeled with transgenetic Yellow Fluorescent Protein (YFP) and excited by a 514nm laser to generate channel #3. Channel #2 was captured by using Differential Interference Contrast (DIC) as a transmitted light image with enhanced contrast. The biologists attempted to gain a better understanding of the interaction between proteins GCR1 and GCR2 through interactive data visualization than mental reconstruction.

Confocal microscopy introduces so-called z-drop artifacts during volume data acquisition due to light absorption by slices. As a result, the average intensity in lower slices tends to be decreased. To correct the intensity attenuation, we adjusted the average intensity in each slice by multiplying by an exponential function j = a x EXP[ -b x (N - 1 - n) / (N - 1) ]where a and b are user-defined parameters tunable based on the number of slices N, and n is the slice index ranging from 0, the highest slice, to (N - 1), the lowest slice. We chose a = 1.05 and b = 0.15 to correct the aforementioned data.

The raw volume data was first quickly displayed using a projection view, which shows three orthogonal slice images generated by projecting the maximum (or average) values through the volume in the x, y, and z directions, respectively, and color-mapping the resulting values, i.e., by axis-aligned Maximum Intensity Projection (MIP). The Region of Interest (ROI) was then easily located based on the sketch patterns to visually prune blank surrounding regions in the data pre-processing stage, which reduced the computational workload for an increased rendering speed.

Figure 2: Iso-surfaces extracted from channel #1 (GCR1: top-left), channel #2 (DIC: top-right), and channel #3 (GCR2: bottom-left). All the iso-surfaces are displayed in the bottom-right view.

Figure 3: Volume rendering of channel #1 (GCR1: top-left), channel #2 (DIC: top-right), and channel #3 (GCR2: bottom-left). All the channels are volume rendered in the bottom-right view.

Figure 4: Hybrid-rendered images used to highlight channel #1 (GCR1: top-left), channel #3 (GCR2: top-right), channel #1 and channel #3 (bottom-left), and channel #2 (DIC: bottom-right).

Figure 5: Volume rendering of channel #1 minus the overlap (top-left), channel #3 minus the overlap (top-right), the overlap (bottom-left), and all the channels combined (bottom-right).

Figure 6: Hybrid rendering of the volume with the overlap (in blue) extracted to show where GCR1 and GCR2 do and do not interact with each other. Along with the scattered overlap, GCR1 in the left image and GCR2 in the right image are highlighted.

Figure 7: The Amira component network (left window) and the embedded C ++ module (right windows) for channel extraction.

Figure 8: Volume rendered images of the intensity of the interaction between GCR1 and GCR2. The color-map is based on the intensity with blue being lowest and red highest. The left image with a higher overall opacity shows hot spots discernible through translucent interleaved lower-intensity regions while the right image shows interior hot spots revealed using a lower overall opacity.

Figure 9: CAVE-based visualization and exploration of the biomolecular data.

Visualization of confocal microscopic biomolecular data is a challenging research topic due to low signal-to-noise ratio, low image contrast, longitudinal smearing, large data size, amorphous structures, ragged contours, porous surfaces, numerous disconnected small objects, and in particular lack of a priori knowledge about intractable, possibly time-varying micro-structures. Effective visualization techniques need to be developed to address confocal microscopic biomolecular data. The data pre-processing techniques and the correlation-based channel extraction method introduced in this paper are very effective in artifacts suppression, noise removal, and ROI location. Some techniques such as the statistics-based evaluation of intensity of protein interaction, though simple, are very appropriate to meet specific visualization needs of the user. Hybrid rendering is a powerful visualization approach, which draws on the advantages of both isosurface extraction and volume rendering to provide various aspects of insights into the volume data in the same view. Immersive visualization based on CAVE even provides more intuitive perception of the data.

Future work goes in many directions including image de-convolution, image segmentation and registration, effective transfer function design for volume rendering, and feature extraction and tracking in time-varying data.