Center of Excellence in Wireless and Information Technology

>home/research/

Graphics, Imaging & Visualization

Computer Graphics and Visualization Using Conformal Geometry
PIs: Xianfeng David Gu (SBU) Baoquan Chen, Yousef Saad (University of Minnesota-Twin Cities)

Geometry representation is fundamental to modeling, design, analysis, simulation, and graphics rendering which represent key operations in various scientific and engineering applications. However, current geometry representations, such as the most popular ones based on polygonal meshes, are not convenient for the underlying applications for which they are used. This results in nonintuitive, complicated, and sometimes even unachievable operations. The proposed research is to apply conformal surface theory to various geometry representations to compute conformal structures. Using computed conformal structures, new geometry representation and analysis tools can be developed, which will pave the road for advances in multiple fronts of science and engineering. The work proposed herein will especially explore these potentials in computer graphics and visualization. A conformal structure uniquely characterizes surfaces. Many laws of physics are governed by conformal structures. For example, heat diffusion and electromagnetic field distribution on surfaces, tension in soap bubbles and parts of string theory in theoretical physics are determined by conformal surface structures.

For computer graphics, geometric modeling and much of scientific computing, however, the potential of conformal structures has essentially been untapped so far. To illustrate this potential, consider one aspect of conformal structures, namely the canonical flattening of a surface into the plane, resulting in geometry images, by global conformal parameterization.

This recasts many three dimensional (3D) geometric problems into two dimensions (2D) and leads to efficient approaches for a number of fundamental geometric problems. These approaches can then immediately benefit a wide range of applications, such as surface classification, surface matching and shape analysis, geometric modeling, simulation, graphics rendering and visualization. Performing conformal parameterization requires solving large least squares problems. To further push the application of the conformal structure to interactive or time critical operations, the research team will investigate novel iterative methods for least-squares problems based on sparse QR and incomplete sparse QR algorithms. Intellectual merit: The proposed research has the potential to lead to seminal advances in theory, methodology, and techniques of computing conformal structures through the development of (a) conformal parameterization of various geometry representations including polygon meshes, implicit surfaces, B-Spline surfaces, point geometry, and volumes; (b) incomplete QR-based methods for solving large least-squares problems; and (c) geometry-image based geometry analysis, completion, synthesis, rendering and visualization.

The research involves an approach which combines solid mathematical foundations, tool development, and effective domain algorithm designs. Broader Impact: The proposed research may have a major impact on computer graphics and visualization because the new approaches promise a paradigm shift concerning all aspects of graphics processing and rendering. The benefits of the new approach may foster similar adaptation and eventually similar paradigm shifts in other applications which are empowered by geometry representation and analysis and visualization techniques. Furthermore, the use of iterative methods for least-squares problems, though developed mainly to accelerate computing conformal structures, constitute a vital tool in a wide range of applications, primarily those related to optimization and statistics. Overall, the tools developed in this project can help boost the development of effective techniques to deal with the emerging problems related to scientific simulation, data exploration, and the new economy. The project has a natural science and engineering education component through the research participation of graduate and undergraduate students, including three Ph.D. students. Using several mechanisms, the PIs are committed to including undergraduates and underrepresented minority college students in the research. (NSF)

O-Buffer: A Framework for Sample Graphics
PI: Arie Kaufman

In this project we are developing an innovative modeling and rendering primitive, called the O-buffer, for sample-based graphics, such as images, volumes, and points. The two-dimensional or three-dimensional O-buffer is in essence a conventional image or a volume, respectively, except that samples are not restricted to a regular grid. A sample position in the O-buffer is recorded as an offset to the nearest grid point of a regular base grid (hence the name O-buffer). The offset is typically quantized for compact representation and efficient rendering. This project investigates and develops representations, algorithms, applications, and a software system for O-buffers. The intellectual merits of the O-buffer are that it is more precise than a conventional image or volume, provides a more compact representation relative to a finer grid, accelerates rendering and supports low-bandwidth communications, improves image quality, is a flexible representation (e.g., non-regular, non- uniform, multi-resolution), and is a unified framework that can represent, render, and mix unstructured primitives. O-buffers can be harnessed in numerous applications, such as in wireless graphics, Internet graphics, improving simulation systems, scientific visualization, design and manufacturing, and health care, as well as enhancing and upgrading many other computer systems that use sampled-based data, such as images. (NSF)

Conformal Geometry Applied to Shape Analysis and Geometric Modeling
PI: Xianfeng David Gu

The conformal structure is a natural structure associated with Riemann surfaces and plays fundamental roles in geometry and physics. Conformal structure has proven useful in graphics and vision: conformal parameterizations provide high quality texture mapping without local distortion, as well as surface matching and morphing with applications such as brain mapping. Intellectual merit. The overall career goal is to explore the potential of conformal geometry for computer graphics and vision and to ultimately make the interdisciplinary field of computational conformal geometry accessible and useful to society. The direct research goals are to apply conformal geometry to geometric modeling and shape analysis, specific:
1. Construct shape spaces based on Teichmuller space theory, where the space of all surfaces is modeled as a finite dimensional manifold, each point represents a conformal equivalence class of surfaces (a Riemann surface), and the metric of the shape space measures the deviation of conformal structures of the two shapes. Additional related goal is to build geometric search engine.
2. Find a systematic way to generalize geometric constructions defined on planar domains to manifolds, such as manifold triangular BSplines and manifold Powell- Sabin surfaces. Design new surface subdivision schemes by inserting knots into manifold BSpline surfaces. The educational goal is to find an effective way to teach conformal geometry (Riemann surface theory) for non-math majors by visualizing the abstract concepts and implementing practical algorithms to establish the understanding of profound theories.

Broader impact. Conformal geometry theory is mature but abstract. We will build and disseminate a concrete set of software tools for computing and visualizing the conformal structure of arbitrary real surfaces. This will make the theory accessible to students and its practical applications useful to the broader community. Manifold BSpline tools based on conformal geometry will bridge the gap between traditional polygonal meshes in graphics and spline surfaces in CAGD. The generic geometric search engine can be applied to a geometric database and an Internet search engine. We will complement the software development with a systematic development of the classic material in a context that permits integration into the curriculum of non-math majors. Computer graphics, vision, scientific computing, medical imaging, mathematics and physics will benefit from the research and education of computational conformal geometry directly. Computational conformal geometry has already made impacts on the graphics industry and will be more broadly applied in the future. (NSF)

Integrate CAD to Virtual Colonoscopy for Cancer Screening
PIs: Arie Kaufman and Jerome Liang

Studies have shown this non-invasive procedure to be an accurate, comfortable technique for screening large segments of the population. From the physician’s perspective however, VC still requires careful inspection of the entire length of the colon, a somewhat tedious process that takes at least 15 minutes for each scan. For about a decade our research group at Stony Brook University (SBU) in collaboration with industry has been advancing VC development and commercialization. Currently, our focus is on the research and development of 3D texture-based computer aided detection (CAD) techniques using 3D texture information within the colon wall. The goal of this effort is to exploit polyp characteristics in the CT data to automatically identify suspicious regions of the colon wall. The outcome will be to increase the physician’s efficiency reading the VC data as well as the overall sensitivity of the procedure. Our specific aims are as follows:

1) To exploit 3D texture information within the colon wall to rapidly and accurately identify suspicious regions.
2) To develop interactive CAD reading strategies that effectively blend the efficiencies of computer and human skills.
3) To accelerate the CAD processing and interactive rendering using commodity graphics hardware.
4) To integrate CAD and interactive rendering into a commercial VC system.
5) To validate the integrated system on a patient database.
We anticipate that integrating texture-based CAD technology into a commercial VC system will substantially enhance VC by providing superior CAD detection performance while dramatically reducing the physician’s interaction time with the system. Improving the cancer-finding capability and cost-effectiveness of VC will ultimately result in better outcomes for a greater number of people. (NIH)

Medical Image Processing and Intelligent Cerebral Hemorrhage Diagnosis System on Wireless Network
PI: Arie Kaufman

In this project, we develop an alerting and monitoring system for intensive care and first-aid patients in a general hospital through a wireless network using a client/server system. We call it Emergency Alerting and Monitoring System (EAMS). The system consists of Smart Phone-based clients and an EAMS server, which is designed for integration with an existing Hospital Information System (HIS). A doctor may carry a Smart Phone or a PDA and observe the latest status of the patients anywhere at anytime. The Smart Phones allow the doctors to access the wireless network based on CDMA communication as well as IEEE 802.11b wireless LAN. Thus, the doctor can receive an alert message with a link, which point to an access location on the server, through the CDMA communication even if their Smart Phones are not connected to a wireless LAN. In addition, the client program provides a convenient user interface to present the medical images as well as a substantial amount of numerical data, which overcomes the tiny screen size of the Smart Phone. The server consists of four modules: the alerting and monitoring service module, the HL7 message broker module, the JPEG2000 converting module, and the medical image processing module.

First, the alerting and monitoring service module gathers the patients’ information from the HIS and alerts the doctor of an emergency situation if it finds any abnormality in the information. Second, the HL7 message broker module supports the international standard interface for integrating between the EAMS and the HIS. Third, the JPEG2000 converting module converts medical images from PACS to JPEG2000 formatted images for transferring them through the wireless network with minimal loss of image quality. Finally, the medical image processing module is a service module which transfers the medical images from the server to the client so that the doctor can review patient’s images and diagnose the illness. The proposed system can provide doctors with the urgent information in time, by which doctors can see and diagnose their patients’ information through the CDMA communication as well as the Wireless LAN. Furthermore, it can promote the utilization of mobile devices in medical environment and eventually serve the purpose of the ubiquitous computing environment. Moreover, we are implementing automatic cerebral hemorrhage diagnosis function using image processing techniques from a brain tomogram image to decide whether a patient has an cerebral hemorrhage or not. The automatic cerebral hemorrhage diagnosis function operates on the server that transmits the images to the clients. This function automatically operates when the server transmits a brain tomogram image by a request from a client. (Metrosoft; Wireless Internet Research Institute in Gyeonggi Province, Rep. of Korea)

Developing Virtual Colonoscopy for Cancer Screening
PIs: Jerome Liang and Arie Kaufman

Colorectal carcinoma is the third most commonly diagnosed cancer and the second leading cause of death from cancer in the United States. Since most cancers arise from polyps over a 5-15 year period of malignant transformation, screening programs to detect small polyps less than 1 cm in diameter have been advocated. Unfortunately most people do not follow this recommendation. The health relatedness of this project is to dramatically increase the number of people willing to participate in screening programs by using a convenient, risk-free procedure. Virtual colonoscopy (VC) is a new procedure that we developed. It employs computed tomography (CT) scanning and volume visualization, and is poised to become the procedure of choice in lieu of the conventional optical colonoscopy for mass screening for colon polyps — the precursor of colorectal cancer. The patient’s abdomen is imaged by a helical CT scanner during a single-breath-hold. A 3D model of the patient’s colon is then reconstructed from the CT scan by automatically segmenting the colon out of the abdomen followed by electronic cleansing — computer-based removal of residual material in the colon. The system, running on a PC, allows physicians to interactively navigate through the colon and view the inner surface using volume rendering, with tools for measurements, electronic biopsy, to inspect suspicious regions, as well as painting already seen areas to help in visualizing 100% of the surface. Unlike optical colonoscopy, VC is a patient friendly, fast, non-invasive, more accurate, inexpensive procedure. VC has been extended to 3D virtual endoscopy of other organs, such as the heart, arteries, lungs, stomach, and bladder. To further advance this technology, the specific aims of this project are: (1) to investigate low-dose CT techniques for VC towards massive screening of colonic polyps; (2) to extend electronic colon cleansing strategies to extract the colon mucosa layer by mixture-based image segmentation; (3) to investigate integrated feature-extraction techniques for polyp modeling towards computed aided detection (CAD) of colonic polyps; and (4) to extend our current real-time volume-rendering based navigation algorithms to include CAD and interactive virtual biopsy means for analysis of suspected abnormalities.

The research design and methodology will include evaluating the ability to electronically clean the colon lumen and extract the mucosa layer with less-stressful bowel preparation from low-dose CT images; the feasibility of bringing the technology to a readily accessible environment by documenting VC speed and quality with CAD and interactive virtual biopsy tools through the entire colon; and the accuracy by comparing virtual and optical colonoscopy polyp detection in the same patient using a pilot study. (NIH)

Plume Modeling Simulation and Visualization
PIs: Arie Kaufman and Klaus Mueller

We have adopted a numerical method from computational fluid dynamics, the Lattice Boltzmann Method (LBM), for real-time simulation and visualization of flow and amorphous phenomena. Unlike other approaches, LBM discretizes the micro-physics of local interactions and can handle very complex boundary conditions, such as deep urban canyons, curved walls, indoors, and dynamic boundaries of moving objects. Due to its discrete nature, LBM lends itself to multi-resolution approaches, and its computational pattern, which is similar to Cellular Automata, is easily parallelizable. We have accelerated LBM on commodity graphics processing units (GPUs), achieving real-time or even accelerated real-time on a single GPU or on a GPU cluster. Another key innovation of LBM is its extension to support input from pervasive sensors, influencing the simulation so as to maintain its faithfulness to real-time live sensor readings. We have implemented a 3D urban navigation system, featuring a 3D polygonal model GIS with façade texturing, flow visualization streamlines, volume rendering plumes, and information visualization of real-time live sensor data.

We have tested it with a 10-block GIS in the West Village of New York City, overlaid with results of dispersion simulation and real-time readings from 3 meteorological sensors, and with an 851-building area in Times Square of NYC. In addition to a pivotal application in simulation of airborne contaminants in urban environments, our approach will enable the development of other superior prediction simulation capabilities for physically accurate environmental modeling and disaster management. It will also increase the level of accuracy and speed achievable in visual simulations for the computer graphics industries. It may further lead to a novel technology for computational science and engineering, which has the potential to revolutionize the way scientists and engineers conduct their simulations. (DHS, NASA, Millennium Center)

3D Visualization for the Korean Visible Human
PI: Arie Kaufman

We have been developing 3D volume modeling and visualization for the Korean Visible Human. The latter consists of numerous slices, each of which is a color image with a resolution of 1080x1110. We focus primarily on the brain area of 158 slices. The hand-segmentation of 13 sub-organs is also stored in 158 color images. The storage of the segmentation information is reduced by using color index, for a total of about 1GB. In order to render the dataset using a direct volume rendering technique, the opacity for each voxel is determined in a pre-processing step. In that step, we convert the color information from RGB to the YIQ color model, and use the Y component as the density. The volume is then divided into small cells; the non-empty ones are compressed and saved. We use a fast GPU (Graphics Processing Unit)-based object-order ray-casting algorithm to render the dataset, which allows the user to interactively explore the human brain. Our renderer decompresses the non-empty cells and store them into two textures on the GPU. Our cell projection algorithm is implemented using the pixel shader running on the GPU. At each sampling point along a ray segment, color, density, and segment index are obtained through texture lookup. The user interface is also implemented using Direct3D. (HuminTec, Rep. of Korea)

Point-Based and Image-Based Volumetric Rendering and Detail Modeling For Volume Graphics
PI: Klaus Mueller

The objective of this work is to advance and promote the emerging field of volume graphics for science, engineering, medicine, education, and entertainment. It outlines novel data representations, innovative rendering techniques, and creative modeling schemes that are all geared towards making the rendering, processing, and interaction with volumetric datasets more feasible and more realistic in a wide range of application domains. Traditionally, volumetric datasets have been most popular in computational science and medicine, where they are natively produced by mathematical simulations, such as computational fluid dynamics or finite elements, as well as volumetric scanners, such as MRI, PET, SPECT, and CT. Over recent years, the size of these datasets has grown at an alarming pace. Medical datasets of one billion volume elements (or voxels) and more have become quite frequent, and scientific datasets produced in numerical simulations or acquired in observational studies are even larger, by magnitudes. It is projected that scientific datasets soon will be in the range of terabytes and even petabytes, and the need to develop efficient visualization methods for these has been identified as an immediate national interest.
The proposed work encompasses:
1) Point-based representations— A natural compression of the volume as well as of the computational burden can be achieved by storing and rendering only the relevant voxels, each represented as a basic point primitive. Extending previous work, we proposing novel ways for representing, organizing, and rendering volumetric points.
2) Point-based volumetric objects—
By encoding a volumetric object into a set of abstracted points, called rayels, one can obtain a compact representation. Rayel-based objects are rendered very efficiently, with many volumetric effects, such as view-dependent semi-transparencies, natural phenomena, and others.
3) Image-Based Rendering (IBR) Assisted Volume Rendering— IBR is an emerging technology for surface graphics to hide rendering complexity by reusing image information from previously rendered frames. This proposal advances the technology and extends these concepts to capture the intricacies of volumetric integration (VOL-IBR). It provides a performance booster for almost any volume renderer and can be run as a client-server system over the net.
4) Subdivision volumes with detail-on-demand— we introduce a novel methodology that adds artificial or natural detail to the volume whenever the user approaches the resolution limits of the volume. The same datastructure can also be used to resample irregular grids into a grid that can be rendered with multi-resolution. (NSF)

A Unified Framework for Rapid CT on Commodity GPUs
PI: Klaus Mueller

We devise a general and unified framework to harness the abundant performance of current and future generation commodity graphics hardware (GPUs) for the purpose of tomographic reconstruction from projections. Our results indicate that a performance improvement on the order of 1-2 magnitudes over traditional CPU-based approaches can be obtained. Our framework is general and allows the reconstruction from a diverse set of raw data (such as kV X-rays, MV X-rays, and protons), with a diverse set of reconstruction algorithms (such as maximum likelihood algorithms, algebraic methods, and Feldkamp-style filtered back-projection), and within a diverse set of application scenarios (such as CT, SPECT, PET, and Proton CT). Starting from the basic reconstruction operators, we model all dominant physical and algorithmic effects that occur in radiation-based tomography, such as depth weighting, detector geometric response, attenuation weighting, and scatter compensation, using implementations that map optimally to the graphics hardware and take full advantage of its computational architecture. The rapid speeds of reconstruction also enable a new concept that we call Visual Reconstruction Steering (VRS). This VRS framework will consist of a visual interface in which users can build and interactively control reconstructions as they occur on the GPU, and in which they can visualize and assess the results in real time. (NIH)

Intelligent Deformable Models
PIs: Hong Qin, SBU, Demetric Terzopoulos, UCLA, Stanley Osher, UCLA, Ronald Fedkiw, Stanford

Deformable models and level set methods are related techniques that have proven to be phenomenally successful computational tools across a variety of disciplines, ranging from computer vision and image processing, computer graphics and image synthesis, computer-aided design and geometric modeling, as well as in applied mathematics and physics. The two techniques, each a major investigative avenue in itself, are complementary in several fundamental ways. The goal of this project is, for the first time, to harness the complementary strengths of these two physics-based methods. We will do so by unifying them under a biology-based control paradigm derived from the emerging field of artificial life. Our unification will lead to a novel breed of intelligent, deformable organisms capable of performing a wide range of challenging data analysis tasks, such as image segmentation and data reconstruction, in a highly automated fashion.
The technical merit of the proposed research activities is the goal of incorporating within an ultimately symbolic control hierarchy, two fundamentally numeric methods, the first related to the continuum mechanics of solids, the second related to liquids. Hence, deformable models maintain their topological structure as they evolve, while level set methods are topologically adaptive. Enabling these methods seamlessly to join forces within a rigorous mathematical and tractable computational foundation is an intellectually challenging problem. The anticipated outcome is a new, highly automated methodology for analyzing large-scale datasets that are subject to uncertainty and noise. Through an array of research activities, this investigation will dramatically advance the state of the art in the aforementioned fields, all of which have strategic value in the US information technology industry. Complementary subgoals will be to demonstrate that the new modeling paradigm is not only a powerful analysis tool for visual information processing, but that it can potentially serve as a general computational technology to aid in new scientific discovery. In particular, our novel models can be deployed in massive datasets to automatically extract geometric boundaries and discover their unknown topology of structures of interest within the data. For example, with the vast quantities of medical images routinely collected for diagnostic and research purposes, this project will serve the critical need for the development of tools to aid in their analysis. These new models can alleviate the burden of laborious human operation for the processing of large-scale datasets, enhance the efficiency of domain scientists as well as ordinary users, facilitate modeling and rendering tasks, and streamline the entire visual information processing pipeline. (NSF)

MASSIVE: Multiresolution, Adaptive, Subdivision Surfaces for Interactive Visualization and Exploration
PIs: Hong Qin, Joe Mitchell

One fundamental research challenge of advanced computational science is to aid scientists, researchers, engineers, and general users to gain a better understanding of large-scale datasets acquired from either computer simulations or real-world experiments. Therefore, it demands better modeling, analysis, and visualization tools that can reveal the insight from raw datasets and facilitate the interpretation of high-level knowledge. Towards this goal, the technical approach in this multi-disciplinary project is to develop novel algorithmic and computational techniques founded upon the principle of the deformable modeling paradigm for manipulating, simulating, visualizing, and processing any large-scale, complex dataset, hence leading to a better understanding of the higher-level, more meaningful information hidden within raw data subject to uncertainty and noise. This research initiative aims to develop new theoretic, algorithmic, computational, and software techniques within the mathematically rich and broadly applicable deformable models paradigm, with an ambitious goal to further revolutionize deformable models and promote them as a valuable visualization and exploration tool.
The intellectual merit of the proposed research is the unique technical approach of developing a suite of novel deformable models and presenting an integrated methodology for modeling and visualizing both complicated geometric information and arbitrarily, unknown topological structure in large-scale, complex datasets. (NSF)

Acceleration of 3D Electron Microscopy Reconstruction Using Graphics Hardware
PIs: Klaus Mueller, SBU, David Agard, UCSF

We are developing a framework for GPU-accelerated 3D reconstruction of parallel-beam projection data obtained with electron microscopes. In order to overcome and correct inherent device problems with data alignment and other artifacts, we will implement an iterative algorithm in the spirit of SART or SIRT, which will allow certain correction procedures to be applied while the reconstruction is performed. Special emphasis will be given to the fact that the projection data are large, on the order of 2k x 2k pixels, which will require special techniques for management of the limited memory on the graphics card.

The essence of our novel deformable models is multiresolution, level-of-detail (LOD), and subdivision geometry whose topology is also dynamically adaptive subject to variational principles (VPs) and partial differential equations (PDEs). Through an array of research and education activities, this investigation aims to demonstrate that the novel deformable models are not only a powerful modeling, rendering, and simulation tool for visual information processing, but that they can potentially serve as a general computational technology to aid in new scientific exploration. Meanwhile, we will incorporate the newly-developed theory and algorithms into our graduate curricula in computer science, applied mathematics, and statistics of Stony Brook. These curricula will expose graduate students to a novel perspective on visual computing based on deformable models, which will significantly improve their problem-solving skills in an interdisciplinary context. (NSF)

A Haptic-based Interface and Sculpting System for Virtual Environments
PIs: Hong Qin, Arie Kaufman

In the modern information era, the world is faced with ever-increasing expectations of improved product quality and reduced product prices from customers. These challenges demand more advanced design tools and more effective human-computer interactions that permit designers to complete their work as quickly and inexpensively as possible. Thus, the grand success of future information-driven design technology hinges upon the rapid advancement of powerful, efficient design tools coupled with novel human-computer interface. The technical vision of this cross-disciplinary initiative is to develop an interactive, tangible virtual environment that can significantly advance the current state-of-the-art in human-computer interaction through the novel integration of dynamic modeling and real-time haptic sculpting. Computer-centered engineering design consists of a variety of complex and challenging processes, ranging from conceptual design, geometric modeling, evaluation, prototyping, manufacturing, assembly, to production.

To ameliorate CAD/CAM processes, we have pioneered the novel, physics-based modeling technology. To realize the full potential of physics-based design in industrial practice as well as to revolutionize human-computer interaction technology, we center our endeavors on the R&D activities of real-time physical interaction coupled with realistic haptic sculpting capabilities. In this initiative in particular, we propose to develop an interactive design environment founded on the integrated principle of haptic interaction and physics-based modeling. This investigation will significantly advance the state of the art of computer-integrated engineering design in three aspects:
1) The goal is to pioneer a novel theory and technology unifying haptics-based interaction and physics-based modeling, designers will be free to choose assorted haptic sculpting tools for direct interaction of geometric primitives;
2) The objective of the research is to address the symbolic, numerical, geometric, and algorithmic problems inherent to real-time haptic sculpting, the effective simulation of computational physics, and the accurate approximation of diverse geometric primitives in a unified way. Finding satisfactory solutions to these funda-mental issues are of importance to both the theory and practice of human-computer interaction; and
3) We will develop a novel haptics-based virtual design environment, and explore the challenge of applying the haptic interface and interactive system to an array of design activities and visual computing applications including conceptual design, shape sculpting, virtual prototyping, surgery simulation and training, and haptic exploration of scientific visualization.

Novel and natural human-computer interaction techniques underpin the success of future design technology. Therefore, it is our hope that this project will not only advance the state of the knowledge of computer-integrated engineering in improving product quality, reducing product cost, and increasing the effectiveness of design engineers, but that it will also make significant contributions to human-computer interface methodology and interactive techniques. (NSF)

An Interactive Graphical Modeling System based on Dynamic Subdivision Splines
PI: Hong Qin

The technical vision of this ITR project is to develop software environments that can significantly facilitate human-computer interaction through the physics-based modeling of graphical entities. Our novel approach aims to broaden the accessibility of graphical/visual modeling by combining conventional geometric models with computational physics, thus offering novel interactive methodologies based on realistic model behavior. In particular, the technical challenge of our research investigation focuses on the interactive manipulation and direct sculpting of Subdivision Splines (S-splines). Recent years have seen dramatic growth in the use of S-splines for graphical modeling and animation, especially for the representation of smooth, oftentimes complex shapes of arbitrary topology. Unfortunately, conventional interactive approaches to subdivision objects can be extremely laborious and inefficient.

Modelers must carefully specify the initial mesh and/or painstakingly manipulate the control vertices at different levels of the subdivision hierarchy in order to satisfy the functional requirements and aesthetic criteria in the modeled object. In this project, we aim to realize the full potential of combining physics-based interaction with powerful subdivision geometry, both in terms of its theoretical aspects and its practical aspects in visual computing applications. We propose to concentrate our endeavors on dynamic interaction with subdivision geometry through a wide array of research and education undertakings on fundamental theory, integration of theoretical and algorithmic advances, modeling/design/ simulation tools, prototype environments, industrial collaboration, and technical dissemination to U.S. IT enterprises. The primary thrusts of this project are to help improve human-computer interaction methodologies, develop new interface modes, make physics-based interaction accessible to all users, and enhance the capabilities of graphical modelers. The novel framework to be developed in this project will not only advance the state of the art of human-computer interaction technology, but also geometric modeling, computer graphics, visualization, robotics, and virtual environments. (NSF)

Sloan Research Fellowship on Geometric and Visual Computing
PI: Hong Qin

Computer-centered engineering design consists of a variety of complex and challenging processes, ranging from conceptual design, geometric modeling, evaluation, prototyping, manufacturing, assembly, to production. In modern information era, the world is faced with ever-increasing expectations of improved product quality and reduced product prices from customers. These challenges demand more advanced design tools and more effective human-computer interactions that permit designers to complete their work as quickly and inexpensively as possible. Thus, the grand success of future information-driven design technology hinges upon the rapid advancement of powerful, efficient design tools coupled with novel human-computer interface. The on-going research activities and future research initiatives is to develop an interactive, tangible virtual environment that can significantly advance the current state-of-the-art in human-
computer interaction through the novel integation of physics-based dynamic modeling, real-time haptic interaction, and conventional geometric and visual computing.

To realize the full potential of physics-based modeling in geometric and visual computing applications, we will continue our endeavors to investigate new theory, explore novel and powerful models, and develop innovative systems that will lay a solid foundation for the framework of the future generation of physics-based CAD/CAM. In particular, we conduct various research activities towards an interactive, computer-integrated virtual environment for the unified engineering design, analysis, and manufacturing. The objective of the future research is to advance the state-of-the-art knowledge of the entire CAD/CAM processes and forge ahead to achieve the ultimate objective of computer-integrated virtual engineering. CAD/CAM is an iterative operation which comprises conceptual design, detailed part drafting, finite element analysis, evaluation, part assembly, prototyping, and manufacturing. The advent of Virtual Reality (VR) and internet technologies make it both possible and necessary to revolutionize traditional design and manufacturing techniques and systems. The new computer-integrated virtual engineering promises the significant cost-reduction of performing CAD/CAM processes.

The concrete research plan includes:
1) research and develop physics-based interaction tools that permit designers to interactively create and modify product geometry with haptic feedback, and further examine design requirements and criteria in real-time
2) investigate graphical models and visualization tools that support the effective graphical simulation of finite element analysis through the extensive use of D-NURBS finite elements and FEM-based subdivision splines
3) explore and develop graphical tools that can simulate manufacturing process without the need of physically manufacturing the mechanical part
4) develop other simulation tools to facilitate the process of decision making, the generation of alternative design configurations, the assessment of product assembly-ability, etc.

Through the VR-based CAD/CAM simulation, it is also our hope that this system can provide a test-bed to verify new algorithms and ideas for design and manufacturing purposes. Throughout this research, we focus on large scale, complex, real-world CAD models and/or real objects. To make the maximum use of the state-of-the-art graphics and VR facilities in the Center of Visual Computing (CVC) at Stony Brook, we also lead efforts to investigate the design of parallel algorithms and develop distributed design tools that can transform our physics-based CAD/CAM system into a distributed environment. (Alfred P. Sloan Foundation)

Face Analysis and Recognition Under Variable Illumination
PI: Dimitris Samaras

We are investigating face recognition methods from images taken under arbitrary illumination, where a low-dimensional spherical harmonics basis is computed for each image from a statistical representation of spherical harmonics images. Combined with a morphable shape model, this approach has given excellent results in re-synthesis of facial images under different pose, illumination and even expression. (NSF-ITR, Dept of Justice)

Facial Expression Acquisition, Tracking and Learning
PIs: Dimitris Samaras, Peisen Huang, David Gu (SBU), Dimitris Metaxas, Ahmed ElGammal (Rutgers)

This project addresses fundamental issues regarding the use of high quality dense 3-D data samples undergoing motions at video speeds, e.g. human facial expressions. In order to utilize such data for motion analysis and re-targeting, correspondences must be established between data in different frames of the same faces as well as between different faces. This project investigates data driven approaches that consists of:
1) High speed, high accuracy capture
of moving faces without the use of markers,
2) Very precise tracking of facial motion using deformable meshes and conformal geometry,
3) Low dimensional mappings of dynamic facial motion that can separate expression style.

The accuracy and resolution of our method allows us to capture and track subtle expression details. (NSF-ITR, Dept of Justice)

Machine Learning Techniques to Analyze Dynamic Functional Neuroimaging Patterns Underlying Inhibitory Control Mechanisms
PIs: Dimitris Samaras, SBU, Rita Goldstein, Nelly Alia-Klein (BNL)

Investigation of novel computational techniques to analyze brain-behavior relationships underlying mechanisms of inhibitory control, focusing on performing classification of hard-to-categorize groups of subjects based on brain activation response patterns to behavioral challenges of inhibitory control using functional magnetic resonance imaging (fMRI). Unique patterns of variability in brain function can assist in identification of brain mechanisms rooted in compromised inhibitory control. Such patterns will increase understanding of brain connectivity and circuitry between a-priori and exploratory means of describing circuits of inhibitory control. An integrated machine learning framework for the joint exploration of spatial, temporal and functional information of fMRI signals, allows the testing of hypotheses and development of applications that are not supported by traditional analysis methods. (NIH, NIDA)

Using Shared Eyegaze to Coordinate Time-Critical Collaborative Tasks
PIs: Greg Zelinsky, Susan Brennan

Research consists of controlled experiments that explore basic processes of human coordination in time-critical situations. Partners will collaborate on a set of screen-based tasks, and the effects of shared eyegaze will be examined. The effects of these different communication modes will be compared using the grounding framework in order to discover the independent and combined contributions of speech, gaze, and other pointing methods upon interpersonal coordination on a fine timescale (100 milliseconds to seconds). Tasks include searching together to locate a target (as police officers searching for a sniper or a team of ornithologists searching for a bird among trees), tracking moving targets (as security personnel tracking a suspect in a crowd within an auditorium or a pair of marine biologists tracking a swimming animal), and establishing consensus using referential communication (as doctors referring to ambiguous or hard-to-describe patterns in medical images or programmers helping each other debug software). (NSF-HSD)