Astrophysical crash: Computing colliding black holes

Scientists at The University of Texas at Austin, NCSA/UIUC, and six other universities are collaborating on a Grand Challenge effort to investigate what happens when two black holes collide.

NSF's High Performance Computing and Communications Program awarded $3.75 million--$550,000 for NCSA input--to fund the research.

"It's a big project to try and compute," says Ed Seidel, NCSA research scientist and leader of its Relativity Group. "It's a step-by-step process. Each time we add a step there will be a whole new set of problems to solve."

The project will start simply, with the simulation of two nonspinning black holes colliding head-on. Next, spins aligned along the collision axis will be modeled. Finally, the simulation will include the black holes spinning and revolving around each other and colliding.

Principal investigator is Richard Matzner of the Center for Relativity at The University of Texas at Austin. Other co- principal investigators are James Browne (University of Texas, Austin); Larry Smarr and Ed Seidel (NCSA/UIUC); Paul Saylor and Faisal Saied (UIUC); James York and Charles Evans (University of North Carolina, Chapel Hill); Stuart Shapiro and Saul Teukolsky (Cornell University); Geoffrey Fox (Syracuse University); Jeffrey Winicour (University of Pittsburgh); Sam Finn (Northwestern University); and Pablo Laguna (Pennsylvania State University). Each group is contributing their own unique talents and insights. In this article we review only one piece of the NCSA/UIUC effort.

Solving Einstein's theory

"We are solving Einstein's equation for the collision of black holes so we can understand LIGO's data and differentiate black hole collisions from other astrophysical events," says Karen Camarda, UIUC graduate research assistant working with Seidel.

Einstein found that gravity is a property of space-time, not just a force. He predicted that gravitational waves would ripple through space in the same way that waves ripple through a pond. His equations of General Relativity govern these time- changing curvatures.

"After almost 80 years, it is time for us to start solving these equations, and the only way we know how is through supercomputers," Seidel says. "The equations are so complicated that is beyond our capability to find a general analytic solution. Einstein sat down with very basic theories and figured it all out--that's the amazing thing."

Opening the door

"There will be much to do," he says. "One thing will be comparing the precise predictions of Einstein's equations to experimental and observational results that will be available during the early part of the next century. Such comparisons should point the way towards future research.

"For example, one may find that experiments do not agree completely with the theory. If the calculations can be trusted, one would have to consider modifying Einstein's theory. So we will be able to verify the theory in precise detail, and perhaps we will find that it is inadequate in some ways-- although so far every indication is that it is the correct theory of gravity."

If Einstein were still alive, Seidel says, "I think he would like it."

Using multigrids

For the most part, scientists will be solving hyperbolic equations, which run efficiently on scalar machines like NCSA's CM-5. Bottlenecks occur when computing the partial differential equations needed for solving elliptical problems.

Typical methods deal with all scales on a fine grid which wastes resources. Multigrid methods cheaply eliminate the high- frequency error components on the fine grid and deal with lower frequency errors on coarser grids where they are more easily handled. Such methods are "optimal order," meaning that the cost of finding an unknown value remains fixed as the problem size increases--with other methods, the cost increases more rapidly.

Refining the method

Scientists say the numerical solution can drift from the true solution due to numerical error. "Multigrid is the engine making the car go down a curved road. We are the navigation constraining the car to the road," says Saylor. He describes the curved road as an elliptical problem and notes that it is his and Saied's task to make the computations run smoothly.

"An important challenge for current research is to come up with effective parallelizations of multigrid methods on distributed memory parallel machines," Saied says. "The difficulties stem from the global data dependencies, and hence global communication requirements, of elliptical problems." Saylor and Saied note that their work is still in the early stages. "We've done 1D equations and are bridging the gap to 3D," Saylor says.

Others working on the project at NCSA include Director Larry Smarr, co-principal investigator; postdoctoral research associates Peter Anninos and Joan Masso; graduate research assistants Steve Brandt, Joe Libson, Peter Leppik, Bharat Khardia, and Paul Walker; and research programmer John Towns. This project will exploit scalable computing to develop software tools using NCSA's CM-5 and the Pittsburgh Supercomputing Center's T3D system.

Further information

More information from the NCSA Digital Library data base.