The unthinkable has become a reality. A nuclear strike has hit the U.S., and a retaliatory long-range ballistic missile loaded with a nuclear warhead streaks toward its target. As it begins reentry an enemy defensive missile explodes nearby creating extremely high temperatures and radiation aimed at disabling the incoming weapon.
Should this doomsday scenario ever occur, the U.S. must ensure that its nuclear weapons arsenal will be able to withstand such defensive missile blasts and successfully strike their targets.
But how to do so?
Until U.S. ratification of the Nuclear Non-Proliferation treaty in 1970 nuclear tests could be conducted to confirm the viability of nuclear weapons, but the treaty put a halt to such tests.
Prior to 2005, the job of creating a hostile neutron environment similar to what a nuclear weapon system might encounter was given to DOE’s Sandia Pulsed Reactor (SPR), housed at Sandia National Laboratory, Albuquerque, New Mexico. When researchers wanted to determine if the electrical components of a nuclear weapon system could withstand being bombarded by a defensive missile explosion they placed the components in the SPR, hit them with short-pulsed neutrons and gamma rays and then examined their condition.
But then came 9/11 and everything changed.
There was concern that if the nuclear material used in the reactor fell into terrorist hands it could be used to produce “dirty bombs,” or other kinds of nuclear weapons. To eliminate even the remotest possibility of that occurring, the pulse reactor was deactivated. That left those within DOE who are responsible for maintaining the viability of the nation’s nuclear weapons stockpile without an important testing tool.
Compounding the situation is a freeze on the building of new nuclear weapons that is also part of the Nuclear Non-Proliferation treaty. As a result, the U.S. must rely on its existing nuclear weapons stockpile, which is not immune to the effects of aging, and must, therefore, be periodically refurbished. And, “If you refurbish the old systems you have to requalify them,” said Robert Hoekstra, who heads the 13-person Electrical and Microsystems Modeling group at Sandia that has taken on the task of ensuring that the new electrical components used in the refurbishing process can survive extreme environments.
Their goal is to use DOE’s high-powered computing capabilities to model the effects of hostile environments on the electrical devices used in nuclear weapons as a substitute for testing in the SPR. In anticipation of the possible eventual need for such computational modeling, Hoekstra’s department began creating the necessary computer software some two years prior to 9/11. As part of this effort, Hoekstra’s team has been working in conjunction with other groups within Sandia that are providing data necessary to do the modeling, including codes representing material properties and radiation sources. Due to the complexity of the task, and the crucial need for proven accuracy, the development efforts are ongoing, with recent test results that have been promising.
“We had the first prototype demonstration earlier this year and that was very successful,” said Hoekstra. “It was a big milestone toward proving that we can use this modeling for new systems qualification.” But the actual use of this software to qualify new nuclear weapon components is still several years away.
The project designed to accomplish this monumental task is called Qualification Alternative to the Sandia Pulsed Reactor — the acronym being QASPR (pronounced Casper). QASPR consists of several computer codes, with two of the primary codes being developed by Hoekstra’s group. They are known as Xyce (pronounced Zeiss), and Charon, the more powerful of the two.
The River Styx
Because of its ability to solve a broad range of transport problems, including semiconductor physics, Hoekstra’s team named the program Charon, after the Greek mythological boatman who transported the dead across the river Styx.
The name Xyce is a variant of SPICE (Simulation Program with Integrated Circuit Emphasis), a widely used circuit simulation program first developed at the University of California-Berkeley, around 1970. Today there are numerous variants of SPICE that are used commercially in the electronics industry, but, says Hoekstra, none is as powerful as Sandia’s Xyce.
The prototype of Xyce was written some nine years ago by Eric Keiter at Sandia shortly before Hoekstra joined the department. He and Scott Hutchison then worked with Keiter to create the first production version. The team is now working with Xyce version 4.1.
A high-fidelity code in its own right, Xyce can take a computerized snapshot of the electronic forest by modeling the logic and timing of multiple circuits, while Charon can bore down into the individual trees by examining the workings of a single device to the point that it can model the movement of electrons inside a semiconductor material.
“For a single device there may be as many as millions of finite elements to model, so to explore the physics of the materials, to learn how the device changes if it is hit with radiation or some other effect, requires extremely high-fidelity, which is what Charon provides,” explains Hoekstra.
With it “we develop a detailed physics understanding of what the radiation effects are,” he said. “We need to go to a lower-fidelity tool when we want to simulate the full electrical system, because we can’t simulate every device in the electrical circuit at the Charon fidelity: it wouldn’t fit on the largest computers. So we take what we learned with the higher-fidelity Charon and use that to inform the lower-fidelity Xyce, which simulates the full system.”
Going Commercial
While addressing DOE’s nuclear weapons verification needs, Hoekstra’s group is also working with commercial vendors to make Xyce technology available to them. “It will give them the ability to do very large-scale simulation of circuits which is needed as integrated circuits get bigger and bigger,” he says. The companies involved are major electrical design and automation tool companies that develop software and manufacturing processes for integrated circuits. They in turn supply semiconductor manufacturers.
“There are two features of our tools that differentiate them from the commercial tools,” said Hoekstra. “One is that we can go to greater fidelity physics in the models. What goes in hand with that is our ability to scale up to the really large scale computing platforms of ASC (Advanced Simulation and Computing). Our code is the only version that is capable of massively parallel execution, which means we can run dramatically larger problems,” he said.
What makes that possible is the supercomputing power made available through ASC, a DOE program created in 1995, aimed at developing supercomputer capability to simulate the performance, safety and reliability of nuclear weapons and to certify their functionality. It involves the collaboration of three DOE laboratories: Sandia, Los Alamos, and Lawrence Livermore, in conjunction with numerous university researchers.
For the higher-fidelity Charon code Hoekstra’s team has run large-scale calculations using Livermore’s Purple, an ASC supercomputer made up of 12,544 processors. Utilizing over 8,000 of these processors, it takes some two weeks for Charon to model 250 million variables. So far the group has used Purple for approximately 12 weeks during the past year. Because Xyce, powerful as it is, does not require such massive computing capacity, it is being run on a few hundred processors at Sandia on high-capacity computing Linux clusters, which themselves can consist of thousands of processors.
Until recently the suite of codes involved in the complex QASPR process were referred to individually by names such as NuGET, Cascade, and GRASP. Xyce and Charon are at the end of this chain of codes, and actually model the responses of electronic devices to hostile environments. To more tightly integrate the codes for effectiveness and efficiency they have now been coupled under a single umbrella term standing for Radiation Analysis Modeling and Simulation for Electrical Systems, or RAMSES.
Pushing the Limits
When QASPR will be ready for “real world” electronic component qualification is in large measure dictated by the needs of the Electrical and Microsystems Modeling group’s customers — namely the military, who specify the requirements for refurbished nuclear weapons systems, and the DOE scientists and engineers who design the new components. Hoekstra expects that the software to model the devices’ durability will be ready when the components themselves are ready for testing.
In the meantime, much needs to be done. “We have to improve our understanding of the physics and our ability to model that physics,” said Hoekstra. For example, he noted, “The radiation effects are only partially understood, so we are trying to increase our knowledge of those effects and get that knowledge into the models. We are doing tightly coupled experiments and modeling to understand the physics.”
Beyond that there is the issue of scale. “We are really pushing the limits of computing to model these very high-fidelity models. We need to improve our models and increase our computing horsepower,” he said.
Another challenge is to convince those responsible for nuclear weapons qualification that they can rely on the QASPR methodology to accurately depict the response of electronic components to hostile environments. Doing so requires a two-step process known as verification and validation. Verification is more mathematical than empirical in that it verifies that the computer model gives the answer it is supposed to give in mathematical terms.
Validation answers the question, “Do the results represent reality?” To validate RAMSES results, physical tests are run on electronic components in radiation facilities other than the now defunct Sandia Pulsed Reactor. Though the conditions they create are not as close to an actual hostile environment as was achievable with the SPR, “We still have facilities that give us gamma and neutron irradiation. We put the devices and circuits into those test facilities and use the results to validate our models,” says Hoekstra.
Two DOE Computational Science Graduate Fellows (CSGF) have been involved in the development of the suite of codes used by the QASPR group. Judith Hill, who did her DOE CSGF practicum at Sandia in 2000, worked on Charon when she returned to Sandia as a postdoctoral fellow. Similarly, David Ropp, who did his DOE CSGF practicum at Los Alamos in 1993, contributed to the ASC program while a postdoctoral fellow at Sandia from 2000 to 2004. During that period Ropp worked on Trilinos, a package of algorithms designed to run on the large-scale ASC computers such as Purple.
For its work, the Xyce team recently won a prestigious 2008 R&D 100 Award, given by R&D magazine. The magazines’ Web site states that the “Award provides a mark of excellence known to industry, government and academia as proof that the product is one of the most innovative ideas of the year.”
This article originally appeared in DEIXIS: The CSGF Annual, 2008-09.