Editor's Note: Thist story was originally published in the May 1998 edition of Scientific American. We are posting it in light of recent news involving Lehman Brothers and Merrill Lynch.

Months before El Niño– driven storms battered the Pacific Coast of the U.S., the financial world was making its own preparations for aberrant weather. Beginning last year, an investor could buy or sell a contract whose value depended entirely on fluctuations in temperature or accumulations of rain, hail or snow.

These weather derivatives might pay out, for example, if the amount of rainfall at the Los Angeles airport ranged between 17 and 27 inches from October through April. They are a means for an insurer to help provide for future claims by policyholders or a farmer to protect against crop losses. Or the contracts might allow a heating oil supplier to cope with a cash shortfall from a warmer than expected winter by purchasing a heating degree-day floor—a contract that would compensate the company if the temperature failed to fall below 65 degrees as often as expected. “We’re big fans of El Niño because it’s brought us a lot of business,” comments Andrew Freeman, a managing director of Worldwide Weather Trading, a New York City–based firm that writes contracts on rain, snow and temperature.

Weather derivatives mark an example of the growing reach of a discipline called financial engineering. This bailiwick of high-speed computing and the intricate mathematical modeling of mathematicians, physicists and economists can help mitigate the vagaries of running a global business. It entails the custom packaging of securities to provide price insurance against a drop in either the yen or the thermometer. The uncertainties of a market crash or the next monsoon can be priced, divided into marketable chunks and sold to someone who is willing to bear that risk—in exchange for a fee or a future stream of payments. “The technology will effectively allow you to completely manage the risks of an entire organization,” says Robert A. Jarrow, a professor of finance at Cornell University.

The engineering of financial instruments has emerged in response to turbulence during recent decades in ever more interconnected world markets: a result of floating exchange rates, oil crises, interest- rate shocks and stock-market collapses.

The creative unleashing of new products continues with increasingly sophisticated forms of securities and derivatives—options, futures and other contracts derived from an underlying asset, financial index, interest or currency exchange rate. New derivatives will help electric utilities protect against price and capacity swings in newly deregulated markets. Credit derivatives let banks pass off to other parties the risk of default on a loan. Securities that would help a business cope with the year 2000 bug have even been contemplated. This ferment of activity takes place against a tainted background.

The billions of dollars in losses that have accumulated through debacles experienced by the likes of Procter & Gamble, Gibson Greetings and Barings Bank have given derivatives the public image of speculative risk enhancers, not new types of insurance. Concerns have also focused on the integrity of the mathematical modeling techniques that make derivatives trading possible.

Despite the tarnish, financial engineering received a valentine of sorts in October. The Nobel Prize for economics (known formally as the Bank of Sweden Prize in Economic Sciences) went to Myron S. Scholes and Robert C. Merton, two of the creators of the options-pricing model that has helped fuel the explosion of activity in the derivatives markets.

Options represent the right (but not the obligation) to buy or sell stock or some other asset at a given price on or before a certain date. Another major class of derivatives, called forwards and futures, obligates the buyer to purchase an asset at a set price and time. Swaps, yet another type of derivative, allow companies to exchange cash flows—floating-interest- rate for fixed-rate payments, for instance. Financial engineering uses these building blocks to create custom instruments that might provide a retiree with a guaranteed minimum return on an investment or allow a utility to fill its future power demands through contractual arrangements instead of constructing a new plant.

Creating complicated financial instruments requires accurate pricing methods for the derivatives that make up their constituent parts. It is relatively easy to establish the price of a futures contract. When the cost of wheat rises, the price of the futures contract on the commodity increases by the same relative amount. Thus, the relationship is linear. For options, there is no such simple link between the derivative and the underlying asset. For this reason, the work of Scholes, Merton and their deceased colleague Fischer Black has assumed an importance that prompted one economist to describe their endeavors as “the most successful theory not only in finance but in all of economics.”

Einstein and Options
The proper valuation of options had perplexed economists for most of this century. Beginning in 1900 with his groundbreaking essay “The Theory of Speculation,” Louis Bachelier described a means to price options. Remarkably, one component of the formula that he conceived for this purpose anticipated a model that Albert Einstein later used in his theory of Brownian motion, the random movement of particles through fluids. Bachelier’s formula, however, contained financially unrealistic assumptions, such as the existence of negative values for stock prices.

Other academic thinkers, including Nobelist Paul Samuelson, tried to attack the problem. They foundered in the difficult endeavor of calculating a risk premium: a discount from the option price to compensate for the investor’s aversion to risk and the uncertain movement of the stock in the market.

The insight shared by Black, Scholes and Merton was that an estimate of a risk premium was not needed, because it is contained in the quoted stock price, a critical input in the option formula. The market causes the price of a riskier stock to trade further below its expected future value than a more staid equity, and that difference serves as a discount for inherent riskiness.

Black and Scholes, with Merton’s help, came up with their option-pricing formula by constructing a hypothetical portfolio in which a change of price in a stock was canceled by an offsetting change in the value of options on the stock—a strategy called hedging. Here is a simplified example: A put option would give the owner the right to sell a share of a stock in three months if the stock price is at or below $100. The value of the option might increase by 50 cents when the stock goes down $1 (because the condition under which the option can be used has grown more likely) and decrease by 50 cents when the stock goes up by $1.

To hedge against risks in changes in share price, the investor can buy two options for every share he or she owns; the profit then will counter the loss. Hedging creates a risk-free portfolio, one whose return is the same as that of a treasury bill. As the share price changes over time, the investor must alter the composition of the portfolio—the ratio of the number of shares of stocks to the number of options—to ensure that the holdings remain without risk.

The Black-Scholes formula, in fact, is elicited from a partial differential equation demonstrating that the fair price for an option is the one that would bring a risk-free return within such a hedging portfolio. Variations on the hedging strategy outlined by Black, Scholes and Merton have proved invaluable to financial- center banks and a range of other institutions that can use them to protect portfolios against market vagaries—ensuring against a steep decline in stocks, for instance.

The basic options-pricing methodology can also be extended to create other instruments, some of which bear bizarre names like “cliquets” or “shouts.” These colorful financial creatures provide the flexibility to shape the payoffs from the option to a customer’s particular risk profile, placing a floor, ceiling or averaging function on interest or exchange rates, for example.

With the right option, investors can bet or hedge on any kind of uncertainty, from the volatility (up-and-down movement) of the market to the odds of catastrophic weather. An exporter can buy a “look-back” currency option to receive the most favorable dollar-yen exchange rate during a six-month period, rather than being exposed to a sudden change in rates on the date of the contract’s expiration.

In the early 1970s Black and Scholes’s original paper had difficulty finding a publisher. When it did reach the Journal of Political Economy in 1973, its impact on the financial markets was immediate. Within months, their formula was being programmed into calculators. Wall Street loved it, because a trader could solve the equation easily just by punching in a few variables, including stock price, interest rate on treasury bills and the option’s expiration date. The only variable that was not readily obtainable was that for “market volatility”— the standard deviation of stock prices from their mean values. This number, however, could be estimated from the ups and downs of past prices. Similarly, if the current option price was known in the markets, a trader could enter that number into a workstation and “back out” a number for volatility, which can be used to judge whether an option is overpriced or underpriced relative to the current price of the stock in the market.

Investors who buy options are basically purchasing volatility—either to speculate on or to protect against market turbulence. The more ups and downs in the market, the more the option is worth. An investor who speculates with a call—an option to buy a stock—can lose only the cost of purchase, called a premium, if the stock fails to reach the price at which the buyer can exercise the right to purchase it. In contrast, if the stock shoots above the exercise price, the potential for profit is unlimited. Similarly, the investor who hedges with options also anticipates rough times ahead and so may buy protection against a drop in the market.

Physicists on Wall Street
Although it can be reduced to operations on a pocket calculator, the mathematics behind the Black-Scholes equation is stochastic calculus, a descendant from the work of Bachelier and Einstein. These equations were by no means the standard fare in most business administration programs. Enter the Wall Street rocket scientists: the former physicists, mathematicians, computer scientists and econometricians who now play an important role at the Wall Street financial behemoths.

Moving from synchrotrons to trading rooms does not always result in such a seamless transition. “Whenever you hire a physicist, you’re always hoping that he or she doesn’t think of markets as if they were governed by immutable physical laws,” notes Charles Smithson, a managing director at CIBC World Markets, an investment bank. “Uranium 238 always decays to uranium 234. But a physicist must remember that markets can go up as well as go down.”

Recently some universities have opened “quant schools,” programs that educate M.B.A. or other master’s students in the higher applied mathematics of finance, the subtleties of Ito’s lemma and other cornerstones of stochastic calculus. Or else they may train physicists, engineers and mathematicians before moving on to Wall Street. “Market pressures are directing physicists to get more education to try to understand the motivation and intuition underlying financial problems,” says Andrew W. Lo, who heads the track in financial engineering at the Massachusetts Institute of Technology’s Sloan School of Management.

As part of their studies, financial engineers in training learn about the progression of mathematical modeling beyond the original work of Black, Scholes and Merton. The basic Black-Scholes formula made unrealistic assumptions about how the market operates. It takes a fixed interest rate as an input, but of course interest rates change, and that influences the value of an option—particularly an option on a bond. The formula also assumes that changes in the growth rate of stock prices fall into a normal statistical distribution, a bell curve in which events cluster around the mean. Thus, it fails to take into account extraordinary events such as the 1929 or 1987 stock market crashes. Black, Scholes and Merton—and legions of quants—have spent the ensuing years refining many of the original ideas.

Emanuel Derman, head of the quantitative strategies group at Goldman Sachs, is a physicist-turned-quant whose job over the past 13 years has been to tackle the imperfections of the Black- Scholes equation. Derman, a native of Cape Town, South Africa, received his doctorate from Columbia University in 1973 for a thesis on the weak interaction among subatomic particles. He went on to postdoctoral positions, including study of neutrino scattering at the University of Pennsylvania and charmed quark production at the University of Oxford’s department of theoretical physics. In the late 1970s Derman decided to leave academia: “Physics is lonely work. It’s a real meritocracy. In physics, you sometimes feel like you’re either [Richard] Feynman or you’re nobody. I liked physics, but maybe I wasn’t as good as I might have been.”

So in 1980 he went to Bell Laboratories in New Jersey, where he worked on a computer language tailored for finance. In 1985 Goldman Sachs hired him to develop methods of modeling interest rates. He has worked there since, except for a year spent at Salomon Brothers. At Goldman, he met the recently recruited Fischer Black, and the two began working with another colleague, William W. Toy, on a method of valuing bond options. Derman remembers Black as a bluntly truthful man with punctilious writing habits who wore a Casio Data Bank watch. “Black was less powerful mathematically than he was intuitively,” Derman says. “But he always had an idea of what the right answer was.”

Physics Versus Finance
Much of Derman’s recent work on the expected volatility of stock prices continues to refine the original 1973 paper. The Black-Scholes equation was to finance what Newtonian mechanics was to physics, Derman asserts. “Black-Scholes is sort of the foundation on which the field rests. Nobody knows what to do next except extend it.” But the field, he fears, may never succeed in producing its own Einstein—or some unified financial theory of everything. Finance differs from physics in that no mathematical model can capture the multitude of ever mutating economic factors that cause major market perturbations— the recent Asian collapse, for instance. “In physics, you’re playing against God; in finance, you’re playing against people,” Derman declares.

Outside the domain of Wall Street, the parallels between physical concepts and finance are sometimes taken more literally by academics. Kirill Ilinski of the University of Birmingham in England has used Feynman’s theory of quantum electrodynamics to model market dynamics, while employing these concepts to rederive the Black-Scholes equation. Ilinski replaces an electromagnetic field, which controls the interaction of charged particles, with a so-called arbitrage field that can describe changes in option and stock prices. (Trading that brings the value of the stock and the option portfolio into line is called arbitrage.)

Ilinski’s theory shows how quantum electrodynamics can model Black, Scholes and Merton’s hedging strategy, in which market dynamics dictate that any gain in a stock will be offset by the decline in value of the option, thereby yielding a risk-free return. Ilinski equates it with the absorption of “virtual particles,” or photons, that damp the interacting forces between two electrons. He goes on to show how his arbitrage field model elucidates opportunities for profit that were not envisaged by the original Black-Scholes equation.

Ilinski is a member of the nascent field of econophysics, which held its first conference last July in Budapest. Nevertheless, literal parallelism between physics and finance has gained few adherents. “It doesn’t meet the very simple rule of demarcation between science and hogwash,” notes Nassim Taleb, a veteran derivatives trader and a senior adviser to Paribas, the French investment bank. Ilinski recognizes the controversial nature of his labors. “Some people accept my work, and some people say I’m mad. So there’s a discrepancy of opinion,” he says wryly.

Whether invoking Richard Feynman or Fischer Black, the use of mathematical models to value and hedge securities is an exercise in estimation. The term “model risk” describes how different models can produce widely varying prices for a derivative and how these prices create large losses when they differ from the ones at which a financial instrument can be bought or sold in the market.

Model risk comes in many forms. A model’s complexity can lead to erroneous valuations for derivatives. So can inaccurate assumptions underlying the model—failing to take into account the volatility of interest rates during an exchange- rate crisis, for instance. Many models do not cope well with sudden alterations in the relation among market variables, such as a change in the normal trading range between the U.S. dollar and the Indonesian rupiah. “The model or the way you’re using it just doesn’t capture what’s going on anymore,” says Tanya Styblo Beder, a principal in Capital Market Risk Advisors, a New York City firm that evaluates the integrity of models. “Things change. It’s as if you’re driving down a very steep mountain road, and you thought you were gliding on a bicycle, and you find you’re in a tractor-trailer with no brakes.”

Custom-tailored products of financial engineering are not traded on public exchanges and so rely on valuations produced by models, sometimes making it difficult to compare the models’ pricing to other instruments in the marketplace. When it comes time to sell, the market may offer a price that differs significantly from a model’s estimate. In some cases, a trader might capitalize on supposed mispricings in another trader’s model to sell an overvalued option, a practice known as model arbitrage.

“There’s a danger of accepting models without carefully questioning them,” says Joseph A. Langsam, a former mathematician who develops and tests models for fixed-income securities at Morgan Stanley. Morgan Stanley and other firms adopt various means of testing, such as determining how well their models value derivatives for which there is a known price.

Problems related to modeling have accounted for about 20 percent of the $23.77 billion in derivatives losses that have occurred during the past decade, according to Capital Market Risk Advisors. Last year, however, model risk comprised nearly 40 percent of the $2.65 billion in money lost. The tally for 1997 included National Westminster Bank, with $123 million in losses, and Union Bank of Switzerland, with a $240-million hit.

A conference in February sponsored by Derivatives Strategy, an industry trade magazine, held a roundtable discussion called “First Kill All the Models.” Some of the participants questioned whether the most sophisticated mathematical models can match traders’ skill and gut intuition about market dynamics. “As models become more complicated, people will use them, and they’re dangerous in that regard, because they’ll use them in ways that are deleterious to their economic health,” said Stanley R. Jonas, who heads the derivatives trading department for Societé Generale/FIMAT in New York City. An unpublished study by Jens Carsten Jackwerth of the London Business School and Mark E. Rubinstein of the University of California at Berkeley has shown that traders’ own rules of thumb about inferring future stock index volatility did better than many of the major modeling methods.

One modeler at the session—Derman of Goldman Sachs—defended his craft. “To paraphrase Mao in the sixties: Let 1,000 models bloom,” he proclaimed. He compared models to gedanken (thought) experiments, which are unempirical but which help physicists contemplate the world more clearly: “Einstein would think about what it was like to sit on the edge of a wave moving at the speed of light and what he would see. And I think we’re doing something like that. We are sort of investigating imaginary worlds and trying to get some value out of them and see which one best approximates our own.” Derman acknowledged that every model is imperfect: “You need to think about how to account for the mismatch between models and the real world.”

Financial Hydrogen Bombs
The image of derivatives has been sullied by much publicized financial debacles, which include the bankruptcies of Barings Bank and Orange County, California, and huge losses by Procter & Gamble and Gibson Greetings. Investment banker Felix Rohatyn has been quoted as warning about the perils of twentysomething computer whizzes concocting “financial hydrogen bombs.” Some businesses and local governments have excluded derivatives from their portfolios altogether; fears have even emerged about a meltdown of the financial system.

The creators of these newfangled instruments place the losses in broader perspective. The notional, or face, value of all stocks, bonds, currencies and other assets on which options, futures, forwards and swap contracts are derived totaled $56 trillion in 1995, according to the Bank for International Settlements. The market value of the outstanding derivatives contracts themselves represents only a few percentage points of the overall figure but an amount that may still total a few trillion dollars. In contrast, known derivatives losses between 1987 and 1997 totaled only $23.8 billion. More mundane investments can also hurt investors. When interest rates shot up in 1994, the treasury bond markets lost $230 billion.

Derivatives make the news because, like an airplane crash, their losses can prove sudden and dramatic. The contracts can involve enormous leverage. A derivatives investor may put up only a fraction of the value of an underlying asset, such as a stock or a bond. A small percentage change in the value of the asset can produce a large percentage gain or loss in the value of the derivative.

To manage the risks of owning derivatives and other securities, financial houses take refuge in yet other mathematical models. Much of this work is rooted in portfolio theory, a statistical measurement and optimization methodology for which Harry M. Markowitz received the Nobel Prize in 1990. Markowitz elucidated how investors could minimize risk for a given level of return by diversifying into a range of assets that do not all perform the same way as the market changes.

One hand-me-down from Markowitz is called value at risk. It sets forth a set of techniques that elicits a single worst-case number for investment losses. Value at risk calculates the probability of the maximum losses for every existing portfolio, from currency to derivatives. It then elicits a value at risk for the company’s overall financial exposure: the worst hit that can be expected within the next 30 days with a given statistical confidence interval might amount to $85 million. An analysis of the portfolios shows where risks are concentrated. Philippe Jorion, a professor of finance at the University of California at Irvine, has performed a case study that shows how value-at-risk measures could raise warning flags to even unsophisticated investors. Members of the school boards in Orange County that invested in the county fund that lost $1.7 billion might have reacted differently if they knew that there existed a 5 percent chance of a billion-dollar-plus loss.

Like other modeling techniques, value at risk has bred skepticism about how well it predicts ups and downs in the real world. The most widely used measurement techniques rely heavily on historical market data that fail to capture the magnitude of rare but extreme events. “If you take the last year’s worth of data, you may see a portfolio vary by only 10 percent. Then, if you move a month ahead, things may change by 100 percent,” comments Ron S. Dembo, president of Algorithmics, a Torontobased risk-management software company. Algorithmics and other firms go beyond the simplest value-at-risk methods by providing banks with software that can “stress-test” a portfolio by simulating the ramifications of large market swings.

One modeling technique may beget another, and debates over their intrinsic worth will surely continue. But the ability to put a price on uncertainty, the essence of financial engineering, has already proved worthwhile in other business settings as well as in government policymaking and domestic finance. Options theory can aid in steering capital investments. A conventional investment analysis might suggest that it is better for a utility to budget for a large coal-fired plant that can provide capacity for 10 to 15 years of growth. But that approach would sacrifice the alternative of building a series of small oilfired generators, a better choice if demand grows more slowly than expected. Option-pricing techniques can place a value on the flexibility provided by the slow-growth path.

The Black-Scholes model has also been used to quantify the benefits that accrue to a developing nation from providing workers with a general education rather than targeted training in specific skills. It reveals that the value of being able to change labor skills quickly as the economy shifts can exceed the extra cost of supplying a broad-based education. Option pricing can even be used to assess the flexibility of choosing an “out-ofplan” physician for managed health care. “The implications for this aren’t just in the direct financial markets but in being able to use this technology for how we organize nonfinancial firms and how people organize their financial lives in general,” says Nobelist Merton. Placing a value on the vagaries of the future may help realize the vision of another Nobel laureate: Kenneth J. Arrow of Stanford University imagined a security for every condition in the world—and any risk, from bankruptcy to a rained-out picnic, could be shifted to someone else.