The Paula Gordon Show Logo The Paula Gordon Show
farmer's photo

Doyne Farmer

      . . . has applied nonlinear studies from Los Alamos to Wall Street. One of the early formulators of Chaos Theory, Dr. Farmer earned a B.S. from Stanford and a PhD from the University of California-SantaCruz, both in physics. He was Oppenheimer Fellow at Los Alamos National Labs where he founded the Complex Systems Group. In 1991, with Norman Packard, Dr. Farmer founded Prediction Company, a firm whose business is automatic trading of financial instruments based on time-series based directional forecasting methods. Dr. Farmer is now the McKinsey Professor at the Santa Fe Institute.


      Imagine a physicist who predicts the performance of the world's financial markets. Imagine Doyne Farmer. He has been looking for "signals" within a vast expanse of noise all his life. As a graduate student in physics, he and his friends were captivated by roulette wheel balls. As a scientist at Los Alamos National Lab, he and his colleagues analyzed fluid-flow experiments, ice ages and sunspot cycles. Then Doyne Farmer responded to the challenge, What about finding order in financial markets? Translation: if you can predict the market, you can make money. Lots of it.

      If you responded to this challenge and had invented Chaos Theory, you'd use Chaos Theory to predict the markets, right? Not necessarily. Doyne Farmer (who along with Norman Packard formulated much of Chaos Theory and co-founded Prediction Company) insists that the dominant principle of Prediction Company was good old fashioned trial and error. No Chaos Theory. Since they knew virtually nothing about financial markets going in, Farmer and Packard and their colleagues succeeded the old fashioned way. Hard work. They poured through phenomenal amounts of data, looking for patterns. And they got lucky -- enough.

      Then Doyne Farmer, the entrepreneur, did the unexpected. He left Prediction Company, preferring the pursuit of the theoretical to meeting quarterly earnings. He is now a faculty member at the Santa Fe Institute, where they study complex systems.

      Now that Mr. Farmer is Dr. Farmer again, he declares financial markets a fascinating example of a complex system. Is evolution a vital process in financial markets, playing a role comparable to the role of evolution in biological systems? Maybe. Dr. Farmer is developing a theory in which traders are the individual organisms. Strategies of traders are like the species that distinguish one group from another. These strategies evolve in time. Evolution is modulated by profits in much the same way that survival in a biological ecology is modulated by food. The traders are able to make profits. The strategies that are able to make profits are the strategies that get propagated forward. So, markets are always evolving and changing in time. Maybe.

      What's Doyne Farmer's dream? The dream of every scientist: To solve a little problem so that he can move on to a larger one in the hope that discovering underlying truths. He would like to understand how complex systems work in nature. It's a dream as elusive as the markets themselves. And success is as unpredictable.


Conversation 1

Doyne Farmer distinguishes the way mathematicians and others talk about chaos for Paula Gordon and Bill Russell. He indicates where the deterministic concept of chaos seems to play an important role.

Conversation 2

Distinguishing between the common and technical use of the term chaos, Dr. Farmer describes the utility of the idea in the natural world using examples of how tiny noise fluctuations are amplified. He categorizes the contribution he and his colleagues at the University of California - Santa Cruz made to Chaos Theory, connecting data to theory using analog computers. He describes analog computers and gives examples of how they take advantage of similarities among physical systems. He compares analog and digital computers. He suggests possible reasons for the term ãbutterfly effectä in chaos theory and describes the effect.

Conversation 3

Dr. Farmer points to fundamental limits to prediction, tracing ideas about chaos back to the French mathematician Poincare. Even tiny uncertainties, like those identified in quantum mechanics, get amplified and eventually become important, says Dr. Farmer, who applies that idea to fundamental limits in measuring anything accurately. He explains how this relates to Prediction Company, which he co-founded. Dr. Farmer recounts the curious path his career has followed, from theoretical to applied ideas about how the world works and back again to the theoretical. He describes the immense difficulty in predicting the world's financial markets.

Conversation 4

Dr. Farmer uses the example of a stock as he continues his description of finding the signal in the noise of a financial market. He shows where the neoclassic economists' Efficient Markets theory are temporarily correct (as a first approximation) but on a grander scale, less so. He indicates new ways Prediction Company used to look for patterns (signals). He explains how Prediction Company got better-than-average results, with examples. He reminds us of how hard that was. He remembers when he was and was not surprised. He compares his youthful experiences (motorcycles, physics, predicting the outcome of roulette games) to his adult ones (Los Alamos and Prediction Company).

Conversation 5

In the face of considerable success with Prediction Company, Doyne Farmer is again dealing in the theoretical as a faculty member at the Santa Fe Institute. He describes what motivated this latest move. He describes his continuing fascination with trying to understand why the order and richness of the biological and cultural world. He summarizes his current work with economics. Dr. Farmer applies both Darwin's and Lamarck's ideas to his own. Dr. Farmer explains how he and colleagues from different disciplines learn from each other at the Santa Fe Institute, shaping a new approach to complex questions as well as responding to current academic over-specialization. Physics, Dr. Farmer reminds us, has a long tradition of emigration to other fields. He honors Herbert Spencer's broadest vision.

Conversation 6

Dr. Farmer outlines his current pursuit of a broad understanding of what makes markets tick. He illustrates the common threads for which he is looking among all complex systems. He describes what all scientists seek -- to find the simple problem, solve it and move on to a harder one with the hope of solving truly complex problems.


Doyne Farmer was exceedingly gracious with his time and attention when we visited with him in Santa Fe. It was a great pleasure to be with him, both personally and professionally, and we thank him.

We thank Dr. Yaneer Bar-Yam, President of the New England Complex Systems Institute, for including us in his science distribution list, where, serendipitously, we came across the book recounting Dr. FarmerÔs experience with Prediction Company.

Related Links:
The Santa Fe Institute is home to some of the world's most sophisticated thinking about complex systems.

The Predictors: How a Band of Maverick Physicists Used Chaos Theory to Trade Their Way to a Fortune on Wall Street by Thomas A. Bass recounts the story of Doyne Farmer and Norman Packard's adventures founding and building Prediction Company. The Predictors is published by Henry Holt and Company.

Prediction Company continues to try to stay one step ahead of the financial markets.

Quick buttons

© 2000 The Paula Gordon Show.
All materials contained on this website are
copyrighted by The Paula Gordon Show and may
not be used in any way without the express,
written consent of Paula Gordon.