Sunday, January 29, 2012

Chaos and machine intelligence

"Intelligent design" creationists such as Robert J. Marks II often claim that only intelligence, whatever that is, creates information. Some people, including me, respond that chaotic systems create information. My perspective is somewhat unusual, however, in that I've focused on chaos in artificial neural networks.

Once upon a time, I'd have assumed that Marks' perspective was similar to my own. He has published books on artificial neural networks and on information theory. He knows that artificial neural nets with feedback loops can behave chaotically, and he should know that there is considerable support for the notion that chaos is essential to information processing in biological brains. Furthermore, he should know that Kolmogorov-Sinai entropy and positive Lyapunov exponents are measures of the rate at which a system creates information as it pursues a chaotic orbit.

Many investigators of machine intelligence believe that a system doesn't deserve to be called "intelligent" unless it is creative in some sense. When we build a chaotic system, and then set it loose to interact with an environment, we cannot predict what it will do over the long term. Although it is our artifact, it acquires a "mind" of its own. And thus we gain information by observing it.

I don't want to give the impression that chaos requires "intelligent design." There are extremely simple dynamical systems that behave chaotically, e.g., the logistic map, $$x_{n+1} = r x_n (1 - x_n).$$ (See the Wikipedia article I've linked to for details.) Stephen Wolfram's program of exploration of cellular automata (see A New Kind of Science) has led to the discovery of many chaotic systems that easily could arise by chance.

The upshot is that it makes sense to say that an "intelligent" system with chaotic components creates information, but not that only intelligence creates information.