One of the Dumbest Things I have Ever Read

Let me preface my remarks with a few points to establish my bona fides. First I knew, albeit as a student, Shannon and Wiener at MIT. Second, I taught Information Theory at MIT. Third my first book, Stochastic Systems and State Estimation written in the summer of 1969 was a direct result of the influence of Wiener. Fourth, I really like Wiener as one could surmise from one of my Squirrel Tales.

First, this is NOT a review of Gleick's, The Information, which I have yet to read. It is a review of the review in the NY Times. Second this is NOT in any way a critique of what Shannon did. His was in many ways a singular accomplishment.

The strength of what Shannon did was two fold. First, its very simplicity. It was a simple and clearly written paper which was elegant in the way he approached what was then a highly complex problem. I remember reading it in 1963 or 1964. It was like reading a daily news article, it flowed, it enlightened, and it explained. That in many ways was its strongest contribution since it was readily accessible to so many. Second was the elegance of his proof, showing bounds not necessarily specific answers, but showing what could be done. He did this using simple and well understood concepts. Thus the simplicity, elegance, and the ability to set achievable goals was the essence of his contributions. Shannon then had many brilliant academics follow upon his ideas, people like Fano, Elias, Gallager. The work of those people led to 4G wireless and LTE and the like. Built of course on silicon.

Now to the review in the NY Times. I suspect the reviewer is clueless as regards to facts based on his comments. For he states:

But unlike the equally voguish discipline of cybernetics proposed that same year by Norbert Weiner, which left little behind it but a useful prefix, information theory wound up reshaping fields from economics to philosophy, and heralded a dramatic rethinking of biology and physics. 

First, at least spell Wiener's name correctly! When you say Wiener you pronounce the second vowel, unless of course you do not want to be called what you may be known for as I suspect some politicians have done. But secondly it would have been nice to have to have a reviewer who knew and understood a modicum of Wiener, his work and contributions. I guess the reviewer just wanted to sound smart. Too bad he fell short of that.

You see, Wiener was a real mathematician. He did real mathematics and was well known for that. His view of the world, with feedback, is the very essence of how we all see the world today. The Russians understood that. Wiener even influenced the Chinese when he taught at Tsing Hua University in the 1930s. Wiener understood the world as uncertain, a world with hidden instabilities, and he did a tremendous amount to influence future generations as regards to a holistic view of nature. If only the banking world understood Wiener they would have been more concerned as to the instabilities hidden in its bowels.

It was Wiener who did the reshaping not Shannon. Shannon was highly gifted but he was focused on bits and their transmission. His paper is brilliant for its simple insight. The paper is truly an example of speaking to the masses. Wiener had trouble doing that. He was thinking too much. It is hard to read Wiener at time because behind his persona is the mind of a pure mathematician.

The world of Wiener is the world of cellular pathways, stochastic feedback loops which control cell growth. Shannon with all due respect is no where to be found, except perhaps in his doctoral thesis on genetics which frankly falls far from the mark.

Shannon was also part of the Bell Labs/ATT Public Relations machine. Unlike MIT, ATT used the press and managed to glorify its accomplishments and hide its disasters. Yet today Bell Labs Holmdel is an empty facade, Murray Hill, just down the road from me, is Alcatel, filled with more accountants than engineers or scientists. In contrast, it was Wiener who invented the first bio feedback arm to replace a lost limb using the very theories that the author seems to dismiss, and it was Wiener who developed the radar tracking systems which shot down enemy aircraft in WW II from Navy ships thus saving my father's life, and it was Wiener who inspired my comments to Professors of Finance that they had better watch out for those instabilities, and it was Wiener who discovered the very feedback systems in our neural networks which are now applied to modeling and controlling intracellular pathways in cancer!.

"Left little behind...", well to the author of the review you Sir are in my opinion totally clueless. It would help Sir to know something of Wiener. He did not have the PR machine of a Shannon or a Shockley.  In fact they have equal standing. But alas, this is from a faculty member in Information, most likely called Library Science a decade or so ago, and at Berkeley. Thus why should I even wonder!

One further nit. The reviewer states:

In the 1950s, Francis Crick, the co-­discoverer of the structure of DNA, was still putting “information” in quotation marks when describing how one protein copied a sequence of nucleic acids from another. But molecular biologists were soon speaking of information, not to mention codes, libraries, alphabets and transcription, without any sense of metaphor. In Gleick’s words, “Genes themselves are made of bits.”

The view of genes as bits you miss the point. Genes are stochastic dynamic variables in complex time varying state machines. The interact internally and externally. It was Alan Turing in his last paper describing the pattern generation in Zebras that established the essence of this complex automaton. This was a Wiener world view not a Shannon world view.  This in no way delimits what Shannon did, his result was elegant, it was simple, and it was useful. Wiener's result was hard, complex, and reflective of nature.

The more I read this review the more disappointed I am in the NY Times for having selected this reviewer. Gleick is an excellent science writer, not that he gets everything correct or understands every nuance, but he strikes a chord. This reviewer in my opinion just was not even in the same galaxy!

In contrast there was a review in the Wall Street Journal. Not a word about Wiener, perhaps the writer just wrote aboiut what he knew, and he apparently knew Shannon. He states:

My two-page profile in Scientific American didn't come close to doing justice to Shannon, who died in 2001. After all, this playful polymath—whose work bridged electrical engineering, mathematics, computer science, physics and even philosophical logic—was among our era's most influential thinkers. His work, especially his 1948 paper "The Mathematical Theory of Communication," helped spawn today's digital devices and communications technologies. Information theory has also inspired a radical new scientific worldview, which proposes that reality is composed not of matter but of bits of information.

In the Shannon paper he introduces the simple model, paradigm in Kuhn's world. It contains:

By a communication system we will mean a system of the type indicated schematically...  It consists of essentially five parts:

1. An information source which produces a message or sequence of messages to be communicated  to the receiving terminal. The message may be of various types: (a) A sequence of letters as in a  telegraph t ) as in radio or telephony; (c) A function oftime and other variables as in black and  white television ..

2. A transmitter which operates on the message in some way to produce a signal suitable for transmission over the channel.  In telephony this operation consists merely of changing sound  pressure  into a proportional electrical current. ....

3. The channel is merely the medium used to transmit the signal from transmitter to receiver. It  may be a pair of wires, a coaxial cable, a band of radio frequencies, a beam of light, etc.

4. The receiver ordinarily performs the inverse operation of that done by the transmitter,
reconstructing the message from the signal.

5. The destination is the person (or thing) for whom the message is intended.

These five simple elements were than built upon. This was part of the Shannon elegance. The simplicity was the binary symmetric channel model, the BSC shown below:

This simple model of how one sends a 0 or a 1 across a highly complex communications channel such as a radio network, suddenly drew apart the curtain for many. It was simple and it allowed great steps to be made by many great minds. All the details were contained in the p, the probability that if you sent a one you got a one. Then he put messages together in his information construct and strings of correlated bits. It all became obvious. That was Shannon. Wiener on the other hand was the deep thinker and it took a lot of hard work to understand much of what he did. The Wiener world looked like:

The Wiener world had feedback, random processes and complex systems. The Wiener world had noise and he worked hard to define it in excruciating mathematical detail, thus the Wiener process. Shannon became the recipient of that detail, the simplicity was for him to build upon. Wiener was struggling with complexity, Shannon had found simplicity.

This WSJ review was balanced and on target, for there was no reason to attack Wiener. In fact he did a reasonably good job describing Shannon.

Finally there are a few good works on Wiener and the best is by one of his associates and a world respected mathematician, Masani. It is worth a read. There is a popular author biography by Conway which gets into some of the psycho babble stuff but it is also useful.