The Information by Gleick is a long and winding tale in which the author has focused upon the work of Claude Shannon as exemplar. Gleick is a superb writer who has the ability to engage the reader in his evolving tale about information, its meaning, its use and its transformation. There are however a few problems in this book, style not being one of them. What is more intriguing is why these problems arose. It is most likely that it was not Gleick, who as a good recorder of what he has been told, after all his is no original thinker in this field by any stretch of the imagination. He must have run across some “information machine” programmed to render into the historical record some views which contradict the reality.
The subtext here is the almost angelic and prophetic Claude Shannon and the misanthropic Norbert Wiener. Hero versus arrogant non-contributing interloper, that seems to be the theme. One wonders who told him this tale. Why pick on Wiener as he did and by so doing distort history, and in fact due potential harm to future generations who may read this and be motivated accordingly? Whenever I see such an ad hominem attack in a document I am always suspicious as to agenda and in this case due to my proximity to Wiener and truly wonder. It seems clear to me that perhaps Gleick may not have the technical competence nor the necessary mathematical intellect to have understood Wiener.
Now before I continue some short bona fides. I did my doctoral work at MIT in the area with Shannon and while he was still there in the mid-1960s. My doctoral pedigree is below going back to Gauss, so perhaps there is some credibility there. I spent several tours at Bell Labs, thus knowing its inner workings fairly well and when at NYNEX, now Verizon, I was head of R&D for a period, and was responsible as Chairman of the Research Committee of Bellcore for its Research efforts. Bellcore was the telephone side of Bell Labs after the split. In addition I knew Wiener, albeit at a distance, and my first book in 1969 built upon his work directly. I taught Information theory at MIT for a while and was thus in close proximity to the main players, although I was Wienerian rather than Shannonian.
Before continuing I think a brief not of how people are divided in mind sets is worthwhile. I spent time in France and there is a feeling that every Frenchman, or French woman equally as well, is either a Cartesian or Pascallian. They either start with doubting everything or have a strong set of initial beliefs. Either way they view the world in a dramatically different manner. In Gleick’s book he is a strong Shannonian. That seems less by intellectual or visceral choice but due to some pedagogical inculcation by some person hidden in the shadows of his book.
The author starts at the very beginning beatifying Shannon. Then he excuses in him the inability to solve certain problems, as on p 187, the fire control problem, which was actually being solved by those making electromechanical fire control systems. During WW II the fire control batteries were all a collection of electromechanical computers, brilliant devices made of gears and small synchro motors, all wired to model the tracking of dynamically moving aircraft. We depict such a device on my father’s ship, the DD-649 in the fall of 1944. This was coupled into the radar system and the 5” guns on deck (see my book on the DD-649 which discusses this in detail.
Thus Gleick’s comment regarding Shannon finding the problem too hard with the nonlinear equations was already being solved elsewhere. We will also discuss the solving by Wiener as well.
As to Shannon’s work it is without a doubt one of enduring prominence. The problem he addressed was simple, mainly because he phrased it in a manner in which it became simple. Namely if you were to send a message from point A to point B and I had a transmitter with a fixed level of power and a channel which added so much noise, what is the maximum rate, whatever that means, that I could ever achieve using the best possible technology, using that channel.
Namely given a signal level S and a noise level N, what was the best I could do. It was the establishment of a goal. To solve that problem, to answer the question, he looked at the world in simple terms, say as a telegrapher, but used 0s and 1s, not dots and dashes. For any message he believed could be transformed into such a set of 0s and 1s. Thus the bit. And the theorem said that for a certain S and certain N that the best you could do were some maximum rate R in bits per second per unit bandwidth. That means bits per second per Hertz. We now had a goal. He never said how to get there only that there was what there was! Brilliant. This was the Channel Coding Theorem.
He also posed in the same paper the Source Coding Theorem which became a basis for coding theory. Namely that give a channel with a signal and noise you could find a code which took the message and using some form of redundancy be able to achieve great accuracy.
This theorem drove many to find these codes. For example in cellular phones when I started deploying them twenty five years ago we had about a bit per second per hertz. Now with 4G LTE we have 7 or 8 bits per second per Hertz, which means we can carry 8 times more bits in the same bandwidth. The problem is we have now approached the Shannon Limit!
BSC, which said that if you sent a 1 then with probability p you received a 1 and with probability q=1-p you made an error. Using modulators, demodulators and knowing signal and noise levels one could somewhat readily calculate p. Thus the simple paradigm hid great complexities but it left one with a view of completeness.
Now the irony was that the details on finding the p and q was what Wiener and his students worked out. Depending on the signal and the type of noise on the channel one needed to get optimal transmitters and receivers. It was the Wiener-Hopf filters and receivers which allowed that. Gleick seems to be oblivious of that fact. One must see that there was a flow back and forth between many people at this time.
On p 222 Gleick shows the classic Shannon channel model. It is a shame he did not show the BSC model as well. The confusion is clear however in this model. Shannon used signal as the output of the transmitter, the modulator as it was later called, modem. Thus he was stuck with “signal” at that point and he then used information as the term which led to a signal. There is a difference. The signal can be an encoded concatenation of the information stream. Ironically the information stream may be some correlated set of bits, but frankly what it may carry in human meaning is irrelevant. Shannon did not are whether it was a mother’s day call or the output of an EKG.
Now Gleick on p 233 comments on a review by Doob. Doob was a brilliant mathematician at the University of Illinois who published in 1953 one of the best books on stochastic processes. He was like Wiener a mathematician. Probability had become a branch of pure mathematics separate from say statistics. Doob as Gleick states says that the work is suggestive.
Yet that was the brilliance of the Shannon piece. It was NOT a mathematical work, it was targeted at engineers, and in the late 1940s many engineers could grasp what Shannon wrote about. It did not contain the depth required of an academic journal. It was suggestive, and simplistic, but elegant. It taught the reader and it brought the reader along, never letting them get discouraged. The writing style above all else was its strongest point. The insight it has was of course without par for that field but it was deliberately targeted at engineers and not mathematicians. That also was a source of irritation.
On p 235 we start with Gleick’s attack on Wiener. He uses Pierce, one of the Bell Labs gang. Now I knew Pierce, took a semester seminar from him in late 1960s at MIT. His claim to fame was the traveling wave tube analysis, the TWT being a microwave amplifier. Now Bell Labs during this period was somewhat of an interesting place.
Since 1913 with the Kingsborough decision by the Justice Department giving AT&T a monopoly, the management and most members of AT&T fought tooth and nail to not only keep the monopoly but to expand it in as many ways as possible. Oftentimes to the detriment of technology. Pierce in my opinion was at the forefront of that efforts, namely keeping Bell Labs out in front, and ensuring that it got the fullest share of credit. Thus one often sees that Bell Labs would go to extremes to take work from MIT and other places to gain control.
There was a brutal fight over the Radiation Lab at MIT, which Bush established over the loud objections of AT&T. They wanted all electronics War work done at Bell Labs. The Government did not bend, Bush gave Bell Labs a piece but only a piece. Thus in short order the ship, aircraft and land radars came out of MIT Rad Lab to the chagrin of Bell Labs. Wiener had been a part of that battle and one wonders what the purpose of Pierce’s remark is. Gleick seems not to know of that issue.
If Gleick had studied Wiener a bit more before attacking him perhaps he would have understood this issue better. Pity. If only Gleick had read Masani’s fantastic book on Wiener.
On p 237 Gleick details some of the background on Wiener. One of the statements was that Wiener’s work was too ambitious for the real world. If he had read the Rad Lab book by Nichols and Phillips he would have seen that it was incorporated in the designs! In fact I used the same approach when I designed the Apollo start tracker navigation system actually using Nichols and Phillips and Wiener! It really helps at times to know something.
On p 239 he states that Cybernetics was Wiener’s first book. In fact he had several mathematical treatises before. Perhaps Gleick means his first popular book. On p 240 he states that the book was abstruse and ungainly. Frankly when I read it in the mid-1960s it was smooth as butter. Much of it is mathematical but that is what it was supposed to be. “ungainly” is a rather strange phrase. The author provides little if any basis for it. Perhaps Gleick cannot understand it but perhaps it was not meant for him.
Now on p 263 Gleick states that cybernetics faded. Indeed it changed, it morphed, it took upon itself the complexities that Wiener had set forth. O the same page Gleick makes a rather confusing remark. He states that what McLuhan called the medium Shannon called the channel. Wrong! As Peter Drucker said on seeing McLuhan present his work:
"Did I hear you right," asked one of the professors in the audience, "that you think that printing influenced the courses that the university taught and the role of university all together." "No sir," said McLuhan, "it did not influence; printing determined both, indeed printing determined what henceforth was going to be considered knowledge."
What Drucker implies is that the medium in which we receive the presentation determines what is knowledge, and in effect truth. Thus we Homeric verse, spoken, we have printed words, we have television, we have Internet presentations. The medium, the holistic process of communicating, the totality not just the channel, is the medium, and it determines what “henceforth was going to be considered knowledge”. Thus in our world the use of the Internet as an overlay on books creates a new medium and thus established a new knowledge. This very Internet based review, picked up by hundreds, becomes in a small way knowledge.
Now in Chapter 10 with the genetic code Gleick has apparently not kept up with the current. Regrettably by the time he does it may have moved further on. Let us take cancer. Cancer is a random dynamic process based upon a distributed random state machine with sophisticated feedback in cells communicating with each other. For example, prostate cancer is a pathway disease. The figure below is a current understanding of some of these issues.
It is not a single static code. It is the result of interactions of genes on RNA to proteins and the proteins then interfering with the DNA and its transcription. It is not a one way process as Gleick seems to allude to on p 299. Rather it is a Wienerian cybernetic machine, and one which communicates across many cells in a manner discussed by Alan Turing in his last paper (see my paper on the Turing issue and my book on Cancer Pathways).
Thus the reality is that Shannon made tremendous contributions. Information Theory became a branch of many fields and it continues to thrive in various niches of academia. In contrast Wiener viewed life as complex and messy, a concatenation of signals, noise, feedback and nonlinearities. In fact that is just what we must deal with. The division into a world of Shannonians and Wienerarians is a division into a world which is highly focused in solving the problem of getting more out of a channel and the world which became expansive and trying to understand a control our world fraught with random processes and nonlinearities.
The work Wiener did on the first electronic arm, the Boston Arm, was a simple example of that step. He conceived of and pushed through in his own way that implementation of his cybernetic view. Did Wiener have personality issues? He was an MIT Professor, all one need do is walk down the halls. It is not Harvard folks! But that is the price one pays for brilliance I guess. Wiener was also a product of the old world way of doing things. He was a prodigy, and he recognized that being one was less of a sign of exceptionalism and more a burden to have borne as a child lacking a child hood. Gleick does Wiener a disservice.
Wiener made a tremendous set of contributions to the world as we know it now. The Wiener process, white noise, the Ergodic Theorem, namely the time average is equal to the ensemble average, feedback control with noise, optimal signal processing using matched filters. He did this all without the PR machine of a Bell Labs. On the other hand Bell Labs has a few stumbles. The fact that the CEO of ATT had to go to Canada to get a digital switch, the fact that ARPA had to independently develop what is now the Internet, the fact that almost all digital wireless standards were developed extra Bell Labs.
He was a brilliant man and his ideas permeated many areas. One need think only of the progression from Wiener to Kolmolgorov to Kalman. The development of nonlinear dynamic optimal estimation and control. This is at the heart of all of our current control systems. The beginning, Wiener.
Frankly I had hope Gleick would have done better. Shannon and Wiener were two sides of the same coin. They each contributed in their own way. I like Wiener, much of my thinking has been molded by his way in thinking. In fact if you want a brief on him, read my Squirrel Tale on Norbert.
Posts
