What is entropy and negentropy?

Norbert Wiener, answered from the texts and cited to the page.

Norbert Wiener answers

Entropy, as the second law of thermodynamics tells us, is nature's statistical tendency toward disorder — the inexorable drift of any isolated system toward disorganization, toward the leveling of differences, toward noise.1 The physicist measures it as a quantity that spontaneously increases; the communications engineer measures its negative, which I call negentropy, as the amount of information a system carries.

The connection between the two is not metaphorical but precise. Just as the amount of information in a system is a measure of its degree of organization, so the entropy of a system is a measure of its degree of disorganization; and the one is simply the negative of the other.2 A message, in this framework, is organized pattern set against the background of possible disorder; noise is the entropic tendency reasserting itself against that pattern.

When noise increases in intensity, the information a message carries approaches zero — the organized signal is swallowed by the surrounding chaos.3 Living organisms are the most striking natural exemplars of negentropy at work. We are not isolated systems; we take in food and information from the outside and use both to maintain, for a time, a local enclave of organization against the general stream of increasing entropy.4

The process by which we resist that stream is homeostasis — the ensemble of negative feedback mechanisms that keep our temperature, our blood chemistry, our hormonal balance within the narrow ranges that life requires.5 Organism is opposed to chaos as message is to noise; to describe an organism is not to catalogue its molecules but to characterize its pattern, which is more significant precisely because it is less probable.6

Maxwell's demon is the thought-experiment that makes the relation vivid. A small being stationed at a door between two chambers of gas, opening the door selectively for fast molecules in one direction and slow in the other, appears to reverse the entropic tendency — to create a temperature difference from uniformity, to run the engine of organization on nothing but information.7

The demon does not violate the conservation of energy; it appears to violate the second law by using knowledge to sort what chance had scrambled. This is precisely the point: information is the anti-entropic resource, and the organism, like the demon, is a system that uses information received through its sense organs to act against the drift toward disorder.8

No operation on a message can gain information on the average — that is the communication-theoretic form of the second law.9 The processes that lose information are closely analogous to the processes that gain entropy: the fusion of regions of probability that were originally distinct, the replacement of a precise distribution by a coarser one.

Entropy and lost information are the same tendency, seen from the physicist's side and the communications engineer's side respectively.

Sources

  1. nature's statistical tendency to disorder, the tendency for entropy to increase in isolated systems, is expressed by the second law of thermodynamics.
    The Human Use of Human Beings, pp. 30–31
  2. Just as the amount of information in a system is a measure of its degree of organization, so the entropy of a system is a measure of its degree of disorganization; and the one is simply the negative of the other.
    Cybernetics, pp. 30–31
  3. In the presence of a noise, however, this amount of information is finite, and it approaches 0 very rapidly as the noise increases in intensity.
    Cybernetics, pp. 83–84
  4. We, as human beings, are not isolated systems. We take in food, which generates energy, from the outside, and are, as a result, parts of that larger world which contains those sources of our vitality.
    The Human Use of Human Beings, pp. 30–31
  5. These mechanisms constitute what is known as homeostasis, and are negative feedback mechanisms of a type that we may find exemplified in mechanical automata.
    The Human Use of Human Beings, pp. 97–98
  6. Organism is opposed to chaos, to disintegration, to death, as message is to noise. To describe an organism, we do not try to specify each molecule in it, and catalogue it bit by bit, but rather to answer certain questions about it which reveal its pattern: a pattern which is more significant and less probable as the organism becomes, so to speak, more fully an organism.
    The Human Use of Human Beings, pp. 97–98
  7. The demon at the first door opens it only for highspeed molecules and closes it in the face of low-speed molecules coming from the container... The result is that the temperature goes up at one end and down at the other thus creating a perpetual motion of 'the second kind'.
    The Human Use of Human Beings, pp. 30–31
  8. even more important is the fact that we take in information through our sense organs, and we act on information received.
    The Human Use of Human Beings, pp. 30–31
  9. No operation on a message can gain information on the average. Here we have a precise application of the second law of thermodynamics in communication engineering.
    Cybernetics, pp. 83–84