Editor's note (6/1/2011): We are making the text of this July 1985 article freely available for 30 days to coincide with the publication of a paper on entropy and quantum systems by Vlatko Vedral. He authored our June 2011 cover story and blogs about his latest work, which discusses the research featured in this 1985 article.
A computation, whether it is performed by electronic machinery, on an abacus or in a biological system such as the brain, is a physical process. It is subject to the same questions that apply to other physical processes: How much energy must be expended to perform a particular computation? How long must it take? How large must the computing device be? In other words, what are the physical limits of the process of computation?
So far it has been easier to ask these questions than to answer them. To the extent that we have found limits, they are terribly far away from the real limits of modern technology. We cannot profess, therefore, to be guiding the technologist or the engineer. What we are doing is really more fundamental. We are looking for general laws that must govern all information processing, no matter how it is accomplished. Any limits we find must be based solely on fundamental physical principles, not on whatever technology we may currently be using.
There are precedents for this kind of fundamental examination. In the 1940's Claude E. Shannon of the Bell Telephone Laboratories found there are limits on the amount of information that can be transmitted through a noisy channel; these limits apply no matter how the message is encoded into a signal. Shannon's work represents the birth of modern information science. Earlier, in the mid- and late 19th century, physicists attempting to determine the fundamental limits on the efficiency of steam engines had created the science of thermodynamics. In about 1960 one of us (Landauer) and John Swanson at IBM began attempting to apply the same type of analysis to the process of computing. Since the mid-1970's a growing number of other workers at other institutions have entered this field.
In our analysis of the physical limits of computation we use the term "information" in the technical sense of information theory. In this sense information is destroyed whenever two previously distinct situations become indistinguishable. In physical systems without friction, information can never be destroyed; whenever information is destroyed, some amount of energy must be dissipated (converted into heat). As an example, imagine two easily distinguishable physical situations, such as a rubber ball held either one meter or two meters off the ground. If the ball is dropped, it will bounce. If there is no friction and the ball is perfectly elastic, an observer will always be able to tell what state the ball started out in (that is, what its initial height was) because a ball dropped from two meters will bounce higher than a ball dropped from one meter.
If there is friction, however, the ball will dissipate a small amount of energy with each bounce, until it eventually stops bouncing and comes to rest on the ground. It will then be impossible to determine what the ball's initial state was; a ball dropped from two meters will be identical with a ball dropped from one meter. Information will have been lost as a result of energy dissipation.
Here is another example of information destruction: the expression 2 + 2 contains more information than the expression = 4. If all we know is that we have added two numbers to yield 4, then we do not know whether we have added 1 + 3, 2 + 2, 0 + 4 or some other pair of numbers. Since the output is implicit in the input, no computation ever generates information
In fact, computation as it is currently carried out depends on many operations that destroy information. The so-called and gate is a device with two input lines, each of which may be set at 1 or 0, and one output, whose value depends on the value of the inputs. If both inputs are 1, the output will be 1. If one of the inputs is 0 or if both are 0, the output will also be 0. Any time the gate's output is a 0 we lose information, because we do not know which of three possible states the input lines were in (0 and 1, 1 and 0, or 0 and 0). In fact, any logic gate that has more input than output lines inevitably discards information, because we cannot deduce the input from the output. Whenever we use such a "logically irreversible" gate, we dissipate energy into the environment. Erasing a bit of memory, another operation that is frequently used in computing, is also fundamentally dissipative; when we erase a bit, we lose all information about that bit's previous state.
Rights & Permissions
See what we're tweeting about
8 Comments
Add CommentIn fact, a bit of data is not information, not even for logic gates. Data being operated on by processors are merely transient copies of data representing information - no functional information system destroys any required data, no how many times it's 'ANDed'.
Reply | Report Abuse | Link to thisMoreover, in equating information to bits and energy, physicists apparently aren't aware that data stored in most electronic memories ('chips') require continual energy refreshment or their 'information' will simply dissipate!
Physical memory of all types, from processor registers, caches, memory chips, device buffers, magnetic and even optical media is continuously being released and reallocated, requiring that data be moved into to newly assigned physical memory, overlaying any previously stored data. All this is accomplished without any unintentional loss of information, which is managed and interpreted by software, not logic circuits!
Did you ever try to load a current spreadsheet file into an old copy of the program? If so, you should understand the difference between data and information.
When the article states:
"Are irreversible logic gates and erasures essential to computation? If they are, any computation we perform has to dissipate some minimum amount of energy."
…it ignores the physical realities of actual information processes. Just for example, even if some method of instantaneous computation were devised, a computer system based on that method could only operate as fast as data could be delivered for computation and retrieved from it. That and all data would be useless without properly interpretive software to finally represent it as information. Information management typically requires far more processing than computation of data and the enormous migration of data required for processing requires far more time than computations do.
IMO, the quest for instantaneous computing is much more dependent on the energy requirements and performance of information storage, retrieval and routing than logic gates.
The AND operation and any other irreversible operation does destroy information which costs energy. I think the point made in the article was if you could make a computer that doesn't lose information then you can compute without that expense. Logic gates are the basis of computing. Software is merely a convenience.
Reply | Report Abuse | Link to thisThe AND operation has two parameters: a source data field and a destination field. The source field is not altered by the operation. If the destination field is a copy of an original data field, not only is its data not destroyed, but the AND operation produces additional data.
Reply | Report Abuse | Link to thisBy the author's definition of irreversible operations or erasures, only repeated operations on a single set of data could ever be performed, since no new data could ever be brought into storage and no results could be extracted - that would require erasure of previous data.
Logic gates do absolutely nothing unless instructed by software to operate on data provided by software - very convenient indeed!
Three comments:
Reply | Report Abuse | Link to this1. Every page of the original article contains detailed figures that add much to the authors' arguments. If you're going to publish the article (which is appreciated), please include all of it.
2. Would today's Scientific American publish an article of this technical sophistication and length (nine pages)?
3. Check out what SciAm covers looked like back in the 1980's when this was published (including the July 1985 cover):
http://www.coverbrowser.com/covers/scientific-american/42
The covers are beautiful, informative, and depict real subjects of the science journalism contained in that issue.
Compare that to the current cover:
http://www.scientificamerican.com/media/cover/cover_2011-06.jpg
Or these recent gems:
http://www.scientificamerican.com/media/cover/cover_2008-02.jpg
http://www.scientificamerican.com/media/cover/cover_2008-03.jpg
http://www.scientificamerican.com/media/cover/cover_2010-03.jpg
Why do the covers have to be so freaking goofy? This is embarrassing. What happened to this once-great magazine?
Really excellent comment! I dug back into my 'library' and found the original print issue.
Reply | Report Abuse | Link to thisI think a more direct comparison for the cover illustrations of current issues would be OMNI magazine of the 1970s. Unfortunately, much of the content of recent issues is also comparable to OMNI...
I hadn't even realized that the original figures had been omitted - they really do enhance the reader's ability to understand the article.
However, I still object to many of the premises of the article and I think information theory at least as adopted and incorporated into physics.
The article states:
"Here is another example of information destruction: the expression 2 + 2 contains more information than the expression = 4. If all we know is that we have added two numbers to yield 4, then we do not know whether we have added 1 + 3, 2 + 2, 0 + 4 or some other pair of numbers. Since the output is implicit in the input, no computation ever generates information."
Again, a programmer is not at the mercy of logic gates and can maintain the source variables for any 'destructive' operation. In that case the last statement is doubly incorrect: _all_ computations generate additional information and, if source data is maintained no information is destroyed!
Lastly, the description of first illustration on page 49 in the original issue states:
"The "logic gates" central to the design of a chip expend energy because they discard information."
Does it require more energy for a person of a chip to calculate that 2 + 2 = 4 than it does to calculate that 1 + 1 = 2? In the second case the original values can be derived so, according to the article, no information is lost. I guess it's presumed that a person or a chip cannot remember the numbers that produced the sum of 4...
I don't see that any relationship has been established between and data content or information and any energy requirements, except as some obtuse abstract misconception.
I see the problem as thus:
Reply | Report Abuse | Link to thisLet me go with the ball-system you describe. In order to perform a computation, some minimum amount of energy must be put into the system to move the balls forward. After the computation, in order to USE that information (ie the balls go into another logic gate etc.), some amount of that initial energy must be transfered forward into whatever apparatus uses the result of the computation. The measurement of the result creates some minimum amount of entropy that makes it impossible for the system to completely reverse itself. A simple example would be some sort of light sensor at the end of your computation system that detects the presence of the ball. When the photons bounce off of the ball, they impart a certain minimum amount of energy to the ball (possibly negative if it pushes against its direction of motion). The state of the ball therefore becomes, to some extent, uncertain and more entropic. After the ball bounces against some immovable object that reflects 100% of its energy back towards the beginning part of the machine, the ball(s) will arrive back at the starting location. However, the energy that they impart to the 'first mover' that caused the balls initial motion is now uncertain (by whatever uncertainty the measurement created). Further, there is a non-zero probability that, due to the measurement interaction, the balls do not return to their original states.
Although, in theory, a self-contained, reversible computation machine is possible, in the physical universe, to actually observe (and use) the result of the computation, a certain amount of entropy is introduced into the system which makes it impossible for the system to return to its original state. The minimum amount of energy necessary to perform a computation relates to the entropy imparted by the observation of the computation - A strange but thermodynamically sound conclusion.
Adding in a bit of Quantum Mechanics, until we observe the output of the computation, the computation machine state exists in a superposition of all possible states. In otherwords, no computation (in the classical sense) is possible without an observation. Since observation imparts a certain non-zero entropy into the computation machine state, no computation is possible without the expenditure of a certain minimum amount of energy, and no 'computation machine' is fully reversible in the physical universe.
Errata: Page 4, 4th Para. has been truncated.
Reply | Report Abuse | Link to thisYes, you can save any data within any argument that is passed to some function. However, this "saving" of the inputs requires extra resources in the form of an array or data structure *before* those inputs are entered into the function. These extra resources are not abstract, because the execution of the "saving" and the function require the use of entirely separate groups logic gates (or the same gates at different times). The former group duplicates inputs while the latter computes the programmed function. The function execution does in fact destroy the existence of its copy of the inputs. This is just a fact of Turing equivalent discrete state computation at the physical level. Logic gates don't "remember" unless additional logic gates are employed.
Reply | Report Abuse | Link to thisBut it's even worse than that, because the system must expend even *more* physical resources in order to create a pointer between the inputs and the outputs in order to maintain any relevance between these discrete and independent states. The noiseless environment that is the genius of Turing equivalent computation comes at an entropic cost, and that cost is amplified when trying to preserve, and associate, past states with present states.
We forget this today because memory has become so relatively cheap. But in the good ol' days software engineering really was engineering because the ability to execute some code was very constrained by available time and memory.