In my last post, we traced the early history of the science of steam engine building as a progression of mysterious fluids. We began with Johann Joachim Becher’s 1667 mysterious phlogiston which was replaced in 1783 by Lavoisier’s mysterious caloric, and we ended with a mention of Rudolph Clausius’ mysterious fluid called entropy in 1850. Today we will carry on with entropy. As an IT professional, you might find all these mysterious fluids rather amusing, but I would like to remind you that you spend all day working with another mysterious fluid we call information. Information flows through our computer systems on a 24x7 basis, and when it stops flowing, we all get into a lot of trouble rather quickly. When I left United Airlines about 5 years ago, we lost about $150/second when www.united.com went down because customers could not book flights. And I bet that some of you IT folks on Wall Street can easily lose $100,000/second without much trouble. So let’s take our mysterious fluids seriously. The people working on steam engines in the 18th and 19th centuries certainly did. By the way, have you ever stopped to wonder what information is? I mean, you work with it all day long – right? Strange as it might sound, we will soon see how the struggles of steam engine designers in the 19th century led to the concept of information in physics, so please be patient and try to continue learning things from our counterparts in the Industrial Revolution.
In 1842, Julius Robert Mayer unknowingly published the first law of thermodynamics in the May issue of Annalen der Chemie und Pharmacie using experimental results done earlier in France. In this paper, Mayer was the first to propose that there was a mechanical equivalent of heat. Imagine two cylinders containing equal amounts of air. One cylinder has a heavy movable piston supported by the pressure of the confined air, while the other cylinder is completely closed-off. Now heat both cylinders. The French found that the cylinder with the movable piston had to be heated more than the closed-off cylinder to raise the temperature of the air in both cylinders by some identical value like 10 0F. Mayer proposed that some of the heat in the cylinder with the movable piston was converted into mechanical work to lift the heavy piston, and that was why it took more heat to raise the temperature of the air in that cylinder by 10 0F. This is what happens in the cylinders of your car when burning gasoline vapors cause the air in the cylinders to expand and push the pistons down during the power stroke. Mayer proposed there was a new mysterious fluid at work that we now call energy, and that it was conserved. That means that energy can change forms, but that it cannot be created nor destroyed. So for the cylinder with the heavy movable piston, chemical energy in coal is released when it is burned and transformed into heat energy; the resulting heat energy then causes the air in the cylinder to expand, which then lifts the heavy piston. The end result is that some of the heat energy is converted into mechanical energy. Now you might think that with Mayer’s findings that, at long last, steam engine designers finally had some idea of what was going on in steam engines! Not quite. Mayer’s idea of a mechanical equivalent of heat was not well received at the time because Mayer was a medical doctor and considered an outsider of little significance by the scientific community of the day. The sudden loss of two of his children in 1848 and the rejection of his ideas by some of the most prestigious physicists of the time led Mayer to an attempted suicide on May 18, 1850, after which Mayer was committed to an insane asylum.
About the same time another outsider, James Prescott Joule, was doing similar experiments. Joule was the manager of a brewery and an amateur scientist on the side. Joule was investigating the possibility of replacing the steam engines in his brewery with the newly invented electric motor. This investigation ended when Joule discovered that it took about five pounds of battery zinc to do the same work as a single pound of coal. But during these experiments, Joule came to the conclusion that there was an equivalence between the heat produced by an electrical current in a wire, like in a toaster, and the work done by an electrical current in a motor. In 1843, he presented a paper to the British Association for the Advancement of Science in which he announced that it took 838 ft-lbs of mechanical work to raise the temperature of a pound of water by 1 0F (1 Btu). In 1845, Joule presented another paper, On the Mechanical Equivalent of Heat, to the same association in which he described his most famous experiment. Joule used a falling weight connected to a paddle-wheel in an insulated bucket of water via a series of ropes and pulleys to stir the water in the bucket. He then measured the temperature rise of the water as the weight, suspended by a rope connected to the paddle-wheel, descended and stirred the water. The temperature rise of the water was then used to calculate the mechanical equivalent of heat that equated BTUs of heat to ft-lbs of mechanical work. But as with Mayer, Joule’s work was entirely ignored by the physicists of the day. You see, Mayer and Joule were both outsiders challenging the accepted caloric theory of heat.
All this changed in 1847 when physicist Hermann Helmholtz published On the Conservation of Force in which he referred to the work of both Mayer and Joule and proposed that heat and mechanical work were both forms of the same conserved force we now call energy. Finally, in 1850, Rudolph Clausius formalized this idea as the first law of thermodynamics in On the Moving Force of Heat and the Laws of Heat which may be Deduced Therefrom.
The modern statement of the first law of thermodynamics reads as:
dU = dQ – dW
This equation simply states that a change in the internal energy (dU) of a closed system, like a steam engine, is equal to the amount of heat flowing into the system (dQ), minus the amount of mechanical work that the system performs (dW). Thus steam engines take in heat energy from burning coal and convert some of the heat energy into mechanical energy, exhausting the rest as waste heat.
Remember how Carnot thought that the efficiency of a steam engine only depended upon the temperature difference between the steam from the boiler and the temperature of the room in which the steam engine was running? Carnot believed that when caloric fell through this temperature difference, it produced useful mechanical work. Clausius reformulated this idea with a new concept he called entropy, described in his second law of thermodynamics. Clausius reasoned that there had to be some difference in the quality of different energies. For example, there is a great deal of heat energy in the air in the room you are currently sitting in. According to the first law of thermodynamics, it would be possible to build an engine that converts the heat energy in the air into useful mechanical energy and exhausts cold air out the back. With such an engine, you could easily build a refrigerator that produced electricity for your home and cooled your food for free at the same time. All you would need to do would be to hook up an electrical generator to the engine, and then run the cold exhaust from the engine into an insulated compartment! Clearly, this is impossible. As Clausius phrased it, "Heat cannot of itself pass from a colder to a hotter body".
The second law of thermodynamics can be expressed in many ways. One of the most useful goes back to Carnot’s original idea of the maximum efficiency of an engine and the temperature differences between the steam and room temperature. It can be framed quantitatively as:
Maximum efficiency of an engine = 1 – Tc/Th
where Tc is the temperature of the cold reservoir into which heat is dissipated and Th is the temperature of the hot reservoir from which heat is obtained. For a steam engine, Th corresponds to the temperature of the steam and Tc to the temperature of the surrounding room, with both temperatures measured in absolute degrees Kelvin. So Carnot’s proposal was correct. As the temperature difference between Tc and Th gets larger, Tc/Th gets smaller and the efficiency of the engine increases and approaches the value “1” or 100%. In this form of the second law, entropy represents the amount of useless energy; energy that cannot be turned into useful mechanical work and remains as useless waste heat.
Later, in 1865, Clausius presented the most infamous version of the second law at the Philosophical Society of Zurich as:
The entropy of the universe tends to a maximum.
What he meant here was that spontaneous changes tend to smooth out differences in temperature, pressure, and density. Hot objects cool off, tires under pressure leak air and the cream in your coffee will stir itself if you are patient enough. A car will spontaneously become a pile of rust, but a pile of rust will never spontaneously become a car. Spontaneous changes cause an increase in entropy and entropy is just a measure of the degree of this smoothing-out process. So as the entropy of the universe constantly increases with each spontaneous change - the universe tends to run downhill with time. Later we will see that entropy is also a measure of the disorder of a system at the molecular level. In a sense, Murphy’s law is just a popular expression of the second law.
The first and second laws of thermodynamics laid the foundation of thermodynamics which describes the bulk properties of matter and energy. The science of thermodynamics finally allowed steam engine designers to understand what was going on in the cylinders of a steam engine by relating the pressures, temperatures, volumes, and energy flows of the steam in the cylinders while the engine was running. This ended the development of steam engines by trial and error, and the technological craft of steam engine building finally matured into a science.
In Softwarephysics, we apply these same basic ideas of thermodynamics to the macroscopic behavior of software at the program level. The macroscopic behavior of a program can be viewed as the functions the program performs, the speed with which those functions are performed, and the stability and reliability of its performance, just as the macroscopic behavior of steam in a steam engine can be defined by pressure, temperature, and volume changes. This is the viewpoint of software from the perspective of IT management and end-users. They don’t really care what is going on inside of software; they are only interested in how software behaves. From this perspective, it is well known that the entropy of software tends to spontaneously increase with time; software tends to run downhill. Whenever we change software, there is a very good chance that things will run amuck; that is why IT has developed such elaborate change management procedures. But even when we don’t change software, we still seem to get into trouble. I began the very first post on softwarephysics with the following paragraph:
Have you ever wondered why your IT job is so difficult? Have you ever noticed that whenever we change software, performance can drastically decline? Have you observed that performance can drastically decline even when we don’t change software; that sometimes applications spontaneously get slow and then spontaneously return to normal response times without any intervention? Have you noticed that 50% of the time we never find a root cause for problems and that we just start bouncing things at random until performance improves? Have you ever wondered why software behaves this way? Is there anything we can do about all this?
My contention is that these observed behaviors of software are due in part to a simulation of the second law of thermodynamics and the natural tendency for the entropy of software to increase with time at the program level. For deeper insights into this phenomenon of software behavior, we will need to drill down to a lower level and turn to another effective theory of physics called statistical mechanics. Statistical mechanics was developed during the last half of the 19th century and allows us to derive the thermodynamic laws of matter and energy, outlined above, by viewing matter as a large collection of molecules in constant random motion.
Comments are welcome at scj333@sbcglobal.net
To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/
Regards,
Steve Johnston
Friday, October 26, 2007
Entropy - the Bane of Programmers
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment