Entropy - the Bane of Programmers

In my last post, we traced the early history of the science of steam engine building as a progression of mysterious fluids. We began with Johann Joachim Becher’s 1667 mysterious phlogiston which was replaced in 1783 by Lavoisier’s mysterious caloric, and we ended with a mention of Rudolph Clausius’ mysterious fluid called entropy in 1850. Today we will carry on with entropy. As an IT professional, you might find all these mysterious fluids rather amusing, but I would like to remind you that you spend all day working with another mysterious fluid we call information. Information flows through our computer systems on a 24x7 basis, and when it stops flowing, we all get into a lot of trouble rather quickly. When I left United Airlines about 5 years ago, we lost about $150/second when www.united.com went down because customers could not book flights. And I bet that some of you IT folks on Wall Street can easily lose $100,000/second without much trouble. So let’s take our mysterious fluids seriously. The people working on steam engines in the 18th and 19th centuries certainly did. By the way, have you ever stopped to wonder what information is? I mean, you work with it all day long – right? Strange as it might sound, we will soon see how the struggles of steam engine designers in the 19th century led to the concept of information in physics, so please be patient and try to continue learning things from our counterparts in the Industrial Revolution.

In 1842, Julius Robert Mayer unknowingly published the first law of thermodynamics in the May issue of Annalen der Chemie und Pharmacie using experimental results done earlier in France. In this paper, Mayer was the first to propose that there was a mechanical equivalent of heat. Imagine two cylinders containing equal amounts of air. One cylinder has a heavy movable piston supported by the pressure of the confined air, while the other cylinder is completely closed-off. Now heat both cylinders. The French found that the cylinder with the movable piston had to be heated more than the closed-off cylinder to raise the temperature of the air in both cylinders by some identical value like 10 ⁰F. Mayer proposed that some of the heat in the cylinder with the movable piston was converted into mechanical work to lift the heavy piston, and that was why it took more heat to raise the temperature of the air in that cylinder by 10 ⁰F. This is what happens in the cylinders of your car when burning gasoline vapors cause the air in the cylinders to expand and push the pistons down during the power stroke. Mayer proposed there was a new mysterious fluid at work that we now call energy, and that it was conserved. That means that energy can change forms, but that it cannot be created nor destroyed. So for the cylinder with the heavy movable piston, chemical energy in coal is released when it is burned and transformed into heat energy; the resulting heat energy then causes the air in the cylinder to expand, which then lifts the heavy piston. The end result is that some of the heat energy is converted into mechanical energy. Now you might think that with Mayer’s findings that, at long last, steam engine designers finally had some idea of what was going on in steam engines! Not quite. Mayer’s idea of a mechanical equivalent of heat was not well received at the time because Mayer was a medical doctor and considered an outsider of little significance by the scientific community of the day. The sudden loss of two of his children in 1848 and the rejection of his ideas by some of the most prestigious physicists of the time led Mayer to an attempted suicide on May 18, 1850, after which Mayer was committed to an insane asylum.

About the same time another outsider, James Prescott Joule, was doing similar experiments. Joule was the manager of a brewery and an amateur scientist on the side. Joule was investigating the possibility of replacing the steam engines in his brewery with the newly invented electric motor. This investigation ended when Joule discovered that it took about five pounds of battery zinc to do the same work as a single pound of coal. But during these experiments, Joule came to the conclusion that there was an equivalence between the heat produced by an electrical current in a wire, like in a toaster, and the work done by an electrical current in a motor. In 1843, he presented a paper to the British Association for the Advancement of Science in which he announced that it took 838 ft-lbs of mechanical work to raise the temperature of a pound of water by 1 ⁰F (1 Btu). In 1845, Joule presented another paper, On the Mechanical Equivalent of Heat, to the same association in which he described his most famous experiment. Joule used a falling weight connected to a paddle-wheel in an insulated bucket of water via a series of ropes and pulleys to stir the water in the bucket. He then measured the temperature rise of the water as the weight, suspended by a rope connected to the paddle-wheel, descended and stirred the water. The temperature rise of the water was then used to calculate the mechanical equivalent of heat that equated BTUs of heat to ft-lbs of mechanical work. But as with Mayer, Joule’s work was entirely ignored by the physicists of the day. You see, Mayer and Joule were both outsiders challenging the accepted caloric theory of heat.

All this changed in 1847 when physicist Hermann Helmholtz published On the Conservation of Force in which he referred to the work of both Mayer and Joule and proposed that heat and mechanical work were both forms of the same conserved force we now call energy. Finally, in 1850, Rudolph Clausius formalized this idea as the first law of thermodynamics in On the Moving Force of Heat and the Laws of Heat which may be Deduced Therefrom.

The modern statement of the first law of thermodynamics reads as:

dU = dQ – dW

This equation simply states that a change in the internal energy (dU) of a closed system, like a steam engine, is equal to the amount of heat flowing into the system (dQ), minus the amount of mechanical work that the system performs (dW). Thus steam engines take in heat energy from burning coal and convert some of the heat energy into mechanical energy, exhausting the rest as waste heat.

Remember how Carnot thought that the efficiency of a steam engine only depended upon the temperature difference between the steam from the boiler and the temperature of the room in which the steam engine was running? Carnot believed that when caloric fell through this temperature difference, it produced useful mechanical work. Clausius reformulated this idea with a new concept he called entropy, described in his second law of thermodynamics. Clausius reasoned that there had to be some difference in the quality of different energies. For example, there is a great deal of heat energy in the air in the room you are currently sitting in. According to the first law of thermodynamics, it would be possible to build an engine that converts the heat energy in the air into useful mechanical energy and exhausts cold air out the back. With such an engine, you could easily build a refrigerator that produced electricity for your home and cooled your food for free at the same time. All you would need to do would be to hook up an electrical generator to the engine, and then run the cold exhaust from the engine into an insulated compartment! Clearly, this is impossible. As Clausius phrased it, "Heat cannot of itself pass from a colder to a hotter body".

The second law of thermodynamics can be expressed in many ways. One of the most useful goes back to Carnot’s original idea of the maximum efficiency of an engine and the temperature differences between the steam and room temperature. It can be framed quantitatively as:

Maximum efficiency of an engine = 1 – T_c/T_h

where T_c is the temperature of the cold reservoir into which heat is dissipated and T_h is the temperature of the hot reservoir from which heat is obtained. For a steam engine, T_h corresponds to the temperature of the steam and T_c to the temperature of the surrounding room, with both temperatures measured in absolute degrees Kelvin. So Carnot’s proposal was correct. As the temperature difference between T_c and T_h gets larger, T_c/T_h gets smaller and the efficiency of the engine increases and approaches the value “1” or 100%. In this form of the second law, entropy represents the amount of useless energy; energy that cannot be turned into useful mechanical work and remains as useless waste heat.

Later, in 1865, Clausius presented the most infamous version of the second law at the Philosophical Society of Zurich as:

The entropy of the universe tends to a maximum.

What he meant here was that spontaneous changes tend to smooth out differences in temperature, pressure, and density. Hot objects cool off, tires under pressure leak air and the cream in your coffee will stir itself if you are patient enough. A car will spontaneously become a pile of rust, but a pile of rust will never spontaneously become a car. Spontaneous changes cause an increase in entropy and entropy is just a measure of the degree of this smoothing-out process. So as the entropy of the universe constantly increases with each spontaneous change - the universe tends to run downhill with time. Later we will see that entropy is also a measure of the disorder of a system at the molecular level. In a sense, Murphy’s law is just a popular expression of the second law.

The first and second laws of thermodynamics laid the foundation of thermodynamics which describes the bulk properties of matter and energy. The science of thermodynamics finally allowed steam engine designers to understand what was going on in the cylinders of a steam engine by relating the pressures, temperatures, volumes, and energy flows of the steam in the cylinders while the engine was running. This ended the development of steam engines by trial and error, and the technological craft of steam engine building finally matured into a science.

In Softwarephysics, we apply these same basic ideas of thermodynamics to the macroscopic behavior of software at the program level. The macroscopic behavior of a program can be viewed as the functions the program performs, the speed with which those functions are performed, and the stability and reliability of its performance, just as the macroscopic behavior of steam in a steam engine can be defined by pressure, temperature, and volume changes. This is the viewpoint of software from the perspective of IT management and end-users. They don’t really care what is going on inside of software; they are only interested in how software behaves. From this perspective, it is well known that the entropy of software tends to spontaneously increase with time; software tends to run downhill. Whenever we change software, there is a very good chance that things will run amuck; that is why IT has developed such elaborate change management procedures. But even when we don’t change software, we still seem to get into trouble. I began the very first post on softwarephysics with the following paragraph:

Have you ever wondered why your IT job is so difficult? Have you ever noticed that whenever we change software, performance can drastically decline? Have you observed that performance can drastically decline even when we don’t change software; that sometimes applications spontaneously get slow and then spontaneously return to normal response times without any intervention? Have you noticed that 50% of the time we never find a root cause for problems and that we just start bouncing things at random until performance improves? Have you ever wondered why software behaves this way? Is there anything we can do about all this?

My contention is that these observed behaviors of software are due in part to a simulation of the second law of thermodynamics and the natural tendency for the entropy of software to increase with time at the program level. For deeper insights into this phenomenon of software behavior, we will need to drill down to a lower level and turn to another effective theory of physics called statistical mechanics. Statistical mechanics was developed during the last half of the 19th century and allows us to derive the thermodynamic laws of matter and energy, outlined above, by viewing matter as a large collection of molecules in constant random motion.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

Computer Science as a Technological Craft

A few posts back I described how James Watt made improvements to the Newcomen Steam engine that raised the energy efficiency of steam engines from 1% to 3% and in the process sparked the Industrial Revolution. The reason for exploring the history of steam engines from an IT perspective is two-fold. First of all, it will lead us to some very applicable concepts in the form of the second law of thermodynamics and the interplay of entropy (disorder) and information at the coding level; secondly, it is an interesting story of a technological craft developing into a science. A technological craft is a collection of skills and techniques gained through trial and error that is passed down through the generations. A prime example is early metallurgy. People began to smelt copper about 3800 B.C. in Iran. A thousand years later, in 2800 B.C., the Sumerians in Iraq learned how to combine copper and tin into a very hard and durable alloy called bronze - the distinguishing hallmark of civilization. Around 1500 B.C. the Hittites began to work with iron, which was softer than bronze at the onset, but which made possible the discovery in 1000 B.C. that when iron was reheated with charcoal, it formed a very hard alloy we now call steel. Over many thousands of years of trial and error, people learned many useful metallurgical techniques, like how to harden metal by quenching hot metal in water and then to temper the brittle quenched metal by reheating it and allowing the metal to slowly anneal. Over thousands of years, mankind developed many impressive metallurgical skills and practices, but during all this time, nobody really had any idea of what was going on in the metal during all these intricate manipulations. Acquiring technology by trial and error is a very slow process indeed. Today, metallurgy has evolved into a branch of the material sciences and studies metals at the atomic level within crystal lattices. A modern metallurgist might use x-ray diffraction patterns, electron microscopy, neutron bombardment, or a mass spectrometer to figure out what is going on in a new alloy under study.

Although IT and computer science are populated by very intelligent and gifted people, I would like to suggest that computer science is still an emerging technological craft on the threshold of becoming a true science. In the current state of affairs, even software engineering is little more than the formal instructional framework of a guild. It outlines the procedures to follow to develop and maintain software without dealing with the ultimate nature of software itself. It’s like describing the formal procedures to heat treat the blade of a sword without going into the nature of the steel in the blade itself. In the modern world, civil engineers would find it very difficult indeed to design bridges without knowledge of the tensile strength of structural steel. That is where softwarephysics can be of help to software engineers by providing a theory of software behavior, just as physics comes to the aid of civil engineers by offering a model for the behavior of steel under load.

In 1979, when I transitioned from being an exploration geophysicist to become an IT professional, I left a diverse exploration team exploring for oil in the Gulf of Suez to become a programmer in Amoco’s IT department. Exploration teams are a multidisciplinary team consisting of geologists, geophysicists, petrophysicists, geochemists, and paleontologists. Oil companies throw all the science they can muster at trying to figure out what is going on in a prospective basin before they start spending lots of money drilling holes, just as it is prudent to try to understand the internal nature of an alloy before trying to improve it. When I moved into Amoco’s IT department, I came into contact with many talented and intelligent people, but I was dismayed to discover that there was little sharing of ideas from the other sciences like I had found in my old exploration teams. It seemed that computer science was totally isolated from the outside scientific world. I realized that computer science was a very young science at the time and that it was more of a technological craft than a science, but that was nearly 30 years ago! It’s time for computer science to learn from the other sciences! Fortunately, we are beginning to see this in computer science with the Biologically Inspired Computing (BIC) community in the computer science departments of many universities. The BIC community is, at long last, bringing in ideas from biology, physics, and biochemistry into mainstream computer science and ultimately IT.

Now back to steam engines. Watt did not know that he had increased the energy efficiency of steam engines because he had never even heard of the term energy. The concept of energy did not arrive on the scene until 1850 when Rudolph Clausius published the first law of thermodynamics. But Watt did know that his engine used about 1/3 the coal of a standard Newcomen steam engine with the same horsepower. Unfortunately, the science of the day was not of great help to 18th-century steam engine designers. They had to deal with some very poor effective theories at the time. There was a lot of confusion about the nature of heat in those days. In 1667, Johann Joachim Becher published the phlogiston theory of combustion. The phlogiston theory was an effective theory that proposed that flammable substances such as coal contained a mysterious substance called phlogiston. When coal burned, it released phlogiston to the air. The ash that remained behind was the residual "dephlogisticated" form of coal, while the fumes from the burning coal were "phlogisticated air". Air could only hold so much phlogiston before it became saturated, and that was why a candle could be snuffed out by an overturned glass. The phlogiston theory of combustion held sway until it was replaced by the caloric theory of heat by Antoine Lavoisier in 1783. Lavoisier proposed that when coal burned, it really combined with the newly discovered gas found in air called oxygen and released another mysterious substance called caloric. Caloric was the “substance of heat” and always flowed from hot bodies to cold bodies. Naturally, the hot flue gasses from the burning coal expanded as they took up caloric, making hot air balloons possible. Now during all this time, Daniel Bernoulli had proposed an effective theory of heat that is still in force today known as the kinetic theory of heat. In 1738, Bernoulli proposed that gasses are really composed of a very large number of molecules bouncing around in all directions. Gas pressure in a cylinder was simply the result of a huge number of molecular impacts from individual gas molecules against the walls of a cylinder, and heat was just a measure of the kinetic energy of the molecules bouncing around in the cylinder. But the kinetic theory of heat was not held in favor in Watt’s day, and unfortunately for early steam engine designers, none of these theories of heat related heat energy to mechanical energy, so Watt and the other 18th-century steam engine designers were pretty much on their own, without much help from the available science of the day. In the absence of a useful scientific model for energy, steam engine building in the 18th century became a technological craft, just as computer science has become a technological craft lacking a practical model for software behavior.

This all changed in 1824 when Sadi Carnot published a small book entitled Reflections on the Motive Power of Fire, which was the first scientific treatment of steam engines. In this book, Carnot asked lots of questions. Carnot wondered if a lump of coal could do an infinite amount of work. “Is the potential work available from a heat source potentially unbounded?". He also wondered if the material that the steam engine was made of or the fluid used by the engine made any difference. "Can heat engines be in principle improved by replacing the steam by some other working fluid or gas?". And he came to some very powerful conclusions. Carnot proposed that the efficiency of a steam engine only depended upon the difference between the temperature of the steam used by the engine and the temperature of the room in which the steam engine was running. He envisioned a steam engine as a sort of caloric waterfall, with the difference between the temperature of the steam from the boiler and the temperature of the room the steam engine was in being the height of the waterfall. Useful work was accomplished by the steam engine as caloric fell from the high temperature of the steam to the lower temperature of the room, like water falling over a waterfall doing useful work on a paddlewheel. It did not matter what the steam engine was made of or what fluid was used.

"The motive power of heat is independent of the agents employed to realize it; its quantity is fixed solely by the temperatures of the bodies between which is effected, finally, the transfer of caloric."

“In the fall of caloric the motive power evidently increases with the difference of temperature between the warm and cold bodies, but we do not know whether it is proportional to this difference.”

“The production of motive power is then due in steam engines not to actual consumption of the caloric but to its transportation from a warm body to a cold body.”

Later we will see that Carnot’s ideas were really an early expression of the second law of thermodynamics.

Although we no longer have much confidence in the caloric theory of heat, and now favor the kinetic theory of heat in its place, Carnot was able to make a great contribution to the science of steam engines and thermodynamics by creating an abstract model of steam engines that provided a direction for thought. However, Reflections on the Motive Power of Fire created little attention in 1824 and quickly went out of print. Carnot’s ideas were later revived by work done by Lord Kelvin in 1848 and by Rudolph Clausius in 1850 when Clausius published the first and second laws of thermodynamics in On the Moving Force of Heat and the Laws of Heat which may be Deduced Therefrom. In a similar manner, softwarephysics attempts to provide an abstract model for the behavior of software that provides a direction for thought.

So what happened to Sadi Carnot? In 1832, Carnot died in a cholera epidemic at the age of 36, a victim of the miasma theory of disease described in an earlier post. Unfortunately, because of the fear of choleric miasma, many of his writings were also buried along with him, leaving only a few additional surviving scientific writings.

Next time we will continue on to define the first and second laws of thermodynamics and encounter another strange mysterious substance called entropy – the true bane of all programmers.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

So Why Are There No Softwarephysicists?

I started working on softwarephysics in 1979 and I have struggled with this question for many years. In my last post, I proposed the idea of thinking of both software and money as being virtual substances. This brought to mind an old high school experience of mine from 1969. During the last semester of my senior year, while I and all of my fellow classmates were well established in our well deserved senior slump, I came across one of those teachers who remain with you for the rest of your life. In this very last semester of high school, we all had to take a mandatory course in economics, in which, from the onset, I totally had no interest, having already mentally departed for the University of Illinois to study physics. However, to my surprise, this teacher totally won me over on the very first day of class with an introduction to the course that went something like this:

We have had money in circulation for several thousand years. We have had domestic and international trade for several thousand years. We have had governments collecting taxes, tariffs, and tribute for several thousand years. We have had banks and money lending for several thousand years. Writing and mathematics have been with us for several thousand years too, and they were invented primarily so that we could do accounting, which has also been with us for several thousand years. And all of these things had life or death consequences for the people of the time. Wars were fought over these issues. Kingdoms and civilizations rose and fell over these issues. All of human history was shaped by these issues. So the question is, dear student, how come there were no economists until the 18th century? Everything that modern economists study and work with had been around for several thousand years, and yet there were no economists! Why? British economist Adam Smith is credited with inventing economics when he published The Wealth of Nations in 1776. Why did that take several thousand years? What changed? What changed was a way of thinking brought about by Galileo, Des Cartes, Spinoza, and Newton. As Thomas Paine put it, it was The Age of Reason. The Enlightenment of the 18th century brought on by the Scientific Revolution of the 17th century created a worldview capable of contemplating economic theories. In this worldview, the Universe was, at last, understandable and rational - it followed physical laws and so too could economic activities.

It has been 66 years since the Age of Software began in the spring of 1941 when Konrad Zuse completed his Z3 computer. But in all that time, I have never seen a theory for software behavior, outside of softwarephysics, that depicted software with invariant tangible physical properties and a set of underlying principles or laws that govern those properties. I have never seen an economic theory of software. Yet deep down I am convinced that nearly all IT professionals really do think of software as a virtual substance. At times we curse software, at other times we cajole software, but at all times we are obsessed with software. I started programming in the fall of 1972 when I took the obligatory course in FORTRAN programming that nearly all physics majors took at the time, and ever since it has been the same story no matter where I go or what I do. I have programmed on punched cards, punched tape, magnetic tape and disk drives using no editor, line editors, full-screen editors, and CASE editors. I have used 3rd generation languages, 4th generation languages, CASE languages, compiled languages and interpreted languages and it has always been predictably the same. When I start working on some code, it’s like nothing has really changed in 35 years. It’s all the same problems over and over, day in and day out. In physics, we would say that software is homogeneous, isotropic and time-invariant. That means that software is the same no matter where you go, where you look, or where you find yourself in time. It’s always the same thing over and over.

Such symmetries have profound implications in modern physics. Emmy Noether was a brilliant mathematical genius, who like Einstein before her, fled Nazi Germany in 1933. In 1918, she published Noether’s Theorem which has become a fundamental tenet in theoretical physics. Her theorem states that there is a one-to-one correspondence between the conservation laws and the symmetries of nature. For example, let’s suppose you do a simple high school physics experiment like colliding two billiard balls on a pool table. Now move the whole contraption 100 miles east and do the same exact experiment. You get the same results. That means the results are symmetric under a spatial translation. Noether showed that this translational symmetry implied the law of the conservation of momentum – the bad stuff that happens when you run into a parked car. Now rotate the pool table 180⁰ and repeat the experiment. Again you get the same result. Symmetry under rotation implies the conservation of angular momentum – why a skater speeds up when she pulls in her arms in a spin. Do the same experiment a month later, and again you will get the same results. The symmetry over time implies the conservation of energy. So the fact that software is homogeneous, isotropic, and time-invariant just screams out for some underlying laws at work. Something has to be going on!

Next time I really will pick up again with our steam engine designers and thermodynamics. I am a little hesitant to get deeper into physics, since there is the chance of losing some of you, given the very low level of popularity of physics with the general public. But I am going to forge ahead with as little math as possible and try to stick to the main ideas of physics from an IT perspective. My old physics department had a plaque on the wall that read “I understand the material; I just can’t do the problems”. I have frequently thought a more fitting plaque would have been “I can do the problems; I just don’t understand the material.” In my opinion, one of the major failings of teaching physics in this country has been an overemphasis on problem-solving. After all, most physics students never end up being professional physicists, but the underlying concepts of physics can be understood by most people and can be used by all on a daily basis to improve their lives.

So where are all the softwarephysicists? Why you’re one of them! You just don’t know it yet.


Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

Software as a Virtual Substance

Before diving into thermodynamics, let’s map out the road ahead. Remember that the underlying concept of softwarephysics is that the global IT community has unintentionally created a pretty decent computer simulation of the physical Universe that softwarephysics depicts as the Software Universe. We then use this simulation in reverse. Understanding how the physical Universe operates, allows us to better model how software behaves in the Software Universe. We do this by visualizing software as a virtual substance. Now how do you model a virtual substance? It might help to think of another virtual substance that we are all more familiar with – money. Many people devote their entire educations and careers to learning how to manipulate and model the virtual substance we call money. Nobody gets squeamish about the Federal Reserve Board expanding or contracting the money supply, manipulating interest rates, or trying to raise or lower our exchange rate with other currencies. In fact, you can win a Nobel Prize for such efforts. But for the most part, money is simply a collection of bits stored in a network of computers. Is software any less real than money? Now imagine trying to run the modern world economy without the benefit of macro and microeconomic theories to model the ebb and flow of money throughout the world economies. The aim of softwarephysics is to achieve a similar ability to model the behavior of software at differing levels of software architecture by using the physical Universe as a guide.

The Value of Effective Theories
Our current understanding of the physical universe is based upon a collection of effective theories in physics. Recall that an effective theory is an approximation of reality that only holds true over a certain restricted range of conditions and only provides a certain depth of understanding of the problem at hand. Although all effective theories are fundamentally “wrong”, they are still exceedingly useful. All of the technology surrounding you - your PCs, your cell phones, your GPS units, and your air conditioners, vacuum cleaners, cars, and TV sets were all built using the current approximate effective theories of physics. It’s amazing that you can build all these useful gadgets using theories that are all fundamentally “wrong”, but that just highlights the value of effective theories. The crown jewel of physics is the effective theory called QED - Quantum Electrodynamics. QED has predicted the gyromagnetic ratio of the electron, a measure of its intrinsic magnetic field, to 11 decimal places. This prediction has been validated by rigorous experimental data. As Richard Feynman has pointed out, this is like predicting the exact distance between New York and Los Angeles to within the width of a human hair! Yet most physicists believe that someday an even better effective theory will come along to replace QED. I try to carry over this idea of approximate effective theories into my personal and professional lives too, by trying to keep in mind that my own deeply held personal opinions are also just approximations of reality too. This makes it easier to live with, and work with, people of differing viewpoints.

It might seem disheartening that we don’t really understand what is going on in our own Universe and that all we have is a set of very useful approximations of reality. But as Ayn Rand cautioned us, be sure to “check your premises”. What exactly do you mean by reality? Physicists and philosophers have been debating the nature of reality for thousands of years. My personal favorite definition is:

Physical Reality – Something that does not go away even when you stop believing in it.

I find this definition to be flexible enough to keep most people happy, including physicists, theologians, and even most philosophers. When we discuss the implications of quantum mechanics for software, you may be surprised to learn that about 60% of physicists still ascribe to the old Copenhagen interpretation of quantum mechanics (1927) in which absolute reality does not even exist. In the Copenhagen interpretation, there are an infinite number of potential realities. About 30% adhere to a variation of the “Many-Worlds” interpretation of Hugh Everett (1957) which admits an absolute reality, but claims that there are an infinite number of absolute realities spread across an infinite number of parallel universes. And the remaining 10% bank on the really strange interpretations! They all agree on the underlying mathematics of quantum mechanics; they just don’t agree on what the mathematics is trying to say. It’s like trying to figure out what the mathematical formula:

Glass = 0.5

is trying to tell you. Is the glass half empty or half full? So many physicists skirt the whole issue by adopting a positivist approach to the subject. Logical positivism is an enhanced form of empiricism, in which we do not care about how things “really” are. We are only interested in how things are observed to behave, and effective theories are a perfect fit in that regard. In softwarephysics, I am also taking a positivist approach to software. I don’t care what software “really” is, I only care about how software is observed to behave.

Softwarephysics is a Simulated Science
Just as physics is a collection of useful effective theories, softwarephysics is also a matching collection of effective theories. Because softwarephysics is a simulated science, the challenge for softwarephysics is to find the corresponding matching effective theory of physics to apply to software at each level of complexity. At the highest level, we will be using chaos theory which is a theory for nonlinear dynamical systems developed in the 1970s and 1980s. In the physical Universe, one might apply chaos theory to the traffic patterns on the network of expressways found in a large metropolitan area such as Chicago. In a similar fashion, we could apply chaos theory to the complex traffic patterns for a large corporate website residing on 100 production servers - load balancers, firewalls, proxy servers, webservers, WebSphere Application Servers, CICS Gateway servers to mainframes, mailservers, etc. Any IT professional who has ever worked on a large corporate website might indeed guess that chaos theory would provide a fitting description of their daily life, without ever even having heard of softwarephysics! In fact, chaos theory has already been applied to computer networks by many others.

Thermodynamics
If we pull one of the cars out of a traffic jam on one of the expressways and take a look under the hood, we come to the next level in the hierarchy of effective theories - thermodynamics which was developed during the last half of the 19th century. Thermodynamics allows us to understand the macroscopic behaviors of matter. For example, thermodynamics allows us to understand what is going on in the cylinders of a car by relating the pressures, temperatures, volumes, and energy flows of the gases in the cylinders while the engine is running. We can apply these same basic ideas to the macroscopic behavior of software at the program level. The macroscopic behavior of a program can be viewed as the functions the program performs, the speed with which those functions are performed, and the stability and reliability of its performance.

Statistical Mechanics
The next lower level theory in the hierarchy of effective theories is called statistical mechanics. Statistical mechanics was also developed during the last half of the 19th century and allows us to derive the thermodynamic properties of the gases in the cylinders of a car by viewing the gases as a large collection of molecules bouncing around in the cylinders. It also provides us with a definition of information and some very powerful insights into how information operates in the physical Universe. We will see that we can apply these ideas to software at the line of code level by examining the interplay of information and entropy (disorder) at the coding level.

QED and Chemistry
Going still deeper we come to QED (Quantum Electrodynamics) which is the basis for chemistry at the molecular level. QED is an effective theory that reached maturity in 1948 and which is an amalgam of two other effective theories; quantum mechanics (1925) and Einstein’s special theory of relativity (1905). We will use ideas from QED at the line of code level by depicting lines of code as interacting organic molecules, similar to the chemical reactions back at the refinery that made the gasoline for the car in question.

Quantum Mechanics
Deeper still we finally come to quantum mechanics (1925). Quantum mechanics describes the structure and behavior of individual atoms and we will use concepts from quantum mechanics to describe software at the level of individual characters in source code like the carbon and hydrogen atoms found in gasoline molecules.

In summary:

Software Element	Physical Counterpart	Effective Theory
Computer Networks	Expressway Traffic	Chaos Theory
Programs	Gas in a Cylinder	Thermodynamics and Statistical Mechanics
Lines of Code	Organic Molecules	QED and Chemistry
Source Code Characters	Atoms	Quantum Mechanics

When we are finished with all of this, we will have a collection of effective theories for softwarephysics that will allow us to define a self-consistent model for software behavior that can be used for making day-to-day decisions in IT and provide a direction for thought. It will also lead us to the fundamental problem of software and the suggestion that a biological solution is in order.

Next time we will continue on with the saga of steam engine designers and how their struggle led to the development of the first and second laws of thermodynamics, the concept of entropy (disorder) and the discovery of information itself.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

A Lesson From Steam Engines

Let’s get back to exploring the benefits of applying science to computer science. Since the rebirth of science about 400 years ago, we have had two major economic revolutions; the Industrial Revolution and the Information Revolution. During the Industrial Revolution, mankind began to manipulate large quantities of energy; while during the Information Revolution, mankind began to manipulate large quantities of information. Both have had huge economic impacts. We can date the dawn of the Information Revolution to the spring of 1941 when Konrad Zuse built the Z3 with 2400 telephone relays. Similarly, we can date the dawn of the Industrial Revolution to 1712 when Thomas Newcomen invented the first commercially successful steam engine.

As an IT professional you are a warrior in the Information Revolution. You work with information all day long. You create, maintain, and operate software which processes information in huge quantities. Software is also a form of information, so essentially you get paid to process information with information. Have you ever stopped to wonder what information is? Is information “real” or just something we made up? Is information a tangible part of the physical Universe, or is it just a useful human contrivance like the names we use for the days of the week? Over the past 400 years, the role of information in physics has taken on more and more significance, to the point that many eminent physicists, such as John Wheeler, have proposed that the physical Universe is simply made out of information - “It from Bit”. Over the years, the concept of information has arisen in physics in several effective theories, most notably in thermodynamics and Einstein’s special theory of relativity. Today we will lay the foundations for the concept of information in thermodynamics, and leave Einstein for another time. Now let’s see if we can learn a lesson from the past warriors of the Industrial Revolution.

The early factories of the 18th century were forced to run on water power. This required them to be located in the highlands near fast-moving water, far from the lowland cities where workers and consumers resided and distant from many natural resources required for production. What was needed was a portable source of power. The Newcomen steam engine was the first commercially successful steam engine and consisted of an iron cylinder with a movable piston. Low-pressure steam was sucked into a cylinder by a rising piston. When the piston reached its maximum extent, a cold water spray was shot into the cylinder causing the steam to condense and form a partial vacuum in the cylinder. External atmospheric air pressure forced the piston down during the power stroke. In the 18th-century, steam engines used low-pressure steam and were thus called atmospheric steam engines because the power stroke came from atmospheric air pressure. High-pressure steam boilers in the 18th century were simply too dangerous to use for steam engines. The Newcomen steam engine was used primarily to pump water out of coal mines. It had an efficiency of about 1%, meaning that about 1% of the energy in the coal used to fuel the engine ended up as useful mechanical work, while the remaining 99% ended up as useless waste heat. This did not bother owners of steam engines in the 18th century because they had never even heard of the term energy. The concept of energy did not come into existence until 1850 when Rudolph Clausius published the first law of thermodynamics. However, they did know that the Newcomen steam engine used a lot of coal. This was not a problem if you happened to own a coal mine, but for 18th-century factory owners, the Newcomen steam engine was far too expensive for their needs.

You can see the oldest surviving Newcomen steam engine at the Henry Ford Museum in Dearborn Michigan just outside of Detroit, as well as Thomas Edison’s original Menlo Park Laboratory, which has also been relocated to the adjoining Greenfield Village museum. This engine was built in 1760 and pumped water from an English coal pit until 1834. I had the chance to see this steam engine a few years ago. It was as big as a house and weighed in at a whopping 15 horsepower, about the horsepower of a modern riding lawnmower. You might wonder why anybody would go to the trouble of building such an engine, but you have to compare it to the effort involved in the care and feeding of 15 horses!

Figure 1 – The first commercially successful steam engine was invented by Thomas Newcomen in 1712. The Newcomen steam engine had an efficiency of 1%.

In 1763, James Watt was a handyman at the University of Glasgow building and repairing equipment for the University. One day the Newcomen steam engine at the University broke, and Watt was called upon to fix it. During the course of his repairs, Watt realized that the main cylinder lost a lot of heat through conduction and that the water spray which cooled the entire cylinder below 212 ⁰F required a lot of steam to reheat the cylinder above 212 ⁰F on the next cycle. In 1765, Watt had one of those scientific revelations in which he realized that he could reduce the amount of coal required by a steam engine if he could just keep the main cylinder above 212 ⁰F for the entire cycle. He came up with the idea of using a secondary condensing cylinder cooled by a water jacket to condense the steam instead of using the main cylinder. He also added a steam jacket to the main cylinder to guarantee that it always stayed above 212 ⁰F for the entire cycle. In 1765, Watt conducted a series of experiments on scale model steam engines that proved out his ideas.

Figure 2 – In 1765, James Watt improved the Newcomen steam engine by introducing a condensing cylinder to condense steam during the power stroke and by using a steam jacket to always keep the main cylinder at a temperature higher than the boiling point of water. Watt's improved steam engine had an efficiency of 3%, and on that basis, launched the Industrial Revolution

To learn moree about the Newcomen and Watt steam engines go to:

https://en.wikipedia.org/wiki/Watt_steam_engine

Watt’s steam engine had an efficiency of 3% which may still sound pretty bad, but that meant it only used 1/3 the coal of a Newcomen steam engine with the same horsepower. So the Watt steam engine became an economically viable option for 18th-century factory owners. We will discuss the second law of thermodynamics at a later time. But just for the sake of comparison, the second law allows us to calculate that the maximum efficiency of a steam engine running at a room temperature of 72 ⁰F using 212 ⁰F steam is 21%.

The Industrial Revolution was delayed by more than 50 years because nobody bothered to try to understand what was going on in a Newcomen steam engine. This was overcome by James Watt when he unknowingly applied the scientific method to steam engines. Based upon some empirical evidence gathered while repairing a Newcomen steam engine, he had a moment of inspiration. He then proceeded to deduce the implications of his revelation and came up with the design for a new kind of steam engine. He then tested his design with a series of controlled experiments.

We are now some 60+ years into the Information Revolution, and like our counterparts in the Industrial Revolution, we are still struggling with the inefficiency of creating and operating software. And like our counterparts, we know that we are very inefficient at software, but we really do not have a clue as to how inefficient we may truly be. Softwarephysics proposes that we stop and take a look into our engine compartment.

The Problem with Common Sense
Like the 18th-century engineers struggling with steam engines, IT professionals have developed a common sense approach to software based upon a set of heuristics. In IT, you quickly learn that when coding software it never works the first time. If you are lucky it will work on the 10th try. If it does not work by the 100th try, you need to look for another profession. But common sense is just another effective theory. Recall that an effective theory is an approximation of reality that only holds true over a certain restricted range of conditions and only provides a certain depth of understanding of the problem at hand. Take a ballpoint pen from your desk and one of your shoes. Drop them both from shoulder height and see which one hits the ground first. For nearly 2,000 years, common sense and the teachings of Aristotle held that the shoe will hit the ground first. It was not until the late 16th century that Galileo demonstrated that they hit the ground at the same time. He also discovered that if you doubled the time of a fall, the distance traveled increased by a factor of four (the square of the time). This was one of the first uses of a mathematical model in physics. The purpose of softwarephysics is to go beyond IT common sense and come up with an effective theory of software behavior at a deeper level.

Next time we will continue on with thermodynamics and see how it led to an effective theory of information and how softwarephysics incorporates that theory into a model for software behavior.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

How To Think Like A Scientist

I was just about to tell you about applying science to computer science when I realized I was getting ahead of myself. First I need to define what I mean by science. As with all of softwarephysics, this is my own operational definition. However, I think it is pretty close to the mainstream concept of what science is as held by the majority of the scientific community.

First of all, science is a way of thinking. Science has a methodology to aid in this way of thinking which has been very successful over the past 400 years. The purpose of the scientific method is to formulate theories or models of the Universe. A scientific model is a simplified approximation of reality that allows people to gain insight into the real structure and operation of true reality. Scientists create models to explain observations, predict future observations, and provide direction for thought. The scientific method is a little different than the way most people think in their daily lives, so let’s examine some of the ways people come up with ideas with a little help from our philosophical friends.

There are three main approaches to gaining knowledge:

1. Inspiration/Revelation
These are ideas that just come out of the blue with no apparent source. I find that IT people are very good at this. For example, on a conference call for a website outage, I am frequently surprised at the incredible level of troubleshooting skill of many of the participants. I frequently wonder to myself “Where did that insight come from?” when somebody nails a root cause out of the blue.

Most of the great ideas in science have also come from inspiration/revelation. For example, in 1900 Max Planck had the insight that he could solve the Ultraviolet Catastrophe by assuming that charged particles in the walls of a room could only oscillate with certain fixed or quantized frequencies. The classical electromagnetic theory of the day predicted that the room you are currently sitting in should be bathed in a lethal level of ultraviolet light and x-rays and that the walls of the room should be at a temperature of absolute zero having turned over all of their available energy into zapping you to death. This was clearly evidence of a theory missing the mark by a wide margin! Planck thought that his fixed frequency solution was just a mathematical trick, but in 1905 Einstein had the revelation that maybe this was not just a trick. Maybe light did not always behave as an electromagnetic wave. Maybe light sometimes behaved like a stream of particles we now call photons that only came in fixed or quantized amounts of energy. The fixed energy of the photons would match up with the fixed frequencies of the charged particles in the walls of your room. In 1924, Louis de Broglie had another revelation and suggested that particles, like electrons, might behave like waves too, just as a stream of photons sometimes behaved like an electromagnetic wave. In 1925, Werner Heisenberg and Erwin Schrödinger developed quantum mechanics based upon these insights, and in 1948 the transistors in your PC were invented at Bell Labs based upon quantum mechanics.

The limits of Inspiration/Revelation:
You never know for sure that your idea is correct.

2. Deductive Rationalism
With deductive rationalism, you make a few postulates which usually come from inspiration/revelation and then you deduce additional ideas or truths from them using pure rational thought. Plato and Des Cartes were big fans of deductive rationalism. It goes like this:

If A = B
And B = C
Then A = C

The limits of deductive rationalism:
In 1931, Kurt Gödel proved that no self-consistent mathematical theory could deduce all truths and that no self-consistent mathematical theory could prove that it was always self-consistent (does not contradict itself). So you cannot deduce all truths.

3. Inductive Empiricism
With inductive empiricism, you make a lot of observations and then reverse the deductive rationalism process. Aristotle and John Locke were big fans of inductive empiricism. If I observe that 99.99% of the time that A = C, then I will assume that A is really equal to C, and I will chalk up the .01% discrepancy to observational error. I don’t know anything about B at this point because I have no observations of B’s state. However, if I make some more observations and find that 99.99% of the time that B = C, then I will infer that B is really equal to C, and therefore, that A is really equal to B too.

If A = C 99.99% of the time
And B = C 99.99% of the time
Then B = C, A = C, and A = B

The limits of empirical induction:
The above may all just be coincidences and you have to have good technology in order to make accurate observations. Most Ancient Greek philosophers did not like inductive empiricism because they thought that all physical measurements on Earth were debased and corrupt. They believed in the power of pure uncorrupted rational thought. This was largely due to the poor level of measurement technology they possessed at the time (they had no Wily). But even in the 17th century when Galileo was demonstrating experiments to his patrons that proved, contrary to Aristotle’s teachings, that all bodies fell with the same acceleration, they thought his experimental demonstrations were magic tricks!

People get into trouble when they only use one or two of the above three approaches to knowledge to make decisions. I know that I do. Politicians have frequently been known to not use any of them at all! The power of the scientific method is that it uses all three of the above approaches to knowledge. Like the checks and balances in the U.S. Constitution, this helps to keep you out of trouble.

The Scientific Method
1. Formulate a set of hypotheses based upon inspiration/revelation with a little empirical inductive evidence mixed in.

2. Expand the hypotheses into a self-consistent model or theory by deducing the implications of the hypotheses.

3. Use more empirical induction to test the model or theory by analyzing many documented field observations or performing controlled experiments to see if the model or theory holds up. It helps to have a healthy level of skepticism at this point. As philosopher Karl Popper has pointed out, you cannot prove a theory to be true, you can only prove it to be false. Galileo pointed out that the truth is not afraid of scrutiny, the more you pound on the truth, the more you confirm its validity.

Effective Theories
The next concept that we need to understand is that of effective theories. Physics currently does not have an all-encompassing unifying theory or model. Researchers are looking for a TOE – Theory of Everything in physics, but currently, we do not have one. Instead, we have a series of pragmatic effective theories. An effective theory is an approximation of reality that only works over a certain range of conditions. For example, Newtonian mechanics allowed us to put men on the Moon, but it cannot explain how atoms work or why the clocks on GPS satellites run faster than clocks on Earth. All of the current theories in physics are effective theories that only work over a certain range of conditions. Physics currently comes in three sizes – Small, Medium, and Large

• Small – less than 10^-10 meter and tiny masses
Quantum Mechanics – atomic bombs and transistors
• Medium – 19th-Century Classical Physics
Newtonian Mechanics – space shuttle launches
Maxwell’s Electromagnetic Theory – electric motors
Thermodynamics – air conditioners
• Large – greater than 20,000 miles/sec or very massive objects
Einstein’s General Theory of Relativity – cosmology, black holes, and GPS satellites

Since all of the other sciences are built upon a foundation of underlying effective theories in physics, that means that all of science is “wrong”! But knowing that you are “wrong” gives you a huge advantage over people who know that they are “right” because knowing that you are “wrong” allows you to keep an open mind to search for models that are better approximations of reality.

In addition to covering different ranges of conditions, effective theories also come in different levels of depth with more profound effective theories providing deeper levels of insight. For example, Charles’ Law is a very high-level effective theory that states that at a constant pressure, the volume of a gas in a cylinder is proportional to the temperature of the gas. If you double the temperature of a gas in a cylinder having a freely moving piston, its volume will expand and double in size. A more profound effective theory for the same phenomena is called statistical mechanics which views the gas as a large number of molecules bouncing around in the cylinder. When you double the temperature of the gas, you double the energy of the molecules, so they bounce around faster and take up more room. An even deeper effective theory is called quantum mechanics which views the molecules as standing waves in the cylinder.

The goal of softwarephysics is to provide a pragmatic high-level effective theory of software behavior at a level of complexity similar to that of Charles’ Law. Having an effective theory of software behavior is useful because it allows you to make day-to-day IT decisions with more confidence. For example, suppose you learn 30 minutes before your maintenance window goes down that you have a new EJB that must go into production, but that it corrupts 0.5% of a certain new database transaction. A young programmer on your team quickly produces a “fixed” version of the EJB, but he does not have time to regression test it. Do you put the “fixed” EJB into production, or do you go with the one with the known 0.5% bug with the hope that the corrupted database records can be corrected later? As we shall see later, softwarephysics helps in such situations.

The Most Difficult Thing in Science
The final concept of the scientific method is the most difficult for human beings. In science, you are not allowed to believe in things. You are not allowed to accept models or theories without supporting evidence. However, you are allowed to have a level of confidence in models and theories. For example, I do not “believe” in Newtonian mechanics because I know that it is “wrong”, but I do have a high level of confidence that it could launch me into an Earth orbit. I might get blown up on the launch pad, but like all of our astronauts, I would bet my life on Newtonian mechanics getting me into an Earth orbit instead of plunging me into the Sun if I do my calculations properly! Similarly, I have a low level of confidence in the old miasma theory of disease. In the early 19th century, it was thought by the scientific community that diseases were caused by miasma, a substance found in foul smelling air. And there was a lot of empirical evidence to support this model. For example, people who lived near foul smelling 19th-century rivers were more prone to dying of cholera than people who lived further from the rivers. We had death certificate data to prove that empirical fact. If you were running a cesspool cleaning business in the 19th century, you knew that on the first day of work your rookies were likely to get sick and vomit when they were exposed to the miasma from their first cesspool and a few days later they might come down with a fever and die on you! The miasma theory of disease even had predictive power! If you were running a 19th-century cesspool cleaning business in the middle of a cholera epidemic, and you shut down your operation during the epidemic, while your competitors kept theirs open, you would probably enjoy a larger market share when the epidemic subsided. This just highlights the dangers of relying too heavily on the inductive empiricism approach to gaining knowledge.

As a human being, it is hard not to believe in things. I have been married for 32 years and I have two wonderful adult children. And I truly believe in them all! If somebody confronted me with incontrovertible evidence that one of my children had embezzled funds, my first thought would be that there must be some horrible mistake. However, in scientific matters, you are not allowed this luxury.

The Scientific Method and Softwarephysics
So what does all of this have to do with softwarephysics? Softwarephysics is a high-level effective theory of software behavior. It is a simulated science for the simulated Software Universe that we are all immersed in. Let me explain. In the 1970s, I was an exploration geophysicist writing FORTRAN software to simulate geophysical observations for oil companies. When I transitioned into IT in 1979, it seemed like I was trapped in a frantic computer simulation, just like the ones I used to program for oil companies. After a few months in Amoco’s IT department, I had the following inspiration/revelation:

The Equivalence Conjecture of Softwarephysics

Over the past 70 years, through the uncoordinated efforts of over 50 million independently acting programmers to provide the world with a global supply of software, the IT community has accidentally spent more than $10 trillion creating a computer simulation of the physical Universe on a grand scale – the Software Universe.

I soon realized that I could use this simulation in reverse. By understanding how the physical Universe behaved, I could predict how the Software Universe would react to stimuli, and I proceeded to deduce many implications for software behavior based upon this insight. This was a bit of a role reversal; in physics, we use software to simulate the behavior of the Universe, while in softwarephysics we use the Universe to simulate the behavior of software.

The one problem that I have always had with softwarephysics has been with the confirmation of the model via inductive empiricism. How do you produce and analyze large amounts of documented field observations of software behavior or run controlled experiments for a simulated science? “Hey, Boss I would like to run a double-blind experiment where we install software into production, but only half of it goes through UAT testing. The other half comes straight from the programmers as is, and we don’t know which is which in advance”. Unfortunately, I have always had a full-time job without the luxury of graduate students! So I am relying on 30+ years of personal anecdotal observation of software behavior to offer softwarephysics as a working hypothesis.

Next time I will describe why applying science to computer science is a good idea using the challenges faced by steam engine designers in the 18th century as a case study.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

So You Want To Be A Computer Scientist?

As professional IT people, we are constantly being called upon to innovate. Unfortunately, at most places where I have worked over the past 32 years, that has meant innovating using conventional ideas – a truly difficult thing to do. I know that softwarephysics can be a bit daunting, especially as presented in SoftwarePhysics 101 – The Physics of Cyberspacetime because the course is designed for several audiences – IT people, physicists, and biologists, and none of these folks talk to each other much. So I would like to break down softwarephysics into some smaller chunks that might be easier to absorb from an IT perspective. I work with a large number of my fellow IT people on a daily basis, and I frequently hear that “Why is this happening to me?” sound in their voices at 3:00 AM. This might help.

Let’s begin where it all started in the spring of 1941 when Konrad Zuse built the Z3 with 2400 electromechanical telephone relays. The Z3 was the world’s first full-fledged computer. You don’t hear much about Konrad Zuse because he was working in Germany during World War II. The Z3 had a clock speed of 5.33 Hz and could multiply two very large numbers together in 3 seconds. It used a 22-bit word and had a total memory of 64 words. It only had two registers, but it could read in and store programs via a punched tape. In 1945, while Berlin was being bombed by over 800 bombers each day, Zuse worked on the Z4 and developed Plankalkuel, the first high-level computer language more than 10 years before the appearance of FORTRAN in 1956. Zuse was able to write the world’s first chess program with Plankalkuel. And in 1950 his startup company Zuse-Ingenieurbüro Hopferau began to sell the world’s first commercial computer, the Z4, 10 months before the sale of the first UNIVAC.

Figure 1 – Konrad Zuse with a reconstructed Z3 in 1961 (click to enlarge)

Figure 2 – Block diagram of the Z3 architecture (click to enlarge)

Now in the past 66 years hardware has improved by a factor of about a billion. You can now go to Best Buy with $500 bucks and buy a machine that is approximately a billion times faster than the Z3 with nearly a billion times as much memory. So how much progress have we made on the software side of computer science in this same period of time? How far have we come since Plankalkuel? Now be careful! A billion seconds is 32 years, and I know that some of you have not quite reached that milestone yet. I would estimate that at most we are perhaps 100 – 1,000 times better off at creating, maintaining, and operating software than Zuse was with writing Plankalkuel on punched tape. And I think I am being generous here. So although we have made great strides in software, how come the hardware guys beat us out by a factor of between 1 – 10 million over the past 60 some years? My suggestion is that this was not a fair fight because the hardware guys were cheating - they were using science! Yes, softwarephysics makes the outrageous suggestion that computer scientists try using science! This has already started to happen in academic computer science with the Biologically Inspired Computing community spread across many universities, but it has not yet filtered down much to the commercial IT community.

Next time I would like to discuss why in the world would you possibly want to apply science to computer science? People working on steam engines in the 18th century asked this very same question.

So what happened to Konrad Zuse? Zuse died in 1995 after making many contributions to computing that you use in your job on a daily basis. You can read about his adventures in computing in his own words at:

http://ei.cs.vt.edu/~history/Zuse.html

Being the unsung genius that he was, Zuse published Calculating Space in 1967, in which he proposed that the physical Universe was a giant computer! This crazy idea has recently been adopted and expanded upon by such huge intellects as physicists John Wheeler, Seth Lloyd, David Deutsch and many others now working on quantum computers. In 1687, Newton published his Principia in which he presented the world with Newtonian mechanics. The Newtonian clockwork model of the Universe, which depicted the world as a huge machine relentlessly moving in deterministic paths, dominated Western thought throughout the 18th and 19th centuries. But the rise of quantum mechanics and chaos theory in the 20th century has recently caused many physicists and philosophers to adopt a new model of the Universe which depicts the Universe as a huge quantum computer constantly calculating how to behave.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

SoftwarePhysics Revisited

Since it appears that nobody has solved the fundamental problem of software outlined last year in my original blog at:

https://softwarephysics.blogspot.com/2006/07/softwarephysics.html

I thought there would be no harm in folks taking another look at softwarephysics. I personally have been using softwarephysics on a daily basis for over 25 years to help make decisions about software and to help me cope with the daily mayhem of life in IT, and it might help you too.

For those of you already familiar with softwarephysics, I have a revised version of SoftwarePhysics 101 – The Physics of Cyberspacetime, which is now available on Google Drive. Please note that some of the formulas do not render properly, especially exponents which do not display as superscripts, so please use your imagination.

Part1 - Part 1 of the original PowerPoint document.
Part 2 - Part 2 of the original PowerPoint document.
Entropy – A spreadsheet referenced in Part 1
BSDE – A 1989 document describing how to use BSDE - the Bionic Systems Development Environment - to grow applications from genes and embryos within the maternal BSDE software.

In the new edition, I have expanded on how lines of code can be thought of as organic molecules composed of atoms (characters) in fixed quantum states and beefed up the section on quantum mechanics in general. I also expanded the section covering Einstein's Special Theory of Relativity in relation to causality and the flow of information in the universe. As a supplemental reading, you can find an excellent treatment of the Special Theory of Relativity with very little math, at professor John D. Norton’s course HPS 0410 Einstein for Everyone at:

http://www.pitt.edu/~jdnorton/teaching/HPS_0410/chapters/Special_relativity_rel_sim/index.html

Be sure to investigate the animated graphic on the Relativity of Simultaneity towards the middle of the webpage, which is far more illuminating than my static slide. Quantum mechanics and relativity are both difficult topics, and I hope the additional slides help clarify these ideas. There have been many other minor changes and additions too, that might warrant another look. Again, this course was designed to be given as a series of one-hour seminars over an extended period of time to keep the lethal effects of physics in large doses down to a minimum. When the first director of Fermilab was asked by the FBI what he would do if Fermilab was stormed by an angry mob of anti-war protestors, he responded with "Don't worry, we have a devastating physics lecture under lock and key in our armory for just such an occasion; thank God we have never had to use it!".

For those of you not familiar with softwarephysics, please consider the following thought experiment. Sometime in the spring of 2008, CERN hopes to crank up its new 14 TEV LHC accelerator which will supersede Fermilab's record-breaking 4 TEV accelerator. Imagine that you are a senior physicist on the LHC team that morning and you hear a commotion down the hall. One of your postdocs sticks her head into your office doorway and says, "Hey boss, we finally got the beams up to 14 TEV! We did not find the Higgs boson yet, but we found something even stranger. Nobody has ever seen anything like it before! We decided to call it "software"." I had just such an Alice in Wonderland experience in 1979 when I made a career change and transitioned from being a geophysicist into being a professional IT person. Suddenly one Monday morning, I found myself in alien territory on the floor of Amoco's IT department surrounded by all these strange IT people. They seemed really good at weird complex abstract thought, much more like physicists than the general public at large for sure, but they were always in such a hurry! Every once in a while, I would grab one of them by the scruff of the neck and ask, "So what is this software stuff made of? What are its physical properties? How does it behave? Does it follow physical laws?". The answer was always, "Sorry. Don't have time to think about that now, I have a deadline to make!". That's when I decided to become a softwarephysicist. Since all I knew was physics and a little FORTRAN that I used for modeling geophysical observations, I needed something to help me find my bearings in this new profession. I guess people always gravitate back to what they are comfortable with when placed in an awkward position. I knew a guy who moved from Chicago to the western suburbs when he was a teenager. Whenever he had to go somewhere in the Chicago area, the first thing he did was to drive back to Chicago. I did basically the same thing when I first landed in Amoco’s IT department. The first thing I did was to envision software as a virtual substance. Then all I had to do was follow the normal drill of theoretical physics - make some field observations of software behavior, formulate a model that explained that behavior, and then test the model as best I could.

Software is indeed a strange substance. It can be as hard as nails and as fast as lightning most of the day, and then suddenly it goes through a phase change and becomes a viscous tar-like, goopy, sludge that is as slow as molasses. It can be incredibly stable for many months at a time, and then with a very tiny alteration, it begins to behave like nitroglycerin on a hot summer day. Why does it do that? Is there anything we can do about it? These are some of the questions that softwarephysics tries to answer. In truth, all IT people are really softwarephysicists already. They just don't know it yet. And that goes for nearly the entire population of the modern world too. I get several phone calls per week from my wife concerning cybersludge, and I am sure that you get desperate pleas from friends and relatives too. If software weren't so useful, nobody would put up with the aggravation.

My immediate impression in 1979 was that the software side of computer science seemed to be a little isolated without much interaction with the other sciences, while out of necessity, the hardware side of computer science was heavily dependent upon physics, electrical engineering, and the material sciences. Since the software side of computer science was relatively new at the time, this was quite understandable. So I decided that it would make sense to throw all the physics, chemistry, and biology I could muster at the problem of commercial software development and support to make my life easier as a professional IT person. I called the approach softwarephysics. I figured if you could apply physics to geology, why not apply physics to software? The basic idea was that many concepts in physics suggested to me that the IT community had accidentally created a pretty decent computer simulation of the physical universe and that I could use this simulation in reverse to better understand the behavior of commercial software. This reverse simulation also suggested that a biological approach to software development and support would be a better approach to take than the current traditional IT methodologies.

In 1987, Chris Langton was at the Los Alamos National Laboratory. Just as the web browser, HTML, and the World-Wide Web were invented by Tim Berners-Lee in 1989 in his spare time at CERN, Chris Langton came up with the idea of Artificial Life in his spare time while working at Los Alamos. The basic idea was that biologists could learn a lot about living things by creating artificial simulations of living things with software. Following a few conferences on Artificial Life at Los Alamos, there was a flurry of excitement in many computer science and biology departments at leading universities and research institutes throughout the world in the early 1990s. As with Artificial Intelligence, this initial excitement dropped off a bit as reality set in. However, about 10 years ago the Artificial Life people in academia came to the realization that they could use this simulation approach in reverse too. They figured IT people could learn how to build and operate better software by using concepts from biology. In the U.S., this became known as "biologically inspired computing" and in the U.K. they call it "natural computing". About 5 years ago a dozen or so universities began to offer graduate and undergraduate computer science courses in "biologically inspired computing" and "natural computing". You can find information about these courses and professional conferences on these topics by Googling for "biologically inspired computing" and "natural computing" on the Internet.

So the Artificial Life people have caught onto the biological aspects of softwarephysics, but they still have not caught onto the aspects of softwarephysics based upon physics and biochemistry to a great extent yet. And as far as I can tell, the first successful commercial application of these ideas, aside from some early commercial applications of genetic algorithms, was in 1985 when I developed the BSDE (Bionic Systems Development Environment) tool at Amoco.

Because softwarephysics is still a bit unconventional, to say the least, softwarephysics has been stuck at the first stage of the life-cycle for all new theories for nearly 30 years:

1. First it is ignored
2. Then it is wrong
3. Then it is obvious
4. Then somebody else thought of it 20 years earlier!

You know you are treading on new ground when even the Artificial Life folks think that your ideas are rather exotic! So this is your chance to get in on the ground floor. You can be the first softwarephysicist in your cube farm if you act now!

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston

SoftwarePhysics

Have you ever wondered why your IT job is so difficult? Have you ever noticed that whenever we change software, performance can drastically decline? Have you observed that performance can drastically decline even when we don’t change software; that sometimes applications spontaneously get slow and then spontaneously return to normal response times without any intervention? Have you noticed that 50% of the time we never find a root cause for problems, and that we just start bouncing things at random until performance improves? Have you ever wondered why software behaves this way? Is there anything we can do about all this?

If you have had such thoughts, then you need to take a Twilight Zone trip into cyberspacetime and take a look at the material for my course for IT people called SoftwarePhysics 101 – The Physics of Cyberspacetime . Softwarephysics is an approach to software development, maintenance, and support based upon concepts from physics and biology that I have been using for over 25 years. The purpose of softwarephysics is to explain why IT is so difficult, to suggest possible remedies, and to provide a direction for thought. In addition, several universities also now offer courses on Biologically Inspired Computing which cover the biological aspects of softwarephysics, and the online content for some of these courses can be found by Googling for Biologically Inspired Computing.

SoftwarePhysics 101 – The Physics of Cyberspacetime is now available on Google Docs. Please note that some of the formulas do not render properly, especially exponents which do not display as superscripts, so please use your imagination.

Part 1 - Part 1 of the original PowerPoint document.
Part 2 - Part 2 of the original PowerPoint document.
Entropy – A spreadsheet referenced in Part 1
BSDE – A 1989 document describing how to use BSDE - the Bionic Systems Development Environment - to grow applications from genes and embryos within the maternal BSDE software.

Before proceeding further, let me describe a typical IT scenario that might help explain the utility of softwarephysics. I am currently in the Middleware Operations group supporting the website for a major corporation. The website is comprised of about 90 production servers – load balancers, firewalls, proxy servers, webservers, WebSphere Application Servers, CICS Gateway servers to mainframes, and mailservers. Every few weeks, the website will become extremely slow and all sorts of symptoms will begin to emerge on the servers. For example, I might initially see that the number of connections to the Oracle databases on one of the eight WebSphere Application Servers suddenly rise and WebSphere begin to get timeouts for database connections. This symptom may then spread to the other seven WebSphere servers causing the number of connections on an internal firewall to max out. Transactions begin to back up into the webservers and eventually max out the load balancers in front of the webservers. The whole architecture of 90 servers can spin out of control and grind to a halt within a period of several seconds to several hours, depending upon the situation. When this happens, our internal monitors will detect a mounting problem and will page out the appropriate people to jump onto the problem. A conference call will be convened and perhaps 10 people will begin looking for a root cause for the problem. Was some new application code released into production last night? Were the WebSphere configuration parameters changed? Was Oracle upgraded with a patch from the vendor? Are we under a hacker attack? While this analysis is proceeding, perhaps 100,000 people cannot use the website to check their balance or pay their bill electronically. It can get worse, when I was at United Airlines supporting www.ual.com, we would begin losing about $150/second because people could not book flights online. While on the conference call, there is a tension between finding a root cause for the escalating problem and bouncing the servers having problems. Bouncing is a technical term for stopping and restarting a piece of software or server to alleviate a problem, and anyone who owns a PC running the Windows operating system should be quite familiar with the process. The fear is that bouncing software may temporarily fix the problem, but the problem may eventually come back unless the root cause is determined. Also, it might take 30 minutes to bounce all of the affected servers and there is the risk that the problem will immediately reappear when the servers come back up. However, about 50% of the time we never find a root cause, and delaying bouncing can cause additional servers to spin out of control making recovery even more time consuming. As Ilya Prigogine has pointed out, cause and effect get pretty murky down at the level of individual transactions. In the chemical reaction

A  +  B  ↔   C

at equilibrium, do A and B produce C, or does C disassociate into A and B? We have the same problem with cause and effect in IT when trying to troubleshoot a large number of servers that are all in trouble at the same time. Is a rise in database connections a cause or an effect? Unfortunately, it can be both depending upon the situation. For the IT Operations department of a major corporation, the above scenario happens to several major systems on a daily basis, and this sort of activity has been going on for about 50 years in IT.

Since software and hardware are a product of human intelligence, IT people naturally think they understand what is going on at a reductionist metaphysical level. But understanding that software eventually translates into machine instructions playing quantum mechanical tricks with iron atoms on disk drives and silicon atoms in chips does not help with making day-to-day decisions in IT. What was needed was a pragmatic effective theory of software, and that is what softwarephysics is all about.

Outlined below are some of my adventures as a softwarephysicist exploring the fascinating physics of cyberspacetime. Softwarephysics is a fun, but useful, combination of physics, biology, and computer science that I have been using for more than 25 years, that might be of interest to you, if not entirely in a serious manner, perhaps at least in an entertaining one. Softwarephysics is a simulated science for the simulated software universe that we are all immersed in.

The Origin of Softwarephysics
In 1979, I was a happy exploration geophysicist exploring for oil in the Gulf of Suez out of Amoco’s Chicago exploration office. At the time, I was writing FORTRAN code for geophysical models to simulate the observed data from seismic surveys. Then one sad day, I learned that the whole Chicago exploration office was being transferred to Houston. Just six months prior to this announcement, I had left a really great job in Shell’s exploration office in Houston in order to return my family to our hometown of Chicago. To get myself out of this mess, I finagled my way into Amoco’s IT department supporting geophysical application software. After a couple of months in the IT department, I came to two conclusions:

1. Doing IT for a living was a lot harder than being an exploration geophysicist.

2. It seemed like these strange IT people had created their own little universe, or
at least a pretty good computer simulation of their own little universe.

It seemed like I was trapped in a frantic computer simulation, buried in punch card decks and fan-fold listings. I began to think of this computer-simulated universe as the "Software Universe". The Software Universe is a bit like Edwin A. Abbott’s Flatland, only a little more tangible. Flatland is a delightful little story written in 1884 about a charming little 2-dimensional universe called Flatland and its strange inhabitants that you can read about on the Internet. I soon noticed that the Software Universe was not as chaotic as it first appeared. It seemed to follow some "laws", and it seemed like I had seen many of these "laws" before when studying physics. The longer I studied the Software Universe, the more I became convinced that the IT community had accidentally produced a pretty decent computer simulation of the physical universe, just like I used to do on purpose when writing FORTRAN code to simulate geophysical models of seismic data. Soon I realized that I could use this accidental simulation in reverse. By understanding how the physical universe behaved, I could predict how the Software Universe would respond to stimuli. So to help myself cope with the daily mayhem of life in IT, I developed a scientific approach to software during my first few years that I called softwarephysics. I figured if you could apply physics to geology; why not apply physics to software? This was a bit of a role reversal. In physics, we use software to simulate the behavior of the universe, while in softwarephysics we use the universe to simulate the behavior of software.

The Software Universe is a 2-dimensional cyberspacetime universe consisting of a time dimension and processor space dimension. Cyberspacetime is not a continuum - both dimensions are quantized. The cyberspace dimension is defined by the 1 – 10 trillion currently active discrete microprocessors, wherever they might be, and the individual system clocks of each microprocessor quantize the time dimension. Cyberspacetime is not fixed and absolute because processes can jump from one processor to another and the system clocks all measure local time and have different drift rates. The Software Universe is not made of things; it consists only of processes and flows of information between discrete events in the two dimensions of cyberspacetime. There is the illusion that the Software Universe is filled with real things, such as files and databases with purchase orders and inventory levels on them, but this is an illusion. For example, when you edit a file with notepad, you are not interacting with a file; you are interacting with a process. Just press CTRL-ALT-DEL to open Windows Task Manager, click on the Processes tab and look for notepad.exe. The PID is the process ID for the notepad process you are interacting with. The CPU Time is the distance the notepad process has traveled along the time dimension of cyberspacetime measured in dedicated CPU cycles to the notepad process. Nothing ever interacts with "things" in cyberspacetime because the "things" have to first be read into the memory of a computer and placed under the control of a process. Only the processes interact with each other in cyberspacetime. Therefore files and databases on disk drives, tapes, and CDs are not part of the Software Universe; they are part of the physical universe. In the Software Universe, we have two processes called "read" and "write" which allow us to pass information into and out of cyberspacetime. The Software Universe began about 1.9 billion seconds (60 years) ago as a few bytes of machine code on a single computer, and has expanded and evolved into the complex Software Universe we see today, consisting of millions of terabytes of software residing on trillions of microprocessors. In softwarephysics, we use this unintentional simulation in reverse to help us understand the nature of software by observing how the physical universe operates.

Currently the physics community is hard at work trying to unify general relativity, the physics of large velocities and large masses, with quantum mechanics, the physics of tiny things. One of the promising approaches is called loop quantum gravity. Loop quantum gravity proposes that space and time are not continuous, but are quantized into tiny discrete chunks and that all of the particles and forces in the physical universe are really illusions. Loop quantum gravity postulates that all tangible things are merely a set of interacting processes that run with a clock speed of 10⁴³ Hz. So the Software Universe seems to do a fair job of simulating a universe built on loop quantum gravity only with a clock speed of about 10⁹ Hz.

Softwarephysics is the study of cyberspacetime. Softwarephysics begins by visualizing software as a virtual substance in the Software Universe. You then follow the normal steps a theoretical physicist would take when studying any new substance or phenomena. You gather some observations on how software seems to behave, develop a model that explains that behavior, and then try to test your model as best you can. Nearly everybody in IT has developed a model, or an intuitive feeling, for how software seems to behave in the universe. Softwarephysics is simply an explicit articulation of a physical model for software behavior. During the early 1980s, I developed a model that described software behavior in terms of a nonlinear system of quantum particles besieged by the second law of thermodynamics.

Physicists call the study of the macroscopic properties of matter thermodynamics. For example, a gas in a cylinder has certain macroscopic properties such as a volume, a pressure, and a temperature. Software can also be viewed macroscopically as the functions the software performs, the speed with which those functions are performed, and the stability and reliability of its performance. Physicists call the study of the microscopic properties of matter quantum mechanics. Quantum mechanics is what makes the transistors in your PC work and predicts that all matter exists as particles in discrete quantum states of energy and momentum. Quantum mechanics tells us that the universe is digital and not analog. Similarly, software can also be viewed microscopically as the interaction of a large number of bytes, each byte existing in one of 256 discrete quantum microstates. As with quantum mechanics, the macroscopic properties of software can be thought of as the sum total of the microscopic interactions of a large number of bytes in fixed quantum states. For a practical example, let us compare the college student’s favorite molecule, ethyl alcohol, with a line of code.

    H   H
    │   │
H-C-C-OH        for (i=0; i<10;i++)
    │   │
    H  H

Each atom in the molecule of ethyl alcohol has a set of electrons in fixed quantum states. Physicists express this concept in terms of a quantum wavefunction for each atom. Similarly, the line of code consists of a set of characters (atoms), with each character in one of 256 possible quantum ASCII states. The atoms in the ethyl alcohol molecule bond together into a molecule, which has its own quantum wavefunction. Large numbers of the resulting molecule interact with the "operating system" of the physical universe through the electromagnetic force to produce a clear liquid with a low boiling point that produces the desired effect when placed into the hands of a college student. After compilation, the properly formed line of code also interacts with an operating system to produce a desired effect.

Because of these close similarities to matter in the real world, software also is subject to the equivalent of a second law of thermodynamics - software naturally tends to a state of increased entropy, or disorder, and an attendant decrease in information content whenever software is worked on by programmers. The entropy of software has a tendency to increase because the number of incorrect versions of a piece of software vastly outnumber the correct versions of a piece of software. Going back to the above example, you can compare a methyl alcohol molecule to a line of code with a bug in it.

    H
    │
H-C-OH        for (i=0; i<0;i++)
    │
    H

(Note: the correct line of code will execute a loop of code 10 times, while the line of code with the bug in it will only execute the loop once because the programmer left out the "1" in the "10".)

The methyl alcohol molecule also produces a clear liquid with a low boiling point and the desired effect of inebriation, but it makes you go blind. Living things have to form complex organic molecules from simple atoms to perform the functions of life. In a similar fashion, programmers must assemble complex patterns of characters into lines of code to instruct a computer to perform useful functions. Both must deal with the second law of thermodynamics that demands the total entropy (disorder) of the universe must increase whenever a change is made. However, both living things and programmers are able to construct low entropy molecules and lines of code by dumping entropy into disordered kinetic energy, also known as heat. However, living things do this on a much grander scale and with much greater accuracy than any programmer.

Software also seems to exhibit nonlinear behavior because small changes to software, or the processing load that the software handles, can cause extreme fluctuations in the macroscopic behavior of the software. If you have ever driven on an expressway at rush hour, you have enjoyed the experience of dealing with a nonlinear system. Drivers watching a man change a tire 200 feet away across the median strip can cause a massive traffic jam on your side of the highway. Thirty minutes after the man with the flat has left the scene, the traffic jam still persists, and the luckless victims wonder what the root cause of the traffic jam was all about. Linear systems are like a highway with very little traffic. Small changes in the traffic density will cause small changes in the average traffic velocity when traffic is light. As the traffic density rises, there comes a tipping point, when the highway goes from linear to nonlinear behavior. After the tipping point, small changes to traffic density can cause drastic changes to throughput. The traffic becomes chaotic and lurches forward in spasms like beer going glug.. glug.. glug.. out of a bottle tipped over too far. Because nonlinear differential equations cannot generally be solved with calculus, scientists and engineers tended to entirely ignore nonlinear systems throughout history, until the 1970s, when computers appeared that could provide numerical solutions for nonlinear differential equations. Only then did scientists and engineers begin to realize the strange behavior of nonlinear systems through the invention of chaos theory.

Scientific Models
So what is the value in having a model for software behavior? The value is that a model allows you to make decisions about software, and it provides a direction for thought. Scientific models are analogies or approximations of reality that explain observations and make predictions. Most physicists gave up searching for true reality at the close of the 19th century, when they discovered that Newtonian mechanics and classical electrodynamics were only approximations of reality and did not work at very large velocities or for very small things like atoms. Instead, they adopted a worldview called positivism, in which physics only seeks out models of reality - not reality itself. All models are approximations - all models are "wrong". But knowing that you are "wrong" gives you a great advantage over people who know that they are "right", because knowing that you are "wrong" allows you to seek improved models of reality. However, having an approximate model that is "wrong" is better than no model at all. After all, Newtonian mechanics did allow us to put men on the Moon. So how do you put a model for software behavior to use? Here is an example. Softwarephysics suggests that you should not be too timid about bouncing software. On a conference call for a website outage, there is always a tension between bouncing software to restore functionality and searching for the root cause of the problem in an effort to prevent future occurrences. Softwarephysics predicts that, like the man with the flat tire, many times the root cause of a problem may be long gone even though its impact still remains.

All of the above leads us to the fundamental problem of software:

The Fundamental Problem of Software

1. The second law of thermodynamics tends to introduce small bugs into software that are never detected through testing.

2. Because software is inherently nonlinear, these small bugs cause general havoc when they reach production.

3. But even software that is absolutely bug-free can reach a critical tipping point and cross over from linear to nonlinear behavior.

So what can we do about the fundamental problem of software? Softwarephysics proposes that since the most complex systems in the physical universe that deal well with the second law of thermodynamics and nonlinearity are living things, we should take a biological approach to software. In fact, we already have.

The Software Universe is populated by living things. In my youth, we called these living things "computer systems", but today we call them "Applications". The Applications exist by exchanging information with each other, and sadly, are parasitized by viruses and worms and must also struggle with the second law of thermodynamics and nonlinearity. Since the beginning of the Software Universe, the architecture of the Applications has evolved through a process of innovation and natural selection that has followed a path very similar to the path followed by living things on Earth. I believe this has been due to what evolutionary biologists call convergence. For example, as Richard Dawkins has pointed out, the surface of the Earth is awash in a sea of visible photons, and the concept of the eye has independently evolved more than 40 times on Earth over the past 600 million years to take advantage of them. An excellent treatment of the significance that convergence has played in the evolutionary history of life on Earth, and possibly beyond, can be found in Life’s Solution (2003) by Simon Conway Morris. Programmers and living things both have to deal with the second law of thermodynamics and nonlinearity and there are only a few optimal solutions. Programmers try new development techniques and the successful techniques tend to survive and spread throughout the IT community, while the less successful techniques are slowly discarded. Over time, the population distribution of software techniques changes.

Unstructured Period (1945 – 1972)
During the Unstructured Period, programs were monolithic structures with lots of branch or GOTO statements and very little internal structure. These programs were similar to the early prokaryotic bacteria that appeared over 4,000 million years ago on Earth and lacked internal structure. Bacteria consist essentially of a polysaccharide cell wall filled with a lot of spaghetti code organic molecules. Just as bacteria still flourish today, many of these programs are still in production. At United Airlines, my former employer, there are still many spaghetti code FORTRAN II applications that were optimized in the late 1960s that are still in production.

Structured Period (1972 – 1992)
During the Structured Period, structured programming techniques were adopted by the IT community, and the GOTO statements were replaced by subroutines and indented code with lots of internal structure like the eukaryotic structure of modern cells that appeared about 1,500 million years ago. Eukaryotic cells are found in the bodies of all complex organisms from single-celled yeasts to you and me and divide up cell functions amongst a collection of organelles (subroutines) such as mitochondria, chloroplasts, Golgi bodies, and the endoplasmic reticulum.

Object-Oriented Period (1992 – Present)
During the Object-Oriented Period, programmers adopted a multicellular organization for software in which programs consisted of many instances of objects (cells) that were surrounded by membranes studded with exposed methods (membrane receptors). Multicellular organisms appeared about 900 million years ago and send messages between cells (objects) by secreting organic molecules that bind to the membrane receptors on other cells and induce those cells to execute exposed methods. For example, your body consists of about 100 trillion independently acting cells, and not a single cell in the collection knows that the other cells even exist. In an object-oriented manner, each cell just responds to the organic molecules that bind to its membrane receptors, and in turn, sends out its own set of chemical messages that bind to the membrane receptors of other cells in your body. When you wake to the sound of breaking glass in the middle of the night, your adrenal glands secrete the hormone adrenaline (epinephrine) into your bloodstream, which binds to the getScared() receptors on many of your cells. In an act of object-oriented polymorphism, your liver cells secrete glucose into your bloodstream and your heart cells constrict harder when their getScared() methods are called.

Distributed Objects Period
Currently we are entering the Distributed Objects Period, which is very similar to the Cambrian Explosion. During the Cambrian Explosion, 541 million years ago, complex body plans first evolved, which allowed cells in multicellular organisms to make RMI calls on the cells of remote organs to accomplish biological purposes. In the Distributed Objects Period, we are using common EJB components in J2EE appservers to create Applications with complex body plans. The J2EE appservers perform the functions of organs like kidneys, lungs and livers. I am discounting CORBA here as a failed precursor because CORBA never became ubiquitous as EJB seems to be heading. In the evolution of any form of self-replicating information, there frequently are many failed precursors. There is a growing body of evidence beginning to support the geological "Snowball Earth" hypothesis that the Earth went through a period of 100 million years of extreme climatic fluctuations just prior to the Cambrian Explosion. During this period, the Earth seesawed between being completely covered with a thick layer of ice and being a hot house with a mean temperature of 140 ⁰F. It has been suggested that the resulting stress on the Earth's ecosystems sparked the Cambrian Explosion. In 1995, the IT community was also hit with a similar cataclysmic event called Java, which put extreme stress on the IT community and eventually sparked EJB.

It has taken the IT community nearly 60 years to develop a distributed objects architecture based upon multicellular organization. This was achieved through a slow evolutionary process via innovation and natural selection performed by millions of independently acting programmers. Granted, this occurred much faster than the three billion years nature took to come up with the same architecture, but we could have done this back in the 1960s if we had known better – after all, the object-oriented language Simula was developed in 1965. Softwarephysics proposes that we use concepts from biology to skip to solutions directly.

Applying biological concepts to software is not always obvious. The remainder of this posting is a rather long-winded case study from the 1980s that might get a bit tedious, but since it is not fully covered in SoftwarePhysics 101, I will tell the tale here. In 1985, while at Amoco, I figured that since the most complex nonlinear systems in the universe that dealt well with the second law of thermodynamics were living things, a biological or "bionic" approach to software development was the best approach to take. This did not happen all at once, as I shall explain later, but in 1985 I began developing an early mainframe IDE (Integrated Development Environment) called BSDE (Bionic Systems Development Environment) that developed software based upon concepts from molecular biology. During the 1980s BSDE, which was a direct application of softwarephysics, put several million lines of code into production at Amoco. BSDE generated a 10,000 line of code "embryo" for an application based upon the application’s "genes". The genes for an application were the DDL (Data Definition Language) statements used to create its DB2 relational database. (Note: In the 1980s IBM’s relational database was called SQL/DS on the VM/CMS operating system and DB2 on the MVS operating system – I will refer to them both simply as DB2 for the sake of clarity) Each embryo grew and differentiated into a unique individual application within BSDE by having a programmer turn on and off its set of unique genes to generate screens, reports, and SQL code. The first language was REXX and later BSDE generated PL/I and COBOL. Today these techniques are called wizards. The BSDE.txt file mentioned above describes the typical life cycle of an application developed in the 1980s using BSDE. It is best to view this file with a Fixedsys font so that the line printer graphics align properly.

Because BSDE generated applications using ISPF Dialog Manager screens and REXX, I was able to use BSDE to generate code for itself and have BSDE evolve through small incremental changes using concepts from evolutionary biology. The next generation of BSDE was grown inside of its maternal release. Over a period of seven years, from 1985 – 1992, more than 1,000 generations of BSDE were generated, and BSDE slowly evolved into a very sophisticated tool through small incremental changes.

The evolution of BSDE had an interesting issue. Like the origin of life on Earth, I had to deal with the bootstrap problem – how do you get started? BSDE began as a few simple ISPF edit macros running under ISPF edit. ISPF is the software tool that mainframe programmers still use today to interface to the IBM MVS and VM/CMS mainframe operating systems and contains an editor that can be greatly enhanced through the creation of edit macros written in REXX. I began BSDE by writing a handful of ISPF edit macros that could automate some of the editing tasks that a programmer needed to do when working on a program that used a DB2 database. These edit macros would read a Control File which contained the DDL statements to create the DB2 tables and indexes. The CREATE TABLE statements in the Control File were the equivalent of genes and the Control File itself performed the functions of a chromosome. For example, a programmer would retrieve a skeleton COBOL program, with the bare essentials for a COBOL/DB2 program, from a stock of reusable BSDE programs. The programmer would then position their cursor in the code to generate a DB2 SELECT statement and hit a PFKEY. The REXX edit macro would read the genes in the Control File and would display a screen listing all of the DB2 tables for the application. The programmer would then select the desired tables from the screen, and the REXX edit macro would then copy the selected genes to an array (mRNA). The mRNA array was then sent to a subroutine that inserted lines of code (tRNA) into the COBOL program. The REXX edit macro would also declare all of the SQL host variables in the DATA DIVISION of the COBOL program and would generate code to check the SQLCODE returned from DB2 for errors and take appropriate actions. A similar REXX ISPF edit macro was used to generate screens. These edit macros were also able to handle PL/I and REXX/SQL programs. They could have been altered to generate the syntax for any programming language such as C, C++, or Java. As time progressed, BSDE took on more and more functionality via ISPF edit macros. Finally, there came a point where BSDE took over and ISPF began to run under BSDE. This event was very similar to the emergence of the eukaryotic architecture for cellular organisms. BSDE consumed ISPF like the first eukaryotic cells that consumed prokaryotic bacteria and used them as mitochondria and chloroplasts. With continued small incremental changes, BSDE continued to evolve.

I noticed that I kept writing the same kinds of DB2 applications, with the same basic body plan, over and over. From embryology, I got the idea of using BSDE to read the Control File for an application and to generate an "embryo" for the application based upon its unique set of genes. The embryo would perform all of the things I routinely programmed over and over for a new application. Once the embryo was generated for a new application from its Control File, the programmer would then interactively "grow" code and screens for the application. With time, each embryo differentiated into a unique individual application until the fully matured application was delivered into production by BSDE. At this point, I realized that I could use BSDE to generate code for itself, and that is when I started using BSDE to generate the next generation of BSDE. This technique really sped up the evolution of BSDE because I had a positive feedback loop going. The more powerful BSDE became, the faster I could add improvements to the next generation of BSDE through the accumulated functionality inherited from previous generations.

Embryos were grown within BSDE using an ISPF split screen mode. The programmer would start up a BSDE session and run Option 4 – Interactive Systems Development from the BSDE Master Menu. This option would look for an embryo, and if it did not find one, would offer to generate an embryo for the programmer. Once an embryo was implanted, the option would turn the embryo on and the embryo would run inside of the BSDE session with whatever functionality it currently had. The programmer would then split his screen with PF2 and another BSDE session would appear in the lower half of his terminal. The programmer could easily toggle control back and forth between the upper and lower sessions with PF9. The lower session of BSDE was used to generate code and screens for the embryo on the fly while the embryo in the upper BSDE session was fully alive and functional. This was possible because BSDE generated applications that used ISPF Dialog Manager for screen navigation, which was an interpretive environment, so compiles were not required for screen changes. If your logic was coded in REXX, you did not have to do compiles for logic changes either, because REXX was an interpretive language. If PL/I or COBOL were used for logic, BSDE had facilities to easily compile code for individual programs after a coding change, and ISPF Dialog Manger would simply load the new program executable when that part of the embryo was exercised. These techniques provided a tight feedback loop so that programmers could immediately see the effects of a change as the embryo grew and differentiated.

This was still the early days of relational databases in IT, and I naively bought into the prevalent idea of the day that in the future we would be storing all the data for a corporation on a large number of shared DB2 tables, rather than having thousands of individual applications that all used their own stand-alone set of files. So I came up with the idea of the "corporate gene pool". I used BSDE to generate code for a new generation of BSDE that could read the DB2 System Catalog tables by splicing in the genes for the DB2 System Catalog tables into BSDE’s own Control File. This allowed BSDE to retrieve the genes for any existing DB2 application from the DB2 System Catalog. Now BSDE could create an embryo for a new application that was an amalgam of genes from existing applications with the addition of some new genes for the new DB2 tables required by the new application, and I actually started to generate applications that shared data, rather than relying on inter-application communication via extract files. Sadly, even today, we still usually write applications that "own" their own set of stand-alone relational tables.

BSDE was originally developed for my own use, but fellow programmers soon took note, and over time, an underground movement of BSDE programmers developed at Amoco. Encouraged by my supervisor, I began marketing BSDE and softwarephysics within Amoco. Being a rather conservative oil company, this was a hard sell, and I am not much of a salesman. By now it was late 1986, and I was calling their programmers "software engineers" and was pushing the unconventional ideas of applying physics and biology to software. Fortunately for me, all of Amoco's income came from a direct application of geology, physics, or chemistry, so many of our business partners were geologists, geophysicists, chemists, or chemical, petroleum, electrical, or industrial engineers, and that was a big help. Computer viruses began to appear about this time, and they provided some independent corroboration for the idea of applying biological concepts to software, and in 1987 we also started to hear some things about artificial life from Chris Langton out of Los Alamos, but there still was a lot of resistance to the idea back at Amoco. One manager insisted that I call genes "templates". I used the term "bionic" in the product name after I checked in the dictionary and found that bionic meant applying concepts from biology to solve engineering problems, and that was exactly what I was trying to achieve. There were also a couple of American television programs in the 1970s that introduced the term to the American public, The Six Million Dollar Man and The Bionic Woman, which featured superhuman characters performing astounding feats. In my BSDE road shows, I would demo BSDE in action spewing out perfect code in real time and about 20 times faster than normal human programmers could achieve, so I thought the term "bionic" was fitting. I am still not sure that using the term "bionic" was the right thing to do. IT was not "cool" in the 1980s, as it is in today’s ubiquitous Internet Age, and was dominated by very serious and conservative people with backgrounds primarily in accounting. However, BSDE was pretty successful and by 1989, BSDE was finally recognized by Amoco IT management and was made available to all Amoco programmers. A BSDE class was developed and about 20 programmers became active BSDE programmers. A BSDE COI (Community of Interest) was formed, and I used to send out weekly email newsletters about applying physics and biology to software to the COI members. In October 1991, an article on BSDE appeared as the cover story of the Enterprise Systems Journal.

Figure 1 – BSDE appeared as the cover story of the October 1991 issue of the Enterprise Systems Journal

During the period of 1985-1989, BSDE only ran under the IBM VM/CMS operating system. The VM/CMS operating system was an interactive operating system that IBM developed in the 1970s, which was somewhat similar to the Unix operating system being developed at Bell Labs at the same time. The other major IBM operating system was MVS, a batch operating system that was the direct descendant of the original 1965 IBM OS/360 operating system. In the 1970s, IBM added a time-sharing option to MVS called TSO. Now it just happened that MVS/TSO had all the software infrastructure components I needed for an MVS/TSO version of BSDE, but there were subtle syntax differences between the two operating systems. I realized that with some gradual incremental changes to the VM/CMS version of BSDE, I could evolve an MVS/TSO version. So I took the code for the VM/CMS version of BSDE and copied it over to MVS/TSO in a more or less brute force manner. This was very much like the first lungfish to crawl up onto the land. BSDE VM/CMS did not run very well under MVS/TSO because of the syntax differences, but it did just barely survive the transition. At this point the two versions of BSDE began to diverge. I kept evolving BSDE MVS/TSO to correct the syntax errors as they arose during use, and slowly BSDE MVS/TSO adapted to the new operating system. At the same time, I began to incrementally modify BSDE VM/CMS so that programmers could develop applications on VM/CMS and then port them to MVS/TSO. The computer charge rate for VM/CMS was substantially less than for MVS/TSO, so it was much cheaper to grow an embryo within BSDE VM/CMS and then port the nearly fully grown larval stage embryo to MVS/TSO for the final delivery into production. BSDE MVS/TSO could then be used to maintain the fully-grown adult MVS/TSO application. So BSDE generated MVS/TSO applications took on a two-stage life cycle. The bulk of their development took place within BSDE VM/CMS as a larval stage embryo in an environment where computer charges were cheap and the living was easy. The adult stage application fluttered about on MVS/TSO fulfilling its purpose in life.

Because all BSDE generated applications had the same biochemistry down at the coding level, the effort to maintain BSDE generated applications was much less than for normal applications. Maintenance is a big problem in IT because 80% of your budget goes to supporting old applications that may be 10 years old and which are littered with the historical programming styles of dozens of programmers who have long since gone. Once a programmer became familiar with one BSDE generated application, it was very easy for the programmer to support other BSDE applications for the same reason that we do not have hundreds of different kinds of veterinarians taking care of domestic animals.

I wish that I could claim that I was smart enough to have sat down and thought up all of this stuff from first principles, but that is not what happened. It all just happened through small incremental changes over a very long period of time and most of the design work was done subconsciously, if at all. Even the initial BSDE ISPF edit macros happened through serendipity. When I first started programming DB2 applications, I found myself copying in the DDL CREATE TABLE statements from the file I used to create the DB2 database into the program that I was working on. This file, with the CREATE TABLE statements, later became the Control File used by BSDE to store the genes for an application. I would then go through a series of editing steps on the copied in data to transform it from a CREATE TABLE statement into a DB2 SELECT, INSERT, UPDATE, or DELETE statement. I would do the same thing all over again to declare the host variables for the program. Being a lazy programmer, I realized that there was really no thinking involved in these editing steps and that an ISPF edit macro could do the job equally as well, only very quickly and without error, so I went ahead and wrote a couple of ISPF edit macros to automate the process. I still remember the moment when it hit me. For me it was very much like the scene in 2001 - A Space Odyssey, when the man-ape picks up the wildebeest thighbone and starts to pound the ground with it. My ISPF edit macros were doing the same thing that happens when the information in a DNA gene is transcribed into a protein! A flood of biological ideas poured into my head over the next few days, because at last I had a solution for my pent-up ideas about nonlinear systems and the second law of thermodynamics that were making my life so difficult as a commercial software developer. We needed to "grow" code – not write code!

So that is how BSDE got started by accident. Since I did not have any budget for BSDE development, and we charged out all of our time at Amoco to various projects, I was forced to develop BSDE through small incremental enhancements that always made my job on the billable projects a little easier. I new that was how evolution worked, so I was not too concerned. In retrospect, this was a fortunate thing. In the 1980s, the accepted theory of the day was that you needed to prepare a very thick user requirements document for a new application and then you would code from the requirements as a blueprint. This was before the days of prototyping and development through small incremental releases that you find today. In the mid 1980s, prototyping was still a heretical proposition in IT. But I don’t think that I could have built BSDE using the traditional blueprint methodology of the day.

Unfortunately, the early 1990s saw the downfall of BSDE. The distributed computing model hit with full force, and instead of deploying applications on mainframe computers, we began to distribute applications across a network of servers and client PCs. Since BSDE generated applications for mainframe computers, it could not compete, and BSDE quickly went extinct in the minds of Amoco IT management. I was left with just a theory and no tangible product, and it became much harder to sell softwarephysics at that point. So after a decade of being considered a little "strange", I decided to give it a rest. But I made a promise to myself. I would let 25 years go by, until the year 2004, and if I did not see softwarephysics appear elsewhere in the IT community, I would give it another shot.

Anyway, over the years, I have found softwarephysics to be very useful and also a lot of fun, and maybe you might too.

Comments are welcome at scj333@sbcglobal.net

To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/

Regards,
Steve Johnston