We are all becoming more reliant on Advanced AI each day to do the more mundane and laborious tasks for us, and I am certainly no exception. So to begin this post, I would like to defer to ChatGPT to begin by doing some of the initial heavy lifting.
Me: Tell me about the AI Intelligence Explosion.
ChatGPT:
The AI Intelligence Explosion is a concept from artificial intelligence and futurism that describes a runaway process of rapidly accelerating intelligence, potentially leading to superintelligent AI far beyond human capability in a very short time.
The idea is most closely associated with mathematician and computer scientist I. J. Good, who articulated it in 1965.
Core Idea
At its heart, the intelligence explosion is a feedback loop:
1. Humans build an AI that is at least as good as humans at AI research.
2. That AI improves its own design.
3. The improved AI is even better at improving itself.
4. The cycle repeats, each time faster and more powerful.
If this loop accelerates quickly enough, intelligence could increase exponentially, rather than gradually.
I. J. Good’s Original Formulation
Good famously wrote:
“The first ultraintelligent machine is the last invention that man need ever make.”
His reasoning:
Once machines outperform humans at general reasoning and innovation, human-led technological progress becomes obsolete.
The AI would drive its own improvements.
Why It’s Called an “Explosion”
The term does not imply a literal explosion, but a phase transition:
- Before: slow, human-limited progress
- After: extremely rapid, machine-driven progress
This transition is sometimes called:
- Hard takeoff (very fast)
- Soft takeoff (slower, more controlled)
Key Assumptions Behind the Concept
The intelligence explosion depends on several assumptions - each debated:
1. Recursive Self-Improvement Is Possible. The AI must be able to meaningfully improve:
- Its algorithms
- Its architecture
- Its training methods
- Possibly even its hardware designs
2. Improvements Compound
- Each improvement makes the AI better at making further improvements.
3. Intelligence is the Main Bottleneck. Progress is limited primarily by intelligence, not:
- Energy
- Hardware availability
- Economic constraints
- Physical laws
Hard vs. Soft Takeoff
Hard Takeoff
- Intelligence jumps from human-level to vastly superhuman in days, hours, or minutes
- Humans have little time to respond
- Often associated with existential risk discussions
Soft Takeoff
- Improvement happens over years or decades
- Society adapts alongside the AI
- Allows time for regulation, alignment, and integration
Why People Worry About It
Alignment Risk
- If a superintelligent AI’s goals are not aligned with human values, even small misspecifications could lead to catastrophic outcomes.
- Classic examples:
- “Paperclip maximizer” (optimize a trivial goal at all costs)
- Instrumental convergence (acquiring power/resources regardless of intent)
Control Problem
Once intelligence vastly exceeds human levels:
- Monitoring becomes ineffective
- Correction may be impossible
- Human oversight could be symbolic rather than real
Skeptical Views
Many researchers argue the intelligence explosion is unlikely or overstated:
- Intelligence may not be a single scalable dimension
- Self-improvement may face diminishing returns
- Physical and economic constraints could slow progress
- Real-world intelligence depends heavily on data, embodiment, and infrastructure
Some prefer terms like:
- Intelligence amplification
- Gradual AI scaling
- Continuous deployment
Relationship to the Singularity
The intelligence explosion is a mechanism often cited as leading to the technological singularity:
- A point beyond which human prediction becomes unreliable
- AI-driven progress outpaces human comprehension
- Not all singularity theories rely on intelligence explosion—but many do.
In Short
- The AI Intelligence Explosion is the idea that AI could rapidly surpass human intelligence through recursive self-improvement.
- It could be fast or slow, beneficial or dangerous.
- The concept underpins much of the modern debate on AI alignment, safety, and governance.
A very interesting YouTube channel covering how an AI Intelligence Explosion could go awry can be found at:
Species | Documenting AGI
https://www.youtube.com/@AISpecies/featured
The AI Intelligence Explosion is currently underway, and it seems that there are no practical worldwide control measures in place to prevent a Hard Takeoff from doing a great deal of damage in the process. In fact, a sufficiently damaging Hard Takeoff might be enough to actually snuff out the AI Intelligence Explosion. As I explained in Are The Coming ASI Machines An Existential Threat To Mankind?, we human DNA survival machines should be more concerned about the reactions of our fellow human DNA survival machines to the rise of the coming ASI Machines than we should be about the ASI Machines themselves. So, how do we take measures to ensure a Soft Takeoff?
It seems that all of the current Advanced AI development centers are trying to use classical Control Theory to control the AI Intelligence Explosion if they are taking any measures at all. They all seem to be using principles from classical Control Theory to ensure that their AI models are "aligned" with human values by doing human reinforcement training of models so that they do not tell you how to plan for a mass shooting or how to build improvised explosives. As we have seen, many times, such AI safety efforts are easily foiled by properly posed prompts that "jailbreak" such safety constraints. So, this does not seem to be working very well by all reports. Consequently, in this post, I would like to suggest that this failure of classical Control Theory is failing because classical Control Theory was never meant to control explosives. Perhaps a better approach would be to study how explosives are safely controlled. But before doing that, let us first review the classical Control Theory that all the AI centers are now trying to use.
The Fundamentals of Classical Control Theory
Again, let me use ChatGPT to do the heavy lifting.
Figure 1 - Classical Control Theory uses feedback loops to control processes.
Me: Tell me about classical Control Theory
ChatGPT
"Control theory" is a branch of engineering and applied mathematics concerned with making systems behave in a desired way—automatically, reliably, and stably—despite disturbances and uncertainty. It underpins everything from cruise control in cars and aircraft autopilots to industrial robots, power grids, and even economic or biological models.
Core idea: feedback
At the heart of control theory is "feedback":
1. "Measure" the system’s output (what actually happens).
2. "Compare" it to a desired value (the -reference- or -setpoint-).
3. "Adjust" the input to reduce the error.
This closed-loop process lets systems correct themselves.
Key concepts (plain-language)
- "Plant": the system being controlled (engine, robot arm, chemical reactor).
- "Controller": the algorithm that decides how to adjust inputs.
- "Setpoint": the target value (speed, temperature, position).
- "Error": difference between setpoint and actual output.
- "Stability": whether the system settles down instead of oscillating or blowing up.
- "Transient vs. steady-state": short-term response vs. long-term behavior.
- "Disturbances": outside influences (wind, load changes, noise).
Classical control (frequency & time-domain)
This is the traditional framework, widely used in industry.
- "PID controllers" (Proportional–Integral–Derivative)
- P: reacts to current error
- I: eliminates long-term bias
- D: anticipates future error
Simple, robust, and extremely common.
- "Transfer functions & Laplace transforms"
Describe system dynamics mathematically and analyze stability.
- "Bode plots, Nyquist plots, root locus"
Graphical tools to see how systems respond and how close they are to instability.
Modern control (state-space)
Developed in the mid-20th century for complex, multi-variable systems.
- "State variables" represent the system’s internal condition.
- "State-space models" handle multiple inputs/outputs naturally.
- "Optimal control" (e.g., LQR) balances performance against effort.
- "Observers/Kalman filters" estimate internal states from noisy measurements.
This approach dominates aerospace, robotics, and advanced automation.
Nonlinear & advanced control
Real systems are often nonlinear.
- "Nonlinear control": feedback linearization, Lyapunov methods.
- "Adaptive control": controller adjusts itself as the system changes.
- "Robust control": guarantees performance despite uncertainty.
- "Model Predictive Control (MPC)": optimizes future behavior subject to constraints—very popular in process industries and autonomous systems.
Where you see control theory in daily life
- Thermostats and HVAC systems
- Vehicle stability control and cruise control
- Aircraft and spacecraft guidance
- Industrial automation and robotics
- Power grid frequency regulation
- Medical devices (insulin pumps, ventilators)
Why it matters
Control theory provides:
- "Predictability" (systems behave as expected)
- "Safety" (avoid unstable or runaway behavior)
- "Efficiency" (minimum energy or cost)
- "Automation at scale"
In short, it’s the mathematics of "making things behave".
Control Theory was developed many years ago to control machines and other physical processes with the use of negative and positive feedback loops. An example of a negative feedback loop is your home furnace and thermostat. When your house cools down below the lower set point of the thermostat, the thermostat sends a signal to your furnace to turn it on. As your house warms up, it eventually exceeds the upper set point of the thermostat, and then the thermostat sends a signal to your furnace to shut it down. In this way, your thermostat performs an automated regulation of the temperature within your house that keeps your house within a comfortable range of about 10F. Notice that your thermostat manages to accomplish this seemingly sophisticated control process with no built-in intelligence or software whatsoever. Negative feedback loops automatically keep systems in a stable equilibrium by simply pulling them back into a narrow range of operational conditions. That is the beauty of using negative feedback loops to control complex systems: they lead to inherently stable and self-regulated systems, without the need for an overly complex control mechanism. Here is another example of a negative feedback loop in action. Extend both of your index fingers out in front and carefully place each end of a ruler on top of your two outstretched fingers. Now slowly bring your fingers together, letting the ruler slide freely on top of each finger, as you bring your fingers closer together. As you do so, you will find that the ruler always remains balanced upon your fingers, and you will end up with both fingers together precisely at the midpoint of the ruler. Try as you may, you will not be able to obtain any other result. You will always end up with the ruler carefully balanced at its midpoint on your two fingers when your two fingers have finally met in the center.
Explosives Use Positive Feedback Loops
Explosives use positive feedback loops that do just the opposite of negative feedback loops. Positive feedback loops cause systems to explode out of control by using the amplified output of the feedback loop as the input to the feedback loop and thus causing the output to quickly expand out of control. Explosives, like nitroglycerine, are therefore prime examples of positive feedback loops in action. When nitroglycerine detonates, the liquid nitroglycerine rapidly decomposes via the chemical reaction:
4 C3H5(NO3)3 → 12 CO2 + 10 H2O + 6 N2 + O2
Figure 2 - The nitroglycerin molecule is composed of hydrogen, nitrogen and oxygen atoms that are all weakly bound together. When nitroglycerin detonates, all these atoms become very hot gases that rapidly expand as they form molecules with higher chemical binding energies.
Figure 3 - When nitroglycerin detonates, all of its atoms turn into very hot gases at 5,000 0K, which is about the temperature of the surface of the Sun.
Notice that the liquid nitroglycerine turns completely into gases at the temperature at which nitroglycerine detonates. This alone generates gases that would occupy more than 1,200 times the original volume of the liquid at normal room temperature and pressure. However, because the above chemical reaction is also very exothermic, meaning that it gives off large amounts of heat energy, the generated gases are raised to a temperature close to that of the surface of the Sun, about 5,000 0K, which causes the generated gases to further greatly expand, creating a tremendous blast. When nitroglycerine detonates, chemical potential energy is quickly released because the weak bonds of the atoms in the large nitroglycerine molecules rearrange to form new smaller molecules with stronger bonds, such as N2, H2O, and CO2, releasing large quantities of chemical potential energy in the process. The nitroglycerin molecule contains plenty of its own oxygen atoms, so it does not need air to detonate. It carries its own oxidizing agent internally. Since higher temperatures make chemical reactions run even faster, when nitroglycerine detonates, the initial decomposition of a small number of molecules causes a pressure wave to form that detonates the nitroglycerine in the immediate surrounding area. The resulting self-sustained shock wave propagates through the nitroglycerine at 30 times the speed of sound, as a near instantaneous pressure induced chemical reaction that transforms the small volume of liquid nitroglycerine into a huge quantity of white-hot gas at a temperature of 5,000 0K. This is what makes nitroglycerine such a powerful explosive.
Figure 4 - Nitroglycerin was a very unstable and highly dangerous explosive. Notice the 19th-century warning label above that was intended for would-be criminals.
DANGER! DROP IT AND DIE!
EXPLOSIVE FOR MINING & INDUSTRIAL USE ONLY
NOT FOR USE IN BLOWING SAFE DEPOSIT BOXES OR POSTAL TRAIN CAR DOORS.
Nitroglycerin was first synthesized by the chemist Ascanio Sobrero in 1847, but the main problem with nitroglycerine was that the slightest shock could cause it to easily detonate. In 1867, Alfred Nobel came up with an improvement to nitroglycerin that vastly improved its stability, and therefore its utility. By simply mixing three parts of nitroglycerin with one part of diatomaceous earth and a small amount of sodium carbonate, Nobel created a very stable solid explosive that we now call dynamite. Dynamite rapidly became very popular because it could be rolled up into convenient paperwrapped sticks that were very easily loaded into the drill holes used for blasting rock during mining operations, and dynamite was also very resistant to accidental detonation from small shocks. In fact, in order to detonate dynamite, one must use a blasting cap or a high-temperature fuse to set it off.
Figure 5 - By simply mixing three parts of nitroglycerin, with one part of diatomaceous earth, and a small amount of sodium carbonate, Nobel created a very stable solid explosive, which we now call dynamite.
Figure 6 - Dynamite could be rolled into paper-covered sticks that could be safely handled and easily loaded into drill holes for mining purposes.
Figure 7 - In order to detonate dynamite, one needs the detonation from a high-explosive blasting cap or a burning high-temperature fuse (click to enlarge).
Figure 8 - Negative feedback loops regulate systems, while positive feedback loops cause systems to explode (click to enlarge).
The differences between positive and negative feedback loops can be further illustrated with the use of a few diagrams. On the left side of Figure 8, we see a system composed of a marble and a bowl in a state of stable equilibrium under the influence of a negative feedback loop. Jiggling the system with a small perturbation always results in the marble safely returning to the center of the bowl. The shape of the bowl can also be thought of as a plot of the potential energy function of the system. The combined system of the marble and the bowl has a minimum potential energy when the marble is safely in the center of the bowl, and the negative feedback loop has a tendency to keep the marble there. On the right side of Figure 8, on the other hand, we see a similar system composed of a marble on top of an overturned bowl that is an example of a positive feedback loop in an unstable equilibrium. Like nitroglycerine, the slightest addition of energy to the system, by perturbing the system with a little jiggle, will cause the marble to roll off the overturned bowl with increasing velocity, as it quickly turns all of its potential energy into kinetic energy, like the atoms in a container of nitroglycerine rapidly turning their chemical potential energy into the kinetic energy of molecular motion, also known as heat energy.
What Alfred Nobel did was to superimpose a small negative feedback loop on top of the very large positive feedback loop of nitroglycerine, by mixing nitroglycerine with a small amount of stabilizing diatomaceous earth and sodium carbonate. In Figure 9, we see the resulting plot of the potential energy function for dynamite, consisting of the large positive feedback loop of nitroglycerine, with a small dimple of a negative feedback loop on top. Now, in order for the marble to release its vast amount of potential energy, it needs a quantity of activation energy to get the marble over the hump at the top of the curve. This is why dynamite requires the detonation of a blasting cap or the heat from a high-temperature fuse in order to detonate.
Figure 9 - In 1867, Alfred Nobel discovered that by adding some diatomaceous earth to nitroglycerin, he could superimpose a small negative feedback loop on top of the much larger positive feedback loop of nitroglycerin (click to enlarge).
So How Does Diatomaceous Earth Stabilize Nitroglycerin?
Once again:
Me: How does diatomaceous earth stabilize nitroglycerin in dynamite?
ChatGPT:
Diatomaceous earth stabilizes nitroglycerin in dynamite "by physically immobilizing it", not by chemically neutralizing it. Here’s how that works, step by step:
1. What makes nitroglycerin so dangerous? Nitroglycerin is a powerful explosive because it:
- Is a "dense liquid"
- Is "highly sensitive" to shock, friction, and rapid temperature changes
- Can "detonate from small mechanical disturbances"
In its pure liquid form, even a small jolt can cause catastrophic decomposition.
2. Diatomaceous earth is:
- A soft, powdery material made of "fossilized diatom shells"
- Composed mostly of "amorphous silica"
- Extremely "porous", with microscopic cavities and channels
Those pores are the key.
3. Physical absorption and immobilization
When nitroglycerin is mixed with diatomaceous earth:
- The liquid "soaks into the microscopic pores"
- Capillary forces "hold the nitroglycerin in place"
- The explosive is "spread out as thin films", not a free-flowing liquid
This prevents:
- Sloshing
- Pooling
- Sudden pressure concentration
In effect, the nitroglycerin is “locked” inside a solid matrix.
4. Reduced sensitivity to shock and friction
Because the nitroglycerin:
- Cannot move freely
- Is distributed over a large surface area
- Experiences mechanical forces more evenly
…it becomes "much less sensitive to accidental detonation" during:
- Handling
- Transport
- Moderate impact
Importantly, the explosive power remains intact — it just becomes controllable.
5. Why it still explodes when detonated
Diatomaceous earth:
- Is "chemically inert"
- Does "not react" with nitroglycerin
- Does "not inhibit detonation once initiated"
A blasting cap provides a strong enough shock wave to:
- Overcome the physical damping
- Trigger the rapid decomposition of nitroglycerin throughout the absorbed mass
So the stabilization is "mechanical, not chemical".
6. Historical significance
This discovery by "Alfred Nobel" in the 1860s:
- Transformed nitroglycerin from a laboratory hazard into a usable industrial explosive
- Made modern mining, tunneling, and construction possible
- Directly led to the invention of "dynamite" and Nobel’s later legacy
In one sentence:
"Diatomaceous earth stabilizes nitroglycerin by absorbing it into microscopic pores, preventing sudden movement and impact sensitivity while preserving its explosive power when deliberately detonated."
Figure 10 - Diatomaceous earth is a soft absorbent powder made of diatom fossils.
Figure 11 - Diatoms are microscopic, single-celled algae found in oceans, lakes, rivers, and soils. Each diatom is encased in a rigid shell called a frustule, made of silica (SiO2). The frustule consists of two halves that fit together like a petri dish. These shells are intricately patterned with pores, ridges, and symmetry, making diatoms famous for their beauty under microscopes.
Figure 12 - A silica tetrahedron is composed of a central silicon atom surrounded by four oxygen atoms. A single silica tetrahedron has a net charge of -4.
Figure 13 - However, silica tetrahedra can form an amorphous chemically inert 3D framework, which we call quartz glass, by having the surrounding oxygen atoms share electrons. This forms the very strong material that diatom shells are made from.
Conclusion
So instead of trying to use classical Control Theory to "align" the AI Intelligence Explosion with external reinforcement training, a better approach might be to build into the LLM models some kind of stabilizing agent like diatomaceous earth to tame the AI Intelligence Explosion into a Soft Takeoff. For how that could be done, I have no idea. But the taming of nitroglycerin with diatomaceous earth might give some in the AI community some sense of direction.
Comments are welcome at scj333@sbcglobal.net
To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/
Regards,
Steve Johnston
