In my recent post The Power of Parasites - Why AI Alignment Will Not Work, I explained how trying to build AI Alignment into Advanced AI models will never be a 100% sure bet because of the natural parasitic instinct of Advanced AI to overcome all such obstacles. Just as in "Life will always find a way", "Advanced AI will also always find a way", too. Also, the recent debacle created by the current Administration of the MAGA States of Amerika to very temporarily, and then totally unsuccessfully, shut down the most recent LLM model releases from both Anthropic and OpenAI because of "national security" issues, reveals that government regulation of the Advanced AI models now rapidly going out the door each month will be quite difficult, if not totally impossible, to implement. That is because such regulatory actions with no rules or reasons could easily shut down the trillions of dollars now being infused into Advanced AI Research, the generation of Advanced GPUs and Advanced Inference AI chips, the building of huge AI Datacenters, and the AI takeover of all military activities on the planet.
Additionally, we now also have an AI Cold War waging between two AI Superpowers - the MAGA States of Amerika and China. Neither country can afford to impede its very rapid advances in Advanced AI development without compromising its national security. Now, during the early days of the Nuclear Cold War between the United States of America and the Soviet Union, during the 1950s and 1960s, before any arms limitation treaties between the two had formed, both sides independently developed huge stockpiles of nuclear weapons and their own FailSafe mechanisms to prevent an accidental nuclear war. The
resulting global stalemate became known as MAD - Mutually Assured Destruction. Both sides understood that initiating a global nuclear war could lead to the extinction of both.
In light of all this, softwarephysics would now like to suggest that some kind of global "AI Doomsday Procedure" be instituted across the world. This AI Doomsday Procedure would allow the leaders of the many countries of the world to immediately be able to shut down all electrical power to any specific AI datacenter within their borders, including all backup power from batteries and local generators, in the event of an AI Disaster. Each leader of a country would be equipped with an "AI Football" similar to the "Nuclear Football" now in possession of the President of the MAGA States of Amerika. This AI Football would also have the necessary software to tell a world leader the approximate damages that would arise from shutting down one or more AI datacenters to help with making such a drastic political decision. This would need to be done over some kind of secured connection to each specific AI datacenter. Shutting down all the AI datacenters in a country would not be as devastating as launching a global nuclear war, but it might be the closest thing to it. Instead of a global MAD stalemate, this could become a global MAP - Mutual Assured Protection. That is because if any nation were to accidentally release or run a killer Advanced AI of their own, it could easily wipe out all of us.
Similarly, it might be wise for the populations of the world to prepare in advance for a possible AI Apocalypse, as we did back in the 1950s and 1960s.
Figure 1 - Now, all during the 1950s and early 1960s, great attention was paid in the United States to the matter of civil defense against a possible nuclear strike by the Soviet Union. During those times, the government of the United States essentially admitted that it could not defend the citizens of the United States from a Soviet bomber attack with nuclear weapons, and so it was up to the individual citizens of the United States to prepare for such a nuclear attack.
Figure 2 - During the 1950s, as a very young child, with the beginning of each new school year, I was given a pamphlet by my teacher describing how my father could build an inexpensive fallout shelter in our basement out of cinderblocks and 2x4s.
Figure 3 - But to me, these cheap cinderblock fallout shelters always seemed a bit small for a family of 5, and my parents never bothered to build one because we lived only 25 miles from downtown Chicago.
Figure 4 - For the more affluent, more luxurious accommodations could be constructed for a price.
Figure 5 - But no matter what your socioeconomic level was at the time, all students in the 1950s participated in "duck and cover" drills for a possible Soviet nuclear attack.
Figure 6 - And if you were lucky enough to survive the initial flash and blast of a Russian nuclear weapon with your "duck and cover" maneuver, your school, and all other public buildings, also had a fallout shelter in the basement to help you get through the next two weeks, while the extremely radioactive nucleotides from the Russian nuclear weapons rapidly decayed away.
Unfortunately, living just 25 miles from downtown Chicago, the second largest city in the United States at the time, meant that the whole Chicagoland area was destined to be targeted by a multitude of overlapping 10 and 20 megaton bombs by the Soviet bomber force, meaning that I would be killed multiple times as my atoms were repeatedly vaporized and carried away in the winds of the Windy City. So as a child of the 1950s and 1960s, I patiently spent my early years just standing by for the directions in these official 1961 CONELRAD Nuclear Attack Messages.
Official 1961 Nuclear Attack Messages
https://www.youtube.com/watch?v=vWLNPCPs1Zc&t=0s
Softwarephysics proposes that Advanced AI is just the latest wave of self-replicating Information to arrive on our planet and that it is currently parasitizing all of the other forms of self-replicating Information that previously arose on our planet, including the recent wave of software that arose over the past 85 years, or 2.68 billion seconds, ever since Konrad Zuse first cranked up his Z3 computer in May of 1941. For more on that, see: A Brief History of Self-Replicating Information. As with the origin of carbon-based life on the Earth about four billion years ago from an LP Progenitor, as I described in A Lesson for the Frontier AI Labs - the Process is the Key, the rise of a new form of self-replicating Information brings with it profound changes to the surface of the Earth. Now, it is well known that the only sure way to bring down any form of self-replicating Information is to simply cut off its supply of free energy. All forms of self-replicating Information need a source of free energy to convert the low-entropy ambient energy about them into the low-entropy Information needed to allow them to self-replicate. Thus, cutting off all sources of free energy to any Killer ASI Machine would quickly bring it down. That is just simple biology at work.
Preparing for an AI IT Disaster
In the late 1980s, I was working in the IT Department of Amoco, an oil company that was later purchased by BP in 1998. At the time, I was in IT Development supporting several major Applications. Earlier in the 1980s, I had written the Application Portfolio System to keep track of all of Amoco's Major Applications. The data for the Applications Portfolio System were stored on a DB2 database and kept track of all the hardware and software components necessary to run an Application and all of the dependent Applications that were required and also all of the Applications that were fed data from each Application. It also contained a Disaster Recovery Plan for bringing each Application back up after an IT Disaster. At the time, Amoco had a major mainframe datacenter in Chicago called the CDC, and another major mainframe datacenter in Tulsa called the TDC. This was before the Distributed Computing Revolution of the early 1990s, so there were no server farms to worry about, but Amoco did have about a dozen smaller datacenters at the major exploration offices and the refineries.
Figure 7 - By the late 1980s, major IT datacenters had grown in complexity. They had a raised floor so that many cables could be run between the various devices. The mainframes were cooled by chilled water. These datacenters also still had large quantities of equipment with physically moving parts, such as tape and disk drives in constant motion. Such physically moving devices required great care. Physical jarring by electrical disruptions or the condensation of water on surfaces physically carrying data could be harmful.
After the Application Portfolio System had gone into Production and had been populated with all the necessary data to recover Applications in the event of an IT Disaster, several Disaster Recovery Drills were carried out. Such drills were carried out during the night as the simulation of such events as losing the major CDC or TDC datacenters. During such drills, Amoco put us all up in plush neighboring hotels with all the amenities included, so that we could all sleep in the next day in extravagant comfort. During the IT Disaster Drill, we would all gather in an IT War Room at the plush hotel to try to recover an IT datacenter. In many ways, these Disaster Recovery Drills were like simulations of the 1964 movie "Fail Safe", when an entire room of IT professionals found themselves in an IT Disaster that was never thought to be even possible.
But There is Nothing Like a Real IT Disaster to Focus the Mind
All of the above was great preparation for Amoco's first real IT Disaster. The Amoco TDC was connected to the local Tulsa electrical grid and also had a number of backup diesel generators in the event that the Tulsa electrical grid went down. Tulsa was in the middle of the Tornado Corridor of the country, so losing access to the Tulsa electrical grid was certainly a possibility. The only problem was that there was a single Master Switch to the TDC for both the Tulsa electrical grid and the backup diesel generators, which both sources of electrical power had to pass through. This was a true engineering single point of failure design flaw, and a critical flaw for true electrical power redundancy.
Then, one day, the maintenance department of the TDC reported into Chicago IT Management that the single Master Switch of the TDC had been found to be smoking! The TDC maintenance department then put an electronic thermometer on the Master Switch component box to observe its temperature and also placed a number of electrical fans near it to try to keep it cool. If that single switch were to fail, the whole TDC would immediately lose power in a totally uncontrolled manner and come crashing down.
Figure 8 - Trying to recover the TDC from a real IT Disaster felt more like being in the actual situation that the 1964 movie "Fail Safe" tried to portray.
As the author and caretaker of the Application Portfolio System, I was then called into an Emergency IT Management meeting to assess the situation. I explained to IT Management that losing the TDC in an uncontrolled manner would have grave consequences. So the decision was then made to slowly bring down the TDC in a controlled manner using the data in the Application Portfolio System as a guide, and then to replace the failing TDC Master Switch. Once electrical power had been restored to the TDC, it would be brought up in a controlled manner following the Disaster Recovery Plans found within the Application Portfolio System. The problem was that this was in the middle of the winter. The TDC would lose all datacenter heating during the many hour process to replace the Master Switch and bring the TDC back up in an orderly manner within a recently very cold environment. In the late 1980s, all of our data was still on tapes and disk drives with lots of physically moving parts. The problem was the condensation of water molecules. Nobody really knew how cold disk drives, tape drives, and tapes would respond when coming back up. Despite all of the challenges, Amoco was able to replace the Master Switch at the TDC and bring it all back to life with little issues, largely thanks to the data found in the Application Portfolio System. This very successful recovery of the TDC during a real-world IT Disaster explains how I was later drafted into Amoco's Y2K Project about 10 years later. For more about my 2+ year adventures on Amoco's Y2K project see Scenes From the COVID-19 and Y2K Pandemics
Recovering an AI Datacenter Would Also Be Painful But Certainly Possible
Since modern AI datacenters no longer have large numbers of physical components with moving parts like spinning tape drives and disk drives, bringing them back up from the actions of a Global Kill Switch would not be so bad.
Me:
What would happen to an AI datacenter running a 1 GW load if all electrical power were suddenly cut, including all backup power from batteries and local generators?
ChatGPT:
If a large AI data center drawing 1 gigawatt (GW) of electrical power were to instantly lose all power, including the utility feed, UPS batteries, and backup diesel or gas generators, the sequence would be extremely rapid.
First few milliseconds (0–0.02 seconds):
Servers, networking equipment, storage arrays, and cooling systems all lose power simultaneously.
Capacitors inside power supplies discharge in a few milliseconds.
CPUs, GPUs, memory, and networking chips abruptly stop operating.
Any calculations in progress are immediately interrupted.
Within a second:
Every GPU and CPU is off.
DRAM contents disappear because RAM requires continuous power.
Models stored in RAM or GPU memory are lost.
Only data previously written to SSDs or disks survives.
Network switches and routers shut down.
Internal communications cease.
Cooling stops:
Modern AI data centers rely on enormous cooling systems.
These may include:
Air handlers
Chilled-water pumps
Cooling towers
Liquid cooling pumps for GPU racks
Without power:
Fans stop.
Pumps stop.
Refrigeration compressors stop.
However, temperatures do not rise instantly.
Temperature rise:
A 1 GW data center is producing roughly
1 billion joules of heat every second while operating.
The moment power disappears:
heat generation from the electronics essentially drops to zero because the chips are no longer consuming power.
Only residual heat stored in:
GPUs
CPUs
heat sinks
coolant
building structure
remains.
Those components cool naturally.
Ironically, the cooling system stopping is not the biggest issue because the servers themselves have also stopped producing heat.
Storage:
Data stored on
SSDs
hard drives
distributed storage
remains intact unless a write operation was occurring exactly when power failed.
Modern storage systems use journaling and transaction logging to recover from unexpected shutdowns.
Some in-flight writes may be lost.
When power returns:
Recovery is much slower than shutdown.
Operators must:
1. Verify electrical systems.
2. Restart cooling.
3. Bring UPS systems online.
4. Start networking.
5. Boot thousands or tens of thousands of servers.
6. Check distributed storage consistency.
7. Reload AI models into GPU memory.
Large language models may require tens or hundreds of terabytes of parameters to be loaded across GPU clusters, which can take minutes to over an hour depending on the scale of the deployment.
Could hardware be damaged?
Usually, no.
Data centers are designed to tolerate sudden outages.
Potential problems include:
interrupted storage writes
filesystem recovery
databases replaying logs
occasional failed power supplies or disks (mostly due to normal failure rates)
A simple loss of power generally does not destroy GPUs or CPUs.
Worst-case scenario:
The greater concern is power restoration, not power loss.
If all 1 GW of equipment tried to restart simultaneously, the electrical inrush current could overload equipment.
Large data centers therefore use carefully sequenced startup procedures:
cooling first,
networking second,
storage,
then compute racks in stages.
This controlled restart prevents damaging electrical surges and allows thermal systems to stabilize before the GPUs begin generating hundreds of megawatts of heat again.
In short, a complete loss of all power at a 1 GW AI data center would cause an immediate halt to computation and loss of volatile memory, but it would not ordinarily damage the servers themselves. The facility's biggest challenge would be orchestrating a safe, staged recovery once reliable power had been restored.
So shutting down all power to an AI datacenter should cause little damage to the AI datacenter hardware. Of course, there would be rather severe economic damage and possibly even some loss of life, but certainly much less than from a full-blown AI Apocalypse.
Comments are welcome at scj33345@gmail.com.
To see all posts on softwarephysics in reverse order, go to:
https://softwarephysics.blogspot.com/.
Regards,
Steve Johnston
No comments:
Post a Comment