I have been watching the recent YouTube videos by Kyle Kabasares demonstrating the scientific value of the new OpenAI ChatGPT o1-preview and ChatGPT o1-mini LLM models which are only a few days old. On his YouTube channel, Kyle Kabasares describes himself as:
Kyle Kabasares
I am a recent Physics PhD graduate from the University of California, Irvine. I currently work at the Bay Area Environmental Research Institute (BAERI) at NASA’s Ames Research Center in Silicon Valley.
Here is his website:
Kyle K. M. Kabasares
https://www.kylekabasares.com/
Here is his YouTube channel:
Kyle Kabasares
https://www.youtube.com/@KMKPhysics3
Kyle's YouTube channel features several hundred videos covering his adventures as an undergraduate and graduate student in physics. Following obtaining his Ph.D., Kyle Kabasares recently made the transition to become a data scientist at the Bay Area Environmental Research Institute (BAERI) at NASA’s Ames Research Center in Silicon Valley. In the process, Kyle Kabasares has become acquainted with the AI Revolution currently underway and has recently done some amazing research demonstrating how valuable Advanced AI could be for doing advanced research in physics. His work will be the subject of this post.
What is New About the OpenAI ChatGPT o1-preview and ChatGPT o1-mini LLM Models?
It has long been recognized that by simply putting "tell me step-by-step how to" into the prompt for an LLM model the output response from the LLM model is greatly improved because it encourages the LLM model to enter into a "chain of thought" analysis similar to human reasoning. Somehow the new ChatGPT o1-preview and ChatGPT o1-mini models have built-in human "chain of thought" reasoning. Instead of immediately running your prompt through the LLM generative neural network to produce a response, ChatGPT o1-preview and ChatGPT o1-mini spend a few seconds to a few minutes "reasoning" through the problem before passing it to the LLM generative neural network.
Here are some of Kyle Kabasares's latest YouTube videos which capture the power of using Advanced AI tools for scientific research purposes:
Can ChatGPT o1-preview Solve PhD-level Physics Textbook Problems?
https://www.youtube.com/watch?v=scOb0XCkWho&t=0s
Can ChatGPT o1-preview Solve PhD-level Physics Textbook Problems? (Part 2)
https://www.youtube.com/watch?v=a8QvnIAGjPA&t=0s
ChatGPT o1 preview + mini Wrote My PhD Code in 1 Hour*—What Took Me ~1 Year
https://www.youtube.com/watch?v=M9YOO7N5jF8&t=0s
Live Testing ChatGPT o1 With College and PhD-level Physics Problems
https://www.youtube.com/watch?v=GaAaFkipaTQ&t=0s
Addressing Some Questions...(ChatGPT o1-preview + o1-mini video(s) follow up)
https://www.youtube.com/watch?v=wgXwD3TD43A&t=0s
Fact-Checking OpenAI o1-preview on Graduate-Level Astronomy Problems
https://www.youtube.com/watch?v=Ww13-AWpWRk
Kyle Kabasares is currently uploading a great deal of content on this subject, so I would highly recommend looking at the
videos on his YouTube channel for additional videos. I have not seen anybody else put ChatGPT o1-preview and ChatGPT o1-mini or any other Advanced AI model through such rigorous testing so it would be very worthwhile for you to take a look.
These demonstrations of ChatGPT o1-preview and ChatGPT o1-mini doing advanced physics show how Advanced AI will be used to do scientific research in the near future as the AI Revolution continues to unfold. Currently, it seems that Advanced AI cannot do the whole job yet, but even today, Advanced AI can now be used to dramatically improve the efficiency of scientific research. At the rate that Advanced AI is now progressing, that near future might be defined in terms of just a few months, or possibly years because, eventually, Advanced AI should be able to do the whole job by itself.
Has Advanced AI Already Done So?
The researchers at Sakana AI already contend that it has. See their description of The AI Scientist at:
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
https://sakana.ai/ai-scientist/
Their paper for AI Scientist can be downloaded at:
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
https://arxiv.org/abs/2408.06292
For Kyle's thoughts on the AI Scientist see:
Reacting to the AI Scientist (As a Real Scientist)
https://www.youtube.com/watch?v=ACjB_5Tu1to&t=0s
Preparing for Humanity's Last Exam
Kyle Kabasares is currently using ChatGPT o1-preview and ChatGPT o1-mini to help formulate tough Ph.D.-level physics problems for the Humanity's Last Exam competition. The sponsors of the Humanity's Last Exam competition recognize that many of the current benchmarks used to test the current capabilities of Advanced AI are no longer up to the task. So they are currently holding a competition to prepare a really tough qualifying exam for Humanity's Last Exam. You can contribute a test question for the competition at:
Submit Your Toughest Questions for Humanity's Last Exam
https://scale.com/blog/humanitys-last-exam
The administrators of the competition explain:
September 16, 2024
Scale AI and CAIS are excited to announce the launch of Humanity's Last Exam, a project aimed at measuring how close we are to achieving expert-level AI systems. The exam is aimed at building the world's most difficult public AI benchmark gathering experts across all fields. People who submit successful questions will be invited as coauthors on the paper for the dataset and have a chance to win money from a $500,000 prize pool.
Déjà vu All Over Again
This reminds me very much of my own experiences from 50 years ago using computers to do geophysical research. I finished up my B.S. in physics at the University of Illinois in 1973 with the sole support of my trusty slide rule, but fortunately, I did take a class in FORTRAN programming during my senior year. I then immediately began working on an M.S. degree in geophysics at the University of Wisconsin at Madison. For my thesis, I worked with a group of graduate students who were shooting electromagnetic waves into the ground to model the conductivity structure of the Earth’s upper crust. We were using the Wisconsin Test Facility (WTF) of Project Sanguine to send very low-frequency electromagnetic waves, with a bandwidth of about 1 – 20 Hz into the ground, and then we measured the reflected electromagnetic waves in cow pastures up to 60 miles away. All this information has been declassified and is available on the Internet, so any retired KGB agents can stop taking notes now and take a look at:
Extremely Low Frequency Transmitter Site Clam Lake, Wisconsin
http://www.fas.org/nuke/guide/usa/c3i/fs_clam_lake_elf2003.pdf.
Project Sanguine built an ELF (Extremely Low Frequency) transmitter in northern Wisconsin and another transmitter in northern Michigan in the 1970s and 1980s. The purpose of these ELF transmitters is to send messages to our nuclear submarine force at a frequency of 76 Hz. These very low-frequency electromagnetic waves can penetrate the highly conductive seawater of the oceans to a depth of several hundred feet, allowing the submarines to remain at depth, rather than coming close to the surface for radio communications. You see, normal radio waves in the Very Low Frequency (VLF) band, at frequencies of about 20,000 Hz, only penetrate seawater to a depth of 10 – 20 feet. This ELF communications system became fully operational on October 1, 1989, when the two transmitter sites began synchronized transmissions of ELF broadcasts to our submarine fleet.
Figure 1 – Some graduate students huddled around a DEC PDP-8/e minicomputer. Notice the teletype machines in the foreground on the left that were used to input code and data into the machine and to print out results as well.
Anyway, back in the summers of 1973 and 1974, our team was collecting electromagnetic data from the WTF using a DEC PDP 8/e minicomputer. The machine cost about $30,000 in 1973 dollars and was about the size of a side-by-side refrigerator with 32K of magnetic core memory. We actually hauled this machine through the lumber trails of the Chequamegon National Forest and powered it with an old diesel generator to digitally record the reflected electromagnetic data in the field. For my thesis, I then created models of the Earth’s upper conductivity structure down to a depth of about 12 miles, using programs written in BASIC. The beautiful thing about the DEC PDP 8/e was that the computer time was free so I could play around with different models until I got a good fit to what we recorded in the field. The one thing I learned by playing with the models on the computer was that the electromagnetic waves did not go directly down into the Earth from the WTF like common sense would lead you to believe. Instead, the ELF waves traveled through the air to where you were observing and then made a nearly 900 turn straight down into the Earth, as they refracted into the much more conductive rock. So at your observing station, you really only saw ELF waves going straight down and reflecting straight back up off the conductivity differences in the upper crust, and this made modeling much easier than dealing with ELF waves transmitted through the Earth from the WTF. And this is what happens for our submarines too; the ELF waves travel through the air all over the world, channeled between the conductive seawater of the oceans and the conductive ionosphere of the atmosphere, like a huge coax cable. When the ELF waves reach a submarine, they are partially refracted straight down to the submarine. I would never have gained this insight by solving Maxwell’s differential equations for electromagnetic waves alone!
Using a computer as a research tool was a completely new experience for me after my four years of doing physics problems solely with a pen, paper and slide rule! Of course, using computers to model complex physical systems today is quite common and nearly universal. But at the time, I found the experience quite novel and mind-expanding. It allowed me to think of problems in an entirely new manner. I am sure that the new Advanced AI tools of today will do the same for others.
Comments are welcome at
scj333@sbcglobal.net
To see all posts on softwarephysics in reverse order go to:
https://softwarephysics.blogspot.com/
Regards,
Steve Johnston
No comments:
Post a Comment