Mike's Notes
The GitHub link contains the experiment. Yuanzhao has done research into reservoir computing, a future enhancement for Pipi 11.
Resources
- https://santafe.edu/news-center/news/chaos-in-the-machine-how-foundation-models-can-make-accurate-predictions-in-time-series-data
- https://santafe.edu/people/profile/yuanzhao-zhang
- https://y-zhang.com
- https://iclr.cc/virtual/2025/poster/29513
- https://github.com/GilpinLab/dysts
- https://y-zhang.com/publications.html
- https://y-zhang.com/machine-learning.html
- https://arxiv.org/abs/2506.05292
References
- Reference
Repository
- Home > Ajabbi Research > Library > Subscriptions > SFI News
- Home > Handbook >
Last Updated
08/06/2025
Chaos in the machine: How foundation models can make accurate predictions in time-series data
Yuanzhao Zhang, Complexity Postdoctoral Fellow, Omidyar Fellow, Santa Fe Institute
Yuanzhao was born and raised in a small coastal city in China. His research focuses on collective dynamics on complex networks. In particular, he is interested in how individual differences shape collective behaviors and how complex dynamical patterns emerge from decentralized interactions.
Yuanzhao received a B.Sc. in Mathematics from Zhejiang University in 2014, an M.Sc. in Applied Mathematics in 2015 and his Ph.D.in Physics from Northwestern University.
Until recently, using machine learning for a specific task meant training the system on vast amounts of relevant data. The same was true for data representing a system that changes over time, says SFI Complexity Postdoctoral Fellow Yuanzhao Zhang. “The traditional paradigm in forecasting dynamical systems has always been that you need to train on the system you want to predict,” he says. If you want to forecast the weather in Santa Fe, start by training your model on the area’s historical weather data.
But the advent of foundation models — a term coined in 2021 to describe the architecture at the heart of today’s AI systems — has changed the game. These models, like previous systems, train on large datasets. But unlike earlier, specialized deep-learning models, they’re designed to carry out a wide range of tasks. “They work right out of the box,” Zhang says. Notably, they can complete new tasks that weren’t included in their training data. For large language models, those include tasks like generating computer code or translating between languages. Reports of this behavior, called “zero-shot learning,” ignited a global race to build models that can similarly make zero-shot predictions for time-series data.
Zhang wanted to understand whether existing foundation models could predict chaotic systems and, if so, how they do it. In a recent analysis, Zhang and William Gilpin, a physicist at the University of Texas at Austin, reported that a foundation model called Chronos could generate predictions of chaotic dynamical systems at least as accurately as models trained on relevant data. Their paper was accepted to the Thirteenth International Conference on Learning Representations, which focuses on deep learning approaches in AI and was held in Singapore in April 2025.
Zhang says the paper represents the first test of zero-shot learning in forecasting chaotic systems, such as the weather and financial markets, which are governed by mathematical equations and extremely sensitive to small changes in initial conditions. Zhang and Gilpin tested their idea by using Chronos to predict how 135 chaotic systems would change over time. They tested each system using 20 distinct initial conditions. They compared the short- and long-term predictions of the model to deep learning models specifically trained using chaotic data.
“We wanted to compare this zero-shot paradigm with the old paradigm and see if the foundation model can outperform the traditional models,” Zhang says.
The promising results show that foundation models can make accurate predictions after training on data from any time series — not just data from the system or task that a user wants to predict. Forecasting the weather in Santa Fe may not require historical data, just other time-series behaviors in which the model could identify patterns.
The study raises interesting ideas about what kind of training is required to accurately perform time-series tasks. “There’s this question: Do you actually need to learn chaos to have a good forecasting performance for chaotic systems?” Zhang asks. “I think the answer is no.”
Zhang and Gilpin’s current work only looks at one-dimensional data; in future work, Zhang says he hopes to expand that to more complicated, multidimensional data. He’d also like to determine how the system carries out these tasks. “Is it, in some sense, learning the dynamics?” he asks. “Is it using anything more sophisticated than parroting?”
The new study offers a step forward in answering those larger, deeper questions, he says.
No comments:
Post a Comment