Time-series foundation models: how one pretrained model forecasts data it has never seen

A time-series foundation model is a neural network pretrained on a very large and varied collection of time series, such as sensor readings, sales figures or web traffic. Once trained, it can forecast a series it has never seen, with no training on that series. This “zero-shot” use replaces the older routine of building and tuning a separate model for every dataset.

Training once instead of once per dataset

Classic forecasting fits one model to one series or one group of series. An analyst picks a method such as ARIMA or exponential smoothing, or trains a deep network on the company’s own history, and repeats the work whenever the data changes. Each model starts knowing nothing about the world.

A foundation model flips this. During pretraining it sees an enormous number of series and learns recurring shapes: daily and weekly cycles, trends, level shifts, spikes, noise. At forecast time it receives the recent history of a new series as context and continues it. A rough analogy is an experienced weather observer who has watched thousands of seasons and can read a new town’s pattern after a few days, instead of a newcomer who must learn from scratch.

Most of these models are also probabilistic. Rather than one line, they return a range of plausible futures, usually as quantiles, so a planner can see a likely value and an uncertainty band around it.

Borrowing the language-model toolkit for numbers

The core trick is to make a sequence of measurements look like a sequence of tokens, so transformer machinery built for text can be reused. Two designs from 2024 show the two main routes.

Amazon researchers, led by Abdul Fatir Ansari, published Chronos (arXiv, March 2024). It scales the values and quantizes them into a fixed vocabulary, then trains T5-family language models, from 20M to 710M parameters, on those tokens with the usual cross-entropy loss. The authors added synthetic series generated from Gaussian processes to improve generalization. On a benchmark of 42 datasets, they report that Chronos clearly beats other methods on datasets that were in its training corpus, and is comparable to, and occasionally better than, methods trained specifically on the data when tested zero-shot on new datasets.

Google Research took a different route with TimesFM, described in a February 2024 blog post by Rajat Sen and Yichen Zhou and accepted at ICML 2024. It is a decoder-only model, with causal attention like a chatbot, of about 200M parameters. Instead of quantizing, it groups consecutive time points into patches and turns each patch into a token with a small neural layer. Its output patches can be longer than its input patches, so a long forecast needs fewer generation steps, which the authors say reduces accumulated error. Pretraining used about 100 billion real-world time points, a large share of them search-interest data from Google Trends, plus Wikipedia pageviews and synthetic series.

Measuring “works on anything” with public benchmarks

Claims of generality need a shared test. In October 2024, Taha Aksu and colleagues at Salesforce AI Research published GIFT-Eval, a benchmark aimed at foundation models. According to the paper, it covers zero-shot forecasting across 23 datasets, more than 144,000 series and 177 million data points, seven domains and ten sampling frequencies, with multivariate inputs and horizons from short to long. It also ships a pretraining dataset of roughly 230 billion points designed not to leak test data, and the authors evaluated 17 baselines spanning statistical, deep learning and foundation models. Salesforce’s own blog and the Hugging Face dataset card disagree on the dataset count (28 versus 23), so figures may differ by version. Other leaderboards, such as TIME and fev-bench, now sit alongside it, and new releases are usually ranked on all of them.

Where Timestral fits in

The newest entrant is Mistral AI’s Timestral, covered in our news story. Per Mistral’s announcement, it comes in four sizes (5M, 28M, 123M and 209M parameters) sharing one architecture, outputs 21 quantiles trained with a pinball loss, and takes a context of 32,768 time steps. Mistral reports that the 209M model ranks first among pretrained zero-shot models on GIFT-Eval, TIME and fev-bench from a single checkpoint without ensembling, and that its 28M model beats Chronos-2, a model about four times larger, on GIFT-Eval and TIME. These are the company’s own figures, and a full technical report is still to come.

Two design choices stand out. First, covariates: each input series gets a role, either target, past-only covariate or known-future covariate such as a holiday calendar or weather forecast, and attention lets information flow from covariates to targets but not back, so a target cannot leak its own future. Second, the training data: Mistral says most of it is synthetic, with pretraining on purely synthetic data at a context of 8,192, then post-training on a mix of real and synthetic data at a context four times longer. The company also says the models run on a server CPU, with about 10 ms for the smallest and 140 ms for the largest on one specific one-day-ahead task. Customers can adapt the model through Mistral Forge, and Mistral describes pairing it with language-model agents that explain flagged anomalies.

Limits that benchmarks do not show

Zero-shot success on public datasets does not guarantee success on a given factory or trading desk. Benchmark series are often cleaner than production data, and real data drifts over time. Mistral itself notes that production data can differ from benchmark data and that adaptation and retraining may be needed. Known-future covariates only help when that future information exists and is reliable.

Evidence is also thinner than the leaderboard rankings suggest. Google’s TimesFM post reports that it beats most, not all, supervised approaches, and its long-horizon comparison covers only a few datasets. Most reported results are self-reported by the releasing lab, which is why independent submissions to shared leaderboards matter. Contamination is a standing risk too: if a model’s training data overlaps a test set, scores look better than they are. That is why GIFT-Eval ships a non-leaking pretraining set, and why Mistral points to control over the training mix as a way for customers to make sure nothing they backtest appeared in pretraining.

A smaller practical point is that a general model is not always the best tool. A simple statistical method can still win on a stable, well-understood series, and a foundation model adds a dependency that needs checking against the baseline.

What to watch next

The first thing to watch is Mistral’s technical report, which should show whether synthetic-heavy training and the forecast-token design hold up under outside scrutiny. The second is independent evaluation on private, messy data rather than public benchmarks. The third is the trend toward small, CPU-friendly models: if a 28M or 123M model is close to the largest in accuracy, as Mistral claims, forecasting can run on ordinary servers and inside air-gapped environments. Finally, the combination of forecasters with language-model agents, where one model spots the anomaly and another explains it, is an early pattern to see tested in real operations.