Time-Series Forecasting with LLM using Chronos

Utilizing the Large Language Model for time-series prediction

Photo by Nathan Dumlao on Unsplash

The large Language Model (LLM) is identical to text generation tasks such as question-answering, summarization, translation, and text-related tasks. Even for prediction, text classification is possible with LLM.

Recent developments have shown that LLM can perform classic machine learning tasks, including time Series prediction: Introduction Chronos, a pre-trained time series forecasting model based on language model architectures.

Developed by AWS teams, the LLM model can show excellent performance on the time-series task. So, how do they work, and how do they perform the prediction with Chronos? Let’s learn together.

Chronos Summary

Before going into the hands-on approach for the Chronos, let’s understand how the model works.

Chronos is a novel framework that uses language modeling techniques to leverage pre-trained probabilistic time series models. Developed by a team from AWS AI Labs and various universities, Chronos simplifies time series forecasting by converting time series values into a tokenized format through scaling and quantization. This allows transformer-based language models to be trained on these tokenized time series without modifying the model architectures.

The Chronos’ overall structure is shown in the image below.

Chronos overall architecture (Ansari et al. 2024)

Chronos was pre-trained on a large collection of publicly available datasets and a synthetic dataset generated via Gaussian processes, which allowed the model to be generalized well. The model was evaluated on 42 diverse datasets, significantly outperforming both classical models and deep learning methods on trained datasets while also showing superior zero-shot performance on new datasets.

Chronos Performance based on the benchmarks dataset (Ansari et al. 2024)

The models are based on the T5 architecture and have several size selections. The only difference is the vocabulary size.

Ultimately, Chronos can improve zero-shot accuracy and simplify forecasting processes, making it an ideal candidate for further development as a general-purpose time series model.

That’s enough theory for now. Let’s move on to a more hands-on approach.

Chronos Hands-On Approach

Let’s start with the package installation. You can do that with the following code.

!pip install git+https://github.com/amazon-science/chronos-forecasting.git

As the Chronos model is based on LLM architecture, you might need a GPU to use it properly, but the minimal example we use might already be enough for you, who can only work with a CPU.

Next, we would download the dataset for the example. We can do that with the following code.

import pandas as pd 
import torch
from chronos import ChronosPipeline

df = pd.read_csv("https://raw.githubusercontent.com/AileenNielsen/TimeSeriesAnalysisWithPython/master/data/AirPassengers.csv")

We would have a sample dataset like the one above. It shows the Month and the numerical values for the training data.

We can then set the model pipeline with the code below.

pipeline = ChronosPipeline.from_pretrained(
    "amazon/chronos-t5-small",
    device_map="cuda",
    torch_dtype=torch.bfloat16,
)

You can change the CUDA to the CPU if necessary. The model pipeline is ready, and we can use it for inferences.

forecast = pipeline.predict(
    context=torch.tensor(df["#Passengers"]),
    prediction_length=12,
    num_samples=20,
)

With the code above, we already forecast our next 12 steps of the time-series data. If we visualize the result, it would be like this.

import numpy as np
import matplotlib.pyplot as plt 
forecast = pipeline.predict(
    context=torch.tensor(y_train),
    prediction_length=12,
    num_samples=20,
)

forecast_index = range(len(df), len(df) + 12)
low, median, high = np.quantile(forecast[0].numpy(), [0.1, 0.5, 0.9], axis=0)

plt.figure(figsize=(10, 6))
plt.plot(df["#Passengers"], color="royalblue", label="Historical Data")
plt.plot(forecast_index, median, color="tomato", label="Median Forecast")
plt.fill_between(forecast_index, low, high, color="tomato", alpha=0.3, label="80% Prediction Interval")
plt.legend()
plt.grid()
plt.xlabel("Time")
plt.ylabel("Number of Passengers")
plt.title("Passenger Forecasting with Chronos")
plt.show()

That’s all you need to use Chronos. It’s pretty intuitive and easy for beginners. Try to tweak the parameters to understand the model better.

I hope that helps!

---

Don’t forget to like, subscribe, and share the post if you find it useful.