Comment by Fripplebubby

I think that the concept of a "foundation model" for time series is actually a bit flawed as presented in this blog post. A foundation model is interesting because it is capable of many tasks _beyond the target tasks_ that it was trained to do, whereas what the author is looking for is a time-series model that can make out-of-distribution predictions without re-training - which is, in my opinion, a problem that is pretty well solved by existing ARIMA and (especially) Prophet models (Yes, you have to re-fit the model on your distribution, but this is not at all akin to the task of training or fine-tuning an LLM, it's something you can do in seconds on a modern CPU, and yes, there are certain hyperparameters that may need to be selected, but they are actually fairly minimal).

But for a model to make out-of-distribution predictions does not make it a foundation model for time series, really that's just the basic task that all time series forecasting models do. A more interesting question is, does an LLM architecture seem to improve the task of univariate or multivariate time-series prediction? I don't think the answer is yes, although, depending on your domain, being able to use language inputs to your model may have a positive impact, and the best way to incorporate language inputs is certainly to use a transformer architecture, but that isn't what is addressed in this post.