Time Series Forecasting with Deep Learning
Forecasting that survives review: start with naive baselines, build leak-free lag features, train an LSTM in PyTorch, and judge it with walk-forward validation.

A demand forecast I built with an LSTM looked excellent in review: a low percentage error and a prediction line hugging the actuals. Then a colleague overlaid the two series and zoomed in. The prediction was the actual curve shifted by one step. The network had learned to repeat the last value it saw, which is exactly what a one-line baseline does, and it had cost us two weeks of training experiments to match it.
Deep learning can improve time series forecasts, but only if you prove it beats the dumb alternatives under an honest evaluation. This post is the workflow I use to do that, with code you can run on your own series. If you need the neural-network fundamentals first, start with Deep Learning Fundamentals.
Quick takeaways
- Always compute a naive forecast and a seasonal-naive forecast first. If your model cannot beat them, it has learned nothing useful.
- Build lag and rolling features with
shift(1)so no feature ever sees the value it predicts. - Fit scalers on the training fold only, and evaluate with walk-forward splits instead of random ones.
- For a single series with limited history, gradient boosting on lag features is often as good as an LSTM and much cheaper to run.
- Deep models earn their cost on many related series, long histories and rich external signals.
What is the first model I should build?
Two baselines, written in minutes. The naive forecast predicts that the next value equals the last one. The seasonal-naive forecast predicts it equals the value one season ago, for example the same hour yesterday or the same weekday last week. Many production systems quietly run on these, and they set the bar. Classical models are the next rung: ARIMA is a solid, interpretable baseline for a single series, and Prophet is quick to prototype when you have strong calendar effects.
Know what you are fighting. A series is trend plus seasonality plus noise, and the hard parts are non-stationarity, multiple seasonalities, missing values and concept drift, where the pattern itself changes. Plot the series and its autocorrelation before you model anything.
How do I engineer features without leaking the future?
Lag features, rolling statistics and calendar fields cover most of the signal. The leak to avoid is a rolling window that includes the current value, which hands the model the answer:
import pandas as pd
def add_features(df: pd.DataFrame, target: str = "demand") -> pd.DataFrame:
out = df.copy()
for lag in (1, 24, 168): # previous step, same hour yesterday, same hour last week
out[f"lag_{lag}"] = out[target].shift(lag)
# shift(1) before rolling so the window never includes the value being predicted
out["roll_mean_24"] = out[target].shift(1).rolling(24).mean()
out["roll_std_24"] = out[target].shift(1).rolling(24).std()
out["hour"] = out.index.hour
out["dow"] = out.index.dayofweek
return out.dropna()
The frame needs a DatetimeIndex. Add holidays, promotions, weather or prices as external features when you will know their values at prediction time; a feature you only learn afterwards is a leak. With these columns you can train gradient boosting (LightGBM, XGBoost) in a few lines, and it is the model I try before any neural network.
How do I train an LSTM forecaster in PyTorch?
LSTMs and GRUs read a window of past values and predict the next one. The code below turns a series into windows, defines a small model and trains it with an L1 loss:
import numpy as np
import torch
from torch import nn
def make_windows(series: np.ndarray, lookback: int):
X, y = [], []
for i in range(len(series) - lookback):
X.append(series[i : i + lookback])
y.append(series[i + lookback])
return np.array(X, dtype=np.float32)[..., None], np.array(y, dtype=np.float32)
class LSTMForecaster(nn.Module):
def __init__(self, hidden: int = 32):
super().__init__()
self.lstm = nn.LSTM(input_size=1, hidden_size=hidden, batch_first=True)
self.head = nn.Linear(hidden, 1)
def forward(self, x):
out, _ = self.lstm(x)
return self.head(out[:, -1]).squeeze(-1)
def fit(model, X, y, epochs: int = 20, lr: float = 1e-3):
X_t, y_t = torch.from_numpy(X), torch.from_numpy(y)
opt = torch.optim.Adam(model.parameters(), lr=lr)
loss_fn = nn.L1Loss()
for _ in range(epochs):
model.train()
for idx in torch.randperm(len(X_t)).split(64):
opt.zero_grad()
loss = loss_fn(model(X_t[idx]), y_t[idx])
loss.backward()
opt.step()
return model
Temporal convolutional networks replace the recurrence with dilated convolutions, which train in parallel and are often faster. Transformer-based models capture long-range dependencies and shine on large datasets with many series. All of them consume the same windowed input, so the evaluation code below works for any of them.
How do I evaluate a forecast honestly?
Random train and test splits let the model peek at the future, so they are wrong for time series. Use walk-forward validation: train on the past, test on the block that follows, then slide forward. Scale with statistics from the training fold only, and compare against both baselines in the same loop:
from sklearn.model_selection import TimeSeriesSplit
LOOKBACK = 48 # values: a 1-D float array of the series, hourly data in this example
for fold, (tr, te) in enumerate(TimeSeriesSplit(n_splits=3, test_size=168).split(values)):
mu, sigma = values[tr].mean(), values[tr].std() # statistics from the training fold only
scaled = (values - mu) / sigma
X_tr, y_tr = make_windows(scaled[tr], LOOKBACK)
# test windows may look back into the training fold, never forward
X_te, y_te = make_windows(scaled[te[0] - LOOKBACK : te[-1] + 1], LOOKBACK)
model = fit(LSTMForecaster(), X_tr, y_tr)
model.eval()
with torch.no_grad():
pred = model(torch.from_numpy(X_te)).numpy()
naive = X_te[:, -1, 0] # last observed value
seasonal = X_te[:, -24, 0] # same hour yesterday
mae = lambda p: float(np.abs(y_te - p).mean())
print(fold, f"naive {mae(naive):.3f} seasonal {mae(seasonal):.3f} lstm {mae(pred):.3f}")
Report MAE and RMSE in the units your stakeholders understand. MAPE is popular but explodes when actuals are near zero. MASE divides the error by a naive forecast's error, so a value below 1 means you beat the baseline, which is the question you actually care about. If the LSTM column is not clearly lower than the others across folds, stop tuning the network and improve the features.
When do other architectures and ensembles help?
On one series with a few thousand points, I rarely see a deep model beat well-engineered lag features with gradient boosting, and the boosted model is easier to explain and retrain. Deep models start to pay off when you train one global model across thousands of related series, when you have long histories, or when you combine text, images or many external signals. Ensembles are reliable: averaging a statistical model, a boosted model and a neural one usually beats each alone, because their errors differ. Whatever you ship, treat drift as a given and monitor it; the pipeline side is covered in Building Production ML Pipelines, and the data checks in Data Engineering Best Practices.
What should I do this week?
Take one forecast you already run. Compute the naive and seasonal-naive errors over the last four weeks with walk-forward folds, and put the three numbers next to your current model's error in one table. If the model does not win clearly, you have found your next project, and it is a cheaper one than training another network.
Uğur Kaval
AI/ML Engineer & Full Stack Developer specializing in building innovative solutions with modern technologies. Passionate about automation, machine learning, and web development.


