RNN vs DNN for Time Series | Dr. Pedram Jahangiry

Understanding RNN vs DNN for Time Series

This interactive visualization demonstrates a critical concept in time series forecasting: Dense Neural Networks (DNNs) can work well on raw time series data by treating lags as independent features, while Recurrent Neural Networks (RNNs) require data preprocessing to effectively capture temporal patterns.

Using the classic airline passenger dataset, you'll see how DNNs achieve good performance on raw data, while RNNs initially struggle but excel after log transformation and differencing (making data stationary).

Key Learning: Model choice matters, but so does understanding what preprocessing each architecture needs!

🤔 What's Going On? Is RNN Failing?

Note that the Dense Neural Network (DNN) model works well on the unprocessed airline passenger data, even without preprocessing steps like differencing or logarithmic transformation, while the Recurrent Neural Network (RNN) requires these preprocessing steps to perform effectively.

This can be attributed to several factors related to the nature of DNNs and how they process information compared to RNNs:

📊 DNNs and Feature Representation

Independent Features: Treat lags as independent features, capturing trend and seasonality without understanding sequential data.
Pattern Mapping: Learn patterns from raw data without requiring stationarity.
No Temporal Dynamics: Perform well if input-output relationships are straightforward, even without sequential understanding.

🔄 RNNs and Sequential Data

Sequential Processing: Capture temporal dependencies but can struggle with non-stationary data.
Need for Stationarity: Preprocessing (e.g., differencing) helps reveal patterns by stabilizing mean and variance.

💡 Conclusion

DNNs excel with raw data by leveraging absolute lag values.
RNNs require preprocessing to focus on temporal relationships effectively.
Model choice and preprocessing depend on data characteristics and forecasting goals.

Configuration

🔧 Shared Configuration

Data Preprocessing

Raw Data

Log + Differencing

Training Epochs

100

🔷 DNN Configuration

Number of Lags (Input Features)

Hidden Layer 1 Units

Hidden Layer 2 Units

🔶 RNN Configuration

Sequence Length (Memory Window)

RNN Units (Memory Depth)

Results

🔷 DNN Performance

Test MAPE: -

Test RMSE: -

Training Time: -

🔶 RNN Performance