Taxi demand prediction based on LSTM with residuals and multi-head attention

This paper presents a simple yet effective framework to accurately predict the taxi demands of different regions in a city in the near future. This framework is based on a deep-learning structure with residual connections in the LSTM layers and the attention mechanism. We found that adding residuals accelerates optimization and that adding the attention mechanism makes the model better predict the taxi demands, especially when the demand fluctuates greatly in the peak hours and off-peak hours. We conducted extensive experiments by comparing the proposed models to the time-series model (ARIMA), traditional supervised learning model (ridge regression), strong machine learning model that won many Kaggle competitions (Gradient Boosted Decision Tree implemented in the XGBoost library), and deep learning models (LSTM and DMVST-Net) on two real and open-source datasets. Experimental results show that the proposed models outperform the baselines for most cases. We believe the greatest improvement comes from the attention mechanism, which helps distinguish the demands in the peak hours and off-peak hours. Additionally, the proposed model runs 10% to 40%-times faster than the other deep-learning-based models. We applied the models to participate in a taxi demand prediction challenge and won second place out of hundreds of teams.