Automating Demand Forecasting with Machine Learning

Will Goodrum, Ph.D.

August 3, 2018

BLOG_Automating Demand Forecasting

Elder Research implemented an automated framework for time-series forecasting at a major logistics company. Our system, combining R and Apache Spark™, produces 35 million forecasts in under one hour, and selects the optimal time-series forecast algorithm in each of three forecasting windows. Forecast results from our framework were 88% accurate at a four-week horizon.

The Challenge

Demand Forecasting 1Logistics is a mature, technologically-advanced, and analytically-sophisticated industry. Still, even after decades of improvements coming from the Industrial Engineering and Operations Research fields, major efficiencies can still be realized by applying advanced analytics, data infrastructure, and computing power. All business processes in logistics rely on accurate demand forecasting in the short, medium, and long-term to inform resourcing, planning, and staffing to support future needs. Our client was three months into a highly-visible, strategic analytics project and with an urgent need to have forecast results in their production system. Given the strategic importance of this project, they needed to quickly scale a prototype forecast model into their automated production system that interfaces with a new platform for planners.

The Solution

Elder Research was hired to provide a bridge between technical experts and the application development team responsible for time-series forecast implementation. We worked collaboratively with the prototype model authors, software developers and architects, database administrators, and business stakeholders to ensure that our production solution would meet requirements, interface with existing systems, and provide the flexibility required for future development. We also provided a valued perspective on statistical and optimization methods and techniques for the operations research team that created the prototype model.


Demand GraphIn three weeks we delivered a functioning production time-series forecasting framework using R and Spark. After six months we had scaled to a refined framework that produces 35 million forecasts in under one hour on over 2000 locations in our client’s network. This framework features automated execution and algorithm selection at short, medium, and long-term horizons. At a two-week interval, our forecasts had a median accuracy of 88%, despite high variance in the characteristics of the entities being forecast. We also developed a flexible forecasting model API to enable easy inclusion/exclusion of time-series algorithms as better techniques are identified or existing algorithms are replaced.


Benefit Summary-1

About the Author

Will Goodrum, Ph.D. Dr. William Goodrum has nearly a decade of experience in the management and delivery of projects and products that embed Data Science and numerical methods in software. At Elder Research, Dr. Goodrum leads a team of six Data Scientists who deliver custom Data Science training and create advanced analytical solutions and strategy for private sector clients around the globe. Dr. Goodrum has experience consulting across different industries, including logistics, software, and philanthropic development. Additionally, Dr. Goodrum has acted as PI on a NASA Phase II STTR program that implemented validated models of corrosion behavior for gas turbine engine rotors. Prior to Elder Research, Dr. Goodrum worked at a global engineering software firm where he supported customers in the Aerospace & Defense, manufacturing, and automotive industries. Dr. Goodrum’s PhD research estimated lifetime highway maintenance costs for the government of New South Wales, Australia.