Scaling Your Time Series Forecasting Project

Scaling Your Time Series Forecasting Project

This Blog is based on my talk at the International Symposium on Forecasting ISF2020, titled — Best Practices for Scaling Sales Forecasting.

What Are We Forecasting and Why?

At for the last few years we’ve been using time-series forecasting models as part of our data science projects for forecasting Wix’s future collections. This allowed the company two important things: (1) Better budget planning based on future collections; (2) Accurate guidance to the stock market.

Forecasting collections is a challenging task (every forecasting task is) but it’s one we’ve constantly improved on and achieved amazing results. In this blog I want to share some of our insights and practices for scaling a forecasting project.

Scaling Starts from the Bottom — The Importance of Design

For our system to scale and accommodate more features, more models and eventually more forecasters we divided it to the above building blocks. Each building block stands on its own but also knows how to communicate and work with others. A short explanation of each (hopefully I would describe them in more depth at the future, comment below/DM with requests):

  • Data Sources —Used to access and query all of our different tables and DB’s.
  • Feature Store — Uses the data from the Data Sources to create time-series features. Given a list of requested features it will return a time-series dataset.
  • *Models — *This is included for clarity, references open source models for time-series.
  • Models Library — Wrappers for Models, with our choice of hyper-params and compatible to datasets originating from the Feature Store.
  • Forecaster — Takes a dataset from the Feature Store and a model from the Model Library and creates a time-series forecaster for our required task.
  • Airflow DAG — The DAG%20as%20code.) plays several roles in our system, in this case (more will follow) we’re referring to it scheduling the Forecaster’s run (daily in our case)

As mentioned, this design supports scaling and as important it supports fast experimentation which is the basis of every successful data science project.

data-science scaling forecasting time-series-forecasting machine-learning

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

How to use Deep Learning for Time Series Forecasting

How to use Deep Learning for Time Series Forecasting. An application of the RNN family

What is Time Series Forecasting?

In this article, we will be discussing an algorithm that helps us analyze past trends and lets us focus on what is to unfold next so this algorithm is time series forecasting. In this analysis, you have one variable -TIME. A time series is a set of observations taken at a specified time usually equal in intervals. It is used to predict future value based on previously observed data points.

Most popular Data Science and Machine Learning courses — July 2020

Most popular Data Science and Machine Learning courses — August 2020. This list was last updated in August 2020 — and will be updated regularly so as to keep it relevant

15 Machine Learning and Data Science Project Ideas with Datasets

Learning is a new fun in the field of Machine Learning and Data Science. In this article, we’ll be discussing 15 machine learning and data science projects.

Time Series Forecasting: Limitations of LSTMs

While LSTMs have become increasingly popular for time series analysis, they do have limitations. Long-short term memory networks (LSTMs) are now frequently used for time series analysis.