Enhanced Sales Forecasting Model Using Textual Search Data: Fusing Dynamics With Big Data

Forecasting sales is an essential marketing function, and, for most businesses, sales are driven by own and competitive activities. Most firms use their own marketing efforts or a selection of their competitor’s marketing efforts for forecasting sales. Due to data availability limitations, data on the full set of competitors are rarely used when forecasting sales. The emergence of online search data provides access to a novel data source on own as well as never-before observed competitive activities. The authors propose a novel regularized dynamic forecasting model utilizing all available competitive search data in a market vs. constructing ad-hoc and potentially subjective smaller competitive sets. The authors' model addresses the inherent statistical issue that arises when including a large number of competitive effects and parsimoniously utilizes all competitive data. The authors demonstrate their model using data from the US automobile industry over a twelve-year period and forecast car-model sales for 14 exemplary car-models utilizing multiple search measures for all 374 potential competitive car-models. The authors show that their model fits and forecasts sales better than models not leveraging the full competitive search data, e.g., using subjective sets of relevant competitors or narrowly defined category competitors. The authors also find that market research done via novel large-language models (also called LLMs) to obtain a narrower set of competitors does not outperform the authors' proposed model that includes the full set of competitors.