Testing the Validity of a Demand Model: An Operations Perspective
Coauthor(s): Omar Besbes, Assaf Zeevi.
The fields of statistics and econometrics have developed powerful methods for testing the
validity (specification) of a model based on its fit to underlying data. Unlike statisticians,
managers are typically more interested in the performance of a decision rather than the statistical
validity of the underlying model. We propose a framework and a statistical test that incorporates
decision performance into a measure of statistical validity. Under general conditions on the
objective function, asymptotic behavior of our test admits a sharp and simple characterization.
We develop our approach in a revenue management setting and apply the test to a data set used
to optimize prices for consumer loans. We show that traditional model-based goodness-of-fit tests
may consistently reject simple parametric models of consumer response (e.g., the ubiquitous logit
model), while at the same time these models may "pass" the proposed performance-based test.
Such situations arise when decisions derived from a postulated (and possibly incorrect) model generate results that cannot be distinguished statistically from the best achievable performance—i.e., when demand relationships are fully known.
Source: Manufacturing & Service Operations Management
Besbes, Omar, Robert Phillips, and Assaf Zeevi. "Testing the Validity of a Demand Model: An Operations Perspective." Manufacturing & Service Operations Management 12, no. 1 (Winter 2010): 162-183.