"Inflated T-Values in Scanner Data Models"
In scanner models, non-promoted weeks with lower, more stable sales often outnumber promoted weeks with higher, less predictable volume spikes. Yet standard t-values used to assess accuracy of promotion parameter estimates skew in favor of more accurate predictions in non-promoted weeks. Thus, they overstate precision and give modelers too much confidence in rejecting the null hypothesis that a coefficient is 0. This is demonstrated with simulated data. The conclusion: t-values should be used with some caution as a model diagnostic.