🔥 Model calibration is a hot topic in Marketing Mix Modeling right now
Model calibration is a key step in the modeling workflow for ensuring the accuracy of ROI estimates.
How does it work?
👉 In the Bayesian modeling approach, one can use "priors" to set assumptions for the model on the likely range where the ROI for a marketing activity lies.
👉 Instead of using non-informative priors, where the aim is not to guide the model, the calibration approach uses informative priors, i.e. it gives the model narrower range where to search the ROI from.
The main assumption in model calibration is, that you have strong evidence, which justifies creating priors with a narrow distribution and thus limiting the range of possible ROI estimates. This makes model calibration tricky - you need to be sure that you understand the biases and shortcomings in the calibration data that you use.
We typically leverage four types of data in our calibration framework, when forming prior distributions for marketing activities:
1️⃣ Randomized control trials, e.g., Facebook Conversion Lift tests.
2️⃣ Geo tests: Experiments where control geos and test geos are compared against each other in an attempt to understand the effect of a change in marketing activity.
3️⃣ Shutdown tests: Experiments where an activity is stopped for a certain period of time, after which the ”on” and ”off” periods are compared.
4️⃣ Triangulation based on incrementality factor benchmarks for the same marketing activity in similar companies.