Counterfactual Analysis of Business Decisions Using PoS Data: What Would Have Happened If You Had Not Changed the Price?
Apply synthetic-control and causal-forest methods to estimate counterfactual outcomes for past business decisions using observational PoS transaction data.
Key Takeaways
- Counterfactual analysis estimates what would have happened in the absence of a business decision, enabling rigorous evaluation of pricing, promotional, and assortment changes using observational PoS data.
- The synthetic control method constructs a counterfactual baseline by weighting control products or time periods to match the pre-intervention trajectory of the treated product, providing a credible comparison for single-unit interventions.
- Heterogeneous treatment effect estimation through causal forests reveals how the impact of a business decision varies across products, time periods, and customer segments, enabling targeted strategy refinement.
The Counterfactual Question in Retail Decision-Making
Every business decision a retailer makes — changing a price, running a promotion, discontinuing a product, adjusting store hours — has consequences that are only partially observable. The retailer observes what happened after the decision, but not what would have happened had the decision not been made. This unobserved alternative — the counterfactual — is precisely what is needed to evaluate whether the decision was beneficial. A price reduction that precedes a sales increase may appear successful, but if sales would have increased anyway due to seasonal trends, the price reduction may have unnecessarily eroded margin. Conversely, a promotional campaign that coincides with flat sales may appear ineffective, but if sales would have declined without the promotion, it may have prevented a larger loss. Counterfactual analysis provides the statistical methodology to construct credible estimates of the unobserved alternative outcome, enabling rigorous before-after comparison that accounts for confounding trends, seasonality, and concurrent events. The challenge is constructing a counterfactual that is credible: it must represent a plausible trajectory of what would have occurred in the absence of the intervention, estimated from observed data. askbiz.co applies counterfactual analysis to evaluate the impact of retailer business decisions, providing evidence-based assessments of whether pricing changes, promotions, and assortment decisions achieved their intended effects.
Synthetic Control Method for Single-Unit Interventions
The synthetic control method (SCM), developed by Abadie and Gardeazabal (2003), constructs a counterfactual for a treated unit by creating a weighted combination of untreated control units that closely matches the treated unit pre-intervention trajectory. In the retail context, the treated unit might be a product that received a price change, and the control units are similar products whose prices remained unchanged. The synthetic control is a weighted average of control product sales trajectories, with weights chosen to minimize the discrepancy between the synthetic control and the treated product in the pre-intervention period. If the synthetic control closely tracks the treated product before the intervention, it provides a credible estimate of what the treated product sales would have looked like post-intervention in the absence of the price change. The treatment effect is then estimated as the difference between the observed post-intervention sales and the synthetic control prediction. Inference is conducted through permutation tests: the same synthetic control procedure is applied to each control product in turn (as if it had been treated), producing a distribution of placebo effects against which the actual treatment effect is compared. A treatment effect larger than all placebo effects provides strong evidence of a genuine causal impact. askbiz.co implements the synthetic control method for evaluating pricing and promotional interventions, automatically identifying suitable control products and computing treatment effects with permutation-based confidence intervals.
Difference-in-Differences and Interrupted Time Series
When the synthetic control approach is infeasible (insufficient control units or poor pre-intervention fit), alternative counterfactual methods provide useful approximations. Difference-in-Differences (DiD) compares the change in the treated unit outcome before and after the intervention to the change in a control group over the same period. The identifying assumption is that treated and control units would have followed parallel trends in the absence of the intervention — a weaker assumption than the level-matching required by SCM. In retail, DiD can evaluate the impact of a store-wide policy change (e.g., new store hours) by comparing revenue changes at the affected store to revenue changes at similar stores that did not change hours. Interrupted Time Series (ITS) analysis uses only the treated unit data, fitting a time-series model to the pre-intervention period and extrapolating it to the post-intervention period as the counterfactual. The treatment effect is the difference between observed and extrapolated values. ARIMA models, segmented regression, and Bayesian structural time series (BSTS, implemented in Google CausalImpact) all provide ITS frameworks with different flexibility-robustness tradeoffs. BSTS is particularly attractive because it incorporates covariates, handles seasonality explicitly, and produces posterior distributions over the counterfactual that naturally yield credible intervals for the treatment effect. askbiz.co offers BSTS-based interrupted time series analysis as its primary single-unit counterfactual method, supplemented by DiD analysis when comparable control stores or products are available.
Heterogeneous Treatment Effects With Causal Forests
Average treatment effects, while useful, may mask important variation: a price change might boost sales for some customer segments while having no effect on others, or a promotion might be highly effective on weekdays but counterproductive on weekends. Heterogeneous treatment effect (HTE) estimation identifies how the causal impact of an intervention varies across subpopulations or conditions. Causal forests, proposed by Wager and Athey (2018), extend the random forest algorithm to estimate conditional average treatment effects. Each tree in the forest partitions the covariate space (product attributes, temporal features, customer characteristics) into regions with different treatment effects, and the forest average provides a smooth estimate of the treatment effect as a function of covariates. The resulting HTE function reveals which products, time periods, or customer segments respond most strongly to the intervention, enabling targeted strategy refinement. For example, a causal forest analysis of a promotional campaign might reveal that the promotion increased sales primarily among infrequent customers during weekday afternoons, suggesting a targeted rather than store-wide deployment in the future. Honest estimation, where the tree structure is learned on one subsample and treatment effects are estimated on a separate subsample, provides valid inference and confidence intervals for the estimated effects. askbiz.co applies causal forest analysis to evaluate how business decision impacts vary across products, time periods, and customer segments, enabling retailers to refine their strategies based on granular causal evidence.
Practical Considerations and Decision Support
Translating counterfactual analysis into a practical decision-support tool requires addressing several practical challenges. First, the timing of the analysis relative to the intervention matters: too early and the treatment effect may not have fully materialized; too late and confounding from subsequent events may contaminate the estimate. A rolling analysis that tracks the cumulative treatment effect over time, with widening confidence intervals reflecting increasing uncertainty as the intervention recedes into the past, provides a dynamic view. Second, multiple concurrent interventions (changing price and running a promotion simultaneously) create identification challenges because their individual effects cannot be separated without additional assumptions or variation. Factorial experimental designs, where different intervention combinations are applied to different products or time periods, enable joint estimation of individual and interaction effects. Third, spillover effects — where an intervention on one product affects sales of related products — must be accounted for to avoid biased treatment effect estimates. Including potentially affected products in the outcome vector and analyzing the full spillover pattern provides a more complete picture of intervention impact. askbiz.co maintains a decision journal that logs all retailer interventions with timestamps and automatically applies counterfactual analysis to evaluate each decision, building a cumulative evidence base that informs future strategy by revealing which types of decisions have historically produced the largest positive impacts.