Hierarchical Forecasting in Product Category Structures: Reconciling SKU-Level and Category-Level Predictions From PoS Data
Address the coherence problem in multi-level forecasting, ensuring SKU-level predictions sum to category-level forecasts using reconciliation approaches.
Key Takeaways
- Independent forecasts at different aggregation levels (SKU, subcategory, category, total store) are generally incoherent — they do not sum consistently across the hierarchy, creating conflicting planning signals.
- Optimal reconciliation methods such as MinT produce coherent forecasts that are provably at least as accurate as the best independent forecast at each level of the hierarchy.
- The choice between top-down, bottom-up, and optimal reconciliation depends on data availability and forecast accuracy at different hierarchy levels, with bottom-up generally preferred when SKU-level data is sufficient.
The Coherence Problem in Hierarchical Data
Retail product catalogs are inherently hierarchical: individual SKUs belong to subcategories, which belong to categories, which aggregate to total store level. Business decisions are made at different levels of this hierarchy — purchasing decisions are SKU-specific, category management operates at the category level, and financial planning uses total-store aggregates — and each level requires forecasts. When forecasts are generated independently at each level using the data and models appropriate to that level, the results are almost certainly incoherent: the sum of SKU-level forecasts within a subcategory does not equal the subcategory forecast, and the sum of subcategory forecasts does not equal the category forecast. This incoherence creates conflicting signals that undermine coordinated planning. A purchasing manager ordering individual SKUs based on SKU-level forecasts may procure a total quantity inconsistent with the category-level demand that the category manager is planning for, and the aggregate procurement spend may not match the financial plan based on total-store forecasts. Reconciliation — the process of adjusting forecasts across the hierarchy to ensure coherence while preserving or improving forecast accuracy — resolves this conflict. askbiz.co automatically reconciles PoS-derived demand forecasts across the product hierarchy, ensuring that every level of aggregation receives coherent, consistent planning signals.
Top-Down, Bottom-Up, and Middle-Out Approaches
Three classical approaches to hierarchical forecasting address the coherence problem by generating forecasts at only one level and deriving other levels through aggregation or disaggregation. Bottom-up forecasting generates independent forecasts at the lowest (SKU) level and aggregates them upward through the hierarchy by summation. This approach preserves SKU-level detail and does not lose information through aggregation, but SKU-level forecasts are often noisy due to sparse data, and errors accumulate through aggregation. Top-down forecasting generates a single forecast at the highest (total store or category) level and disaggregates it downward using historical proportions — each SKU receives a share of the category forecast proportional to its historical share of category demand. This approach leverages the smoother, more predictable aggregate series but imposes the strong assumption that historical proportions persist, failing to capture SKU-level trend changes or new product introductions. Middle-out approaches forecast at an intermediate level (subcategory) and use bottom-up aggregation upward and top-down disaggregation downward, attempting to balance the advantages of both extremes. All three classical approaches sacrifice information by constraining the forecasting to a single hierarchy level. askbiz.co evaluates the relative forecast accuracy at each hierarchy level and selects the classical approach that minimizes aggregate error when a simpler method is preferred to optimal reconciliation.
Optimal Reconciliation: The MinT Framework
Optimal reconciliation, formalized by Wickramasuriya, Athanasopoulos, and Hyndman (2019) through the Minimum Trace (MinT) reconciliation method, generates independent base forecasts at every level of the hierarchy and then adjusts them to achieve coherence while minimizing the total forecast error variance. The method projects the vector of base forecasts onto the coherent subspace (the set of forecast vectors where aggregation constraints are satisfied) using a linear mapping that depends on the covariance matrix of the base forecast errors. When the base forecast error covariance is accurately estimated, MinT-reconciled forecasts are provably at least as accurate as the base forecasts at every level of the hierarchy — the reconciliation process improves accuracy by exploiting the information contained in forecasts at all levels. Practical estimation of the error covariance matrix is challenging, particularly for retailers with large product hierarchies, and several approximations have been proposed. The simplest assumes equal variance for all series (ordinary least squares reconciliation). The diagonal approximation (WLS reconciliation) estimates only the variances, ignoring cross-series correlations. The full sample covariance (MinT with sample covariance) captures correlations but requires sufficient historical data to estimate the high-dimensional covariance matrix. Shrinkage covariance estimators provide a compromise between the stability of diagonal estimates and the informativeness of full covariance. askbiz.co implements MinT reconciliation with shrinkage covariance estimation, automatically generating coherent demand forecasts across the product hierarchy from independently produced base forecasts.
Temporal Hierarchies and Cross-Temporal Reconciliation
The hierarchical coherence problem extends beyond product aggregation to temporal aggregation. Daily forecasts should sum to weekly forecasts, which should sum to monthly forecasts, which should sum to quarterly forecasts. Independent forecasts at different temporal granularities typically violate these summation constraints, just as independent forecasts at different product hierarchy levels do. Temporal hierarchical reconciliation, introduced by Athanasopoulos, Hyndman, Kourentzes, and Petropoulos (2017), applies the same reconciliation framework to the temporal dimension. Cross-temporal reconciliation, the most comprehensive approach, reconciles simultaneously across both the product and temporal dimensions, ensuring that forecasts are coherent in both dimensions. This cross-temporal approach produces the most constrained and therefore most coherent forecasts but requires the largest covariance matrix estimation, which can become computationally challenging for large product hierarchies at fine temporal granularity. In practice, micro-retailers with moderate catalog sizes (hundreds to a few thousand SKUs) can implement full cross-temporal reconciliation with manageable computational cost, while larger retailers may need to apply reconciliation sequentially (product reconciliation first, then temporal) as an approximation. askbiz.co supports both product-dimension and temporal-dimension reconciliation, ensuring that demand forecasts are coherent whether they are consumed for daily operational planning, weekly purchasing, or monthly financial forecasting.
Implementation and Forecasting System Integration
Implementing hierarchical reconciliation in a production forecasting system requires attention to several engineering and process considerations. The product hierarchy must be clearly defined and consistently maintained — changes to the hierarchy (adding categories, reclassifying products, introducing new hierarchy levels) require re-mapping that can affect reconciliation performance. Base forecast generation should be parallelizable across the hierarchy, with each series receiving the most appropriate forecasting method for its aggregation level and data characteristics. SKU-level series may benefit from intermittent demand methods, while category-level series may be well-served by exponential smoothing or ARIMA. The reconciliation step itself is a matrix operation that can be computed efficiently but requires maintaining the estimated covariance matrix, which must be updated periodically as forecast error properties evolve. Forecast accuracy monitoring should track both base forecast accuracy (before reconciliation) and reconciled forecast accuracy (after reconciliation) at each hierarchy level to verify that reconciliation is providing the expected improvement. If reconciled forecasts are worse than base forecasts at certain levels, this may indicate covariance estimation problems or hierarchy structural issues that need attention. askbiz.co integrates hierarchical reconciliation as an automatic post-processing step in its forecasting pipeline, monitoring accuracy improvements from reconciliation at each hierarchy level and alerting when reconciliation performance degrades.