A historical forecast comparison
An archived internal analysis compares monthly forecasts with recorded vaccine administration volumes for 41 pediatric offices. The project records identify the dataset as calendar-year 2024 and 2025 Vaccines for Children (VFC) administration data.
- Unit of evaluation: total VFC doses per office, per month; 492 office-months.
- Comparison period: calendar year 2025.
- Baseline: predict the same number of doses as the corresponding month in 2024.
- Error measure: monthly mean absolute percentage error (MAPE). Lower is better.
The baseline is a forecasting comparison. It does not represent the offices’ actual purchase orders or establish how individual teams made ordering decisions.
Results in the archived dataset
| Measure | Archived forecast | Prior-year baseline |
|---|---|---|
| Mean monthly error (MAPE) | 15.5% | 18.7% |
| Offices with lower monthly error | 38 of 41 | 3 of 41 |
For each office-month, percentage error is the absolute difference between forecast and recorded volume, divided by recorded volume. We average across the 492 office-months. Each office-month has equal weight; all recorded volumes in this dataset are greater than zero. Results are rounded to one decimal place.
The archived forecasts had lower average monthly error than the prior-year baseline at 38 offices, or approximately 93% of this sample. This is the share of offices where the forecast performed better. It is not a 93% forecast-accuracy rate.
What these results establish
The saved forecast and actual-volume series support the arithmetic above. They describe an earlier internal analysis, not an independent validation of the current product.
- These are office-level monthly totals. They do not establish accuracy for individual vaccines or specific order quantities.
- The analysis covers VFC volumes in one network. It does not establish commercial purchasing outcomes or performance across other health systems.
- The current product has changed since the historical analysis. Prospective performance requires its own evaluation with forecasts recorded before the outcome period.
- These figures do not measure savings, prevented stockouts, reduced waste, or staff time saved.
Measuring operational impact
We work with each organization to define the participating offices, baseline, observation period, and data needed for the outcome being evaluated.
- Inventory held: track recorded commercial stock and potential excess with a clearly labeled price basis. Lower inventory held and realized savings are different measures.
- Purchasing and waste: use actual purchasing costs, receipts, and documented waste to evaluate changes. List-price estimates do not establish realized savings.
- Staff time: measure the full workflow, including counting and ordering. Time spent entering data alone does not establish time saved.
- Availability: distinguish a low-stock alert from a documented stockout, and track events over the agreed period.
Missing counts, unrecorded deliveries, and changes in office participation can affect comparisons. Those limitations need to be visible in the result.
Discuss your evaluation
We can discuss the data inputs, measurement definitions, and evaluation plan appropriate for your organization.