Accuracy & Limitations

Accuracy and limitations

Forests don’t move on startup timelines. A tree planted today won’t tell you whether we were right for years. We take this seriously — it means we owe you more transparency, not less.

Model performance

Mycel v0.2 performance metrics will be published here as validation analysis is completed. We are committed to reporting discrimination (AUC-ROC), calibration (predicted vs. observed survival rates), and per-species accuracy transparently.

We validate through hindcasting — predicting outcomes for historical sites where we already know what happened, using only the information that would have been available at planting time.

What the model does well

Species-site matching. The model captures the ecological relationships between species and site conditions that experienced foresters know intuitively — Douglas Fir thrives on moderate, well-drained slopes; Sitka Spruce needs coastal moisture; Ponderosa Pine tolerates dry, east-side conditions. These patterns are grounded in decades of FIA field observations.

Relative ranking. Even where absolute probabilities carry uncertainty, the model is strong at ranking sites from best to worst for a given species. If Canopi predicts 85% at Site A and 65% at Site B, Site A is very likely to outperform Site B even if the absolute numbers shift.

Risk factor identification. The SHAP-based risk factors identify genuine ecological stressors. High vapor pressure deficit, low soil water capacity, and elevation-related cold stress consistently appear as risk factors in contexts where forestry literature confirms their importance.

Known limitations

Match distance. Predictions come from the nearest pre-computed data point, not from your exact coordinates. In areas with sparse plot coverage, the matched site may be kilometers away, and local microsite conditions (aspect, soil pockets, drainage patterns) may differ significantly.

Extreme events. The model learns from historical climate patterns. A once-in-a-century drought, an unprecedented heat dome, or a novel pest outbreak are outside the training distribution. Canopi’s predictions assume future conditions broadly resemble historical patterns.

Seedling vs. established tree. FIA measures trees that were already established enough to be inventoried. There’s an implicit assumption that conditions favoring established tree survival also favor seedling survival. This is generally true but imperfect — first-year seedling mortality involves stressors (transplant shock, browse, competition from grass/shrubs) that FIA’s mature-tree data doesn’t directly capture.

Planting method evidence. The method effect in the model is based on how the planting method variable interacts with site conditions in the training framework. Direct paired-comparison data (same site, both methods, measured outcomes) is limited. We are transparent about this and actively seeking validation partnerships with reforestation operators who can provide real-world outcome data.

Geographic coverage. Currently limited to Oregon and Washington. Predictions are unavailable for other regions. National expansion is in development.

Temporal resolution. Three time horizons (1, 3, 5 years) provide decision-relevant intervals but don’t capture month-by-month survival dynamics. The critical first 90 days after planting — when the highest mortality occurs — is not separately modeled.

Our commitment

We will be wrong sometimes. Ecological systems are complex and stochastic. A freak ice storm, an unexpected pest outbreak, or a drought that exceeds historical precedent can invalidate any prediction.

What we promise is not perfection but transparency: that we report our accuracy honestly, that we include uncertainty where appropriate, that we state our limitations clearly, and that we improve continuously as evidence accumulates. Each growing season makes Canopi more accurate and more trustworthy.

If you have planting outcome data and would be interested in a validation partnership, we’d welcome the conversation. Every real-world data point makes the root network stronger.