Model Accuracy
Active models
| Model | AUC-ROC | Decile lift |
| MLB Start/sit confidence v5
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2024 · tested on 2025 |
0.625 | 27.0x |
| MLB Pitcher form v1
Whether a start clears the pitcher's own trailing three-start baseline. Shown as Trending Up, Steady or Trending Down, not a percentage.
Trained 2021–2024 · tested on 2025 |
0.636 | 2.5x |
| NBA Start/sit confidence v3
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2024 · tested on 2025 |
0.847 | 71.3x |
| NBA Heat (hot/cold form) v2
Whether a player is running hot or cold against his own recent baseline.
Trained — · tested on — |
0.741 | 4.7x |
| NFL Start/sit confidence v1
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2024 · tested on 2025 |
0.697 | 4.6x |
| NFL Heat (hot/cold form) v2
Whether a player is running hot or cold against his own recent baseline.
Trained 2021–2024 · tested on 2025 |
0.623 | 25.6x |
| NHL Start/sit confidence v1
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2023 · tested on 2024 |
0.669 | 37.0x |
Retired models
Replaced or pulled. Their numbers stay here.
| MLB Start/sit confidence v1
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2024 · tested on 2025 |
0.570 | —x |
| NBA Start/sit confidence v1
A calibrated probability shown on player pages and in lineup advice.
Trained 2021–2024 · tested on 2025 |
0.845 | 45.1x |
| NFL Heat (hot/cold form) v1
Whether a player is running hot or cold against his own recent baseline.
Trained 2021–2024 · tested on 2025 |
0.534 | —x |
How accurate are PlayCaller's fantasy predictions?
PlayCaller runs 7 production models. Measured on a held-out season none of them trained on, their AUC-ROC ranges from 0.623 to 0.847 (highest: NBA Start/sit confidence). Each one cleared its promotion gate on held-out data before it shipped.
What do AUC-ROC and decile lift mean?
AUC-ROC is the chance the model ranks a random hit above a random miss: 0.5 is a coin flip, 1.0 is perfect. Decile lift compares the hit rate of the model's top 10% of predictions with its bottom 10% on the test season.
Does PlayCaller show scores without a validated model?
No. A score, rating or confidence value only appears when a model with a measured AUC backs it. Anything else is labeled as a heuristic or not shown.