Churn prediction system
SignalForge: churn prediction with statistical rigor
Churn prediction system with statistical rigor. 0.849 AUC on 7,043 customers, $1.67M revenue at risk, and a retention ROI framework.
7,043 rows
7,043 customer records
0.849 ± 0.012 AUC
95% CI [0.828, 0.869] • p=0.016
$1.67M Revenue at Risk
1.21x-1.81x Expected ROI
Statistical Methods
Bootstrap Confidence Intervals
Quantify uncertainty in all metrics
5-Fold Cross-Validation
Robust estimates vs single train/test split
Statistical Significance Testing
Prove model superiority with p-values
Calibration Analysis
Verify predicted probabilities match reality
Learned Feature Weights
Data-driven optimization vs hard-coded weights
Key Findings
Churn prediction system built with 5-fold CV, Optuna Bayesian tuning (20 trials), bootstrap confidence intervals, and significance testing. Real data, real methods, real results.
Logistic Regression achieved 0.849 +/- 0.012 AUC [95% CI: 0.828, 0.869], statistically significantly better than Random Forest (p=0.016) but not GB (p=0.130). Simple model, competitive performance, interpretable.
$1.67M annual revenue at risk. Built a retention ROI framework showing 1.21x-1.81x expected return on intervention spend. The question isn't whether to act - it's how much to spend.
Feature engineering via Ridge regression. Contract type turned out to be 2x more important than initial assumptions suggested. Data-driven discovery, not hard-coded guesses.
Predict who is risky, target who is saveable
A churn-risk model, uplift-based intervention strategy, and drift monitor packaged into an executive-facing decision science demo.
ROC AUC on holdout months
Higher churn concentration in the most at-risk accounts
Correlation between predicted and true intervention lift
$251.4K net value from a 32-account save budget
Monthly Model Tracking
Why It Matters
The model is evaluated on future months instead of a random split, which makes the score more believable in a real customer-success workflow.
SignalForge is intentionally not a perfect synthetic problem. AUC is strong enough to be useful, but not so high that it looks fabricated.
Latest average uplift: 18.0% per targeted account.