Predicting SaaS Churn with Machine Learning — Before It Happens

Every SaaS finance leader knows the feeling: a customer cancels, and in hindsight the signals were there — declining login frequency, a support ticket that went unanswered, a renewal conversation that never happened. The problem with traditional churn analysis is that it is backward-looking. You measure churn after it has happened and ask why. Machine learning lets you measure churn risk before it happens and ask who.

This article explains how classification models — specifically Random Forest and XGBoost — can be applied to SaaS customer data to produce a weekly churn probability score per account, and what that score enables in practice.

Why Churn Is a Classification Problem

At its core, predicting churn is a binary classification task: for each customer, in the next 30, 60 or 90 days, will they churn (1) or not (0)? Classification algorithms learn from historical examples where the outcome is already known — customers who churned in the past — to recognise the pattern of features that preceded that churn.

The three steps are: engineer the features (what do we know about each customer right now?), train the model on historical data (which features predicted who churned?), and score every live customer on a recurring basis (who looks like a churner today?).

Feature Engineering: What Do You Actually Know About a Customer?

The quality of a churn model depends almost entirely on the quality of its input features. For a typical B2B SaaS business, the most predictive features fall into five categories:

• Usage behaviour: login frequency over the last 7, 30 and 90 days; number of active users vs. licensed seats; feature adoption rate (what % of available features has the account ever used?); time since last meaningful session.
• Support signals: number of support tickets in the last 90 days; sentiment of recent tickets (if available); unresolved ticket count; escalated ticket history.
• Commercial signals: contract value; contract age; number of previous renewals; whether they came in on a discount; expansion or contraction since original contract.
• Payment behaviour: days overdue on most recent invoice; number of late payments in contract history; whether the direct debit has ever failed.
• Engagement signals: NPS score trend; attendance at webinars or training sessions; usage of the knowledge base; whether there is an active CSM relationship.

You do not need all of these. A model with five well-chosen features often outperforms one with fifty poorly-chosen ones. Start with what you can extract cleanly from your billing system and your product usage data, and build from there.

Model Choice: Why Random Forest and XGBoost

Logistic regression is often the starting point for a classification problem — it is fast, interpretable and gives you a probability output. Use it as your baseline. If it produces acceptable precision and recall, it may be all you need.

Random Forest and XGBoost consistently outperform logistic regression on real SaaS datasets because churn prediction has two characteristics that suit ensemble methods:

• Non-linear relationships: The relationship between feature values and churn probability is rarely linear. An account that logs in twice a week for three months and then drops to once a month is far more at risk than one that has always logged in once a month — but a linear model cannot capture this change signal without additional feature engineering.
• Feature interactions: A customer with declining logins AND a recent support ticket AND a contract up for renewal in 45 days is much more at risk than any one of those signals alone. Tree-based models capture these interactions naturally.

Random Forest builds many decision trees on random subsets of data and features, then averages their predictions. It is robust, handles missing data reasonably well, and provides built-in feature importance scores that tell you which signals are driving the predictions.

XGBoost (Extreme Gradient Boosting) builds trees sequentially, each one correcting the errors of the previous. It typically achieves higher accuracy than Random Forest but is more sensitive to hyperparameter choices. For a SaaS churn model, the improvement over Random Forest is often marginal but worth testing.

Model Evaluation: The Metrics That Matter

Accuracy — the % of predictions that are correct — is the wrong metric for churn prediction. If 5% of your customers churn in a given quarter, a model that predicts “no one will churn” for every customer has 95% accuracy. It is useless.

The metrics that matter for a churn model are:

• Precision: Of the accounts the model flagged as high-risk, what % actually churned? Low precision means the customer success team is wasting time on false alarms.
• Recall: Of the accounts that actually churned, what % did the model catch? Low recall means churners are slipping through without being flagged.
• F1 Score: The harmonic mean of precision and recall — useful when you need a single number to compare models.
• ROC-AUC: The area under the receiver operating characteristic curve — measures how well the model distinguishes churners from non-churners across all possible probability thresholds. A value above 0.80 is good for a SaaS churn model on real data.

The trade-off between precision and recall is a business decision, not a technical one. A customer success team of 3 can only handle 20 flagged accounts per week — you want high precision. A high-value enterprise customer base where missing one churn costs £100k — you want high recall and are willing to accept more false positives.

What the Output Looks Like in Practice

Once trained, the model scores every active customer and assigns a churn probability — a number between 0 and 1. A score of 0.78 means the model estimates a 78% probability that this account will churn in the next 90 days based on its current feature values.

The output is delivered as a simple ranked table, updated weekly, that the customer success team works from the top down. It answers one question: who should I call first this week? The model does not tell the CSM what to say — that requires human relationship context — but it answers the prioritisation question with statistical rigour rather than gut feel.

The second output is feature importance. Random Forest and XGBoost both provide a score for each feature reflecting how much it contributed to predictions across the entire dataset. In a typical SaaS model, the top features are usually login frequency trend (change over 90 days), days to renewal, and support ticket volume. Feature importance makes the model explainable to a board — you can say exactly why an account was flagged, not just that the algorithm said so.

Building This Into Finance Reporting

From a CFO perspective, the churn model has a direct financial application beyond customer success prioritisation. The weekly probability scores can be used to build a statistically-grounded churn forecast: multiply each at-risk account’s ARR contribution by its churn probability, sum across the book, and you have an expected churn figure for the next quarter based on the current risk profile of the customer base.

This feeds directly into the ARR waterfall forecast and the cashflow model. Instead of forecasting churn as “same as last quarter” or “judgment call at 5%”, you have a bottom-up, customer-by-customer estimate grounded in actual usage data. When you present this to a PE board, you can show them the methodology — and when they ask why churn is forecast to increase next quarter, you can point to the specific cohort of accounts that are showing elevated risk signals right now.

This is what machine learning actually delivers in a finance context: not magic, but a systematic, repeatable, explainable way to turn operational data into financial forecasts that are more accurate than gut feel.