Uplift Modeling: Targeting the Persuadable

Back in Lesson 1, a churn programme lost money by sending discounts to the riskiest customers. Now we can say precisely what went wrong. The decision needed the effect of the offer on each customer, and the model provided the risk of churning. Uplift modeling estimates the effect directly.

Four kinds of customer

Every customer, for a given action, falls into one of four groups depending on their two potential outcomes:

Only persuadables create value. Sure things take the discount and would have stayed anyway: pure cost. Lost causes leave regardless. Sleeping dogs are real and common: a retention email reminds a dormant subscriber that they’re paying for something they don’t use, and they cancel.

Risk-based targeting picks high-churn customers, which is a mix of persuadables and lost causes, and the proportion is unknown. Uplift-based targeting aims at persuadables directly.

Data: randomized is best

Uplift models learn from data where some customers received the action and some didn’t. If assignment was randomized, the comparison is clean. If it was driven by a past policy, you’re back to the confounding problems of the last lesson and need the adjustments of Lesson 12.

For a new programme, the standard approach is a randomized pilot: send the offer to a random half of an eligible population for a month or two, record outcomes, and train on that. Ongoing, the permanent randomized holdout keeps feeding fresh, unconfounded data.

Meta-learners

Meta-learners build uplift estimators out of ordinary supervised models. Three are worth knowing.

S-learner (“single”): train one model with the treatment as a feature, then predict each customer twice, once with treatment = 1 and once with treatment = 0. Simple, but regularized models often barely use the treatment feature when its effect is small compared to the baseline signal, so S-learner uplift is shrunk towards zero.

T-learner (“two”): train separate models on the treated and control groups, and take the difference.

import numpy as np
from sklearn.ensemble import HistGradientBoostingClassifier

def t_learner(X, t, y):
    m1 = HistGradientBoostingClassifier(max_iter=300).fit(X[t == 1], y[t == 1])
    m0 = HistGradientBoostingClassifier(max_iter=300).fit(X[t == 0], y[t == 0])
    def uplift(X_new):
        # Estimated change in P(retained) from sending the offer
        return m1.predict_proba(X_new)[:, 1] - m0.predict_proba(X_new)[:, 1]
    return uplift

Straightforward and often decent, but each model sees only part of the data, and the difference of two noisy estimates is noisier still. With unbalanced groups (5% treated) the small model is weak.

X-learner (Künzel et al., 2019): starts like a T-learner, then imputes individual effects for each group using the other group’s model (for treated customers: observed outcome minus control-model prediction; for controls: treated-model prediction minus observed outcome), fits models on those imputed effects, and blends them weighted by the propensity. It does especially well when one group is much smaller than the other, which is the usual situation.

Beyond meta-learners, causal forests and double machine learning (next lesson) estimate CATE directly with built-in honesty and confidence intervals. Libraries such as EconML and CausalML implement all of these.

Evaluating uplift: Qini curves

You can’t compute the accuracy of an uplift model on individuals, because nobody’s true effect is ever observed. Instead, evaluate on held-out randomized data by ranking.

Sort customers by predicted uplift, highest first. For each fraction k of the ranked list, compare the outcome rate among treated customers in the top k to the rate among controls in the top k. A good model concentrates the real effect at the top.

def qini_curve(uplift_pred, t, y, n_points=50):
    order = np.argsort(-uplift_pred)
    t, y = t[order], y[order]
    n_t = np.cumsum(t)               # treated so far
    n_c = np.cumsum(1 - t)           # controls so far
    y_t = np.cumsum(y * t)           # treated successes so far
    y_c = np.cumsum(y * (1 - t))     # control successes so far
    # Incremental successes if the top-k had all been treated,
    # scaling control successes to the treated group size.
    qini = y_t - y_c * np.divide(n_t, n_c, out=np.zeros_like(n_t, dtype=float), where=n_c > 0)
    idx = np.linspace(0, len(y) - 1, n_points).astype(int)
    return idx / len(y), qini[idx]

Compare the curve with the straight line from random targeting. The area between them (the Qini coefficient) summarizes how well the model ranks. Also check the uplift by decile: the observed treated-minus-control difference in each predicted decile. You want it to fall monotonically from top to bottom decile, and the bottom deciles may well be negative (sleeping dogs).

Watch the sample sizes. Uplift is a difference of two rates, so it’s noisy. Bootstrapped confidence bands on the Qini curve stop you from celebrating noise.

From uplift to a targeting policy

Uplift is a change in probability. To decide, convert it to money:

def expected_gain(uplift, p_retain_control, clv, discount_cost, contact_cost):
    # Gain from customers the offer saves...
    saved_value = uplift * clv
    # ...minus the discount paid by everyone who stays with the offer,
    # including the sure things who would have stayed anyway.
    p_retain_treated = p_retain_control + uplift
    discount_paid = p_retain_treated * discount_cost
    return saved_value - discount_paid - contact_cost

That second term matters. The discount is paid by everyone who stays while holding the offer, sure things included. An offer with positive uplift can still be unprofitable if most recipients are sure things. This is why uplift modeling must estimate both the effect and the baseline.

With multiple treatments (10% discount, 25% discount, upgrade, call), estimate uplift for each treatment against control, compute expected gain for each, and choose the best per customer, with “none” worth zero. When treatments share a budget, these expected gains are exactly the value[c, a] inputs to the allocation model in Lesson 7.

Practical cautions

• Uplift signals are weak. Effects of a few percentage points are typical. You need large samples, careful validation and humility.
• Effects change. An offer that worked last year may not work after a competitor launches a cheaper plan. Retrain on recent randomized data.
• Effects interact with frequency. The second discount in a year is worth less than the first, and customers learn to wait for offers. Track this explicitly.
• Don’t extrapolate. If the randomized pilot only covered annual-plan customers, the model knows nothing about monthly ones.

When randomized data isn’t available, you need to estimate effects from observational logs. That’s harder, riskier and sometimes unavoidable, and it’s next.