The Decision Stack: Rules, Predictions, Optimization & LLMs

Once the decision is specified, the next question is how to implement it. There are four families of technique, and almost every argument about “rules vs ML” or “should we use an LLM for this?” is really an argument about which family fits which part of the decision.

The answer in production is nearly always all of them, in layers. A loan decision might use rules for eligibility, a model for default probability, an optimizer for credit limits under portfolio risk constraints, and an LLM to read uploaded payslips. Each layer does what it’s good at.

Layer 1: Rules

Rules are deterministic if-then logic: “if age < 18, decline”, “if account is in collections, no offers”, “if order value > $5,000 and new customer, route to manual review”.

Good at: legal and policy requirements, crisp eligibility criteria, encoding expert knowledge when there’s little data, and anything an auditor will read line by line. Rules are transparent, testable and fast.

Bad at: weighing many weak signals, adapting to change, and handling the long tail. Rule systems grow by accretion. After a few years you have 3,000 rules with unclear precedence that nobody dares delete.

Use rules for what is allowed, not for what is best. Lesson 16 covers decision tables and the DMN standard, which keep rule sets maintainable.

Layer 2: Predictive models

Predictive models estimate the uncertain quantities in the decision: the probability of default, expected demand, the chance a customer churns under each possible offer.

Good at: combining hundreds of signals, learning from history, producing probabilities that can be weighed against costs.

Bad at: telling you what to do. A model outputs a number; turning it into an action needs an objective. They also only know about actions that appear in the training data, and they confuse correlation with effect. A model trained on history where the account team called only the most loyal customers will “learn” that calls cause loyalty.

Two specializations matter enough to get their own modules:

• Causal models (Module 4) estimate what an action changes, the difference between the outcome with and without it. That’s what targeting decisions need.
• Sequential and bandit methods (Module 5) handle decisions where acting generates the data you learn from, and where you have to explore actions you’re unsure about.

Layer 3: Optimization

Optimization chooses the best combination of actions given an objective and constraints. Linear programming, mixed-integer programming, constraint programming and their relatives.

Good at: allocation under shared limits. Budget across customers, inventory across stores, drivers across deliveries, staff across shifts. Whenever one decision affects what’s available for another, you need optimization, because choosing the best action for each unit independently ignores the coupling.

Bad at: dealing with uncertainty by itself. A solver takes inputs as given. Feed it point forecasts and it’ll confidently exploit their errors. Lessons 8 and 9 deal with this.

Layer 4: LLM components

Large language models are the newest layer, and the one most often misused.

Good at: reading unstructured input (support tickets, emails, contracts, call transcripts) and producing structured output; classifying intent and sentiment; drafting explanations and customer messages; acting as a flexible interface for people who review decisions.

Bad at: consistent arithmetic, producing calibrated probabilities, behaving identically on identical inputs, and resisting adversarial input. An LLM that decides loan approvals directly is a system you can’t audit, can’t reproduce, and can’t defend.

The pattern that works: LLMs as sensors (turn text into features the rest of the stack consumes) and drafters (produce explanations or messages that are checked before use), wrapped in validation. Lessons 17 and 18 build this.

How the layers fit together

The standard execution order in a production decision service:

flowchart TB
    IN["Decision request\nunit + context"] --> E["1. Eligibility rules\nremove disallowed actions"]
    E --> F["2. Feature building\nincl. LLM extraction from text"]
    F --> S["3. Scoring\npredict outcome per eligible action"]
    S --> O["4. Optimization / selection\nmaximize objective under constraints"]
    O --> G["5. Guardrails\npost-checks, escalation to humans"]
    G --> OUT["Action + reason codes\nlogged with propensity"]

    style IN fill:#EEF0F7,stroke:#6366F1,color:#0F172A
    style E fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style F fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style S fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style O fill:#f0fdf9,stroke:#0D9488,color:#0F172A
    style G fill:#fff7ed,stroke:#f59e0b,color:#0F172A
    style OUT fill:#EEF0F7,stroke:#6366F1,color:#0F172A

Rules come first because there is no point scoring actions you aren’t allowed to take, and because filtering early makes the scoring and optimization cheaper. Guardrails come last because they check the final decision against limits the optimizer might have bent: unusual action volumes, conflicts with recent decisions for the same customer, or inputs that look like an attack.

The output isn’t just an action. It’s an action plus the reason codes that explain it and the propensity (the probability this action was chosen), both logged. You’ll need reasons for explanations and audits (Lesson 22) and propensities for evaluating future policies from logs (Lesson 15).

Choosing the right layer

When you’re deciding how to implement a piece of a decision, these questions point at the right layer:

Here’s what the running retention example looks like as a skeleton. Every later module fills in one of these functions:

from dataclasses import dataclass

@dataclass
class Decision:
    customer_id: str
    action: str
    expected_value: float
    reasons: list[str]
    propensity: float          # P(this action | context) under the policy
    policy_version: str

def decide_batch(customers, budget) -> list[Decision]:
    # 1. Rules: eligible actions per customer (Lesson 16)
    eligible = {c.id: eligible_actions(c) for c in customers}

    # 2-3. Features + scoring: uplift per (customer, action) (Modules 2, 4, 6)
    values = {
        (c.id, a): expected_margin_gain(c, a)
        for c in customers for a in eligible[c.id]
    }

    # 4. Optimization: allocate the budget (Module 3)
    plan = allocate_offers(values, costs=OFFER_COSTS, budget=budget)

    # 5. Guardrails + exploration + logging (Modules 5, 6, 8)
    return [apply_guardrails(d) for d in add_exploration(plan)]

What not to do

Three failure patterns show up again and again:

• One layer for everything. A 3,000-rule system trying to do what a model should, or a model trying to learn a legal requirement from examples. Use each layer for its strength.
• LLM as the whole stack. “Give the model the customer record and ask what offer to send” produces a demo in a day and a system you can’t evaluate, explain or control in production.
• Optimization on uncalibrated inputs. Solvers trust their inputs completely. If the probabilities going in are off, the optimizer will find and exploit every error.

That last point is where Module 2 starts: making predictions good enough for a decision to trust.