Rational choice maximizes expected utility, not expected money — the same dollar amount carries less subjective value the more you already hold. This seemingly technical correction underpins the entire modern decision framework, from insurance to portfolio theory. Yet it carries a time bomb inside it: real humans systematically violate its axioms.
Von Neumann and Morgenstern laid down four axioms (completeness, transitivity, continuity, independence) and proved that if your preferences satisfy them, there must exist a utility function u such that you "ought to" pick the option with the highest expected utility. The concavity of u captures risk aversion — another 1M means far less to someone already worth tens of millions than to someone with nothing. That is exactly why people buy insurance: trading a certain small loss to erase an uncertain catastrophe.
| Experiment | Option A / C | Option B / D | Majority pick |
|---|---|---|---|
| First | 1M for sure | 89% get 1M · 10% get 5M · 1% nothing | A (play safe) |
| Second | 11% get 1M · 89% nothing | 10% get 5M · 90% nothing | D (chase upside) |
The St. Petersburg paradox: a gamble with infinite expected payoff, yet no one will pay more than a few dollars to enter — expected money collapses here, and only a logarithmic utility yields a sane bid. Sharper still is the Allais paradox above: the same people reverse preferences across two structurally equivalent choices, declaring that real preferences cannot be represented by any utility function at all. That crack later gave birth to prospect theory.
Concave utility is diminishing marginal utility, sharing roots with economic marginalism. In reinforcement learning the agent maximizes "expected return," and reward shaping is essentially designing a utility function for the system. In evolutionary biology, "fitness" — what natural selection optimizes — is life's utility; and risk aversion follows rigorously from "going bust means elimination" under multiplicative growth, the heart of the Kelly criterion.
When you design an AI product's objective function, you are in fact defining "utility" on the system's behalf. Optimize only for expected clicks and it will diverge from the user's long-run utility — and the spot where the independence axiom is shattered by reality is often exactly where a product begins to do harm. Investing is the same: think in utility rather than dollars and you will naturally decline bets that are "positive expected value but might force you off the table."
▸ Prompt: In your last major bet, were you optimizing expected money — or did you seriously price out whether the worst case could knock you out of the game?The brain doesn't compute the optimum; it runs "good enough" shortcuts — heuristics. Efficient and reliable in ancestral settings, they fail systematically in modern statistical contexts. Bias is not random noise but a predictable, directional distortion. And because it is predictable, it can be both exploited and hedged against.
Simon proposed "bounded rationality": with limited cognitive resources, people satisfice rather than optimize. Kahneman and Tversky distilled three master heuristics — representativeness (substituting similarity for probability, thus ignoring base rates), availability (substituting ease of recall for true frequency), and anchoring-and-adjustment (grabbing a starting number and adjusting insufficiently). The key word is "systematic": the same person leans the same direction in the same kind of situation, so bias can be predicted, engineered, or deliberately corrected.
The Linda problem: after describing a philosophy graduate concerned with social justice, most people judge "she is a bank teller and active in the feminist movement" more probable than "she is a bank teller" — violating the most basic law of probability, since a conjunction can never be more likely than one of its parts. The anchoring experiment is even more absurd: spin an obviously random wheel, then estimate the share of African nations in the UN — the larger the wheel's number, the higher the estimate. A knowingly irrelevant number still hijacked judgment.
The opposing camp offers "ecological rationality": those same fast-and-frugal heuristics (like the recognition heuristic) often beat complex models in environments whose information structure matches them. This maps neatly onto the bias-variance tradeoff in machine learning — simple, high-bias models generalize more stably on small samples. And in AI safety, large language models faithfully inherit the human biases in their training data, amplifying cognitive shortcuts into systematic output tendencies.
As a decision-maker, the practical move isn't "eliminating bias" (nearly impossible) but reshaping the decision environment to hedge it: a base-rate checklist against representativeness, pre-committed criteria against anchoring. The same applies with kids — a child's fear of the dark is usually availability at work: vivid, terrifying imagery overwhelms the true probability, and switching on a nightlight beats any lecture.
▸ Prompt: Recall the last time you snap-judged someone or some plan as "solid." Which shortcut did you use? If you'd been forced to write down the relevant base rate first, would the conclusion survive?Major decisions are almost never one-dimensional. Decomposing the fuzzy agony of "which house, which offer, which hire" into an explicit "attributes × weights" structure tends to beat holistic intuition — not because the arithmetic is precise, but because it forces hidden tradeoffs onto the table.
Score each option on several attributes (salary, growth, commute, culture…), assign weights to the attributes, and take a weighted sum — this is multi-attribute utility theory (MAUT). Its core value lies not in the final number but in the process: it forces you to surface what you actually care about (the weights), how far apart the options truly are (the scores), and breaks the holistic intuition that gets "hijacked" by a single vivid attribute.
| Attribute (weight) | Offer A | Offer B |
|---|---|---|
| Salary ×0.30 | 9 → 2.70 | 7 → 2.10 |
| Growth ×0.35 | 6 → 2.10 | 9 → 3.15 |
| Commute ×0.15 | 8 → 1.20 | 5 → 0.75 |
| Culture ×0.20 | 7 → 1.40 | 8 → 1.60 |
| Weighted total | 7.40 | 7.60 |
Franklin's "moral algebra": faced with a hard call, fold a page in half, list pros on one side and cons on the other, assign each a weight, cancel them out, and see what remains. Counterintuitively, research shows that for extremely complex multi-attribute choices, "understand fully, then let it incubate unconsciously" can be more satisfying than item-by-item calculation — linear models excel at consistency, intuition at integrating many weak cues. The best strategy is often to cross-check the two: when the computed winner and your gut collide, that conflict is exactly where you should dig.
It is formally identical to machine learning's "weighted sum of features" (linear regression, the perceptron) — attributes are features, weights are parameters; a recommender's ranking is large-scale multi-attribute decision making. In management it's the "decision matrix" and goal-alignment scoring; in ethics, Bentham's felicific calculus (intensity × duration × probability of pleasure) is humanity's earliest attempt at multi-attribute utility.
When hiring for a senior role or choosing a tech stack, rather than getting stuck on "this one feels better," first get the team to agree on the weights — that step alone dissolves much of the disagreement — then have everyone score independently. But remember: when the matrix's winner makes your stomach drop, don't ignore that signal; it usually points to a hidden weight you never wrote down.
▸ Prompt: For your last agonizing choice, if forced to write out attribute weights, which weight was actually your largest — yet one you'd be embarrassed to say out loud?When probabilities simply cannot be estimated — true uncertainty, not quantifiable risk — "maximize expected utility" breaks down. A different criterion surfaces: not the optimum, but the choice you'll regret least afterward. It shifts the anchor of decision from "predicting the future" to "managing how you will look back on this moment."
Regret = the payoff of the best choice in hindsight − the payoff of what you actually chose. The minimax-regret criterion picks the option whose worst-case regret is smallest. It leans naturally toward robustness and reversibility: preserve optionality, avoid irreversible blunders. In game theory and online learning, "no-regret" learning (such as the CFR algorithm) lets an agent converge to equilibrium through repeated play — the very engine behind top poker AIs.
| Option ＼ State | Market takes off | Market stalls | Max regret |
|---|---|---|---|
| Bet on startup | 0 | 50 | 50 |
| Stay at big firm | 60 | 0 | 60 |
Bezos's "regret minimization framework": deciding whether to quit and start a company, he didn't compute the odds of success but imagined his 80-year-old self looking back — the regret of "never trying" far outweighed "trying and failing." The conclusion was instantly clear. Psychology backs this: over the long run, people almost always regret what they didn't do rather than what they did, because the regret of inaction is boundless and cannot be rationalized away.
Online learning's no-regret algorithms are the shared bedrock of modern game AI (solving poker via CFR) and online ad auctions; in investing it maps to "avoiding permanent loss of capital before maximizing returns" — Buffett's first rule. In engineering and complex systems it reincarnates as a robust design philosophy of "preserve redundancy and reversibility," kin to antifragility and the "reversible door vs. one-way door" taxonomy of decisions.
Facing high-uncertainty technical bets (which AI framework, which lane to enter), don't pretend you can estimate the odds precisely. Ask two questions instead: which choice is reversible even if it's wrong? And which mistake would knock me out permanently? Rule out the irreversible blunders, then experiment boldly in the reversible space — this is precisely the "super-individual's" meta-strategy for hedging uncertainty.
▸ Prompt: The thing you're hesitating over — is it a "reversible door" or a "one-way door"? Imagining yourself looking back in ten years, which choice's regret would be harder to swallow?