Scientific Method

Hypothesis testing is the central loop of the scientific method: state a testable hypothesis, design an experiment, gather data, and judge whether the evidence supports or refutes the claim. The value of the loop is not "proving things right" — it is approaching truth through relentless attempts to falsify. The cognitive payoff: it forces a hazy intuition into a precise statement that reality can defeat. "I think this product has a market" is not a hypothesis. "Conversion rate exceeds 3% within one week on channel A" is.

The non-trivial insight: the most common error is not "the hypothesis failed" but "we tested the wrong hypothesis." People spend enormous effort validating "does this solution work?" without testing the more fundamental premise — "does the problem actually exist?" In a startup, founders carefully validate feature feasibility but skip the deeper assumption "do users really have this pain?" The correct order in hypothesis testing is bottom-up: test the most fundamental, cheapest-to-overturn assumptions first.

How to apply it: (1) decompose any decision into 3-5 layered hypotheses ranked from foundational to derived, and start at the bottom; (2) define a "killer metric" for each hypothesis up front — a number that, below the threshold, you abandon the hypothesis, and above which you commit; this eliminates after-the-fact rationalization; (3) cultivate the habit of pre-registration — write down your predicted outcome before the experiment so you cannot "reinterpret" the data after seeing it (the HARKing trap).

Hypothesis-Testing is the engine of scientific progress: form a specific, falsifiable prediction, design an experiment, collect data, and update your beliefs accordingly. In business and investment contexts, it translates to disciplined small-scale testing before committing major resources — replacing "I think this will work" with "here's how we'll know if it works."

Karl Popper argued that to be scientific a theory must be falsifiable — there must exist some possible observation that could refute it. A theory that no evidence can refute is not science; it is faith. The key distinction: a scientific claim says "I have not been refuted yet" (provisional), while a pseudoscientific claim says "I can never be refuted" (eternally true). This is why "the market may go up or down tomorrow" is worthless — it can be falsified by no outcome, so it contains no predictive information.

The non-trivial insight: falsifiability is not only a yardstick for scientific theories but also a sharp blade for testing the quality of personal beliefs. The more non-falsifiable a person's belief system, the more fragile their cognitive system — because it has refused all feedback from reality. The most dangerous narratives in investing share the non-falsifiable feature: "it will rise in the long run" — no matter how long it drops, you can always say "not long enough." Falsifiable thinking demands you answer one question before forming a belief: what evidence would make me give up this belief? If you cannot answer, the belief has slipped out of the rational domain.

How to apply it: (1) for every important judgment you hold, write a "falsification checklist" — concrete, observable, time-bounded ("if X does not occur within 6 months, I abandon this judgment"); (2) watch for "non-falsifiability drift" — when you find yourself constantly revising the conditions to fit new data, your belief may have slid from science into faith; (3) when evaluating others' predictions or theories, the first question is always "under what circumstances would this prediction be proven wrong?" — if they cannot answer, the prediction carries no information.

Falsifiability, proposed by Karl Popper, is the criterion that separates science from pseudoscience: a genuine theory must make predictions that could, in principle, be proven wrong. In investing, strategy, and everyday reasoning, falsifiability forces you to define in advance what evidence would change your mind — preventing the intellectual trap of theories that can explain everything and therefore predict nothing.

Controlling variables is the foundational move of experimental science: when you want to know A's effect on B, every other factor must be held constant — otherwise you cannot distinguish whether the change came from A or from something else. The mathematical basis is the partial derivative — to measure A's effect on B, you fix all other variables and take the derivative. This principle is routinely ignored in everyday decisions because real-world variables move at the same time, and the brain naturally treats "happens together" as "happens because."

The non-trivial insight: the biggest difficulty in controlling variables in practice is not "knowing what to control" but "not knowing what you didn't control" — the unknown confounders. That is why the randomized controlled trial (RCT) is the gold standard of causal inference: randomization does not require you to know every confounder; probability guarantees that unknown variables are evenly distributed across treatment and control groups. When an RCT is not possible (investing, education), the fallbacks are natural experiments and difference-in-differences (DiD) — finding quasi-random historical events as the basis for causal inference.

How to apply it: (1) when you change something in life or work, change only one variable at a time and observe for 2-4 weeks before introducing the next; (2) when multiple variables change together (the real-world default), at least list every variable you can identify and admit "I cannot tell which one is the cause"; (3) when evaluating someone else's causal claim, ask "what's the control group?" — a conclusion with no control group is an anecdote, not evidence.

Controlled Variables is the experimental principle of isolating cause from effect: to understand the impact of variable A on outcome B, everything else must be held constant. Without this discipline, we mistake correlation for causation and attribute effects to the wrong causes. In product testing, investing, and parenting — wherever you're trying to learn what actually works — controlling variables is the difference between knowledge and noise.

Correlation ≠ causation is the most basic statistical principle and the most frequently violated: two variables moving together in the data (correlation) does not mean one causes the other. Correlations can arise from three structures: (1) A really does cause B; (2) B in fact causes A (reverse causation); (3) an invisible third variable C drives both A and B (confounder). The brain is wired to read correlation as causation, because rapidly inferring cause and effect ("the bush moved → predator may be near") gave a huge survival edge to our ancestors, even with occasional false alarms. But in a data-driven modern world this instinct becomes a systematic cognitive trap.

The non-trivial insight: big data has not reduced this trap — it has grown it exponentially. When the dataset is large enough, you can find a statistically significant correlation between almost any two variables (the multiple-comparisons problem). More subtle is the "inverted causal chain" — we see "people who meditate make better decisions" and roll out a meditation program, without testing whether a third variable like "high self-discipline" drives both meditation habits and good decisions. Judea Pearl's ladder of causation makes the point: going from "observation" to "intervention" to "counterfactual" requires fundamentally different reasoning tools at each level; pure data mining is stuck on the first rung.

How to apply it: (1) for any "A and B are correlated" finding, immediately sketch all three causal diagrams (A→B, B→A, C→A+B) and ask which is most plausible; (2) ask "if I intervene on A (rather than passively observe), does B still change?" — that is the gold question for separating correlation from causation; (3) in life and investment decisions, prioritize RCT-based conclusions and stay highly skeptical of causal claims from purely observational studies.

Correlation ≠ Causation is perhaps the most important statistical principle for clear thinking. Just because two variables move together doesn't mean one causes the other — there may be a third confounding variable, or the relationship may be pure coincidence. In the age of big data, distinguishing genuine causal signals from spurious correlations is one of the highest-value cognitive skills.