LTV:CAC and CAC Payback
The ratio of a customer's expected lifetime gross profit to the cost of acquiring them, with payback period as the time taken to recover that cost.
Breaks when LTV is a forecast, and extrapolating it from early cohorts is the most common error in the field. Note also that the 3:1 ratio and the twelve-month payback rule are conventions, not findings — they are routinely presented as research.
Brand Tracking
Repeated survey measurement of a brand's presence in memory — recall, consideration, asset attribution — reported as a time series.
Breaks when Question wording determines the answer; mix prompted and unprompted recall and the series becomes meaningless. Change methodology and the entire history is void.
Attribution Models
Rules that assign credit for a conversion across the touchpoints that preceded it.
Breaks when Observational, not causal. Benchmarked against experiments it systematically inflates cheap and search channels. It cannot substitute for an incrementality test, and it constantly is.
Net Promoter Score
A single-question metric scoring likelihood to recommend on a 0–10 scale and reporting the share of promoters minus the share of detractors.
Breaks when The growth-prediction claim has been repeatedly refuted. Keiningham et al. examined 21 firms and 15,500+ customer interviews across five industries and found no support for NPS as the best predictor of growth — correlations were inconsistent and mostly non-significant, and in two of the three US industries Reichheld showcased, ACSI outperformed it. Useful as an internal trend; not an external truth.
Goodhart's Law
The principle that an observed statistical regularity collapses once it is used as a target for control.
Breaks when It does not break. What breaks is the measurement system that forgot it.
Share of Search
A brand's share of category-related search volume, used as a fast and free leading indicator of market share.
Breaks when It is a proxy, and proxies drift from what they proxy. Search volume rises with newsworthiness as well as with demand, so a crisis and a successful campaign look the same. In categories where buying does not involve search, or where the brand name is also an ordinary word, the signal is mostly noise. And because it is cheap to move — brand-name bidding, PR spikes — it degrades the moment it becomes a target, which is the standard Goodhart path.
Test & Roll
An experiment design that sizes a test to maximise total profit across the test and the rollout, instead of sizing it to reach statistical significance.
Breaks when It optimises the profit of one decision, so it deliberately accepts a higher error rate than a scientific test would — it will sometimes roll out the worse arm, by design, because the expected cost of that is lower than the cost of the larger test. That makes it the wrong instrument when the result has to generalise, feed a model, be reused across markets, or be defended to someone outside the team. It also needs a prior on the effect size, and a badly chosen prior moves the recommended sample size a long way.
Split-Run Testing
Running two versions of the same advertisement to randomly divided halves of one audience and keeping the version that produces more response.
Breaks when Most tests are underpowered for the difference they are looking for, so a large share of declared winners are noise that will not repeat. The design also only compares the variants you thought of: it optimises inside a set and never reports that the set was wrong, which is how a headline test returns a clean 12% lift on a page selling the wrong thing. And it measures response, not profit — Hopkins' keyed coupon counted replies and the modern equivalent counts clicks.
Customer-Based Brand Equity
Brand equity defined as the difference in how a customer responds to marketing for a branded product versus the identical product unbranded, located in brand knowledge held in memory.
Breaks when It never specifies the weights. Nothing in the framework says how much awareness is worth against how much favourability, so every tracker built on it makes those trade-offs implicitly and reports the result as a score. Defined as a differential response, it is also close to unfalsifiable in practice: any gap between a brand and an unbranded equivalent counts as evidence for it, and the framework names no case in which the differential should be zero.