Asset-pricing replication failure estimates are definition-sensitive, not one settled rate

Abstract

Five source-diverse asset-pricing replication receipts report definition-specific failure estimates from 2.0% to 87.2%. The spread is the signal: the estimates move with the replication definition, hurdle rate, sample construction, and microcap or data-snooping adjustment, so the memo should be read as a map of method sensitivity rather than a pooled failure-rate estimate.

Research question

How much do factor-premia replication failure estimates vary when asset-pricing papers change the replication definition, hurdle, and sample restrictions?

Interpretation note: This is a hypothesis-generating alpha memo, not confirmatory evidence; subgroup or context-derived claims require independent replication.

Why this is surprising

The bounded signal is method-sensitive disagreement, not a settled failure rate. The receipts share a common frame: published cross-sectional equity return predictors and factor premia are re-tested under replication, robustness, or multiple-testing screens. They do not share an identical estimand.

The low-end receipt, Chen and Zimmermann, is explicitly definition-mismatched: it measures t-statistic survival among originally significant predictors. The high-end receipts use stricter or different failure definitions, such as single-test hurdle failure, independent-determinant survival, and false-rejection rates. The useful alpha is therefore not the midpoint; it is that asset-pricing replication claims can flip depending on what counts as failure.

Estimate map

fact_id	estimate	definition	hurdle / threshold	sample and restrictions
finance-replication-v3-001	65.0%	Share of 452 anomalies failing the single-test replication hurdle	Absolute t-statistic 1.96	Microcaps mitigated with NYSE breakpoints; value-weighted returns
finance-replication-v3-002	87.2%	Implied share of 94 characteristics not remaining reliable independent determinants	Joint Fama-MacBeth screen with data-snooping adjustment	U.S. monthly stock returns, 1980-2014; avoids overweighting microcaps
finance-replication-v3-003	45.3%	Expected false-rejection proportion under anomaly search without multiple-testing adjustment	Multiple-hypothesis thresholds calibrated from trading strategies	Over 2 million generated strategies plus publication-survivor strategy set
finance-replication-v3-004	44.4%	Complement of a 55.6% baseline U.S. factor replication rate	Significant OLS t-statistics for average raw factor returns	Longer U.S. factor sample and added factors versus the Hou-Xue-Zhang comparison
finance-replication-v3-005	2.0%	Complement of 98% t-stat survival among originally significant predictors	Long-short portfolio t-statistic above 1.96	Open-source replication against original-paper t-statistics for clearly significant predictors

Evidence shape

• population: published cross sectional equity return predictors and factor premia
• intervention: replication or multiple testing robustness screen
• comparator: original anomaly evidence at conventional thresholds
• outcome: method-specific predictor survival after replication screen
• metric: definition-specific replication failure estimate
• study_design: empirical asset pricing replication
• dataset: published stock return anomaly libraries
• estimation_method: asset pricing replication robustness screen
• identification_strategy: empirical asset pricing replication

Evidence receipts

• fact_id=finance-replication-v3-001 (A_core) - For factor premia returns, Hou, Xue, and Zhang report a definition-specific replication failure estimate of 65% for 452 anomalies under a single-test t-statistic hurdle after microcap mitigation and value-weighted returns.
• fact_id=finance-replication-v3-002 (A_core) - For factor premia returns, Green, Hand, and Zhang imply a definition-specific replication failure estimate of 87.2% because 12 of 94 characteristics remain reliable independent determinants under microcap and data-snooping adjustments.
• fact_id=finance-replication-v3-003 (A_core) - For factor premia returns, Chordia, Goyal, and Saretto estimate a definition-specific replication failure estimate of 45.3% as the false-rejection proportion for anomaly searches that omit multiple hypothesis testing adjustments.
• fact_id=finance-replication-v3-004 (A_core) - For factor premia returns, Jensen, Kelly, and Pedersen imply a definition-specific replication failure estimate of 44.4% from a 55.6% baseline replication rate for U.S. factors.
• fact_id=finance-replication-v3-005 (A_core) - For factor premia returns, Chen and Zimmermann imply a definition-specific replication failure estimate of 2.0% because 98% of clearly significant original predictors still have long-short portfolio t-statistics above 1.96.

What would weaken this

• A rerun that forces the same failure definition, threshold, sample period, and microcap rule across all five source families collapses the spread.
• Source verification shows the Chen-Zimmermann 2.0% estimate is not an appropriate complement to the reported 98% t-stat survival result.
• Additional source-diverse replication papers show that hurdle choice and sample construction do not materially change the reported failure estimate.