Frame the question
Specify the hypothesis, eligible data and failure conditions before looking at the answer.
RESEARCH / METHODOLOGY
Discover. Evaluate. Operationalize.
DEO connects a research question to an evidence review and, only when separately warranted, an engineering decision. The stages remain distinct so a historical result cannot silently become an operational claim.
Challenge the idea
Use causal tests, chronological separation and realistic friction; record supportive and unsupportive outcomes.
Review readiness
Assess execution, risk and monitoring separately. No model moves forward merely because a historical test passed.
EVALUATION DETAILS
Six checks behind the workflow.
01 / QUESTION
Pre-register the question
Define the hypothesis, eligible data, evaluation horizon and decision criteria before inspecting the holdout.
02 / DATA
Preserve causality
Use only information available at the decision point. Audit timestamps, revisions, survivorship and look-ahead or label leakage.
03 / DEVELOPMENT
Limit overfitting
Keep model selection within development data. Track multiple trials and parameter sensitivity instead of selecting a single favorable run.
04 / VALIDATION
Test chronologically
Distinguish an independent candidate-family out-of-sample test from a reused-family holdout. A historical pass remains historical evidence.
05 / FRICTION
Model practical costs
Include commissions, spread, slippage and execution assumptions. Recheck conclusions under less favorable but plausible costs and broker risk constraints.
06 / REPORT
Preserve uncertainty
Report small samples, event concentration, multiple testing, regime dependence, uncertainty and failed tests. Keep the verdict distinct from archive status.
Evidence distinctions: Source or log screening is not a reproduced backtest. Historical development testing, independent candidate-family out-of-sample testing, reused-family holdouts, controlled forward demo/simulation and audited live realized trading are separate evidence levels. A secondary portfolio-check pass applies only to that predeclared portfolio scope; it does not validate a new strategy or establish economic superiority.