RESETRESEARCH STUDIO
Methodology / Model review

Make room for a model to fail.

A model review is more useful when it actively seeks the conditions under which the model stops helping.

Define a fair comparison

State the decision the model is meant to improve and choose a relevant comparison before reviewing its result. Keep the test horizon, the available information and the evaluation rule explicit.

Expose the assumptions

Write down the treatment of transaction costs, liquidity, missing observations and rebalancing. Vary assumptions separately so the reviewer can see which one is doing the work.

Preserve a separate evaluation

Distinguish the observations used to construct a rule from those reserved to evaluate it. Document changes to the rule, including changes made after an unfavorable result.

A minimal review checklist

Ask: What information was available? Which costs were included? How sensitive is the result to parameters? What changes across conditions? What evidence contradicts the proposed mechanism? This checklist is a research aid, not evidence of an investable strategy.

RESET methodology note. This document describes a proposed research practice; it reports no investment results or proprietary dataset.
Return to the data directory ↗