Selection bias, named out loud
Hand-pick a watchlist and your exact rules are re-run on the S&P 500 as it stood at each rebalance, not on today's winners. Hindsight gets reported, not rewarded.
Educational research tool · Nothing here is advice
Every retail backtester would have shown me that first number and none of the corrections. So I built the one that argues back — it runs a full honesty audit on every result, and there is no way to turn it off.
136% a year over 40 months, on 44 tickers picked with hindsight. Max drawdown 45%, against SPY's 8%.
Different rules: momentum plus low volatility, top 25 names, 2016 to 2026, picking only from that month's index members. SPY returned +362% over the same years.
A dip-buyer on the S&P 500 as it stood each day had a Sharpe of 0.81 before costs, against SPY's 0.83. At five basis points a day it was up 22% (Sharpe 0.19), against SPY's 344%.
Watch it argue back
The strategy “Each month, buy the 5 strongest stocks in my 44-ticker watchlist by 6-month return. Equal weight.” is read back as a spec — top 5, 126-day return, monthly rebalance, equal weight, vs spy — and audited: five checks pass, and one is fatal: the 44-ticker universe is hand-picked. Same rules, the S&P 500 as it stood each month: +363.5% vs your +1646.6%. Hand-picking alone: 3.77× in final wealth. Verdict: red — Result not trustworthy. Selection bias detected — Re-run this spec on the full S&P 500. Strategy versus SPY buy-and-hold: Total return +1646.6% vs +41.9%, CAGR +348.9% vs +20.2%, Sharpe 2.03 vs 0.92, Max drawdown −61.2% vs −18.8%.
04 · Verdict
Killed — red verdict
Result not trustworthy.
Selection bias detected — Re-run this spec on the full S&P 500
Benchmark shown on every result. No way to turn the audit off.
A replay of report rep_9534f1bae0, the same report as the export further down this page. It is a separate test on the same 44 tickers, ranked by 6-month return from mid-2024 to May 2026, so its numbers differ from the backtest above. Its selection-bias check compares against the S&P 500 as it stood each month. Every number is from the real report.
Every run ends here
Plain English goes in; a precise, editable spec comes back for you to approve. Nothing runs until you confirm exactly what will be tested — and what comes out is this.

Six ways it tries to kill your idea
Hand-pick a watchlist and your exact rules are re-run on the S&P 500 as it stood at each rebalance, not on today's winners. Hindsight gets reported, not rewarded.
Scaled by your strategy's measured turnover. The report states the cost level at which the edge dies — often lower than you'd guess.
Thirty per cent of the window is held out. A large in-sample advantage that vanishes there is flagged as the overfit signature it is.
Signals at day t, fills at the close of t+1, enforced inside the engine rather than by convention. Altering future prices cannot change past weights.
If your window contains no sustained bear market, the report says so. A strategy tested only through a bull run has not been tested.
Every run freezes its price panel under a content hash and replays it exactly. Your journal cannot quietly drift underneath you.
Terms
The audit is never paywalled. Charging you to discover that your strategy is broken would invert the point of the product — so the honesty is free, and the paid tier is better data, which is where the real cost actually sits.
The complete honesty audit, unabridged.
Survivorship-free data and forward tracking. Priced once the data licence is signed, not before.
No pricing is committed, and nothing on this page is for sale. The paid tier ships only after a licensed data source and a legal review are in place.
Validation, in the open
I'm testing whether this is worth building for anyone but me. Leave an email and, if you like, the strategy you'd test first — that answer shapes what gets built next.