Commit 2d9bf67
fix(robustness): use raw daily net returns for null suite
Task 1 of the PR #356 DECISION_GRADE escalation. Switches the null suite
off the cumret-derived pct_change proxy and onto mathematically exact
daily log-returns, and fixes a degenerate null family that the switch
exposed.
## Input-data change (Task 1 literal mandate)
The frozen demo bundle ships strategy_cumret (cumulative wealth) but no
raw net_ret column. Contract now derives daily returns as:
r_t = log(cumret_t) − log(cumret_{t-1})
This is mathematically exact (not an approximation) for the hypothetical
raw net_ret series that produced the wealth trajectory. Log returns
are the honest time-additive representation and preserve independence
under permutation/resampling, which is the contract assumed by the
bootstrap null families. Derivation documented in
results/cross_asset_kuramoto/robustness_v1/ROBUSTNESS_PROTOCOL.md.
## Null-family fix (bug exposed by Task 1, not introduced by it)
The switch to log returns surfaced a structural bug: the old
'iid_permutation' family was *degenerate* for a Sharpe statistic on a
single return stream, because Sharpe is order-invariant on a given
vector (permutation preserves mean and std exactly up to float noise).
The p-value was trivially ≈ 1.0 by construction; the previous p=0.088
on pct_change was a floating-point artefact, not a real signal.
Fix: replaced with 'iid_bootstrap' — sample with replacement from the
empirical marginal distribution. This changes the realised mean and std
of each draw and is the proper iid null for a Sharpe statistic on a
single return stream. Literal type, family names, docstrings, and tests
updated; null_audit logic otherwise untouched.
## Verdict evolution (numbers on disk)
Observed Sharpe (log returns): 0.4832 (was 0.5775 on pct_change)
iid_bootstrap p-value: 0.5045 (was 0.0878 on proxy / degenerate permutation)
stationary_bootstrap p-value: 0.5235 (was 0.5170)
Verdict label: FAIL → FAIL (unchanged). The honest real-returns null
gives p ≈ 0.50, consistent with SEPARATION_FINDING.md: the *realised*
daily return stream is statistically indistinguishable from bootstrap
resamples, because most alpha lives in a narrow HIGH_SYNC regime. This
is NOT a proxy artefact — marked FAIL_ON_DAILY_RETURNS in verdict.json.
## Evidence artefacts
- verdict.json now carries input_source: 'daily_log_returns' and
label_qualifier: 'FAIL_ON_DAILY_RETURNS'.
- Renamed ROBUSTNESS_v1.md → ROBUSTNESS_RESULTS.md per task convention.
- ROBUSTNESS_PROTOCOL.md introduced to pin the derivation.
- cpcv_summary.json, null_summary.json, jitter_summary.json regenerated.
## Guarantees
- 28/28 frozen SOURCE_HASHES artefacts unchanged.
- Shadow timer still active.
- 58/58 tests/research/robustness/ green.
- mypy --strict clean across 21 source files.
- Signal code untouched; framework-only change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 5df09ed commit 2d9bf67
11 files changed
Lines changed: 2179 additions & 1033 deletions
File tree
- research/robustness/protocols
- results/cross_asset_kuramoto/robustness_v1
- scripts
- tests/research/robustness
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
187 | 188 | | |
188 | 189 | | |
189 | 190 | | |
190 | | - | |
191 | | - | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
192 | 206 | | |
193 | | - | |
194 | 207 | | |
195 | 208 | | |
196 | 209 | | |
197 | | - | |
198 | | - | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
199 | 217 | | |
200 | | - | |
201 | 218 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
13 | | - | |
14 | | - | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
15 | 21 | | |
16 | 22 | | |
17 | 23 | | |
| |||
31 | 37 | | |
32 | 38 | | |
33 | 39 | | |
34 | | - | |
| 40 | + | |
35 | 41 | | |
36 | 42 | | |
37 | 43 | | |
| |||
117 | 123 | | |
118 | 124 | | |
119 | 125 | | |
120 | | - | |
| 126 | + | |
| 127 | + | |
121 | 128 | | |
122 | 129 | | |
123 | 130 | | |
124 | | - | |
125 | | - | |
| 131 | + | |
| 132 | + | |
126 | 133 | | |
127 | 134 | | |
128 | 135 | | |
129 | 136 | | |
130 | | - | |
| 137 | + | |
131 | 138 | | |
132 | 139 | | |
133 | 140 | | |
| |||
Lines changed: 82 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
Lines changed: 3 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | | - | |
14 | | - | |
| 13 | + | |
| 14 | + | |
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| |||
Lines changed: 37 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
8 | | - | |
| 8 | + | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| |||
46 | 46 | | |
47 | 47 | | |
48 | 48 | | |
49 | | - | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
50 | 82 | | |
51 | | - | |
52 | | - | |
53 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
54 | 86 | | |
55 | 87 | | |
56 | 88 | | |
0 commit comments