Commit eb3aac8
docs(robustness): explicit alpha threshold and PSR caveat
Task 4 of the DECISION_GRADE escalation. Pins every statistical
threshold to a canonical location and documents the PSR autocorrelation
limitation so no reader confuses PSR=1.0 with definitive significance.
## ROBUSTNESS_PROTOCOL.md § 3 — Statistical thresholds
Nine thresholds tabulated verbatim with their module-level source:
null_alpha = 0.05 kuramoto_null_suite.NULL_PASS_P_THRESHOLD
pbo_max = 0.50 kuramoto_cpcv_suite.PBO_PASS_THRESHOLD
loo_pbo_max = 0.50 kuramoto_cpcv_suite.LOO_PBO_PASS_THRESHOLD
psr_min = 0.95 kuramoto_cpcv_suite.PSR_PASS_THRESHOLD
jitter_floor_ratio = 0.80 kuramoto_jitter_suite default
sharpe_tolerance = 0.20 kuramoto_jitter_suite.DEFAULT_SHARPE_TOLERANCE
pbo_tautological_n = 3 kuramoto_cpcv_suite.PBO_TAUTOLOGICAL_CUTOFF
pbo_weak_n = 5 kuramoto_cpcv_suite.PBO_WEAK_CUTOFF
null_convergence_tol = 0.02 analysis_null_convergence.CONVERGENCE_TOLERANCE
The file is explicit that documentation mirrors the code constants,
never the other way round. Threshold drift between code and doc is a
bug in the doc.
## ROBUSTNESS_LIMITATIONS.md (new)
Five honest catalogue entries:
1. PSR has no autocorrelation adjustment.
Lopez de Prado Eq. 14.1 corrects skew + kurtosis, not serial
correlation. Regime-following strategies have inflated effective
sample sizes; PSR=1.0 on the frozen bundle should not be read as
definitive significance. HAC (Newey-West) is the forward fix.
2. Jitter evaluator is placeholder — forced abstain, not pass.
3. LOO-grid PBO has only 5 paths — wide CI on the 0.20 point estimate.
4. Null families are single-stream (no benchmark-matched test).
5. Contract covers frozen bundle only; no re-simulation.
Each entry is explicit that it is NOT a bug and NOT required for a
valid verdict — only things a reader must account for.
## ROBUSTNESS_RESULTS.md wiring
- CPCV row now reads 'PSR (daily, no HAC)' so the caveat is visible
at-a-glance in the main results table.
- Notes section cross-references ROBUSTNESS_PROTOCOL.md § 3 for
thresholds and ROBUSTNESS_LIMITATIONS.md § 1 for the PSR caveat.
## Integrity
- Code constants unchanged (per R6: do not change verdict by
threshold manipulation). Documentation mirrors existing code.
- 63/63 tests/research/robustness green.
- mypy --strict clean across touched files.
- 28/28 frozen artefacts intact.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>1 parent 9f14490 commit eb3aac8
4 files changed
Lines changed: 123 additions & 10 deletions
File tree
- results/cross_asset_kuramoto/robustness_v1
- scripts
Lines changed: 89 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
Lines changed: 21 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
66 | | - | |
67 | | - | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
68 | 81 | | |
69 | 82 | | |
70 | 83 | | |
| |||
Lines changed: 3 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
| 38 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
110 | 110 | | |
111 | 111 | | |
112 | 112 | | |
113 | | - | |
| 113 | + | |
114 | 114 | | |
115 | 115 | | |
116 | 116 | | |
| |||
189 | 189 | | |
190 | 190 | | |
191 | 191 | | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
192 | 201 | | |
193 | 202 | | |
194 | 203 | | |
| |||
0 commit comments