Skip to content

Commit 4f0f4a5

Browse files
Technologicatclaude
andcommitted
CLAUDE.md: document unpythonic.test.fixtures framework semantics
Expanded the brief "Reading test results" note into a dedicated subsection explaining: - Rationale for a custom test framework at all: pytest installs an assert-rewriting import hook and mcpyrate installs a macro-expanding import hook, and Python only supports one source-rewriting import hook at a time, so the two loaders can't be chained. Macro expansion is non-negotiable for code that uses macros; assert rewriting is therefore out. - The Pass / Fail / Error / Warn distinction, with the semantically load-bearing Fail-vs-Error line spelled out: Fail = test ran but expectation didn't hold; Error = test didn't run to completion because something escaped the expression. "Error" is what you look at first in CI. - The `the[]` value-capture helper: its use, the implicit-LHS default for comparison expressions, the trivial-literal skip optimization, and which constructs don't support it. - Etymology of the `the[]` name (English-reading-order nod, Common-Lisp-`THE` pun) and the `\bthe\[` grep idiom to cut through the word-boundary noise. No code changes. Makes the framework's public-API status actionable: anyone reusing `unpythonic.test.fixtures` in their own macro-enabled project now has the rationale and result semantics in one place. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 33c1590 commit 4f0f4a5

1 file changed

Lines changed: 27 additions & 1 deletion

File tree

CLAUDE.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,33 @@ Test suites discovered by `runtests.py`:
6161

6262
Each test module exports a `runtests()` function. Tests are grouped with `testset()` context managers.
6363

64-
**Reading test results**: The framework reports Pass/Fail/Error/Total per testset. "Error" means an unexpected exception inside a `test[]` expression — this includes intentional skip-with-message patterns (e.g. "SymPy not installed"), so a few errors from optional-dependency tests are normal. Look at the actual error messages, not just the count. Nested testsets show hierarchy with indentation and asterisk depth (`**`, `****`, `******`, etc.).
64+
**Reading test results**: The framework reports Pass/Fail/Error/Total (plus optional `+ N Warn`) per testset. Nested testsets show hierarchy with indentation and asterisk depth (`**`, `****`, `******`, etc.). The distinction between Fail and Error is semantically load-bearing — see the next subsection.
65+
66+
### The `unpythonic.test.fixtures` framework
67+
68+
Part of unpythonic's **public API** (`unpythonic.test.fixtures`, `unpythonic.test.runner`). Reusable by any project that writes macro-enabled Python tests. Rationale for not using pytest:
69+
70+
- pytest installs an import hook that rewrites `assert` statements (to give you the informative "assert x == 42 where x was 41" diagnostics you're used to).
71+
- mcpyrate installs its own import hook to macro-expand source before compilation.
72+
- Python only supports one source-rewriting import hook at a time; the two loaders can't be chained. So if you want both "nice assert messages" *and* "macro expansion", you have to pick one — and macro expansion is non-negotiable for code that uses macros.
73+
74+
`unpythonic.test.fixtures` is the answer: instead of overriding the `assert` keyword, it provides `test[expr]`, `test_raises[cls, expr]`, `test_signals[cls, expr]`, and `warn[msg]` **macros** that construct test assertions at the AST level, and route results through `mcpyrate`'s condition system. The result categories:
75+
76+
- **Pass**: the `test[...]` expression evaluated to a truthy value (or `test_raises[...]` saw exactly the expected exception, etc.). The test ran to completion and met its expectation.
77+
- **Fail**: the test ran to completion, but the expectation was not met — `test[x == 42]` saw `x == 41`, or `test_raises[TypeError, ...]` saw the expression return normally. This is the "your code is wrong" category.
78+
- **Error**: the test did **not** run to completion. An unhandled exception (or unhandled `error`/`cerror` condition) escaped the `test[...]` expression itself. This is the "the test infrastructure or the code *under* test crashed in a way the test didn't expect" category — semantically distinct from Fail, because the test never got to judge the expectation. An Error in CI means something is broken in a way that needs investigation, not just "the assertion didn't hold."
79+
- **Warn**: advisory, emitted via `warn[msg]` (or by the runner itself for version-gated skips like "this test requires Python 3.14+, skipping on 3.13"). Does **not** count toward Pass/Fail/Error totals and does **not** fail the testset. Used for temporarily disabled tests, optional-dependency skips, and similar soft signals.
80+
81+
**Capturing values with `the[]`**: when a `test[]` fails, you want to see *what the interesting subexpression actually evaluated to*, not just "the assertion was falsy." The `the[...]` helper macro marks a subexpression for capture; at run time, when the test fires, the framework formats a failure message with the source text and captured value of each `the[]`. The name is chosen to mostly preserve English reading order at the use site (`test[the[x] == 42]` reads roughly as "test that the `x` equals 42"), and is also a nod to Common Lisp's `THE` special form — though CL's `THE` is a *type-declaration* construct, so it's a name pun, not a semantic port. Heads-up for grepping: `the` is a word-boundary nightmare; anchor searches with `\bthe\[`. Usage:
82+
83+
- `test[the[x] == 42]` → on failure, reports `x` and the value it had.
84+
- `test[f(the[a]) == g(the[b])]` → reports both `a` and `b`, in evaluation order. A `test[]` can contain any number of `the[]`, including nested (`the[outer(the[inner])]`).
85+
- **Default**: if the top-level expression of `test[]` is a comparison and no explicit `the[]` is present, the leftmost term is **implicitly** wrapped — so `test[x == 42]` already reports `x` without you having to write `the[x]`. This is the common case.
86+
- Use explicit `the[]` when you want to capture something *other* than the LHS of the top-level comparison — e.g. a subexpression inside a function call, a term in a non-comparison assertion, or multiple values at once.
87+
- The helper is smart enough to skip trivial captures (literal values), so `test[4 in the[(1, 2, 3)]]` won't clutter the output with `(1, 2, 3) = (1, 2, 3)`.
88+
- **Not supported** inside `test_raises`, `test_signals`, `fail`, `error`, or `warn` — only in `test[...]` and `with test:` blocks.
89+
90+
**Debugging cheat sheet**: a small number of **Warn**s on CI is expected (optional dependencies, version gates). **Fail** means a real expectation mismatch — read the captured values from `the[]` in the message. **Error** is the one you should *always* look at first: it means control flow in the test went somewhere unexpected, and the count alone won't tell you where. The log above the summary line has the actual traceback.
6591

6692
## Linting
6793

0 commit comments

Comments
 (0)