You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Expanded the brief "Reading test results" note into a dedicated
subsection explaining:
- Rationale for a custom test framework at all: pytest installs
an assert-rewriting import hook and mcpyrate installs a
macro-expanding import hook, and Python only supports one
source-rewriting import hook at a time, so the two loaders
can't be chained. Macro expansion is non-negotiable for
code that uses macros; assert rewriting is therefore out.
- The Pass / Fail / Error / Warn distinction, with the
semantically load-bearing Fail-vs-Error line spelled out:
Fail = test ran but expectation didn't hold; Error = test
didn't run to completion because something escaped the
expression. "Error" is what you look at first in CI.
- The `the[]` value-capture helper: its use, the implicit-LHS
default for comparison expressions, the trivial-literal
skip optimization, and which constructs don't support it.
- Etymology of the `the[]` name (English-reading-order nod,
Common-Lisp-`THE` pun) and the `\bthe\[` grep idiom to
cut through the word-boundary noise.
No code changes. Makes the framework's public-API status
actionable: anyone reusing `unpythonic.test.fixtures` in their
own macro-enabled project now has the rationale and result
semantics in one place.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: CLAUDE.md
+27-1Lines changed: 27 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -61,7 +61,33 @@ Test suites discovered by `runtests.py`:
61
61
62
62
Each test module exports a `runtests()` function. Tests are grouped with `testset()` context managers.
63
63
64
-
**Reading test results**: The framework reports Pass/Fail/Error/Total per testset. "Error" means an unexpected exception inside a `test[]` expression — this includes intentional skip-with-message patterns (e.g. "SymPy not installed"), so a few errors from optional-dependency tests are normal. Look at the actual error messages, not just the count. Nested testsets show hierarchy with indentation and asterisk depth (`**`, `****`, `******`, etc.).
64
+
**Reading test results**: The framework reports Pass/Fail/Error/Total (plus optional `+ N Warn`) per testset. Nested testsets show hierarchy with indentation and asterisk depth (`**`, `****`, `******`, etc.). The distinction between Fail and Error is semantically load-bearing — see the next subsection.
65
+
66
+
### The `unpythonic.test.fixtures` framework
67
+
68
+
Part of unpythonic's **public API** (`unpythonic.test.fixtures`, `unpythonic.test.runner`). Reusable by any project that writes macro-enabled Python tests. Rationale for not using pytest:
69
+
70
+
- pytest installs an import hook that rewrites `assert` statements (to give you the informative "assert x == 42 where x was 41" diagnostics you're used to).
71
+
- mcpyrate installs its own import hook to macro-expand source before compilation.
72
+
- Python only supports one source-rewriting import hook at a time; the two loaders can't be chained. So if you want both "nice assert messages" *and* "macro expansion", you have to pick one — and macro expansion is non-negotiable for code that uses macros.
73
+
74
+
`unpythonic.test.fixtures` is the answer: instead of overriding the `assert` keyword, it provides `test[expr]`, `test_raises[cls, expr]`, `test_signals[cls, expr]`, and `warn[msg]`**macros** that construct test assertions at the AST level, and route results through `mcpyrate`'s condition system. The result categories:
75
+
76
+
-**Pass**: the `test[...]` expression evaluated to a truthy value (or `test_raises[...]` saw exactly the expected exception, etc.). The test ran to completion and met its expectation.
77
+
-**Fail**: the test ran to completion, but the expectation was not met — `test[x == 42]` saw `x == 41`, or `test_raises[TypeError, ...]` saw the expression return normally. This is the "your code is wrong" category.
78
+
-**Error**: the test did **not** run to completion. An unhandled exception (or unhandled `error`/`cerror` condition) escaped the `test[...]` expression itself. This is the "the test infrastructure or the code *under* test crashed in a way the test didn't expect" category — semantically distinct from Fail, because the test never got to judge the expectation. An Error in CI means something is broken in a way that needs investigation, not just "the assertion didn't hold."
79
+
-**Warn**: advisory, emitted via `warn[msg]` (or by the runner itself for version-gated skips like "this test requires Python 3.14+, skipping on 3.13"). Does **not** count toward Pass/Fail/Error totals and does **not** fail the testset. Used for temporarily disabled tests, optional-dependency skips, and similar soft signals.
80
+
81
+
**Capturing values with `the[]`**: when a `test[]` fails, you want to see *what the interesting subexpression actually evaluated to*, not just "the assertion was falsy." The `the[...]` helper macro marks a subexpression for capture; at run time, when the test fires, the framework formats a failure message with the source text and captured value of each `the[]`. The name is chosen to mostly preserve English reading order at the use site (`test[the[x] == 42]` reads roughly as "test that the `x` equals 42"), and is also a nod to Common Lisp's `THE` special form — though CL's `THE` is a *type-declaration* construct, so it's a name pun, not a semantic port. Heads-up for grepping: `the` is a word-boundary nightmare; anchor searches with `\bthe\[`. Usage:
82
+
83
+
-`test[the[x] == 42]` → on failure, reports `x` and the value it had.
84
+
-`test[f(the[a]) == g(the[b])]` → reports both `a` and `b`, in evaluation order. A `test[]` can contain any number of `the[]`, including nested (`the[outer(the[inner])]`).
85
+
-**Default**: if the top-level expression of `test[]` is a comparison and no explicit `the[]` is present, the leftmost term is **implicitly** wrapped — so `test[x == 42]` already reports `x` without you having to write `the[x]`. This is the common case.
86
+
- Use explicit `the[]` when you want to capture something *other* than the LHS of the top-level comparison — e.g. a subexpression inside a function call, a term in a non-comparison assertion, or multiple values at once.
87
+
- The helper is smart enough to skip trivial captures (literal values), so `test[4 in the[(1, 2, 3)]]` won't clutter the output with `(1, 2, 3) = (1, 2, 3)`.
88
+
-**Not supported** inside `test_raises`, `test_signals`, `fail`, `error`, or `warn` — only in `test[...]` and `with test:` blocks.
89
+
90
+
**Debugging cheat sheet**: a small number of **Warn**s on CI is expected (optional dependencies, version gates). **Fail** means a real expectation mismatch — read the captured values from `the[]` in the message. **Error** is the one you should *always* look at first: it means control flow in the test went somewhere unexpected, and the count alone won't tell you where. The log above the summary line has the actual traceback.
0 commit comments