Skip to content

BIP-0327: avoid using sys.path[0] to find current working directory#2158

Open
omipheo wants to merge 2 commits into
bitcoin:masterfrom
omipheo:bip-0327-avoid-sys-path-0
Open

BIP-0327: avoid using sys.path[0] to find current working directory#2158
omipheo wants to merge 2 commits into
bitcoin:masterfrom
omipheo:bip-0327-avoid-sys-path-0

Conversation

@omipheo
Copy link
Copy Markdown
Contributor

@omipheo omipheo commented May 14, 2026

Summary

bip-0327/reference.py and bip-0327/gen_vectors_helper.py both locate the test-vector JSON files via os.path.join(sys.path[0], 'vectors', '<file>.json'). As Sebastian Falbesoner noted in 4e18ee6 ("BIP-374: avoid using sys.path[0] to find current working directory"), this pattern is incompatible with any future sys.path.insert(0, ...) extension — for example, the sys.path.insert(0, str(Path(__file__).parent / "secp256k1lab/src")) shim that bip-0352 and bip-0374 use to locate their vendored secp256k1lab copies. Should bip-0327 follow the same vendoring pattern, every test-vector load would silently miss its file.

This PR switches both files to the Path(__file__).parent / 'vectors' / '...' pattern so the path is always resolved relative to the script itself, matching the BIP-374 precedent. import os and import sys are removed from reference.py (no remaining uses); gen_vectors_helper.py gets an explicit from pathlib import Path since it can no longer inherit those names through from reference import *.

Diff

Old call (×10 across 2 files) New call
os.path.join(sys.path[0], 'vectors', 'X.json') Path(__file__).parent / 'vectors' / 'X.json'

reference.py: 8 call sites (lines 499, 508, 532, 559, 577, 660, 717, 764). gen_vectors_helper.py: 2 call sites (lines 21, 49).
Imports adjusted to match.

Verification

$ python3 bip-0327/reference.py
$ echo $?
0

$ python3 bip-0327/gen_vectors_helper.py > /dev/null
$ echo $?
0

Both scripts continue to exit 0 with all test vectors loading.

(Note: bash bip-0327/tests.sh exits non-zero on master too — current local mypy versions flag four pre-existing bytearray/bytes mismatches at reference.py:354,355,671,672 that are unrelated to this PR. CI is presumably pinned to an older mypy; the runtime tests themselves pass on both branches.)

Notes

  • Direct follow-up to 4e18ee6 ("BIP-374: avoid using sys.path[0]…").
    Same shape, same justification, applied to the next reference implementation in the same family.
  • No behavior change beyond path-resolution robustness.

`bip-0327/reference.py` and `bip-0327/gen_vectors_helper.py` both
locate the test-vector JSON files via
`os.path.join(sys.path[0], 'vectors', '<file>.json')`. As Sebastian
Falbesoner noted in 4e18ee6 ("BIP-374: avoid using sys.path[0] to
find current working directory"), this pattern is incompatible with
any future `sys.path.insert(0, ...)` extension — for example, the
`sys.path.insert(0, str(Path(__file__).parent / "secp256k1lab/src"))`
shim that bip-0352 and bip-0374 use to locate their vendored
`secp256k1lab` copies. Should bip-0327 follow the same vendoring
pattern, every test-vector load would silently miss its file.

Switch both files to the `Path(__file__).parent / 'vectors' / '...'`
pattern so the path is always resolved relative to the script
itself, matching the BIP-374 precedent. `import os` and `import sys`
are removed from `reference.py` (no remaining uses); `gen_vectors_helper.py`
gets an explicit `from pathlib import Path` since it can no longer
inherit those names through `from reference import *`.

Verified that `python3 reference.py` and `python3 gen_vectors_helper.py`
still exit 0 with all 8 (resp. 2) vector files loading correctly. No
behavior change beyond the path-resolution robustness.
Copy link
Copy Markdown
Member

@jonatack jonatack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK, it's a minor refactor but more idiomatic and robust and would serve as a better reference for developers.

@jonatack
Copy link
Copy Markdown
Member

jonatack commented May 14, 2026

@omipheo note that BIPs 89, 328 and 340 also still use the same pattern. Maybe update them all here in one go.

…rectory

Same fix as the previous commit on BIP-0327, applied to the remaining
reference implementations that still use the
`os.path.join(sys.path[0], ...)` pattern: bip-0089/reference.py
(8 sites), bip-0328/reference.py (1 site), and bip-0340/reference.py
(1 site). All resolve test-vector paths via `Path(__file__).parent /
...` so future `sys.path.insert(0, ...)` extensions for vendored deps
won't silently miss the data files.

`import os` and `import sys` are dropped from each file (no remaining
uses). bip-0340/test-vectors.py imports `sys` itself for `sys.stdout`
so its wildcard `from reference import *` is unaffected by the
import cleanup in bip-0340/reference.py.

Verified: `python3 reference.py` for each of bip-0089, bip-0328,
bip-0340 still exits 0 with all test vectors loading; bip-0340's
test-vectors.py importer also still exits 0.

Per @jonatack's review note on PR bitcoin#2158.
@omipheo
Copy link
Copy Markdown
Contributor Author

omipheo commented May 14, 2026

Done — pushed b39a896e28 extending the same fix to bip-0089/reference.py (8 sites), bip-0328/reference.py (1), and bip-0340/reference.py (1). Same pattern: os.path.join(sys.path[0], '...')Path(__file__).parent / '...', with import os / import sys removed where they had no remaining uses.

Verified each script still runs clean:

$ for d in bip-0089 bip-0327 bip-0328 bip-0340; do (cd $d && python3 reference.py >/dev/null && echo "$d ok"); done
bip-0089 ok
bip-0327 ok
bip-0328 ok
bip-0340 ok

$ (cd bip-0327 && python3 gen_vectors_helper.py >/dev/null) && echo ok
ok

$ (cd bip-0340 && python3 test-vectors.py >/dev/null) && echo "test-vectors.py ok (wildcard importer of bip-0340/reference.py is unaffected by the os/sys removal — it imports sys itself for sys.stdout)"
test-vectors.py ok (wildcard importer of bip-0340/reference.py is unaffected by the os/sys removal — it imports sys itself for sys.stdout)

Happy to squash both commits into one if you'd prefer a single atomic change before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants