Add path-based asset backend proxy routing#668
Conversation
aram356
left a comment
There was a problem hiding this comment.
Summary
Path-based asset proxy routing closely follows the spec, with a clean separation from the publisher pipeline and a conservative forwarded-header set. Concerns are concentrated around (1) origin_url validation that lets path/query through silently, (2) missing test coverage for spec-stated invariants — most importantly that asset failures must not silently fall back to publisher.origin_url, and (3) one CLAUDE.md doc-comment compliance gap.
Blocking
🔧 wrench
- Missing doc comments on
ProxyAssetRoute(crates/trusted-server-core/src/settings.rs:336-339) origin_urlvalidation accepts URLs with path/query that are silently dropped at runtime (crates/trusted-server-core/src/settings.rs:740-766)- No test that asset-origin failures stop at 502 without falling back to
publisher.origin_url(crates/trusted-server-adapter-fastly/src/route_tests.rs, around theasset_routes_*tests) — spec §9 forbids the fallback; the impl is correct but unpinned by tests.
Non-blocking
🤔 thinking
Strict-Transport-Securityfrom asset origin forwarded unchanged (crates/trusted-server-core/src/proxy.rs:657-663) — same class of risk as upstreamSet-Cookie, which is correctly stripped. Either strip HSTS too or document explicitly.- Spec-listed test gaps: HEAD passthrough, non-GET/HEAD bypass, redirect pass-through, query-string-ignored-for-matching. All small additions given existing helpers.
- String-prefix matching can match non-segment boundaries (
prefix = "/static"matches/staticfile.js). Spec is explicit, but every example uses a trailing/. Worth a sentence in the toml comment and field doc.
♻️ refactor
- Idiomatic method-set check in
route_request(crates/trusted-server-adapter-fastly/src/main.rs:153-156) — replace thematch &method { &Method::GET | &Method::HEAD => ... }withmatches!(...).then(...).flatten(). - Test-stub duplication:
StaticResponseHttpClientcould move intoplatform::test_supportby extendingStubHttpClientwithpush_response_with_headers. - Validation split between
Proxy::normalizeandProxy::prepare_runtimeis inconsistent with the rest ofSettings, which uses#[validate]attributes. Either consolidate or comment the rationale.
🌱 seedling
- WASM heap pressure for large asset bodies (
crates/trusted-server-adapter-fastly/src/platform.rs:223-237→fastly_response_to_platformcallstake_body_bytes();crates/trusted-server-core/src/proxy.rs:657re-buffers viaset_body(Vec<u8>)). Same asPublisherResponse::PassThrough, so not a regression — but this PR is specifically targeting asset traffic, where bodies are routinely larger than HTML. Track as a streaming pass-through follow-up mirroringPublisherResponse::Stream. (Anchored in the body sinceplatform.rsis unchanged in this PR.)
⛏ nitpick
- Validation error wording (
crates/trusted-server-core/src/settings.rs:716,validate_no_trailing_slash): the spec says "must not include a trailing slash"; the error message says "origin_url must not end with '/'". Aligning the wording reads slightly better in operator-facing errors. The function itself is unchanged in this PR but is reached via the newvalidate_proxy_origin_url. (Anchored in the body since the line is outside the diff.) - Redundant
to_ascii_lowercase()ontarget_url.scheme()— already lowercase from URL parsing. See inline comment. Set-Cookiestrip test only verifies a single header is removed; should test multiple. See inline comment.
CI Status
- fmt: PASS
- clippy: PASS
- rust tests: PASS (858 tests, 0 failures locally; CI green)
- vitest: PASS
- format-typescript: PASS
- format-docs: PASS
- browser & integration tests: PASS
- CodeQL / Analyze: PASS
|
Tested proxying, it does work, however proxying to a s3 bucket that requires authentication isn't implemented. |
aram356
left a comment
There was a problem hiding this comment.
Summary
Asset-proxy routing closely follows the spec, every prior blocking comment has been addressed with a paired test, and CI is green. Two blockers remain: (1) path_pattern regex is recompiled on every request despite a working OnceLock pattern ten lines up in the same file, and (2) the spec checked in by this PR explicitly lists path rewrite as out of scope but the PR adds path rewrite. Two follow-up question/scope items, plus a handful of non-blockers around header handling and validation tightness.
Blocking
🔧 wrench
path_patternregex recompiled on every request (crates/trusted-server-core/src/settings.rs:435-447) —Handlerten lines up caches viaOnceLock.prepare_runtimealready compiles for fail-fast and throws the result away. Hot-path WASM perf hole for the Cloudinary-style example shipped intrusted-server.toml.- Spec contradicts implementation in the same PR (
docs/superpowers/specs/2026-04-28-multi-backend-asset-proxy-design.md:55-58) — spec says path rewrite and regex matching are out of scope, but the PR adds both. Update the spec or split the rewrite into a follow-up.
❓ question
- Why is the publisher Host-override change in this PR? (
crates/trusted-server-core/src/settings.rs:18-20) — separate publisher-pipeline feature stacked on top of asset-routing. Independent surface, independent rollback risk.
Non-blocking
🤔 thinking
validate_host_header_valueis too permissive (crates/trusted-server-core/src/settings.rs:898-907) — passes spaces, slashes, query strings. Used both as a literalHost:header and to format a URL.Clear-Site-Datanot stripped from asset responses (crates/trusted-server-core/src/proxy.rs:660-661) — same threat class as HSTS / Set-Cookie which are correctly stripped.X-Forwarded-Forlisted but always empty (crates/trusted-server-core/src/proxy.rs:28) —sanitize_forwarded_headersstrips XFF at the edge before routing. Asset CDNs see Trusted Server's IP only.
♻️ refactor
- Dead-code error path in
target_path_for(crates/trusted-server-core/src/settings.rs:459-466) — unreachable.ok_or_else(...)with a misleading error message. from_url_with_first_byte_timeout_and_override_host(crates/trusted-server-core/src/backend.rs:303-318) — 51-character method name; the API is asking for a builder. Out of scope for this PR.
🌱 seedling
- Backend-name collision via
replace(['.', ':'], \"_\")(crates/trusted-server-core/src/backend.rs:144-149) — operator-controlled so practical risk is near-zero, but a hash or non-replacing escape would be more defensive.
⛏ nitpick
matched_asset_routecomputed eagerly (crates/trusted-server-adapter-fastly/src/main.rs:153-155) — only used in the catch-all branch. Move inside the_ =>arm.- Double Host header in publisher.rs (
crates/trusted-server-core/src/publisher.rs:528) —BackendConfig::override_hostis already authoritative; the manualset_headeris redundant. - POST-bypass test could be stronger (
crates/trusted-server-adapter-fastly/src/route_tests.rs:399-410) —assert_ne!(OK)is true for any non-200; pinBAD_GATEWAYlike the other tests.
CI Status
- fmt: PASS
- clippy: PASS
- rust tests: PASS (875 tests on this worktree, 0 failures)
- vitest: PASS
- format-typescript: PASS
- format-docs: PASS
- browser & integration tests: PASS
- CodeQL / Analyze: PASS
b2edefe to
8c8a51a
Compare
8c8a51a to
b4f13f9
Compare
Summary
[[proxy.asset_routes]]path-prefix routing so selected first-party asset paths can proxy to a different backend origin thanpublisher.origin_url.GET/HEAD, uses longest-prefix-wins, and preserves the full incoming path/query while swapping only the origin.Changes
docs/superpowers/specs/2026-04-28-multi-backend-asset-proxy-design.mdcrates/trusted-server-core/src/settings.rsProxyAssetRoute,proxy.asset_routesconfig support, asset-route normalization/validation, duplicate-prefix warnings, and longest-prefix matching helpers.crates/trusted-server-core/src/proxy.rscrates/trusted-server-adapter-fastly/src/main.rscrates/trusted-server-adapter-fastly/src/route_tests.rstrusted-server.toml[[proxy.asset_routes]]configuration shape with a commented example.Closes
Closes #663
Test plan
cargo test --workspacecargo clippy --workspace --all-targets --all-features -- -D warningscargo fmt --all -- --checkcd crates/js/lib && npx vitest runcd crates/js/lib && npm run formatcd docs && npm run formatcargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1fastly compute serveChecklist
unwrap()in production code — useexpect("should ...")tracingmacros (notprintln!)