Skip to content

Commit 898d8d8

Browse files
ovadiagalovadiagal
committed
.
Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> . Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> rename Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> , Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> work Signed-off-by: Gal Ovadia <govadia@palantir.com> Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> clean Signed-off-by: Gal Ovadia <govadia@palantir.com> Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> cleanuP Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> update to latest sdk Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> cleanup Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> cleanup Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> style match lib.rs Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> add tests, readme, Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> clean Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> Update lib.rs Signed-off-by: Gal Ovadia <ggalovadia@gmail.com> Update README.md Signed-off-by: Gal Ovadia <ggalovadia@gmail.com>
1 parent 2e468f3 commit 898d8d8

11 files changed

Lines changed: 2075 additions & 12 deletions

File tree

‎rust/Cargo.lock‎

Lines changed: 759 additions & 9 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎rust/Cargo.toml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,19 @@ repository = "https://github.com/envoyproxy/dynamic-modules-example"
77

88
[dependencies]
99
# The SDK version must match the Envoy version due to the strict compatibility requirements.
10-
envoy-proxy-dynamic-modules-rust-sdk = { git = "https://github.com/envoyproxy/envoy", rev = "6d9bb7d9a85d616b220d1f8fe67b61f82bbdb8d3" }
10+
envoy-proxy-dynamic-modules-rust-sdk = { git = "https://github.com/envoyproxy/envoy", rev = "f0e51db62b58196f012f93f20899d86ec81c63e6" }
1111
serde = { version = "1.0", features = ["derive"] }
1212
serde_json = "1.0"
1313
rand = "0.9.0"
1414
matchers = "0.2.0"
15+
dashmap = "6.1.0"
16+
once_cell = "1.20.2"
17+
hickory-proto = "0.24"
18+
parking_lot = "0.12"
19+
prost = "0.13"
20+
21+
[build-dependencies]
22+
prost-build = "0.13"
1523

1624
[dev-dependencies]
1725
tempfile = "3.16.0"

‎rust/build.rs‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
fn main() {
2+
let mut config = prost_build::Config::new();
3+
config.type_attribute(".", "#[derive(serde::Deserialize, serde::Serialize)]");
4+
config.field_attribute(".", "#[serde(default)]");
5+
config
6+
.compile_protos(
7+
&["src/dns_gateway/dns_gateway.proto"],
8+
&["src/dns_gateway/"],
9+
)
10+
.unwrap();
11+
}

‎rust/src/dns_gateway/README.md‎

Lines changed: 330 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,330 @@
1+
# DNS Gateway
2+
3+
Envoy dynamic module filters that intercept DNS queries and route TCP connections to external
4+
domains via virtual IP allocation.
5+
6+
![DNS Gateway diagram](diagram.png)
7+
8+
## Prerequisites
9+
10+
Requires iptables/nftables rules to redirect application traffic to Envoy:
11+
12+
- **DNS**: UDP port 53 redirected to Envoy's DNS listener (e.g. port 15053)
13+
- **TCP**: Outbound connections redirected to Envoy's TCP listener (e.g. port 15001)
14+
15+
See [`connectivity-iptables`](../../../../../connectivity-iptables) for setup scripts.
16+
17+
## How it works
18+
19+
1. **`dns_gateway`** (UDP listener filter) — Intercepts DNS queries. If the queried domain matches
20+
a configured pattern, allocates a virtual IP from a private subnet and responds with an A record.
21+
Caches the mapping from virtual IP to domain and metadata. Non-matching queries pass through.
22+
23+
2. **`cache_lookup`** (network filter) — On new TCP connections, looks up the destination virtual IP
24+
in the shared cache and sets the resolved domain and metadata as Envoy
25+
[filter state](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/data_sharing_between_filters#primitives)
26+
for use in routing.
27+
28+
```
29+
Application
30+
| DNS query: "bucket-1.aws.com"
31+
v
32+
dns_gateway
33+
| matches "*.aws.com", allocates 10.10.0.1, responds with A record
34+
v
35+
Application
36+
| TCP connect to 10.10.0.1:443
37+
v
38+
cache_lookup
39+
| resolves 10.10.0.1 -> domain="bucket-1.aws.com", metadata.cluster="aws"
40+
v
41+
tcp_proxy
42+
| routes to upstream cluster using filter state
43+
v
44+
External service (bucket-1.aws.com)
45+
```
46+
47+
## Filter state
48+
49+
`cache_lookup` sets the following keys, accessible via `%FILTER_STATE(...)%`:
50+
51+
| Key | Example |
52+
| ---------------------------------- | -------------------------------- |
53+
| `envoy.dns_gateway.domain` | `bucket-1.aws.com` |
54+
| `envoy.dns_gateway.metadata.<key>` | value from matched domain config |
55+
56+
Usage in Envoy config:
57+
58+
- `%FILTER_STATE(envoy.dns_gateway.domain:PLAIN)%`
59+
- `%FILTER_STATE(envoy.dns_gateway.metadata.cluster:PLAIN)%`
60+
- `%FILTER_STATE(envoy.dns_gateway.metadata.auth_token:PLAIN)%`
61+
62+
## Domain matching
63+
64+
- **Exact**: `"example.com"` — matches only `example.com`
65+
- **Wildcard**: `"*.aws.com"` — matches any subdomain (e.g. `bucket-1.aws.com`,
66+
`sub.api.aws.com`) but not `aws.com` itself
67+
68+
## Configuration reference
69+
70+
### `dns_gateway`
71+
72+
| Field | Type | Description |
73+
| -------------------- | ------- | ---------------------------------------------------------------- |
74+
| `base_ip` | string | Base IPv4 address for virtual IP allocation (e.g. `"10.10.0.0"`) |
75+
| `prefix_len` | integer | CIDR prefix length (1-32). A `/24` gives 256 IPs. |
76+
| `domains` | array | Domain matchers |
77+
| `domains[].domain` | string | Exact (`"example.com"`) or wildcard (`"*.example.com"`) pattern |
78+
| `domains[].metadata` | object | String key-value pairs exposed via filter state |
79+
80+
### `cache_lookup`
81+
82+
No configuration. Use `filter_config: {}`.
83+
84+
## Manual testing
85+
86+
End-to-end test with docker-compose. I recommend using a clean linux VM for this.
87+
88+
Create the following files:
89+
90+
**docker-compose.yml**:
91+
92+
```yaml
93+
services:
94+
envoy:
95+
image: <your-envoy-image>
96+
network_mode: host
97+
volumes:
98+
- ./envoy.yaml:/etc/envoy/envoy.yaml
99+
command: ["envoy", "-c", "/etc/envoy/envoy.yaml", "-l", "debug"]
100+
101+
upstream-1:
102+
image: python:3.12-slim
103+
network_mode: host
104+
volumes:
105+
- ./upstream_1.py:/app/server.py
106+
command: ["python3", "/app/server.py"]
107+
108+
upstream-2:
109+
image: python:3.12-slim
110+
network_mode: host
111+
volumes:
112+
- ./upstream_2.py:/app/server.py
113+
command: ["python3", "/app/server.py"]
114+
```
115+
116+
**upstream_1.py** (port 18001):
117+
118+
```python
119+
from http.server import HTTPServer, BaseHTTPRequestHandler
120+
121+
class Handler(BaseHTTPRequestHandler):
122+
def do_CONNECT(self):
123+
print(f"\nCONNECT {self.path}")
124+
for key, value in self.headers.items():
125+
print(f" {key}: {value}")
126+
127+
self.send_response(200)
128+
self.end_headers()
129+
130+
request = self.connection.recv(4096)
131+
132+
body = f"cluster_1\nCONNECT: {self.path}\n"
133+
for key, value in self.headers.items():
134+
body += f"{key}: {value}\n"
135+
136+
resp = f"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nX-Upstream: cluster_1\r\nContent-Length: {len(body)}\r\n\r\n{body}"
137+
self.connection.sendall(resp.encode())
138+
139+
HTTPServer(("0.0.0.0", 18001), Handler).serve_forever()
140+
```
141+
142+
**upstream_2.py** (port 18002):
143+
144+
```python
145+
from http.server import HTTPServer, BaseHTTPRequestHandler
146+
147+
class Handler(BaseHTTPRequestHandler):
148+
def do_CONNECT(self):
149+
print(f"\nCONNECT {self.path}")
150+
for key, value in self.headers.items():
151+
print(f" {key}: {value}")
152+
153+
self.send_response(200)
154+
self.end_headers()
155+
156+
request = self.connection.recv(4096)
157+
158+
body = f"cluster_2\nCONNECT: {self.path}\n"
159+
for key, value in self.headers.items():
160+
body += f"{key}: {value}\n"
161+
162+
resp = f"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nX-Upstream: cluster_2\r\nContent-Length: {len(body)}\r\n\r\n{body}"
163+
self.connection.sendall(resp.encode())
164+
165+
HTTPServer(("0.0.0.0", 18002), Handler).serve_forever()
166+
```
167+
168+
**envoy.yaml**:
169+
170+
```yaml
171+
static_resources:
172+
listeners:
173+
- name: dns_listener
174+
address:
175+
socket_address:
176+
address: 0.0.0.0
177+
port_value: 15053
178+
protocol: UDP
179+
listener_filters:
180+
- name: envoy.filters.udp_listener.dynamic_modules
181+
typed_config:
182+
"@type": type.googleapis.com/envoy.extensions.filters.udp.dynamic_modules.v3.DynamicModuleUdpListenerFilter
183+
dynamic_module_config:
184+
name: connectivity_envoy_module
185+
do_not_close: true
186+
filter_name: dns_gateway
187+
filter_config:
188+
"@type": type.googleapis.com/google.protobuf.Struct
189+
value:
190+
base_ip: "10.10.0.0"
191+
prefix_len: 24
192+
domains:
193+
- domain: "*.aws.com"
194+
metadata:
195+
cluster: cluster_1
196+
auth_token: "abc123"
197+
- domain: "example.com"
198+
metadata:
199+
cluster: cluster_2
200+
auth_token: "def456"
201+
- name: envoy.filters.udp_listener.dns_filter
202+
typed_config:
203+
"@type": type.googleapis.com/envoy.extensions.filters.udp.dns_filter.v3.DnsFilterConfig
204+
stat_prefix: dns_fallback
205+
client_config:
206+
max_pending_lookups: 256
207+
dns_resolution_config:
208+
resolvers:
209+
- socket_address:
210+
protocol: TCP
211+
address: 172.20.0.10
212+
port_value: 53
213+
dns_resolver_options:
214+
no_default_search_domain: true
215+
use_tcp_for_dns_lookups: true
216+
server_config:
217+
inline_dns_table: {}
218+
219+
- name: tcp_listener
220+
address:
221+
socket_address:
222+
address: 0.0.0.0
223+
port_value: 15001
224+
listener_filters:
225+
- name: envoy.filters.listener.original_dst
226+
typed_config:
227+
"@type": type.googleapis.com/envoy.extensions.filters.listener.original_dst.v3.OriginalDst
228+
filter_chains:
229+
- filters:
230+
- name: envoy.filters.network.dynamic_modules
231+
typed_config:
232+
"@type": type.googleapis.com/envoy.extensions.filters.network.dynamic_modules.v3.DynamicModuleNetworkFilter
233+
dynamic_module_config:
234+
name: connectivity_envoy_module
235+
do_not_close: true
236+
filter_name: cache_lookup
237+
filter_config: {}
238+
# IMPORTANT! Setting an upstream cluster directly in TCP proxy config with FILTER_STATE(...)
239+
# is not supported. Instead, write the value of FILTER_STATE(...) to 'envoy.tcp_proxy.cluster'
240+
- name: envoy.filters.network.set_filter_state
241+
typed_config:
242+
"@type": type.googleapis.com/envoy.extensions.filters.network.set_filter_state.v3.Config
243+
on_new_connection:
244+
- object_key: envoy.tcp_proxy.cluster
245+
format_string:
246+
text_format_source:
247+
inline_string: "%FILTER_STATE(envoy.dns_gateway.metadata.cluster:PLAIN)%"
248+
- name: envoy.filters.network.tcp_proxy
249+
typed_config:
250+
"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
251+
stat_prefix: egress
252+
cluster: default
253+
tunneling_config:
254+
hostname: "%FILTER_STATE(envoy.dns_gateway.domain:PLAIN)%"
255+
headers_to_add:
256+
- header:
257+
key: "X-Auth-Token"
258+
value: "%FILTER_STATE(envoy.dns_gateway.metadata.auth_token:PLAIN)%"
259+
260+
clusters:
261+
- name: cluster_1
262+
type: STATIC
263+
load_assignment:
264+
cluster_name: cluster_1
265+
endpoints:
266+
- lb_endpoints:
267+
- endpoint:
268+
address:
269+
socket_address:
270+
address: 127.0.0.1
271+
port_value: 18001
272+
273+
- name: cluster_2
274+
type: STATIC
275+
load_assignment:
276+
cluster_name: cluster_2
277+
endpoints:
278+
- lb_endpoints:
279+
- endpoint:
280+
address:
281+
socket_address:
282+
address: 127.0.0.1
283+
port_value: 18002
284+
```
285+
286+
### 2. Start
287+
288+
```bash
289+
docker-compose up
290+
```
291+
292+
### 3. Set up iptables redirect
293+
294+
```bash
295+
# Redirect DNS (UDP 53) to Envoy's DNS listener
296+
sudo iptables -t nat -A OUTPUT -p udp --dport 53 -j DNAT --to-destination 127.0.0.1:15053
297+
298+
# Redirect TCP to virtual IPs (10.10.0.0/24) to Envoy's TCP listener
299+
sudo iptables -t nat -A OUTPUT -p tcp -d 10.10.0.0/24 -j DNAT --to-destination 127.0.0.1:15001
300+
```
301+
302+
### 4. Test
303+
304+
```bash
305+
# Will allocate sequentially increasing virtual IPs
306+
dig one.s3.aws.com
307+
dig two.s3.aws.com
308+
dig example.com
309+
310+
# Unmatched domain, will defer to external DNS
311+
dig github.com
312+
313+
# Will reach cluster_1
314+
curl http://s3.aws.com./
315+
# Note that the trailing dot is necessary, Coder might try and append extra parts to the domain because of ndots funkiness
316+
317+
# Will reach cluster_2
318+
curl http://example.com./
319+
320+
# See logs for upstream-1 and upstream-2
321+
docker-compose logs upstream-1
322+
docker-compose logs upstream-2
323+
```
324+
325+
### 5. Clean up iptables
326+
327+
```bash
328+
sudo iptables -t nat -D OUTPUT -p udp --dport 53 -j DNAT --to-destination 127.0.0.1:15053
329+
sudo iptables -t nat -D OUTPUT -p tcp -d 10.10.0.0/24 -j DNAT --to-destination 127.0.0.1:15001
330+
```

0 commit comments

Comments
 (0)