When golang-filter is used, if the backend service is unhealthy, envoyf returns a local response 503. However, if the client still sends request data, a panic occurs, causing the Envoy process to crash.

[NetRanger-222]

If you are reporting any crash or any potential security issue, do not
open an issue in this repo. Please report the issue via emailing
envoy-security@googlegroups.com where the issue will be triaged appropriately.

Title: When golang-filter is used, if the backend service is unhealthy, envoyf returns a local response 503. However, if the client still sends request data, a panic occurs, causing the Envoy process to crash

Description:

When golang-filter is used, if the backend service is unhealthy, envoyf returns a local response 503. However, if the client still sends request data, a panic occurs, causing the Envoy process to crash.

Logs:

2026-04-08T10:27:13.077096Z debug envoy upstream external/envoy/source/common/upstream/cluster_manager_impl.cc:2001 no healthy host for HTTP connection pool thread=487
2026-04-08T10:27:13.077294Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:40 golang filter onLocalReply, decoding state: decoder:WaitingData, encoding state: encoder:WaitingHeader, code: 503 thread=487
2026-04-08T10:27:13.077306Z debug envoy http external/envoy/source/common/http/filter_manager.cc:1034 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] Preparing local reply with details no_healthy_upstream thread=487
2026-04-08T10:27:13.077320Z debug envoy http external/envoy/source/common/http/filter_manager.cc:1076 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] Executing sending local reply. thread=487
2026-04-08T10:27:13.077328Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:91 golang filter encodeHeaders, encoding state: encoder:WaitingHeader, end_stream: false thread=487
2026-04-08T10:27:13.077331Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:253 golang filter doHeaders, state: encoder:WaitingHeader, end_stream: false thread=487
2026-04-08T10:27:13.077334Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:240 golang filter passing header to golang, state: encoder:ProcessingHeader, end_stream: false thread=487
2026-04-08T10:27:13.077345Z debug envoy http external/envoy/contrib/golang/filters/http/source/processor_state.cc:52 golang filter handle header status, state: encoder:ProcessingHeader, status: 2 thread=487
2026-04-08T10:27:13.077348Z debug envoy http external/envoy/contrib/golang/filters/http/source/processor_state.cc:89 golang filter after handle header status, state: encoder:WaitingData, status: 2 thread=487
2026-04-08T10:27:13.077376Z debug envoy http external/envoy/source/common/http/conn_manager_impl.cc:1899 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] encoding headers via codec (end_stream=false):
':status', '503'
'content-length', '19'
'content-type', 'text/plain'
'date', 'Wed, 08 Apr 2026 02:27:12 GMT'
'server', 'istio-envoy'
'connection', 'close'
thread=487
2026-04-08T10:27:13.077395Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:110 golang filter encodeData, encoding state: encoder:WaitingData, data length: 19, end_stream: true thread=487
2026-04-08T10:27:13.077398Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:283 golang filter doData, state: encoder:WaitingData, end_stream: true thread=487
2026-04-08T10:27:13.077401Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:269 golang filter passing data to golang, state: encoder:WaitingData, end_stream: true thread=487
2026-04-08T10:27:13.077408Z debug envoy http external/envoy/contrib/golang/filters/http/source/processor_state.cc:96 golang filter handle data status, state: encoder:ProcessingData, status: 2 thread=487
2026-04-08T10:27:13.077411Z debug envoy http external/envoy/contrib/golang/filters/http/source/processor_state.cc:160 golang filter after handle data status, state: encoder:Done, status: 2 thread=487
2026-04-08T10:27:13.077413Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:317 golang filter doData, return: true thread=487
2026-04-08T10:27:13.077424Z debug envoy http external/envoy/source/common/http/conn_manager_impl.cc:2014 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] Codec completed encoding stream. thread=487
2026-04-08T10:27:13.077439Z debug envoy http external/envoy/source/common/http/conn_manager_impl.cc:257 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] doEndStream() resetting stream thread=487
2026-04-08T10:27:13.077446Z debug envoy http external/envoy/source/common/http/conn_manager_impl.cc:1971 [Tags: "ConnectionId":"779","StreamId":"7815793805644531524"] stream reset: reset reason: local reset, response details: - thread=487
2026-04-08T10:27:13.078453Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:144 golang filter on destroy thread=487
2026-04-08T10:27:13.078482Z debug envoy connection external/envoy/source/common/network/connection_impl.cc:150 [Tags: "ConnectionId":"779"] closing data_to_write=178 type=2 thread=487
2026-04-08T10:27:13.078493Z debug envoy connection external/envoy/source/common/network/connection_impl_base.cc:51 [Tags: "ConnectionId":"779"] setting delayed close timer with timeout 1000 ms thread=487
2026-04-08T10:27:13.078510Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:418 after done handle golang status, status: 2, state: decoder:WaitingData, done: true, seen trailers: false thread=487
2026-04-08T10:27:13.078514Z debug envoy http external/envoy/contrib/golang/filters/http/source/golang_filter.cc:269 golang filter passing data to golang, state: decoder:WaitingData, end_stream: false thread=487
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xffff9886090c]

goroutine 99 [running, locked to thread]:
github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.getState(...)
/usr1/envoy_source/envoy/contrib/golang/filters/http/source/go/pkg/http/shim.go:161
github.com/envoyproxy/envoy/contrib/golang/filters/http/source/go/pkg/http.envoyGoFilterOnHttpData(0x2678fea2a050, 0x0, 0x2678feba7180, 0x1b69)
/usr1/envoy_source/envoy/contrib/golang/filters/http/source/go/pkg/http/shim.go:239 +0x7c

Note: If there are privacy concerns, sanitize the data prior to
sharing.

Call Stack:

If the Envoy binary is crashing, a call stack is required.
Please refer to the Bazel Stack trace documentation.

[NetRanger-222]

The getState function needs to handle the exception when the request does not exist.

[wdauchy]

We (datadog) will take a look at this issue

[henrymwang]

👋 Hi,

I've proposed a fix in this PR #44429. I've created a minimal Docker reproduction environment with: Envoy on the latest version, HTTP client that sends a POST request to Envoy with a body, and the backend has its health-check flapping. After applying the patch in our local sandbox environment, we no longer see this happening.

[wdauchy]

@botengyao I think we can close this issue