Feat invalid Unicode surrogate crash in JSON serialization#15403
Conversation
- Add SurrogateStrippingStringSerializer - Integrate serializer in JacksonMapper - Add comprehensive tests covering edge cases and issue kestra-io#14806
loicmathieu
left a comment
There was a problem hiding this comment.
I understand the rational but I'm not fan.
Doing this is highly opinionated and would incure a performance cost of reading the whome JSON byte array.
I would prefer that we fail gracely if there is an unsupported character sequence, we already handled that in the queue and he executor I think (we catch the Postgres specific error message and fail the exec).
A better approach may also to have validation whenever possible to avoid such un-processable strings to be set (like in inputs).
|
Hello @loicmathieu , I also tightened the queue-side detection so unsupported Unicode payload failures are still converted to UnsupportedMessageException, including lone surrogate cases, and kept regression coverage on the Postgres queue tests without relying on one exact Postgres error string. I couldn’t fully verify the Postgres test locally because my local test environment does not currently have Postgres available on localhost:5432, but the code compiles and the change is now targeted to the existing recoverable failure path rather than global string replacement. |
ed13d6f to
80370c2
Compare
|
Backend tests themselves passed; the failing job is in the reporting step after Gradle and looks related to fork PR permissions ("Resource not accessible |
Added support for handling invalid UTF-16 surrogate sequences during JSON serialization.
This prevents crashes when serializing malformed Unicode strings (e.g., PostgreSQL jsonb rejection).
Fixes #14806