Skip to content

fix(executor): defer WorkerJobRunningStateStore and CREATED state event to fix flaky test initialization#15395

Open
ammeek wants to merge 1 commit intodevelopfrom
fix/executor-flaky-bean-init
Open

fix(executor): defer WorkerJobRunningStateStore and CREATED state event to fix flaky test initialization#15395
ammeek wants to merge 1 commit intodevelopfrom
fix/executor-flaky-bean-init

Conversation

@ammeek
Copy link
Copy Markdown
Contributor

@ammeek ammeek commented Apr 6, 2026

✨ Description

defer WorkerJobRunningStateStore and CREATED state event to fix flaky test initialization

🛠️ Backend Checklist

  • Code compiles successfully and passes all checks
  • All unit and integration tests pass

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 6, 2026

🐋 Docker image

ghcr.io/kestra-io/kestra-pr:15395
docker run --pull=always --rm -it -p 8080:8080 --user=root -v /var/run/docker.sock:/var/run/docker.sock -v /tmp:/tmp ghcr.io/kestra-io/kestra-pr:15395 server local

🧪 Java Unit Tests

TestsPassed ✅Skipped ⚠️FailedTime ⏱
Java Tests Report4552 ran4529 ✅23 ⚠️0 ❌46m 11s 253ms

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 6, 2026

Tests report quick summary:

success ✅ > tests: 4553, success: 4529, skipped: 23, failed: 1

unfold for details
Project Status Success Skipped Failed
cli success ✅ 47 0 0
core success ✅ 2108 1 0
executor success ✅ 31 0 0
jdbc success ✅ 11 0 0
jdbc-h2 success ✅ 549 0 0
jdbc-mysql success ✅ 541 0 0
jdbc-postgres success ✅ 541 0 0
processor success ✅ 7 0 0
queue success ✅ 31 0 0
runner-memory success ✅ 1 0 0
scheduler success ✅ 57 0 0
script success ✅ 31 0 0
storage-local success ✅ 64 0 0
webserver success ✅ 420 0 0
worker success ✅ 51 0 0
worker-controller success ✅ 61 0 0

Develocity build scan: https://develocity.kestra.io/s/soc6bgxcsfiha

Copy link
Copy Markdown
Member

@loicmathieu loicmathieu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fhussonnois WDYT?
Should we always swtich to CREATED in a postconstruct?

@ammeek It seems a bit odd to me, I agree to use a Provider to break dependency cycle but I'm not sure I understand why you have to set the CREATED state later

@ammeek
Copy link
Copy Markdown
Contributor Author

ammeek commented Apr 7, 2026

@loicmathieu setState publishes a ServiceStateChangeEvent which is consumed by the ServiceLivenessManager which I thought caused this bean to be initialised before the beans it needs are available. After taking a second look I can see that ServiceLivenessManager is a Context bean so the event publishing does not cause this issue like I thought.

I think that the event publishing should still be done in the post construct as it's not required to initialise the bean.

I'll take another look into what is causing the issue and update the description of this PR to include a more detailed description of the fix I find for it.

@ammeek
Copy link
Copy Markdown
Contributor Author

ammeek commented Apr 7, 2026

The failing JavaSecurityTest has stumped me. It looks like the bean definition for ServerConfig$Liveness does not exists. This is strange because the correct application.yaml values have been set. I thought that this might have been caused by the class being compiled before the correct configs are loaded but this does not seem to be the case.

I'll continue to investigate this. @loicmathieu if you have any idea of how i can go about this it would be appreciated.

https://github.com/kestra-io/kestra-ee/actions/runs/24029838381/job/70090388092

@ammeek
Copy link
Copy Markdown
Contributor Author

ammeek commented Apr 14, 2026

After some investigation last week we found out that the JavaSecurityTest was a lot less flaky than we originally thought .The test report was caching the results from earlier runs on the CI pipeline. Since JavaSecurityTest is no longer a blocking issue I am going to drop the investigation.

@fhussonnois would you be able we weigh in on Loïc comment?

@fhussonnois WDYT? Should we always switch to CREATED in a postconstruct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: To review

Development

Successfully merging this pull request may close these issues.

4 participants