Skip to content

Introduce datumaro integration tests for Geti Sample types#6053

Merged
itallix merged 4 commits intodevelopfrom
vitalii/6047-ci-datumaro-regression
Apr 15, 2026
Merged

Introduce datumaro integration tests for Geti Sample types#6053
itallix merged 4 commits intodevelopfrom
vitalii/6047-ci-datumaro-regression

Conversation

@itallix
Copy link
Copy Markdown
Contributor

@itallix itallix commented Apr 7, 2026

Summary

The goal of this PR is to be more proactive when detecting regressions related to datumaro integration.
New test suite will be executed on pre-collected datasets that cover all supported formats and App Sample types.

The failures will indicate either the issues in experimental datumaro logic (import_dataset + convert_schema) or the outdated App sample type definitions.

No job orchestration involved (this is covered in BDD)

Let's merge after experimental API will be healthy:

Checklist

  • The PR title and description are clear and descriptive
  • I have manually tested the changes
  • All changes are covered by automated tests
  • All related issues are linked to this PR (if applicable)
  • Documentation has been updated (if applicable)

@itallix itallix requested review from a team as code owners April 7, 2026 15:11
@itallix itallix linked an issue Apr 7, 2026 that may be closed by this pull request
Copilot AI review requested due to automatic review settings April 7, 2026 15:11
@github-actions github-actions bot added TEST Any changes in tests BUILD Geti Backend Issues related to the Geti application server labels Apr 7, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new backend test suite intended to catch regressions in Datumaro import + schema conversion against pre-collected “real” datasets, and wires it into CI so failures surface early when sample schemas or experimental Datumaro logic drift.

Changes:

  • Added tests/datumaro_integration pytest suite that downloads a regression dataset bundle and validates import_dataset() + convert_to_schema() for supported sample types.
  • Added a just test-datumaro-integration target and executed it in the backend CI workflow.
  • Updated uv.lock (Datumaro git revision and related wheel URL entries).

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
application/backend/tests/datumaro_integration/test_datasets.py New smoke test covering import + schema conversion per archive type.
application/backend/tests/datumaro_integration/conftest.py Downloads/extracts regression archives and parametrizes tests over them.
application/backend/Justfile Adds a dedicated command to run the new test suite (and tweaks BDD invocation verbosity/path).
.github/workflows/backend-lint-and-test.yaml Runs Datumaro integration tests as part of backend CI.
application/backend/uv.lock Moves pinned Datumaro revision and updates some locked artifact URLs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread application/backend/tests/integration/datumaro/conftest.py
Comment thread application/backend/tests/integration/datumaro/conftest.py
Comment thread application/backend/tests/integration/datumaro/conftest.py
Comment thread application/backend/tests/integration/datumaro/conftest.py
Comment thread application/backend/tests/datumaro_integration/test_datasets.py Outdated
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 7, 2026

🐳 Docker image sizes

Device Size
cpu 3592.9 MB (3.51 GB)
xpu 10561.5 MB (10.31 GB)
cuda 11713.8 MB (11.44 GB)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 7, 2026

📊 Test coverage report

Metric Coverage
Lines 56.2%
Functions 78.2%
Branches 87.4%
Statements 56.2%

@codecov-commenter
Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Comment thread application/backend/tests/integration/datumaro/conftest.py
@itallix itallix force-pushed the vitalii/6047-ci-datumaro-regression branch from 56fd33f to d826019 Compare April 13, 2026 11:44
Comment thread application/backend/tests/integration/datumaro/conftest.py Outdated
@itallix itallix added this pull request to the merge queue Apr 15, 2026
Merged via the queue into develop with commit 82467ab Apr 15, 2026
35 checks passed
@itallix itallix deleted the vitalii/6047-ci-datumaro-regression branch April 15, 2026 09:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

BUILD Geti Backend Issues related to the Geti application server TEST Any changes in tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ci: Add regression tests step for datumaro integration

5 participants