CI tests take too much time

[campospinto]

As of today (March 13, 2025), the continuous integration tests take more than 1h.

We should reduce this time if possible. We could start by profiling the tests and see if some tests are expensive with no good reason.

[jowezarek]

@kvrigor once presented a tool during our Psydac meeting that was able to show which tests take up the most time! @kvrigor, could you let us know which tests take up the most time / how to use this tool?

[kvrigor]

@jowezarek this should be addressed by #458. Unfortunately I don't yet have time to implement this; but all the necessary info to replicate the tests should be in that PR.

[kvrigor]

How to run profiling and visualize the results:

# Install profiling tools
pip3 install pytest-profiling snakeviz

# Run profiling on one test
pytest --profile psydac/feec/tests/test_global_projectors.py -k "test_derham_projector_3d"

# Visualize profiling results on a web browser
snakeviz prof/test_derham_projector_3d/combined.prof

# Repeat for another slow test
pytest --profile psydac/feec/tests/test_commuting_projections.py -k "test_3d_commuting_pro_3"
snakeviz prof/test_3d_commuting_pro_3/combined.prof

[campospinto]

thanks @kvrigor ! This will be useful. We can probably gain some time already by running some tests on coarse grids. I will run some profiling.

[kvrigor]

One quick way to reduce test time is to decrease the testing combinations below (if it's possible). Otherwise more effort would be involved (i.e. #458).

psydac/psydac/feec/tests/test_global_projectors.py

Lines 251 to 256 in ae3e803

	@pytest.mark.parametrize('ncells', [[30,30,30]])
	@pytest.mark.parametrize('degree', [[2,2,2], [2,3,2], [3,3,3]])
	@pytest.mark.parametrize('periodic', [[False, False, False], [True, True, True]])
	@pytest.mark.parametrize('multiplicity', [[1,1,1], [1,2,2], [2,2,2]])
	def test_derham_projector_3d(ncells, degree, periodic, multiplicity):

psydac/psydac/feec/tests/test_commuting_projections.py

Lines 187 to 192 in ae3e803

	@pytest.mark.parametrize('Nel', [8, 12])
	@pytest.mark.parametrize('Nq', [8])
	@pytest.mark.parametrize('p', [2,3])
	@pytest.mark.parametrize('bc', [True, False])
	@pytest.mark.parametrize('m', [1,2])
	def test_3d_commuting_pro_3(Nel, Nq, p, bc, m):

[campospinto]

Indeed -- in particular given that the commuting property does not really depend on the grid resolution.

[yguclu]

For starters I think we just need to invoque Pytest with the flag --durations=10, which will list the 10 slowest tests and their runtime. See https://docs.pytest.org/en/6.2.x/usage.html#profiling-test-execution-duration.

[anushkasinghh]

I ran pytest --durations=10 to review the slowest tests. Here's the current top 10:

320.48s call     psydac/api/tests/test_ci_performance.py::test_maxwell_2d_2_patch_dirichlet_1
135.64s call     psydac/api/tests/test_ci_performance.py::test_navier_stokes_2d
131.46s call     psydac/api/tests/test_ci_performance.py::test_time_harmonic_maxwell_pretzel_f_nc
126.92s call     psydac/api/tests/test_ci_performance.py::test_time_harmonic_maxwell_pretzel_f
111.83s call     psydac/api/tests/test_ci_performance.py::test_poisson_pretzel_f_nc
104.76s call     psydac/api/tests/test_ci_performance.py::test_poisson_pretzel_f
72.83s call     psydac/api/tests/test_ci_performance.py::test_maxwell_3d_2_mult
48.48s call     psydac/api/tests/test_ci_performance.py::test_maxwell_2d_2_patch_dirichlet_0
29.46s call     psydac/api/tests/test_ci_performance.py::test_maxwell_2d_2_patch_dirichlet_2

I’m unsure about the best way to proceed here; any suggestions would be appreciated.