|
1 | 1 | <h2>Why is this an issue?</h2> |
2 | | -<p>Tests should always:</p> |
| 2 | +<p>Randomness in test code, whether introduced intentionally to cover multiple scenarios or unintentionally through non-deterministic library |
| 3 | +functions, undermines the principles of effective testing. In most cases, randomness leads to problems, resulting in code that is unreliable and |
| 4 | +difficult to debug. Consequently, deterministic and reproducible tests are preferred, primarily for the following reasons:</p> |
3 | 5 | <ul> |
4 | | - <li> Make sure that production code behaves as expected, including edge cases. </li> |
5 | | - <li> Be easy to debug, i.e. understandable and reproducible. </li> |
| 6 | + <li> When a test fails, the ability to reproduce the conditions that led to the failure is crucial for effective debugging. Randomness can make it |
| 7 | + difficult or even impossible to pinpoint the root cause, as subsequent runs may not exhibit the same failure. </li> |
| 8 | + <li> Being able to replay a scenario allows us to easily compare logs between different test runs. </li> |
| 9 | + <li> Determinism gives us confidence that a bug is fixed when it no longer appears in tests. If they behave randomly, a passing test after a fix |
| 10 | + might be coincidental due to a specific random input, rather than a genuine resolution of the underlying problem. </li> |
| 11 | + <li> Flaky tests, which pass or fail intermittently without any code changes, are a significant problem for CI pipelines (continuous integration). |
| 12 | + They erode confidence in the CI system, lead to unnecessary investigations and reruns, and ultimately slow down the development and release process. |
| 13 | + A stable CI pipeline relies on deterministic test outcomes. </li> |
6 | 14 | </ul> |
7 | | -<p>Using random values in tests will not necessarily check edge cases, and it will make test logs a lot harder to read. It is better to use easily |
8 | | -readable hardcoded values. If this makes your code bigger you can use helper functions.</p> |
9 | | -<p>There is one valid use case for random data in tests: when testing every value would make tests impractically slow. In this case the best you can |
10 | | -do is use random to test every value on the long run. You should however make sure that random values are logged so that you can reproduce failures. |
11 | | -Some libraries exist to make all this easier. You can for example use property-based testing libraries such as <a |
12 | | -href="https://github.com/jlink/jqwik">jqwik</a>.</p> |
13 | 15 | <p>This rule raises an issue when <code>new Random()</code> or <code>UUID.randomUUID()</code> are called in test code.</p> |
| 16 | +<h2>How to fix it</h2> |
| 17 | +<ul> |
| 18 | + <li> When a test uses random numbers to generate inputs, an easy fix is to replace those random inputs with pseudo-random values generated from a |
| 19 | + known seed. By initializing a pseudo-random number generator with a fixed seed, tests can generate sequences of seemingly random data that are |
| 20 | + reproducible across different test runs. </li> |
| 21 | + <li> When randomness occurs due to the use of a library function, the solution is to replace the call with a constant. For example, rather than |
| 22 | + generating a UUID at random, one should use a fixed value. </li> |
| 23 | +</ul> |
14 | 24 | <h3>Noncompliant code example</h3> |
15 | 25 | <pre> |
16 | 26 | int userAge = new Random().nextInt(42); // Noncompliant |
17 | 27 | UUID userID = UUID.randomUUID(); // Noncompliant |
18 | 28 | </pre> |
19 | 29 | <h3>Compliant solution</h3> |
20 | 30 | <pre> |
21 | | -int userAge = 31; |
| 31 | +static final int SEED = 0x533d; |
| 32 | +int userAge = new Random(SEED).nextInt(42); |
22 | 33 | UUID userID = UUID.fromString("00000000-000-0000-0000-000000000001"); |
23 | 34 | </pre> |
24 | 35 | <h2>Resources</h2> |
25 | 36 | <ul> |
26 | 37 | <li> <a href="https://phauer.com/2019/modern-best-practices-testing-java/#use-fixed-data-instead-of-randomized-data">Modern Best Practices for |
27 | 38 | Testing in Java - Philipp Hauer</a> </li> |
28 | | - <li> <a href="https://jqwik.net/">Jqwik test engine</a> </li> |
29 | 39 | </ul> |
30 | 40 |
|
0 commit comments