|
| 1 | +# What is AcceptPy? |
| 2 | + |
| 3 | +AcceptPy is a minimal acceptance test framework. It can be used to automatically and quickly check that a program operates as a developer intends. What a unit test framework is for functions, AcceptPy is for whole programs. |
| 4 | + |
| 5 | +For example we can use AcceptPy to test the operation of the console `echo` command. To do so we first author a test file (in JSON) which defines a test for the echo command: |
| 6 | + |
| 7 | +``` |
| 8 | +{ |
| 9 | + "test_type": "shell command", |
| 10 | + "command" : "echo have some pie.", |
| 11 | + "expect_exit": 0, |
| 12 | + "expect_stdout": "have some pie.\n" |
| 13 | +} |
| 14 | +``` |
| 15 | +This test checks than executing `echo have some pie.` prints: |
| 16 | + |
| 17 | +``` |
| 18 | +have some pie |
| 19 | +
|
| 20 | +``` |
| 21 | + |
| 22 | +to the console and then exits with code 0. |
| 23 | + |
| 24 | +We can run this test by invoking AcceptPy passing the directory that contains the test file: |
| 25 | + |
| 26 | +``` |
| 27 | +python3 accept.py /home/user/echo_tests |
| 28 | +``` |
| 29 | + |
| 30 | +which produces the following output: |
| 31 | + |
| 32 | +``` |
| 33 | +######################### |
| 34 | + Welcome to AcceptPy |
| 35 | +######################### |
| 36 | +
|
| 37 | +searching for tests... |
| 38 | +
|
| 39 | +/home/user/echo_test/echo_pie.test |
| 40 | +
|
| 41 | +found 1 tests. |
| 42 | +
|
| 43 | +
|
| 44 | +######################### |
| 45 | + Test 1 of 1 |
| 46 | +######################### |
| 47 | +
|
| 48 | +[running: /home/user/echo_tests/echo_pie.test] |
| 49 | +shell command: echo have some pie. |
| 50 | +<begin stdout> |
| 51 | +have some pie. |
| 52 | +
|
| 53 | +<end stdout> |
| 54 | +[PASSED] |
| 55 | +
|
| 56 | +
|
| 57 | +######################### |
| 58 | + Results: |
| 59 | +######################### |
| 60 | +
|
| 61 | +
|
| 62 | +1 of 1 tests passed. |
| 63 | +``` |
| 64 | + |
| 65 | +When running the test defined in `echo_pie.test` AcceptPy: |
| 66 | + |
| 67 | +1. Launched a shell subprocess. |
| 68 | +2. Ran the command `echo have some pie.` in the shell. |
| 69 | +3. Checked that the command's exit code was 0. |
| 70 | +4. Checked that the total of the command's printed stdout was `have some pie.\n` |
| 71 | + |
| 72 | +# Installation |
| 73 | + |
| 74 | +# Running multiple tests |
| 75 | + |
| 76 | +Running multiple tests is accomplished by authoring multiple test files. |
| 77 | + |
| 78 | +For example: |
| 79 | + |
| 80 | +``` |
| 81 | +$ echo '{ "test_type": "shell command", "command" : "echo dog" }' > echo_dog.test |
| 82 | +$ echo '{ "test_type": "shell command", "command" : "echo cat" }' > echo_cat.test |
| 83 | +$ python3 ~/accept.py ./ |
| 84 | +``` |
| 85 | + |
| 86 | +(n.b. in these minimal test files AcceptPy will only check that the echo commands terminated with exit code 0.) |
| 87 | + |
| 88 | +... prodcues the output: |
| 89 | + |
| 90 | +``` |
| 91 | +######################### |
| 92 | + Welcome to AcceptPy |
| 93 | +######################### |
| 94 | +
|
| 95 | +searching for tests... |
| 96 | +
|
| 97 | +./echo_cat.test |
| 98 | +./echo_dog.test |
| 99 | +
|
| 100 | +found 2 tests. |
| 101 | +
|
| 102 | +
|
| 103 | +######################### |
| 104 | + Test 1 of 2 |
| 105 | +######################### |
| 106 | +
|
| 107 | +[running: ./echo_cat.test] |
| 108 | +shell command: echo cat |
| 109 | +<begin stdout> |
| 110 | +cat |
| 111 | +
|
| 112 | +<end stdout> |
| 113 | +[PASSED] |
| 114 | +
|
| 115 | +
|
| 116 | +######################### |
| 117 | + Test 2 of 2 |
| 118 | +######################### |
| 119 | +
|
| 120 | +[running: ./echo_dog.test] |
| 121 | +shell command: echo dog |
| 122 | +<begin stdout> |
| 123 | +dog |
| 124 | +
|
| 125 | +<end stdout> |
| 126 | +[PASSED] |
| 127 | +
|
| 128 | +
|
| 129 | +######################### |
| 130 | + Results: |
| 131 | +######################### |
| 132 | +
|
| 133 | +
|
| 134 | +2 of 2 tests passed. |
| 135 | +``` |
| 136 | + |
| 137 | +When running AcceptPy the program will reclusively search all directories under the given test directory argument |
| 138 | +for all files ending in `.test` or `.py`. |
| 139 | + |
| 140 | +# Types of tests |
| 141 | + |
| 142 | +## Shell command tests |
| 143 | + |
| 144 | +A shell command test is defined in a JSON file with the `.test` extension. It is generally of the form: |
| 145 | + |
| 146 | +``` |
| 147 | +{ |
| 148 | + "test_type": "shell command", |
| 149 | + "command" : "echo have some pie.", |
| 150 | + "expect_exit": 0, |
| 151 | + "expect_stdout": "have some pie.\n" |
| 152 | +} |
| 153 | +``` |
| 154 | + |
| 155 | +When a shell command test file is encountered AcceptPy will spawn a shell subprocess and execute the command string |
| 156 | +identified by `command`. If `expect_exit` is specified AcceptPy will check that the command terminates with the given |
| 157 | +exit code. If not, the tool will check that the command terminated with exit code 0. If `expect_stdout` is specified |
| 158 | +AcceptPy will check that the text sent to standard out by the command matches the given text. |
| 159 | + |
| 160 | +## Python tests |
| 161 | + |
| 162 | +A Python test is defined in a `.py` file. When AcceptPy encounters such a file it will execute the Python program |
| 163 | +in a new subprocess using the command: |
| 164 | + |
| 165 | +`python3 file.py` |
| 166 | + |
| 167 | +and score the test as a pass if the returned exit code is 0. Any other exit code will result in a failed scoring. |
0 commit comments