Commit e4e9941
authored
fix: Fix benchmark results display for paswordless AWS OpenSearch benchmarks (#721)
Fixes #716
Description:
When running a benchmark against AWS OpenSearch, username and password are optional and can we left empty so benchmark can use AWS authentiction tokens in environment variables. This works perfectly, and benchamrk runs and does the expected inserts and queries on the AWS OS cluster. But when trying to look at the benchmark results, http://localhost:8502/results displays this error:
```
pydantic.error_wrappers.ValidationError: 2 validation errors for AWSOpenSearchConfig user Empty string! (type=value_error) password Empty string!
(type=value_error)
File "/Users/jvegas/.cache/uv/archive-v0/DcT73OzatkvN3UXwJBtH_/lib/python3.12/site-packages/vectordb_bench/frontend/pages/results.py", line 62, in <module>
main()
File "/Users/jvegas/.cache/uv/archive-v0/DcT73OzatkvN3UXwJBtH_/lib/python3.12/site-packages/vectordb_bench/frontend/pages/results.py", line 29, in main
allResults = benchmark_runner.get_results()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jvegas/.cache/uv/archive-v0/DcT73OzatkvN3UXwJBtH_/lib/python3.12/site-packages/vectordb_bench/interface.py", line 106, in get_results
return ResultCollector.collect(target_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jvegas/.cache/uv/archive-v0/DcT73OzatkvN3UXwJBtH_/lib/python3.12/site-packages/vectordb_bench/backend/result_collector.py", line 18, in
collect
file_result = TestResult.read_file(json_file, trans_unit=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jvegas/.cache/uv/archive-v0/DcT73OzatkvN3UXwJBtH_/lib/python3.12/site-packages/vectordb_bench/models.py", line 331, in read_file
task_config["db_config"] = db.config_cls(**task_config["db_config"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pydantic/main.py", line 364, in pydantic.main.BaseModel.__init__
```
Looks like during results loading the system is applying a more strict validation that does not allow empty username or password, which was allowed at test creation. This PR fixes validation to allow for empty username or password in both cases, resulting in viewables results for passwordless AWS OS benchmarks1 parent e89c445 commit e4e9941
3 files changed
Lines changed: 87 additions & 7 deletions
File tree
- tests
- vectordb_bench/backend/clients/aws_opensearch
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
20 | 25 | | |
21 | 26 | | |
22 | 27 | | |
23 | | - | |
24 | | - | |
| 28 | + | |
| 29 | + | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| |||
188 | 193 | | |
189 | 194 | | |
190 | 195 | | |
191 | | - | |
| 196 | + | |
192 | 197 | | |
193 | 198 | | |
194 | 199 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
15 | | - | |
| 14 | + | |
| 15 | + | |
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
21 | 23 | | |
22 | 24 | | |
23 | 25 | | |
| |||
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
33 | 47 | | |
34 | 48 | | |
35 | 49 | | |
| |||
0 commit comments