Daniel (dB.) Doubrovkine
af263a060a
* Align pool_maxsize for different connection pool implementations. Signed-off-by: dblock <dblock@amazon.com> * Added benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Multi-threaded vs. async benchmarks. Signed-off-by: dblock <dblock@amazon.com> * Set pool size to the number of threads. Signed-off-by: dblock <dblock@amazon.com> * Added sync/async benchmark. Signed-off-by: dblock <dblock@amazon.com> * Report client-side latency. Signed-off-by: dblock <dblock@amazon.com> * Various updates to benchmarks, demonstrating threading improves throughput. Signed-off-by: dblock <dblock@amazon.com> * Bench info. Signed-off-by: dblock <dblock@amazon.com> * Fixup format. Signed-off-by: dblock <dblock@amazon.com> * Undo async maxsize. Signed-off-by: dblock <dblock@amazon.com> * Moved benchmarks folder. Signed-off-by: dblock <dblock@amazon.com> * Updated documentation and project description. Signed-off-by: dblock <dblock@amazon.com> --------- Signed-off-by: dblock <dblock@amazon.com>
64 lines
4.3 KiB
Markdown
64 lines
4.3 KiB
Markdown
- [Benchmarks](#benchmarks)
|
|
- [Start OpenSearch](#start-opensearch)
|
|
- [Install Prerequisites](#install-prerequisites)
|
|
- [Run Benchmarks](#run-benchmarks)
|
|
|
|
## Benchmarks
|
|
|
|
Python client benchmarks using [richbench](https://github.com/tonybaloney/rich-bench).
|
|
|
|
### Start OpenSearch
|
|
|
|
```
|
|
docker run -p 9200:9200 -e "discovery.type=single-node" opensearchproject/opensearch:latest
|
|
```
|
|
|
|
### Install Prerequisites
|
|
|
|
Install [poetry](https://python-poetry.org/docs/), then install package dependencies.
|
|
|
|
```
|
|
poetry install
|
|
```
|
|
|
|
Benchmarks use the code in this repository by specifying the dependency as `opensearch-py = { path = "..", develop=true, extras=["async"] }` in [pyproject.toml](pyproject.toml).
|
|
|
|
### Run Benchmarks
|
|
|
|
Run all benchmarks available as follows.
|
|
|
|
```
|
|
poetry run richbench . --repeat 1 --times 1
|
|
```
|
|
|
|
Outputs results from all the runs.
|
|
|
|
```
|
|
Benchmarks, repeat=1, number=1
|
|
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
|
|
┃ Benchmark ┃ Min ┃ Max ┃ Mean ┃ Min (+) ┃ Max (+) ┃ Mean (+) ┃
|
|
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
|
|
│ 1 client vs. more clients (async) │ 1.640 │ 1.640 │ 1.640 │ 1.102 (1.5x) │ 1.102 (1.5x) │ 1.102 (1.5x) │
|
|
│ 1 thread vs. 32 threads (sync) │ 5.526 │ 5.526 │ 5.526 │ 1.626 (3.4x) │ 1.626 (3.4x) │ 1.626 (3.4x) │
|
|
│ 1 thread vs. 32 threads (sync) │ 4.639 │ 4.639 │ 4.639 │ 3.363 (1.4x) │ 3.363 (1.4x) │ 3.363 (1.4x) │
|
|
│ sync vs. async (8) │ 3.198 │ 3.198 │ 3.198 │ 0.966 (3.3x) │ 0.966 (3.3x) │ 0.966 (3.3x) │
|
|
└───────────────────────────────────┴─────────┴─────────┴─────────┴─────────────────┴─────────────────┴─────────────────┘
|
|
```
|
|
|
|
Run a specific benchmark, e.g. [bench_sync.py](bench_sync.py) by specifying `--benchmark [name]`.
|
|
|
|
```
|
|
poetry run richbench . --repeat 1 --times 1 --benchmark sync
|
|
```
|
|
|
|
Outputs results from one benchmark.
|
|
|
|
```
|
|
Benchmarks, repeat=1, number=1
|
|
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
|
|
┃ Benchmark ┃ Min ┃ Max ┃ Mean ┃ Min (+) ┃ Max (+) ┃ Mean (+) ┃
|
|
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
|
|
│ 1 thread vs. 32 threads (sync) │ 6.804 │ 6.804 │ 6.804 │ 3.409 (2.0x) │ 3.409 (2.0x) │ 3.409 (2.0x) │
|
|
└────────────────────────────────┴─────────┴─────────┴─────────┴─────────────────┴─────────────────┴─────────────────┘
|
|
```
|