actions/opensearch-pyd
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat: Added pool_maxsize for RequestsHttpConnection (#216)

Browse Source 
Signed-off-by: Niket Singh <[email protected]>

Signed-off-by: Niket Singh <[email protected]>
Co-authored-by: Niket Singh <[email protected]>
Co-authored-by: Harsha Vamsi Kalluri <[email protected]>
This commit is contained in:
Neckbuster
2022-12-01 11:24:59 -08:00
committed by GitHub
co-authored by Niket Singh Harsha Vamsi Kalluri
parent d6e994a411
commit b6b35f4666
3 changed files with 21 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+1
View File
@@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
## [Unreleased] ## [Unreleased]
### Added ### Added
- Added Point in time API rest API([#191](https://github.com/opensearch-project/opensearch-py/pull/191)) - Added Point in time API rest API([#191](https://github.com/opensearch-project/opensearch-py/pull/191))
- Added pool_maxsize for RequestsHttpConnection ([#216](https://github.com/opensearch-project/opensearch-py/pull/216))
- Github workflow for changelog verification ([#218](https://github.com/opensearch-project/opensearch-py/pull/218)) - Github workflow for changelog verification ([#218](https://github.com/opensearch-project/opensearch-py/pull/218))
- Added overload decorators to helpers-actions.pyi-"bulk" ([#239](https://github.com/opensearch-project/opensearch-py/pull/239)) - Added overload decorators to helpers-actions.pyi-"bulk" ([#239](https://github.com/opensearch-project/opensearch-py/pull/239))
- Document Keberos authenticaion ([214](https://github.com/opensearch-project/opensearch-py/pull/214)) - Document Keberos authenticaion ([214](https://github.com/opensearch-project/opensearch-py/pull/214))
USER_GUIDE.md
+9 -6
View File
@@ -1,6 +1,6 @@
- [Getting started with the OpenSearch Python client](#getting-started-with-the-opensearch-python-client) - [User guide of OpenSearch Python Client](#user-guide-of-opensearch-python-client)
- [Setup](#setup) - [Setup](#setup)
- [Sample code](#sample-code) - [Example](#example)
- [Creating a client](#creating-a-client) - [Creating a client](#creating-a-client)
- [Creating an index](#creating-an-index) - [Creating an index](#creating-an-index)
- [Adding a document to an index](#adding-a-document-to-an-index) - [Adding a document to an index](#adding-a-document-to-an-index)
@@ -9,8 +9,8 @@
- [Searching for a document](#searching-for-a-document) - [Searching for a document](#searching-for-a-document)
- [Deleting a document](#deleting-a-document) - [Deleting a document](#deleting-a-document)
- [Deleting an index](#deleting-an-index) - [Deleting an index](#deleting-an-index)
- [Making API calls](#making-api-calls) - [Making API Calls](#making-api-calls)
- [Point in time API](#point-in-time-api-calls) - [Point in Time API](#point-in-time-api)
- [Using plugins](#using-plugins) - [Using plugins](#using-plugins)
- [Alerting plugin](#alerting-plugin) - [Alerting plugin](#alerting-plugin)
- [**Searching for monitors**](#searching-for-monitors) - [**Searching for monitors**](#searching-for-monitors)
@@ -390,7 +390,9 @@ Refer the AWS documentation regarding usage of IAM credentials to sign requests
Opensearch-py client library also provides an in-house IAM based authentication feature, `AWSV4SignerAuth` that will help users to connect to their opensearch clusters by making use of IAM roles. Opensearch-py client library also provides an in-house IAM based authentication feature, `AWSV4SignerAuth` that will help users to connect to their opensearch clusters by making use of IAM roles.
##### Pre-requisites to use `AWSV4SignerAuth` `AWSV4SignerAuth` uses RequestHttpConnection as transport class for communication with opensearch clusters. Opensearch-py client library provides `pool_maxsize` option to modify default connection-pool size.
#### Pre-requisites to use `AWSV4SignerAuth`
- Python version 3.6 or above, - Python version 3.6 or above,
- Install [botocore](https://pypi.org/project/botocore/) using pip - Install [botocore](https://pypi.org/project/botocore/) using pip
@@ -413,7 +415,8 @@ client = OpenSearch(
http_auth = auth, http_auth = auth,
use_ssl = True, use_ssl = True,
verify_certs = True, verify_certs = True,
connection_class = RequestsHttpConnection connection_class = RequestsHttpConnection,
pool_maxsize = 20
) )
q = 'miller' q = 'miller'
opensearchpy/connection/http_requests.py
+11
View File
@@ -66,6 +66,8 @@ class RequestsHttpConnection(Connection):
:arg http_compress: Use gzip compression :arg http_compress: Use gzip compression
:arg opaque_id: Send this value in the 'X-Opaque-Id' HTTP header :arg opaque_id: Send this value in the 'X-Opaque-Id' HTTP header
For tracing all requests made by this transport. For tracing all requests made by this transport.
:arg pool_maxsize: Maximum connection pool size used by pool-manager
For custom connection-pooling on current session
""" """
def __init__( def __init__(
@@ -82,6 +84,7 @@ class RequestsHttpConnection(Connection):
headers=None, headers=None,
http_compress=None, http_compress=None,
opaque_id=None, opaque_id=None,
pool_maxsize=None,
**kwargs **kwargs
): ):
if not REQUESTS_AVAILABLE: if not REQUESTS_AVAILABLE:
@@ -94,6 +97,14 @@ class RequestsHttpConnection(Connection):
for key in list(self.session.headers): for key in list(self.session.headers):
self.session.headers.pop(key) self.session.headers.pop(key)
# Mount http-adapter with custom connection-pool size. Default=10
if pool_maxsize and isinstance(pool_maxsize, int):
pool_adapter = requests.adapters.HTTPAdapter(
pool_maxsize=pool_maxsize
)
self.session.mount("http://", pool_adapter)
self.session.mount("https://", pool_adapter)
super(RequestsHttpConnection, self).__init__( super(RequestsHttpConnection, self).__init__(
host=host, host=host,
port=port, port=port,