opensearch-pyd/GETTING_STARTED.md

• Getting Started with the OpenSearch Python Client
  • Setup
  • Sample code
  • Using IAM credentials for authentication
    • Pre-requisites to use AWSV4SignerAuth

Getting Started with the OpenSearch Python Client

Setup

To add the client to your project, install it using pip:

pip install opensearch-py

Then import it like any other module:

from opensearchpy import OpenSearch

If you prefer to add the client manually or just want to examine the source code, see opensearch-py on GitHub.

Sample code

Creating a client

from opensearchpy import OpenSearch

host = 'localhost'
port = 9200
auth = ('admin', 'admin') # For testing only. Don't store credentials in code.
ca_certs_path = '/full/path/to/root-ca.pem' # Provide a CA bundle if you use intermediate CAs with your root CA.

# Optional client certificates if you don't want to use HTTP basic authentication.
# client_cert_path = '/full/path/to/client.pem'
# client_key_path = '/full/path/to/client-key.pem'

# Create the client with SSL/TLS enabled, but hostname verification disabled.
client = OpenSearch(
    hosts = [{'host': host, 'port': port}],
    http_compress = True, # enables gzip compression for request bodies
    http_auth = auth,
    # client_cert = client_cert_path,
    # client_key = client_key_path,
    use_ssl = True,
    verify_certs = True,
    ssl_assert_hostname = False,
    ssl_show_warn = False,
    ca_certs = ca_certs_path
)

Creating an index

# Create an index with non-default settings.
index_name = 'python-test-index3'
index_body = {
  'settings': {
    'index': {
      'number_of_shards': 4
    }
  }
}

response = client.indices.create(index_name, body=index_body)
print('\nCreating index:')
print(response)

Adding a document to an index

document = {
  'title': 'Moneyball',
  'director': 'Bennett Miller',
  'year': '2011'
}
id = '1'

response = client.index(
    index = index_name,
    body = document,
    id = id,
    refresh = True
)

print('\nAdding document:')
print(response)

Adding documents in bulk

docs = '{"index": {"_index": "index-2022-06-08", "_id": "1"}} \n 
{"name": "foo"} \n 
{"index": {"_index": "index-2022-06-09", "_id": "2"}} \n 
{"name": "bar"} \n 
{"index": {"_index": "index-2022-06-10", "_id": "3"}} \n 
{"name": "baz"}'

response = client.bulk(docs)

print('\nAdding bulk documents:')
print(response)

Adding documents in bulk using helper functions

docs = []
def generate_data():
    mywords = ['foo', 'bar', 'baz']
    for index, word in enumerate(mywords):
        docs.append({
            "_index": "mywords",
            "word": word,
            "_id": index
        })
    return docs

response = helpers.bulk(client, generate_data(), max_retries=3)

print('\nAdding bulk documents using helper:')
print(response)

Searching for a document

q = 'miller'
query = {
  'size': 5,
  'query': {
    'multi_match': {
      'query': q,
      'fields': ['title^2', 'director']
    }
  }
}

response = client.search(
    body = query,
    index = index_name
)
print('\nSearch results:')
print(response)

Deleting a document

response = client.delete(
    index = index_name,
    id = id
)

print('\nDeleting document:')
print(response)

Deleting an index

response = client.indices.delete(
    index = index_name
)

print('\nDeleting index:')
print(response)

Using IAM credentials for authentication

Refer the AWS documentation regarding usage of IAM credentials to sign requests to OpenSearch APIs - Signing HTTP requests to Amazon OpenSearch Service.

Opensearch-py client library also provides an in-house IAM based authentication feature, AWSV4SignerAuth that will help users to connect to their opensearch clusters by making use of IAM roles.

Pre-requisites to use AWSV4SignerAuth

• Python version 3.6 or above,

• Install botocore using pip

  pip install botocore

Here is the sample code that uses AWSV4SignerAuth -

from opensearchpy import OpenSearch, RequestsHttpConnection, AWSV4SignerAuth
import boto3

host = '' # cluster endpoint, for example: my-test-domain.us-east-1.es.amazonaws.com
region = 'us-west-2'
credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, region)
index_name = 'python-test-index3'

client = OpenSearch(
    hosts = [{'host': host, 'port': 443}],
    http_auth = auth,
    use_ssl = True,
    verify_certs = True,
    connection_class = RequestsHttpConnection
)

q = 'miller'
query = {
  'size': 5,
  'query': {
    'multi_match': {
      'query': q,
      'fields': ['title^2', 'director']
    }
  }
}

response = client.search(
    body = query,
    index = index_name
)

print('\nSearch results:')
print(response)