37 Commits

Author SHA1 Message Date
Harshil Jani 4cdf81e741 Added new test case for bulk delete with single item (#878)
Signed-off-by: Harshil Jani <harshiljani2002@gmail.com>
2024-12-29 07:57:01 -05:00
nathaliellenaa b9e48dc847 Fix AuthorizationException with AWSV4SignerAsyncAuth when the doc ID has special characters. (#848)
* Lifecycle integration tests.

Signed-off-by: dblock <dblock@amazon.com>

* Added a test that makes sure the slash is properly encoded.

Signed-off-by: dblock <dblock@amazon.com>

* Added more tests for signer and _make_path.

Signed-off-by: Nathalie Jonathan <nathhjo@amazon.com>

* Prevent AIOHttpConnection from encoding the url a second time.

Signed-off-by: Nathalie Jonathan <nathhjo@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
Signed-off-by: Nathalie Jonathan <nathhjo@amazon.com>
Co-authored-by: dblock <dblock@amazon.com>
2024-11-27 17:50:22 -05:00
Daniel (dB.) Doubrovkine 6fa54a719a Updated YAML REST tests known to fail. (#838)
Signed-off-by: dblock <dblock@amazon.com>
2024-10-24 16:38:00 -04:00
Étienne Beaulé 1b0440a33f Ignore name of index when fetching settings (#823)
This commit fixes a situation where an index cannot be updated through this client as the index is aliased, with the client pointing to the alias.

As the `GET /<index>/_settings` request will only ever return the settings for the specified index (through the alias), it would only have one key, so the name of the key would not matter. We can pop the key to get the settings object for the index through the alias.

Signed-off-by: Étienne Beaulé <beauleetienne0@gmail.com>
2024-09-26 14:02:13 -04:00
Daniel (dB.) Doubrovkine 45d817289a Update dev guide, lock aiohttp to a version that works. (#828)
* Update developer guide with test setup instructions using OpenSearch 2.12+.

Signed-off-by: dblock <dblock@amazon.com>

* Lock down aiohttp to <= 3.10.5.

Signed-off-by: dblock <dblock@amazon.com>

* Exclude failing integ test.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2024-09-25 14:47:22 -07:00
Daniel (dB.) Doubrovkine be56ae82b8 fix: retry_on_conflicts is not bool (#795)
Signed-off-by: dblock <dblock@amazon.com>
Co-authored-by: Patrick Rice <patrickrice20@gmail.com>
2024-08-15 10:34:37 -04:00
Daniel (dB.) Doubrovkine 55f9940d51 Remove HEAD-handling hack. (#794)
* remove HEAD -> GET workaround

Signed-off-by: dblock <dblock@amazon.com>

* Removed remaining HEAD-handling code.

Signed-off-by: dblock <dblock@amazon.com>

* Fixed remaining references to admin:admin.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
Co-authored-by: Vincent Castaneda <vincent.castaneda@sage.com>
2024-08-15 10:15:13 -04:00
Hugo van Kemenade 6e3f1a1194 Remove redundant mock backport dependency and upgrade syntax for Python 3.8+ (#785)
* Upgrade syntax with pyupgrade --py38-plus

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

* Convert to f-strings with flynt

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

* Format with Black

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

* Remove redundant mock backport dependency

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

* isort imports

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

* Add changelog entry

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>

---------

Signed-off-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-07-20 16:19:20 -04:00
Daniel (dB.) Doubrovkine 1ef788a55a Skip elements with titles. (#777)
* Skip elements with titles.

Signed-off-by: dblock <dblock@amazon.com>

* Simplify and just skip by ref.

Signed-off-by: dblock <dblock@amazon.com>

* Re-generate API spec.

Signed-off-by: dblock <dblock@amazon.com>

* Do not skip alias tests.

Signed-off-by: dblock <dblock@amazon.com>

* Actually skip node_id_or_metric.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2024-07-18 13:24:38 -07:00
Daniel (dB.) Doubrovkine 04b870d68a Skip put_aliases tests that don't require both index and alias parameters. (#778)
Signed-off-by: dblock <dblock@amazon.com>
2024-07-18 08:44:41 -07:00
AbitraryYu 0caacf870c Added Tests for Search Pipeline and Notifications Plugin (#668)
* Added Tests for Notification and Search Pipeline.ml

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>

* Add back the lines of comment in indices.py during resolving merge conflict.

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>

* Fix notification plugin tests, from async def to def.

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>

* Fix test errors. Rename function names and rewrite assertion tests.

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>

* Fix formatting errors. Added typings to CONTENT.

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>

---------

Signed-off-by: AbitraryYu <nikkoyhc@gmail.com>
2024-04-30 16:51:22 -04:00
Sai Medhini Reddy Maryada fd48eb450a added tests (#726)
Signed-off-by: saimedhi <saimedhi@amazon.com>
2024-04-23 15:47:15 -04:00
Sai Medhini Reddy Maryada 0e28f5ffe9 Updated code generator to use new version of OpenAPI specification (#721)
* Updated code generator to use new version of OpenAPI specification

Signed-off-by: saimedhi <saimedhi@amazon.com>

* Updated code generator to use native OpenAPI specification

Signed-off-by: saimedhi <saimedhi@amazon.com>

---------

Signed-off-by: saimedhi <saimedhi@amazon.com>
2024-04-17 17:22:10 -04:00
Sai Medhini Reddy Maryada 58e9b1d66c Introduced service time metrics to OpenSearch-Py client. (#716)
* Introduced service time metrics to opensearch-py client

Signed-off-by: saimedhi <saimedhi@amazon.com>

* Introduced service time metrics to opensearch-py client

Signed-off-by: saimedhi <saimedhi@amazon.com>

* Introduced service time metrics to opensearch-py client

Signed-off-by: saimedhi <saimedhi@amazon.com>

* Added service time metrics

Signed-off-by: saimedhi <saimedhi@amazon.com>

---------

Signed-off-by: saimedhi <saimedhi@amazon.com>
Signed-off-by: Sai Medhini Reddy Maryada <117196660+saimedhi@users.noreply.github.com>
2024-04-15 07:20:46 -04:00
Sai Medhini Reddy Maryada b2a1796b8e Fixed yaml test runner (#696)
Signed-off-by: saimedhi <saimedhi@amazon.com>
2024-03-16 07:58:10 -04:00
Sai Medhini Reddy Maryada 94e33c6899 skipping failing integ test that is newly added (#652)
* skipping a newly added test

Signed-off-by: saimedhi <saimedhi@amazon.com>

* skipping a newly added test

Signed-off-by: saimedhi <saimedhi@amazon.com>

---------

Signed-off-by: saimedhi <saimedhi@amazon.com>
2024-02-02 09:34:31 -08:00
Mark Cohen a80bab2ad5 Assignment from no return (#658)
* added unnecessary-dunder-call to pylintrc files; disabled for certain lines

in run_tests.py, exception thrown by 'git remote add origin' when the remote already exists will not exit

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updates to adhere to assignment-from-no-return lint

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* simplified get_value_filter in Facet to return None
added assert to test get_value_filter returning None

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* added option to output HTML test coverage locally from run_tests.py

returning None from test_faceted_search.Facet.get_value_filter

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* added unused-variable lints; replaced unused variables with _ or referenced them

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated CHANGELOG to point to the right PR

Signed-off-by: Mark Cohen <markcoh@amazon.com>

---------

Signed-off-by: Mark Cohen <markcoh@amazon.com>
2024-01-25 15:17:09 -08:00
Mark Cohen 6e58837993 added unnecessary-dunder-call to pylintrc files; disabled for certain lines (#655)
in run_tests.py, exception thrown by 'git remote add origin' when the remote already exists will not exit

Signed-off-by: Mark Cohen <markcoh@amazon.com>
2024-01-22 09:12:57 -05:00
Mark Cohen 0ddbf8cafa Pylint integration updates (#643)
* updated files with docstrings to pass pylint

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated samples to prepare for enabling missing-docstring linter; will continue to work on this before committing setup.cfg

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* removed missing-function-docstring from setup.cfg so the linter doesn't fail while work on docstrings continues

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* corrected unnecessary return docstring values

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* fixing failure in 'black' on reformatting

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated utils to pass missing-function-docstring tests

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated functions with missing docstrings or pylint ignore instructions; added a utility to automatically add these ignore instructions to most functions that should be self-describing; rolled back some automatically generated code mistakenly changed

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* * ignoring opensearchpy for pylint and then added it back to noxfile.py
* fixed some lints; created a feature flag for newer dynamic pylint so now lints can be fixed first in legacy code and then enabled by multiple people
* extracted a method for per-folder linting
* updated noxfile.lint_per_folder with type hints
* enabled unspecified-encoding in pylint
* added disable missing-function-docstring pragma to test_clients.py in test_async and test_server
* added more encodings to pass unspecified-encoding pylint tests
* updated changelog
Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated CHANGELOG.md entry
removed the feature flag for pylint lint_per_folder
fixed failures from mypy and pylint
removed pylint MESSAGE CONTROL config from setup.cfg after relocating to lint_per_folder method
Signed-off-by: Mark Cohen <markcoh@amazon.com>

* removed pylint ignore missing-function-docstring

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* added pylint.extensions.docparams plugin

updated some docstrings to correct parameters

removed pylint from setup.cfg

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* added four lints for opensearchpy/

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* adding await back to client.info() call

Signed-off-by: Mark Cohen <markcoh@amazon.com>

* updated TODOs as requested

renamed test_opensearchpy.test_async.test_server.test_helpers.conftest.setup_ubq_tests to setup_update_by_query_tests

added
OpenSearch-main/rest-api-spec/src/main/resources/rest-api-spec/test/indices/stats/50_noop_update[0]
to skip tests list

run_tests.py catches a CalledProcessError when the git repo already exists and the command to add the origin fails in fetch_opensearch_repo()

Signed-off-by: Mark Cohen <markcoh@amazon.com>

---------

Signed-off-by: Mark Cohen <markcoh@amazon.com>
2024-01-19 13:36:05 -05:00
odelmarcelle 3eba72c320 Reusable async client (#639)
* set aiohttp.ClientSession to None after close()

Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* add test

Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* update changelog

Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* update changelog + format
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* update changelog
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* add tests using `with` and synchronous client
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* fix `urllib3.exceptions.ClosedPoolError` breaking synchronous client after `close`
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* update changelog
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* separate tests
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

* refactor pool factory as lambda
Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>

---------

Signed-off-by: odlmarce <delmarcelle.olivier@gmail.com>
2024-01-02 13:33:24 -05:00
Samuel Orji 6f26eb3e8e remove unnecessary utf-8 header in .py files (#615)
* remove unnecessary utf-8 header in .py files

Signed-off-by: samuel orji <awesomeorji@gmail.com>

* review feedback: add link to changelog

Signed-off-by: samuel orji <awesomeorji@gmail.com>

---------

Signed-off-by: samuel orji <awesomeorji@gmail.com>
2023-11-24 16:19:50 -05:00
Daniel (dB.) Doubrovkine 1801ada270 Added pylint, enforce naming. (#590)
* Added pylint.

Signed-off-by: dblock <dblock@amazon.com>

* Enforce pylint:invalid-name.

Signed-off-by: dblock <dblock@amazon.com>

* Updated the generated code header to prevent broken links.

Signed-off-by: dblock <dblock@amazon.com>

* Swapped order of messages.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-11-21 10:04:39 -08:00
Daniel (dB.) Doubrovkine e079c00b48 Remove support for local client in testing. (#573)
Signed-off-by: dblock <dblock@amazon.com>
2023-11-12 08:35:06 -08:00
Daniel (dB.) Doubrovkine 56c96d7c4f Expanded type coverage to benchmarks, samples and tests. (#566)
* Renamed json samples to fix duplicate module name.

Signed-off-by: dblock <dblock@amazon.com>

* Enabled mypy on all source files.

Signed-off-by: dblock <dblock@amazon.com>

* Added missing types.

Signed-off-by: dblock <dblock@amazon.com>

* Added CHANGELOG.

Signed-off-by: dblock <dblock@amazon.com>

* Move type: ignore to fix untyped decorator makes function untyped.

Signed-off-by: dblock <dblock@amazon.com>

* Fix nox -rs lint-3.7.

Signed-off-by: dblock <dblock@amazon.com>

* Fixed incorrect import.

Signed-off-by: dblock <dblock@amazon.com>

* Fix broken test.

Signed-off-by: dblock <dblock@amazon.com>

* Fixed TestBulk::test_bulk_works_with_bytestring_body.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-11-09 10:51:20 -05:00
Daniel (dB.) Doubrovkine dcb79cc322 Merge .pyi type stubs inline (#563)
* Merged types into .py code.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: nox -rs generate.

Signed-off-by: dblock <dblock@amazon.com>

* Updated CHANGELOG.

Signed-off-by: dblock <dblock@amazon.com>

* Use lowest common python version for lint.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: don't typeshed.

Signed-off-by: dblock <dblock@amazon.com>

* Removed unneeded comment.

Signed-off-by: dblock <dblock@amazon.com>

* Simplify OPENSEARCH_URL.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: positional ignore_status used as chunk_size.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: parse version string.

Signed-off-by: dblock <dblock@amazon.com>

* Remove future annotations for Python 3.6.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: types in documentation.

Signed-off-by: dblock <dblock@amazon.com>

* Improve CHANGELOG text.

Signed-off-by: dblock <dblock@amazon.com>

* Re-added missing separator.

Signed-off-by: dblock <dblock@amazon.com>

* Remove duplicate licenses.

Signed-off-by: dblock <dblock@amazon.com>

* Get rid of Optional[Any].

Signed-off-by: dblock <dblock@amazon.com>

* Fix docs with AsyncOpenSearch.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: undo comment.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-11-06 10:08:19 -08:00
Daniel (dB.) Doubrovkine 0da60b2623 Added a utf-8 header to all .py files (#557)
Signed-off-by: dblock <dblock@amazon.com>
2023-10-26 13:34:34 -04:00
Daniel (dB.) Doubrovkine 627e717534 Remove support for Python 2.x. (#548)
Signed-off-by: dblock <dblock@amazon.com>
2023-10-24 07:57:21 -07:00
Daniel (dB.) Doubrovkine 2bfc409177 Fix: typos. (#526)
* Fix: typo.

Signed-off-by: dblock <dblock@amazon.com>

* Fix: typo.

Signed-off-by: dblock <dblock@amazon.com>

* Fixed its.

Signed-off-by: dblock <dblock@amazon.com>

* Added Visual Code settings to .gitignore.

Signed-off-by: dblock <dblock@amazon.com>

* Added loop type for async client.

Signed-off-by: dblock <dblock@amazon.com>

---------

Signed-off-by: dblock <dblock@amazon.com>
2023-10-09 10:45:18 -07:00
florianvazelle 2e8d5ce1a5 Index State Management support (#398)
* feat(plugins): add index management client plugin

Signed-off-by: florian <florian@harfanglab.fr>

* chore(CHANGELOG): added entry for ISM api support in changelog

Signed-off-by: florian <florian@harfanglab.fr>

* test(plugins): use assertEqual to compare call parameters

Signed-off-by: florian <florian@harfanglab.fr>

* test(plugins): edit policy to support older versions of opensearch

Signed-off-by: florian <florian@harfanglab.fr>

* test(plugins): ignore plugin tests when opensearch is unreleased

Signed-off-by: florian <florian@harfanglab.fr>

* test(plugins): move plugin tests into separate files

Signed-off-by: florian <florian@harfanglab.fr>

* test(plugins): fix import of OpenSearchTestCase

Signed-off-by: florian <florian@harfanglab.fr>

* chore(USER_GUIDE): add a index management plugin part

Signed-off-by: florian <florian@harfanglab.fr>

---------

Signed-off-by: florian <florian@harfanglab.fr>
2023-06-28 16:56:00 -04:00
Sai Medhini Reddy Maryada e5789c7b52 Enhanced the Yaml test runner to utilize the rest-api-spec YAML tests from OpenSearch repo as the input source (#414)
Signed-off-by: saimedhi <saimedhi@amazon.com>
2023-06-26 19:07:06 -04:00
Sai Medhini Reddy Maryada c58375aa7b Merging opensearch-dsl-py to opensearch-py (#287)
Signed-off-by: saimedhi <saimedhi@amazon.com>
2023-02-14 15:03:56 -08:00
Arnav Das 2d03dc773b Alerting Plugins (#93)
* Alert Plugin

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* lint file header license

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Tests For Alerting[Monitors]

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Tests For Alerting[Destinations]

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* dynamic lookup 90#issuecomment-1003396742

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Alerting Async

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Lint Changes

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* alerting plugins integration tests

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Integ UnitTests Skip on OPENSEARCH_VERSION >= 2.0.0

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Unit Test Skip Complete

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* License Headers fixed

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* test_urllib3_connection timeout

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* lint

Signed-off-by: Arnav Das <arnav.das88@gmail.com>

* Fix licenses in all files

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

* Remove elastic search licenses on new files

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

* Fix formatting

Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

Signed-off-by: Arnav Das <arnav.das88@gmail.com>
Signed-off-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
Co-authored-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>
2022-10-03 14:45:18 -04:00
Vacha Shah 3944cc2928 Re-enable integration tests across multiple versions of OpenSearch (#163)
* Re-enable integration tests for 1.x versions of OpenSearch

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Fixing integ tests after re-enabling 1.x versions

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Separating out versions

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-05-03 09:27:25 -07:00
Vacha Shah 1569c05f08 Remove include_type_name parameter from APIs (#156)
* Remove include_type_name parameter from APIs

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Applying formatting

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-04-20 13:58:13 -07:00
Vacha Shah b1eaba0064 Type removal breaking changes (#151)
* Breaking changes for type removal

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Adding support to test against unreleased OpenSearch

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Formatting

Signed-off-by: Vacha Shah <vachshah@amazon.com>

* Addressing comments to refactor run-opensearch.sh

Signed-off-by: Vacha Shah <vachshah@amazon.com>
2022-04-13 10:47:48 -07:00
Shivam Dhar 25ff092c8f removes cert usage from CI module
Signed-off-by: Shivam Dhar <dhshivam@amazon.com>
2022-01-08 15:15:15 +05:30
Rushi Agrawal ef0c23c0e4 Rename module to opensearchpy
To avoid conflict with an existing package by name 'opensearch' being
present

Signed-off-by: Rushi Agrawal <rushi.agr@gmail.com>
2021-09-16 21:23:38 +05:30