Pylint integration updates (#643)

* updated files with docstrings to pass pylint

Signed-off-by: Mark Cohen <[email protected]>

* updated samples to prepare for enabling missing-docstring linter; will continue to work on this before committing setup.cfg

Signed-off-by: Mark Cohen <[email protected]>

* removed missing-function-docstring from setup.cfg so the linter doesn't fail while work on docstrings continues

Signed-off-by: Mark Cohen <[email protected]>

* corrected unnecessary return docstring values

Signed-off-by: Mark Cohen <[email protected]>

* fixing failure in 'black' on reformatting

Signed-off-by: Mark Cohen <[email protected]>

* updated utils to pass missing-function-docstring tests

Signed-off-by: Mark Cohen <[email protected]>

* updated functions with missing docstrings or pylint ignore instructions; added a utility to automatically add these ignore instructions to most functions that should be self-describing; rolled back some automatically generated code mistakenly changed

Signed-off-by: Mark Cohen <[email protected]>

* * ignoring opensearchpy for pylint and then added it back to noxfile.py
* fixed some lints; created a feature flag for newer dynamic pylint so now lints can be fixed first in legacy code and then enabled by multiple people
* extracted a method for per-folder linting
* updated noxfile.lint_per_folder with type hints
* enabled unspecified-encoding in pylint
* added disable missing-function-docstring pragma to test_clients.py in test_async and test_server
* added more encodings to pass unspecified-encoding pylint tests
* updated changelog
Signed-off-by: Mark Cohen <[email protected]>

* updated CHANGELOG.md entry
removed the feature flag for pylint lint_per_folder
fixed failures from mypy and pylint
removed pylint MESSAGE CONTROL config from setup.cfg after relocating to lint_per_folder method
Signed-off-by: Mark Cohen <[email protected]>

* removed pylint ignore missing-function-docstring

Signed-off-by: Mark Cohen <[email protected]>

* added pylint.extensions.docparams plugin

updated some docstrings to correct parameters

removed pylint from setup.cfg

Signed-off-by: Mark Cohen <[email protected]>

* added four lints for opensearchpy/

Signed-off-by: Mark Cohen <[email protected]>

* adding await back to client.info() call

Signed-off-by: Mark Cohen <[email protected]>

* updated TODOs as requested

renamed test_opensearchpy.test_async.test_server.test_helpers.conftest.setup_ubq_tests to setup_update_by_query_tests

added
OpenSearch-main/rest-api-spec/src/main/resources/rest-api-spec/test/indices/stats/50_noop_update[0]
to skip tests list

run_tests.py catches a CalledProcessError when the git repo already exists and the command to add the origin fails in fetch_opensearch_repo()

Signed-off-by: Mark Cohen <[email protected]>

---------

Signed-off-by: Mark Cohen <[email protected]>

test_opensearchpy/run_tests.py

@@ -37,10 +37,16 @@ import subprocess
 import sys
 from os import environ
 from os.path import abspath, dirname, exists, join, pardir
+from subprocess import CalledProcessError
 from typing import Any
 
 
 def fetch_opensearch_repo() -> None:
+    """
+    runs a git fetch origin on configured opensearch core repo
+    :return: None if environmental variables TEST_OPENSEARCH_YAML_DIR
+    is set or TEST_OPENSEARCH_NOFETCH is set to False; else returns nothing
+    """
     # user is manually setting YAML dir, don't tamper with it
     if "TEST_OPENSEARCH_YAML_DIR" in environ:
         return
@@ -77,12 +83,20 @@ def fetch_opensearch_repo() -> None:
     # make a new blank repository in the test directory
     subprocess.check_call("cd %s && git init" % repo_path, shell=True)
 
-    # add a remote
-    subprocess.check_call(
-        "cd %s && git remote add origin https://github.com/opensearch-project/opensearch.git"
-        % repo_path,
-        shell=True,
-    )
+    try:
+        # add a remote
+        subprocess.check_call(
+            "cd %s && git remote add origin https://github.com/opensearch-project/opensearch.git"
+            % repo_path,
+            shell=True,
+        )
+    except CalledProcessError as e:
+        # if the run is interrupted from a previous run, it doesn't clean up, and the git add origin command
+        # errors out; this allows the test to continue
+        remote_origin_already_exists = 3
+        print(e)
+        if e.returncode != remote_origin_already_exists:
+            sys.exit(1)
 
     # fetch the sha commit, version from info()
     print("Fetching opensearch repo...")
@@ -90,6 +104,17 @@ def fetch_opensearch_repo() -> None:
 
 
 def run_all(argv: Any = None) -> None:
+    """
+    run all the tests given arguments and environment variables
+    - sets defaults if argv is None, running "pytest --cov=opensearchpy
+    --junitxml=<path to opensearch-py-junit.xml>
+    --log-level=DEBUG --cache-clear -vv --cov-report=<path to output code coverage"
+    * GITHUB_ACTION: fetches yaml tests if this is not in environment variables
+    * TEST_PATTERN: specify a test to run
+    * TEST_TYPE: "server" runs on TLS connection; None is unencrypted
+    * OPENSEARCH_VERSION: "SNAPSHOT" does not do anything with plugins
+    :param argv: if this is None, then the default arguments
+    """
     sys.exitfunc = lambda: sys.stderr.write("Shutting down....\n")  # type: ignore
     # fetch yaml tests anywhere that's not GitHub Actions
     if "GITHUB_ACTION" not in environ: