Pylint integration updates (#643)

* updated files with docstrings to pass pylint

Signed-off-by: Mark Cohen <[email protected]>

* updated samples to prepare for enabling missing-docstring linter; will continue to work on this before committing setup.cfg

Signed-off-by: Mark Cohen <[email protected]>

* removed missing-function-docstring from setup.cfg so the linter doesn't fail while work on docstrings continues

Signed-off-by: Mark Cohen <[email protected]>

* corrected unnecessary return docstring values

Signed-off-by: Mark Cohen <[email protected]>

* fixing failure in 'black' on reformatting

Signed-off-by: Mark Cohen <[email protected]>

* updated utils to pass missing-function-docstring tests

Signed-off-by: Mark Cohen <[email protected]>

* updated functions with missing docstrings or pylint ignore instructions; added a utility to automatically add these ignore instructions to most functions that should be self-describing; rolled back some automatically generated code mistakenly changed

Signed-off-by: Mark Cohen <[email protected]>

* * ignoring opensearchpy for pylint and then added it back to noxfile.py
* fixed some lints; created a feature flag for newer dynamic pylint so now lints can be fixed first in legacy code and then enabled by multiple people
* extracted a method for per-folder linting
* updated noxfile.lint_per_folder with type hints
* enabled unspecified-encoding in pylint
* added disable missing-function-docstring pragma to test_clients.py in test_async and test_server
* added more encodings to pass unspecified-encoding pylint tests
* updated changelog
Signed-off-by: Mark Cohen <[email protected]>

* updated CHANGELOG.md entry
removed the feature flag for pylint lint_per_folder
fixed failures from mypy and pylint
removed pylint MESSAGE CONTROL config from setup.cfg after relocating to lint_per_folder method
Signed-off-by: Mark Cohen <[email protected]>

* removed pylint ignore missing-function-docstring

Signed-off-by: Mark Cohen <[email protected]>

* added pylint.extensions.docparams plugin

updated some docstrings to correct parameters

removed pylint from setup.cfg

Signed-off-by: Mark Cohen <[email protected]>

* added four lints for opensearchpy/

Signed-off-by: Mark Cohen <[email protected]>

* adding await back to client.info() call

Signed-off-by: Mark Cohen <[email protected]>

* updated TODOs as requested

renamed test_opensearchpy.test_async.test_server.test_helpers.conftest.setup_ubq_tests to setup_update_by_query_tests

added
OpenSearch-main/rest-api-spec/src/main/resources/rest-api-spec/test/indices/stats/50_noop_update[0]
to skip tests list

run_tests.py catches a CalledProcessError when the git repo already exists and the command to add the origin fails in fetch_opensearch_repo()

Signed-off-by: Mark Cohen <[email protected]>

---------

Signed-off-by: Mark Cohen <[email protected]>

utils/license_headers.py

@@ -47,11 +47,16 @@ def find_files_to_fix(sources: List[str]) -> Iterator[str]:
 
 
 def does_file_need_fix(filepath: str) -> bool:
+    """
+    checks if the correct license header exists at the top of the file
+    :param filepath: an absolute or relative filepath to a file to check
+    :return: True if the file needs a header, False if it does not
+    """
     if not re.search(r"\.py$", filepath):
         return False
     existing_header = ""
-    with open(filepath, mode="r") as f:
-        for line in f:
+    with open(filepath, mode="r", encoding="utf-8") as file:
+        for line in file:
             line = line.strip()
             if len(line) == 0 or line in LINES_TO_KEEP:
                 pass
@@ -64,20 +69,30 @@ def does_file_need_fix(filepath: str) -> bool:
 
 
 def add_header_to_file(filepath: str) -> None:
-    with open(filepath, mode="r") as f:
-        lines = list(f)
+    """
+    writes the license header to the beginning of a file
+    :param filepath: relative or absolute filepath to update
+    """
+    with open(filepath, mode="r", encoding="utf-8") as file:
+        lines = list(file)
     i = 0
     for i, line in enumerate(lines):
         if len(line) > 0 and line not in LINES_TO_KEEP:
             break
     lines = lines[:i] + [LICENSE_HEADER] + lines[i:]
-    with open(filepath, mode="w") as f:
-        f.truncate()
-        f.write("".join(lines))
+    with open(filepath, mode="w", encoding="utf-8") as file:
+        file.truncate()
+        file.write("".join(lines))
     print(f"Fixed {os.path.relpath(filepath, os.getcwd())}")
 
 
 def main() -> None:
+    """
+    arguments:
+        fix: find all files without license headers and insert headers at the top of the file
+        check: prints a list of files without license headers
+        list of one or more directories: search in these directories
+    """
     mode = sys.argv[1]
     assert mode in ("fix", "check")
     sources = [os.path.abspath(x) for x in sys.argv[2:]]