actions/opensearch-pyd
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity
1,082 Commits 3 Branches 0 Tags
07fef33accc28e057c9edf2cd730c72d3a835b21
Commit Graph
68 Commits
Author SHA1 Message Date
Nick LangandGitHub 206f5e2754 Black (#948) 2019-05-10 09:16:33 -06:00
Asher MancinelliandNick Lang 85573db275 Restructuring helpers dir (#886) 
* PEP8 formatting & package structuring

* adding internal function imports into helpers init

* changed absolute paths for mock

* removed debugging print statements

* resolving @fxdgear 's comment
 2019-01-14 11:27:02 -08:00
Nick LangandGitHub f3f7266c1f better logic to handling unicode and chunking. (#870) 
Chunking was looking at the length of a string but not accounting for the
byte length. closes #716
 2018-11-26 10:58:47 -07:00
ItayandNick Lang de8413fdbd typo fix in helpers (#854) 2018-10-11 03:05:11 -06:00
MillyandNick Lang cd871244e0 TransportError not raised in helpers.streaming_bulk() with max_retries. (#775) 
* Add test to raise `TransportError` in `helpers.streaming_bulk()` with `max_retries`.

* Re-raise `TransportError` if the last retry fails.
 2018-04-24 11:35:54 -06:00
Nick LangandGitHub 9721e4bc92 Nick/query params bulk helper (#774) 
* cleanup doc strings closes #766

* adding *args to bulk helpers so values such as timeout can be passed in. Closes #773
 2018-04-20 13:11:37 -06:00
umeandNick Lang 4e95fe320d Support renamed routing bulk parameter (#735) 2018-03-04 15:13:52 -08:00
Mike Sukmanowsky 197ad82e18 Use proper retry_on_conflict for helpers.expand_action 
As per
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html#bulk-update,
retry_on_conflict should not be preceeded by an underscore.
 2018-02-13 12:06:29 -05:00
konradkonradandGitHub 9b288a1e92 docstring typo fixes 
cheers
 2017-11-16 17:07:57 +01:00
Honza Král 4e6a05f614 Added scroll_kwargs to scan helper 2017-11-10 21:43:16 +01:00
Honza Král 92763522a9 Check successful shards instead of failed 
Fixes #660, thanks @juke1
 2017-11-09 20:35:53 +01:00
Honza Král 8f73594cac Tests for retry logic in streaming_bulk 2017-07-22 13:40:22 -04:00
Honza Král ba8772a66f When retrying we get deserialized data 
This is just a hotfix until a better solution with proper tests come
along
 2017-07-22 13:13:53 -04:00
Honza Král 2c0f6d8c04 refactor the helpers a bit for more consistent errors 
Now every error (either returned by streaming_bulk or collected by bulk)
contains the original data as well
 2017-07-20 13:53:23 -04:00
Honza Král a16679b2cf draft of docs for backoff helper 2017-07-18 18:08:00 -04:00
Honza Král b31b5ddae6 Merged backoff into streaming_bulk 2017-07-18 17:44:43 -04:00
Honza Král 66c380420f Forgot to assign data 2017-07-18 16:20:36 -04:00
Honza Král 957fcad34b Do not serialize the data in bulk_data 
This can lead to multiple serializations when retrying with backoff
helper
 2017-07-18 16:15:08 -04:00
Honza Král accb19ab90 Action must be a dict, not serialized string 2017-07-18 15:45:43 -04:00
Honza Král f9f4f0c61a have _chunk_actions always return data and actions 2017-07-18 14:18:09 -04:00
Honza Král c8231cad0e Initial implementation of a backoff bulk helper 2017-07-18 14:02:02 -04:00
Honza Král 620afc667f Set a limit on the internal queue of the parallel_bulk helper 
Fixes #547
Solution taken from https://bugs.python.org/msg90067
 2017-07-12 23:48:58 +02:00
Honza Král 5c658fcfb8 Allow pipeline as a parameter in bulk actions 
Fixes #564
 2017-04-17 22:12:41 +02:00
Honza Král c2fc5aa18c Be more explicit with stats_only behavior 2016-12-15 10:21:38 +11:00
Honza Král 5fe3b4baf8 Scan bug that caused the scan to be sorted 2016-11-02 19:29:41 +01:00
Honza Král aa3a7431ff Add note about reindex API to the reindex helper 2016-10-17 14:01:52 +02:00
Will McGinnisandHonza Král ca2dd242c4 optionally don't explicitly delete scrolls in helper (#454) 2016-10-11 15:00:59 +02:00
Honza Král d4efb81b06 Propagate request_timeout to scroll calls 
Fixes #434
 2016-08-29 15:46:46 +02:00
Honza Král 10c013a067 stored_fields=_parent no longer needed with 5.0 2016-07-28 11:29:01 +02:00
Honza Král ef41d893aa fields was renamed to stored_fields in elasticsearch 2016-07-12 17:52:50 +02:00
Honza Král 1e2770fb5d Change default size for scan to 1000 
Fixes #408
 2016-06-03 15:07:09 -07:00
Honza Král 1ce17950e0 Make sure scan helper cleans up after itself 
Closes #389
 2016-05-17 15:14:27 +02:00
Honza Král 6e94bf76ad 5.0 compatibility 2016-03-22 21:04:07 +01:00
Honza Král 8342e0efb1 map already returns an iterator, no need to wrap it in iter 
Thanks to andreip for pointing it out!
 2016-01-15 14:15:53 +01:00
Honza Král 99c284d015 Fixing more typos. Thanks density! 
ref #333
 2016-01-12 11:33:03 +01:00
Honza Král 0f4fa3c97f Fixing typo in max_chunk_bytes 
Fixes #333 Thanks kengruven
 2016-01-07 20:35:27 +01:00
Honza Král b62d950bb8 Make sure we always close the pool when doing parallel_bulk 2015-12-27 21:10:55 +01:00
Honza Král 5e97002f28 Fix documentation for the query param of the scan helper 
Thanks apandita for the report!
 2015-10-23 15:21:28 +02:00
Dmitry SadovnychyiandHonza Král 2d04178215 Move import statement into a function 
To fix a bug on App Engine where multiprocessing is not allowed.
Closes #282
 2015-10-19 15:20:39 +02:00
Honza Král 22943e7e47 Move bulk helpers' docs outside of streaming_bulk's docstring 2015-10-11 23:45:09 +02:00
Honza Král 9d415fadef Adding tests for indexing strings via bulk helpers 2015-10-11 05:00:14 +02:00
Honza Král acde15e742 we use our own compat module, not six 2015-10-10 18:57:21 +02:00
Honza Král 92c1757ad1 Allow people to just pass raw json strings to bulk helpers 
This should allow for significant speedups when indexing json documents
from a file
 2015-10-10 18:40:18 +02:00
Honza Král 5d4640d4c2 Move parallel_bulk helper into helpers/__init__ 2015-10-10 18:35:50 +02:00
Honza Král ccef253001 Merge branch 'parallel' 2015-10-07 18:28:33 +02:00
Sujay MansinghandHonza Král e3d25e8fe3 [#256] Fix exception message 
* Typo: Scrol -> Scroll
* We need to format the string with the args ourselves (ScanError won't
  do it).
  Otherwise the args in the ScanError won't be substituted.
 2015-10-07 17:53:39 +02:00
Honza Král 2208387e0b Extract the processing of individual bulk request to a separate method 2015-10-06 02:10:42 +02:00
Honza Král 3400179153 Allos the size of the bulk request to be defined in bytes 
Fixes #199
 2015-09-30 21:06:24 +02:00
Honza Král 928fc000d1 Serialize the data in bulk helper 2015-09-30 19:25:17 +02:00
Honza Král 9e4d0dd25d Move the bulk body construction into _chunk_actions 2015-09-30 19:16:21 +02:00