Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct single quote escape character in DynamoDB [#3664] #3667

Merged

Conversation

dlvenable
Copy link
Member

Description

This PR adds unit tests which replicate the behavior in #3664 of single quotes causing data loss. It fixes this by updating the AWS SDK to 2.21.23. We probably should update the version anyway to keep up with the latest.

This PR also adds different error handling when failures occur from parsing data that cannot work. Now the failure is logged and the error metric is incremented.

Issues Resolved

Resolves #3664

Check List

  • New functionality includes testing.
  • New functionality has a documentation issue. Please link to it in this PR.
    • New functionality has javadoc added
  • Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

…r's ObjectMapper. The EnhancedDocument is escaping single quotes. Also, skip data that cannot be parsed entirely rather than silently send empty data. Resolves opensearch-project#3664.

Signed-off-by: David Venable <[email protected]>
@dlvenable dlvenable changed the title 3664 dynamodb escape chars Correct single quote escape character in DynamoDB (#3664) Nov 15, 2023
@dlvenable dlvenable changed the title Correct single quote escape character in DynamoDB (#3664) Correct single quote escape character in DynamoDB [#3664] Nov 15, 2023
graytaylor0
graytaylor0 previously approved these changes Nov 15, 2023
Signed-off-by: David Venable <[email protected]>
@dlvenable dlvenable merged commit 1ffb572 into opensearch-project:main Nov 15, 2023
67 of 70 checks passed
@dlvenable dlvenable deleted the 3664-dynamodb-escape-chars branch November 16, 2023 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Unrecognized character escape issue for stream data in DynamoDB source
3 participants