[DBZ-PGYB] Non-key columns should always be optional for replica identity CHANGE #147
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Suppose we create a table which has a non-key column which also has a constraint
NOT NULL
- when the connector will start and CDC service will send aRELATION
message, the column will be marked as "required" since aNOT NULL
column means that the value should be present.But for a table with replica identity
CHANGE
the column will not be present inUPDATE
operations if the column is not updated. This will violate the assumption of the column being marked as "required" since the value will not be present and the connector will end up throwing error of a similar format as:Solution
This PR introduces a fix to the above problem by adding a check while decoding the relation message - the check essentially verifies that if the replica identity of the table is
CHANGE
then we explicitly mark the column as optional if it is a non key column.