Replies: 2 comments 1 reply
-
|
In the kafka adapter if you pass start_offset = KafkaStartOffset.START_TIME it will play back all data from start time ( which can be in the past ) and the seamlessly switch to realtime mode if your endtime extends into the future https://github.com/Point72/csp/wiki/Input-Output-Adapters-API#api |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
But thats means you end up with duplicates on the sink side? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
I see what you mean, you would have to ensure you dont republish on playback yes |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When using kafka as a source, after restarting a server you can re-read from a topic to build up the states.
But how can you seamlessly go from historical to realtime data without manually change the settings and without duplicates towards a sink?
Beta Was this translation helpful? Give feedback.
All reactions