You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When requesting data for a symbol, redis loads the complete Rdata file. This can consume memory if there are many large Rdata files
When we try to save Rdata file after appending new data, the process for saving takes longer, especially on a spindle disk
Both problems 1 and 2 will magnify as we move into saving smaller time resolutions and increase history.
Solution
Save historical data as .rds files that serialize R object.
The name of the rds object should specify the symbol and the period of the data held. If we are able to specify the default number of bars in each rds file, it is possible that we can save the rds file with symbol + starttime.
loadSymbol should load a defined number of recent bars, unless a start and end date is specified. The default number of recent bars should be equal to the default number of bars expected in each .rds file.
Alternative
The current alternative is to keep data in kairosdb (the current solution). However, this has issues of speed while accessing web api for each data call. In addition there is significant maintenance and hardware costs for the solution.
The text was updated successfully, but these errors were encountered:
Problem
Both problems 1 and 2 will magnify as we move into saving smaller time resolutions and increase history.
Solution
Alternative
The current alternative is to keep data in kairosdb (the current solution). However, this has issues of speed while accessing web api for each data call. In addition there is significant maintenance and hardware costs for the solution.
The text was updated successfully, but these errors were encountered: