Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve behavior across multiple session #17

Open
piccolbo opened this issue May 29, 2015 · 0 comments
Open

improve behavior across multiple session #17

piccolbo opened this issue May 29, 2015 · 0 comments

Comments

@piccolbo
Copy link
Collaborator

What happens right now all tbls become stale, src becomes stale and one needs to refresh by hand.
Possible steps:

  • src stores creation call, uses it to refresh itself
  • tbl can temporary (TT) or persistent (PT). TT are just a backing for an in-memory object, PT have a life of their own. The table behind a TT should be destroyed at gc time, the one behind PT should outlive it. When a TT is saved, meaning it has a chance to be loaded in another session it should become a PT. The latter is problematic because the risk of leaving behind, but it mimicks the behavior for regular objects.
> z = data.frame(col1, col2)
> quit()
save? N
$ R
> #z is gone
> z = data.frame(col1, col2)
> quit()
save? Y
$ R
> print(z)
> #z is still there
> rm(z) #z gone

There's a third possibility where you can save z to a file and load it later. Now in an ideal world we can go through the same use case with a tbl_SparkSQL and never have z in some butchered state where it exists but is invalid. This may be a pipe-dream because I don't think save is a generic so that we can do something with it, but I thought I'd write it down.

piccolbo added a commit that referenced this issue Aug 14, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant