Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spark remote client fails with passworded login from Windows #78

Open
hongooi73 opened this issue Sep 26, 2017 · 2 comments
Open

Spark remote client fails with passworded login from Windows #78

hongooi73 opened this issue Sep 26, 2017 · 2 comments
Labels
RevoScaleR issue

Comments

@hongooi73
Copy link
Contributor

hongooi73 commented Sep 26, 2017 •
edited
Loading 

rxSparkConnect(
    reset=TRUE,
    hdfsShareDir=paste("/user/RevoShare", user, sep="/"),
    shareDir=paste("/var/RevoShare", user, sep="/"),
    sshUsername=<sshuser>,
    sshHostname=<host>,
    sshSwitches='-pw <password>)'

rxHadoopCopyFromClient("mtcars.csv", hdfsDest="copied")
#copyFromLocal: `/tmp/mtcars.csv': No such file or directory

This is because RevoScaleR:::rxScp trashes the switches:

        switches <- gsub("-p", "-P", switches, fixed = TRUE)
@hongooi73
Copy link
Contributor Author

TFS item 89502

@hongooi73 hongooi73 changed the title rxHadoopCopyFromClient fails on passworded login rxHadoopCopyFromClient fails on passworded login from Windows Sep 26, 2017
hongooi73 added a commit that referenced this issue Sep 26, 2017
@hongooi73
work around #78
10bee65
@hongooi73 hongooi73 changed the title rxHadoopCopyFromClient fails on passworded login from Windows Spark remote client fails with passworded login from Windows Sep 26, 2017
@hongooi73
Copy link
Contributor Author

Broader than just copying files; rxDataStep fails as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
RevoScaleR issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@hongooi73