Skip to content

XR

Jump to bottom
Philippe Coval edited this page May 21, 2019 · 55 revisions

WoTxR : WoT to XR(VR/AR)

Presentation

WORK IN PROGRESS

As a proof of concept, webthing-webapp can be also declined to use 3D widgets and provide immersive web experience:

Special thanks to Fabien, for A-Frame PoC:

Work is still in progress at:

#wotxr-20190320rzr

DEMO

Usage:

Misc:

TWINS

COMPONENTS:

CONTROLLER:

Is there any X/Y track pad events ?

com.sec.android.app.svrbrowser.apk

WEBAPP:

webthings-webapp-aframe

More details on WebApp page

TODO

How to clean cache of sVRbrowser ?

XR

More hints to come at:

MISC

Demos:

Ideas:

  • visualize opensensemap's sensors
  • generic 3d browser

Log:

A-FRAME:

ANDROID:

package=com.sec.android.app.sbrowser
url='http://is.gd/webxr'
activity='com.sec.android.app.sbrowser/.SBrowserMainActivity'

adb kill-server
adb devices -l # Plug USB before
#| List of devices attached
#| ad0ffffffffffffff      device

# Stop package
adb shell am force-stop $package

# Clean cache
adb shell pm clear $package

# Start package
adb shell monkey -p $package -c android.intent.category.LAUNCHER 1

# Start activity and open URL
adb shell am start \
-n $activity \
-a android.intent.action.VIEW \
-d "$url"

# Open URL (for default browsers)
adb shell am start -a android.intent.action.VIEW -d "$url"

adb shell dumpsys package $package

adb shell ip address show dev wlan0 # note ip

adb tcpip 5555 # then disconnect usb

adb connect 192.168.1.58 # Or adapt
#| connected to 192.168.1.58:5555

SVRBROWER

package=com.sec.android.app.svrbrowser
activity=com.sec.android.app.svrbrowser/.UnityPlayerNativeActivity
application=com.sec.android.app.svrbrowser.VrBrowserApplication
url='https://is.gd/webxr'

adb shell pm list package | grep ${package}
adb shell dumpsys package $package | grep ${activity}
adb shell am force-stop $package
adb shell pm clear $package 

# Plug in GearVR headset, it will switch to VR dashboard
adb shell am start -n "$activity"

# TODO: pass url
adb shell am start \
 -n $activity \
 -a android.intent.action.VIEW \
 -d "$url"


# Workaround:

text="is.gd/webxr"
# text="http%3A%2F%2F$text"

# 1st Focus on URL bar (TODO : use adb shell input tap 300 700 ?)
adb shell input keyevent 67 && adb shell input text "$text" && adb shell input keyevent 66

LOGIN:

Automated login hack:

gw_url="https://sosg.mozilla-iot.org"
token="eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Ijg1NWEyYWVjLWI2NTMtNDQ3Mi04MDllLTQyNGI4ZTk1NjQyMiJ9.eyJjbGllbnRfaWQiOiJsb2NhbC10b2tlbiIsInJvbGUiOiJhY2Nlc3NfdG9rZW4iLCJzY29wZSI6Ii90aGluZ3M6cmVhZHdyaXRlIiwiaWF0IjoxNTUyNzM5MzQ4LCJpc3MiOiJodHRwczovL3Nvc2cubW96aWxsYS1pb3Qub3JnIn0.JLgc7Gd5R5IvG5AHdQb1zBG311sOyGjT4bPlqPUYma1IPaxDJYCJQvfurqkNVd2l7MtLP7IGZlBdN7B3pDLzMA"
base_url="https://webthings-webapp.samsunginter.net/"
base_url="${base_url}/login.html"
url="$base_url#?url=${gw_url}&token=${token}"

# On GNU/Linux:
xdg-open "$url"

# With exokit (TBC):
url='http://rzr.online.fr/x/vr/' # TODO
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"


#git clone ... 
base_url="file://$PWD//"
base_url="${base_url}/login.html"
url="$base_url#?url=${gw_url}&token=${token}"

# On GNU/Linux
xdg-open "$url"

# With exokit:
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"

AR:

Normally it should display a cube that goes from Red to Green and then Blue:

url=http://samsunginter.net/color-sensor-js/example/color-sensor-webthing/extra/aframe.html
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"

wotxr-aframe-20190320rzr

SETUP

See WebApp page

oauthClients.register(
  new ClientRegistry(new URL('https://webthings-webapp.samsunginter.net/login.html'),
                     'webthings-webapp.samsunginter.net',
                     'WebApp Token Service',
                     'webthings-webapp.samsunginter.net',
                     '/things:readwrite')
);


oauthClients.register(
  new ClientRegistry(new URL('https://webthings-webapp.samsunginter.net/index.html'),
                     'webthings-webapp.samsunginter.net',
                     'WebApp Token Service',
                     'webthings-webapp.samsunginter.net',
                     '/things:readwrite')
);

BABYLONJS: (WIP)

BabylonJs

ACTUATOR

LINKS:

LICENSE:

LOST ?

Check Concept page for overview, Gateway to get started, IotJs page to install runtime to build webthing as explained in Home page.

For further experiments check Social and Sensor, or Extra parts like WebApp (for Tizen or PWA) or MCU info about running on other microcontrollers not supported by TizenRT.

While Home focus mostly on using iotjs to build webthings (on GNU/Linux or TizenRT for ARTIK05X devices).

FEEDBACK: WELCOME

This document is still in draft state, but reviews are always welcome, if you try to replicate it and stuck on missing instructions I would appreciate that you file issues or even better make pull request (just edit in github) that insert "TODO marks" in following chapters, like:

  • TODO: please explain more this chapter and then remove this TODO line

Community contributions are welcome at:

Support is also possible, ask in:

DISCLAIMER:

WARNING: Developement branches could break over time.

Instead of maintaining "quick and dirty" demo code, I decided to split demo in smaller independents parts (which can reused) and I am upstreaming the most I can.

Then support can be done on mainline branches (or released versions).

Note that, Upstreaming can be a slow process, so snapshots links will remain until 100% of code is upstreamed.

Licence:

Reference documentation is at:

INDEX

Clone this wiki locally