Oculus Quest/Android port

[jackjansen]

We need to port cwipc to Android cwi-dis/cwipc#24

Then we can get cwipc_unity working, which this issue is about.

The end goal is cwi-dis/VR2Gather#18 i.e. running VR2Gather experiences with pointclouds on a standalone Quest

This issue is created mainly to keep a record of what we've found, and what issues there are..

[jackjansen]

Currently there are two known issues (using the synthetic pointcloud viewer):

• The pointcloud shader doesn't work. It shows kaleidoscopic color effects in the left eye, nothing in the right eye.
• After about 20 seconds we start getting log messages about refcount=-1 on BaseMemoryChunk.

[jackjansen]

https://developer.oculus.com/blog/debugging-unity-apps-for-oculus-quest-in-headset/ has information in how to do on-device debugging.

https://docs.unity3d.com/Manual/SL-PlatformDifferences.html and https://docs.unity3d.com/Manual/SL-VertexFragmentShaderExamples.html have information on writing shaders.

The latter two have been used to add documentation to PointCloudTextured.shader to help future readers to understand the structure of our shader code.

[jackjansen]

On-device debugging seems to work if you follow the steps very very carefully. There's quite a bit of functionality missing though (such as examining the number of running threads, and I think the stack trace may be incomplete too).

[jackjansen]

Found out that the debugging steps also work for debugging Desktop Unity Player apps (as opposed to the "Attach to Unity" button in VS which attaches to the editor.

It's also useful to select the "Wait for managed debugger" checkbox in the build settings, which waits for the debugger to be attached early during app startup.

[jackjansen]

Bingo! The second error (the refcount one) is not specific to Oculus Quest/Android builds: it is dependent on the backend used. The error occurs when building with the IL2CPP backend.

Found this by doing an IL2CPP build for the desktop (as opposed to the Mono builds we usually do) and seeing the same error.

My assumption is that the IL2CPP garbage collector has a different access pattern than the Mono garbage collector.

I think no memory is actually being leaked, but not 100% sure.

[jackjansen]

There is a GPU profiler/debugger for the Quest that may be useful: https://developer.oculus.com/documentation/unity/ts-renderdoc-for-oculus/

[jackjansen]

Quick check to see whether Z-buffer direction may be the culprit. Doesn't look like it: reverse-Z is true both on Windows desktop and Quest.

[jackjansen]

It's working! The problem was the Stereo Rendering mode (Player Settings->XR Plugin management -> Oculus -> Android).

It was set to "multi-view" but it should be "multi-pass".

This setting is pretty similar to desktop setting Single Pass Instanced or multi-pass, and the former stopped working for us about a year ago. Apparently a shader needs to be adapted for it.

On the desktop the only problem was that some textures were only seen with one eye, but apparently on the Quest it makes everything render in a weird place (very close to the camera).

[jackjansen]

Now that cwi-dis/cwipc#91 is nearing completion this should be revisited again.