How to Create Mixed Reality Videos for the Vive - with Two Controllers

http://secondreality.co.uk/blog/how-to-create-mixed-reality-videos-for-the-vive-with-two-controllers/

16 MAY 2016

Introduction

Mixed reality videos are catching on.

They give a good sense of what it's like to be in VR. In order to create mixed reality videos you normally need three controllers, however I found a solution for two controllers.

Before reading on, read these articles for a good introduction on how to make mixed reality videos:

• http://uploadvr.com/make-mixed-reality-video-livestream-two-realities/
• http://northwaygames.com/how-to-mixed-reality/

As you have read, three controllers are normally needed - two for the VR user, and one attached to the camera. This allows the physical camera to stay in sync with the virtual camera - even when it is being moved.

Solution

There are two options for a two controller setup:

• VR user only has a single controller, the other controller is attached to the physical camera.
• VR user has both controllers, the virtual camera position is moved to the position of the physical camera (somehow).

I'm going to detail a solution for the second scenario. The disadvantage to this solution is that you can not move the physical camera while you are recording.

I made a Vive controller driver that simply 'tricks' SteamVR into thinking a third controller is attached. This will enable mixed reality mode in most games/apps that were built using Unity.

After this we must find the orientation and position of the physical camera so we can put it in externalcamera.cfg. I created a very simple Unity application that would let you do this easily - you simply put your controller next to your physical camera and click a button, and you can then save the orientation/position straight to the externalcamera.cfg file.

Step by Step Software Setup

1: Download and Install the Virtual Controller Driver

Download MixedRealityTwoController-0-0-3.zip and extract it. 
There is a folder called virtualcontroller inside it. Copy this folder, and move it to your Steam drivers folder. Your full path to the dll should be something like:

C:\Program Files (x86)\Steam\steamapps\common\SteamVR\drivers\virtualcontroller\bin\win64\driver_virtualcontroller.dll

Update: The SteamVR folder may be called OpenVR.

2: Enable Multiple Driver Support

Edit your config/steamvr.vrsettings to enable "activateMultipleDrivers". This is what allows the virtual controller driver to co-exist with the Vive drivers. Be sure to mind your commas. Check vrserver.txt log to see if there were parse errors. Many of the settings are described athttps://developer.valvesoftware.com/wiki/SteamVR/steamvr.vrsettings .

{ ... "steamvr" : { "activateMultipleDrivers" : true } }

3: Make the controllers show up in the correct order

You want the virtual controller to be recognized by SteamVR as the third controller, here is how to do that:

• Exit SteamVR
• Turn on your two controllers (press the system button)
• Start SteamVR and make sure both the controllers are seen by your lighthouse (not flashing green).

The third controller will appear after a few seconds. If your physical controllers go to sleep you may need to repeat this process.

4: Generate your externalcamera.cfg file

Included in the zip you downloaded is an application called cameraAlign. Run this program, put on your headset and follow the instructions. When you hit save the externalcamera.cfg file in the directory is updated.

5: Record your videos

Move the externalcamera.cfg file to your game/application directory that want to record. So far I've had luck with Tiltbrush, Space Pirate Trainer and Job Simulator. From here on out follow the usual instructions to create mixed reality videos.

Conclusion

Mixed reality with two controllers is tricky, but possible. Let me know if you have any ideas to improve the process. The source code for the Virtual Controller Driver is available here.

Updates

A number of people have successfully used the driver to create videos.

• Jessie made a great tutorial on how to use the driver!
• Jessie also made a whole series of mixed reality of videos (playlist)
• Excellent review of Cyberpong by Mike
• Zombie Training Simulator gameplay by Christopher Davis

Have you used it to create something cool? Let me know at smr@secondreality.co.uk.

Problems

If the third controller doesn't show for you, you can contact me at smr@secondreality.co.uk. Please include virtual controller in the subject, and the SteamVR log file which is usually found at C:\Program Files (x86)\Steam\logs\vrserver.txt.

Changelog

• Version 0.0.3: Now includes 64 bit DLL. DLLs now contain version info in their property pages. Additional driver logging for troubleshooting.
• Version 0.0.2: Fixed bug where you couldn't make selections if hands were too near. Made cusorSprite more visible.
• Version 0.0.1: First release

Donate

Donations to Child's Play Charity are appreciated.