Introducing Holographic Emulation

Holographic Emulation is a new feature that vastly reduces iteration time when developing holographic applications in Unity. Developers creating applications for Microsoft HoloLens will immediately benefit by being able to prototype, debug, and iterate on design directly from the Unity Editor without getting bogged down by long build and deploy times. Holographic Emulation works in two different modes: Remoting and Simulation.

Holographic Remoting allows you to run your application directly from the Editor after connecting to a Windows Holographic device (Microsoft HoloLens). The application will behave as if it were deployed to the device (with full sensor data and positioning), but will actually be running on the host machine. The Game view window will also allow you to see what is being rendered on the device (absent, of course, anything that the wearer of the device sees from the real world).

Holographic Simulation goes one step further, allowing you to run on a simulated device directly in the editor — in other words, no physical device is necessary. This is a great option for development when you have a limited number of devices to share among your team, or you want to get started early with holographic development before getting physical hardware.

Requirements

In order to use Holographic Emulation, you’ll need to be running Unity 5.5 (now currently in beta) on a machine with Windows 10 Anniversary update (or later) installed.

Getting Started

To enable remoting or simulation, open the Holographic Emulation control window from the Editor Windows menu by selecting “Holographic Emulation”:

The Holographic Emulation control window can be moved and docked like any other window; we recommend keeping it visible during development because it has a few settings you’ll want access to when starting your application.

Initially the Emulation Mode will be set to None, which means that your application will run in the Editor without any of the Holographic API functionality.

Holographic Remoting

To enable Remoting, change the Emulation Mode to “Remote to Device”. The interface will change to reflect the additional options available when remoting.

To connect to your device, you’ll need to do three things. First, you’ll need to install and run the Remoting Player, available from the Windows Store. Launch the application on the device, and it will enter a waiting state and show the IP address of the device. Additional information about the Remoting Player, including how to enable connection diagnostics, can be found on the Microsoft Windows Dev Center.

Second, enter the IP address of your device in the Remote Machine field. The dropdown button to the right of the field will allow you to select recently used addresses.

Finally, press the “Connect” button.  The connection status should change to a green light with a connected message. You’re now ready to hit Play in the Editor and run your device remotely. You can pause, inspect objects, and debug just like you are running an app in the Editor, because that’s essentially what’s happening, but with video, audio, and device input transmitted back and forth across the network between the host machine and the device. Remoting isn’t useful for validating performance (since it’s using your host machine for all the heavy lifting), but it is a great way to iterate quickly during development.

Holographic Simulation

Running in Simulation mode is even simpler than Remoting. Just change the mode to “Simulate in Editor” and press the Play button. Your application will start with a emulator built into the Editor — no headset required. You can choose from one of five virtual rooms (the same as those supplied with the XDE HoloLens Emulator) as well as specify which virtual hand (left or right) performs gestures.

When in Simulation mode, you control a virtual human with a game controller.

Note that in order to use the game controller, the game window will need to have focus. Just clicking the Game view window once after doing anything else with the UI should give it focus.

Known Limitations

• Speech (PhraseRecognizer) is not currently supported via Holographic Remoting, and instead will intercept speech from the Editor-hosted machine.
• PhotoCapture can be performed during Holographic Simulation, but since the capture will be performed via a local camera (such as a web cam), you will not be able to retrieve a matrix with TryGetProjectionMatrix or TryGetCameraToWorldMatrix.
• During Remoting, all audio on the hosted PC (not just that from the app) will redirect to the device.