Developers excited about HoloLens, say it needs a whole new way of thinking

Peter Bright

8ninths virtual workstation. Video courtesy of 8ninths/David Dossett, edited by Nathan Fitch

The first HoloLens Development Edition kits are shipping today to developers selected to be in the first wave of availability. Over the weekend, the $3,000 kits should get into the hands of developers not bound by NDAs and other restrictions, making this the first time we'll start to see and understand just what the device can be used for.

However, these aren't the first developers outside of Redmond to use the HoloLens. The company has been working with a select group of teams to develop apps for the augmented reality headset and to better learn how to use the hardware.

The recurring theme is that HoloLens is very new—and very different from what has come before. The developers we spoke to had some experience with virtual reality but told us that VR did much more to build on existing 3D game knowledge. Although VR does have its own problems—especially nausea for the user—its application in world-building and interactivity is much more familiar. The melding of the physical and virtual world that HoloLens offers is a big shift.

8ninths, a Seattle-based agency, used this shift to good effect in its Virtual Workstation prototype developed for financial traders at Citi. The workstations use conventional PCs—the multimonitor trading desks that are endemic in the financial industry aren't going away soon—but added a Surface Pro 4 for touch and a HoloLens for 3D. 8ninths' aim was to give traders something richer than the grids of numbers they normally work with. 8ninths had to devise new approaches to visualizing data and presenting it, dividing the workspace into distinct areas into which 3D objects are projected.

This process involved considerable trial and error; it's intuitively easy to use HoloLens to build, for example, an even bigger virtual multimonitor system, but existing physical systems already push humans to the limit. It also demonstrated the benefits of the augmented reality approach, as traders don't want to get rid of their existing systems or lose the comfort and familiarity of touchscreens, mice, and keyboards. With HoloLens, these existing systems could be extended, unlike a virtual reality approach that would tend to require them to be dropped.

Both 8ninths and Pop Agency, another Seattle design agency, told us that physical modeling is essential. They found that many ideas they thought up didn't work when implemented and that it was hard to take existing approaches to interface design and apply them to HoloLens' much more "physical" approach. So they ended up building simple cardboard models to get a sense of how things would fit together in real space.

They also had to discard existing practices. Pop Agency is working on a system for MLS to enable teams to analyze their performance, examine how they performed set pieces, and so on. This system uses lots of video that's captured at each match. Initially, the developers created a scrubber of the kind that existing 2D video apps already routinely use. Only after using it did they realize that it didn't translate well to HoloLens' interactions. Eventually, they settled on something that was better tailored to spatial manipulation.

In addition to requiring new approaches to designing systems, the HoloLens requires new software. For example, it needs algorithms to turn the spatial data captured by its Kinect-like sensors into "walls," "floors," and so on. Microsoft is developing a set of libraries to help with these tasks—similar to its D3DX library for performing common Direct3D tasks. This time around, however, the library is going to be open source and published on Github.

It's clear that HoloLens and its augmented reality development are still in their infancy, and it's going to take some time for developers to figure out the right metaphors and approaches to building their apps. The wider availability of the hardware is sure to yield considerable progress in this area, and the future is quite exciting.

Listing image by 8ninths