Microsoft HoloLens – Mixed Reality now here

Microsoft is betting that MR is the productivity tool the world needs. MR is a clever mix of holographic images overlayed on the real world. After an amazing, hands-on demonstration today I can see what it means – let games have the VR world, but almost all else is more suited to MR.

Microsoft HoloLens is the first fully untethered, holographic computer running Windows 10. It is completely self-contained, with no wires, phones, or connection needed to a PC.

It was the first time I have used HoloLens for an extended period. In a blog-like manner let me tell you about this incredible experience.

{loadposition ray}

First, let’s eliminate any preconceptions you may have – it is not VR where the entire view is created and rendered on a computer and viewed on a wireless or wired, fully enclosed, cloistered, headset through fisheye lenses from a small screen (usually either a 5+” smartphone or LCD/OLED screen).

You don the HoloLens like a baseball cap, a fully self-contained Windows 10 device, where you can still see the world around you, but with the addition of a “white dot” that is, in reality, a mouse cursor – showing where you are looking.

The headband weights 579g, but it does not feel heavy or uncomfortable. Adjust the headband for a firm fit (more to ensure it does not move when you move around). You can even wear glasses. I am long sighted and need reading glasses but I did not need them here as images were crisp and clear.

It has two, transparent, polycarbonate lenses (sitting inside the outer visor) that provide amazing, full colour, HD holographic images that have “real” texture and opaqueness – in other words, solid images as if they are there. No, they won't fool you as “real” yet but when it comes in 4K it might. I was impressed!

I was instructed to look at a table in the corner of the room – and to be sure there was a Space Shuttle model and a Pokémon style yellow bird. I walked up to them, put my hands through them, and the colours were solid – I could not see my hands beneath. With a single finger gesture (equivalent to a mouse click) the bird sang.

Then there was a demonstration of the solar system with the planets revolving around the sun – it filled the 46m2 room (about 6 x 7m). I could walk around the “solid” image, under it, over it, through it and select — grab — planets, drill down and more. Pretty cool.

Later I saw a demonstration of a real world app by Saab Australia (former car maker and defence/security contractor) that set up a war “theatre” on an island replete with full, topographic detail, and all sorts of military intelligence and saw how to strategise and rescue hostages. But more impressive was that the two developers were “barely out of nappies” (no offence meant) and took to this medium like ducks to water.

I spent over half an hour immersed in MR and never once felt vertigo or discomfort that VR can cause. I was “safe” in the room with others, and we could discuss, and collaborate normally, each seeing what the other was.

Let’s get technical

The headband contains the HoloLens computer – it is more powerful than an average laptop and has a CPU (central processing unit), GPU (graphics processing unit) and the secret sauce – an HPU (holographic processing unit).

It has an array of sensors to capture where you are looking, movement, gestures, spatial stereo speakers (the red things either side give a precise locational sound without drowning out the real world), battery (about two to three hours use and micro-USB rechargeable), and it communicates wirelessly with a Windows 10 device – a Surface Pro or a desktop is fine.

• 2 x HD, 16:9 lenses capable of 2.5k light points per radiant for realistic, solid holographs;
• 12MP HD video camera to record mixed reality as well;
• 4 environment understanding camera (think Kinect);
• 4 microphones;
• 2 x stereo spatial speakers;
• Human understanding – gaze tracking, gesture input, voice commands;
• Wi-Fi AC; and
• 2GB RAM for CPU, plus 1GB for the HPU and 64GB storage.

And it costs US$3000 for a Development Edition and US$5000 for a commercial edition. Details here.

What some developers are doing

Case Western Reserve University: “This device allows us to engage students in unprecedented ways,” said Mark Griswold, faculty director for Case Western Reserve’s Interactive Commons and leader of the university’s work with Microsoft HoloLens. “The HoloLens allows students to explore the world in completely new ways. The mixed-reality view means students and faculty can interact with one another and the holographic information the entire time, preserving the critical human connection that is such an essential part of learning. We have just begun to explore the potential of HoloLens, and I am already confident that in time the device will be as common in student backpacks as laptops and smartphones.”

Lowe’s Home Improvement: “At Lowe’s we’ve endeavoured to build a mixed-reality solution to help change the future of our retail experience. When we saw HoloLens, we knew we had found a solution that would allow us to create a collaborative process, enabling customers to make decisions about their home improvement projects quickly and confidently,” said Kyle Nel, executive director of Lowe’s Innovation Labs. “We’re investing the most valuable thing we have: our time and our focus. Microsoft’s deep history of delivering meaningful solutions with other companies assures us that our partnership will help provide a differentiating customer experience.”

thyssenkrupp: “With elevators transporting over 1 billion people each day, we have a critical role to play in keeping cities moving. At thyssenkrupp, we are focused on leading the much-needed transformation of the global elevator industry to dramatically increase the efficiency and availability of elevators, and HoloLens is a key element in helping us achieve these goals,” said Andreas Schierenbeck, chief executive of thyssenkrupp Elevator. “The application of HoloLens in our operations can reduce service intervention times by up to four times, and such a feat was only made possible through our strong collaboration with Microsoft.”

NASA: Our partnership with NASA brings mixed reality to space exploration and discovery, creating new experiences and educational opportunities for scientists and consumers alike. Together with NASA’s Jet Propulsion Laboratory, we have created four groundbreaking mixed-reality applications. The OnSight project enables scientists to use HoloLens to “work on Mars” together, directly from their offices, allowing them a means to plan and, along with the Curiosity Mars Rover, conduct science operations on the Red Planet. By extending the OnSight project, in September we opened “Destination: Mars” at the Kennedy Space Center to the public, which allows visitors the only opportunity possible today to walk around the Red Planet, just as NASA scientists do today. The Sidekick project brought HoloLens to the International Space Station to enable station crews to get remote expert assistance when and where they need it, reducing crew training requirements and increasing the efficiency at which astronauts can work in space. The NASA team created the ProtoSpace HoloLens application to build the next generation of spacecraft and space rovers; the application brings 3-D spacecraft designs into the world to help improve the design process.

Saab: “Microsoft HoloLens is the perfect platform for learning, collaborating or visualizing complex information,” said Inger Lawes, Saab Australia’s Mixed Reality Applications Program Head. “For example, in the training environment, it allows both trainers and trainees to share a visually rich interactive experience where the real world can be overlaid with fully interactive holograms. This technology is transformational, and we have had significant interest from a diverse range of industries such as healthcare and mining, keen to understand how Microsoft HoloLens and tailored mixed-reality applications can add new value to their businesses.” 

Airbus: “Airbus believes in mixed-reality technologies, which are already deployed within our products and processes. Microsoft HoloLens is a promising platform, bringing mobility and new ways to consume and link users with digital information. Following our co-development with Microsoft teams, the device is concretely being tested and challenged in various environments across the company to understand if the technology will meet our business expectations. It is a very exciting phase, and we are looking forward to sharing concrete results based on our investigation.”

Audi: “Audi is invested in leading the future of automotive design through the use of cutting-edge technologies. Technology like Microsoft HoloLens could open up new opportunities for our services in many ways-from engineering reviews and collaboration to after sales scenarios and new ways of customer experiences - there are many use cases to be realized," said Jan Pflüger, Co-ordination Augmented- & Virtual Reality at Audi IT. “We innovate to improve service quality, cut time and costs required for maintenance, as well as combining it with a new way of customer communication, a mixed reality solution like HoloLens seems very promising in achieving these goals. We see an exciting future in this technology and look forward to expanding its use at Audi.”

Comment

VR is great for games and things requiring a completely immersive experience and MR is for all else. Besides at the introductory price of $5K it is not for the consumer – not just yet anyway.

But I can see advances in this product bringing it to the home in the not too distant future. For example HR resolution and real colours makes it possible to “HoloLens” a 100“ TV set replete with over the air or video content – that could spell the end of that device as we know it. Similarly, computer monitors could be completely replaced by HoloLens.

But its uses in architecture, design, engineering, medicine, military, and commerce are where it will first take hold. Imagine walking into a showroom and building/decorating/colour coordinating your dream kitchen from where you stand.

Today marks the commercial availability of HoloLens and like NASA’s Neil Armstrong once said, “One small step for man, one giant leap for mankind.” This is epoch making.