How Microsoft Created Holographic AR Content For Hololens
Microsoft first revealed HoloLens AR headset back in January 2015, the company’s augmented reality headset that enables users to see and interact with holographic objects and content. The AR headset mixes physical reality with augmented reality holographic objects that user can interact with. Below is the first HoloLens demo from January 2015:
Microsoft’s HoloLens demo at E3 had one of the longest lines we have seen at the 3 days event. The HoloLens demo was truly impressive and mind-blowing! However, limited amount of details had been unveiled since.
NASA was one of the first to get their hands on and experiment with Hololens!
HoloLens: A Solution For Professional & Enterprise
Surprisingly, Microsoft recently announced that HoloLens was not originally designed and built for gaming or entertainment purposes; it was designed to be an enterprise-level solution for professional developers, business sectors, and of course NASA, who had been experimenting with HoloLens on various space projects. Below HoloLens demos are from Build, E3 2015 and the most recent from Microsoft Worldwide Partner Conference.
Microsoft recently uploaded a video on Youtube that shows how Microsoft created holographic AR content. The video demo showed how holographic content of a diver, martial artist, juggling clown, Japanese samurai, animal, weight lifter, basketball player, astronaut, Marie-Antoinette and the traditional Maori Haka dancers were created.
The below image is a studio located in Redmond, WA. This is where these holographic contents were created. In this case, the demo demonstrated the process of how a high quality streamable free viewpoint video was created in this custom built studio covered with calibrated green screen.
How Microsoft Produce Holographic AR Content @ Redmond, WA Studio
The studio is equipped with 106 synchronized RGB and infrared cameras, which are used to capture the silhouette of objects or humans. When all background had been removed, all the data such as depth and texture will then be processed and created.
The first 3D point cloud generated by stereo depth map is rendered, then a multi-view stereo algorithm refines the model into tens of thousands of points, which is ultimately reduced into water type mesh surface and is applied at over one million triangles per frame. Details such as face and hands will be determined; the geometrical shape and texture of these details would be preserved and rendered in targeted density.
Next step is to establish temporal coherence to compress mesh motion and texture, The final step is to unwrap the meshes, generate texture, compress and encode the date into a single stream-able impact file at the target bit rate. The potential usage and application of this technology is unlimited, we can’t wait to see how it’s going to be applied for augmented reality and virtual reality as well!
Hololens Developer Version Expected to Ship In 2016
In an interview with the BBC, Microsoft CEO Satya Nadella said the Hololens developer versions would be made available first, Satya went on and indicated that developers shall expect to receive Hololens sometimes in 2016. Satya also said “We’re looking forward to getting the V1 out, which is more around developers and enterprises in the Windows 10 time frame, Which means it is within the next year.”
