Facebook Patent Describes how to Integrate Real World Objects into Virtual World
Users familiar with VR headsets is aware of a so-called protection system that can alert users in a series of different ways when they approach the boundaries of the play area or physical objects. This is because the VR headset will replace the real world around the user with digital content, so in order to prevent the user from colliding with real objects due to the loss of visual perception of the physical world, the device needs a alert system.
Few days ago, the U.S. Patent and Trademark Office announced a Facebook patent application entitled "Feed-forward collision avoidance for artificial reality environments". This invention mainly introduces a technology that uses front camera feedback to render the relative position of physical objects. Specifically, Facebook hopes to use MR to superimpose image data to avoid collisions between users and any physical objects in the physical environment, with a goal to keep users safe and to help ensure an uninterrupted virtual reality experience.
Facebook pointed out that VR content presented by the VR system is not necessarily consistent with the physical dimensions and physical objects of the physical environment. Before the user interacts with the virtual reality system, the user may already have a mental imagery of the physical environment, including location of physical objects in the physical world. But when the user enters the VR and moves in the physical environment, the user's mental imagery of the physical world may become blurred, thereby increasing the risk of collision with physical objects.
As a solution to the above mentioned issue, this virtual reality system patent filed by Facebook includes a physical object overlay engine that can generate overlay image data representing physical objects in a room, and a front camera that provides view feedback for the physical object overlay engine. As an example, the superimposed image data may include a partial image, a single image, or a group of images.
Generally, the superimposed image data can be generated in a way that matches the virtual reality view, and then superimposed to the virtual reality view in a non-obtrusive manner, allowing the user to determine its relative to the physical object without interrupting the virtual reality experience s position. For example, the virtual reality system can simultaneously present the superimposed image of the physical object and the virtual reality image, thereby allowing the user to see the virtual reality content and the superimposed image data at the same time.
In one example, the virtual reality system can automatically superimpose image data of the physical object when the user approaches the physical object. The superimposed image data may be temporary and displayed only when the user's proximity to the physical object has a risk of collision between the user and the physical object. For this, it can be determined based on the proximity of the user to the physical and the threshold value.
In another example, the user can manually command the physical object superimposition engine to generate and superimpose image data of the physical object, for example, through a button of a handheld device. In this case, the superimposed image data may be temporary, and may only be displayed when activated by the user. In addition to buttons, overlay images can be generated and displayed in response to other user activation conditions, such as voice commands, gestures, or other forms of user input.
Through the above two examples, the physical overlay engine can provide images of physical objects, and users can use superimposed images to update their own mental map of the room, thereby avoiding collisions with objects in the room. In addition, users can use the superimposed images to plan their movements or other behaviors in the space.
In the above-mentioned patent illustration, patent FIG. 7A is a virtual reality view presented by a virtual reality headset. In Figure 7B of the patent, in response to the user's proximity to the physical object reaching or exceeding the threshold, or in response to the user's active activation, the virtual reality system can use the video feedback of the front camera and cause the physical object overlay engine to generate the physical object The image data is then superimposed on the physical object image data in the virtual reality view in Figure 7A of the patent.
In this way, the user can use the superimposed image to update his or her mental map of the room, thereby avoiding collisions with objects in the room. In addition, users can use the superimposed images to plan their movements or other behaviors in the space.
