Virtual reality can allow for very immersive experiences, with Google actively investing in capturing moments with the point and shoot-like VR180 and professional Jump camera rigs. The company’s VR division is now experimenting with light fields to capture the world with much more realism.
Current methods do not capture how “light bounces off surfaces in different ways” or from different perspectives. This can be replicated through the use of light fields, or advanced capture, stitching, and rendering algorithms.
With light fields, nearby objects seem near to you—as you move your head, they appear to shift a lot. Far-away objects shift less and light reflects off objects differently, so you get a strong cue that you’re in a 3D space.
The end result of recording “different rays of light coming into a volume of space” is “motion parallax and extremely realistic textures and lighting.” So far, Google has captured the Gamble House in Pasadena, Mosaic Tile House in Venice, St. Stephen’s Church in Granada Hills, and the Space Shuttle Discovery.
Google modified a GoPro Odyssey Jump camera from a circular lens arrangement to a vertical arc of 16 cameras mounted on a rotating platform.
It takes about a minute for the camera rig to swing around and record about a thousand outward-facing viewpoints on a 70cm sphere. This gives us a two-foot wide diameter volume of light rays, which determines the size of the headspace that users have to lean around in to explore the scenes once they are processed.
As a demo, Google has released a “Welcome to Light Fields” app for the HTC Vive, Oculus Rift, and Windows Mixed Reality headsets.