Google AI

Google’s RawNeRF research brings us closer to yelling ‘enhance’ at images

Abner Li | Aug 24 2022 - 3:36 pm PT

Earlier this year, Google Research presented its work on RawNeRF with “NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images” at the CVPR (Computer Vision and Pattern Recognition) 2022 conference.

A neural radiance field (or NeRF) is a neural network that can take 2D images and create a 3D scene. Google has created one called RawNeRF for dark scenes — originally shared in June (h/t TechCrunch) — that combines “images taken from many different camera viewpoints to jointly denoise and reconstruct the scene.”

RawNeRF, which uses RAW images, does a very good job of removing noise. In the first example below, applying Google’s method can clean up enough the noise so that the highlighted sign becomes legible.

Compared to other NeRFs, Google’s RawNeRF can “recover much more accurate color and detail throughout the scene.” The accuracy is approaching the TV trope of characters demanding an image be enhanced and the computer returning something incredibly (and implausibly) high-resolution.

Meanwhile, Google is quick to point out that RawNeRF is more than a denoiser and “can vary the camera position to view the scene from different angles,” as well as vary exposure, tonemap, and focus (specifically, “render synthetic defocus with accurate bokeh effects”).

Google’s six-minute video provides a good overview of RawNeRF, while the paper, data, and code is also available. This work is in the research phase, and it’s not certain whether it will ever be used in a product. Practically speaking, users would have to enable RAW capture, which takes up more space, and snap multiple images. Then again, Google Photos does offer Cinematic photos to replicate 3D motion that might have happened in the lead-up to taking a still image.