Pixel Night Sight tidbits: Improving daylight photos, handling motion, how dark is too dark, more

Ben Schoon | Nov 15 2018 - 9:48 am PT

The camera on Google’s Pixel phones is nothing short of incredible, and it just keeps getting better with each new release. This week, Google has released actual magic onto its phones with Pixel Night Sight, and the results are simply amazing. Let’s take a deeper look into some details about Night Sight you might not have known.

Pixel Night Sight can improve daylight photography too

Night Sight, for obvious reasons, works best when it’s dark. If you take a shot in the daylight, more than likely you’re not going to notice any difference in the lighting. However, what’s interesting to note is that Night Sight can still improve the shot through the amount of detail it shows.

In a sample photo shot by a Googler below, the company details how Night Sight can improve the detail in a normal daytime photograph as well as improving noise reduction.

If you use Night Sight during the day, and are willing to hold still after shutter press, you’ll get the denoising and resolution improvements of Super Res Zoom, as shown in this landscape shot in Marin County, California.

Night Sight lets you shoot the stars

As good as low-light performance has gotten on smartphones, even with Google’s Pixel, you’ll still run into trouble taking photos of starry nights. Even when you manually hold a Pixel, Night Sight can already capture more of a starry night than you’d be able to get with the default settings, but when you add a tripod to the mix, Google can capture stars even better.

(Left) Crop from a handheld Night Sight shot of the sky (full resolution image here). There was slight handshake, so Night Sight chose 333ms x 15 frames = 5.0 seconds of capture. (Right) Tripod shot (full resolution image here). No handshake was detected, so Night Sight used 1.0 second x 6 frames = 6.0 seconds. The sky is cleaner (less noise), and you can see more stars.

How is motion handled?

One of the primary components of how Night Sight works is by using long exposure photography. While shooting in the mode, motion is typically just a big no-no because the subject always ends up blurred. However, Google has built Pixel Night Sight to account for the natural movements of your hand as well as motion in the scene to ensure shots are still as sharp as possible. It’s impossible to account for all motion, but the feature is smart enough to adjust when motion is detected for a better end result. An example of this can be found below.

To combat motion blur that OIS can’t fix, the Pixel 3’s default picture-taking mode uses “motion metering”, which consists of using optical flow to measure recent scene motion and choosing an exposure time that minimizes this blur. Pixel 1 and 2 don’t use motion metering in their default mode, but all three phones use the technique in Night Sight mode, increasing per-frame exposure time up to 333ms if there isn’t much motion.

If the camera is being stabilized (held against a wall, or using a tripod, for example), the exposure of each frame is increased to as much as one second. In addition to varying per-frame exposure, we also vary the number of frames we capture, 6 if the phone is on a tripod and up to 15 if it is handheld. These frame limits prevent user fatigue (and the need for a cancel button). Thus, depending on which Pixel phone you have, camera selection, handshake, scene motion and scene brightness, Night Sight captures 15 frames of 1/15 second (or less) each, or 6 frames of 1 second each, or anything in between.

Center: Night Sight shot with motion metering disabled, causing this phone to use 73ms exposures. Right: Night Sight shot with motion metering enabled, causing this phone to notice the motion and use shorter 48ms exposures.

How dark is too dark?

Pixel Night Sight seriously works magic, but it can’t get the shot in every single situation. At a certain point, Night Sight just doesn’t have enough light to capture a shot or even get focus on the subject. Google details that once a scene has brightness beneath 0.3 lux, autofocus fails. To account for that, Night Sight actually enables “Near” (4ft) and “Far” (12ft) manual focus toggles for situations this dark. It’s not perfect, but it should help keep things in focus.

Focus buttons only available on Pixel 3 devices

Below 0.3 lux, autofocus begins to fail. If you can’t find your keys on the floor, your smartphone can’t focus either. To address this limitation we’ve added two manual focus buttons to Night Sight on Pixel 3 – the “Near” button focuses at about 4 feet, and the “Far” button focuses at about 12 feet. The latter is the hyperfocal distance of our lens, meaning that everything from half of that distance (6 feet) to infinity should be in focus.

Despite autofocus failing, though, 0.3 lux is still enough to capture a shot. Eventually, though, Night Sight just can’t capture enough light. You can experiment with this yourself, but typically speaking, if your own eyes can’t see anything, Night Sight won’t be able to either. As it gets darker too, the amount of noise in the photo grows exponentially. Google summarizes the issue well: “Ultra-long exposure photography is hard.”

How far can we take this? Eventually one reaches a light level where read noise swamps the number of photons gathered by that pixel. There are other sources of noise, including dark current, which increases with exposure time and varies with temperature. To avoid this biologists know to cool their cameras well below zero (Fahrenheit) when imaging weakly fluorescent specimens — something we don’t recommend doing to your Pixel phone! Super-noisy images are also hard to align reliably. Even if you could solve all these problems, the wind blows, the trees sway, and the stars and clouds move. Ultra-long exposure photography is hard.

What’s the difference with Pixel Night Sight on older devices?

Pixel Night Sight isn’t exclusvie to the Pixel 3 family. Because the feature mostly relies on software, it’s also available on the original 2016 Pixel and Pixel XL, as well as the Pixel 2 and Pixel 2 XL. Everything works very similarly, but there are some differences to how it works.

For example, the original Pixel doesn’t have OIS, so Google has less tools to work with when accounting for handshake. The white-balancer is also better optimized on Pixel 3, so colors may look different on older phones as well.

Night Sight works best on Pixel 3. We’ve also brought it to Pixel 2 and the original Pixel, although on the latter we use shorter exposures because it has no optical image stabilization (OIS). Also, our learning-based white balancer is trained for Pixel 3, so it will be less accurate on older phones. By the way, we brighten the viewfinder in Night Sight to help you frame shots in low light, but the viewfinder is based on 1/15 second exposures, so it will be noisy, and isn’t a fair indication of the final photograph. So take a chance — frame a shot, and press the shutter. You’ll often be surprised!