Stanford professor/iPhone camera app developer to take two years off to work at Google

Seth Weintraub | Jun 23 2011 - 12:18 pm PT

Google’s photo software is about to get upgraded…

Stanford professor and iPhone Camera app developer Marc Levoy is going to Google for two years according to his Stanford bio page:

I will be on part-time leave of absence from June 2011 through June 2013, to pursue a project at Google.

This stint at Google won’t be his first. He co-designed the Google book scanner and launched Google’s Street View project.

Levoy’s current interests include light fields, optical microscopy, and computational photography – meaning computational imaging techniques that extend the capabilities of digital photography. Levoy’s recent research focuses on camera applications.

My research has recently focused on making cameras programmable. One concrete outcome of this project is our Frankencamera architecture, published in this SIGGRAPH 2010 paper. To help me understand the challenges of building photographic applications for a mobile platform, I tried writing a cell phone app myself. The result is SynthCam. By capturing, tracking, aligning, and blending a sequence of video frames, the app makes the near-pinhole aperture on an iPhone camera act like the large aperture of a single-lens-reflex (SLR) camera. This includes the SLR’s shallow depth of field and resistance to noise in low light. The app is available for $0.99 in the iTunes app store. I don’t expect to get rich from this app, but I learned a lot by writing it, and yes – seeing it appear in the app store was a thrill. Here are a few of my favorite reviews of the app: MIT Technology Review, WiReD.

What’s Levoy going to be working on at Google?

It isn’t immediately certain, but his iOS app focuses (SWIDT?) on making cameras in smartphones better using existing hardware.

But his work is much much bigger than that. He calls his Camera 2.0 project Frankencamera. The programmable camera software can use post-processing algorithms to help with metering, focusing, demosaicing, denoising and white balancing.

Along with Google, the FCamera project has been funded by grants from Nokia, Stanford, Adobe Systems, Kodak, Sony, Hewlett-Packard, The Walt Disney Company, Intel, Texas Instruments, and NVIDIA.

It would appear then that Google wants to integrate these type of super camera features into the Android OS, making the smartphone camera more of a professional quality experience.

As a side note, there’s been word that the new HTC device from T-Mobile (announced next week) will have the most advanced cameras of any smartphone out there (including Nokia’s N9) and be marketed as a camera with a phone attached.

Google also uses cameras in its Streetview cars and Book scanning products which Levoy has co-developed with Google. The post production software could also be applied to already taken photos in Google users’ Picasa or Photovine albums. Theoretically the same algorithms could be used in video as well, improving Youtube video quality.

Levoy has worked directly with Google CEO Larry Page before (both Books and Streetview are Page pet projects) so this is likely a high level project – perhaps the Photovine social network is the type of high level ambitious project that a professor would take two years off of work for.

The combination of better hardware and this kind of work in software will have point and shoot cameras relegated to niche items and if the below photograph is anything to go by SLRs may one day be in jeopardy.

Taken with an iPhone 4 and SynthCam.

Thanks Jonnie!