Google’s Project Starline has a secret weapon – eye contact

After experiencing a video call through Google’s Project Starline, the most impressive part of the demo was the ability to make meaningful eye contact.

Like many others, I’ve participated in more than my fair share of video calls in the past four years. Work moved online, which meant meetings moved online, and for a time, our friends and family were only available online too. We’re past the worst of that period, but the inherent downsides of video calls still feel fresh and obvious. “Zoom fatigue” is still very much in our common vocabulary.

One of the many things I’m self-conscious about during video calls is where I should be looking. The other person is often squarely in the middle of my screen, while the camera is typically at the top. At first, my eye is drawn to look at the person, but on camera, I appear to be looking downward. If I look at the camera instead, I only see the other person from my peripheral vision.

Can you tell I’ve thought a bit too much about this?

It all became a bit more real for me a few years ago, when my partner moved across the country to accept a job offer. I was self-conscious about where to look in video calls before, and I was doubly so until I could also move to rejoin her. Standard video calls were all we had, so we made do. But it was much harder to make the human connection.

In the last few years, Google has been building a better vision of what video calling could be, having first unveiled Project Starline in 2021. The premise is an interesting one. Instead of seeing a 2D video of the other person, you’re shown a live-animated 3D render of them — created in real time, no less.

I had the opportunity to try out Project Starline last month at Google I/O, and I thought I knew what to expect. “3D” became something of a buzzword in the early 2010s following the breakout success of Avatar in 2009. Best Buy had demos of 3D televisions, and even some phone makers were experimenting with glasses-free 3D screens – anyone remember the HTC Evo 3D?

My contribution to that fad was preordering the Nintendo 3DS ahead of its 2011 launch (shoutout to fellow “3DS Ambassador Program” members). The handheld’s main idea was to split the screen’s visual output to show different images to each of your eyes. Since the Nintendo 3DS had no way of tracking your face, you essentially had to hold it at a specific distance and angle for the effect to work correctly.

Was it impractical? Yes. But was it very, very cool? Also yes.

From those early experiences with glasses-free 3D, I expected Project Starline to be a bit wonky but workable. I spoke briefly in person with Andrew Nartker, Google’s GM for Project Starline, then sat at an empty table across from the unassuming hardware. It’s been over a month since my demo, and I can still distinctly remember my face lighting up with amazement when the call began.

Andrew was now sitting across from me, as though a portal had opened where a flat screen once was. I shifted side-to-side in my swivel chair, but the illusion didn’t break down. Moving to the side simply revealed a different angle of the other person. Clearly (and perhaps unsurprisingly), 3D technology has progressed significantly since the Nintendo 3DS’s release. I assume that the same cameras that were recording me in 3D were also adjusting the display to my eyes’ locations.

What struck me most about the experience, though, was where my eyes were drawn. Just as though we were in the same room, I was naturally inclined to make eye contact. This change alone made Starline significantly better than any video call I’d ever done – so much so that I hesitate to even refer to it as a video call. Instead, it felt like a genuine “virtual meeting.”

While the underlying tech was quite impressive, it was also easy to see and imagine its current limitations. To borrow a phrase from Fozzie Bear, one of the “cheap 3D tricks” saw Starline relatively accurately depicting a bright green apple that Andrew picked up from his table. If something is happening within range of what I’d call the “stage,” it’s properly shown to the other person. However, the stage is a bit smaller than you might first think.

I didn’t notice until my host reached his arm slightly beyond it, but a key part of Project Starline’s illusion is a fake 3D background. These backgrounds (of which, there were a few to choose from) give the appearance of being in a larger space while hiding the ends of the stage. The quality of these backgrounds was actually quite stunning.

Another more noticeable issue – and one that should surely improve with time and development – is that you lose some sharpness and fidelity in the transition from real person to 3D render. It wasn’t enough to break me out of the illusion, but some things would look a bit fuzzy or odd.

However, the biggest problem with Project Starline as it exists today is that it’s only really going to be helpful for one-to-one meetings. Barring the arrival of a significantly more complex glasses-free 3D display (or the development of Star Wars-style holograms), it’s only possible for a single person to see the full illusion. Even if we supposed that three separate Starline owners wanted to connect, I can’t imagine the experience would be as immersive when trying to squeeze multiple people onto the single display.

Despite these drawbacks, I can’t help but still be impressed by what Google has delivered with Project Starline. Conversations flowed naturally, body language was visible, and eye contact was instinctual. It all felt human.

Google knows that it’s onto something and is now looking to spread the magic of Starline. The company recently announced a partnership with HP to produce Starline hardware for companies to purchase. Perhaps more crucially, Google is also ensuring that the 3D video meetings can be held in Meet and Zoom alike.