Google’s Starline project is the real deal

It looked like I could grab the apple. Jason Lawrence, a Google researcher, sat across from me, holding the fruit in his hand. I could see it, it was red and shiny, and my brain was telling me it was right there. But Lawrence and the apple were actually in another room – they were just projected in front of me via Google’s Project Starline.

Project Starline is Google’s next-gen 3D video chat booth that it first introduced at Google I/O 2021. Slip into a booth and your image is supposed to be projected onto another booth in real time, as if you were actually sitting with someone else around a table. In a touching video, Google showed family and friends happily connecting with each other using Starline, and the virtual recreations looked remarkably realistic. “It was mind-blowing,” one person says in the video. “I’ve seen a lot, but I’ve never seen this,” said another.

But in the year since, Google has largely kept the project secret – until now. The company invited me to experience Starline for myself, and I was eager to find out if it would live up to the hype.

“We believe we have a breakthrough in communications technology that allows you to feel closer and more connected with people who could be anywhere in the world,” said Andrew Nartker, director of product management for Starline. Nartker was seated across from me at a table in one of Google’s conference rooms for a normal face-to-face meeting. He described the Starline experience as “a magical window” that lets you feel like “you’re connected and together” with other people.

Eventually, we moved on to testing Starline. The machine was stuffed into a small conference room, taking up the vast majority of the space. To one side was a long wooden bench with a seat cushion right in the middle. On the other, a display showed the empty cushion in the other Starline booth. When I took my seat, more than a dozen cameras and sensors were trained on me. It was nerve wracking – I could tell my every move was tracked.

This picture of a Google research paper gives you a simplified idea of ​​what I was watching sitting in front of the screen. There are a lot of sensors.
Image: Google

But when Nartker slid into the frame of its Starline booth, the technology largely faded and we were able to immediately resume our conversation as if we had moved from table to table.

Starline does a wonderful job of creating a 3D representation of the person you’re talking to. Nartker and Lawrence looked exactly the same as when I shook hands with them a few minutes earlier. Virtual shadows behind both helped sell the effect. It was even possible to estimate how far Lawrence’s apple was in front of his body.

The whole thing felt much more natural than a Zoom call. There was no noticeable latency in their movements or in our conversation, so talking was like talking. There were no weird audio or visual delays. It was easier to believe I was actually with someone because I was making real eye contact with life-size virtual humans instead of squinting at a tiny Zoom window.

Part of the reason Starline is so compelling is that you’re not just looking at a screen, but a series of lenses in front of a screen, or lenticular array. The principle is similar to holographic cards which can display a different image or 3D effect when you rock them back and forth, Lawrence said. Starline goes a step further by following your eyes to know where to direct the images it displays to you.

One person talks to another about pottery using Google's Starline.

Note that I had to compress this GIF, which adds artifacts to Starline that I didn’t see when I used it myself.
GIF: Google

The illusion was not perfect. Looking closer, you could tell the person was being recreated; a head might not be perfectly round, for example, and the top of Lawrence’s hair might get quite wiry. If the person you’re talking to moves within the confines of what Starline was actively reconstructing (Nartker estimated it was about a cubic meter of space), they’ll blur, shatter into glitched polygons, and end up by completely disappearing.

Starline is awesome and has definitely improved video chat. I can see it being useful for one-on-one conversations, especially in places like a doctor’s office or a customer service environment. But Starline clearly has some limitations. The Starline stand is huge and laden with presumably expensive technology. Nartker refused to share a price when I asked, and you’ll need two to use Starline.

Still, Google is moving forward with Starline by slightly expanding who will be able to try it. This week, the company announced that it would be installation of Starline cabins for certain business partners. Companies such as Salesforce, WeWork, T-Mobile and Hackensack Meridian Health will test the technology as part of an early access program. Google employees have already put in thousands of hours, Nartker said, and the company said it has invited “more than 100 partner companies” in industries including media, healthcare and retail to try Starline in Google offices.

“It’s not a product at this stage”

There’s still a long way to go before Starline is widely available for purchase – if it ever is. “It’s early-stage technology that we think is very exciting and a breakthrough in space,” Nartker said. “But it’s not a product at this point.”

Throughout our chat, Nartker talked about how Starline dramatically improved telepresence by replicating a face-to-face conversation in person. Having experienced Starline for myself, I now agree that better telepresence can have a significant impact on virtual interactions, and I feel like something like Starline needs to be in our future – even if that future is years away and hopefully isn’t a giant video booth.

The same day, listening Interview of Meta CEO Mark Zuckerberg with The edge, I was struck by how it made a similar pitch for a very different set of devices. “The magic of virtual reality, for people who have experienced this, is that it immediately convinces your mind that you are present in another place and with the people there,” Zuckerberg said. He explained how, in the future, the interview could be done with him in hologram form, which is a big part of what amazed me with Starline.

I found it striking that two of the biggest tech companies in the world are trying to improve their virtual presence. Of course, both would benefit; Meta is all about its metaverse concept, while Google is pushing hard to take over the company. Chat booths and expensive mixed reality screens aren’t practical or desirable for most people right now, but maybe the thing that really takes off will happen somewhere in the middle.

Nartker ended our Starline conversation with a virtual punch, something he said he does with every demo. As with Lawrence’s apple, I logically figured out that I wouldn’t feel Nartker’s knuckles touching mine. But when we entered the bump, my brain anticipated that my fist would still make contact, and my hand still felt a sensation. When our hands crossed, we laughed at the shared moment, even though we were apart.

Leave a Comment

Your email address will not be published. Required fields are marked *