Say hello to your digital twin. Deep tech experts in Israel can capture a highly realistic 3D image of you using multiple cameras and AI.
Then they bring your twin to life by either doing things they actually filmed you doing — playing the guitar, taking a golf swing, jumping out of a plane — or doing things you didn’t do.
Your digital twin can be programmed to mimic the actions of a library full of things other people have done while you’re sitting at home enjoying a coffee.
Yoom captures a photorealistic volumetric image using multiple cameras and powerful AI. decency
Founded in Tel Aviv, Yoom is one of the leading companies in what is known as volumetric video capture – creating a 3D image with endless possibilities.
With “Immersive Experiences” the Internet takes another leap into the future. This means that we will not only see a character in two dimensions. For example, when we watch a singer sing, we move the mouse on our device to swing left to right, zoom in or out, go down or up, making the experience unique and personal.
Volumetric video recording is the technology that makes this possible. Yoom has been working on it since 2016 and says the AI powering their system is among the best because it has learned from thousands of acquisitions and has been tweaked and refined over the years.
It is an expensive and complicated process, so currently the main customers are sports stars, entertainers, game developers and retailers.
As the “real” and virtual worlds continue to blend—that’s what the Metaverse and Web3 are all about—demand for immersive and interactive content will grow exponentially. Consumers will expect a 3D experience, rather something that’s flat and 2D.
“We enable people to be creative without physical boundaries,” says Bonnka Lim, Chief Marketing Officer at Yoom.
“We are collaborating with a Canadian artist named Riell. She had a creative vision to make a music video about her childhood and the struggles she had.
Montage images show the AI capturing Riell as she moves around the studio. decency
“But she didn’t have the resources to actually shoot with the team in Europe and elsewhere. So we captured her in the studio and created the whole world she had in her head.
“We produced a music video clip that ties into her vision as a creator. Ultimately, our technology lowers the barriers to creating content because once we have your 3D model, we can create any environment you want.”
The images then go through a complex post-production process. decency
Filmed with a set of eight cameras, Riell’s movements were processed by Yoom’s sophisticated neural networks – a suite of algorithms that mimic how the human brain works to perfect rendering, texture and lighting.
Sign up for our free weekly newsletterSubscribe
The images they captured resulted in their digital twin on an apocalyptic city street and a burning house for their single End It – in scenes that would otherwise have been impossible on a budget.
Cameras capture Riell from all angles to create a 3D or volumetric image. decency
“I didn’t know anything about volumetric capture before,” she said. “I didn’t know it existed, nor did I understand the first one as it was explained to me three times.
“I’m really amazed that it’s a camera with an algorithm that creates a 3D version of you. It took me a little while to get that in my head. It recreates you, which I find sick!”
The Yoom team also arrested Dustin Brown to mark his retirement from the LA Kings hockey team in the US after nearly two decades.
Hockey star Dustin Brown is being filmed as he retires for an augmented reality tribute. decency
His digital twin appeared on a giant billboard across from the team’s stadium, appeared to be smashing through a window, and held aloft the Stanley Cup, the National Hockey League’s championship trophy. That was the result of shooting its volumetric image in the studio, then enhancing it and adding effects.
Paul Oakenfold, the godfather of electronic music and legendary British record producer and trance DJ, is another of Yoom’s acclaimed volumetric recordings. He appears in PerfectoVerse, a 72-minute paid concert film and immersive experience that combines his lyrics and music with computer-generated visual magic.
The digital twin of Paul Oakenfold, the godfather of electronic music, in the film and immersive experience PerfectoVerse. decency
“He came into the studio and we captured him and created a whole experience where people can actually see him perform, be close to him and interact with the content. When you click, there are sparks and other effects,” says Lim.
The recording process can require as little as 30 minutes of filming, although there is a lot of technical work involved afterwards.
Lim, who was at Warner Bros. for 12 years, came to Yoom because he believes the technology developed by Yoom represents the future of content creation and production.
“The wonderful thing is that once we have your digital twin, we can put you in any type of digital environment: we can put you in a game, and we can put you in an AR (augmented reality) experience.
“If you look at the history of content on the web, we started with text, then moved to images and then video. But videos are a very passive experience. The only interaction you have with videos is play, pause, skip forward, go back, share. And that’s it.
Volumetric capture is becoming more popular with the advent of Metaverse and Web3. decency
“I believe that the future will be more interactive, so your experience will be different from mine. And this is possible because of the technology we have developed. As you interact with it, you become more immersed in the content, which means you are more engaged.”
