Digital Humans Becoming Reality

© 1982 The Walt Disney Company

It was 1982 when I first saw Flynn get digitized in the film “Tron” and transported to the Grid, an immersive world inside the computer. I was hooked immediately. Could it be possible to digitize humans in my lifetime? 

Fast forward 30+ years...

Last year, inspired by the world’s renewed interest in VR and AR, I decided to pursue a way to digitize real humans, objects, and environments so that they could be viewed from any angle in an immersive 3D space. Some very smart colleagues and I formed HEXXU CORP to accomplish this mission. When I saw the first moving tests of one of our digital humans in a virtual world, I was transported back to that influential scene from Tron.

YouTube link to Virtual Carl in Motion

YouTube link to Virtual Carl in Motion

Why Create Digital Humans?
As if turning Tron’s science fiction into science fact wasn’t a good enough reason, it turns out digital humans are on the verge of becoming big business - really big business. A new format for delivering live-action content for VR, AR, and mobile will have a disruptive and beneficial impact on many sizable industries including film, television, broadcast sports, Fantasy Sports, eSports, advertising, corporate telepresence, customer support, and social media.

Facebook Social VR demo at Oculus Connect

Facebook Social VR demo at Oculus Connect

As an example, you might have seen the amazing Facebook Social VR demo at the 2016 Oculus Connect Keynote with cartoonish avatars. Check it out if you haven’t seen it. Imagine if that same long distance personal connection could happen with a digital version of the real you instead of a cartoon representing you.

Since beginning this challenge, we’ve noticed several other companies validating the need for this while demonstrating the problems with different approaches. The consensus among them is to call this new experience Volumetric Live-Action. This is why engineers aren’t usually part of marketing. It doesn’t really roll off the tongue, but it does aptly describe what makes the various approaches similar. The ability to capture and play back live action events in a way that can be viewed from anywhere inside a given volume/space. 

What separates these companies and their approaches are their features and visual results. Some don’t even bother trying to look realistic. Some limit the viewer’s movement to just turning their head and leaning, but not really moving around. Some require minutes to process a single frame or hours to process a short clip. These are all important steps in the journey to volumetric live-action but currently they fall short of what should be possible with live-action VR/AR.

Suspension of Disbelief
HEXXU is the only company we know of that is actually capturing high-fidelity digital copies of real people and objects and giving the viewer unlimited 3D motion and personalized viewing angles while playing back motion in a recording or a live event. Our goal is to suspend disbelief and have the viewer think they are looking at a film or HD broadcast, with the added bonus of being able to move around inside it. 

Here’s another way to describe it. When your friend calls you on the telephone or messenger call, you don’t think about the fact that what you are hearing is a compressed digital signal originally sampled periodically over time while your friend talked. It sounds close enough to reality that you feel like you are hearing your friend’s voice and talking with them. HEXXU is doing the same thing with visual and motion quality of digital humans so that you believe you are looking at and talking with your friend even though you know they are 100 miles away. That is what differentiates HEXXU’s approach for digital humans.

I could go into a bunch of technobabble about how HEXXU eliminates the capture issues of occlusion, camera resolution, and depth sensors, but the bottom line is HEXXU’s patent pending technology simply looks better and more realistic, whether you are one inch or one hundred yards away. 

Up close and personal with HEXXU Virtual Carl

Up close and personal with HEXXU Virtual Carl

Are we there yet?
Phase 1 on our R&D schedule was about testing different scanning technologies and gaining the knowledge required to reproduce and automate the process of creating visually realistic looking digital humans. We are already suspending disbelief with a large percentage of the audience when using still images of our digital humans in testing. Can we fool 100% of the audience? No, not yet. What’s missing? In a word, motion. Communication, that personal connection, and believing a live person is there in front of you requires accurate and natural looking motion.

We spent much of this year researching and testing popular motion capture solutions on the market in industries such as film visual effects, game development, or simple live chats. While there are great solutions for each of those fields, we concluded there isn’t a single head-to-toe solution that works for where digital humans are headed. As a result, HEXXU is entering Phase 2 on our R&D schedule, developing a proprietary motion capture and playback technology.

An ENCOM Digitizing Laser in Every Home
We can’t all afford Tron’s multimillion dollar digitizing laser from ENCOM, not to mention it is a fictional device. But based on our first year of research and tests, HEXXU is moving in the right direction to develop a real, fast, and affordable technology that will make realistic digital humans a part of everyday life in every home. 

Tron Image: © 1982 The Walt Disney Company.