The Valley of the Shadow of Death

In 1970, the roboticist Masahiro Mori published an article entitled Bukimi no Tani; English translation: The Uncanny Valley. And it is to this concept that the rather overly dramatic title of this post refers.

In short, the theory asserts that the more closely a robot resembles a human being, the more positively we will respond to it. However there is a drastic plunge as the correlation between the robot’s likeness to a human and our reaction to it deviates. Mori noticed that we will only respond positively up to a certain point. At the point where the resemblance to humans is close, but not quite perfect, our response suddenly shifts to feelings of unease and discomfort; this chasm in the correlation is referred to as the Uncanny Valley. What with all of our fancy schmancy technological advancements, the pseudoscientific theory can now be extended and applied to the realms of media such as movies and video games.

Many brave souls have tried to traverse the Valley but as critics recoil in repulsion from their offerings, the footholds that at once seemed so sure begin to loosen until our intrepid pioneers find themselves out on a severed limb, tumbling haplessly headfirst into the yawning maw of the Valley, like something from a Looney Tunes sketch.

 

One such disaster in film is Robert Zemeckis’ Polar Express. Nothing encapsulates the general feeling shared by critics and movie-goers alike than this review by Stella Papamichael from the BBC website “Forget Christmas cheer – kids will run screaming from The Polar Express after seeing Tom Hanks looking like he’s been killed, embalmed and resurrected by lightning.” This resonates with mortality salience cited in relation to robotics; looking at a robot that falls into Uncanny Valley territory evokes our fear of death. Viewing something with an outwardly human appearance and mechanical interior ignites the subconscious angst we all hold; that we are nothing but soulless machines. Insert a computer generated, motion captured Tom Hanks into the equation and pop on a pair of cold dead eyes not dissimilar to those of the aquatic harbingers of death and what we have is a pretty weird idea for a festive family film with its main protagonist parading jauntily around with a face like a death mask. Ghastly!

 Compare this to The Incredibles, as indeed many critics did and we can see a world of difference. The cartoonish characters are much more appealing than the big bowl of wrong courtesy of Zemeckis. Pixar even stated that the choice of aesthetic was not due to any technological limitations in creating realistic characters but was a concerted effort to avoid the pitfalls on the downwards path to the Uncanny Valley. And that’s why we all love Shrek…well the first two movies anyway. Valley or no, by the time the 44^th installment in that franchise rolls around we’ll be sick of the sight of him.

At this juncture one might pipe up and ask “But what about Avatar? It was like, so awesome and I wish I lived on a magical Pandora and was big, blue and beautiful”. My initial answer would be to stop being so sodding pretentious about what is essentially a remake of FernGully and side note – if you were blue you’d be erring more on the side of smurf. But an interesting point nonetheless. Avatar was visually stunning with characters and environments that seemed so real you could almost reach out and touch them…unless you saw it in 2D. Did James Cameron and his team traverse the divide and conquer the Uncanny Valley? In my opinion, the simple answer is no. If that film were remade with CG humans I think we’d have a different story and not just because it would topple the basis of the entire plot of the film with the abrupt absence of aliens.

The trick to bridging the Valley, or avoiding it altogether, is to suspend disbelief. Avatar portrayed ethereal beings which were pleasing on the eye. They were human enough for us to relate to but they were non-human beings and we therefore didn’t expect anything above and beyond the behaviours and appearances exhibited. The Valley was subsequently avoided.

It’s in the grey area of physical mimicry to ourselves as a species that we begin to feel uncomfortable. This is becoming ever more apparent in video games, particularly as it’s the player that holds control of the characters. Not only do we expect the visuals to portray a level of realism, we expect the animation to support the level of realism we’re looking at. Having a perfectly formed character model that moved within its environment like an 8-bit Mario would be repugnant. But it would be just as ludicrous to watch an imperfect model move with the ease and grace of a gazelle. We need to marry the two concepts together whilst also taking into consideration that the character must react to the players control with precision timing.

 But up until recently it was mostly the graphics that were the issue. How could we forget the Heavy Rain demo at E3 dubbed ‘the casting’ or the creepy ass Alias game depicting the face of Jennifer Garner; devoid of all human nuances and emotion? Mass Effect raised the bar but even so, the lack of focus in the eyes, though they were more expressive, threw people off, along with the ever present problem of mouths and teeth. Enslaved kicked it up a notch with mouth watering environments, superb voice acting and outstanding character animation. The game didn’t look photo realistic by any means but there’s something about the experience that makes it…believable. 

Elspeth Tory, Animation Project Manager for Assassin’s Creed raises an interesting point “A character that’s realistic will seem to have ticked off a checklist of human characteristics, but a believable one will display nuances and subtleties that make them seem unique and alive”. She goes on to include weight and timing as other factors necessary in contributing to the believability of a character. Two of Ubisoft’s biggest franchises, Prince of Persia and Assassins Creed, have believability coming out the wazoo but I wouldn’t tag the animation of the characters in either to be photorealistic. Alex Drouin, Animation Artistic Director on both Sands of Time and Assassin’s Creed describes the tools used to bring a level of believability and realism to the games, such as “blending of animations to get special effects…and different kinds of interpolations to fluidly link different moves together. [The same tools were used] for Assassins Creed but we added full body IK (inverse kinematics) and a rag doll tool”. And hey presto, the goal was achieved without getting mired in a quest to bring about a kind of realism that wasn’t required and that was perhaps unattainable…until now.

Thanks to Team Bondi founder Brendan McNamara and his pal Oliver Bao, we now have MotionScan, or Depth Analysis as it’s now called. This technology combines high definition facial scanning with intelligent algorithms capable of capturing high definition footage of an actor’s face and converts it to a virtual 3D head ready to be stuck onto a CG body. The set up is insane. When it was moved from Sydney to LA, it took Bao over ten 16-20 hour days to install the rig. The reason for the move was indirectly down to the technology; the entire set up allows only 50cm of leeway for the actor’s head movements. Many of the Australian actors initially cast were unable to emulate the US accent whilst sitting still.

  

Depth Analysis is revolutionary in that it eliminates the necessity to animate a characters face, so the results are not only faster but amazingly advanced. And what better way to showcase this technology than with Rockstar’s new title L.A. Noire. It is now entirely possible to use facial cues as a means of progressing through the story and by doing so, adding a new level of immersion to the gameplay.

 

There are still flaws, such as textures of skin and clothes but I think this new advancement shouldn’t be viewed as a means to increase the realism of a game, but the believability. Photorealism is neither here nor there. If you have a character that engages you, you can relate to that character more so than you would do to a perfect avatar devoid of displays of emotion and personality. Rockstar have deftly skirted the boundaries of the Uncanny Valley and emerged unscathed on this one.