How 3D changes film: thoughts on Coraline

I've never really been a fan of 3D movies. It's always seemed to me like a gimmick, just an excuse to poke things in the audience's face and make them go "oooh", without actually adding anything of interest to the film. But watching Coraline in 3D yesterday has made me begin to change my mind. It actually seems to nearly work.

Before I get stuck into some of the filmic details, I should just say that wearing the 3D glasses is a pain. The glasses themselves are damn good, way better than the old cardboard red/blue ones, and the 3D image is uncannily good, with a great sense of depth. Without my regular glasses, however, I find it hard to see the screen as crisply as I would like. It's not that I can't see properly, but my tired middle-aged eyes do find everything a bit blurry. And these modern 3D glasses, while all very slick, aren't really compatible with my specs. Now it may be that some of the visual issues I experience wouldn't affect me if I had 20/20 vision, but I don't, and neither do a lot of the rest of the population.


For me, the shots that worked best in 3D weren't the effects shots where things come out of the screen at you. I mainly found them tawdry and predictable. What worked well were the shots where they just added depth, and created a heightened sense of realism. Looking out through woods over a valley to Coraline's house (don't worry, no spoilers) looked truly amazing, and for the first time I got a sense of how flat 2D films and photographs actually are by comparison. What was odd, though, was having to relearn how to see. From birth, we're naturally used to focusing our eyes at the distance of the object we're looking at, and we actually have quite a narrow depth of field. As a result, when we shift our point of view from a near object to a faraway object, our eyes move and deform to get the right focus. However, in a 3D film, the objects aren't at the distance you think they are, and so you find yourself focusing on the wrong place. (Which is, I should add, made worse by the aforementioned glasses problem.) If the entire scene is in focus, but feels 3D, you have to learn not to refocus your eyes as you look around the image on the screen.



It's worth a slight digression here on how 3D works. It's not actually 3D, it's stereoscopic, and it's an optical illusion. The glasses are used to split the image on the screen, so each eye sees the world from a slightly different angle, as if it were looking at a real object. However, all you're looking at is a flat image (or two flat images if it helps to think of it that way). Everything you see is at the same distance - in my case, about ten rows or thirty feet away, so to get optimum focus, you should always focus at 30 feet.

Hence the confusion your visual system (eyes and brain) suffers: those hands that appear to be just inches away are in fact thirty feet away, and the trees outside the window that appear to be a hundred yards away are also thirty feet away. As a result, your brain doesn't know where to look to get the best view. And, more confusingly, as you move your head side to side, you can't get a different view of what's in front of you - you'll never see the back of Dad's hand, no matter where you stand. It ain't real. It's an illusion.


Which leads me to the second major issue, that of focus in the film itself. Over the last 80-odd years, we developed a grammar of film that we all naturally understand. Essentially, the thing that's in focus is the thing you should look at. That's how you direct your audience's view around a 2D image. But in a 3D world, it feels odd that there should exist parts of the world that you can't focus on, no matter how you try. In the real world, your focus is where you are directing your vision. You've never experienced a world where you can't control the focus. When you watch a 3D film, you feel you're looking into a "real" environment, even in a puppet show like Coraline, and using focus adds a strange and unexpected level of artificiality that makes you aware that this is still a film construct, not a miniature world.

It's even stranger when they rack focus during a shot. In the 2D world, we're used to this bit of film grammar. First we focus on Fred, and then as Jo speaks her lines, we focus on her. But in the 3D world, we're not used to having our focus pulled around in this way, and our brains have to work out what's going on. The result is quite disorienting at times, only momentarily, perhaps, but that's enough to interfere with your sense of relaxed viewing and with your total involvement in the story.

Staying with the theme of disorientation, cutting styles in 3D are something else that you need to relearn how to interpret. We're all used to cuts in films, whether they're from person to person, as in a dialogue, or from a long shot to an close shot. We've spent tens of thousands of hours getting used to watching it, and we know what it means. (And anyone who's ever done any editing will know only too well what it takes to learn to do it right, and how unwatchable it can be if you do it wrong.) In 3D, though, the experience is quite different, because of the heightened level of immersion. You suddenly go from looking down on a world, to being in it, or you teleport from one position in the world to another one. Again, it may be only momentary, but that's enough to make the narrative feel jerky. In a fast-cut sequence, where you may be changing your viewpoint every second or two, that's enough to make it completely unintelligible.

So, what does this mean for movies?

At times, watching Coraline, I felt like the early film audiences must have felt watching D.W. Griffith films, trying to make sense of his cutting style. It reminded me what it was like playing my first 3D first person shooter games.

Today, I saw the exact same movie trailers I saw yesterday, but this time in 2D. That really threw into sharp relief what works in 3D, and what doesn't.

I think we're going to have to develop new forms of cinematography for 3D if we're going to make it work. When it works, it's amazing. I found myself watching Dark Crystal today and thinking how incredible it would look in 3D. But we're going to have to adapt the rules of film to the new medium. We have to re-understand how to use focus; we have to come up with less confusing ways of cutting; we have to work out which shots work best in 3D, and how to composite the frame differently. And, perhaps most importantly, we have to drop the gimmicky shots that are only there to show the audience we can shove stuff in their faces. Do we really need any more tyrannosaurs leaping out of the screen right at us?

The technology is damn near there now, after over half a century of experimentation. What needs to happen next is for film-makers to learn how to use it effectively.

Oh, my review of Coraline? Go see it. Go see it in a cinema, in 3D, while you still can. Don't wait for the DVD and watch it at home, you'll be missing out. I nearly did.

Destroy the galaxy!

So there we were last weekend, happily photographing M-109, which as I'm sure you all know isn't a German WW2 fighter plane, but is in fact a barred spiral galaxy approximately 46 million light-years away in the constellation Ursa Major, when Rhys decides he needs a pee. So seeing as it was bloody dark, what with it being the middle of the night in the middle of nowhere, Paul gives him a green laser to light his way as far as the hedge. Well, what do you expect when you give a teenager a laser? Rhys waves the laser around, right across the view of the telescope. Which ruined the picture.


Or did it?

We're all clones

Look at this picture. It's an ordinary group of people.


Now, show me a comparable shot in a machinima film. You'll find it hard.

Why?

Because they're all different heights and different builds, and they all have unique face shapes. Which is, of course, normal for human beings, but very rare indeed in machinima. Given that it's one of the most requested features in Moviestorm - in fact possibly the most requested feature - you might wonder why we don't just do it. The answer, according to the tech guys, is that it's hard. Well, they always say that, don't they? And as a film-maker, I don't care if it's hard. I just want my characters to look like real people.

Well, it turns out it's a damn sight harder than most of us realise. When EA did the market research for Sims 3, one of the most requested features was "I want to make a character who looks like me." It's a reasonable request, you'd have thought. Surely we don't have to be limited to one standard height for men and one for women? Or to fat people who merely look like they've just had a good meal? We can give people double chins, jowls, or curved spines? After all, you can have all these characters in games.

Well, no, you can't. Not easily. Most game characters are carefully hand-built, and only have a very limited amount of customisation and things they can do. Sims characters and Moviestorm characters come with an immense range of customisable costumes and props, and an almost infinite selection of animations once you start combining them in strange ways, and that's where we reach the limits of real-time animation technology. If you don't mind intersections, or you're going for a cartoony look, you can get away with it, but if you're going for something akin to realism, it goes into the same category as flowing hair and pulling shirts.

Take these examples:

• Make a character's face really fat. Now give him specs. Unless the specs can deform to match the face, then they'll sink into his face. When I'm at the optician, I spend time trying different sizes of frame to get one that fits. I can do this because there are spectacle manufacturers in the real world who are prepared to spend their lives making different sizes of every frame. In the machinima world, we'd have to build lots of different sizes of each frame, and then you'd have to spend time trying them on your characters. That would be time-consuming and expensive, and simply not worth the effort.
• Get a character to sit in a chair and fold his hands across his belly. In real life, we stop bending our knees when our butt hits the seat, and we stop moving our arms when our hands hit our stomachs. Animated characters don't work like that: they apply a defined movement to a skeleton. A tall character and a short character have to bend their knees different amounts to achieve butt-seat contact, which would mean having to calculate that movement in real time individually for each character. Same with the hand movement: we'd have to calculate where the belly actually is, and then calculate exactly how much to move the shoulder, elbow and wrist to achieve perfect placement. Being out by a few degrees will result in the guy floating above the chair with his hands in his entrails, or squidged into it, Casino Royale style, with his hands in a ballet posture.
• Now make two characters kiss. You may remember from your teenage years how long it took to learn not to bash her face with your nose, and to hold her gently without grabbing her or putting your hands where you shouldn't. And meanwhile she's learning to avoid your clumsy movements so it all appears graceful. If you know exactly what height and build both characters are, and can manoeuvre them to exactly the right starting place, you can easily define how much you need to move the limbs to get them to look right. Performing that calculation in real time is seriously tricky.
  

So it's not surprising that we tend to fall back on a small range of standard character sizes and canned animations. It makes it much, much easier to create something that looks good.

But surely all these problems are soluble? It's just maths, after all. A bit of complicated 3D geometry. Shouldn't be that hard.

Well, you probably could solve the geometric problems given time and a good mathematician, and then code them up. Whether you could run them in real time on a desktop PC is a whole different question. That would require some very nifty algorithms, and probably more powerful machines than we have right now.

Somewhere about now, you're probably going to tell me to stop with the excuses. It's just a programming problem. Just get some decent coders and make it work. But check this. EA tried to do this for Sims 3. And despite spending several million bucks on the problem, they couldn't do it. They concluded it was just too hard, and reverted to the old system of fixed heights and modest changes in body shape.

And, frankly, if EA can't do it satisfactorily, I'm inclined to give the Moviestorm dev team a bit of a break. For the time being, we just have to accept this as another limitation of the medium, and live with identikit characters.

Crowdsourcing movies

Amplified09
Originally uploaded by Documentally

In a darkened room in London, a group of shady individuals discuss the future of movies, and whether it's possible to make movies without the backing of a movie studio.

The answer, of course, was yes. Provided, of course, you're not expecting to see multi-million dollar blockbusters coming out of it. By using the power of social media, people from round the world can hook up and contribute to large-scale productions. You still need a guiding mind - crowd-sourcing doesn't mean a free-for-all, where everyone gets to do what they want. But if you're prepared to take people's ideas, and accept the help they can give, you can suddenly find yourself with a huge resource at your disposal.

Photo by Documentally (aka Christian Payne).

Amplified09 was a social media networking event that took place in London on the 24th February 2009 at Tiger Tiger in Haymarket.

Hard at work...

Some of you expressed disbelief when I told you how hard I work, and how little time I get for relaxation these days. Well look, here's me in the office, so exhausted by 11am I can barely keep my eyes open. Now do you believe me? Do ya? Do ya? And you still expect me to find time to blog?

Picture by Chris Ollis

Make it flow!

Over at Moviestorm Towers, we've been thinking about hair. We're planning a hairstyles pack, and so we've been looking at what's possible from a technical point of view, and what's desirable from the movie-maker's point of view. It's one of those issues like grabbing a shirt. What's simple in real life ain't so in animation, and damn nigh impossible in machinima.


OK, I might as well 'fess up. I like long hair. A lot. I find it incredibly sensual. And it looks great when photographed too. Perhaps most importantly for the film-maker, it looks amazing when it moves.

It's not just the huge movements that make the difference, although those can be spectacular. It's the small shimmers as the hair blows in the wind, or as the actress turns her head. When hair drapes over the shoulders, it completely changes the shape of the face as the head moves, obscuring different parts of the face and the light catches the hair in different ways.

However, you try doing that in machinima. It won't work. It just won't.

For a start, most hairpieces in games are pretty low-poly. They're about as much like hair as the plastic hairpieces you get on Lego characters. They don't move like real hair, they don't shine like real hair, they don't have the translucency of real hair, and they don't have all the different colours of real hair. They're basically simple blocks of stuff that look like they've been attacked with several coats of varnish. That's basically because for games, it doesn't matter too much. The gamer's attention is focused elsewhere.

In a film, however, you're much more likely to want to go to close-up, and so there's much more emphasis on making humans look right. We're not bad at faces these days, but it so often looks wrong because of the hair.


Yeah, so what about Lara Croft's ponytail or the hair in Heavenly Sword, you ask. Surely they're good? Well, true, they do have a basic spring mesh in, and so they do actually move. But let's just mention the dread word "intersection". Machinima really isn't sophisticated enough to replicate how hair actually moves, and really isn't sophisticated enough to model how it drapes over the human body. Watch Heavenly Sword carefully, and you'll see her hair go through her clothes, her shoulder, her legs, her weapons, and itself. It looks pretty awesome in motion when you're in the middle of a battle sequence, but watch the extracts as a piece of film and they just look wrong. With less advanced engines, like Moviestorm, The Sims, or SL, they just don't care: hair will cut through anything. And that always bugs the hell out of me.


When machinima can give me hair like this, I'll be impressed. Until then, I'll always be wanting more than is technically possible. Like the shirts I mentioned earlier, it's even a real bastard to do with "traditional" CG animation. There are some things where actors and cameras win hands down.

Oh, and to save me writing another whole post on the subject of things you can't do with machinima: the same goes for long, flowing clothing. I like that too, and machinima just can't do it. Just replace the word "hair" with "cloth" and it's pretty much the same. Bah.

And no, I'm not having a downer on machinima. It's more that after four years working full-time in the medium, I'm getting a strong sense of where its true limitations are, and are always likely to be.

Reality Checkpoint

Reality Checkpoint
Originally uploaded by Matt Kelland

I could have stayed at home today instead of coming into the office. Instead, I wore my warmest Soviet Army greatcoat and my furry Russian hat (made of genuine bear fur), and walked through a magical wintry Cambridge, taking photographs.

There are more on my Flickr site.