This week, WAMG had the opportunity to participate in a virtual roundtable with Ben Snow, the Visual Effects Supervisor at Industrial Light and Magic (ILM) for PIRATES OF THE CARIBBEAN: ON STRANGER TIDES, and ask him some questions about his experiences in the visual effects world.
Raised in rural Australia, Ben Snow studied computing and film at the University of Canberra. He did a variety of work and traveled extensively before working as a runner for a computer graphics house in London. Snow later returned to Australia to set up the computer animation department for a company in Sydney where he worked on commercials, broadcast idents and openers, including a computer graphics title sequence for Beyond 2000.
Snow left Australia to join Industrial Light & Magic in 1994, where his first project was to help create the three-dimensional computer graphics image of the “Enterprise B” for Star Trek: Generations. This version had to intercut seamlessly with the motion control model shot on ILM’s soundstage, one of the first such endeavors for a feature film.
In addition to his feature credits, Snow has been instrumental in the research and development required for the groundbreaking images seen in Twister, Deep Impact and Pearl Harbor. Snow has been honored with Academy Award nominations for best achievement in visual effects on three occasions, Pearl Harbor, Star Wars: Episode II “Attack of the Clones,” and most recently for the breakthrough work on the first Iron Man film.
ILM CREDITS / Feature Films
- 2011 PIRATES OF THE CARIBBEAN: ON STRANGER TIDES (currently in production) – Visual Effects Supervisor
- 2010 IRON MAN 2 – Visual Effects Supervisor Academy Award Nomination — Best Visual EffectsVisual Effects Society Award Nomination – Outstanding Visual Effects
- 2009 TERMINATOR SALVATION – Visual Effects SupervisorVisual Effects Society Award Nomination – Best Single Visual Effects of the Year
- 2008 IRON MAN – Visual Effects Supervisor Academy Award Nomination — Best Visual Effects British Academy Award Nomination – Best Visual EffectsHollywood Film Festival Award — Best Visual EffectsVisual Effects Society Award Nomination – Outstanding Visual Effects Visual Effects Society Award Nomination – Best Single Visual Effects of the Year
- 2003 VAN HELSING – Visual Effects Supervisor2002 STAR WARS: EPISODE II “Attack of the Clones” – Visual Effects Supervisor Academy Award Nomination – Best Visual Effects
- 2001 PEARL HARBOR – Associate Visual Effect Supervisor Academy Award Nomination – Best Visual Effects
- 1999 GALAXY QUEST – Associate Visual Effects Supervisor
- 1999 THE MUMMY – Computer Graphic Supervisor
- 1998 DEEP IMPACT – Computer Graphics Supervisor / Head of Research & Dev. Team
- 1997 THE LOST WORLD: JURASSIC PARK – Comp. Graphics Sequence Supervisor
- 1996 DRAGONHEART – Technical Director
- 1996 TWISTER – Technical Director / Research & Development Team
- 1996 MARS ATTACKS! – Computer Graphics Sequence Supervisor
- 1995 CASPER – Technical Director1994 STAR TREK: GENERATIONS – Technical Director
Q. When producing the mermaids-scenes, what where your inspirations for their movement and the general “look” of the mermaids?
A. We wanted to avoid the classic ‘woman in a mono-fin’ that has been seen in past mermaid films so we studied a variety of marine mammals and fish for the motion of the mermaids. We also had synchronized swimmers and professional athletes that we filmed on set to help guide the animation. As for the look, we were trying to create something different than what you’ve seen before but keep it rooted in reality. We used reference of sea life, human skin, etc. and also bought some fish at the fish market and photographed the heck out of them. In the end, the mermaids had a scaly body with a membrane of sheer, almost jelly-fish like, tendrils.
Q. How much influence did filming in 3D have on your workflow, and which problems / possibilities did you have with that?
A. We were excited by the prospect of 3D. ILM’s 3D pipeline was fairly solid after our work on Disney’s Star Tours 3D and Avatar, and we did some initial work to make sure the 3D set-up for visual effects was rock solid. Creatively we played with the concept of adding more depth for the underwater scenes. We also made sure we were viewing our work in the largest theater at ILM because with 3D the size of the image is important when reviewing work – we always want to see it as an audience would.
Q. When you are working on an animated character, what is most important to you?
A. We want the character to be something unique and fantastic but also rooted enough in reality that it is believable. For the mermaids having the footage of marine animals and the swimmers and actors for reference in the filmed material helped make this possible.
Q. If you compare your work on Pirates of the Caribbean to Iron Man, and before that to The Mummy (for example), what are the most notable differences in your workflow and in the tools you use?
A. Pirates had more organic creatures rather than the metallic hard surfaces of Iron Man, so it had some of the challenges of The Mummy. However today’s tools have advanced a long way since I worked on that film. We were still experimenting with facial motion capture on that film and since then it has become a useful and reliable technique. Our abilities to capture the motions of performers on real sets or locations is far more advanced now than what we had for The Mummy. We were able to leverage improvements we had made to those techniques for Iron Man and other films and push them even further to get a seamless blend between the actresses’ bodies and the mermaid tails.
Q. If you compare Pirates of the Caribbean: On Stranger Tides to the other parts of the franchise, where do you think are the main visual differences?
A. We were conscious of fitting into the world of the Pirates films and to live up to the quality of the effects that had been established in previous entries. However, Rob Marshall brought his own visual sense to the series. He wanted to be sure that the most fantastical elements also were rooted in reality, and brought a grittiness to the film. He also emphasized the story and characters in Pirates of the Caribbean: On Stranger Tides and the visuals, though still high quality, are in more of a supporting role this time ’round – it’s more about the adventure and the story.
Q. What’s the most fun part of your job?
A. The best part of my job is coming up with what something is going to look and move like, and then how on earth we’re going to execute it. And the mermaids represented a tremendous opportunity in that way. I’ve been a fan of mermaids for years and particularly liked the fact these were predatory rather than the friendly mermaids. The biggest challenge was maintaining their beauty while making them scary. Working with Rob Marshall, Charlie Gibson and the team to come up with the look was a lot of fun, and the enthusiasm of Astrid Berges-Frisby, who came to ILM and let us photograph and scan her and have her do motion tests to help us make a better mermaid, was infectious. Working out how to best capture the performances on set and apply them to our CG mermaid was very cool because it involved using all our latest motion tracking tools and some new ones we invented as well.
Q. Ben, you have a wonderful career in working with a good number of popular blockbusters but what are your favorite films that you have worked on in your career?
A. I loved working on Pirates of the Caribbean: On Stranger Tides. Even with the challenges of finding the right look for the mermaids. I also had a lot of fun on Galaxy Quest, which is a film I still enjoy watching. The Iron Man films were also a blast because we got to collaborate with Jon Favreau and Marvel in coming up with the filmic persona of a new hero and helped launch a franchise. And my work on Star Wars Episode 2 was both a great opportunity in terms of career and also gave me a lot of freedom to come up with new ways of doing effects because of the sheer amount of the work and the collaborative relationship George Lucas fostered.
Q. Where did you get the ideas for the look of the mermaids from?
A. The look of the mermaids followed a lengthy discussion process with Rob Marshall and a lot of concept art exploration. In talking to Rob he showed us some previous design work that had been done as well as some classic paintings of mermaids, underwater fashion photography and even a couple of mermaid pinups. He told Aaron McBride, our VFX art director, and myself what he liked and didn’t like about this material. One image he liked had a woman in a sheer fabric tail that you could still see the outline of her legs through. He also talked a lot about the beauty of Astrid Berges-Frisby and how some of the previous concepts had been too creature-like. At the same time we had been looking at different types of marine life, and trying to tie the mermaid into the marine life feel of previous Pirates characters we wanted to integrate, and also discussed how she might actually transform. So we came up with a slightly scaly body covered with a sheer membrane. The idea was that when she was wet the membrane was transparent and you could see flashes of scales, but as she dried it became more like human skin. It was a fun process to invent the science behind these characters.
Q. During the planning of the scenes, what were your major concerns for the final product?
A. Our major concerns were to deliver the best mermaids we could – to try and create definitive filmic mermaids, and to live up to the quality of work that we’ve seen in previous Pirates of the Caribbean films.
Q. Please describe how you used 3D in the effects process. Was it a mixture of conversion and shot-for-3D, and how did you resolve any 3D challenges?
A. We made the decision to shoot in 3D early on, so the only conversion we really had was for underwater shots (which all heavily involved digital effects anyway). Something that helped simplify our lives a little bit in dealing with 3D was the decision to shoot with parallel cameras which made it possible to play with the 3D more during the editing process than has ever been possible on previous films. We had several 3D experts on our crew (ILM has been doing 3D projects dating back to Captain EO in the ’80s, and we had one of our senior crew members who’d been at ILM when they were doing that project) and there was a lot of expertise on the Disney/client side.
Q. What kind of education did you get to get where you are today?
A. I studied computing in college with a major in film (which was somewhat unusual back then and I had to get special permission). But after a couple of years of working in computing and doing film as a hobby, I actually started in the industry as a runner – delivering parcels and making cups of tea. So it was kind of the school of hard knocks. Luckily for me this was 1988 and The Abyss and Terminator 2 came out within a couple of years of each other and the industry exploded with a need for computer skilled filmmakers.
Q. Normally, ILM works on single characters or effects, but with ”Rango” they did their first complete movie. Would you be tempted to work on such a movie?
A. A lot of my friends at ILM had wanted to do an animated film for years, and I’m very happy they got a chance to do so and I am very impressed by the result. I’m more interested in the magic trick aspect of visual effects, in trying to get the audience caught up in the fantasy and wondering later on how we did it. While I prefer effects for live action, I do love animated films and I hope ILM gets to work on another one.
Q. Do you involve the actors in your work and if so, how?
A. Yes, we depend upon the collaboration of the actors in our work. One of the sequences we worked on had Jack Sparrow playing with a droplet of water and I had to draw dots on Johnny Depp’s fingers with a sharpie. He was very helpful and tolerant. Johnny and several of the other actors went through an elaborate photographic and scanning process so we could make digital versions of them if needed. For the mermaids, it was even more elaborate but they were great about it. We’d put them in these psychedelic bathing suits we had designed to make tracking them easier and blending the tails with their bodies, and painted dots all over their faces. Astrid Berges-Frisbey (the key mermaid) was a little concerned when we had to paint dots on her face for a key emotional scene (she was originally going to transform at the end of it) but I promised her they’d be painted out and no one would know. Thank goodness for our talented roto and paint crew who were able to paint out every single dot, and in 3D no less.
Q. When you look at all the films you have made, has post production and Visual effects changed its role in film making in general? Has the perception of “those FX-guys” changed to a more creative part in the production, or does it slowly develop into something that is “sourced out and forgotten”?
A. We’re still a creative part of production and in some ways have more respect afforded to us in recent years since the visual effects are such a big part of the process now. Of course our credits are still way down in the end of the film and I’m hoping that will change. It’s certainly never outsourced and forgotten because on a project like Pirates the visual effects are such a key part of the film.
Q. In your opinion: when James Cameron and Peter Jackson can convince the industry to go up to 60 frames per second: will this benefit the experience of visual effects, or just make it that much harder for the post-production?
A. It will certainly make it harder for post-production. The faster frame rate should make for something that feels more real to the audience but that could make it harder to fool them with our visual effects magic. I’m excited to see the results.
Q. What initially got you interested in visual effects?
A. My dad was a film fanatic and we went to the drive-in a lot when I was growing up. But I pin my initial interest in visual effects to finding a book called “Monsters from the Movies” in my high school library and then later in high school reading an American cinematographer article by Dennis Muren about his visual effects photography for The Empire Strikes Back. So I’d say it was a combination of that and an interest in stage magicians of the golden age.
Q. If you have to decide on one single shot that represents the film’s visual effects work, which one would it be and why?
A. I’m cheating with two shots. I love the shot where the Queen Anne’s Revenge comes chasing the mermaids belching fire. CG ship, CG fire, CG water and CG mermaids and then panning onto real pirates on a set with a digital extension. It was an exciting shot that used a lot of different techniques. My other favorite is panning down from Syrena to her tail just after Phillip immobilizes her with his sword. Astrid looks great in the shot. The lighting on the tail is lovely and the blend is seamless, so I think it’s a beautiful shot that showcases the mermaid tails nicely.
Q. You have been nominated a few times for an Oscar. Do you feel any pressure to uphold or improve that level of “perfection” in future projects?
A. Absolutely. We always try and top ourselves with each project.
Q. Besides the mermaids, you also were involved in the shrinking ships in bottles. What could you explain to us about this?
A. This was a fun sequence that was done by our Singapore ILM studio under my supervision. We shot some ships in bottle props on set and then replaced the ships in the closer shots with a full environment with the Black Pearl, stormy CG water, lightning and so on. In one shot we used a real monkey and in others it was a digital monkey.
Q. Did Disney give you any guidelines as to how scary or dark you could go with the mermaids?
A. Yes, they actually encouraged us to really explore the scary side. In the end, Rob Marshall decided to reign back a little on the more creaturesque aspects and go for something which preserved more of the beauty of the original actresses. And I think this was the right decision – they still seem plenty scary enough.
Q. When putting the different scenes together – at what point did you switch between the actresses and the swimmer athletes, and how did you comp it together?
A. The swimmers were used as a reference guide for our animators adding CG mermaids to shots. We used the actresses when we were closer to the mermaids and added scales, tendrils and tails.
Q. You said that in 3D size is important – concerning that, what are your opinions towards mobile TV / Movies on handheld devices?
A. I had a lot of fun playing with the small screens of 3D portable players but I’m not sure how a 3D film would play on them. I think it’s fun but I’d prefer Pirates of the Caribbean: On Stranger Tides be seen in a theater or on a large 3D television screen and in Blu-ray 3D. That said, I did recently see the film on a plane in 2D and equally enjoyed the story and characters…so it works in any format.
Q. Did a fluid medium (something the normal watcher of a movie doesn’t experience on a daily basis, compared to moving on land) give you a bigger range of possible actions, or do you think that when switching mediums you have to be more constricted to physical possibilities?
A. We tried to keep things as physically real as possible but definitely exploited the underwater medium to play with depth and perspective with the mermaids – so it was quite freeing in that way.
Q. Who are your mentors and do you confer with them frequently?
A. For Pirates it was great to have John Knoll and Hal Hickel who worked on the previous films at ILM close by. John has been a mentor over the years – he was a supervisor on Star Trek: Generations, which was the first film I did at ILM. Dennis Muren has long been an inspiration and a mentor – he was a big collaborator on Star Wars, Episode II and is someone I can always bounce ideas off of or ask for an opinion.
Q. Its one thing to use ILM’s tracking suits on dry land / on the deck of a ship in “Pirates of the Caribbean: Dead Man’s Chest,” but quite another to use this kind of technology in a water-filled environment. Can you talk about how you and your team worked around / dealt with these production challenges on “On Stranger Tides”?
A. We definitely had to up the ante. While we utilized our patented Imocap suits and technology, we also developed some new tracking approaches for getting a match to the surface (since we had to seamlessly blend skin to skin) and tools for trying to cancel out the refraction of the water.
Q. How much input do directors have in your work? Are some more “hands on” while others give you more freedom to work?
A. Some directors are more hands on than others but all have lots of input and the final say. I’ve been lucky that most directors I’ve worked with are very collaborative, inviting us to contribute ideas, using some and rejecting others. It’s all part of the creative process.
Q. What makes a visual effect stunning and memorable?
A. For me it’s when the effect is wonderful and fantastical and something that you haven’t seen before. When you either don’t realize it’s an effect or are scratching your head because you know it can’t be real but can’t work out how it was done.
Q. Did you have any conversations with Astrid in regards to the CG component of her character? What she could expect to see in the finished film, etc?
A. Astrid came to Industrial Light & Magic for scanning and photography her first day on the film and right from the get go, discussed her character and how we could make the mermaid work well. She was conscious of helping us do our job and even let us re-photograph her later on to help with one of the trickier shots once the filming was done and she happened to be visiting San Francisco and came to say hello. She was a terrific collaborator and making her character Syrena work was something she was passionate about.
Q. What were the visual effects created by Industrial Light & Magic besides the amazing mermaids?
A. We did the ships in bottles, the fountain of youth scenes, Blackbeard’s death, a bunch of digital matte paintings for the island and the mermaid cove. We also supervised work done at Scanline VFX in Germany on the sequences when Jack Sparrow had drops of water dancing on his fingers and the animated water as they enter the fountain.
Q. What proved to be the biggest challenge in the making of the mermaid scenes?
A. The biggest challenge was the creative one of how much to play them as creatures and how much as human. Ultimately, Rob Marshall decided to keep them as human as possible, and that was itself a huge challenge because of a phenomenon that they call “The Uncanny Valley” in robotics. The more human something artificial starts looking, the creepier it looks. So it was a lot of work to make the CG mermaids more and more human looking, changing their seaweed hair to human hair and their faces to matching the actresses more exactly.
Q. What do you do when you want to create a certain effect in a film and you don’t have the technology to obtain that effect? Was there ever the case?
A. A big part of the fun of my job is coming up with the technology to achieve a certain effect. Luckily I work with an amazing group of people here at ILM and I don’t think we would ever turn down a challenge.
Q. Do you have any anecdotes that you can share about the Mermaid Lagoon shoot on the back lot of Universal Studios Hollywood?
A. We had a great time working on the mermaid lagoon, surrounded by the lovely mermaid actresses and swimmers. My colleagues who got to apply our motion tracking bands to the mermaid costumes told me I’d given them the greatest job of their careers. The mermaids all knew their names and would call out friendly greetings as they were working around the set, arousing the envy of the rest of the crew. It was also a blast working at Falls Lake in the middle of Universal Studios and being part of the studio tour as we were setting up for the evenings work.
Q. What advice would you give to someone who dreams about wanting to get a job in the visual effects industry?
A. Don’t give up! I was a dreamer for many years. I started in the industry at the bottom as a runner and then worked my way up. Try and get the skills you think would help but most importantly, grab a video camera and make movies with your friends, at school, etc. You really have to love this job because of the hours and the stress but it is immensely rewarding.
Q. After working in the ‘Iron Man’ franchise, with cold technological armors, you had to deal with mermaids in ‘Pirates’. How it was the experience? Where did you find the main challenges in a first approach?
A. It was great to work on an organic creature again after all the metal of Iron Man and Terminator. But like Iron Man, we had to use the best of our existing on-set tracking tools and invent a few new ones to match the bodies to the performers but in this case it was the surface of their skin. We also had to create skin, hair and everything else that goes into an organic creature, which is a different set of challenges to a metal suit.
Q. You used a hybrid approach for the mermaids, with CG and real action performance. Was it particularly complicated to find the right balance? How did you create these mermaids?
A. It was complicated, and a big creative question we struggled with. Ultimately Rob Marshall decided to go with more human mermaids and we used the actresses when we had them, got a couple of the mermaids back to film them for a couple of shots that had been more creaturesque, and put all our skills with creating digital doubles to use. We used every trick in the book – match-animation, facial motion capture, digital skin and a lot of compositing techniques with splashes and fog to make it all hang together.
Q. Ben, any final thoughts that you’d like to share on Pirates of the Caribbean: On Stranger Tides?
A. I’ve really enjoyed the questions in this roundtable session, so thank you all. I loved the chance to create our version of the mermaids for Pirates of the Caribbean: On Stranger Tides. There were definitely some big challenges but I’m happy with the results and am very glad that audiences around the world have embraced the film, and the mermaids. Although I’d warn people to be careful about embracing mermaids as the movie shows.
Setting sail on an ocean full of hi-def adventures, Pirates of the Caribbean: On Stranger Tides will debut on October 18th, in a Limited Edition 5-Disc Combo Pack (1-Disc Blu-ray 3D + 2-Disc Blu-ray + 1-Disc DVD + 1-Disc Digital Copy), a 2-Disc Combo Pack (1-Disc Blu-ray + 1-Disc DVD), and for the ultimate fan a 15-Disc Four Movie Collection. Additionally the movie will be available in 3D and High-Definition Movie Download.
