In this series of episodes, we talk to many of the scientists at Blue Sky Studios, which created the Ice Age series of animated features, including the recently released Ice Age: Dawn of the Dinosaurs. In episode 2, we hear from the research and development team about their backgrounds, the kinds of technical challenges they face and the ways they use math and computers to solve those problems. Web sites related to this episode include www.blueskystudios.com; www.iceagemovie.com; www.scientificamerican.com/article/magic-and-the-brain

Podcast Transcription

Steve: Welcome to part 2 of our series of Science Talk interviews with the scientists and other creative members of the team at Blue Sky studios, which brings you the Ice Age animated films. Ice Age: Dawn of the Dinosaurs is just out and playing everywhere, including in 3-D at some theaters.

(excerpts from Ice Age movie)

The biggest event in two million years is about to go to a whole new dimension.

voice 1: Nobody move a muscle.

voice 2: We've been living above an entire world and we didn't even know it.

voice 3: I feel so puny!

Steve: In this episode we will meet the Blue Sky research and development team, everyone introduces him or herself with the exception of someone you'll hear called only Maurice that is Maurice van Swaaij, head of the R&D team. Hearing these R&D folks should convince any kids listening that one great way to get into the movie business is to study a lot of math and science. In fact if you go to the FAQ page of their Web site blueskystudios.com, you'll see tips about what to study if you're interested in getting involved in this kind of work.

Ludwig: I'm Carl Ludwig, and I'm one of the founders of the company, [and] I'm the chief technology officer. My background is in electrical engineering, and I started out in aerospace, but I always was, [even] when I was a youngster, I was a very visual person, I was very interested in art and things of that nature. So I worked in aerospace for a number of years and then from there I went and I started working with a company of a friend of mine from school actually. And we developed some digital film recording stuff for NASA, for [the] Landsat program. And that digital film recorder ended up getting us involved with the people at MAGI, which is the company that did TRON, and from there I ended up working with MAGI and that's how I got into this business. So when I went over there and I saw what everyone was doing, I said, "This is so cool, this is exactly the fit for me." And so I got involved. Then here at Blue Sky, you know, I'm responsible for all the technology, [the] software, and I'm also pretty much the person that writes the rendering software; and Eugene is, this is how we started out—Eugene does all the tracking software and the difficult mathematical calculations for the intersections and then everything else, shadowing and all that, the lighting and lighting techniques and diffusion, transmittance and things of that nature, are all things that I've been involved in. So rendering lends itself to me. I look out that window, and I see what's going on and how do you get that? Why does it look the way it looks? What's going on there? And those physical characteristics, be it a cloud, be it a leaf [those are] the things that I'm able to use my technical background, mostly physics, to get [to] work the way we would like to see [it]. So, you know, this team, everyone brings [a] great deal to the table, and everyone has their areas of expertise and their unique qualities, and it's a beautifully balanced group.

Hadsell: I'm Dick Hadsell and I was trained as a physicist, I have a degree in physics. But pretty soon after I started, I realized that I enjoyed programming computers more than playing with electronics and everything else that's involved in physics. So I got a job working on just computers, and I've been doing it ever since.

Thomson: I'm Trevor Thomson, a senior research associate. My background is in math and computer science, as an undergraduate degree, and then my graduate work was actually in computer art at the School of Visual Arts in New York City. And that's how I met, you know, Carl and the whole gang as they had, you know, a couple of the founders who are involved in the school as well and once I came up here to visit, it was like, this is the place to be, so...

Senguttuvan: I'm Vinoad Senguttuvan. My undergraduate degree was in aerospace engineering and graduate degree was computation engineering. So, I mostly did physics-based simulations from my school and then I moved here and continued to work on fluid simulations, smoke and some collisions.

Steve: Great. Thanks.

Reed: Hi, I'm Michael Reed. I'm a physics undergrad and a CS grad and my work here is mostly on geometry problems and rendering.

Steve: Geometry problems?

Reed: Geometry problems. I mean, the problems here break down to different flavors, there's flavors depending on what the look is, how things look when they show up in an image, and there are problems that are based on basically the geometry in the scene, and I mostly work on the problems with the geometry in the scene.

Roberg-Clark: My name is Gates Roberg-Clark and I'm a research associate. I started out as a lighting TA, and my undergrad is in computer science; and then I went into production engineering here which is like tool development for the studio and to each production. And I'm pretty much continuing that work from the research side of it and developing tools for lighting, animation, whatever department really needs it.

Steve: "Tools" is software?

Gates: Software to help the artists develop, you know, the look and basically some of the tools [I've] written to help them figure out what's going on in the scene.

Borse: Hi I'm Jitendra Borse. I'm originally from Mumbai. That's where I did my undergrad in computer science and then I did my masters at North Carolina State University. And after that I've been, I was lucky to get my first job here. Since then I've been working on different tools to generate—I've been working for a while on this tool, where if you have to generate a forest or something rather than, like, someone going in and bringing in, like, thousands or several thousand trees, you can do it procedurally where they can say, "Hey this is the kind of look we want, and we want to place certain bunch of objects." Those are the kind[s] of software I've been working here, so ...

Steve: So, simplify the whole process of a really complex environment.

Borse: Yeah, kind of make it little automated and, you know, and give them some controls to fine tune it.

McDuffee: I'm Sean McDuffee. I just started here, like three weeks ago. I'm just working for this summer. I've [an] undergrad background in physics. I worked in particle physics for a while for a couple of years, mostly particle accelerators. I got involved with some image processing stuff; I met some graphics guys, realized I didn't want to do that anymore and graphics is a lot more fun and had art involved in it, too. So [I] went back to school, and I'm finishing up a masters in computer science focusing on graphics.

Kwin: Hi, my name is [Kwin Din]; I'm a new hire here. My undergraduate degree was in computer engineering, and then I did a computer science for my graduate degree. Prior to this, I was a faculty in computer science for about six years. And what I really find fascinating is the difference, the contrast between academic research and research here, where you really have to build solutions that fit within the production pipeline. And, yeah, so I'm pretty much interested in all of the graphics-related problems of the geometry and rendering.

Palmer: Hi, my name is Sean Palmer and actually my undergraduate is in fine art, actually I studied...

Steve: What are you doing in here?

Palmer: Well, it's a long story. I've been at Blue Sky for about four years now, but actually started off as an effects technical director and that was primarily like visual solutions, visual scripting and you know, some light programming, you know; but again very sort of results driven, like putting out shot work and things like [that]. But over time, I became more interested in, sort of, the technical solutions and coming up with, I guess, more effective elegant ways to address visual problems that we have here. Like one thing is always like the amount of data required to represent a complex scene and how to best optimize that to create, you know, sort of a visual result that this is just as appealing but can be done in, you know, minutes instead of hours, or you know, or even hours instead of weeks. You know, so as part of that, I started working with Maurice, you know, he's the manager of the group, and kind of through that collaboration I became more and more interested in working in R&D. So I came over last June actually, and I've been working since then on the particle- and fur-rendering technology that we have here, which is completely proprietary. And again just working on ways to sort of further optimize and then structure that to give the artists the tools that they need to get the stuff rendered and out [the] door as efficiently as possible.

Steve: That's great. Let me ask you Maurice, what's the basic mission of R&D here?

van Swaaij: It's kind of two things. One is to identify what artists are struggling with and what we can do to make their life easier. But on the other hand, we also have to look at, you know, the problem of replicating, or showing nature in a computer-generated image and find new ways to do that. It's quite fun to build a tool that does a certain thing for an artist and then see how they can use—and misuse sometimes—it to create wonderful things. It's a real fun environment that way, and I wouldn't want it any other way. Just problem solving is great too, but doing, finding something that you like to do and building a little gadget for somebody to use and then create a way, it's great, it's fantastic.

Steve: What are the some of the, well, you talk about fur. I mean, I remember when this whole world started, I used to always hear people talk about how difficult it was to do water. But when you're doing fur, you know, what are the challenges in fur, just [that] there's so much of it on any individual organism?

Ludwig: Well, [there's] that [and the]in a way gets rendered. Well, Maurice is really the guy who started down that road, and he's really done a fantastic job. But it's the idea of, How do you begin to take care of all these little details and render them properly? Because they're so tiny. So, you know, the approach that he's taken draws on some work that's been done; but its unique and the tools that have been written to control the way things are groomed are also very, very powerful. And it lends itself again to the way we render with ray tracing, which gives us an advantage in that these are things that are rendered not to difficultly with the ray tracing approach. Maurice, if you want to fill in some more, you can.

van Swaaij: I think you said, what's kind of funny about it is that the approach is almost accidental because it started out as a method to render particles. and that's what it's still called. And at some point I thought, "Well, if I can render points, why not lines and spines, hair-like things?" And that's just how it like [rolled] around and then it looks so great by the way that it seemed obvious that there was a method to pursue. And one of the things I'm really proud about is as a team effort we build tools to groom fur on animals. That was one of the challenges and, you know, the basic approach that others had taken was to be like a hair dresser, to take like a virtual comb and maybe virtual scissors and then do the grooming that way. But it's a lot of work and then if things, if a director says, "No I really want the hair cut [a] little differently," you have to start over. And so we build these tools that are kind of complicated at first and sometimes daunting for an artist, but it basically create[s] flow, they create vector fields around objects with various tools. You have magnets to pull the vector to you or push the vectors away from you; line them up with something or not line them up; you know, do things with them that way. And that produced such a great tool that once the artists got used to it, they created these, like, crazy effects with it. A while back, I went back to get my master's degree in the scientific computing, I went to Courant Institute in NYU. And it was, you know, in that setting, its math and computation, and all they care about is, is your computation correct to a certain error; scientific computing is very much about that. It's about, you know, being able to say, "I'm right within that tolerance" and that's all you know, computing that tolerance and proving that you're right. So, applying some of these tools in our field, at some point, I had to realize that math says it's correct within this tolerance, but my eye says it's not right; there's something wrong about it. And it was quite a discovery on my own end, that I had to say, "No, all [methods] are not the same." Because often, you know, in scientific computing you can do things in different ways, but mathematically they're all the same; they all have the same error tolerance. But when you look at the image, you say, "No, this clearly is much better than that," although the mathematical science of it says that they're the same. And I think with some of our volume rendering of small [objects], we have the same kind of problem. We're doing these Monte Carlo integrations and you can prove that this integration has the same error as that integration, but when you look at it, there are discontinuities; there are problems that as a visual result are not good, even though mathematically the problem is solved. And that's kind of very interesting. I really didn't think that, I [thought], "I'm going to do scientific computing; they tell me how to compute things and that's the end of it." But it wasn't, then it was quite a discovery.

Ludwig: When we're doing things visually, it's always the visual quality that matters in the end, because we're making a movie, and we're making imagery. So, the precision, what you have to do is you constantly have to do this iterative study where you basically do the scientific work, then you look at the image, then you look at reality, and you see what's different and why and you discover after a while, you know, this is the thing that Maurice discovered, that some things are more important than other things from a visual standpoint. So, you spend the time computing those things, which are more important, you attach a higher importance to it, and you don't spend as much time doing those things that really don't affect things visually much at all. And you're constantly making those judgments about what needs to be computed more accurately and what can be afforded to just be computationally average.

Steve: Do you ever look at the neurology of vision and use that in your work?

Ludwig: Yes. You know, when you do a lot of studies, you know, about how to get noise reduction in Monte Carlo integration, from a visual standpoint, you know, one of the things you look at is, you know, you look at the character [of] the eye and what's happening there and the way the sensors are distributed and things like that. And you discover that, you know, obviously, you know, pseudorandom distributions are very, very good, Poisson distribution is ideal, but its expensive to compute. So you look always for distributions and things that visually are going to give you better results.

Steve: We just did an article by Penn and Teller. Penn and Teller contributed and an actual neurologist wrote the article and discussed that in magic sometimes, it's not just a question of distraction and you don't realize you saw it; but because of the neurology of the eye and the brain, you literally cannot see the thing that the magician is pulling out on the stage. But of course, the magician[s] figured that out before the neurologists did.

Palmer: It is interesting, it's you know, it's definitely that scientists side and then the magician side; you know, so it's a good analogy because you really find especially. You know, one of our biggest issues is finding ways to optimize as much as possible, and you can use those ideas [as] concepts, you know, to really find ways where you don't need to represent, you know, a fully accurate model of something; and because of that, you know, you can find ways to, you know, compute things more optimally or just save lot of time and then render in another case. I[t']s like, you know, one example that we had was just [scenes] with [an] incredible amount of motion blur, we didn't need to build as much fur in those scenes; because [by] the time everything had been averaged, and then you get the final image, you know, you can optimize quite a bit and come out with [a] image that is the same quality as what you would have experienced if you had actually computed, you know, all the extra, you know, computation and done the extra, you know, time for the render, so, there is that aspect.

van Swaaij: But it's also the other way round. Sometimes you think [dang], your eye just picked something up, and you just can't believe [it]; you think its also continuous, there can't be any problem and yet your eyes see something and where is it coming from? And often your eye just makes up things. Like one of the other things that I discovered sort of recently, and it's like "How did I discover this only recently?" You take a bitmap image, you know, with pixels, each pixel has a constant value basically; if it's far away, [the pixels are] very tiny and you can't see them. If you zoom in and the pixels get bigger and bigger, what you end up seeing is that it looks like the pixels are not constant value anymore. You should do this once; you should try this out. What you start seeing is like as if there's ridges around the pixels; you think that is not possible, and it's just your eye making it up and so sometimes when you look at an artifact, you'll think, "I'll just zoom in and I'll see what the problem is and then I can fix and decode"; you zoom in, you see all these pixels that are not constant value anymore, there are ridges everywhere and you don't know what to do anymore.

Steve: Because you've blown it up so big, we've created edges where there weren't one before and the eyes and the brain is really weird about edges.

van Swaaij: Your eyes is making it up and so it's almost impossible to even look at it anymore but, you know, there's something there because when you're playing it, you just see that little pop happening which is distracting. So it's on one hand, you can do a lot of things, your eyes says, "Yeah, it's fine,"; on the other hand there's these little glitches that just happen and it's just distracting.

Steve: That's it for Part 2 for the Science Talk series on Blue Sky Studios. Don't forget to check in for Part 3 coming your way very soon. I'm Steve Mirsky.

I feel so puny!