Skip To Content
NSF Logo Search GraphicGuide To Programs GraphicImage Library GraphicSite Map GraphicHelp GraphicPrivacy Policy Graphic
OLPA Header Graphic
 
     
 

NSF Press Release

 


NSF PR 03-80 - July 31, 2003

Media contact:

 Josh Chamot

 (703) 292-7730

 jchamot@nsf.gov

Program contact:

 Mary Harper

 (703) 292-8930

 mharper@nsf.gov

Behind the Blockbusters--Special Effects Tool Locks Characters onto Film

ARLINGTON, Va.--A motion-tracking software called Fastrack has helped a Hollywood special effects house rapidly stitch computer graphics into several of this year's biggest movie hits.

Developed by researchers at the National Science Foundation's (NSF) Integrated Media Systems Center (IMSC), the Fastrack technology has helped specialists at Academy Award-winning special effects studio Rhythm & Hues drastically reduce production time for such films as X-Men 2, Daredevil, and the upcoming Dr. Seuss' 'The Cat in the Hat.'

"Fastrack is capable of tracking hundreds of features from one frame to another with sub-pixel accuracy in only a few seconds on a standard personal computer," said Eugene Vendrovsky, principal graphics scientist at Rhythm & Hues in Los Angeles, Calif. With the software, effects artists can process roughly 40 percent of movie shots without having to provide extensive input to the computer.

"This is a huge productivity leap for us," said Vendrovsky. "We are almost twice as productive thanks to Fastrack."

The software tracks motion between each frame of film to carefully wed "real" objects, such as an actor, with computer-generated special effects, such as a supersonic jet, a flying car or a raging river.

"At a broader level, special effects involve superimposing something synthetic onto something real," said Fastrack co-developer Ulrich Neumann, associate professor of computer science and Director of IMSC, an NSF Engineering Research Center at the University of Southern California. "The difficult part is to get the motion exact so the objects move correctly relative to each other," he added.

IMSC develops new multimedia technologies for entertainment, security, communications, and education applications-from advanced 10.2 channel sound systems for movie theaters to three-dimensional surveillance technology for airport security.

"The research at IMSC brings engineering, art, mathematics, psychology, and computer science together," said Mary Harper, the NSF program officer who oversees support of the center and its research, education and industrial collaboration programs. "The breakthroughs coming out of IMSC affect everyday life," she added, "engineering that directly impacts education, business and, of course, entertainment."

Neumann and IMSC assistant professor Suya You developed their tracking technology with cinema and training video applications in mind. The researchers used a set of mathematical algorithms to determine which features in a scene provide the best frame of reference for a computer to track.

Reference points, such as corners on a doorframe or a stop sign on street, can be inherent in a scene or they can be fiducials, objects such as large plastic balls, that substitute for a soon-to-be-animated character during filming. The software tracks the reference points, helping the computer glue every digital component-such as an animated character-to each frame of film. Neumann and You originally described their breakthrough approach in the March 1999 issue of IEEE Transactions on Multimedia.

"You create a lot of image layers and superimpose them later," said Neumann. "The layers can be real people and objects in a studio or synthetic, digital graphics, but they are all interleaved in the final image."

Rhythm & Hues bought the right to use the technology from IMSC in 2002, named the software Fastrack, and continually modifies it for use in films requiring complex special effects.

Until the advent of computer graphics technology in the 1980s, a team of animators had to draw many effects onto the film one frame at a time-much like cartoonists-or shoot miniature models frame by frame.

Computers have enabled machines to take over many of the hand-drawn tasks, although effects artists can spend many hours or days smoothly matching a computer graphic to background film.

If the frame of reference moves, even digital processes can be difficult and time-consuming. For example, in X-Men 2, a camera pans around aircraft flying through numerous tornadoes, all in front of the backdrop of a sunset. The camera view of the digital tornadoes has to match exactly with the imagery of the aircraft and the pilot's motions in the filmed world.

Fastrack can do the initial, difficult matching, processing each frame in just seconds. A person then performs final edits and adjustments to tweak the film into a finished product.

For a given scene, the first step is to film the live set and any objects, such as a street with people. Then, Fastrack software analyzes the film and tracks camera motion and staging, saving production time early in the filming process.

Effects artists next create necessary digital elements-for example, swirling tornadoes-animate them, and add any other effects, such as explosions. In the final step, computer tools combine the film and computer elements and transfer the scene from digital data onto film.

"Someone starts the software, looks over the results, and cleans up the shot," said Vendrovsky, "in a process that now takes a couple of hours instead of couple of days."

In the future, Rhythm & Hues hopes to further modify Fastrack for "flexible body tracking," where effects artists superimpose a digital character over an actor in a special body suit.

"Multidisciplinary projects tend to attract more diverse interest and help draw students into engineering, a discipline that has seen enrollment fall over the past decade," said NSF's Harper. "Kids don't know what's possible as a career, and they don't realize you can actually do cool things that will impact the way you live," she added. "IMSC makes engineering important to everyone."

-NSF-

Media Contacts:
IMSC Researcher: Ulrich Neumann, (213) 740-0877, uneumann@graphics.usc.edu
IMSC Media Contact: Rick Keir, (213) 740-4878, rkeir@imsc.usc.edu
Rhythm & Hues Media Contact: Scot Byrd, (310) 448-7477, scotb@rhythm.com


The National Science Foundation (NSF) is an independent federal agency that supports fundamental research and education across all fields of science and engineering, with an annual budget of nearly $5.3 billion. NSF funds reach all 50 states through grants to nearly 2,000 universities and institutions. Each year, NSF receives about 30,000 competitive requests for funding, and makes about 10,000 new funding awards. The NSF also awards over $200 million in professional and service contracts yearly.

Receive official NSF news electronically through the e-mail delivery system, NSFnews. To subscribe, send an e-mail message to join-nsfnews@lists.nsf.gov. In the body of the message, type "subscribe nsfnews" and then type your name. (Ex.: "subscribe nsfnews John Smith")

Useful NSF Web Sites:
NSF Home Page: http://www.nsf.gov
News Highlights: http://www.nsf.gov/od/lpa
Newsroom: http://www.nsf.gov/od/lpa/news/media/start.htm
Science Statistics: http://www.nsf.gov/sbe/srs/stats.htm
Awards Searches: http://www.fastlane.nsf.gov/a6/A6Start.htm

Photo of scene from X-Men 2
Download video:
http://nsfvideo.nomex.net/virt_mike/x2.qt

View streaming video:
pr0380_video2.htm

Scenes from the films "X-Men 2" and "Daredevil", both of which incorporate Fastrack technology. "X-Men 2" and "Daredevil" are Trademark and © 2003, Twentieth Century Fox, All Rights Reserved. "X-Men 2" and "Daredevil" Character likenesses Trademark and © 2003, Marvel Characters, Inc., All Rights Reserved.

Credit: Images courtesy Rhythm & Hues.

Photo of outdoor scene shot from a handheld camcorder
Download video:
http://nsfvideo.nomex.net/virt_mike/ scene3-f.mpg

View streaming video:
pr0380_video3.htm

Photo of outdoor scene shot from a handheld camcorder
Download video:
http://nsfvideo.nomex.net/virt_mike/scene4-f.mpg

View streaming video:
pr0380_video4.htm

These films depict outdoor scenes shot from a handheld camcorder that were processed using the IMSC feature tracking technology. Even though there is motion and shake, the dots show where the tracking algorithm found good features to track. The dot stability and consistency are indications of how reliably the software tracks the features. Note that the system finds these points automatically as the "best suited" scene points for tracking.

Photo Credit: Ulrich Neumann, IMSC

Illustration of models of humans surrounded in a room surrounded by speakers
Download video:
http://nsfvideo.nomex.net/virt_mike/ virt_mike.mov

View streaming video (captioned):
pr0380_video.htm

Film depicting the virtual microphone and immersive audio technologies developed by IMSC's Immersive Audio Lab.

Credit: Victor LaCour, IMSC, imsc.usc.edu

Illustration depicting immersive audio
View video:
http://imsc.usc.edu/research/ project/immersive_audio.html

Film describing the immersive audio technology and its potential uses.

Credit: Victor LaCour, IMSC, imsc.usc.edu

Illustration depicting building and a car
View video:
http://imsc.usc.edu/research/project/ virt_camp.html

One of the latest technologies under development at IMSC is the Augmented Virtual Environment (AVE). The film describes how AVE technology works, highlighting its potential role as a tool to enhance surveillance capabilities for airport, military, and civilian security applications.

Credit: Victor LaCour, IMSC, imsc.usc.edu

Still Images

Illustration of the 10.2 channel sound system
Illustration of the 10.2 channel sound system developed by IMSC's Immersive Audio Lab.
Credit: Victor LaCour, IMSC, imsc.usc.edu
Select image for larger version
(Size: 1.2MB)

A still image of the University of Southern California.
A still image of the University of Southern California created from airborne, three-dimensional scans that were modified and enhanced by IMSC researchers. In order to convert the imagery into an Augmented Virtual Environment, the researchers wed the scans to ground-based, global positioning system (GPS)-correlated video of the same structures.
Credit: Victor LaCour, IMSC, imsc.usc.edu
Select image for larger version
(Size: 504KB)

Larger versions (Total Size: 1.74MB) of both still images from this document

 Note About Images

 
 
     
 

 
National Science Foundation
Office of Legislative and Public Affairs
4201 Wilson Boulevard
Arlington, Virginia 22230, USA
Tel: 703-292-8070
FIRS: 800-877-8339 | TDD: 703-292-5090
 

NSF Logo Graphic