Vision in film and special effects

Abstract

Computer vision techniques have become increasingly important in visual effects. When a digital artist inserts an effect into a live-action scene, the more information that they know about the scene, the better. In visual effects, we use vision techniques to calculate the location of the camera, construct a 3D model of the scene and follow objects as they move through the scene.Think back to the 1980s, before computer graphics in special effects really took hold. In most of the films of that era, it was very easy to spot the "effects shot" in a film. The give-away was the sudden halting of a previously dynamic, moving camera. This was done because inserting a special effect character or object into a scene with a moving camera was very difficult. The camera move used in filming the live action part of the scene must be exactly reproduced by the camera on the effects set when the model or puppet is filmed. To accomplish this, one of two techniques were employed: 1) mechanical devices were used to encode the motion of the camera and then the encoded information was used to control the motion of the camera on the effects set, or 2) an experienced motion control engineer matched the move by eye. Both of these techniques are terribly prone to error and were used only on high budget productions.In the 1990s with the advent of widespread computer generated effects and digital scanning of film, it became possible to apply computer vision techniques to extract information from the scene to aid the digital artist in blending an effect with the reality of the filmed image.