Gourt Home::Gourt :: Camera tracking
Camera tracking or matchmoving refers to the process of matching the position and angle of CGI (such as a spaceship or animated creature) to real footage shot with a film or video camera. This process can be accomplished manually on a frame by frame basis. More common is the use of automated matchmoving software based on computer vision techniques that can extract camera position and rotation values over time from a frame sequence of digitized footage with minimal user intervention. Done correctly, matchmoving allows the seamless compositing of additional elements into camera footage that may have been too expensive or impossible to achieve otherwise.
Tracking software takes each frame of a sequence and analyses the direction and speed of unique pixel regions that it can 'see' using specialised algorithms. The software will find and track any high-contrast or unique patterns of pixels that can be located precisely in each frame. The software will then use other algorithms to reconstruct a camera animation path in 3D space that tries to match the exact movements of the camera in the video footage. 2D and 3D effects can then be inserted into the shot, and the postions of the new objects will match seamlessly with the video footage.
If a scene lacks contrasting or patterned objects, the tracking engine won't have enough data to solve an accurate matchmove. To improve tracking results, artificial track points are often added by a set technician. These are typically made from small, round matte objects such as Ping-Pong balls, clay, or paint dots. Survey measurements are also entered into the program in order to improve accuracy.
Camera tracking usually applies to shots that involve moving cameras. If the camera in a shot to be composited is static, then the CGI artist only needs to match the position and perspective once (in the 3D animation software), and the rest of the objects placed in the scene will have the correct perspective as well. However, the difficulty comes when the shot is moving. In this case, the 3D artist would need to match the shot frame by frame by eye, so that the 3D camera has the correctly matched perspective in each frame of the composite. There are several disadvantages to this. Firstly, it's tedious and time-consuming. Secondly, it's prone to error in perception on the part of the artist who's performing the manual camera match. With matchmoving software, tracking is much faster and more accurate because quick mathematical operations are employed, rather than human spacial perception.
Matchmoving works by the principle of parallax. Abiding to this optical phenomenon, when the camera is placed in a moving vehicle and static objects are observed, the objects nearest to the camera will move across the screen the fastest (taking the shortest time to move from one end of the frame to the other). The furthest object will travel the slowest (taking the slowest time to get from one end of the frame to the other). When the matchmoving software processes moving footage, it will use this principle to track fast- and slow-moving points in the shot and, by comparing them to each other, try to derive the angle, distance and speed at which the camera is moving. The method mentioned above is only used for solving tracking shots. More complex algorithms in the software will solve more complicated camera moves in the shot, such as panning, tilting and zooming. The software will also need to compensate for lens distortion in the footage, or if the camera was using wide-angle lenses that exaggerate perspective; or narrow-angle lenses that flatten perspective.
The matchmoving software will have difficulty with accuracy in solving under these circumstances:
1. Poor quality of original footage. This could be due to compression or encoding problems for digital footage. In the case of film, it could be caused by scratches and dirt on the film, or from light leaking into the camera during shooting; or the film not properly seated on the sprockets.
2. Movement of objects in the footage is too fast, producing motion-blurring in the shot. This makes tracking quite hard, because the track point now becomes blurred.
3. Change of focus in the camera. When the focus changes, different portions of the image become blurred, and the other blurred portions become sharper. This can make the software lose track of the points.
4. Changes in illumination/colour of the track points. In the course of the shot, if the object suddenly goes into a darker or brighter area, or the light that illuminates the object changes colour, the software may not be able to follow the change and would lose the point.
5. Tracked points become occluded. When another object comes between the camera and the points being tracked, the software will lose the track points.
In the case where the solution for the matchmove is less than satisfactory, many software solutions have tools for the matchmoving artist to help it along; to tell it where the track points are. It may also allow the user to input camera settings, such as the type of lens, the focal distance, the height of the camera, the known distance between certain track points to the camera, or distances between specific track points. All of these will help to achieve better accuracy. For this reason, film productions that require matchmoving facilities often have camera assistants tasked to log the camera settings and distances between the camera and certain objects for those shots. These guys are usually referred to as Visual Effects Supervisors. Often, after the software has solved the matchmove, a matchmove artist will need to step in to refine the animation of the 3D camera, to refine its trajectory.
External links
"Camera tracking"