Source engine

The Source engine (officially the Valve Source Engine) is a 3D game engine developed by Valve Corporation. Its unique features include a very large degree of modularity and flexibility, an artist-driven, shader-based renderer, industry-leading lip sync and facial expression technology, and a powerful, efficient and completely network-enabled physics system.

Source supports both 32-bit and 64-bit environments and the Microsoft Windows, Xbox, Xbox 360 and Playstation 3 platforms. It debuted in November 2004 with Source and Half-Life 2.

Technology overview

For a full overview, see Source Engine Features on the Valve Developer Community Wiki.

Modularity: See Modularity and Notable Upgrades

Rendering: Shader-based; DirectX 9 and Shader model 2.0 or lower only; Deformable terrain; 3D skybox; an area can be displayed as a skybox at up to 16x its actual size with full 3D parallax.; High dynamic range rendering (see High Dynamic Range Rendering section)

Animation: Any animation can merge seamlessly with any other animation at any time; Inverse Kinematics to ensure that character's limbs react to environments; Originate from Havok 2 physics engine; Highly-tuned by Valve; Processor-efficient; Software only; Fully networked with low bandwidth requirements; Vehicle physics including torque, power, gears, tire material, suspension and mode.

Audio: Full DSP system; Full Surround sound support; Software only; Low and high frequency components merged depending on surrounding area and relative position of origin

Scalability: Supports DirectX 6 and upwards; Modularity allows for all current and future Source projects to scale back to DX6 if they desire

Facial expressions: Full range of human and non-human facial movements; Based on Professor Paul Ekman's Facial Action Coding System; Over eighty-four "digital muscles"

Lip-sync: Works in tandem with facial expressions; Auto-generated, but completely configurable; Stored in sound file itself for localization

Modularity and Notable Upgrades

Source is designed from the ground up to be highly modular. This allows for the easy upgrade and modification of certain features without breaking other areas of the engine, or breaking engine continuity (that is to say, there need be no 'version jumps' from 1.0 to 2.0). When coupled with Steam, these updates can be distributed retroactively and automatically. For instance, if Source is upgraded to support hardware physics, every Source title on Steam will instantly benefit. Entirely new features such as High Dynamic Range (HDR) Rendering have been shown to require developer input, however.

High Dynamic Range Rendering

First seen in Source (released 26 September 2005), HDR was the first major instance of Source's modularity in use. However, whilst in theory all Source engine games and mods were able to use HDR immediately after its release the game code required to 'hook in' to the new system is not yet available to modders (as of 28 June 2006). Official licensees and Valve themselves have all made use of the technology.

Facial Animation 2

When Episode One was released on 1 June 2006 it introduced the second version of Valve's proprietary facial animation system. Ken Birdwell explains the upgrade's features in the game's commentary track:

Future Technologies

Dynamic Lighting and Shadowing 2

A new dynamic lighting and shadowing system is being developed for Source, replacing the somewhat limp existing system.

Currently, all dynamic shadows in a map:

Come from the same angle regardless of the area's light sources
Cast through all objects save world geometry/BSP
Only cast shadows on world geometry
Do not blend with pre-compiled static shadows
Do not self-shadow (i.e. a character's arm will not cast a shadow on its torso)

The new system is currently known to:

Go some way towards unifying pre-compiled and dynamic shadowing
Allow objects to cast multiple soft shadows
Allow shadows to dynamically react to at least three light sources
Include new Phong shading and rim lighting effects
Include per-vertex prop lighting

Next-gen Renderer

An upgraded rendering path is in development for future Source engine games on Xbox 360. PC development is not confirmed, but extremely likely. No other details are known at this point. Possible features include support for the Shader Model 3.0 (Pixel Shaders) and DirectX 10.

Landscape and Flora Rendering

The trailer for Episode Two makes extensive use of large-scale natural environments that have traditionally been a weakness for the Source engine.

Soft-Particle System

During the July 2006 Electronic Arts Summer Showcase press conference, Gabe Newell mentioned that a new soft-particle system will be introduced into the Source Engine in the upcoming title Episode Two.

Cinematic Physics

During the July 2006 Electronic Arts Summer Showcase press conference, it was revealed that former Weta Digital employee Gray Horsfield, special effects destruction lead on The Return of the King (film) and King Kong among other roles, is building a "Cinematic Physics" system for Source. GameSpy described the new system in their conference report:

No other details are currently available.

Unconfirmed Future Technology

Cinematic effects

With color correction and film grain already released, Valve intend to add their other cinematic effects such as motion blur and depth of field to Source when hardware is able to render it to their satisfaction. While all of these features can be seen in the Source video Prelude to Victory, they cannot be run at acceptable framerates on today's systems.

A preview of the cinematic technologies is available at bit-tech.

Image-Based Rendering

Image-Based Rendering is a technique in which 2D elements are manipulated to appear in a 3D world. In the context of a 3D game, it delivers a significant performance boost by replacing 3D geometry that is far enough away for the transition to be imperceptible with a 2D image. A current implementation of the technology in this role can be found in Total War. The technology had been in development for Half-Life 2, as a 2003 interview with Driverheaven.net shows, but was cut. It was mentioned again by Gabe Newell during 1UP's 'Valve week':

The June 2006 Source update included an "image-based texture blending shader", and Episode Two's expansive environments seems ideally suited to the technology, but it has yet to be dated or even officially announced.

File streaming

One of the technologies developed for Half-Life 2's Xbox release was file streaming, wherein a map's resources could be loaded as the player moved around in it rather than in one operation before playability. With the system in place, loading times were reduced to as little as fifteen seconds. The system expanded on the caching system already implemented. There is no time frame for its release, as implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems (see Stutter section).

Origins

Although Valve have explicitly stated that the Source engine has been built internally from the ground up, rumors and myths persist that it is instead merely derived from the original Quake codebase via Valve's GoldSrc offshoot. The primary reasons for this are the manner in which the engine uses similar development interfaces to GoldSrc (to aid transitioning developers), and John Carmack's comment on his blog that "there are still bits of early Quake code in Half-Life 2", expanded through hearsay to be a confirmation that large swathes of code are identical, when no such conclusion can be drawn from the statement. There remains no solid proof that Source is derived from the GoldSrc codebase - and indeed, given the fact that the 2003 code leak did not produce any such claims it can only be assumed that no incriminating evidence was to be found.

However, it is known that Source was developed part-by-part, slowly replacing the GoldSrc engine in Valve's internal projects. This explains its modular nature, and suggests that, even if Source was not derived from GoldSrc, GoldSrc was at the very least modified to plug into it during development.

Common issues

Stutter

The Source Engine uses a caching system, whereby the loading of certain resources is handled and managed on the fly, rather than in a single operation behind a traditional loading screen. Texture and sound data are the primary areas in which this occurs. Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested.

Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

When Half-Life 2 was first released and stuttering became a widely-known problem, community member Mark McWilliams set up a page covering the issue and Valve's communication and work on resolving it. Several updates were released by Valve, the effects of which varied from complete fixes for some users to previously smooth systems becoming "infected" with the problem.

Example 1 (.WMV), showing a relatively low level of stutter. The hitches coincide with the drawing of new areas (a process that is by design not visible without debugging tools).
Example 2 (.WMV), showing a far more violent case. This user's stutter seems tied to audio.

Most recently (February 7 2006), changes to the Source engine were introduced alongside a beta test of Steam's Friends chat service, with the aim of 'narrowing down' the problem. The update featured a limited implementation of Source's file streaming system (see above). Generally, the response was very positive.

Looping Audio

The Source Engine suffers from an error whereby the asynchronous loading (see Stutter, above) of a new sound file will cause the engine to lock up with looping audio. Because of the nature of DirectX, once the engine enters such a state it will remain on the screen unless the user can blindly terminate the program, or reboots their computer. The error occurs in a standard Windows library associated with on-board sound cards, and can be resolved by decreasing DirectSound Acceleration.

Reports of looping audio crashes increased around the release of Episode One. While it is likely that the spike was simply because there was an unusually high number of people playing the game, changes to the engine, of which there were many for the game, cannot be ruled out. It has been noted that people who have been able to play Half-Life 2 without any crashes and audio errors in fact do find Episode 1 to be more prone to that problem.

The Valve Developer Community

On the 28th of June 2005, Valve opened the Valve Developer Community Wiki. The VDC replaced the previously available Source SDK documentation with a full MediaWiki-powered community site. Within a matter of days Valve reported that "the number of useful articles ^* nearly doubled". These new articles covered the previously undocumented Source bot (added by the bot's author, Mike Booth) and Half-Life 2 AI, source control for Source engine mods, and more.

Licensed Source games

Valve: Counter-Strike: Source; Source; Source; Half-Life 2; Half-Life 2 Episodes ( Episode One, Episode Two and Episode Three); Portal; Team Fortress 2

Other developers: Infested; Dark Messiah of Might and Magic; Dino Hunters; 2" target="_blank" >^*; Source ^*; Prime; SiN Episodes; The Ship; Lost Souls; After the Fall (production resumed); The Masquerade - Bloodlines; Unannounced Turtle Rock Studios game; Unannounced Warren Spector game

References

External links

Game engines | Half-Life 2

Valve Source Engine | Source Engine | Source engine | Source | Source

Gourt Home::Gourt :: Source engine