New Age Graphics Moving Closer to Photorealism

Tucked in a corner of a large, swanky office, a group of animators form a quiet huddle, glued to a large monitor, where an actress is portraying a series of human emotions ranging from laughter, confusion, relief, stress and depression. On another screen, you find a computer generated image trying to mimic those emotions. This is another test of the kind of photorealism that the latest graphics equipment in studios can achieve. The virtual characters seem to completely embody the personality of their real life counterparts.

More than the prominent cheek bones and the checkered brow, there is that intensity that seems to be emanating from deep inside the character's rendered body. And of course it's more than being a little bit weird. You can feel the subtleness of the inner part of the lips, the movement of the tongue or the eyes, to almost every aspect of human emotion. Then you notice the physics involved in the movements of the character, imitating those of the real character with full glory. Artists might love to call that soul transformation from the real to the reel character. But can you even imagine what kind of technologies or designs are involved in achieving such pristine graphics animation. It goes beyond the 700 million odd transistors that you normally have on processors. It even goes beyond the multi-core or multi-processor architecture that is almost ubiquitous or the ever more complex APIs to realize their full potential.

Dermal Nanotech Display

Do these visuals look scary or exciting? Depends on whether you want to put your body to such use or not. This is an artist's imagination of a 'programmable dermal display' where about 3 billion pixel robots for display are permanently implanted just about a fraction of a mm under the epidermis, making up a 6 cm x 5 cm display area on the back of your palm. The display image is formed by photons emitted by these pixel bots. Each display shows the data received from all the medical bots flowing inside the body. And this is a two-way traffic. The same bots can be instructed by the user to perform specific tasks. What's more, the display can be activated or deactivated by a gentle tap of the finger on the skin surface!

By the end of the 1990s, the usage of dedicated GPUs to provide visually intense, interactive 3D experience had become a norm. These GPUs grew in popularity as more and more graphics intensive applications such as Google Earth, Picasa and games such as Crysis, Halo 2, Flight Simulator, etc became available. To get into the minutest of detail of an object, you need processing power of several hundred cores and at extremely high speeds. In the quest to reach closer to life-like realism, the major graphics hardware and animation software vendors are involved in continuous research and development. The technology should ideally enable you to fathom each and every minute detail, be it the texture of the skin or the crevices in the walls of a building, the clouds in the sky or the discrete water drops in a waterfall. With the advancement in technologies to pack more and more transistors on a single die, the parallel processing power of a GPU can be harnessed in more ways than one.

Today, the processing power of even a multi-core CPU pales in comparison to the mammoth power that you can generate out of a GPU, and this chasm is steadily widening. Moreover, GPUs have crossed the thin line from being a fixed 3D graphics processing pipeline to become flexible general-purpose computational engines. These GPUs implement many parallel algorithms that use all the underlying computational power to achieve tremendous computing speeds. Let's take a look at some of the new techniques in this highly mesmerizing field.

Previously, there were separate processing pipelines for pixel and vertex shaders. But now these pipelines can be used by all shaders

Unified Shader Architecture



The advent of DirectX 10 means that GPUs need to have an architecture to support its unified pipeline model. So we have graphics vendors coming up with Unified Shader Architecture. Earlier, game programmers had no control over transformation, lighting and pixel rendering mainly because all calculation models were fixed on the chip. Unified architecture has a single floating point shader core with multiple independent processors called stream processors. Each of these processors is capable of handling all shading operations: pixel, vertex, geometry and now even physics acceleration. In simpler terms, earlier you had separate pipelines for different shaders, where one of the pipelines used to be fully clogged while the other ones were under utilized, but now you have 100% utilization (see the infographic), thanks to unified shader processors (stream processors), that can be utilized simultaneously. So, the more the number of shader processors the better the processor utilization, resulting in more computing power and faster rendering. Previous generation shader units operate on data in a vector fashion, but now each is a scalar and thus can operate on only one component at a time, making them simple while being flexible at the same time. All this makes today's GPU much more efficient and powerful.

Direct X 10



This API is meant to improve the ease of programming and allow game designers to easily implement enhanced graphics, visual effects and more. It is based on the Shader Model 4 with some big enhancements, like addition of instructions for integers and bitwise calculation and debut of geometry shaders. For details on geometry shaders, please refer to p32, Feb 2007 issue. The most important feature is the unification of shader pipelines, i.e. now fixed pipelines have been done away with in favor of fully programmable pipelines, so that there is no fixed pipeline for pixel or vertex shaders. DirectX 10 reduces the CPU overhead by reducing the amount that the CPU gets involved in the rendering process. Thanks to geometry shaders more manipulation of vertex and other objects are possible which means more ways to process, access and move data. All this extra level of manipulation provides options to developers to add new features into games and utilize GPU for more than just rendering a scene.

Let's talk about some of the features DirectX 10 has on offer. Texture array, predicted draw and stream out are some of the new features that DirectX 10 has. Texture array allows the graphics card to process upto 512 textures in an array at one time. Earlier it was done on the CPU, but now it is done on a GPU without interference from the CPU.

Visual effects in Dhoom2: You can see the actual set for the shot (inset). This was gloriously enhanced by compositing with the image of the city

Stream out allows the graphics card to output the pixel and vertex shader information directly to the memory, which means faster access. Predicted rendering allows drawing calls to be avoided when not required, preventing the object from being rendered if it is not visible or too far to be visible. DirectX 10 has support for Instancing 2.0 which allows multiple instances of similar meshes, like armies, grass or tree to be rendered in a single draw cell reducing processing time needed for multiple similar objects to that of a single one.

Technologies for HD



With movies meant for Blu-ray and HD-DVD being anticipated to release sooner than you and me would have thought, you need graphics technologies that utilize every bit of their functionality to provide quality movie playback at resolutions up to 1080p. Both these technologies provide up to six times more visual detail than your regular DVDs. We have the major graphics vendors coming out with technologies to support them. Let's take a look.

PureVideo HD



NVIDIA's PureVideo HD is a combination of HD movie decode acceleration and post processing, HDCP circuitry and integration with HD movie players, aimed at providing life-like quality for movies on Blu-ray and HD-DVD. It supports formats such as H.263, WMV and MPEG-2 HD. It works by sharpening HD and standard definition interlaced content on progressive displays, which in turn makes it suitable for high-end home theater systems. Using sub-pixel processing, you can scale videos to the required size, so that even the small videos would seem like they've been recorded at high resolution. The rough edges are smoothened using spatial temporal de-interlacing, inverse telecine (where you can recover original film images from films-converted-to-video), and advanced scaling technologies to ensure superior image clarity.

Universal Video Decoder (UVD)



AMD's UVD is a video decoding technique that supports hardware decoding of MPEG-4 and VC-1 codecs. Based on a Xilleon graphics processor, this technique is meant to handle Blu-ray and HD-DVD decoding in a GPU at 40 Mbps. Dual stream decoding techniques are used to ensure HD decoding at 1080p. UVD handles bitstream decoding/entropy decode, pixel prediction and deblocking, and frequency transform techniques; instead of passing these to the CPU for further processing. High definition video playback is enhanced through integrated HDMI and support for HDCP. This is an advancement over the previous AVIVO video processing technology. With this technique, you don't need to play audio on the PCI-e graphics cards, as is currently done by HDMI-based systems. Rather it can also be played on a GPU, eliminating the need for a sound card altogether.

Left: An example of the photorealism we can achieve with DirectX 10. See the discrete drops of water. Right: Geometry shaders allow the manipulation of the vertex to show deformities of the stretched skin

Multi GPU Solutions



Individual graphics cards are capable of delivering huge amount of processing power. Imagine what if we can combine the processing power of two such cards to get a single output on the screen? Seems lucrative right? This is exactly what SLI or CrossFire do. They combine the potential of two or even more graphics card (multiple GPUs) to provide a single output, enhancing the overall visual effects. The basic idea behind a multi-GPU solution is to allow two or more GPUs to share the work load when rendering a 3D scene. Two identical graphics card are installed on a motherboard and setup in a master-slave configuration. Both cards are given the same part of the game or scene to render, but eventually half of the work load is sent to the slave card. The slave then sends its rendered output to the master card, where it is incorporated in master card's own image and sent to the display output.

SLIis the multi-GPU technology from NVIDIA and offers two rendering and one anti-aliasing techniques for splitting the work between the video cards. It uses Split Frame Rendering (SFR), to analyze the rendered image so as to split the workload equally between two GPUs. Another rendering technique is Alternate Frame Rendering where each GPU renders entire frames in sequence, with one GPU processing even frames and the second one processing odd frames, one after the other. The SLI anti-aliasing is a standalone rendering mode that offers up to double the anti-aliasing performance by splitting the anti-aliasing workload between the two graphics cards. There are cards having two GPUs in one single card, also using the SLI technology. This is Quad SLI which enables you to attain 32x anti-aliasing in SLI AA mode and support 2560x1600 resolution with smooth frame rates.

Pankaj Khandpur Creative Director, VCL

Visual Computing Labs (VCL) does animation and visual FX for Bollywood and Hollywood. They also address the global animation markets of 3D episodic television work, Game Asset Building and CG Features. Some of their recent projects have been Dhoom 2, Kabhi Alvida Na Kahna and Rang De Basanti and One Night with The King. What sort of graphics/animation systems do you use in your movies? Maya is used extensively (almost 90%), plus, we also use AutoDesk Flame, Digital Fusion, Shake, Combustion, 3D Equalizer and Houdini. We use a mix of technology available. Systems could have single or multiple GPUs. Again the needs vary depending on the specific task done. The lighting guys need more of GPU power while animation guys can do with less. Name a technology/product that has revolutionized the way special effects are generated? Very extensive compositing is required to make things photoreal. For eg, in the title song of the movie ‘Ta Ra Rum Pum,’ (composited using AutoDesk Flame) we computed 28,000 trillion particles just to render the bear’s fur. Similarly, there are 45 min of hidden effects in the car racing scene. All the 40,000 people in the stadium, the smoke and all the 40 cars have been computer generated. Maya has made a lot of difference to help achieve this. Plus, we’ve also used Visual Fusion and Shake. Are you satisfied with the progress that technology has made in this field? Animation is catching up in India as we have moved from an era where one animated film was made in four years to having four such films in one year. In one of our upcoming films, Jodha Akbar, you’ll see battle sequences with a lot of computer generated imagery depicting horses, soldiers, and the like. 22 years ago you didn’t have graphics card but animation was still there. But, now with the advent of more powerful graphics cards and better software, you have the potential to show more.

CrossFire is ATI’s multi-GPU solution. It has four different rendering modes. The first one is Super Tiling, which divides the screen like a checkerboard, allocating adjacent squares to alternate GPUs. The second is Scissor which divides the screen into two rectangles one above the other. The third is Alternate Frame Rendering, which sets one GPU to render odd frame and the other to render even frames. The fourth and the final one is Crossfire Super AA which improves the quality of the frame rendered rather than concentrating on increasing the frame per second rate.

Since both CrossFire and SLI perform the same role, the comparison between the two is inevitable. Both have their own advantages and disadvantages. For example, the newer ATI CrossFire bridge has higher bandwidth than SLI, even though it is clocked a little slower. Also, high end CrossFire implementation requires an external dongle to operate in CrossFire mode while NVIDIA has done way with an internal bridge, making it less cumbersome.

Remote Graphics Unit



These days we see an influx of remote consoles, thin clients and other such stuff that enable remote control of your CPU. On the same lines why not have a remote access to your GPU? RGU is a technique of separating your GPU from the I/O devices such as keyboard, mouse, audio and monitors. Similar to a remote KVM, such a technique comes in handy when you need to separate the GUI from the actual computer to prevent unsolicited access in corporate environment, to reduce heat or noise in offices with multiple computers running their individual GPUs or to facilitate proper maintenance by clubbing the hardware together. An RGU splits a computer’s PCIe bus and sends across the bus commands to the remote site. As the bus data is smaller than the rendered graphics data, it is possible to obtain higher resolution even when the distance between the computer and the user I/O devices is fairly large. This concept is supported by bus extension technologies where a GPU is plugged into a remote PCI slot through a typical graphics add-on card.

The famous MiG-21 scene from Rang De Basanti where the plane was transposed on the original image using CGI

Graphics in animation



Graphics technologies are in no way restricted to gaming alone. They have tremendous applications in movies, televisions and high performance computing. Today GPUs are seen as a potential force which can give CPUs a run for money in certain areas. GPUs have the power to not only render images but also do many parallel processing tasks much better and quicker than what a CPU can do. With ever increasing shader processors (cores), GPUs have gained substantial processing power to do processing faster and more efficiently. Graphics technologies are making a sizeable impact in the way we see movies. Be it Hollywood or Bollywood everyone is gunning for better and more powerful hardware and software taking a cue from creative animators and visual effects experts. Not only animation, even a normal scene can be enhanced to include lively effects such as artificial snow, burning trains and so on. Even though the animators or visual effects experts rely mostly on software, but these software are designed to take advantage of the full potential of the powerful GPUs available today, to enhance the way scenes are rendered. Even Bollywood is using animation and visual effects in their films with increasing intensity to add snazziness that was missing a few years back. This is going to be the roadmap for the future. Well and truly, the potential for improvement here is almost limitless.