Using GPUs for Parallel Processing

Thought graphics cards could do only graphics? Think again! If you look at
the action in the x86 graphics processing arena over the past six months or so,
you would notice they are talking about something called 'Stream Computing.' To
be truthful, stream computing by itself is nothing new. It is taking a process
with multiple threads that can be executed independently of each other, and
executing them in one go using multiple processors in parallel. How is this
related to graphics processors? We will get into the details shortly, but
consider this: a modern graphics processor (GPU) consists of up to 128 parallel
processing units called 'pixel shaders.' Now these pixel shaders cannot do
fantastic things by themselves individually. But take a bunch of them and run
parallel threads through them and you will get much better computer performance.

This has value in applications that have a large pool of data and
the same instruction needs to be applied to all of it. Some applications that
require this sort of computing are used for protein sequences analysis and
geographical data mapping from satellites. So how does stream computing on
graphics processors affect you? Until now, stream computing involved special
stream computing machines (much like super computers) and were restricted to
high-budget research institutions. But now, as graphics cards have this spare
computing capacity within them, they can be harnessed for the same effect at a
much lower cost. And, vendors of such cards (like AMD/ATi and NVIDIA) have come
up with mechanisms to let software developers produce stream computing
applications that can run on an off-the-shelf workstation with nothing more
powerful than a modern graphics card. So maybe your next payroll processing
engine will need the latest high-end gaming card instead of the latest in server
clusters to do their job faster.

Inside a modern GPU

Inside a GPU on any modern graphics card you pickup, there are specialized
processors (let's call them 'cores' instead) called 'vertex' and 'fragment'
shaders. A vertex shader operates on a vertex of any polygon being considered
for rendering. For instance, such a processor would determine the position and
color of the vertex. A fragment shader works on sets of pixels and generates
lighting effects (for example) on these pixels. Different shader models that
have evolved in-step with successive DirectX specifications have increased the
capabilities of what GPUs can do. For instance, higher floating point precision
called double-precision, involving 64-bit values instead of traditional 16 or 32
bit ones has enabled a lighting effects paradigm called HDR (High Dynamic Range)
that allows for more realistic shadows and effects.

Consider a GPU with 8 layers of 8 fragment shaders. This makes up 64 cores.
This group can render 64 fragments simultaneously. This sort of processing is of
course called SIMD (Single Instruction Multiple Data) that we discussed in an
earlier part of this series. To sum up, SIMD is an operation where you apply a
single instruction (or transformation) on multiple data elements. For example
increasing the light on a set of rendered pixels is an example of SIMD. Let's
take SIMD a step further.

Stream processing

Because these cores operate independently, they cannot share any data and so
there are two types of memory locations associated with them-one's called a
'Texture' memory which can only be read from; while the other is called
'Framebuffer' which can only be written to. Of course, data in the Framebuffer
can be routed back to the Texture memory to serve as input for another stage of
processing before it is sent to the screen. If you consider that all data in the
texture memory needs to have the same instructions applied to them, then modern
parlance calls this a 'Stream'. The application of the shader logic on this is
called 'Stream Processing' and the logic itself is called 'Kernel'. Related
terms are DPP (Data Parallel Processing) and GPGPU (General Purpose computing on
GPU).

Regular GPU based stream computing platforms utilize identical hardware (as
in graphics cards with identical capabilities). But this need not be the rule
and we can use dissimilar hardware, which AMD calls 'Asymmetric processing'.
ATi's cards can be used this way to handle different tasks depending on the
capabilities of each card plugged in. For instance, one can do physics while the
other can be used for rendering. Of course, in such a system, it is not easy to
scale up the performance by simply adding more cards, as this will only add to
the system's asymmetry. Another problem is workload distribution to optimize
performance. So, asymmetric processing can be used even by the same application
with multiple tasks that are independent, with no direct communication amongst
them.

The GPU contains sets of Pixel Shader cores which can execute parallel threads very quickly

Developer support

Both ATi and NVIDIA have released SDKs (software development kits) to let
developers make applications that make use of GPUs. The problem was that
traditional compilers cannot directly optimize for graphics related processing
easily. NVIDIA's solution is called CUDA (Compute Unified Device Architecture)
while that of ATi is called CTM (Close To the Metal). CUDA is actually a C-like
compiler for GPU applications. The engine provides a thread controller
architecture that takes care of thread management for applications that stream
through the GPU. CTM on the other hand, is more a hardware driver that exposes
API that GPU applications can call to achieve similar end results.

Applications that want to harness the power of stream computing through the
GPU must use CTM or CUDA to route their code through the GPU. CTM or CUDA will
perform the arbitration of which code to route to the GPU and which to the
regular CPU.

Sadly there is no vendor-agnostic way to let applications run seamlessly
regardless of which of the two vendors supplied your graphics card. For now,
early adopters will have to be contended with building uni-vendor stream
computing infrastructure which may not be the best-of-breed or the most cost
effective solution around. Clearly, much work needs to be done and it is very
early days yet to put this down as a success or write it off as too complicated.