Why are Multicore CPUs So Hot?

These days it is common to have processor codenames and architecture names
used with abandon. The names can confuse anyone, and after a while, it becomes
nearly impossible to tell them apart. This month, we take you through this
jargon jungle and unveil the secrets of how modern x86 processors work. There
are two parts of the story. One is the micro-architecture part, which is a
combination of multiple technologies. For instance, when Intel says 'Core 2'
or AMD says 'AM2', it denotes that a particular combination of technologies
is at work to produce a processor that can be called a 'Core 2' or an 'AM2'
product.

The second part to our story is the architecture itself, which is known by a
specific codename (say 'Conroe' or 'Manchester'). This month, we look at
technologies behind the different micro-architectures and next month, we'll
see what each combination (the architectures themselves) is about.

Macro Fusion

A term coined by Intel, it refers to the processor's ability to combine
several instructions into one, thus optimizing it and making for a faster
execute. Thus, if values from two different memory locations already in the
processor cache are to be compared but the instruction set first loads them
elsewhere and then compares them, the Macro Fusion technique would enable you to
directly compare them, skipping a step. Spread across an entire application or
thread, this can significantly boost execution times.

Handling L2 cache

When you have more than one core on a processor die, selecting the right place
to have the L2 cache is key. L1 is not touched here since it is a small-sized
cache that contains instructions being immediately fed to the processing core.
One method uses a common L2 cache for multiple cores (Intel) while the other
seeks a dedicated cache location per core (AMD). There are pros and cons to both
sides of the coin.

When you go the AMD way, you have two L2 caches in a dual core die and four
of them on a quad core. While each core gets its own dedicated cache, the
drawback is that when some cores are more active than the others, their caches
will be overflowing and hence those cores suffer a performance hit even though
the core waits for a fetch from the main memory. Intel's shared core (what it
calls an 'Advanced Smart Cache') uses shared L2 caches between two cores,
letting cores be better utilized when one of the cores is relatively less
loaded. But, this method introduces the headache of having a memory controller
that manages the memory between the two cores (who gets to put what where and so
forth). The memory bandwidth Intel promises with its way out peaks at 96 GB per
second at 3 GHz. Obviously it is hard to take a call on which one is better, as
each method compensates for a different scenario.

In FSB-based architecture, data hits a memory controller which routes it to the CPU, memory, etc. AMD's Direct Connect places the memory controller in the CPU, causing all data to pass through the processor, which then routes it appropriately

Intel Smart Memory Access: So, when you have a shared L2 cache like Intel has
with its Advanced Smart Cache technology, the headache of managing the cache
between two cores falls on a memory controller. This is the bundle they call 'Smart
Memory Access.' Along with memory management, SMA also resolves memory
locations and adjusts internal pointers so that when the core finds an
instruction to jump elsewhere in the code or fetch an instruction or data that's
already been cached, it can be directly fetched from the L2 or main memory
location instead of reloading it. This speeds up out-of-order execution. The
speed arises from the fact that instructions that are independent of each other
can be executed as soon as that instruction has been decoded, without having to
wait for the sequence of instructions before it to finish. Thus, if a block of
code loads two values and stores them elsewhere without any processing with
other logical code in between can be reordered and the LOAD/STORE can be
executed in independent threads and many clock cycles earlier.

Intel's vPro technology lets IT admins create system partitions for troubleshooting, maintenance and inventory management

AMD Direct Connect: AMD solves the problem of managing L2 cache in multi core
or multi processor systems, by doing away with the need for an FSB and talking
directly to the various components through something called the HyperTransport
Link. Taken together this architecture is called 'Direct Connect' (DCA).
Each processor built around DCA has an integrated memory controller and is
HyperTransport enabled. Each processor with DCA is linked to specific portions
of memory. When one CPU needs to access data that is in the memory linked to the
other processor, it will use HyperTransport to link to that processor/memory.
This HyperTransport linkage is called Coherent HyperTransport.

Memory access speeds for processor transactions are boosted when you put the
controller on the processor die itself. But in a multi-processor system, when
you add more processors, you also add more memory controllers. To solve the
problems with access overlaps and violations and race conditions, AMD uses
something called NUMA (Non-Uniform Memory Access) that is similar to Intel's
SMP (Symmetric Multi-Processing). NUMA deviates from the SMP in that NUMA is
asymmetric. Although the original use for NUMA and DCA involve multiple
processors rather than cores, it can be applied to a multi-core system just as
easily.

Multimedia instructions

With the Pentium-1 class of processors, we had the SSE (or AMD's equivalent of
3DNow!), SSE-2 and SSE-3 that themselves are descendants of the now defunct MMX
instruction set. SIMD is one more evolution forwards from there. 128-bit SSE
instructions traditionally take two clock cycles each to execute. SIMD allows
for these instructions to complete in one clock, doubling their throughput. In
addition to all that, SIMD adds about 70 new instructions to process packed
floating-point data, control memory without cache pollution and extend the MMX
instruction set. This lets your application do really parallel threads with
independent control flow. This expanded instruction set is also called the 'Streaming
SIMD extensions' because it works best for streaming data (into the processor)
and letting you do video processing like encoding video and other multimedia
data, faster.

Intel vPro

If you've heard the talk about having system level partitions with each
partition being able to run isolated operating systems and software, and your IT
department being able to manage your system remotely using hidden or access
restricted areas of your system while you were busy working with your section of
it... vPro is the technology that is set to make that a reality.

With vPro, your IT department can remotely monitor, diagnose and repair your
desktop even it is switched off. Also, the system can send out its configuration
information like what cards are installed, how the BIOS has been configured and
so on. This also improves asset tracking and inventory since one no longer needs
to go to each computer to audit this and neither do they need to install
specific software or open (desktop) firewall ports to do this. Additionally,
vPro also has the benefit of lower power usage by turning off processor
functions when they are not in use. Yes, vPro can turn off specific portions
inside the processor when they are not being used rather than turning an entire
core on or off.

New in virtualization

This is a big topic outside the discussion on processors as well. The reason
this is hot is well known-the more the number of applications you can put into
a single box (most usually by cramming more virtual machines into that box) the
lower your cost of ownership and running costs and better your cost efficiency.
But the biggest limiting factor so far to that argument has been that while
virtualization has been possible in the non-x86 world for some time now, it has
been primarily software based in the x86 realms. The VMware, Xen and Virtual
Servers/PCs of the world have ruled this area. However, the performance gained
by doing that while the hardware did not inherently support virtualization is
not that great. The traditional way to boost virtual machine performance is by
jacking up the amount of RAM and the processor speed.

Intel's vPro and VT (Virtualization Technology) and AMD's AMD-V are
rather big steps to taking the x86 into a fully virtualizible environment. We
have discussed vPro above, let's take some time to understand VT and AMD-V.

Intel VT: Codename 'Vanderpool', it enables support for
virtualization software layers to control processing actually being done inside
a virtual machine. This allows such software to monitor virtual systems and
marshal their resource (processor and memory) usage. Intel VT is in use not just
in x86 desktop processors, but also affects Itanium 32 and Itanium 64 bit
families. Three new features for the 32-bit VT (called VT-x) are: a more
coordinated way of dealing with blocked NMI (Non Maskable Interrupts) when VMs
(guest OS in a VM) are exiting, setting up virtual processor IDs by VMMs and
then use these IDs to translate buffer addresses in memory and; instead of
having a common memory page tables for host and guest OS VMs now have their own
page tables. The Xen Hypervisor gets extra support from the Intel VT
architecture, letting it expose resources and configuration to its guest
software that return better virtualized performance. This includes presenting
all the real processor information bits (CPUID) to the guest, with the exception
of the VMX and MCA.

AMD-V: This is a step further from AMD's 64-bit architecture
(AMD64). AMD-V adds two modes of operation to the systems it runs on: Host mode
and Guest mode. Also added is a new instruction called VMRUN that lets virtual
machines along with the guest OS and its applications work a little faster
inside a VM. An AMD-V processor will initially boot up with its new VM
capabilities disabled (the 'guest' mode) until compatible VMM (VM Managers)
are detected. Once such a VMM is detected, the processor switches to 'host'
mode and turns on all its capabilities. An AMD-V processor in host mode features
a number of interesting capabilities like setting up at the hardware level
itself the kinds of resources the system software is allowed to access. Using
these instructions one can even assign different VMs exclusive access to
different resources (like network interfaces).

As we continue adding cores to processors and add more of such processors to
our computer systems, it is natural that their technologies will advance to let
multiple processor cores access what so far has been a resource dedicated to one
processor core. Next month, we will be looking to demystify the various
processor code names in the multi-core domain.